Frequently Asked Questions about using AI in Windows apps

It depends on your target hardware and use case. Windows AI APIs provide the simplest path for Copilot+ PCs — single API calls for OCR, image description, text summarization, and Phi Silica chat, with no model management required. Foundry Local is the right choice when you need a wider model catalog, non-Copilot+ PC hardware support, or OpenAI-compatible API access. Windows ML gives you the most control — bring any ONNX model and manage execution providers directly. See Choose your Windows AI solution for a side-by-side comparison.

It depends on which feature you're using. Windows AI APIs require a Copilot+ PC with an NPU. Foundry Local runs on any Windows device with a DirectX 12-capable GPU — including devices without an NPU. Windows ML supports a broad range of CPUs, GPUs, and NPUs. See the supported hardware list for details.

Yes. Foundry Local exposes an OpenAI-compatible REST API. Start the local service with foundry service start, then point your OpenAI client at the local endpoint — your model name, messages, and streaming calls work without changes. See Get started with Foundry Local for the endpoint format.

The native SDKs (C#, Python, JavaScript, Rust) are currently in alpha/pre-release. The underlying ONNX Runtime and model serving infrastructure is production-grade. For shipping apps today, treat the SDK API surface as subject to change and pin your package versions. See What is Foundry Local for the latest stability status.

Yes, once a model is downloaded and cached on the device, Foundry Local performs inference entirely on-device with no cloud dependency. The initial model download requires internet access. On startup, Foundry Local may attempt to refresh the model catalog, but this is not required — if the device is offline, it falls back to the cached catalog and inference continues normally. The SDK's IsCachedAsync (C#) / is_cached (Python) method lets you check model availability before attempting inference so you can handle the offline case gracefully.

Foundry Local detects available hardware at startup and selects the best execution provider — Qualcomm NPU (QNN), any DirectX 12-capable GPU via WinML/DirectML (AMD, Intel, NVIDIA, Qualcomm), NVIDIA GPU via CUDA, or CPU fallback. You request a model by alias (for example, phi-3.5-mini) and Foundry Local automatically serves the hardware-optimized variant for the device. Your app code is the same across all hardware configurations.

No. Foundry Local runs entirely on-device. Inference input and output never leave the machine. The only network traffic is the initial model download and optional catalog metadata refreshes. On startup, Foundry Local may attempt to refresh the model catalog, but this is not required — if the device is offline, it falls back to the cached catalog and inference continues normally.

Windows AI APIs process data locally on the device using the NPU. Input data is not sent to Microsoft servers. For guidance on responsible AI practices, consent, and transparency, see Developing Responsible Generative AI Applications and Features on Windows.

Windows ML enables your app to use a shared system-wide ONNX Runtime and dynamically download vendor-specific execution providers (EPs) so that model inference is optimized for the device's CPU, GPU, or NPU — without your app carrying large runtime or EP binaries. Use it when you need to bring your own ONNX model or want full control over EP selection. See Windows ML overview.

DirectML is a low-level GPU acceleration API for machine learning, built on Direct3D 12. It supports all DirectX 12-capable GPUs from AMD, Intel, NVIDIA, and Qualcomm. ONNX Runtime uses DirectML as an execution provider on Windows when CUDA is not available.

foundry-local-sdk-winml is the Windows-specific package — it includes hardware acceleration via Windows ML and is recommended on Windows. foundry-local-sdk is the cross-platform package for macOS, Linux, or Windows without hardware acceleration. Install only one: the two packages pin different versions of onnxruntime-core and will conflict if both are installed. Note that foundry-local on PyPI (without -sdk) is an unrelated third-party package — install foundry-local-sdk or foundry-local-sdk-winml.

Open Task Manager (Ctrl+Shift+Esc), select the Performance tab, and look for GPU and NPU entries in the left panel. You can also run foundry model list from the command line — Foundry Local will display which execution providers are active for your hardware when it starts the service.