How to use all-MiniLM-L6-v2 in .net android?

Hello, and thank you for your question.

The all-MiniLM-L6-v2 model is an embedding model rather than a chat model. It is designed to understand and compare text by converting it into a format that applications can search, match, or retrieve information from. It cannot generate responses on its own.

If your goal is to create a .NET MAUI Android app that can work with text files, you can build a simple workflow:

1. Install ONNX Runtime and add the ONNX model (you should have downloaded it somewhere) to your MAUI project

dotnet add package Microsoft.ML.OnnxRuntime 

2. Import ONNX and load by its directory

Option 1: Load the model from MAUI Resources/Raw (recommended)

using Microsoft.ML.OnnxRuntime; 
var stream = await FileSystem.OpenAppPackageFileAsync("all-MiniLM-L6-v2.onnx"); 
var tempPath = Path.Combine(FileSystem.CacheDirectory, "all-MiniLM-L6-v2.onnx"); 
using (var file = File.Create(tempPath)) 
{ 
	await stream.CopyToAsync(file); 
} 
using var session = new InferenceSession(tempPath); 

Option 2: Load the model directly from an absolute path

using Microsoft.ML.OnnxRuntime; 
string modelPath = Path.Combine( 
FileSystem.AppDataDirectory, 
"all-MiniLM-L6-v2.onnx"); 
using var session = new InferenceSession(modelPath); 
 

3. Make your application loads the text file by choice. Here is an example for the code-behind:

private async void OnPickFileClicked(object sender, EventArgs e) 
{ 
try 
{ 
	var result = await FilePicker.Default.PickAsync(new PickOptions 
	{ 
		PickerTitle = "Select a text file" 
	}); 
	if (result == null) 
		return; 
	string text = await File.ReadAllTextAsync(result.FullPath); 
	FileNameLabel.Text = result.FileName; 
	ContentEditor.Text = text; 
	// Generate embeddings using the ONNX model. 
	// Tokenization and tensor creation are omitted 
	// from this example for brevity. 
	float[] embedding = await GenerateEmbeddingAsync(text); 
	// Use the embedding as input to the chat workflow. 
	string response = await QueryChatModelAsync(text, embedding); 
	ResultEditor.Text = response; 
} 
catch (Exception ex) 
{ 
	await DisplayAlert("Error", ex.Message, "OK"); 
} 
} 

(You may want to add some equivalent UI features to the XAML file like this:)

<VerticalStackLayout Padding="20"> 
<Button 
	Text="Pick Text File" 
	Clicked="OnPickFileClicked"/> 
<Label 
	x:Name="FileNameLabel" 
	Text="No file selected" /> 
<Editor 
	x:Name="ContentEditor" 
	AutoSize="TextChanges" 
	HeightRequest="400" /> 
</VerticalStackLayout> 

4. (Optional) Since you are using an embedding model, you may want to consider adding a helper method to convert document text into embeddings.

private async Task<float[]> GenerateEmbeddingAsync(string text) 
{ 
	// Placeholder for embedding generation. 
	// The implementation depends on the 
	// model and libraries being used. 
	await Task.CompletedTask; 
	return Array.Empty<float>(); 
} 
 

1. As said above, because the embedding model can't generate a response on its own, you will need to send the data to a chat model.

private async Task<string> QueryChatModelAsync( 
string text, 
float[] embedding) 
{ 
	string prompt = 
	$""" 
	Please answer questions about the following document. 
	Document: 
	{text} 
	"""; 
	// Replace with your preferred LLM API or SDK. 
	string response = 
	await YourLlmService.GenerateAsync(prompt); 
	return response; 		
} 

For more information on how to call an LLM API from a .NET or .NET MAUI application, refer to the following Microsoft documentation: build-chat-app

Tip: The examples above use a hardcoded prompt for simplicity. If you would like users to enter their own questions, consider adding a text input field to the UI and passing its contents to QueryChatModelAsync() instead.

If you found my response helpful or informative, I would greatly appreciate it if you could follow this guide for your confirmation. Thank you.