BetaEvaluatorsOperations Class

Warning

DO NOT instantiate this class directly.

Instead, you should access the following operations through

<xref:evaluators> attribute.

Constructor

BetaEvaluatorsOperations(*args, **kwargs)

Methods

cancel_generation_job	Cancel an evaluator generation job. Cancels an evaluator generation job by its ID.
create_generation_job	Create an evaluator generation job. Creates an evaluator generation job. The service generates rubric-based evaluator definitions from the provided source materials asynchronously.
create_version	Create an evaluator version. Creates a new evaluator version with an auto-incremented version identifier.
delete_generation_job	Delete an evaluator generation job. Deletes an evaluator generation job by its ID. Deletes the job record only; the generated evaluator (if any) is preserved.
delete_version	Delete an evaluator version. Removes the specified evaluator version. Returns 204 whether the version existed or not.
get_credentials	Get evaluator credentials. Retrieves SAS credentials for accessing the storage account associated with the specified evaluator version.
get_generation_job	Get an evaluator generation job. Gets the details of an evaluator generation job by its ID.
get_version	Get an evaluator version. Retrieves the specified evaluator version, returning 404 if it does not exist.
list	List latest evaluator versions. Lists the latest version of each evaluator.
list_generation_jobs	List evaluator generation jobs. Returns a list of evaluator generation jobs. The List API has up to a few seconds of propagation delay, so a recently created job may not appear immediately; use the Get evaluator generation job API with the job ID to retrieve a specific job without delay.
list_versions	List evaluator versions. Returns the available versions for the specified evaluator.
pending_upload	Start a pending upload. Initiates a new pending upload or retrieves an existing one for the specified evaluator version.
update_version	Update an evaluator version. Updates the specified evaluator version in place.

cancel_generation_job

Cancel an evaluator generation job.

Cancels an evaluator generation job by its ID.

cancel_generation_job(job_id: str, **kwargs: Any) -> EvaluatorGenerationJob

Parameters

Name	Description
job_id Required	str The ID of the job to cancel. Required.

Returns

Type	Description
EvaluatorGenerationJob	EvaluatorGenerationJob. The EvaluatorGenerationJob is compatible with MutableMapping

Exceptions

Type	Description
HttpResponseError

create_generation_job

Create an evaluator generation job.

Creates an evaluator generation job. The service generates rubric-based evaluator definitions from the provided source materials asynchronously.

create_generation_job(job: _models.EvaluatorGenerationJob, *, operation_id: str | None = None, content_type: str = 'application/json', **kwargs: Any) -> _models.EvaluatorGenerationJob

Parameters

Name	Description
job Required	EvaluatorGenerationJob or <xref:JSON> or IO[bytes] The job to create. Is one of the following types: EvaluatorGenerationJob, JSON, IO[bytes] Required.

Keyword-Only Parameters

Name	Description
operation_id	str Client-generated unique ID for idempotent retries. When absent, the server creates the job unconditionally. Default value is None. Default value: None

Returns

Type	Description
EvaluatorGenerationJob	EvaluatorGenerationJob. The EvaluatorGenerationJob is compatible with MutableMapping

Exceptions

Type	Description
HttpResponseError

create_version

Create an evaluator version.

Creates a new evaluator version with an auto-incremented version identifier.

create_version(name: str, evaluator_version: _models.EvaluatorVersion, *, content_type: str = 'application/json', **kwargs: Any) -> _models.EvaluatorVersion

Parameters

Name	Description
name Required	str The name of the resource. Required.
evaluator_version Required	EvaluatorVersion or <xref:JSON> or IO[bytes] Is one of the following types: EvaluatorVersion, JSON, IO[bytes] Required.

Returns

Type	Description
EvaluatorVersion	EvaluatorVersion. The EvaluatorVersion is compatible with MutableMapping

Exceptions

Type	Description
HttpResponseError

delete_generation_job

Delete an evaluator generation job.

Deletes an evaluator generation job by its ID. Deletes the job record only; the generated evaluator (if any) is preserved.

delete_generation_job(job_id: str, **kwargs: Any) -> None

Parameters

Name	Description
job_id Required	str The ID of the job to delete. Required.

Returns

Type	Description
None	None

Exceptions

Type	Description
HttpResponseError

delete_version

Delete an evaluator version.

Removes the specified evaluator version. Returns 204 whether the version existed or not.

delete_version(name: str, version: str, **kwargs: Any) -> None

Parameters

Name	Description
name Required	str The name of the resource. Required.
version Required	str The version of the EvaluatorVersion to delete. Required.

Returns

Type	Description
None	None

Exceptions

Type	Description
HttpResponseError

get_credentials

Get evaluator credentials.

Retrieves SAS credentials for accessing the storage account associated with the specified evaluator version.

get_credentials(name: str, version: str, credential_request: _models.EvaluatorCredentialRequest, *, content_type: str = 'application/json', **kwargs: Any) -> _models.DatasetCredential

Parameters

Name	Description
name Required	str Required.
version Required	str The specific version id of the EvaluatorVersion to operate on. Required.
credential_request Required	EvaluatorCredentialRequest or <xref:JSON> or IO[bytes] The credential request parameters. Is one of the following types: EvaluatorCredentialRequest, JSON, IO[bytes] Required.

Returns

Type	Description
DatasetCredential	DatasetCredential. The DatasetCredential is compatible with MutableMapping

Exceptions

Type	Description
HttpResponseError

get_generation_job

Get an evaluator generation job.

Gets the details of an evaluator generation job by its ID.

get_generation_job(job_id: str, **kwargs: Any) -> EvaluatorGenerationJob

Parameters

Name	Description
job_id Required	str The ID of the job. Required.

Returns

Type	Description
EvaluatorGenerationJob	EvaluatorGenerationJob. The EvaluatorGenerationJob is compatible with MutableMapping

Exceptions

Type	Description
HttpResponseError

get_version

Get an evaluator version.

Retrieves the specified evaluator version, returning 404 if it does not exist.

get_version(name: str, version: str, **kwargs: Any) -> EvaluatorVersion

Parameters

Name	Description
name Required	str The name of the resource. Required.
version Required	str The specific version id of the EvaluatorVersion to retrieve. Required.

Returns

Type	Description
EvaluatorVersion	EvaluatorVersion. The EvaluatorVersion is compatible with MutableMapping

Exceptions

Type	Description
HttpResponseError

list

List latest evaluator versions.

Lists the latest version of each evaluator.

list(*, type: Literal['builtin'] | Literal['custom'] | Literal['all'] | str | None = None, limit: int | None = None, **kwargs: Any) -> ItemPaged[EvaluatorVersion]

Keyword-Only Parameters

Name	Description
type	str or str or str or str Filter evaluators by type. Possible values: 'all', 'custom', 'builtin'. Is one of the following types: Literal["builtin"], Literal["custom"], Literal["all"], str Default value is None. Default value: None
limit	int A limit on the number of objects to be returned. Limit can range between 1 and 100, and the default is 20. Default value is None. Default value: None

Returns

Type	Description
ItemPaged[EvaluatorVersion]	An iterator like instance of EvaluatorVersion

Exceptions

Type	Description
HttpResponseError

list_generation_jobs

List evaluator generation jobs.

Returns a list of evaluator generation jobs. The List API has up to a few seconds of propagation delay, so a recently created job may not appear immediately; use the Get evaluator generation job API with the job ID to retrieve a specific job without delay.

list_generation_jobs(*, limit: int | None = None, order: str | PageOrder | None = None, before: str | None = None, **kwargs: Any) -> ItemPaged[EvaluatorGenerationJob]

Keyword-Only Parameters

Name	Description
limit	int A limit on the number of objects to be returned. Limit can range between 1 and 100, and the default is 20. Default value is None. Default value: None
order	str or PageOrder Sort order by the `created_at` timestamp of the objects. `asc` for ascending order and`desc` for descending order. Known values are: "asc" and "desc". Default value is None. Default value: None
before	str A cursor for use in pagination. `before` is an object ID that defines your place in the list. For instance, if you make a list request and receive 100 objects, ending with obj_foo, your subsequent call can include before=obj_foo in order to fetch the previous page of the list. Default value is None. Default value: None

Returns

Type	Description
ItemPaged[EvaluatorGenerationJob]	An iterator like instance of EvaluatorGenerationJob

Exceptions

Type	Description
HttpResponseError

list_versions

List evaluator versions.

Returns the available versions for the specified evaluator.

list_versions(name: str, *, type: Literal['builtin'] | Literal['custom'] | Literal['all'] | str | None = None, limit: int | None = None, **kwargs: Any) -> ItemPaged[EvaluatorVersion]

Parameters

Name	Description
name Required	str The name of the resource. Required.

Keyword-Only Parameters

Name	Description
type	str or str or str or str Filter evaluators by type. Possible values: 'all', 'custom', 'builtin'. Is one of the following types: Literal["builtin"], Literal["custom"], Literal["all"], str Default value is None. Default value: None
limit	int A limit on the number of objects to be returned. Limit can range between 1 and 100, and the default is 20. Default value is None. Default value: None

Returns

Type	Description
ItemPaged[EvaluatorVersion]	An iterator like instance of EvaluatorVersion

Exceptions

Type	Description
HttpResponseError

pending_upload

Start a pending upload.

Initiates a new pending upload or retrieves an existing one for the specified evaluator version.

pending_upload(name: str, version: str, pending_upload_request: _models.PendingUploadRequest, *, content_type: str = 'application/json', **kwargs: Any) -> _models.PendingUploadResponse

Parameters

Name	Description
name Required	str Required.
version Required	str The specific version id of the EvaluatorVersion to operate on. Required.
pending_upload_request Required	PendingUploadRequest or <xref:JSON> or IO[bytes] The pending upload request parameters. Is one of the following types: PendingUploadRequest, JSON, IO[bytes] Required.

Returns

Type	Description
PendingUploadResponse	PendingUploadResponse. The PendingUploadResponse is compatible with MutableMapping

Exceptions

Type	Description
HttpResponseError

update_version

Update an evaluator version.

Updates the specified evaluator version in place.

update_version(name: str, version: str, evaluator_version: _models.EvaluatorVersion, *, content_type: str = 'application/json', **kwargs: Any) -> _models.EvaluatorVersion

Parameters

Name	Description
name Required	str The name of the resource. Required.
version Required	str The version of the EvaluatorVersion to update. Required.
evaluator_version Required	EvaluatorVersion or <xref:JSON> or IO[bytes] Evaluator resource. Is one of the following types: EvaluatorVersion, JSON, IO[bytes] Required.

Returns

Type	Description
EvaluatorVersion	EvaluatorVersion. The EvaluatorVersion is compatible with MutableMapping

Exceptions

Type	Description
HttpResponseError

Feedback

Was this page helpful?