API Reference¶
Packages¶
litellm.litellm.ai/v1alpha1¶
Package v1alpha1 contains API Schema definitions for the litellm v1alpha1 API group.
Resource Types¶
ConnectionRef¶
Appears in: - ModelSpec
| Field | Description | Default | Validation |
|---|---|---|---|
secretRef SecretRef |
|||
instanceRef InstanceRef |
DatabaseSecretKeys¶
Appears in: - DatabaseSecretRef
| Field | Description | Default | Validation |
|---|---|---|---|
hostSecret string |
|||
passwordSecret string |
|||
usernameSecret string |
|||
dbnameSecret string |
DatabaseSecretRef¶
Appears in: - LiteLLMInstanceSpec
| Field | Description | Default | Validation |
|---|---|---|---|
nameRef string |
|||
keys DatabaseSecretKeys |
Gateway¶
Appears in: - LiteLLMInstanceSpec
| Field | Description | Default | Validation |
|---|---|---|---|
enabled boolean |
|||
host string |
Ingress¶
Appears in: - LiteLLMInstanceSpec
| Field | Description | Default | Validation |
|---|---|---|---|
enabled boolean |
|||
host string |
InitModelInstance¶
model instance used to create proxy server config map
Appears in: - LiteLLMInstanceSpec
| Field | Description | Default | Validation |
|---|---|---|---|
modelName string |
|||
requiresAuth boolean |
|||
identifier string |
|||
modelCredentials ModelCredentialSecretRef |
|||
liteLLMParams LiteLLMParams |
|||
additionalProperties RawExtension |
InstanceRef¶
Appears in: - ConnectionRef
| Field | Description | Default | Validation |
|---|---|---|---|
namespace string |
|||
name string |
LiteLLMInstance¶
LiteLLMInstance is the Schema for the litellminstances API.
Appears in: - LiteLLMInstanceList
| Field | Description | Default | Validation |
|---|---|---|---|
apiVersion string |
litellm.litellm.ai/v1alpha1 |
||
kind string |
LiteLLMInstance |
||
kind string |
Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds |
||
apiVersion string |
APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources |
||
metadata ObjectMeta |
Refer to Kubernetes API documentation for fields of metadata. |
||
spec LiteLLMInstanceSpec |
|||
status LiteLLMInstanceStatus |
LiteLLMInstanceList¶
LiteLLMInstanceList contains a list of LiteLLMInstance.
| Field | Description | Default | Validation |
|---|---|---|---|
apiVersion string |
litellm.litellm.ai/v1alpha1 |
||
kind string |
LiteLLMInstanceList |
||
kind string |
Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds |
||
apiVersion string |
APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources |
||
metadata ListMeta |
Refer to Kubernetes API documentation for fields of metadata. |
||
items LiteLLMInstance array |
LiteLLMInstanceSpec¶
LiteLLMInstanceSpec defines the desired state of LiteLLMInstance.
Appears in: - LiteLLMInstance
| Field | Description | Default | Validation |
|---|---|---|---|
image string |
ghcr.io/berriai/litellm-database:main-v1.74.9.rc.1 | ||
masterKey string |
|||
databaseSecretRef DatabaseSecretRef |
|||
redisSecretRef RedisSecretRef |
|||
ingress Ingress |
|||
gateway Gateway |
|||
replicas integer |
1 | ||
models InitModelInstance array |
|||
extraEnvVars EnvVar array |
LiteLLMInstanceStatus¶
LiteLLMInstanceStatus defines the observed state of LiteLLMInstance.
Appears in: - LiteLLMInstance
| Field | Description | Default | Validation |
|---|---|---|---|
observedGeneration integer |
ObservedGeneration represents the .metadata.generation that the condition was set based upon | ||
lastUpdated Time |
LastUpdated represents the last time the status was updated | ||
configMapCreated boolean |
Resource creation status | ||
secretCreated boolean |
|||
deploymentCreated boolean |
|||
serviceCreated boolean |
|||
ingressCreated boolean |
|||
conditions Condition array |
Conditions represent the latest available observations of a LiteLLM instance's state |
LiteLLMParams¶
LiteLLMParams defines the LiteLLM parameters for a model.
Appears in: - InitModelInstance - ModelSpec - ModelStatus
| Field | Description | Default | Validation |
|---|---|---|---|
inputCostPerToken string |
InputCostPerToken is the cost per input token | ||
outputCostPerToken string |
OutputCostPerToken is the cost per output token | ||
inputCostPerSecond string |
InputCostPerSecond is the cost per second for input | ||
outputCostPerSecond string |
OutputCostPerSecond is the cost per second for output | ||
inputCostPerPixel string |
InputCostPerPixel is the cost per pixel for input | ||
outputCostPerPixel string |
OutputCostPerPixel is the cost per pixel for output | ||
apiKey string |
APIKey is the API key for the model | ||
apiBase string |
APIBase is the base URL for the API | ||
apiVersion string |
APIVersion is the version of the API | ||
vertexProject string |
VertexProject is the Google Cloud project for Vertex AI | ||
vertexLocation string |
VertexLocation is the location for Vertex AI | ||
vertexCredentials string |
VertexCredentials is the credentials for Vertex AI | ||
regionName string |
RegionName is the region name for the service | ||
awsAccessKeyId string |
AWSAccessKeyID is the AWS access key ID | ||
awsSecretAccessKey string |
AWSSecretAccessKey is the AWS secret access key | ||
awsRegionName string |
AWSRegionName is the AWS region name | ||
watsonxRegionName string |
WatsonXRegionName is the WatsonX region name | ||
customLLMProvider string |
CustomLLMProvider is the custom LLM provider | ||
tpm integer |
TPM is tokens per minute | ||
rpm integer |
RPM is requests per minute | ||
timeout integer |
Timeout is the timeout in seconds | ||
streamTimeout integer |
StreamTimeout is the stream timeout in seconds | ||
maxRetries integer |
MaxRetries is the maximum number of retries | ||
organization string |
Organization is the organization name | ||
configurableClientsideAuthParams RawExtension |
ConfigurableClientsideAuthParams are configurable client-side auth parameters | ||
litellmCredentialName string |
LiteLLMCredentialName is the LiteLLM credential name | ||
litellmTraceId string |
LiteLLMTraceID is the LiteLLM trace ID | ||
maxFileSizeMb integer |
MaxFileSizeMB is the maximum file size in MB | ||
maxBudget string |
MaxBudget is the maximum budget | ||
budgetDuration string |
BudgetDuration is the budget duration | ||
useInPassThrough boolean |
UseInPassThrough indicates if to use in pass through | ||
useLitellmProxy boolean |
UseLiteLLMProxy indicates if to use LiteLLM proxy | ||
mergeReasoningContentInChoices boolean |
MergeReasoningContentInChoices indicates if to merge reasoning content in choices | ||
modelInfo RawExtension |
ModelInfo contains additional model information | ||
mockResponse string |
MockResponse is the mock response | ||
autoRouterConfigPath string |
AutoRouterConfigPath is the auto router config path | ||
autoRouterConfig string |
AutoRouterConfig is the auto router config | ||
autoRouterDefaultModel string |
AutoRouterDefaultModel is the auto router default model | ||
autoRouterEmbeddingModel string |
AutoRouterEmbeddingModel is the auto router embedding model | ||
model string |
Model is the model name | ||
additionalProp1 RawExtension |
AdditionalProps contains additional properties |
Model¶
Model is the Schema for the models API.
Appears in: - ModelList
| Field | Description | Default | Validation |
|---|---|---|---|
apiVersion string |
litellm.litellm.ai/v1alpha1 |
||
kind string |
Model |
||
kind string |
Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds |
||
apiVersion string |
APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources |
||
metadata ObjectMeta |
Refer to Kubernetes API documentation for fields of metadata. |
||
spec ModelSpec |
|||
status ModelStatus |
ModelCredentialSecretKeys¶
Appears in: - ModelCredentialSecretRef
| Field | Description | Default | Validation |
|---|---|---|---|
apiKey string |
|||
apiBase string |
|||
awsSecretAccessKey string |
|||
awsAccessKeyId string |
|||
vertexCredentials string |
|||
vertexProject string |
ModelCredentialSecretRef¶
Appears in: - InitModelInstance
| Field | Description | Default | Validation |
|---|---|---|---|
nameRef string |
|||
keys ModelCredentialSecretKeys |
ModelInfo¶
ModelInfo defines the model information.
Appears in: - ModelSpec
| Field | Description | Default | Validation |
|---|---|---|---|
id string |
ID is the model ID | ||
dbModel boolean |
DBModel indicates if this is a database model | ||
teamId string |
TeamID is the team ID | ||
teamPublicModelName string |
TeamPublicModelName is the team public model name | ||
additionalProp1 RawExtension |
AdditionalProps contains additional properties |
ModelList¶
ModelList contains a list of Model.
| Field | Description | Default | Validation |
|---|---|---|---|
apiVersion string |
litellm.litellm.ai/v1alpha1 |
||
kind string |
ModelList |
||
kind string |
Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds |
||
apiVersion string |
APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources |
||
metadata ListMeta |
Refer to Kubernetes API documentation for fields of metadata. |
||
items Model array |
ModelSpec¶
ModelSpec defines the desired state of Model.
Appears in: - Model
| Field | Description | Default | Validation |
|---|---|---|---|
connectionRef ConnectionRef |
ConnectionRef is the connection reference | ||
modelName string |
ModelName is the name of the model | ||
litellmParams LiteLLMParams |
LiteLLMParams contains the LiteLLM parameters | ||
modelInfo ModelInfo |
ModelInfo contains the model information | ||
modelSecretRef SecretRef |
ModelSecretRef is the model secret reference |
ModelStatus¶
ModelStatus defines the observed state of Model.
Appears in: - Model
| Field | Description | Default | Validation |
|---|---|---|---|
observedGeneration integer |
ObservedGeneration represents the .metadata.generation that the condition was set based upon | ||
lastUpdated Time |
LastUpdated represents the last time the status was updated | ||
conditions Condition array |
Conditions represent the latest available observations of a LiteLLM instance's state | ||
modelName string |
ModelName is the name of the model | ||
litellmParams LiteLLMParams |
LiteLLMParams contains the LiteLLM parameters | ||
modelId string |
ModelId contains the model uuid provided by litellm server |
RedisSecretKeys¶
Appears in: - RedisSecretRef
| Field | Description | Default | Validation |
|---|---|---|---|
hostSecret string |
|||
portSecret string |
|||
passwordSecret string |
RedisSecretRef¶
Appears in: - LiteLLMInstanceSpec
| Field | Description | Default | Validation |
|---|---|---|---|
nameRef string |
|||
keys RedisSecretKeys |
SecretRef¶
Appears in: - ConnectionRef - ModelSpec
| Field | Description | Default | Validation |
|---|---|---|---|
namespace string |
|||
secretName string |