Skip to content

feature: (helm) support custom volumes and volumeMounts for epp #1945

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
delavet wants to merge 1 commit into
base: main
Choose a base branch
from
+22 −1
Open

feature: (helm) support custom volumes and volumeMounts for epp #1945

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions config/charts/inferencepool/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -247,6 +247,21 @@ The following table list the configurable parameters of the chart.
| `inferenceExtension.tracing.otelExporterEndpoint` | OpenTelemetry collector endpoint. |
| `inferenceExtension.tracing.sampling.sampler` | The trace sampler to use. Currently, only `parentbased_traceidratio` is supported. This sampler respects the parent span’s sampling decision when present, and applies the configured ratio for root spans. |
| `inferenceExtension.tracing.sampling.samplerArg` | Sampler-specific argument. For `parentbased_traceidratio`, this defines the base sampling rate for new traces (root spans), as a float string in the range [0.0, 1.0]. For example, "0.1" enables 10% sampling. |
| `inferenceExtension.volumes` | List of volumes to mount in the EPP deployment as free-form YAML. Optional. |
| `inferenceExtension.volumeMounts` | List of volume mounts for the EPP container as free-form YAML. Optional. |
| `inferenceExtension.sidecar.enabled` | Enables or disables the sidecar container in the EPP deployment. Defaults to `false`. |
| `inferenceExtension.sidecar.name` | Name of the sidecar container. Required when the sidecar is enabled. |
| `inferenceExtension.sidecar.image` | Image for the sidecar container. Required when the sidecar is enabled. |
| `inferenceExtension.sidecar.imagePullPolicy` | Image pull policy for the sidecar container. Possible values: `Always`, `IfNotPresent`, or `Never`. Defaults to `IfNotPresent`. |
| `inferenceExtension.sidecar.command` | Command to run in the sidecar container as a single string. Optional. |
| `inferenceExtension.sidecar.args` | Arguments to pass to the command in the sidecar container as a list of strings. Optional. |
| `inferenceExtension.sidecar.env` | Environment variables to set in the sidecar container as free-form YAML. Optional. |
| `inferenceExtension.sidecar.ports` | List of ports to expose for the sidecar container. Optional. |
| `inferenceExtension.sidecar.livenessProbe` | Liveness probe configuration for the sidecar container. Optional. |
| `inferenceExtension.sidecar.readinessProbe` | Readiness probe configuration for the sidecar container. Optional. |
| `inferenceExtension.sidecar.resources` | Resource limits and requests for the sidecar container. Optional. |
| `inferenceExtension.sidecar.volumeMounts` | List of volume mounts for the sidecar container. Optional. |
| `inferenceExtension.sidecar.volumes` | List of volumes for the sidecar container. Optional. |
| `provider.name` | Name of the Inference Gateway implementation being used. Possible values: [`none`, `gke`, or `istio`]. Defaults to `none`. |
| `provider.gke.autopilot` | Set to `true` if the cluster is a GKE Autopilot cluster. This is only used if `provider.name` is `gke`. Defaults to `false`. |

Expand Down
2 changes: 1 addition & 1 deletion config/charts/inferencepool/templates/epp-config.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ data:
{{- end }}

---
{{- if .Values.inferenceExtension.sidecar.enabled }}
{{- if and .Values.inferenceExtension.sidecar.enabled .Values.inferenceExtension.sidecar.configMap }}
apiVersion: v1
kind: ConfigMap
metadata:
Expand Down
6 changes: 6 additions & 0 deletions config/charts/inferencepool/templates/epp-deployment.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -182,8 +182,14 @@ spec:
volumeMounts:
- name: plugins-config-volume
mountPath: "/config"
{{- if .Values.inferenceExtension.volumeMounts }}
{{- tpl (toYaml .Values.inferenceExtension.volumeMounts) $ | nindent 8 }}
{{- end }}
{{- include "gateway-api-inference-extension.latencyPredictor.containers" . | nindent 6 }}
volumes:
{{- if .Values.inferenceExtension.volumes }}
{{- tpl (toYaml .Values.inferenceExtension.volumes) $ | nindent 6 }}
{{- end }}
{{- if .Values.inferenceExtension.sidecar.volumes }}
{{- tpl (toYaml .Values.inferenceExtension.sidecar.volumes) $ | nindent 6 }}
{{- end }}
Expand Down