xmlsec.InternalError: (-1, 'lxml & xmlsec libxml2 library version mismatch')

[vaibhavmano]

Issues with xmlsec.InternalError: (-1, 'lxml & xmlsec libxml2 library version mismatch')

We're currently using python3-saml and we're running into issues during development and production. We have a variety of devices that we use for development including Intel Mac, M-Series Mac and our production deployment is a docker container running on Ubuntu 24.04.

I've tried various fixed including pinning the version of the both xmlsec and lxml. Below are the versions that is currently being used.
"python3-saml==1.16.0"
"lxml>=4.6.5,!=4.7.0,<=5.2.1"

Below is my docker file and

RUN <<EOF
set -e
apt-get update -y
apt-get install -y libxml2-dev libxmlsec1-dev libxmlsec1-openssl pkg-config
rm -rf /var/lib/apt/lists/
EOF


COPY /pyproject.toml /build/
COPY /uv.lock /build/
COPY /baselib /build/baselib
COPY /bo /build/bo
COPY /user /build/user

WORKDIR /build
RUN --mount=type=cache,target=${UV_CACHE_DIR} \
    uv sync --active --frozen --no-group dev --package user --no-binary-package lxml

This seems to fix the issue when I build and run the image locally (Intel Mac) but fails when deploying to production. Is there a proper way to resolve this issue because this seems like it has been going on forever with no real fix other than to build the packages directly from source?

[sawyerb-ksu]

I had this exact issue, I (think) it is a restriction for Ubuntu 24.04 LTS vs 24.10. The libxml2 version for 24.04 is 2.9.14, while Ubuntu 24.10 has libxml2: >2.12.7. This creates a mismatch.

lxml 5.x is compatible with libxml2 2.9.x
lxml 6.x requires libxml2 2.12.0+ (not available on Ubuntu 24.04 LTS)

lxml 6.0.2: Bundles its own libxml2 2.14.6
xmlsec 1.3.16: Links against system libxml2 2.9.14

Setting dependencies to:
"lxml>=5.0.0,<6.0.0"
"xmlsec>=1.3.14,<1.4.0"

Seems to resolve the mismatch until upgrading to 24.10 or future ubuntu release. I also might be crazy and there is a better solution I am missing.

[pitbulk]

Have you tried not installing lxml from binary?

pip install --force-reinstall --no-binary lxml lxml 

See https://github.com/SAML-Toolkits/python3-saml?tab=readme-ov-file#note

I'm not aware of any other workaround, you can directly check at python-xmlsec project.

If you are installing in your system the latest xmlsec 1.3.16, check its release notes