Skip to content

container logs timestamp precision differs in web service vs local #27691

@brasie

Description

@brasie

Issue Description

The log timestamp output is different depending on if one is querying locally vs querying via the podman system service API; the former includes nanosecond precision, and the latter is only second precision.

Steps to reproduce the issue

Assume one is running a container such as podman run alpine ash -c 'n=0; while true; do sleep 1; echo $n; n=$((n+1)); done

Compare the output of podman container logs --timestamps and sudo curl --unix-socket=/path/to/podman.sock http://d/v5.7.0/libpod/containers/<name>/logs --url-query timestamps=true --url-query stdout=true --output -

Describe the results you received

Describe the results you received

If one uses podman container logs --timestamps for a container running locally, the timestamp is formatted with nanosecond precision. Like,

2025-12-04T16:59:40.662732000-08:00 0
2025-12-04T16:59:41.665333000-08:00 1
2025-12-04T16:59:42.668075000-08:00 2
2025-12-04T16:59:43.670710000-08:00 3
2025-12-04T16:59:44.673337000-08:00 4

However, if one queries for the logs with timestamps via the API exposed using podman system service, e.g. at libpod/containers/<name>/logs, then there is one second precision. Like,

2025-12-04T16:59:40-08:00 0
2025-12-04T16:59:41-08:00 1
2025-12-04T16:59:42-08:00 2
2025-12-04T16:59:43-08:00 3
2025-12-04T16:59:44-08:00 4

Describe the results you expected

The inconsistency seems like a bug. It would be nice to have the nanosecond precision in both cases.

I'm not too familiar with the code, but it looks like it might come down to:

frame.WriteString(line.Time.Format(time.RFC3339))

In particular, using time.RFC3339Nano in place of time.RFC3339

podman info output

host:
  arch: amd64
  buildahVersion: 1.42.0
  cgroupControllers:
  - cpuset
  - cpu
  - io
  - memory
  - hugetlb
  - pids
  - rdma
  - misc
  - dmem
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-1:2.1.13-1
    path: /usr/bin/conmon
    version: 'conmon version 2.1.13, commit: 82de887596ed8ee6d9b2ee85e4f167f307bb569b'
  cpuUtilization:
    idlePercent: 96.95
    systemPercent: 1.04
    userPercent: 2.02
  cpus: 4
  databaseBackend: sqlite
  distribution:
    distribution: arch
    version: unknown
  eventLogger: journald
  freeLocks: 2038
  hostname: bradenhp
  idMappings:
    gidmap: null
    uidmap: null
  kernel: 6.17.7-arch1-1
  linkmode: dynamic
  logDriver: journald
  memFree: 432844800
  memTotal: 16142561280
  networkBackend: netavark
  networkBackendInfo:
    backend: netavark
    dns:
      package: aardvark-dns-1.16.0-1
      path: /usr/lib/podman/aardvark-dns
      version: aardvark-dns 1.16.0
    package: netavark-1.16.1-1
    path: /usr/lib/podman/netavark
    version: netavark 1.16.1
  ociRuntime:
    name: crun
    package: crun-1.25-1
    path: /usr/bin/crun
    version: |-
      crun version 1.25
      commit: d9a0adce065c7747ab88ea6ccc42b15a626e08e1
      rundir: /run/crun
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL
  os: linux
  pasta:
    executable: /usr/bin/pasta
    package: passt-2025_09_19.623dbf6-1
    version: |
      pasta 2025_09_19.623dbf6
      Copyright Red Hat
      GNU General Public License, version 2 or later
        <https://www.gnu.org/licenses/old-licenses/gpl-2.0.html>
      This is free software: you are free to change and redistribute it.
      There is NO WARRANTY, to the extent permitted by law.
  remoteSocket:
    exists: true
    path: /run/podman/podman.sock
  rootlessNetworkCmd: pasta
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: false
    seccompEnabled: true
    seccompProfilePath: /etc/containers/seccomp.json
    selinuxEnabled: false
  serviceIsRemote: false
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.3.3-1
    version: |-
      slirp4netns version 1.3.3
      commit: 944fa94090e1fd1312232cbc0e6b43585553d824
      libslirp: 4.9.1
      SLIRP_CONFIG_VERSION_MAX: 6
      libseccomp: 2.5.6
  swapFree: 15062171648
  swapTotal: 17179865088
  uptime: 626h 23m 11.00s (Approximately 26.08 days)
  variant: ""
plugins:
  authorization: null
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  - ipvlan
  volume:
  - local
registries: {}
store:
  configFile: /etc/containers/storage.conf
  containerStore:
    number: 10
    paused: 0
    running: 3
    stopped: 7
  graphDriverName: overlay
  graphOptions:
    overlay.mountopt: nodev
  graphRoot: /var/lib/containers/storage
  graphRootAllocated: 250358661120
  graphRootUsed: 235094360064
  graphStatus:
    Backing Filesystem: extfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Supports shifting: "true"
    Supports volatile: "true"
    Using metacopy: "true"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 1
  runRoot: /run/containers/storage
  transientStore: false
  volumePath: /var/lib/containers/storage/volumes
version:
  APIVersion: 5.7.0
  Built: 1763023417
  BuiltTime: Thu Nov 13 00:43:37 2025
  GitCommit: 0370128fc8dcae93533334324ef838db8f8da8cb
  GoVersion: go1.25.4 X:nodwarf5
  Os: linux
  OsArch: linux/amd64
  Version: 5.7.0

Podman in a container

No

Privileged Or Rootless

Privileged

Upstream Latest Release

Yes

Additional environment details

Additional environment details

Additional information

Additional information like issue happens only occasionally or issue happens with a particular architecture or on a particular setting

Metadata

Metadata

Assignees

Labels

Good First IssueThis issue would be a good issue for a first time contributor to undertake.kind/bugCategorizes issue or PR as related to a bug.triagedIssue has been triagedvolunteers-wantedIssues good for community/volunteer contributions

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions