dCache View pool-info 400 error in dCache 11.2.0

[ArturAkh]

Dear dCache developers,

I've installed a newly released dCache version 11.2.0 on our test instance. While operation-critical functionalities work fine for us, we have found the following issue:

I'm attaching an AI-based analysis document on the cause and a proposed fix.

pool-info-usage-400-issue.md

Thank you very much for fixing that quality-of-life issue in advance :)

Cheers,

Artur

[mksahakyan]

Hi @ArturAkh ,

do you see any error in logs?
thanks

[ArturAkh]

Hi @mksahakyan,

I don't see anything in the frontend logs, but I can easily reproduce with curl:

$ curl -k -H 'Content-Type: application/json' -X GET https://ppsdcacheweb-kit.gridka.de:3880/api/v1/pools/f01-152-138-e_D_shared/usage  --oauth2-bearer $BEARER_TOKEN | jq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   567  100   567    0     0   5224      0 --:--:-- --:--:-- --:--:--  5250
{
  "detail": "Unable to interpret JSON: Java 8 optional type `java.util.Optional<java.lang.Double>` not supported by default: add Module \"com.fasterxml.jackson.datatype:jackson-datatype-jdk8\" to enable handling (through reference chain: org.dcache.restful.providers.pool.PoolInfo[\"poolData\"]->org.dcache.pool.json.PoolData[\"sweeperData\"]->org.dcache.pool.classic.json.SweeperData[\"lastAccessHistogram\"]->org.dcache.util.histograms.CountingHistogram[\"metadata\"]->org.dcache.util.histograms.HistogramMetadata[\"maxValue\"])",
  "title": "Bad Request",
  "status": "400"
}

The following works, though:

$ curl -k -H 'Content-Type: application/json' -X GET https://ppsdcacheweb-kit.gridka.de:3880/api/v1/poolgroups/all-pools/usage  --oauth2-bearer $BEARER_TOKEN | jq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  1498  100  1498    0     0  16359      0 --:--:-- --:--:-- --:--:-- 16461
{
  "cellDataForPools": {
    "f01-152-138-e_wT_shared": {
      "label": "Cell Info",
      "cellName": "f01-152-138-e_wT_shared",
      "cellType": "Generic",
      "cellClass": "Pool",
      "domainName": "f01-152-138-e_wT_sharedDomain",
      "creationTime": 1769104694548,
      "state": 1,
      "eventQueueSize": 0,
      "threadCount": 114,
      "expectedQueueTime": 0,
      "version": "11.2.0(11.2.0)",
      "release": "11.2.0",
      "revision": "11.2.0",
      "roundTripTime": 6
    },
    "f01-152-138-e_sT_shared": {
      "label": "Cell Info",
      "cellName": "f01-152-138-e_sT_shared",
      "cellType": "Generic",
      "cellClass": "Pool",
      "domainName": "f01-152-138-e_sT_sharedDomain",
      "creationTime": 1769104694509,
      "state": 1,
      "eventQueueSize": 0,
      "threadCount": 114,
      "expectedQueueTime": 0,
      "version": "11.2.0(11.2.0)",
      "release": "11.2.0",
      "revision": "11.2.0",
      "roundTripTime": 6
    },
    "f01-152-138-e_D_shared": {
      "label": "Cell Info",
      "cellName": "f01-152-138-e_D_shared",
      "cellType": "Generic",
      "cellClass": "Pool",
      "domainName": "f01-152-138-e_D_sharedDomain",
      "creationTime": 1769104694536,
      "state": 1,
      "eventQueueSize": 0,
      "threadCount": 295,
      "expectedQueueTime": 0,
      "version": "11.2.0(11.2.0)",
      "release": "11.2.0",
      "revision": "11.2.0",
      "roundTripTime": 7
    }
  }
}

[mksahakyan]

Hi Artur, " Unable to interpret JSON: Java 8 optional type ` java.util.Optional < java.lang.Double >" the java version must be 17th. thanks Marina

[XMol]

" Unable to interpret JSON: Java 8 optional type ` java.util.Optional < java.lang.Double >"

According to the process table, we're running dCache v11.2 with Java-21:

[root@dcppsweb ~]# cat /etc/dcache.env
# Managed by Puppet.
# Any modifications will be gone with the next Puppet synchronization!

JAVA_HOME=/usr/lib/jvm/jre-21
JAVA=/usr/lib/jvm/jre-21/bin/java

[root@dcppsweb ~]# dcache services | grep httpdDomain
httpdDomain     httpd      httpd             No      No
httpdDomain     info       info              -       Yes
httpdDomain     statistics PoolStatistics    -       No

[root@dcppsweb ~]# ps -f 53739 | cat
UID          PID    PPID  C STIME TTY      STAT   TIME CMD
scc-dca+   53739       1  0 Jan22 ?        Ssl   15:51 /usr/lib/jvm/jre-21/bin/java -server -Xmx2g -XX:MaxDirectMemorySize=512m -Dsun.net.inetaddr.ttl=1800 -Dorg.globus.tcp.port.range=20000,25000 -Dorg.dcache.dcap.port=0 -Dorg.dcache.ftp.log-aborted-transfers=true -Dorg.dcache.net.tcp.portrange=33115:33145 -Djava.security.krb5.realm= -Djava.security.krb5.kdc= -Djavax.security.auth.useSubjectCredsOnly=false -Djava.security.auth.login.config=/etc/dcache/jgss.conf -Dcontent.types.user.table=/etc/dcache/content-types.properties -Dzookeeper.sasl.client=false -Dcurator-dont-log-connection-problems=true -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/var/log/dcache/httpdDomain-oom.hprof -XX:+ExitOnOutOfMemoryError -XX:+StartAttachListener -javaagent:/usr/share/dcache/classes/aspectjweaver-1.9.24.jar -Djava.net.preferIPv6Addresses=system --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/java.net=ALL-UNNAMED --add-opens=java.base/java.util.concurrent=ALL-UNNAMED --add-opens=java.base/java.text=ALL-UNNAMED --add-opens=java.sql/java.sql=ALL-UNNAMED --add-opens=java.base/java.math=ALL-UNNAMED --add-opens java.base/sun.nio.fs=ALL-UNNAMED -Djava.awt.headless=true -DwantLog4jSetup=n -Dorg.bouncycastle.dh.allow_unsafe_p_value=true -javaagent:/usr/share/java/jolokia-jvm-agent/jolokia-agent-jvm-javaagent.jar=port=7771,host=localhost -Ddcache.home=/usr/share/dcache -Ddcache.paths.defaults=/usr/share/dcache/defaults org.dcache.boot.BootLoader start httpdDomain

[mksahakyan]

Thank you @XMol..

[mksahakyan]

It could be somethig related to jackson... i will check it.

[XMol]

That's also what the AI concludes in Artur's summary document:

Jackson is attempting to serialize a type containing Optional<Double> (in pool usage histogram metadata) but the REST ObjectMapper does not register the JDK8 module. Without jackson-datatype-jdk8, Jackson cannot serialize Optional and returns 400.

[DmitryLitvintsev]

https://rb.dcache.org/r/14616/