Can not add bridge with iOS13?

Since iOS 13 I can not add the bridge. With iOS 12 it works. Is there anything new what is should be aware of? The iPhone says it can not be added because it could not connect to the device. Thanks for help and thank you for the project!

I have the same problem

are you running the latest code from HAP-Java, including https://github.com/hap-java/HAP-Java/commit/0201d9a3c1b5040d19510ed1677e8a0a18ac934f ? That should fix iOS 13. If you're using it within OpenHAB, you might want to check out https://github.com/ccutrer/openhab2-addons/releases/tag/2.5.0-cody2 (but with caveats). Otherwise, you'll need to compile and drop latest HAP-Java into whatever other project yourself; there's not a new release yet.

Pairing is not working for me, too. It seems that netty closes the connection too early. I added some Thread.sleep and it helped to get some steps further, but finally failed.

are you running the latest code from HAP-Java, including 0201d9a ? That should fix iOS 13. If you're using it within OpenHAB, you might want to check out https://github.com/ccutrer/openhab2-addons/releases/tag/2.5.0-cody2 (but with caveats). Otherwise, you'll need to compile and drop latest HAP-Java into whatever other project yourself; there's not a new release yet.

Compiled current code but same issue :/.

I'm having the same issue. I'm seeing the following error when I try to pair my phone to openHAB:
2019-11-01 23:09:59.435 [ERROR] [.hapjava.impl.pairing.PairingManager] - Exception while finalizing pairing org.bouncycastle.crypto.tls.TlsFatalAlert: bad_record_mac(20) 	at io.github.hapjava.impl.crypto.ChachaDecoder.decodeCiphertext(ChachaDecoder.java:31) ~[299:org.openhab.io.homekit:<IP_ADDRESS>911012204] 	at io.github.hapjava.impl.crypto.ChachaDecoder.decodeCiphertext(ChachaDecoder.java:41) ~[299:org.openhab.io.homekit:<IP_ADDRESS>911012204] 	at io.github.hapjava.impl.pairing.FinalPairHandler.decrypt(FinalPairHandler.java:44) ~[299:org.openhab.io.homekit:<IP_ADDRESS>911012204] 	at io.github.hapjava.impl.pairing.FinalPairHandler.handle(FinalPairHandler.java:39) ~[299:org.openhab.io.homekit:<IP_ADDRESS>911012204] 	at io.github.hapjava.impl.pairing.PairingManager.handle(PairingManager.java:59) [299:org.openhab.io.homekit:<IP_ADDRESS>911012204] 	at io.github.hapjava.impl.connections.HttpSession.handlePairSetup(HttpSession.java:109) [299:org.openhab.io.homekit:<IP_ADDRESS>911012204] 	at io.github.hapjava.impl.connections.HttpSession.handleRequest(HttpSession.java:53) [299:org.openhab.io.homekit:<IP_ADDRESS>911012204] 	at io.github.hapjava.impl.connections.ConnectionImpl.doHandleRequest(ConnectionImpl.java:54) [299:org.openhab.io.homekit:<IP_ADDRESS>911012204] 	at io.github.hapjava.impl.connections.ConnectionImpl.handleRequest(ConnectionImpl.java:47) [299:org.openhab.io.homekit:<IP_ADDRESS>911012204] 	at io.github.hapjava.impl.http.impl.AccessoryHandler.channelRead0(AccessoryHandler.java:52) [299:org.openhab.io.homekit:<IP_ADDRESS>911012204] 	at io.github.hapjava.impl.http.impl.AccessoryHandler.channelRead0(AccessoryHandler.java:17) [299:org.openhab.io.homekit:<IP_ADDRESS>911012204] 	at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105) [274:io.netty.transport:4.1.34.Final] 	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359) [274:io.netty.transport:4.1.34.Final] 	at io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:38) [274:io.netty.transport:4.1.34.Final] 	at io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:350) [274:io.netty.transport:4.1.34.Final] 	at io.netty.util.concurrent.DefaultEventExecutor.run(DefaultEventExecutor.java:66) [271:io.netty.common:4.1.34.Final] 	at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:905) [271:io.netty.common:4.1.34.Final] 	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [271:io.netty.common:4.1.34.Final] 	at java.lang.Thread.run(Thread.java:748) [?:?]

@frank-f this is with an upgraded bcprov, right? That‘s not working.

@J-N-K I finally got the version from your repo to compile - it's that one.

home app has different functions for initial devices load (json parsing) and for device updates.
the initial load is more strict if something not exactly to specification it just stops. the procedure was change with ios13.
what helps - start with a small set of device, e.g. lights, do initial load / add bridge, add more device afterwards.
e.g. there was an issue with thermostat.  i add first light and then thermostats.
if you have time, please try to figure out which device exactly is making issue (but trying to bridge with one device with two, etc), so that we can fix it.

home app has different functions for initial devices load (json parsing) and for device updates.
the initial load is more strict if something not exactly to specification it just stops. the procedure was change with ios13.
what helps - start with a small set of device, e.g. lights, do initial load / add bridge, add more device afterwards.
e.g. there was an issue with thermostat. i add first light and then thermostats.
if you have time, please try to figure out which device exactly is making issue (but trying to bridge with one device with two, etc), so that we can fix it.

Hey nice catch. But I even tried adding a bridge without any devices and that did not work as well. :(

Hey nice catch. But I even tried adding a bridge without any devices and that did not work as well. :(
and was it working on ios12? maybe something on the network level.
enable logging with

ssh -p 8101 openhab@localhost
default password is (habopen)
log:set DEBUG org.openhab.io.homekit
log:set DEBUG io.github.hapjava

and you should see the request from the phone in openhab.log

Hey nice catch. But I even tried adding a bridge without any devices and that did not work as well. :(
and was it working on ios12? maybe something on the network level.
enable logging with

ssh -p 8101 openhab@localhost
default password is (habopen)
log:set DEBUG org.openhab.io.homekit
log:set DEBUG io.github.hapjava

and you should see the request from the phone in openhab.log

I am not using openHAB I am using it as Library

if you have time, please try to figure out which device exactly is making issue (but trying to bridge with one device with two, etc), so that we can fix it.

Honestly, I gave up with this.
I installed v2.4.0-homekit-6 (https://github.com/hap-java/openhab2-addons/releases/), which didn't work with iOS 13 either. So I booted up an old an iOS 12 device, added everything flawlessly on the first try and now I'm happy I have my smart home stuff working again.
I have thermostats, lights (on/off and dimmable), a temperature sensor and door/window sensors - if that helps.

I have the same issue. I use the source code from here with a simple java main (from samples branch) but i am unable to bind with my iPhone. But it seams that some kind of timeout happen.
If i couple with on iPhone using iOS 12 it work but the iOS 13 devices still dont have access.
Didt some one mage to get it run? Or is someone looking for an solution.

i got it working. i have around 90 devices of all different types and it works perfect with ios13 as well as with latest Ipados and tvOs.
my changes are merged to master branch here, but it looks like they dont work for everyone.
if you want to give a try,

here is my openhab 2.4 addon jar
https://1drv.ms/u/s!AqqiEfFRr6WAgYQ8qsC_kZS6jTCsDw?e=W2z9kX
here is my java hap jar
https://1drv.ms/u/s!AqqiEfFRr6WAgYQ-iybGMnJ0B6GYRQ?e=A4awvN

the repos
https://github.com/yfre/HAP-Java
https://github.com/yfre/openhab2-addons/tree/2.4.0/new_devices

i got it working. i have around 90 devices of all different types and it works perfect with ios13 as well as with latest Ipados and tvOs.
my changes are merged to master branch here, but it looks like they dont work for everyone.
if you want to give a try,

here is my openhab 2.4 addon jar
https://1drv.ms/u/s!AqqiEfFRr6WAgYQ8qsC_kZS6jTCsDw?e=W2z9kX
here is my java hap jar
https://1drv.ms/u/s!AqqiEfFRr6WAgYQ-iybGMnJ0B6GYRQ?e=A4awvN

the repos
https://github.com/yfre/HAP-Java
https://github.com/yfre/openhab2-addons/tree/2.4.0/new_devices

Does not work for me :(

ok. try to enable the logging "io.github.hapjava" = DEBUG. then you should see the JSON the library sends to home app. this JSON must not have any "null" value or values out of range (json has min/max values as well).
here is an example from my json (target mode for thermostat, which must be between 0 and 3
                  {
                     "iid":11,
                     "type":"33",
                     "perms":[
                        "pw",
                        "pr",
                        "ev"
                     ],
                     "format":"int",
                     "ev":false,
                     "description":"Target Mode",
                     "value":0,
                     "minValue":0,
                     "maxValue":3,
                     "minStep":1
                  },

if you get your JSON, please paste it here,  this will help us to fix the library and make it more robust.

Like the others, 12 worked for me, 13 does not.
The problem appears to be with pairing - pairing completes and it seems (and is reported in HAB as) successful but then HomeKit disconnects.  It does not attempt to reconnect.  HomeKit gives up a few minutes later complaining it could not add the bridge and does not retry.
As far as I can tell it never gets to the step of loading accessories.

Exactly. We get to the final pairing step, then the connection closes. If we add sleeps (so that the connection stays alive a little bit longer), we sometimes get to the accessories read step and fail then.

@yfre I checked out:  https://github.com/yfre/HAP-Java  -b master
Then i add the java main from samples and a log4j2.xml with default log level debug:
The binding still failed. Here is the log from the connection attempt.
Using persisted auth
2019-11-10 20:53:02,482 main DEBUG Initializing configuration XmlConfiguration[location=C:\Users\heiko\workspace\HAP-Java\target\classes\log4j2.xml]
2019-11-10 20:53:02,489 main DEBUG Installed script engines
2019-11-10 20:53:02,824 main DEBUG Oracle Nashorn Version: 1.8.0_201, Language: ECMAScript, Threading: Not Thread Safe, Compile: true, Names: {nashorn, Nashorn, js, JS, JavaScript, javascript, ECMAScript, ecmascript}
2019-11-10 20:53:02,824 main DEBUG PluginManager 'Core' found 107 plugins
2019-11-10 20:53:02,825 main DEBUG PluginManager 'Level' found 0 plugins
2019-11-10 20:53:02,829 main DEBUG Log4j2 ConfigurationScheduler: No scheduled items
2019-11-10 20:53:02,829 main DEBUG PluginManager 'Lookup' found 13 plugins
2019-11-10 20:53:02,831 main DEBUG Building Plugin[name=layout, class=org.apache.logging.log4j.core.layout.PatternLayout].
2019-11-10 20:53:02,844 main DEBUG PluginManager 'TypeConverter' found 23 plugins
2019-11-10 20:53:02,857 main DEBUG PatternLayout$Builder(pattern="%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n", PatternSelector=null, Configuration(C:\Users\heiko\workspace\HAP-Java\target\classes\log4j2.xml), Replace=null, charset="null", alwaysWriteExceptions="null", noConsoleNoAnsi="null", header="null", footer="null")
2019-11-10 20:53:02,858 main DEBUG PluginManager 'Converter' found 41 plugins
2019-11-10 20:53:02,858 main DEBUG Building Plugin[name=appender, class=org.apache.logging.log4j.core.appender.ConsoleAppender].
2019-11-10 20:53:02,868 main DEBUG ConsoleAppender$Builder(target="SYSTEM_OUT", follow="null", direct="null", bufferedIo="null", bufferSize="null", immediateFlush="null", ignoreExceptions="null", PatternLayout(%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n), name="Console", Filter=null)
2019-11-10 20:53:02,869 main DEBUG Jansi is not installed, cannot find org.fusesource.jansi.WindowsAnsiOutputStream
2019-11-10 20:53:02,869 main DEBUG Starting OutputStreamManager SYSTEM_OUT.false.false
2019-11-10 20:53:02,869 main DEBUG Building Plugin[name=appenders, class=org.apache.logging.log4j.core.config.AppendersPlugin].
2019-11-10 20:53:02,871 main DEBUG createAppenders(={Console})
2019-11-10 20:53:02,872 main DEBUG Building Plugin[name=AppenderRef, class=org.apache.logging.log4j.core.config.AppenderRef].
2019-11-10 20:53:02,879 main DEBUG createAppenderRef(ref="Console", level="null", Filter=null)
2019-11-10 20:53:02,880 main DEBUG Building Plugin[name=root, class=org.apache.logging.log4j.core.config.LoggerConfig$RootLogger].
2019-11-10 20:53:02,881 main DEBUG createLogger(additivity="null", level="DEBUG", includeLocation="null", ={Console}, ={}, Configuration(C:\Users\heiko\workspace\HAP-Java\target\classes\log4j2.xml), Filter=null)
2019-11-10 20:53:02,886 main DEBUG Building Plugin[name=loggers, class=org.apache.logging.log4j.core.config.LoggersPlugin].
2019-11-10 20:53:02,887 main DEBUG createLoggers(={root})
2019-11-10 20:53:02,889 main DEBUG Configuration XmlConfiguration[location=C:\Users\heiko\workspace\HAP-Java\target\classes\log4j2.xml] initialized
2019-11-10 20:53:02,889 main DEBUG Starting configuration XmlConfiguration[location=C:\Users\heiko\workspace\HAP-Java\target\classes\log4j2.xml]
2019-11-10 20:53:02,889 main DEBUG Started configuration XmlConfiguration[location=C:\Users\heiko\workspace\HAP-Java\target\classes\log4j2.xml] OK.
2019-11-10 20:53:02,890 main DEBUG Shutting down OutputStreamManager SYSTEM_OUT.false.false-1
2019-11-10 20:53:02,890 main DEBUG Shut down OutputStreamManager SYSTEM_OUT.false.false-1, all resources released: true
2019-11-10 20:53:02,891 main DEBUG Stopped org.apache.logging.log4j.core.config.DefaultConfiguration@311d617d OK
2019-11-10 20:53:02,943 main DEBUG Registering MBean org.apache.logging.log4j2:type=73d16e93
2019-11-10 20:53:02,947 main DEBUG Registering MBean org.apache.logging.log4j2:type=73d16e93,component=StatusLogger
2019-11-10 20:53:02,949 main DEBUG Registering MBean org.apache.logging.log4j2:type=73d16e93,component=ContextSelector
2019-11-10 20:53:02,952 main DEBUG Registering MBean org.apache.logging.log4j2:type=73d16e93,component=Loggers,name=
2019-11-10 20:53:02,954 main DEBUG Registering MBean org.apache.logging.log4j2:type=73d16e93,component=Appenders,name=Console
2019-11-10 20:53:02,957 main DEBUG Reconfiguration complete for context[name=73d16e93] at URI C:\Users\heiko\workspace\HAP-Java\target\classes\log4j2.xml (org.apache.logging.log4j.core.LoggerContext@2a798d51) with optional ClassLoader: null
2019-11-10 20:53:02,957 main DEBUG Shutdown hook enabled. Registering a new one.
2019-11-10 20:53:02,959 main DEBUG LoggerContext[name=73d16e93, org.apache.logging.log4j.core.LoggerContext@2a798d51] started OK.
20:53:02.969 [main] DEBUG io.github.hapjava.HomekitRoot - foo
The PIN for pairing is 131-45-154
20:53:03.614 [main] INFO  io.github.hapjava.HomekitRoot - Added accessory Test Bridge
State has changed! Writing
20:53:03.623 [main] INFO  io.github.hapjava.HomekitRoot - Added accessory Test Lightbulb
20:53:03.643 [main] DEBUG io.netty.util.internal.logging.InternalLoggerFactory - Using SLF4J as the default logging framework
20:53:03.652 [main] DEBUG io.netty.util.internal.PlatformDependent0 - java.nio.Buffer.address: available
20:53:03.653 [main] DEBUG io.netty.util.internal.PlatformDependent0 - sun.misc.Unsafe.theUnsafe: available
20:53:03.653 [main] DEBUG io.netty.util.internal.PlatformDependent0 - sun.misc.Unsafe.copyMemory: available
20:53:03.653 [main] DEBUG io.netty.util.internal.PlatformDependent0 - java.nio.Bits.unaligned: true
20:53:03.654 [main] DEBUG io.netty.util.internal.PlatformDependent - Platform: Windows
20:53:03.655 [main] DEBUG io.netty.util.internal.PlatformDependent - Java version: 8
20:53:03.655 [main] DEBUG io.netty.util.internal.PlatformDependent - -Dio.netty.noUnsafe: false
20:53:03.655 [main] DEBUG io.netty.util.internal.PlatformDependent - sun.misc.Unsafe: available
20:53:03.655 [main] DEBUG io.netty.util.internal.PlatformDependent - -Dio.netty.noJavassist: false
20:53:03.656 [main] DEBUG io.netty.util.internal.PlatformDependent - Javassist: unavailable
20:53:03.656 [main] DEBUG io.netty.util.internal.PlatformDependent - You don't have Javassist in your class path or you don't have enough permission to load dynamically generated classes.  Please check the configuration for better performance.
20:53:03.656 [main] DEBUG io.netty.util.internal.PlatformDependent - -Dio.netty.tmpdir: C:\Users\heiko\AppData\Local\Temp (java.io.tmpdir)
20:53:03.657 [main] DEBUG io.netty.util.internal.PlatformDependent - -Dio.netty.bitMode: 64 (sun.arch.data.model)
20:53:03.657 [main] DEBUG io.netty.util.internal.PlatformDependent - -Dio.netty.noPreferDirect: false
20:53:03.668 [main] DEBUG io.netty.channel.MultithreadEventLoopGroup - -Dio.netty.eventLoopThreads: 8
20:53:03.674 [main] DEBUG io.netty.channel.nio.NioEventLoop - -Dio.netty.noKeySetOptimization: false
20:53:03.675 [main] DEBUG io.netty.channel.nio.NioEventLoop - -Dio.netty.selectorAutoRebuildThreshold: 512
20:53:04.065 [main] DEBUG io.netty.util.internal.ThreadLocalRandom - -Dio.netty.initialSeedUniquifier: 0x4747fc96431efde6 (took 1 ms)
20:53:04.097 [main] DEBUG io.netty.buffer.ByteBufUtil - -Dio.netty.allocator.type: unpooled
20:53:04.097 [main] DEBUG io.netty.buffer.ByteBufUtil - -Dio.netty.threadLocalDirectBufferSize: 65536
20:53:04.400 [main] DEBUG io.netty.util.NetUtil - Loopback interface: lo (Software Loopback Interface 1, <IP_ADDRESS>)
20:53:04.401 [main] DEBUG io.netty.util.NetUtil - \proc\sys\net\core\somaxconn: 200 (non-existent)
20:53:04.407 [nioEventLoopGroup-2-1] INFO  io.netty.handler.logging.LoggingHandler - [id: 0xe64a50a7] REGISTERED
20:53:04.408 [nioEventLoopGroup-2-1] INFO  io.netty.handler.logging.LoggingHandler - [id: 0xe64a50a7] BIND(LAPTOP-D3SSRIQB/<IP_ADDRESS>:9123)
20:53:04.410 [nioEventLoopGroup-2-1] INFO  io.github.hapjava.impl.http.impl.NettyHomekitHttpService - Bound homekit listener to /<IP_ADDRESS>:9123
20:53:04.410 [nioEventLoopGroup-2-1] INFO  io.github.hapjava.impl.jmdns.JmdnsHomekitAdvertiser - Advertising accessory Test Bridge
20:53:04.410 [nioEventLoopGroup-2-1] INFO  io.github.hapjava.impl.jmdns.JmdnsHomekitAdvertiser - Registering _hap._tcp.local. on port 9123
20:53:11.159 [nioEventLoopGroup-2-1] INFO  io.netty.handler.logging.LoggingHandler - [id: 0xe64a50a7, /<IP_ADDRESS>:9123] ACTIVE
20:53:18.156 [nioEventLoopGroup-2-1] INFO  io.netty.handler.logging.LoggingHandler - [id: 0xe64a50a7, /<IP_ADDRESS>:9123] RECEIVED: [id: 0xc4300328, /<IP_ADDRESS>:52940 => /<IP_ADDRESS>:9123]
20:53:18.174 [nioEventLoopGroup-3-1] DEBUG io.netty.util.ResourceLeakDetector - -Dio.netty.leakDetection.level: simple
20:53:18.174 [nioEventLoopGroup-3-1] DEBUG io.netty.util.ResourceLeakDetector - -Dio.netty.leakDetection.maxRecords: 4
20:53:18.198 [defaultEventExecutorGroup-4-1] INFO  io.github.hapjava.impl.http.impl.AccessoryHandler - New homekit connection from /<IP_ADDRESS>:52940
20:53:18.198 [nioEventLoopGroup-3-1] DEBUG io.netty.util.Recycler - -Dio.netty.recycler.maxCapacity.default: 262144
20:53:18.209 [defaultEventExecutorGroup-4-1] INFO  io.github.hapjava.impl.pairing.PairingManager - Starting pair for Test Bridge
20:53:18.330 [defaultEventExecutorGroup-4-1] INFO  io.github.hapjava.impl.http.HomekitClientConnection - 200 /pair-setup
20:53:18.334 [defaultEventExecutorGroup-4-1] DEBUG io.netty.util.internal.Cleaner0 - java.nio.ByteBuffer.cleaner(): available
20:53:21.922 [defaultEventExecutorGroup-4-1] INFO  io.github.hapjava.impl.http.impl.AccessoryHandler - Terminated homekit connection from /<IP_ADDRESS>:52940
---> 5 secs later i am done with enter the secure code on iPhone

thanks for details @GreenRover
i have two good news:

i could reproduce it with the sample and got exactly the same logs, behaviour and network traffic.
i found the issue/solution.

it does not like spaces in the bridge name. :(
please change in Main.java
HomekitRoot bridge = homekit.createBridge(mockAuth, "Test Bridge", "TestBridge, Inc.", "G6", "111abe234");
to
HomekitRoot bridge = homekit.createBridge(mockAuth, "TestBridge", "TestBridge, Inc.", "G6", "111abe234");
and try again.

@yfre No that  changes nothing.
Same problem as before.
Iphone XS  IOS 13.1.3
Swiss/German
Server:
Java 1.8.0 201 Oracle JDK

@yfre No that changes nothing.
Same problem as before.
Iphone XS IOS 13.1.3
Swiss/German
Server:
Java 1.8.0 201 Oracle JDK

Does not change anything for me as well. iPhone XS iOS 13.2.2

hmm. i did some tests. it does not really stable. sometimes it works, sometime it doesnt.. kind of 50/50. need to investigate further
here is my log when it works.
defaultEventExecutorGroup-4-1] INFO io.github.hapjava.impl.pairing.PairingManager - Starting pair for TestBridge
[defaultEventExecutorGroup-4-1] INFO io.github.hapjava.impl.http.HomekitClientConnection - 200 /pair-setup
[defaultEventExecutorGroup-4-1] DEBUG io.netty.util.internal.Cleaner0 - java.nio.ByteBuffer.cleaner(): available
[defaultEventExecutorGroup-4-1] DEBUG io.github.hapjava.impl.pairing.PairingManager - Entering second stage of pair for TestBridge
[defaultEventExecutorGroup-4-1] INFO io.github.hapjava.impl.http.HomekitClientConnection - 200 /pair-setup
[defaultEventExecutorGroup-4-1] DEBUG io.github.hapjava.impl.pairing.PairingManager - Entering third stage of pair for TestBridge
Added pairing for 6e:4f:3f:30:a:ceBA18A45D-721F-48C1-94AD-81F9B120D169
State has changed! Writing
[defaultEventExecutorGroup-4-1] INFO io.github.hapjava.impl.jmdns.JmdnsHomekitAdvertiser - Re-creating service due to change in discoverability to false
[defaultEventExecutorGroup-4-1] INFO io.github.hapjava.impl.jmdns.JmdnsHomekitAdvertiser - Registering _hap._tcp.local. on port 9123
[defaultEventExecutorGroup-4-1] INFO io.github.hapjava.impl.http.HomekitClientConnection - 200 /pair-setup
[defaultEventExecutorGroup-4-1] INFO io.github.hapjava.impl.http.impl.AccessoryHandler - Terminated homekit connection from /<IP_ADDRESS>:53255
[nioEventLoopGroup-2-1] INFO io.netty.handler.logging.LoggingHandler - [id: 0x0d199f2a, /<IP_ADDRESS>:9123] RECEIVED: [id: 0xbb245cae, /<IP_ADDRESS>:53257 => /<IP_ADDRESS>:9123]
[defaultEventExecutorGroup-4-2] INFO io.github.hapjava.impl.http.impl.AccessoryHandler - New homekit connection from /<IP_ADDRESS>:53257
[defaultEventExecutorGroup-4-2] DEBUG io.github.hapjava.impl.pairing.PairVerificationManager - Starting pair verification for TestBridge
[defaultEventExecutorGroup-4-2] INFO io.github.hapjava.impl.http.HomekitClientConnection - 200 /pair-verify
[defaultEventExecutorGroup-4-2] DEBUG io.github.hapjava.impl.pairing.PairVerificationManager - Completed pair verification for TestBridge
[defaultEventExecutorGroup-4-2] INFO io.github.hapjava.impl.http.HomekitClientConnection - 200 /pair-verify
[nioEventLoopGroup-3-2] DEBUG io.github.hapjava.impl.http.impl.BinaryHandler - Received data [/<IP_ADDRESS>:53257]:
GET /accessories HTTP/1.1
Host: TestBridge._hap._tcp.local

@GreenRover  @christiantschoerner
can i ask you to do one more test with this hap jar
https://1drv.ms/u/s!AqqiEfFRr6WAgYQ-iybGMnJ0B6GYRQ?e=QrzsNr
as many are affected i want to get this fixed asap.

@yfre (-; That worked (tested 1 times, later more tests)
2019-11-11 06:27:36,973 main DEBUG Initializing configuration XmlConfiguration[location=C:\Users\heiko\workspace2\HAP-Jar\bin\log4j2.xml]
2019-11-11 06:27:36,980 main DEBUG Installed script engines
2019-11-11 06:27:37,332 main DEBUG Oracle Nashorn Version: 1.8.0_201, Language: ECMAScript, Threading: Not Thread Safe, Compile: true, Names: {nashorn, Nashorn, js, JS, JavaScript, javascript, ECMAScript, ecmascript}
2019-11-11 06:27:37,335 main DEBUG PluginManager 'Core' found 107 plugins
2019-11-11 06:27:37,337 main DEBUG PluginManager 'Level' found 0 plugins
2019-11-11 06:27:37,342 main DEBUG Log4j2 ConfigurationScheduler: No scheduled items
2019-11-11 06:27:37,342 main DEBUG PluginManager 'Lookup' found 13 plugins
2019-11-11 06:27:37,344 main DEBUG Building Plugin[name=layout, class=org.apache.logging.log4j.core.layout.PatternLayout].
2019-11-11 06:27:37,357 main DEBUG PluginManager 'TypeConverter' found 23 plugins
2019-11-11 06:27:37,372 main DEBUG PatternLayout$Builder(pattern="%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n", PatternSelector=null, Configuration(C:\Users\heiko\workspace2\HAP-Jar\bin\log4j2.xml), Replace=null, charset="null", alwaysWriteExceptions="null", noConsoleNoAnsi="null", header="null", footer="null")
2019-11-11 06:27:37,373 main DEBUG PluginManager 'Converter' found 41 plugins
2019-11-11 06:27:37,374 main DEBUG Building Plugin[name=appender, class=org.apache.logging.log4j.core.appender.ConsoleAppender].
2019-11-11 06:27:37,385 main DEBUG ConsoleAppender$Builder(target="SYSTEM_OUT", follow="null", direct="null", bufferedIo="null", bufferSize="null", immediateFlush="null", ignoreExceptions="null", PatternLayout(%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n), name="Console", Filter=null)
2019-11-11 06:27:37,386 main DEBUG Jansi is not installed, cannot find org.fusesource.jansi.WindowsAnsiOutputStream
2019-11-11 06:27:37,387 main DEBUG Starting OutputStreamManager SYSTEM_OUT.false.false
2019-11-11 06:27:37,387 main DEBUG Building Plugin[name=appenders, class=org.apache.logging.log4j.core.config.AppendersPlugin].
2019-11-11 06:27:37,395 main DEBUG createAppenders(={Console})
2019-11-11 06:27:37,396 main DEBUG Building Plugin[name=AppenderRef, class=org.apache.logging.log4j.core.config.AppenderRef].
2019-11-11 06:27:37,405 main DEBUG createAppenderRef(ref="Console", level="null", Filter=null)
2019-11-11 06:27:37,405 main DEBUG Building Plugin[name=root, class=org.apache.logging.log4j.core.config.LoggerConfig$RootLogger].
2019-11-11 06:27:37,406 main DEBUG createLogger(additivity="null", level="DEBUG", includeLocation="null", ={Console}, ={}, Configuration(C:\Users\heiko\workspace2\HAP-Jar\bin\log4j2.xml), Filter=null)
2019-11-11 06:27:37,413 main DEBUG Building Plugin[name=loggers, class=org.apache.logging.log4j.core.config.LoggersPlugin].
2019-11-11 06:27:37,414 main DEBUG createLoggers(={root})
2019-11-11 06:27:37,415 main DEBUG Configuration XmlConfiguration[location=C:\Users\heiko\workspace2\HAP-Jar\bin\log4j2.xml] initialized
2019-11-11 06:27:37,415 main DEBUG Starting configuration XmlConfiguration[location=C:\Users\heiko\workspace2\HAP-Jar\bin\log4j2.xml]
2019-11-11 06:27:37,415 main DEBUG Started configuration XmlConfiguration[location=C:\Users\heiko\workspace2\HAP-Jar\bin\log4j2.xml] OK.
2019-11-11 06:27:37,416 main DEBUG Shutting down OutputStreamManager SYSTEM_OUT.false.false-1
2019-11-11 06:27:37,416 main DEBUG Shut down OutputStreamManager SYSTEM_OUT.false.false-1, all resources released: true
2019-11-11 06:27:37,417 main DEBUG Stopped org.apache.logging.log4j.core.config.DefaultConfiguration@5e8c92f4 OK
2019-11-11 06:27:37,509 main DEBUG Registering MBean org.apache.logging.log4j2:type=73d16e93
2019-11-11 06:27:37,514 main DEBUG Registering MBean org.apache.logging.log4j2:type=73d16e93,component=StatusLogger
2019-11-11 06:27:37,516 main DEBUG Registering MBean org.apache.logging.log4j2:type=73d16e93,component=ContextSelector
2019-11-11 06:27:37,518 main DEBUG Registering MBean org.apache.logging.log4j2:type=73d16e93,component=Loggers,name=
2019-11-11 06:27:37,520 main DEBUG Registering MBean org.apache.logging.log4j2:type=73d16e93,component=Appenders,name=Console
2019-11-11 06:27:37,523 main DEBUG Reconfiguration complete for context[name=73d16e93] at URI C:\Users\heiko\workspace2\HAP-Jar\bin\log4j2.xml (org.apache.logging.log4j.core.LoggerContext@359f7cdf) with optional ClassLoader: null
2019-11-11 06:27:37,525 main DEBUG Shutdown hook enabled. Registering a new one.
2019-11-11 06:27:37,527 main DEBUG LoggerContext[name=73d16e93, org.apache.logging.log4j.core.LoggerContext@359f7cdf] started OK.
06:27:37.983 [main] DEBUG io.github.hapjava.HomekitRoot - foo
The PIN for pairing is 131-45-154
06:27:38.683 [main] INFO  io.github.hapjava.HomekitRoot - Added accessory baaaa
State has changed! Writing
06:27:38.701 [main] INFO  io.github.hapjava.HomekitRoot - Added accessory Test Lightbulb
06:27:38.723 [main] DEBUG io.netty.util.internal.logging.InternalLoggerFactory - Using SLF4J as the default logging framework
06:27:38.735 [main] DEBUG io.netty.util.internal.PlatformDependent0 - java.nio.Buffer.address: available
06:27:38.735 [main] DEBUG io.netty.util.internal.PlatformDependent0 - sun.misc.Unsafe.theUnsafe: available
06:27:38.736 [main] DEBUG io.netty.util.internal.PlatformDependent0 - sun.misc.Unsafe.copyMemory: available
06:27:38.736 [main] DEBUG io.netty.util.internal.PlatformDependent0 - java.nio.Bits.unaligned: true
06:27:38.737 [main] DEBUG io.netty.util.internal.PlatformDependent - Platform: Windows
06:27:38.738 [main] DEBUG io.netty.util.internal.PlatformDependent - Java version: 8
06:27:38.739 [main] DEBUG io.netty.util.internal.PlatformDependent - -Dio.netty.noUnsafe: false
06:27:38.739 [main] DEBUG io.netty.util.internal.PlatformDependent - sun.misc.Unsafe: available
06:27:38.739 [main] DEBUG io.netty.util.internal.PlatformDependent - -Dio.netty.noJavassist: false
06:27:38.740 [main] DEBUG io.netty.util.internal.PlatformDependent - Javassist: unavailable
06:27:38.740 [main] DEBUG io.netty.util.internal.PlatformDependent - You don't have Javassist in your class path or you don't have enough permission to load dynamically generated classes.  Please check the configuration for better performance.
06:27:38.740 [main] DEBUG io.netty.util.internal.PlatformDependent - -Dio.netty.tmpdir: C:\Users\heiko\AppData\Local\Temp (java.io.tmpdir)
06:27:38.741 [main] DEBUG io.netty.util.internal.PlatformDependent - -Dio.netty.bitMode: 64 (sun.arch.data.model)
06:27:38.741 [main] DEBUG io.netty.util.internal.PlatformDependent - -Dio.netty.noPreferDirect: false
06:27:38.751 [main] DEBUG io.netty.channel.MultithreadEventLoopGroup - -Dio.netty.eventLoopThreads: 8
06:27:38.760 [main] DEBUG io.netty.channel.nio.NioEventLoop - -Dio.netty.noKeySetOptimization: false
06:27:38.760 [main] DEBUG io.netty.channel.nio.NioEventLoop - -Dio.netty.selectorAutoRebuildThreshold: 512
06:27:38.825 [main] DEBUG io.netty.util.internal.ThreadLocalRandom - -Dio.netty.initialSeedUniquifier: 0x5c1fe64bbf098685 (took 1 ms)
06:27:38.870 [main] DEBUG io.netty.buffer.ByteBufUtil - -Dio.netty.allocator.type: unpooled
06:27:38.870 [main] DEBUG io.netty.buffer.ByteBufUtil - -Dio.netty.threadLocalDirectBufferSize: 65536
06:27:39.211 [main] DEBUG io.netty.util.NetUtil - Loopback interface: lo (Software Loopback Interface 1, <IP_ADDRESS>)
06:27:39.213 [main] DEBUG io.netty.util.NetUtil - \proc\sys\net\core\somaxconn: 200 (non-existent)
06:27:39.219 [nioEventLoopGroup-2-1] DEBUG io.netty.handler.logging.LoggingHandler - [id: 0xaa9e789b] REGISTERED
06:27:39.221 [nioEventLoopGroup-2-1] DEBUG io.netty.handler.logging.LoggingHandler - [id: 0xaa9e789b] BIND(LAPTOP-D3SSRIQB/<IP_ADDRESS>:9124)
06:27:39.222 [nioEventLoopGroup-2-1] INFO  io.github.hapjava.impl.http.impl.NettyHomekitHttpService - Bound homekit listener to /<IP_ADDRESS>:9124
06:27:39.222 [nioEventLoopGroup-2-1] INFO  io.github.hapjava.impl.jmdns.JmdnsHomekitAdvertiser - Advertising accessory baaaa
06:27:39.222 [nioEventLoopGroup-2-1] INFO  io.github.hapjava.impl.jmdns.JmdnsHomekitAdvertiser - Registering _hap._tcp.local. on port 9124
06:27:46.360 [nioEventLoopGroup-2-1] DEBUG io.netty.handler.logging.LoggingHandler - [id: 0xaa9e789b, /<IP_ADDRESS>:9124] ACTIVE
06:27:50.565 [nioEventLoopGroup-2-1] DEBUG io.netty.handler.logging.LoggingHandler - [id: 0xaa9e789b, /<IP_ADDRESS>:9124] RECEIVED: [id: 0xeb79bb51, /<IP_ADDRESS>:53114 => /<IP_ADDRESS>:9124]
06:27:50.582 [nioEventLoopGroup-3-1] DEBUG io.netty.util.ResourceLeakDetector - -Dio.netty.leakDetection.level: simple
06:27:50.582 [nioEventLoopGroup-3-1] DEBUG io.netty.util.ResourceLeakDetector - -Dio.netty.leakDetection.maxRecords: 4
06:27:50.610 [nioEventLoopGroup-3-1] DEBUG io.netty.util.Recycler - -Dio.netty.recycler.maxCapacity.default: 262144
06:27:50.610 [defaultEventExecutorGroup-4-1] INFO  io.github.hapjava.impl.http.impl.AccessoryHandler - New homekit connection from /<IP_ADDRESS>:53114
06:27:50.619 [defaultEventExecutorGroup-4-1] DEBUG io.github.hapjava.impl.http.HomekitClientConnection - doHandleRequest: /pair-setupis upgraded:false
06:27:50.625 [defaultEventExecutorGroup-4-1] INFO  io.github.hapjava.impl.pairing.PairingManager - Starting pair for baaaa
06:27:50.777 [defaultEventExecutorGroup-4-1] INFO  io.github.hapjava.impl.http.HomekitClientConnection - 200 /pair-setup
06:27:50.785 [defaultEventExecutorGroup-4-1] DEBUG io.netty.util.internal.Cleaner0 - java.nio.ByteBuffer.cleaner(): available
06:27:53.591 [defaultEventExecutorGroup-4-1] DEBUG io.github.hapjava.impl.http.HomekitClientConnection - doHandleRequest: /pair-setupis upgraded:false
06:27:53.591 [defaultEventExecutorGroup-4-1] DEBUG io.github.hapjava.impl.pairing.PairingManager - Entering second stage of pair for baaaa
06:27:53.647 [defaultEventExecutorGroup-4-1] INFO  io.github.hapjava.impl.http.HomekitClientConnection - 200 /pair-setup
06:27:53.697 [defaultEventExecutorGroup-4-1] DEBUG io.github.hapjava.impl.http.HomekitClientConnection - doHandleRequest: /pair-setupis upgraded:false
06:27:53.697 [defaultEventExecutorGroup-4-1] DEBUG io.github.hapjava.impl.pairing.PairingManager - Entering third stage of pair for baaaa
06:27:53.810 [defaultEventExecutorGroup-4-1] DEBUG io.github.hapjava.impl.pairing.FinalPairHandler - handle request
06:27:53.829 [defaultEventExecutorGroup-4-1] DEBUG io.github.hapjava.impl.pairing.FinalPairHandler - decrypt plaintext:[B@e7ca986
06:27:53.829 [defaultEventExecutorGroup-4-1] DEBUG io.github.hapjava.impl.pairing.FinalPairHandler - decrypt username:[B@3b499c5
06:27:53.829 [defaultEventExecutorGroup-4-1] DEBUG io.github.hapjava.impl.pairing.FinalPairHandler - decrypt error:null
Added pairing for d2:bb:de:6d:4b:2eE012AC74-5B04-42DC-81EF-9A82F3FF4F64
State has changed! Writing
06:27:53.837 [defaultEventExecutorGroup-4-1] DEBUG io.github.hapjava.impl.pairing.FinalPairHandler - create response
06:27:53.841 [defaultEventExecutorGroup-4-1] DEBUG io.github.hapjava.impl.pairing.FinalPairHandler - create response done
06:27:53.842 [defaultEventExecutorGroup-4-1] INFO  io.github.hapjava.impl.http.HomekitClientConnection - 200 /pair-setup
06:27:53.911 [defaultEventExecutorGroup-4-1] INFO  io.github.hapjava.impl.http.impl.AccessoryHandler - Terminated homekit connection from /<IP_ADDRESS>:53114
06:27:53.918 [nioEventLoopGroup-2-1] DEBUG io.netty.handler.logging.LoggingHandler - [id: 0xaa9e789b, /<IP_ADDRESS>:9124] RECEIVED: [id: 0xc1e57f64, /<IP_ADDRESS>:53115 => /<IP_ADDRESS>:9124]
06:27:53.922 [defaultEventExecutorGroup-4-2] INFO  io.github.hapjava.impl.http.impl.AccessoryHandler - New homekit connection from /<IP_ADDRESS>:53115
06:27:53.923 [defaultEventExecutorGroup-4-2] DEBUG io.github.hapjava.impl.http.HomekitClientConnection - doHandleRequest: /pair-verifyis upgraded:false
06:27:53.928 [defaultEventExecutorGroup-4-2] DEBUG io.github.hapjava.impl.pairing.PairVerificationManager - Starting pair verification for baaaa
06:27:53.941 [defaultEventExecutorGroup-4-2] INFO  io.github.hapjava.impl.http.HomekitClientConnection - 200 /pair-verify
06:27:54.036 [defaultEventExecutorGroup-4-2] DEBUG io.github.hapjava.impl.http.HomekitClientConnection - doHandleRequest: /pair-verifyis upgraded:false
06:27:54.038 [defaultEventExecutorGroup-4-2] DEBUG io.github.hapjava.impl.pairing.PairVerificationManager - Completed pair verification for baaaa
06:27:54.039 [defaultEventExecutorGroup-4-2] INFO  io.github.hapjava.impl.http.HomekitClientConnection - 200 /pair-verify
06:27:54.068 [nioEventLoopGroup-3-2] DEBUG io.github.hapjava.impl.http.impl.BinaryHandler - Received data [/<IP_ADDRESS>:53115]:
GET /accessories HTTP/1.1
Host: baaaa._hap._tcp.local

06:27:54.069 [defaultEventExecutorGroup-4-2] DEBUG io.github.hapjava.impl.http.HomekitClientConnection - doHandleRequest: /accessoriesis upgraded:true
06:27:54.093 [defaultEventExecutorGroup-4-2] INFO  io.github.hapjava.impl.http.HomekitClientConnection - 200 /accessories
06:27:54.094 [nioEventLoopGroup-3-2] DEBUG io.github.hapjava.impl.http.impl.BinaryHandler - Sending data [/<IP_ADDRESS>:53115]:
HTTP/1.1 200 OK
Content-type: application/hap+json
Content-Length: 1850
Connection: keep-alive

{"accessories":[{"aid":1,"services":[{"iid":1,"type":"3E","characteristics":[{"iid":2,"type":"23","perms":["pr"],"format":"string","ev":false,"description":"Name of the accessory","value":"baaaa","maxLen":255},{"iid":3,"type":"20","perms":["pr"],"format":"string","ev":false,"description":"The name of the manufacturer","value":"foooo","maxLen":255},{"iid":4,"type":"21","perms":["pr"],"format":"string","ev":false,"description":"The name of the model","value":"G6","maxLen":255},{"iid":5,"type":"30","perms":["pr"],"format":"string","ev":false,"description":"The serial number of the accessory","value":"123456789","maxLen":255},{"iid":6,"type":"14","perms":["pw"],"format":"bool","ev":false,"description":"Identifies the accessory via a physical action on the accessory"}]}]},{"aid":2,"services":[{"iid":1,"type":"3E","characteristics":[{"iid":2,"type":"23","perms":["pr"],"format":"string","ev":false,"description":"Name of the accessory","value":"Test Lightbulb","maxLen":255},{"iid":3,"type":"20","perms":["pr"],"format":"string","ev":false,"description":"The name of the manufacturer","value":"none","maxLen":255},{"iid":4,"type":"21","perms":["pr"],"format":"string","ev":false,"description":"The name of the model","value":"none","maxLen":255},{"iid":5,"type":"30","perms":["pr"],"format":"string","ev":false,"description":"The serial number of the accessory","value":"none","maxLen":255},{"iid":6,"type":"14","perms":["pw"],"format":"bool","ev":false,"description":"Identifies the accessory via a physical action on the accessory"}]},{"iid":7,"type":"43","characteristics":[{"iid":8,"type":"23","perms":["pr"],"format":"string","ev":false,"description":"Name of the accessory","value":"Test Lightbulb","maxLen":255},{"iid":9,"type":"25","perms":["pw","pr","ev"],"format":"bool","ev":false,"description":"Turn on and off","value":false}]}]}]}
06:27:54.205 [nioEventLoopGroup-3-2] DEBUG io.github.hapjava.impl.http.impl.BinaryHandler - Received data [/<IP_ADDRESS>:53115]:
PUT /characteristics HTTP/1.1
Host: baaaa._hap._tcp.local
Content-Length: 49
Content-Type: application/hap+json

{"characteristics":[{"aid":2,"iid":9,"ev":true}]}
06:27:54.205 [defaultEventExecutorGroup-4-2] DEBUG io.github.hapjava.impl.http.HomekitClientConnection - doHandleRequest: /characteristicsis upgraded:true
06:27:54.223 [defaultEventExecutorGroup-4-2] DEBUG io.github.hapjava.impl.connections.SubscriptionManager - Added subscription to class io.github.hapjava.impl.characteristics.common.PowerStateCharacteristic for 872651142
06:27:54.224 [defaultEventExecutorGroup-4-2] INFO  io.github.hapjava.impl.http.HomekitClientConnection - 204 /characteristics
06:27:54.225 [nioEventLoopGroup-3-2] DEBUG io.github.hapjava.impl.http.impl.BinaryHandler - Sending data [/<IP_ADDRESS>:53115]:
HTTP/1.1 204 No Content
Content-type: application/hap+json
Content-Length: 0
Connection: keep-alive

06:27:54.392 [nioEventLoopGroup-3-2] DEBUG io.github.hapjava.impl.http.impl.BinaryHandler - Received data [/<IP_ADDRESS>:53115]:
PUT /characteristics HTTP/1.1
Host: baaaa._hap._tcp.local
Content-Length: 49
Content-Type: application/hap+json

{"characteristics":[{"aid":2,"iid":9,"ev":true}]}
06:27:54.392 [defaultEventExecutorGroup-4-2] DEBUG io.github.hapjava.impl.http.HomekitClientConnection - doHandleRequest: /characteristicsis upgraded:true
06:27:54.393 [defaultEventExecutorGroup-4-2] DEBUG io.github.hapjava.impl.connections.SubscriptionManager - Added subscription to class io.github.hapjava.impl.characteristics.common.PowerStateCharacteristic for 872651142
06:27:54.393 [defaultEventExecutorGroup-4-2] INFO  io.github.hapjava.impl.http.HomekitClientConnection - 204 /characteristics
06:27:54.394 [nioEventLoopGroup-3-2] DEBUG io.github.hapjava.impl.http.impl.BinaryHandler - Sending data [/<IP_ADDRESS>:53115]:
HTTP/1.1 204 No Content
Content-type: application/hap+json
Content-Length: 0
Connection: keep-alive

06:27:54.465 [nioEventLoopGroup-3-2] DEBUG io.github.hapjava.impl.http.impl.BinaryHandler - Received data [/<IP_ADDRESS>:53115]:
PUT /characteristics HTTP/1.1
Host: baaaa._hap._tcp.local
Content-Length: 49
Content-Type: application/hap+json

{"characteristics":[{"aid":2,"iid":9,"ev":true}]}
06:27:54.465 [defaultEventExecutorGroup-4-2] DEBUG io.github.hapjava.impl.http.HomekitClientConnection - doHandleRequest: /characteristicsis upgraded:true
06:27:54.466 [defaultEventExecutorGroup-4-2] DEBUG io.github.hapjava.impl.connections.SubscriptionManager - Added subscription to class io.github.hapjava.impl.characteristics.common.PowerStateCharacteristic for 872651142
06:27:54.466 [defaultEventExecutorGroup-4-2] INFO  io.github.hapjava.impl.http.HomekitClientConnection - 204 /characteristics
06:27:54.466 [nioEventLoopGroup-3-2] DEBUG io.github.hapjava.impl.http.impl.BinaryHandler - Sending data [/<IP_ADDRESS>:53115]:
HTTP/1.1 204 No Content
Content-type: application/hap+json
Content-Length: 0
Connection: keep-alive

06:27:54.746 [nioEventLoopGroup-3-2] DEBUG io.github.hapjava.impl.http.impl.BinaryHandler - Received data [/<IP_ADDRESS>:53115]:
GET /characteristics?id=2.9 HTTP/1.1
Host: baaaa._hap._tcp.local

06:27:54.746 [defaultEventExecutorGroup-4-2] DEBUG io.github.hapjava.impl.http.HomekitClientConnection - doHandleRequest: /characteristics?id=2.9is upgraded:true
06:27:54.748 [defaultEventExecutorGroup-4-2] INFO  io.github.hapjava.impl.http.HomekitClientConnection - 200 /characteristics?id=2.9
06:27:54.749 [nioEventLoopGroup-3-2] DEBUG io.github.hapjava.impl.http.impl.BinaryHandler - Sending data [/<IP_ADDRESS>:53115]:
HTTP/1.1 200 OK
Content-type: application/hap+json
Content-Length: 53
Connection: keep-alive

{"characteristics":[{"value":false,"aid":2,"iid":9}]}
06:28:37.382 [nioEventLoopGroup-3-2] DEBUG io.github.hapjava.impl.http.impl.BinaryHandler - Received data [/<IP_ADDRESS>:53115]:

06:28:37.382 [defaultEventExecutorGroup-4-2] INFO  io.github.hapjava.impl.http.impl.AccessoryHandler - Terminated homekit connection from /<IP_ADDRESS>:53115

Looks good for me too, had a chance to try it a few times.  Nice work!

ok. background:
bridge (java-hap) must mark itself as bound/non discoverable once the pairing is successfully completed. this should ensure the bridge is linked to only one controller / icloud account.
we did it a little too early - just before that last step of pairing. ios12 was fine with it, ios13 is dropping pairing immediately when controller gets marked as bound. depending on exact order of messages: bound info or pairing completion, it was sometimes working sometimes not.
the quick fix was: do not mark as bound at all, keep it discoverable.
now i will try to implement the proper solution - mark as bound after the last step of pairing.

Way back in iOS9, there was an issue caused by leaving the advertisement discoverable. Given the success Apple has with driving upgrades, it's probably not worth maintaining backwards compatibility with older iOS versions, but thought I'd point it out: https://github.com/hap-java/HAP-Java/commit/de6822d207c7f508a249806a2e0c438b69175381#diff-066f526e6b621ce14b59cda4b6dc2a81

Way back in iOS9, there was an issue caused by leaving the advertisement discoverable. Given the success Apple has with driving upgrades, it's probably not worth maintaining backwards compatibility with older iOS versions, but thought I'd point it out:

I agree. If this solution works on 10-13 we should be good (10 is the last to work in iPad 4 and iPhone 5s).

So consensus is to merge #94 as-is?

So guys what is the final result. Do we have a fix or not :D

As i understand it it only was a temporary fix. Or is there still a final one?

@yfre can you please update the status?

added one more change to PR to ensure discoverability is disabled once pairing is completed. works fine with ios13 but could not test with older version.
it is final from my perspective.

added one more change to PR to ensure discoverability is disabled once pairing is completed. works fine with ios13 but could not test with older version.
it is final from my perspective.

Okay Am I dump or where can I download this? The last "release" was in March and the last push to production was 2 months ago. And do I need to change anything special in my code?

Hello,
I am trying with the new iOS but its also not working with 13.4. I am the only one struggling with this problem?

Hello,
I am trying with the new iOS but its also not working with 13.4. I am the only one struggling with this problem? #95 fixed the discoverability but the pairing issue is still active

what kind of issue you have?  how the log look like?can you try with reduced number of device - e.g. try with a simple switch / light first and once it works, add other devices

Hello,
I am trying with the new iOS but its also not working with 13.4. I am the only one struggling with this problem? #95 fixed the discoverability but the pairing issue is still active

what kind of issue you have? how the log look like?can you try with reduced number of device - e.g. try with a simple switch / light first and once it works, add other devices

I am trying with the Sample project, with the simple MockSwitch.
Using persisted auth The PIN for pairing is 131-45-154 12:17:17.996 [main] INFO  io.github.hapjava.HomekitRoot - Added accessory TestBridge State has changed! Writing 12:17:18.077 [main] INFO  io.github.hapjava.HomekitRoot - Added accessory TestLightbulb 12:17:18.107 [main] DEBUG i.n.u.i.l.InternalLoggerFactory - Using SLF4J as the default logging framework 12:17:18.122 [main] DEBUG i.n.util.internal.PlatformDependent0 - java.nio.Buffer.address: available 12:17:18.123 [main] DEBUG i.n.util.internal.PlatformDependent0 - sun.misc.Unsafe.theUnsafe: available 12:17:18.123 [main] DEBUG i.n.util.internal.PlatformDependent0 - sun.misc.Unsafe.copyMemory: available 12:17:18.124 [main] DEBUG i.n.util.internal.PlatformDependent0 - java.nio.Bits.unaligned: true 12:17:18.126 [main] DEBUG i.n.util.internal.PlatformDependent - Java version: 8 12:17:18.126 [main] DEBUG i.n.util.internal.PlatformDependent - -Dio.netty.noUnsafe: false 12:17:18.127 [main] DEBUG i.n.util.internal.PlatformDependent - sun.misc.Unsafe: available 12:17:18.127 [main] DEBUG i.n.util.internal.PlatformDependent - -Dio.netty.noJavassist: false 12:17:18.128 [main] DEBUG i.n.util.internal.PlatformDependent - Javassist: unavailable 12:17:18.128 [main] DEBUG i.n.util.internal.PlatformDependent - You don't have Javassist in your class path or you don't have enough permission to load dynamically generated classes.  Please check the configuration for better performance. 12:17:18.129 [main] DEBUG i.n.util.internal.PlatformDependent - -Dio.netty.tmpdir: /tmp (java.io.tmpdir) 12:17:18.129 [main] DEBUG i.n.util.internal.PlatformDependent - -Dio.netty.bitMode: 64 (sun.arch.data.model) 12:17:18.129 [main] DEBUG i.n.util.internal.PlatformDependent - -Dio.netty.noPreferDirect: false 12:17:18.169 [main] DEBUG i.n.c.MultithreadEventLoopGroup - -Dio.netty.eventLoopThreads: 8 12:17:18.180 [main] DEBUG io.netty.channel.nio.NioEventLoop - -Dio.netty.noKeySetOptimization: false 12:17:18.181 [main] DEBUG io.netty.channel.nio.NioEventLoop - -Dio.netty.selectorAutoRebuildThreshold: 512 12:17:18.190 [main] TRACE io.netty.channel.nio.NioEventLoop - Instrumented an optimized java.util.Set into: sun.nio.ch.EPollSelectorImpl@5700d6b1 12:17:18.190 [main] TRACE io.netty.channel.nio.NioEventLoop - Instrumented an optimized java.util.Set into: sun.nio.ch.EPollSelectorImpl@6fd02e5 12:17:18.190 [main] TRACE io.netty.channel.nio.NioEventLoop - Instrumented an optimized java.util.Set into: sun.nio.ch.EPollSelectorImpl@5bcab519 12:17:18.190 [main] TRACE io.netty.channel.nio.NioEventLoop - Instrumented an optimized java.util.Set into: sun.nio.ch.EPollSelectorImpl@e45f292 12:17:18.190 [main] TRACE io.netty.channel.nio.NioEventLoop - Instrumented an optimized java.util.Set into: sun.nio.ch.EPollSelectorImpl@5f2108b5 12:17:18.191 [main] TRACE io.netty.channel.nio.NioEventLoop - Instrumented an optimized java.util.Set into: sun.nio.ch.EPollSelectorImpl@31a5c39e 12:17:18.192 [main] TRACE io.netty.channel.nio.NioEventLoop - Instrumented an optimized java.util.Set into: sun.nio.ch.EPollSelectorImpl@3f49dace 12:17:18.192 [main] TRACE io.netty.channel.nio.NioEventLoop - Instrumented an optimized java.util.Set into: sun.nio.ch.EPollSelectorImpl@1e397ed7 12:17:18.192 [main] TRACE io.netty.channel.nio.NioEventLoop - Instrumented an optimized java.util.Set into: sun.nio.ch.EPollSelectorImpl@27c20538 12:17:18.193 [main] TRACE io.netty.channel.nio.NioEventLoop - Instrumented an optimized java.util.Set into: sun.nio.ch.EPollSelectorImpl@72d818d1 12:17:18.193 [main] TRACE io.netty.channel.nio.NioEventLoop - Instrumented an optimized java.util.Set into: sun.nio.ch.EPollSelectorImpl@6e06451e 12:17:18.193 [main] TRACE io.netty.channel.nio.NioEventLoop - Instrumented an optimized java.util.Set into: sun.nio.ch.EPollSelectorImpl@59494225 12:17:18.193 [main] TRACE io.netty.channel.nio.NioEventLoop - Instrumented an optimized java.util.Set into: sun.nio.ch.EPollSelectorImpl@6e1567f1 12:17:18.194 [main] TRACE io.netty.channel.nio.NioEventLoop - Instrumented an optimized java.util.Set into: sun.nio.ch.EPollSelectorImpl@5cb9f472 12:17:18.195 [main] TRACE io.netty.channel.nio.NioEventLoop - Instrumented an optimized java.util.Set into: sun.nio.ch.EPollSelectorImpl@cb644e 12:17:18.196 [main] TRACE io.netty.channel.nio.NioEventLoop - Instrumented an optimized java.util.Set into: sun.nio.ch.EPollSelectorImpl@13805618 12:17:18.227 [main] DEBUG i.n.util.internal.ThreadLocalRandom - -Dio.netty.initialSeedUniquifier: 0xceaa20b0108ae084 (took 2 ms) 12:17:18.288 [main] DEBUG io.netty.buffer.ByteBufUtil - -Dio.netty.allocator.type: unpooled 12:17:18.289 [main] DEBUG io.netty.buffer.ByteBufUtil - -Dio.netty.threadLocalDirectBufferSize: 65536 12:17:18.292 [main] DEBUG io.netty.util.NetUtil - Loopback interface: lo (lo, 0:0:0:0:0:0:0:1%lo) 12:17:18.293 [main] DEBUG io.netty.util.NetUtil - /proc/sys/net/core/somaxconn: 128 12:17:18.321 [nioEventLoopGroup-2-1] INFO  i.n.handler.logging.LoggingHandler - [id: 0xed094031] REGISTERED 12:17:18.322 [nioEventLoopGroup-2-1] INFO  i.n.handler.logging.LoggingHandler - [id: 0xed094031] BIND(<IP_ADDRESS>/<IP_ADDRESS>:51826) 12:17:18.323 [nioEventLoopGroup-2-1] INFO  i.g.h.i.h.i.NettyHomekitHttpService - Bound homekit listener to /<IP_ADDRESS>:51826 12:17:18.323 [nioEventLoopGroup-2-1] INFO  i.g.h.i.jmdns.JmdnsHomekitAdvertiser - Advertising accessory TestBridge 12:17:18.324 [nioEventLoopGroup-2-1] INFO  i.g.h.i.jmdns.JmdnsHomekitAdvertiser - Registering _hap._tcp.local. on port 51826

Hello,
I am trying with the new iOS but its also not working with 13.4. I am the only one struggling with this problem? #95 fixed the discoverability but the pairing issue is still active

what kind of issue you have? how the log look like?can you try with reduced number of device - e.g. try with a simple switch / light first and once it works, add other devices

I am trying with the Sample project, with the simple MockSwitch.
Using persisted auth
The PIN for pairing is 131-45-154
12:17:17.996 [main] INFO  io.github.hapjava.HomekitRoot - Added accessory TestBridge
State has changed! Writing
12:17:18.077 [main] INFO  io.github.hapjava.HomekitRoot - Added accessory TestLightbulb
12:17:18.107 [main] DEBUG i.n.u.i.l.InternalLoggerFactory - Using SLF4J as the default logging framework
12:17:18.122 [main] DEBUG i.n.util.internal.PlatformDependent0 - java.nio.Buffer.address: available
12:17:18.123 [main] DEBUG i.n.util.internal.PlatformDependent0 - sun.misc.Unsafe.theUnsafe: available
12:17:18.123 [main] DEBUG i.n.util.internal.PlatformDependent0 - sun.misc.Unsafe.copyMemory: available
12:17:18.124 [main] DEBUG i.n.util.internal.PlatformDependent0 - java.nio.Bits.unaligned: true
12:17:18.126 [main] DEBUG i.n.util.internal.PlatformDependent - Java version: 8
12:17:18.126 [main] DEBUG i.n.util.internal.PlatformDependent - -Dio.netty.noUnsafe: false
12:17:18.127 [main] DEBUG i.n.util.internal.PlatformDependent - sun.misc.Unsafe: available
12:17:18.127 [main] DEBUG i.n.util.internal.PlatformDependent - -Dio.netty.noJavassist: false
12:17:18.128 [main] DEBUG i.n.util.internal.PlatformDependent - Javassist: unavailable
12:17:18.128 [main] DEBUG i.n.util.internal.PlatformDependent - You don't have Javassist in your class path or you don't have enough permission to load dynamically generated classes.  Please check the configuration for better performance.
12:17:18.129 [main] DEBUG i.n.util.internal.PlatformDependent - -Dio.netty.tmpdir: /tmp (java.io.tmpdir)
12:17:18.129 [main] DEBUG i.n.util.internal.PlatformDependent - -Dio.netty.bitMode: 64 (sun.arch.data.model)
12:17:18.129 [main] DEBUG i.n.util.internal.PlatformDependent - -Dio.netty.noPreferDirect: false
12:17:18.169 [main] DEBUG i.n.c.MultithreadEventLoopGroup - -Dio.netty.eventLoopThreads: 8
12:17:18.180 [main] DEBUG io.netty.channel.nio.NioEventLoop - -Dio.netty.noKeySetOptimization: false
12:17:18.181 [main] DEBUG io.netty.channel.nio.NioEventLoop - -Dio.netty.selectorAutoRebuildThreshold: 512
12:17:18.190 [main] TRACE io.netty.channel.nio.NioEventLoop - Instrumented an optimized java.util.Set into: sun.nio.ch.EPollSelectorImpl@5700d6b1
12:17:18.190 [main] TRACE io.netty.channel.nio.NioEventLoop - Instrumented an optimized java.util.Set into: sun.nio.ch.EPollSelectorImpl@6fd02e5
12:17:18.190 [main] TRACE io.netty.channel.nio.NioEventLoop - Instrumented an optimized java.util.Set into: sun.nio.ch.EPollSelectorImpl@5bcab519
12:17:18.190 [main] TRACE io.netty.channel.nio.NioEventLoop - Instrumented an optimized java.util.Set into: sun.nio.ch.EPollSelectorImpl@e45f292
12:17:18.190 [main] TRACE io.netty.channel.nio.NioEventLoop - Instrumented an optimized java.util.Set into: sun.nio.ch.EPollSelectorImpl@5f2108b5
12:17:18.191 [main] TRACE io.netty.channel.nio.NioEventLoop - Instrumented an optimized java.util.Set into: sun.nio.ch.EPollSelectorImpl@31a5c39e
12:17:18.192 [main] TRACE io.netty.channel.nio.NioEventLoop - Instrumented an optimized java.util.Set into: sun.nio.ch.EPollSelectorImpl@3f49dace
12:17:18.192 [main] TRACE io.netty.channel.nio.NioEventLoop - Instrumented an optimized java.util.Set into: sun.nio.ch.EPollSelectorImpl@1e397ed7
12:17:18.192 [main] TRACE io.netty.channel.nio.NioEventLoop - Instrumented an optimized java.util.Set into: sun.nio.ch.EPollSelectorImpl@27c20538
12:17:18.193 [main] TRACE io.netty.channel.nio.NioEventLoop - Instrumented an optimized java.util.Set into: sun.nio.ch.EPollSelectorImpl@72d818d1
12:17:18.193 [main] TRACE io.netty.channel.nio.NioEventLoop - Instrumented an optimized java.util.Set into: sun.nio.ch.EPollSelectorImpl@6e06451e
12:17:18.193 [main] TRACE io.netty.channel.nio.NioEventLoop - Instrumented an optimized java.util.Set into: sun.nio.ch.EPollSelectorImpl@59494225
12:17:18.193 [main] TRACE io.netty.channel.nio.NioEventLoop - Instrumented an optimized java.util.Set into: sun.nio.ch.EPollSelectorImpl@6e1567f1
12:17:18.194 [main] TRACE io.netty.channel.nio.NioEventLoop - Instrumented an optimized java.util.Set into: sun.nio.ch.EPollSelectorImpl@5cb9f472
12:17:18.195 [main] TRACE io.netty.channel.nio.NioEventLoop - Instrumented an optimized java.util.Set into: sun.nio.ch.EPollSelectorImpl@cb644e
12:17:18.196 [main] TRACE io.netty.channel.nio.NioEventLoop - Instrumented an optimized java.util.Set into: sun.nio.ch.EPollSelectorImpl@13805618
12:17:18.227 [main] DEBUG i.n.util.internal.ThreadLocalRandom - -Dio.netty.initialSeedUniquifier: 0xceaa20b0108ae084 (took 2 ms)
12:17:18.288 [main] DEBUG io.netty.buffer.ByteBufUtil - -Dio.netty.allocator.type: unpooled
12:17:18.289 [main] DEBUG io.netty.buffer.ByteBufUtil - -Dio.netty.threadLocalDirectBufferSize: 65536
12:17:18.292 [main] DEBUG io.netty.util.NetUtil - Loopback interface: lo (lo, 0:0:0:0:0:0:0:1%lo)
12:17:18.293 [main] DEBUG io.netty.util.NetUtil - /proc/sys/net/core/somaxconn: 128
12:17:18.321 [nioEventLoopGroup-2-1] INFO  i.n.handler.logging.LoggingHandler - [id: 0xed094031] REGISTERED
12:17:18.322 [nioEventLoopGroup-2-1] INFO  i.n.handler.logging.LoggingHandler - [id: 0xed094031] BIND(<IP_ADDRESS>/<IP_ADDRESS>:51826)
12:17:18.323 [nioEventLoopGroup-2-1] INFO  i.g.h.i.h.i.NettyHomekitHttpService - Bound homekit listener to /<IP_ADDRESS>:51826
12:17:18.323 [nioEventLoopGroup-2-1] INFO  i.g.h.i.jmdns.JmdnsHomekitAdvertiser - Advertising accessory TestBridge
12:17:18.324 [nioEventLoopGroup-2-1] INFO  i.g.h.i.jmdns.JmdnsHomekitAdvertiser - Registering _hap._tcp.local. on port 51826

what happening afterwards? can you see the bridge in home app ? does it ask for PIN?
can you run with "-Dorg.slf4j.simpleLogger.defaultLogLevel=DEBUG"

Yes, i can see the bridge in home app, and it is ask for the pin. After that nothing happen (no log record), just wheeling and timeouting the accessory (birdge) adding.

just tried with ios13.4 and it works but maybe i have a newer version of java-hap
which version do you use? the one from master branch or the binary from release 1.1.5?

Just turned off the firewall and it is working like a charm. My fault, so sorry :/
Anyway thank you for your time!

Just turned off the firewall and it is working like a charm. My fault, so sorry :/
Anyway thank you for your time!

cool! added firewall to my checklist :)

Doesn't work for me. Pairing completes, the service says it's re-registering, and then removeUser is called and the device says the process failed. iOS 13.3.1. I already tried with and without re-registering the service, but to no avail.
What I find strange is that the request for accessories is received BEFORE the service is re-registering itself. Could this be the problem? Is there maybe a better way like updating the service instead of re-registering it (by using ServiceInfo.setText)? Or re-register the service immediately after sending the final pairing response, so that the service is ready when the next request comes?

Update: if I remove the MockSwitch and just create the bridge, I can add the bridge successfully. So it must have something to do with the device. I already checked the JSON and it's identical to the one posted here for a successful attempt. No null or anything strange. Any idea?

in the older iOS i had to remove "home" and create again time-to-time to get rid of some strange issues. but with ios 13.3.x i could repeat pairing many times without getting any issues.
can you now add the device? without removing the bridge and home. just add device and restart the sample.

No, I can't, because I have to search again and then the bridge isn't found, because it's paired :-(
I also cleared Home completely, but that changes nothing. Any info I can provide?

i did recently the refactoring of HAP in order to add support for optional attributes. the changes are not merged yet, but it would be interesting to know whether you get the same issue with a newer version.
could you maybe try run the sample that is based on the refactored version?
you can download it here
https://1drv.ms/u/s!AqqiEfFRr6WAgYgRlv15LrGLAPkq8g?e=UsqzQ2
and  run with " java -jar target/hap-sample-0.0.2-SNAPSHOT.jar"
or build it yourself form https://github.com/hap-java/HAP-Java/pull/102

Hmm, the jar is only 10 kb and doesn't seem to work? And building it myself would require updating the library, too, because the sample uses parameters for createBridge that I haven't seen before ;-)

you are right. the libs are missing.
so, i have ziped the target folder with zip and jar.
if you have time, could you please try it one more time
https://1drv.ms/u/s!AqqiEfFRr6WAgYgSE-Tb84IKu4ev9A?e=vwozRf

Hmm, ok, that one works, so the error is probably somewhere on my side. Ok, here's what I'm trying to do: I'm triyng to get HAP-Java working with IKVM in C#/.NET. Which seems to work for the most part, but I have no idea why it fails on this "small" detail.
If you have time, I have put the logs of both your sample and mine (using the same, cross-compiled 2.0 jar). Apart from the logging difference, I can't make out much difference, only that your's works and mine doesn't :-(
https://1drv.ms/u/s!AtK4xQ8gVYfFmF4oiRQcptvbOPZ1?e=1q1JgT

After numerous attempts and switching of library versions, I finally got it working. In the end, the problem was one single character in the Json file ;-)  I returned a .NET bool for the power state, which converted to "False" instead of "false". doh ;-)

Hi, I hope I didn't miss it in the discussion: in which version is this expected to be fixed?
As far as I can tell right now, I tried both 1.1.3 and then 1.1.5 and in both of them the pairing does not work. It happened to me once from like 20-30 attempts that it worked, but it was literally just once.
Is there a plan to port the fix to the original 1.1.x "beowulfe" series? Or do I have to eventually migrate to the 1.2.x series? I am using HAP Java 1.1.x as a "driver" for HAP Scala and my derived personal projects.

Probably never mind. So far it seems that the migration to 1.2 is extremely smooth. I will hopefully not need the older versions. (But maybe somebody else will?)

closing as obsolete


Run a sanity check on the env var and credentials file

Run a sanity check on the env var and credentials file when low profile describes something or activate to prevent orphaned settings.

warn if AWS_DEFAULT_PROFILE is set.


Cannot open ipynb file in vs code

Bug: Notebook Editor, Interactive Window, Editor cells

Steps to cause the bug to occur

Actual behavior
cannot open ipynb file in vscode, always loading
ps: this happens when I update the latest version of vscode, is this a bug???
Expected behavior
expect to open ipynb file and edit normally

Your Jupyter and/or Python environment
Please provide as much info as you readily know

Jupyter server running: Local | Remote | N/A
Extension version: 24,688,503
VS Code version: 1.49.0
Setting python.jediEnabled: true | false
Setting python.languageServer: Jedi | Microsoft | None
Python and/or Anaconda version: 3.8.3
OS: Windows | Mac | Linux (distro):
Virtual environment: conda | venv | virtualenv | N/A | ...

Python Output

Microsoft Data Science for VS Code Engineering Team: @rchiodo, @IanMatthewHuff, @DavidKutu, @DonJayamanne, @greazer, @joyceerhl

Thanks for the bug report. We are tracking this issue in #13901 and it is likely related to microsoft/vscode#106547 upstream. Please subscribe to those threads for updates on this bug. Sorry for the trouble!


chore(CI): simplify testing strategy

Just like I did in https://github.com/eslint-community/mysticatea__eslint-plugin/pull/1 & https://github.com/eslint-community/eslint-plugin-eslint-plugin/pull/313

Codecov Report

:exclamation: No coverage uploaded for pull request base (main@e8eabf4). Click here to learn what that means.
Patch has no changes to coverable lines.

Additional details and impacted files
@@           Coverage Diff            @@
##             main      #378   +/-   ##
========================================
  Coverage        ?   100.00%
========================================
  Files           ?        25
  Lines           ?       626
  Branches        ?       242
========================================
  Hits            ?       626
  Misses          ?         0
  Partials        ?         0

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


[FEATURE REQUEST]Read as little as possible from image during 'open' call

When scanning a drive for images, I am interested in gathering all images that are larger than a specific size.  During the open call, more info than just the header is read in, slowing the call significantly.  Calling open on thousands or millions of images is very time-consuming.
For example, in
auto in = ImageInput::open (filename);
if (! in)
    return;
const ImageSpec &spec = in->spec();
int xres = spec.width;
int yres = spec.height;

the TGA processing reads in the thumbnail data.
I would like to have all pixel processing, even the thumbnails, removed from the open call and just have the minimal spec information read so that this call is performant.

This is an interesting topic, and it's not obvious what the right approach is, so I am definitely interested in feedback from you and anybody else reading.
So, just as background, a common theme in OIIO's APIs is that the whole image can be separated into the pixel data itself, and everything else is stored in ImageSpec. The idea is that the result of a full read cycle is an ImageSpec + a pixel buffer, and so when you write an image, you just pass an ImageSpec and a pixel buffer. The specific thing we tried to avoid was having to read item after item through separate API calls, have the user code carefully tend to those separate pieces of data, and pass them though a succession of special API calls when writing or copying one image to another.
So for that reason, I historically had hesitated to make a separate call to read the thumbnail, or to write it. And it's definitely a priority for me that the simple "read an image from one file, write it out to a different file" be really, really simple, a small number of simple API calls.
But I also see that there are times when you don't want all that expense. And thumbnails aren't the only case like this: ICC profiles can be quite big. PTex files have a big set of information giving the connectivity of the mesh. OpenEXR just merged a patch that adds (enormous!) metadata that associates a manifest of object names with integer "object ID" images. So I think this issue will keep popping up in various forms and that there is merit to addressing it intelligently.
So I've been mulling this over for a few days, and I can think of a few strategies we might take here. These are not necessarily mutually exclusive and are in no particular order:

Add a new hint for the "open with config" call, which indicates that any "big/expensive" metadata should not be read or stored in the spec. You'd tend to use this hint only for cases when you were opening the image for the sole purpose of getting the basic resolution and type info. This has the nice property of not breaking any existing code or ABIs, it's optional (for the author of the format reader) and we can implement it selectively, and it's very straightforward.

Amend ImageInput::valid_file(), which is already explicitly a minimal "I don't need the pixels, just tell me if this file exists and appears to be a valid file of this format, as efficiently as possible", to take an optional ImageSpec* parameter, which if non-NULL will fill in just the basic resolution-like fields but none of the metadata that would go in ImageSpec::extra_attribs. This would break ABI by either changing call signature or adding a new virtual method, but it wouldn't break API compatibility.

Exclude big/expensive things (which by definition would include thumbnails, ICC profiles, big manifests, etc.) from storing in the ImageSpec, and instead either have dedicated new ImageInput API calls to retrieve them (and in ImageOutput to store them?), and they would be lazily read only if retrieved. Or store them in a separate "ParamList for the big things" that's only read optionally. But now we're back to what I said we didn't want, which is a proliferation of calls to ferry a growing list of individual data items in and out, and ways for user code to lose metadata if they aren't careful or as the list of special things grows in the future.

Treat thumbnails as the one special case. So you would need one extra call to retrieve it from the input, one extra call to give it to the output. Cons: (a) Old code wouldn't know about it, and new code could forget to make the calls. (b) It doesn't get us any closer to figuring out what to do with the ID manifest or other large metadata. Possible Pro: You could argue that almost anything you do to an image would/should invalidate any thumbnail it originally had, so maybe it's a good thing to forget it by default and have to take a separate step to generate and attach a correct one when you're outputting an image?

Try to have a fully lazy metadata pipeline: Add a public ImageInput::getattribute() that can retrieve metadata (which would, internally check m_spec.extra_attribs first, and if not found then it would read and add the expensive thing). Small metadata would be added to m_spec.extra_attribs right away upon open, but big things would only be added when they are first requested (and if the user requests the full spec(), anything remaining to be read is read). This is source compatible -- anybody requesting the spec() as a whole would trigger the whole read, but if you're careful to ask for only particular things, they would only be read as (or if) you need them. I'm a little nervous about this, it does seem like the tricky logic of when to read things could have subtle bugs and edge cases.

Don't store the thumbnail in the metadata, always treat it as if it's a second subimage in the file. Pro: It wouldn't be read if you didn't read the second subimage. Cons: (a) breaks old code; (b) need some way to know whether the subimage is specifically a thumbnail of the first image versus an unrelated image.

If this is a special case of the thumbnail read on Targa being especially slow to read, perhaps we can just make that go faster, eliminating the need for this feature generally. (Note: most of the formats we deal with don't have provisions for thumbnails at all, so maybe this is just a matter of optimizing one or two format readers to the point that you no longer consider it a perf issue.)

Do nothing at all, OIIO is optimized for reading full images and it's not worth adding complexity to speed up tools that are merely scraping a disk to find resolutions of all the images but not caring about pixels. (I'm not necessarily making that argument myself, just saying that it is a potentially defensible position.)

Maybe there are other approaches, too. I would love feedback on this matter.

Another one I thought of:

Another variant of "open with config" hint: The hint could be a list of items the app doesn't need, which the reader should avoid reading and putting in to the spec, if possible. This differs from my prior suggestion of a simple "no big things" request, in that supplying a list is more flexible. As an example, maybe an app knows it doesn't need a thumbnail, but it does want the ICC profile if one is present.

There's definitely lots of options, and not all of them are mutually exclusive.
The hint to "skip big metadata" seems like the easiest path forward to me, though it is fairly limited to this particular use case. If you wanted to later get the metadata for a file you had already opened this way, you would need to close it and re-open it "for real" right? The definition of what counts as "big" would also be up to the individual file formats to honor, which could be tricky to homogenize across the format plugins, and hard for users of the library to reason about.
If we had the luxury of rethinking the API some more I would probably argue for making all metadata load on demand (while still keeping provisions for round-tripping everything properly when re-saving files). But this seems like it would be really hard to retrofit on top of existing code like you said.
The issue of how the manage thumbnails specifically seems a bit different to me and worth thinking about separately. We should have some way of addressing the thumbnail in a file as similarly as possible to other pixel data. I'm not sure that using the subimage concept works well - don't some formats support both thumbnails and subimages (exr?). This could get confusing for a user wanting to (say) loop over the subimages in a file, not realizing one of them is the thumbnail. A hint could be given in the subimage's spec (or main file spec) but you would have to know to check for it.
So maybe its worth adding a specific "thumbnail image" concept in the API? Similar to subimages, but addressed as their own thing. That would be an API change though, and break any existing code that manipulates thumbnails via the metadata API.
This is a tricky one!

If you wanted to later get the metadata for a file you had already opened this way, you would need to close it and re-open it "for real" right?
The definition of what counts as "big" would also be up to the individual file formats to honor, which could be tricky to homogenize across the format plugins, and hard for users of the library to reason about.

Yeah, I am assuming that you would need to close and re-open. This features is for the common case when you only need the basics (think: iinfo *.tif, you just want the list of files and resolutions).
One simple unambiguous dividing line is the hard coded fields of ImageSpec (the things that tell you the "shape" of the image, like resolution, nchannels, etc.) versus everything that would go in extra_attribs. In fact, it could correspond to the ImageSpec::copy_dimensions call?

If we had the luxury of rethinking the API some more I would probably argue for making all metadata load on demand (while still keeping provisions for round-tripping everything properly when re-saving files). But this seems like it would be really hard to retrofit on top of existing code like you said.

Maybe not so hard? Most existing code asks for ImageInput::spec() and then pulls out fields from there. We could make the implementation of spec() read the whole thing (un-lazify it), which would make existing code work as-is. And add a new call for retrieving an individual piece of metadata, which would load upon being called (if it wasn't already loaded). So the only programs that have to change are the ones that want to selectively pull things out.

The issue of how the manage thumbnails specifically seems a bit different to me and worth thinking about separately. We should have some way of addressing the thumbnail in a file as similarly as possible to other pixel data.

Thumbnails may indeed be a special case. I'm not averse to adding a single ImageInput call "give me thumbnail", and ImageOutput call "here is the thumbnail". It could even pass ImageBuf's around, very convenient. The only downside to this is that old programs would drop them on the floor, unless updated to make the extra calls. But do we care? Almost none of the formats we use in VFX support thumbnails (OpenEXR does not, BTW).
Grepping... Currently, Targa and PSD are our only format readers that put thumbnails in the metadata! (RAW should, but it's an unimplemented "FIXME".) And we don't have a PSD writer. So the only thing that could get lost at this moment is an app (not part of OIIO, because we would fix that) reading Targa and then writing it back out, losing the thumbnail. Why would you read a Targa and then write a Targa? You're not transcoding. Maybe you're modifying the image itself? But if you do that, the thumbnail is very likely no longer correct anyway.
So maybe thumbnails deserve their own thing. But that does nothing to solve things like the ID manifest. Though maybe that's not a real problem, either, considering they probably aren't present in any file that the reader wouldn't want that info, I don't know.

I would definitely vote for the opt-out of metadata approach, either via the "no big things" or "list of things to ignore" mode. Being at the begining of the pipeline I want everything possible to come along for the ride, but I can see how people at the tail end would just want the pixels.

Almost none of the formats we use in VFX support thumbnails (OpenEXR does not, BTW).
https://github.com/AcademySoftwareFoundation/openexr/blob/master/src/lib/OpenEXR/ImfPreviewImageAttribute.h ?
Unless you meant within OIIO, in which case yes, our OpenEXR plugin doesn't currently support reading or writing thumbnails.

The only downside to this is that old programs would drop them on the floor, unless updated to make the extra calls. But do we care?
I would love support for both reading, writing, and autogenerating thumbnails from within OIIO, and I don't think older apps not having the API support would matter too much. Nobody is currently screaming about the lack of support. A separate API for the thumbnails makes sense to me (definitely not treated as a subimage).

Another option is to do what FBX does.  You specify everything you wish to read upfront via flags.
    if (GetFBXImporter()->IsFBX())
    {
        FbxIOSettings *ioSettings = GetFBXSDKManager()->GetIOSettings();

        IOS_REF.SetBoolProp(IMP_FBX_CONSTRAINT, false);
        IOS_REF.SetBoolProp(IMP_FBX_CONSTRAINT_COUNT, false);
        IOS_REF.SetBoolProp(IMP_FBX_MATERIAL, true);
        IOS_REF.SetBoolProp(IMP_FBX_TEXTURE, true);
        IOS_REF.SetBoolProp(IMP_FBX_LINK, true);
        IOS_REF.SetBoolProp(IMP_FBX_SHAPE, true);
        IOS_REF.SetBoolProp(IMP_FBX_GOBO, false);
        IOS_REF.SetBoolProp(IMP_FBX_ANIMATION, true);
        IOS_REF.SetBoolProp(IMP_FBX_GLOBAL_SETTINGS, true);
        IOS_REF.SetBoolProp(IMP_FBX_CURRENT_TAKE_NAME, true);
    }

  lStatus = GetFBXImporter()->Import(GetFBXScene());

FbxIOSettings Class Reference

@shootfast Right you are! I had forgotten that OpenEXR has a thumbnail option. I guess OIIO's lack of it for the past 13 years has not gotten anybody upset enough to even point out that I never got around to it.
@DougRogers Yeah, that's akin to the configuration hint about what to read. I think the correct default is to read everything, but an exclusion list makes sense, and there can be an alias for "exclude everything but the dimensional info".
So I think this is an initial plan for what seems to be resonating (both with you guys as well as in my own head):

An open() configuration option that lets an app say they don't need individual items (including nearly everything). Readers can, optionally, choose to avoid expensive steps if they consult the list. This is 100% non-breaking and can even be backported to the release branch.

For master only, make a new get/set thumbnail API call in the reader and writer. If the thumbnails are wrapped in an ImageBuf, that also means that generating them from an existing buffer is just one call to IBA::resize/resample.

For later, we can contemplate more lazy behavior. One thing I was thinking is that since all user-side access to the metadata is through the spec() call, we could add an optional parameter to spec() that says "I want it all" or "just the basics". The default would be "all", preserving old behavior. Calling with "all" would do the extra work if not already done (it still would only happen once), and the initial open would only provide the basics. Of course, any other API calls would call spec(all) internally if their operations needed to know the rest. This, too, is changing the call signatures, so would not be able to be backported. But I think it is straightforward.

Of course, any other API calls would call spec(all) internally if their operations needed to know the rest. This, too, is changing the call signatures, so would not be able to be backported. But I think it is straightforward.

Perhaps you could add an optional spec config argument that defaults to 'all' to maintain compatibility.

@DougRogers Yes, exactly what I had in mind.

I'm a little late to the party here, but I like the idea of an optional "exclusion" list, and I'm curious what you're thinking in terms of API.
My assumption after reading through this thread is that exclusions would be specified as names. If so, it may be a good idea (or even necessary) to support pattern matching of some kind, especially if you're doing something like examining a large number of images of unknown provenance (similar to the original issue description), that, for whatever reason, may use inconsistent metadata keys.
That use-case then made me wonder if the additional option would actually make more sense as an include filter (or if both include and exclude modes could be supported), although I still think pattern-matching would be important in that scenario. Am I off in the weeds here?

The most general approach could be maybe "thing1,thing2,thing3" or "all,-thing4,-thing5" so it covers both inclusion and exclusion.  But is a fine-grained control really needed?
I'm still mulling this over in my head and I even started sketching a prototype last night and now I am less sure of some of this. We already mentioned thumbnails as deserving their own API calls and no longer being part of the metadata. There are very few other cases in very few formats where there might be a concrete performance benefit to excluding particular pieces of metadata.  We wouldn't want to slow down the usual case of wanting everything for the sake of slightly speeding up the very rare case of wanting a specific subset of things. I think the relatively common case (once we have excluded thumbnails) is wanting nothing beyond the dimensional information. Maybe that's all we should do for now, get the first stage done, and then see what appetite there is for more?
So after some fooling around with it last night and sleeping on it, now I'm thinking like this:

First, add the "separate thumbnail" API change.

Second, add just enough of the selection hint to say "I don't need any metadata beyond the dimensional info."

Then, wait for further feedback and see if there are any other cases that are worth additional complexity.

I would add some items that should be part of the minimal set:

Image size and pitch
Image format, compressed or not
Image data size
Is thumbnail present and its size
Are previews present, count, size of each and their formats.   Raw format may have previews.

If you look at the ImageSpec definition, there are hard fields for most of those things -- resolution, channels, data type, tile size, whether it's a deep file, etc. Then everything else, all the "arbitrary metadata" gets shoved in the extra_attribs list.
ImageSpec even has two copy methods, operator= which copies everything (including a deep copy of the whole extra_attribs paramlist), and copy_dimensions() which omits extra_attribs and the channel names and is very inexpensive.
So that's a pretty natural dividing line.

It starts to get muddy if you want to ask for the presence or size of a piece of expensive metadata (the metametadata?), but don't want the data itself. It's a lot less clear how often (across items and file formats) you would be able to know those things without incurring most of the expense of just getting the whole thing.

I would exclude those from the basic data and just include that type of data if requested.

There are several COMMON cases:

Wanting everything including the pixels.
Wanting all the metadata, but not the pixels.
Just wanting to know if it seems to be a valid file.

We've got those covered already.
We also have a concrete proposal for these less common but plausible cases:

Don't need the thumbnail, so would like to avoid any expense of decoding it (thumbnails are a special case that could be drastically more expensive than other large metadata, if the thumbnail pixels need decompression).
Don't want any of the arbitrary metadata, just want to know the file's resolution, channels, data type.

We can dismiss the cases where you want to exclude a subset of the small cheap things, because there's no benefit to adding extra logic to skip them. You would never be able to measure the perf gain, and you might even come out behind if the cost of checking item by item was more than just copying the small items.
With thumbnails disposed of, how many cases of large/expensive metadata are there? I can think of only a few cases -- ICC profiles maybe? Name manifests in OpenEXR objectID files? Mesh connectivity in PTex files? Anything else? It's not clear how often there exists any file that has more than one of them, and if there are, when you would want one of those but not the other, and if you did, whether there would be any measurable perf benefit to being that fine-grained.
You have to also understand that for many of the file formats, the underlying reader library may have already gone to the expense of reading the whole header up front as soon as it opens, so imagining that we are saving significant expense by not asking for a few items is closing the barn door after the horse is long gone.

You could have several levels of data.

Simple, size and spec items
Med.  Thumbnails
Image
large meta-data

I had forgotten about this, but in looking at imageio.h, I just realized that the ImageInput already has separate spec() and a spec_dimensions() calls, with the latter returning an ImageSpec copy that only has the important fields filled in. It seems like without breaking the interface, we could easily make it so that most of the metadata is not read until spec() is called or until some other internal functionality needs other metadata. If all the user does is open() and then get spec_dimensions(), it would never worry about most of the metadata. So the simple 2-stage lazy metadata reading is already half plumbed and could even be backported without breaking APIs.

All great points. In that light, the "K.I.S.S." changes you outlined make good sense to me.

Just because I haven't seen anyone mention it yet as a pretty big usecase:
Generally when streaming in tiles (especially from many different textures), I don't believe there is any need at all for reading any metadata beyond the layout (planar/separate if relevant), format, dimension and mip/subimage count.
So much so, that in things I've set up to deal with mipmapping, actually attempt to clear as much arb metadata as possible (SHA-1, Comments, Commandline etc) to prevent any additional reading during renders.

TIFF is reading a lot of data during open().
TIFFInput::readspec() is reading decoding xmp data
// Search for an XML packet containing XMP (IPTC, Exif, etc.)
int xmlsize         = 0;
const void* xmldata = NULL;
if (TIFFGetField(m_tif, TIFFTAG_XMLPACKET, &xmlsize, &xmldata)) {
    // std::cerr << "Found XML data, size " << xmlsize << "\n";
    if (xmldata && xmlsize) {
        std::string xml((const char*)xmldata, xmlsize);
        decode_xmp(xml, m_spec);
    }
}

Bumping this - I also encountered PNGs that are very slow to open due to XMP metadata. Here's a sample file attached
HUD_LowerLeft.png.zip


Move Project Jupyter section to lesson 01

In this PR, I've moved the Project Jupyter section from lesson 02 (shell) to lesson 01 (scientific coding environment). I think this makes sense thematically because Project Jupyter makes an online scientific coding environment! It also fits with the revised CoCo course schedule for week 1.
Note that I also changed the filename of the shell lesson to 02_shell.md to follow the naming convention.

@aguspesce @isamarcortes Could I please get help with Spanish translations for a few short passages:

intro sentences of lesson 01
intro sentence of lesson 02
title of lesson 02

Thank you! | Gracias!


chat with workspace

GitHub Copilot Chat revolutionizes coding by allowing developers to generate entire workspaces from scratch using natural language. However, once a workspace is created, the current iteration lacks the capability for ongoing, in-depth interaction with the codebase. This proposal introduces the concept of expanding Copilot Chat with three distinct "brains" or modes: a Simple Chat, a Code Control Brain, and a Code Conversation Brain, each tailored to different stages and types of user interaction.
Current Limitation:
After initial workspace generation, users may find themselves needing more nuanced and direct interaction capabilities with their code. The current setup does not support continuous, context-aware conversations with the existing code or offer granular control over code manipulation and understanding through the chat interface.
Proposed Feature Details:
Simple Chat Mode: This mode would handle general queries and provide assistance similar to the current functionality but enhanced for more generalist advice, documentation lookup, and coding best practices. It's aimed at broad, non-project-specific interactions.
 It would offer capabilities such as refactoring code, suggesting optimizations, debugging, and adding new features or functionalities directly through conversation. This brain would have deep integration with the user's current workspace, providing precise and context-aware actions.
Code Conversation Brain: Designed for an interactive dialogue about the open code in the editor, this mode would allow developers to discuss their code with Copilot as if they were pair programming with a colleague. Questions could range from "Why is this function implemented this way?" to "What are the potential side effects of this method?" This conversational approach would facilitate a deeper understanding of the codebase and support educational and exploratory coding practices.
Benefits:
Enhanced Productivity: By offering specialized interaction modes, developers can work more efficiently, leveraging the most appropriate type of assistance based on their current task.
Improved Code Quality: Direct manipulation and conversation about the code could lead to better optimization, cleaner code, and more effective debugging.
Educational Support: The Code Conversation Brain can serve as a learning tool, offering explanations and discussions about coding decisions, patterns, and best practices.
Conclusion:
Integrating these three brains into GitHub Copilot Chat would significantly expand its utility, making it not just a tool for initiating projects but a comprehensive companion throughout the coding process. This feature request aims to bridge the gap between initial code generation and ongoing project development, ensuring that developers have access to tailored support at every stage of their work.

Can you tell me more about what you feel is missing with the way things work currently?


How to run tests?

Hi,
I'm trying to package this up for Alpine, and am having trouble getting it to run the tests. It doesn't seem to be able to find the fixture tmp_files:
file /tmp/pytest-of-builder/pytest-0/test_tmp_files_indirect0/test_tmp_files_indirect.py, line 3
  @pytest.mark.parametrize(
          'tmp_files, expected', [
                  ({'file.txt': 'spam'}, 'spam'),
                  ({'file.txt': 'eggs'}, 'not eggs'),
          ],
          indirect=['tmp_files'],
  )
  def test_open(tmp_files, expected):
E       fixture 'tmp_files' not found
>       available fixtures: cache, capfd, capfdbinary, caplog, capsys, capsysbinary, doctest_namespace, monkeypatch, pytestconfig, record_property, record_testsuite_property, record_xml_attribute, recwarn, tmp_path, tmp_path_factory, tmpdir, tmpdir_factory
>       use 'pytest --fixtures [testpath]' for help on them.

The command I'm running is:
PYTHONPATH="$PWD" pytest tests/
I've tried creating a conftest.py containing:
from pytest_tmp_files import plugin

But this doesn't help.
Suggestions appreciated!

Usually this error means that pytest_tmp_files isn't installed.  It shouldn't be necessary to import anything.  If you're curious how pytest finds the tmp_files fixture, it's via the pytest11 entry point (defined in pyproject.toml).

That's the point, I want to run the tests at the package creation point. The package isn't installed yet

Hmmm...  The package has to be registered as a pytest plugin for the tests to run.  I don't know anything about packaging, but I'm surprised there's not a way to deal with this.  Surely there must be lots of packages that have to be installed before they can be tested, right?
In any case, it looks like you might be able to set the $PYTEST_PLUGINS environment variable to get pytest to treat the source code as a plugin, see here and here.  You could also always (i) create a virtual environment, (ii) install the package in there, (iii) run the tests, then (iv) delete the virtual environment.  Your conftest.py idea was good, but I think it didn't work because these tests spawn fresh pytest processes, so anything imported in the parent process isn't seen by its children.
Let me know if any of those ideas help.

The venv idea was the key. I realised that alpine packages up other pytest fixtures, and when I looked at how they are tested, they use the venv approach.
I've duplicated that in this package, and it now tests Ok.
Thanks for the assistance. Much appreciated.

No problem, glad you got it working!


Send data

The current implementation closes the sink after each message. rtcMessageWorker will then loop, detect the closed channel and sleep for 1 second. If multiple message are sent during this time they will be dropped as each message will create (and drop) the channel.
This PR moves closing the sink back into RedisRouter to keep the current functionality. LocalRouter only closes the channel on shutdown.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Document which Rust types can be placed into eBPF maps

It appears that the RedBPF toolchain imposes currently undocumented restrictions on map value types that go beyond those expressed by the generic constraints:

All variants of an enum must have the same size in memory.
Struct fields are (mostly?) required to be aligned to multiples of 32 bits, though sometimes 64-bit alignment is required, and some structs don't work at all despite their fields being aligned.
Tweaking alignments with #[repr(align(n))] has no effect whatsoever. Whether or not #[repr(C)] is used doesn't seem to matter either.
Double-width types like u128 cannot be placed in nested structs. They sometimes work in flat structs though.
Fields with array types are hit-and-miss, and whether they work or not appears to depend on the phase of the moon.
Getting any of this wrong results in an incomprehensible error message from libbpf, which gives no hint whatsoever about what is actually going on.

It would be great if there was documentation explaining precisely what the requirements for Rust types are so they can be placed into the eBPF map types provided by RedBPF.
It might also be a good idea to mark all methods exposed by the map types as unsafe. The reason is that normally, you can rely on Rust's type system to tell you whether your arguments have acceptable types. But here there are secondary requirements that cannot be expressed using trait bounds, but will nevertheless result in catastrophic program failure when disregarded. That's pretty much the definition of unsafety.

I wrote some note about valid structs for BPF maps. I hope this note will be written to other forms of documentation later.
While developing BPF programs that deal with BPF maps, errors due to illegal
data definition frequently occur. It is basically FFI and is naturally a part
of unsafe Rust, so we have to be careful. If something goes wrong, infamous
defined behaviors will happen.
Here I am focusing on defining structs. unions and enums are off the
table here.
I couldn't come up with all possible ways to define valid structures. But users
can avoid pesky problems if they follow the conservative rules I suggest below.

Use #[repr(C)] representation
Since the default representation does not guarantee anything about the
layout, we can not control anything when BPF verifier complaints about the
structure. Hence, let's use repr(C) always and control the layout of the
structure and understand the specific size of the structure and figure out
the alignments of all fields in the structure. Handling BPF maps is
conducted by calling BPF helpers and it is naturally FFI. So repr(C)
deserves to be used in BPF programs.

Add padding fields explicitly
You may see the error if you don't add padding fields correctly:

invalid indirect read from stack R3 off -128+92 size 96

You should add padding fields not only at the tail of structure, but also
before the fields that need to be aligned properly using padding.
Remember the principle of layout:

each field of structure needs to be aligned correctly
whole size of structure is multiple of alignment of the structure
Refer to the rust references for more information:
1 2

Limit the maximum alignment of structure to 8 bytes
In BPF programs, to store values in BPF map bpf_map_update_elem is
called. It is wrapped by redbpf_probes::maps::HashMap::set. Inside
kernel, storing values into BPF maps involves copying values. But Linux
kernel copies values to the 8 bytes aligned memory.
So even if a user defines a structure whose alignment is 32 bytes, the
alignment will not be retained during copying the value into a BPF
map. Thus in BPF programs, the pointer returned by bpf_map_lookup_elem is
not aligned to 32 bytes. But redbpf_probes::maps::HashMap::get
immediately creates a reference from the pointer by &*. However creating
a reference of it is counted as undefined behavior because rust compiler
always expects reference to be properly aligned. If this assumption is
imploded, all code that is built upon the assumption is invalid.

See these kernel source code that allocates memory aligned to 8 bytes:
1
2
See this kernel source code that copy values to memory where address is
aligned to 8 bytes. Note that l_new->key is aligned to 8 bytes and
round_up(key_size, 8) rounds up to 8 bytes.
1

I also mentioned the alignment issue the 3 item points to, at PR #215 ..


feat: Add re-export for ConversionReview type in stackable-webhook

This adds a little more convenience when using ConversionWebhookServer. Users now don't need an additional explicit dependency on kube to import the required type.
Came up in https://github.com/stackabletech/documentation/pull/561

Thanks for the approval, I will add the above mentioned changes https://github.com/stackabletech/operator-rs/pull/749#discussion_r1521059405, which requires another approval when done.


new implementations for variance and stdev reductions

What changes were proposed in this pull request?
new implementations are used for variance,stdev reductions. minor fix
in stdev. Bench and correctness tests are moved inside #ifdef block
How was this patch tested?
unit tests
Quick checklist
The following checklist helps ensure your PR is complete:

[x] Eclipse Contributor Agreement signed, and signed commits - see IP Requirements page for details
[x] Reviewed the Contributing Guidelines and followed the steps within.
[x] Created tests for any significant new code additions.
[x] Relevant tests for your changes are passing.

eclipse/deeplearning4j#8826