New native transport for kqueue

Motivation:
We currently don't have a native transport which supports kqueue https://www.freebsd.org/cgi/man.cgi?query=kqueue&sektion=2. This can be useful for BSD systems such as MacOS to take advantage of native features, and provide feature parity with the Linux native transport.
Modifications:

Make a new transport-native-unix-common module with all the java classes and JNI code for generic unix items. This module will build a static library for each unix platform, and included in the dynamic libraries used for JNI (e.g. transport-native-epoll, and eventually kqueue).
Make a new transport-native-unix-common-tests module where the tests for the transport-native-unix-common module will live. This is so each unix platform can inherit from these test and ensure they pass.
Add a new transport-native-kqueue module which uses JNI to directly interact with kqueue

Result:
JNI support for kqueue.
Fixes https://github.com/netty/netty/issues/2448
Fixes https://github.com/netty/netty/issues/4231

@normanmaurer @nmittler @marcuslinke @timothyjward @shpikat @caarlos0 @ninja- @rnorth @ndeloof @slandelle @headius (everyone on the related issues) - PTAL.
There is a large amount of code here ... sorry about that. However it would be nice to get some miles on it and gain more confidence.
There is also some duplication in the Channel hierarchy between EPOLL and KQUEUE. It is not possible for them to share a common hierarchy so for now I think its is OK to keep the duplication and look to improve/consolidate later.

🎉 🎉 🎉 I'll give it a try as soon as I can (will probably be next week). Thanks a lot!

Build failure seems to be related to not being able to find the artifact from a new sub-module introduced in this PR. I recall seeing this issue in the past on the CI servers, and thought we changed the build order or added an extra build pass to resolve this type of issues ... @trustin @normanmaurer do you recall?

Could not resolve dependencies for project io.netty:netty-transport-native-epoll:jar:4.1.7.Final-SNAPSHOT: Could not find artifact io.netty:netty-transport-native-unix-common:jar:linux-x86_64:4.1.7.Final-SNAPSHOT

nice to see that !

@normanmaurer @trustin - Looks like we do the following on the build servers:
$mvn clean validate
<will fail if new modules are added in the PR>
$mvn test

can we instead do:
$mvn clean install -DskipTests
$mvn test

@Scottmitch Sounds great! Will try it this weekend. Thanks for your effort!

@Scottmitch we not want to do an "install" as this will install "changes" that may break other builds on the same host.  I wonder if we can't tell maven to just resolve this somehow internally or if we can't tell maven to use an temporary local maven repository.

Let me investigate

@Scottmitch ok seems like I can use a custom local maven repos... Let me try this out on the CI and kick off another build. Also thanks for this "massive" work :)

@Scottmitch for the record the build is successful locally here :)

@Scottmitch ok it should now work on the CI as soon as the build is fixed for linux.
The problem is it builds and installs this one:
netty-transport-native-unix-common-4.1.7.Final-SNAPSHOT-linux-x86_64-fedora.jar
But it looks for:
netty-transport-native-unix-common:jar:linux-x86_64:4.1.7.Final-SNAPSHOT
The unix-common should not add -fedora to it.
Please fix and the build should work :)

@normanmaurer - I actually want the classifier on the artifact ... but I was just inconsistently using epoll.classifier and os.detected.arch inconsistently. That should now be fixed.
However it looks like the build server is still doing the clean validate ... do I have to open a new PR to get the changes?

@Scottmitch let me check the config again... maybe I missed to change it to install but at least it uses the local repository now.
Why you need to use the "-fedora" classifier ? We not used it before in the epoll transport as it works on fedora and debian etc....

Why you need to use the "-fedora" classifier

Not specifically the "-fedora" type classifier ... just the same classifier used by the epoll module (e.g. <epoll.classifier>${os.detected.name}-${os.detected.arch}</epoll.classifier>)

When trying to bootstrap my application like this:
EventLoopGroup nioEventLoopGroup = new KQueueEventLoopGroup(0,
                new DefaultThreadFactory(threadPrefix));

            bootstrap.group(nioEventLoopGroup).channel(KQueueDomainSocketChannel.class)
                    .handler(new ChannelInitializer<SocketChannel>() {
                        @Override
                        protected void initChannel(final SocketChannel channel) throws Exception {
                            channel.pipeline().addLast(new LoggingHandler(getClass()));
                            channel.pipeline().addLast(new HttpClientCodec());
                        }
                    });

            return nioEventLoopGroup;

I get an exception
java.lang.UnsatisfiedLinkError: failed to load the required native library
	at io.netty.channel.kqueue.KQueue.ensureAvailability(KQueue.java:70)
	at io.netty.channel.kqueue.KQueueEventLoopGroup.<clinit>(KQueueEventLoopGroup.java:35)
	at com.github.dockerjava.netty.NettyDockerCmdExecFactory$UnixDomainSocketInitializer.init(NettyDockerCmdExecFactory.java:223)
	at com.github.dockerjava.netty.NettyDockerCmdExecFactory.init(NettyDockerCmdExecFactory.java:195)
	at com.github.dockerjava.core.TestDockerCmdExecFactory.init(TestDockerCmdExecFactory.java:91)
	at com.github.dockerjava.core.DockerClientImpl.withDockerCmdExecFactory(DockerClientImpl.java:161)
	at com.github.dockerjava.core.DockerClientBuilder.build(DockerClientBuilder.java:45)
	at com.github.dockerjava.client.AbstractDockerClientTest.beforeTest(AbstractDockerClientTest.java:57)
	at com.github.dockerjava.netty.exec.VersionCmdExecTest.beforeTest(VersionCmdExecTest.java:22)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:86)
	at org.testng.internal.Invoker.invokeConfigurationMethod(Invoker.java:514)
	at org.testng.internal.Invoker.invokeConfigurations(Invoker.java:215)
	at org.testng.internal.Invoker.invokeConfigurations(Invoker.java:142)
	at org.testng.TestRunner.beforeRun(TestRunner.java:656)
	at org.testng.TestRunner.run(TestRunner.java:624)
	at org.testng.SuiteRunner.runTest(SuiteRunner.java:366)
	at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:361)
	at org.testng.SuiteRunner.privateRun(SuiteRunner.java:319)
	at org.testng.SuiteRunner.run(SuiteRunner.java:268)
	at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
	at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86)
	at org.testng.TestNG.runSuitesSequentially(TestNG.java:1244)
	at org.testng.TestNG.runSuitesLocally(TestNG.java:1169)
	at org.testng.TestNG.run(TestNG.java:1064)
	at org.testng.remote.AbstractRemoteTestNG.run(AbstractRemoteTestNG.java:126)
	at org.testng.remote.RemoteTestNG.initAndRun(RemoteTestNG.java:137)
	at org.testng.remote.RemoteTestNG.main(RemoteTestNG.java:58)
Caused by: java.lang.NoSuchMethodError: Method io.netty.channel.unix.Socket.recvFd(I)I not found
	at java.lang.ClassLoader$NativeLibrary.load(Native Method)
	at java.lang.ClassLoader.loadLibrary1(ClassLoader.java:1965)
	at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1890)
	at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1851)
	at java.lang.Runtime.load0(Runtime.java:795)
	at java.lang.System.load(System.java:1062)
	at io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:36)
	at io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:255)
	at io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:222)
	at io.netty.channel.kqueue.Native.loadNativeLibrary(Native.java:103)
	at io.netty.channel.kqueue.Native.<clinit>(Native.java:50)
	at io.netty.channel.kqueue.KQueue.<clinit>(KQueue.java:33)

Maybe have declared the wrong dependencies ?
<dependency>
			<groupId>io.netty</groupId>
			<artifactId>netty-transport-native-epoll</artifactId>
			<version>${netty.version}</version>
			<classifier>linux-x86_64</classifier>
			<exclusions>
				<exclusion>
					<groupId>io.netty</groupId>
					<artifactId>netty-transport-native-unix-common</artifactId>
				</exclusion>
			</exclusions>
		</dependency>
		<dependency>
			<groupId>io.netty</groupId>
			<artifactId>netty-transport-native-kqueue</artifactId>
			<version>${netty.version}</version>
			<classifier>osx-x86_64</classifier>
			<exclusions>
				<exclusion>
					<groupId>io.netty</groupId>
					<artifactId>netty-transport-native-unix-common</artifactId>
				</exclusion>
			</exclusions>
		</dependency>
		<dependency>
			<groupId>io.netty</groupId>
			<artifactId>netty-transport-native-unix-common</artifactId>
			<version>${netty.version}</version>
		</dependency>

You also need to specify the qualifier for the unix-common dependency AFAIC

Am 11.11.2016 um 08:51 schrieb marcuslinke<EMAIL_ADDRESS>When trying to bootstrap my application like this:
EventLoopGroup nioEventLoopGroup = new KQueueEventLoopGroup(0,
new DefaultThreadFactory(threadPrefix));
        bootstrap.group(nioEventLoopGroup).channel(KQueueDomainSocketChannel.class)
                .handler(new ChannelInitializer<SocketChannel>() {
                    @Override
                    protected void initChannel(final SocketChannel channel) throws Exception {
                        channel.pipeline().addLast(new LoggingHandler(getClass()));
                        channel.pipeline().addLast(new HttpClientCodec());
                    }
                });

        return nioEventLoopGroup;

I get an exception
java.lang.UnsatisfiedLinkError: failed to load the required native library
at io.netty.channel.kqueue.KQueue.ensureAvailability(KQueue.java:70)
at io.netty.channel.kqueue.KQueueEventLoopGroup.(KQueueEventLoopGroup.java:35)
at com.github.dockerjava.netty.NettyDockerCmdExecFactory$UnixDomainSocketInitializer.init(NettyDockerCmdExecFactory.java:223)
at com.github.dockerjava.netty.NettyDockerCmdExecFactory.init(NettyDockerCmdExecFactory.java:195)
at com.github.dockerjava.core.TestDockerCmdExecFactory.init(TestDockerCmdExecFactory.java:91)
at com.github.dockerjava.core.DockerClientImpl.withDockerCmdExecFactory(DockerClientImpl.java:161)
at com.github.dockerjava.core.DockerClientBuilder.build(DockerClientBuilder.java:45)
at com.github.dockerjava.client.AbstractDockerClientTest.beforeTest(AbstractDockerClientTest.java:57)
at com.github.dockerjava.netty.exec.VersionCmdExecTest.beforeTest(VersionCmdExecTest.java:22)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:86)
at org.testng.internal.Invoker.invokeConfigurationMethod(Invoker.java:514)
at org.testng.internal.Invoker.invokeConfigurations(Invoker.java:215)
at org.testng.internal.Invoker.invokeConfigurations(Invoker.java:142)
at org.testng.TestRunner.beforeRun(TestRunner.java:656)
at org.testng.TestRunner.run(TestRunner.java:624)
at org.testng.SuiteRunner.runTest(SuiteRunner.java:366)
at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:361)
at org.testng.SuiteRunner.privateRun(SuiteRunner.java:319)
at org.testng.SuiteRunner.run(SuiteRunner.java:268)
at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86)
at org.testng.TestNG.runSuitesSequentially(TestNG.java:1244)
at org.testng.TestNG.runSuitesLocally(TestNG.java:1169)
at org.testng.TestNG.run(TestNG.java:1064)
at org.testng.remote.AbstractRemoteTestNG.run(AbstractRemoteTestNG.java:126)
at org.testng.remote.RemoteTestNG.initAndRun(RemoteTestNG.java:137)
at org.testng.remote.RemoteTestNG.main(RemoteTestNG.java:58)
Caused by: java.lang.NoSuchMethodError: Method io.netty.channel.unix.Socket.recvFd(I)I not found
at java.lang.ClassLoader$NativeLibrary.load(Native Method)
at java.lang.ClassLoader.loadLibrary1(ClassLoader.java:1965)
at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1890)
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1851)
at java.lang.Runtime.load0(Runtime.java:795)
at java.lang.System.load(System.java:1062)
at io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:36)
at io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:255)
at io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:222)
at io.netty.channel.kqueue.Native.loadNativeLibrary(Native.java:103)
at io.netty.channel.kqueue.Native.(Native.java:50)
at io.netty.channel.kqueue.KQueue.(KQueue.java:33)
Maybe have declared the wrong dependencies ?

io.netty
netty-transport-native-epoll
${netty.version}
linux-x86_64

io.netty
netty-transport-native-unix-common

io.netty
netty-transport-native-kqueue
${netty.version}
osx-x86_64

io.netty
netty-transport-native-unix-common

io.netty
netty-transport-native-unix-common
${netty.version}

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

@normanmaurer Using classifier  'osx-x86_64' I get the same exception.

@marcuslinke  - netty-transport-native-unix-common is a transitive dependency that you shouldn't have to explicitly include in your application. This is where all the unix general stuff from the transport-native-epoll module moved to so it could be shared with transport-native-kqueue. The goal is that you should be able to include the same way you do the epoll module http://netty.io/wiki/native-transports.html#using-the-native-transport.
While its in PR form make sure you do the following:
$ cd netty
$ git checkout <this branch>
$ mvn clean install -DskipTests

And your netty.version property should be set to 4.1.7.Final-SNAPSHOT.
Let me know if there are still issues after trying this.

@Scottmitch The problem is that I've to support both systems (linux and macos), so how this should work. As I understand the os-maven-plugin detects the build systems OS so the dependencies classifier is set to 'macos' effectively. Therefore I decided to hardcode the classifiers in the POM somehow. Any ideas?

@Scottmitch Why netty-transport-native-unix-common needs a qualifier? Is it really needed?

The problem is that I've to support both systems (linux and macos), so how this should work.

Depends how you want to achieve this.

isolate all your epoll or kqueue code in a module that is conditionally declared as a dependency in the pom.
Always include them both, and just check KQueue.isAvailable() or EPOLL.isAvailable() before you use either.

OS so the dependencies classifier is set to 'macos' effectively

Perhaps but is their a reasons to hardcode and not just use the recommended approach?

netty-transport-native-unix-common also builds a native library ... although it is static and included in the kqueue's and epoll's native library at build time. It may be possible to omit the clarifier I would prefer not to as it clarifies which platform the artifact was built on, and incase there needs to be any differences between platforms we can accommodate.

@Scottmitch Your branch compiles and installs fine here. But even when I do as suggested I get the same exception...
java.lang.UnsatisfiedLinkError: failed to load the required native library

io.netty
netty-transport-native-epoll
4.1.7.Final-SNAPSHOT
linux-x86_64

io.netty
netty-transport-native-kqueue
4.1.7.Final-SNAPSHOT
osx-x86_64

@Scottmitch You said the native library libnetty-unix-common.a is used only at compile time of the epoll and kqueue modules. Maybe there is something wrong in my local build so this library is linked dynamically and missing at runtime?

Maybe there is something wrong in my local build

Its possible ... thanks for helping me test on different environments 👍
If you run the following all the tests pass, or do you see the same issue?
$ cd transport-native-unix-common-tests
$ mvn clean install
$ cd ../transport-native-unix-common
$ mvn clean install
$ cd ../transport-native-kqueue
$ mvn clean install

I have a build failure in transport-native-unix-common:
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.5.1:compile (default-compile) on project netty-transport-native-unix-common: Compilation failure: Compilation failure:
[ERROR] /Users/marcus/git/netty/transport-native-unix-common/src/main/java/io/netty/channel/unix/PeerCredentials.java:[21] error: cannot find symbol
[ERROR] symbol:   static EMPTY_INTS
[ERROR] location: class
[ERROR] /Users/marcus/git/netty/transport-native-unix-common/src/main/java/io/netty/channel/unix/PeerCredentials.java:[40,35] error: cannot find symbol

please run a mvn clean install -DskipTests at the root level of the repo and then try again.

[INFO] Netty/Transport/Native/Unix/Common ................. SUCCESS [  2.608 s]
[INFO] Netty/Transport/Native/Unix/Common/Tests ........... SUCCESS [  1.550 s]
[INFO] Netty/Transport/Native/KQueue ...................... SUCCESS [ 43.670 s]
[INFO] Netty/All-in-One ................................... SUCCESS [  9.113 s]
[INFO] Netty/Tarball ...................................... SUCCESS [  0.209 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 03:06 min
[INFO] Finished at: 2016-11-11T21:41:46+01:00
[INFO] Final Memory: 72M/871M

And still java.lang.UnsatisfiedLinkError: failed to load the required native library when try to use kqueue from within my project.

java.lang.NoClassDefFoundError: io/netty/channel/unix/LimitsStaticallyReferencedJniMethods here https://github.com/Scottmitch/netty/blob/218592882316d5a070f7d9245fcb08954e32ec5b/transport-native-kqueue/src/main/java/io/netty/channel/kqueue/KQueue.java#L35

When doing the following:
$ cd transport-native-unix-common-tests
$ mvn clean install
$ cd ../transport-native-unix-common
$ mvn clean install
$ cd ../transport-native-kqueue
$ mvn clean install

I get:
Running io.netty.channel.kqueue.KQueueSocketChannelNotYetConnectedTest
21:55:40.819 [main] DEBUG i.n.u.i.l.InternalLoggerFactory - Using SLF4J as the default logging framework
21:55:40.828 [Time-limited test] DEBUG i.n.c.MultithreadEventLoopGroup - -Dio.netty.eventLoopThreads: 16
21:55:40.844 [Time-limited test] DEBUG i.n.util.internal.PlatformDependent0 - java.nio.Buffer.address: available
21:55:40.845 [Time-limited test] DEBUG i.n.util.internal.PlatformDependent0 - sun.misc.Unsafe.theUnsafe: available
21:55:40.845 [Time-limited test] DEBUG i.n.util.internal.PlatformDependent0 - sun.misc.Unsafe.copyMemory: available
21:55:40.846 [Time-limited test] DEBUG i.n.util.internal.PlatformDependent0 - direct buffer constructor: available
21:55:40.846 [Time-limited test] DEBUG i.n.util.internal.PlatformDependent0 - java.nio.Bits.unaligned: available, true
21:55:40.846 [Time-limited test] DEBUG i.n.util.internal.PlatformDependent0 - java.nio.DirectByteBuffer.<init>(long, int): available
21:55:40.847 [Time-limited test] DEBUG io.netty.util.internal.Cleaner0 - java.nio.ByteBuffer.cleaner(): available
21:55:40.847 [Time-limited test] DEBUG i.n.util.internal.PlatformDependent - Java version: 8
21:55:40.847 [Time-limited test] DEBUG i.n.util.internal.PlatformDependent - -Dio.netty.noUnsafe: false
21:55:40.847 [Time-limited test] DEBUG i.n.util.internal.PlatformDependent - sun.misc.Unsafe: available
21:55:40.848 [Time-limited test] DEBUG i.n.util.internal.PlatformDependent - -Dio.netty.noJavassist: false
21:55:40.929 [Time-limited test] DEBUG i.n.util.internal.PlatformDependent - Javassist: available
21:55:40.929 [Time-limited test] DEBUG i.n.util.internal.PlatformDependent - -Dio.netty.tmpdir: /var/folders/y3/8mx9s87n01942z74s5wjsxjw0000hb/T (java.io.tmpdir)
21:55:40.929 [Time-limited test] DEBUG i.n.util.internal.PlatformDependent - -Dio.netty.bitMode: 64 (sun.arch.data.model)
21:55:40.929 [Time-limited test] DEBUG i.n.util.internal.PlatformDependent - -Dio.netty.noPreferDirect: false
21:55:40.930 [Time-limited test] DEBUG i.n.util.internal.PlatformDependent - io.netty.maxDirectMemory:<PHONE_NUMBER> bytes
21:55:40.943 [Time-limited test] DEBUG io.netty.channel.nio.NioEventLoop - -Dio.netty.noKeySetOptimization: false
21:55:40.943 [Time-limited test] DEBUG io.netty.channel.nio.NioEventLoop - -Dio.netty.selectorAutoRebuildThreshold: 512
21:55:40.944 [Time-limited test] DEBUG i.n.util.internal.PlatformDependent - org.jctools-core.MpscChunkedArrayQueue: available
21:55:40.963 [Time-limited test] DEBUG i.n.u.internal.NativeLibraryLoader - -Dio.netty.tmpdir: /var/folders/y3/8mx9s87n01942z74s5wjsxjw0000hb/T (java.io.tmpdir)
21:55:40.963 [Time-limited test] DEBUG i.n.u.internal.NativeLibraryLoader - -Dio.netty.native.workdir: /var/folders/y3/8mx9s87n01942z74s5wjsxjw0000hb/T (io.netty.tmpdir)
21:55:40.979 [Time-limited test] DEBUG io.netty.util.NetUtil - -Djava.net.preferIPv4Stack: false
21:55:40.979 [Time-limited test] DEBUG io.netty.util.NetUtil - -Djava.net.preferIPv6Addresses: false
21:55:40.982 [Time-limited test] DEBUG io.netty.util.NetUtil - Loopback interface: lo0 (lo0, 0:0:0:0:0:0:0:1)
21:55:40.983 [Time-limited test] DEBUG io.netty.util.NetUtil - /proc/sys/net/core/somaxconn: 128 (non-existent)
dyld: lazy symbol binding failed: Symbol not found: _clock_gettime
  Referenced from: /private/var/folders/y3/8mx9s87n01942z74s5wjsxjw0000hb/T/libnetty-transport-native-kqueue4723678605595081288.dylib (which was built for Mac OS X 10.12)
  Expected in: /usr/lib/libSystem.B.dylib
dyld: Symbol not found: _clock_gettime
  Referenced from: /private/var/folders/y3/8mx9s87n01942z74s5wjsxjw0000hb/T/libnetty-transport-native-kqueue4723678605595081288.dylib (which was built for Mac OS X 10.12)
  Expected in: /usr/lib/libSystem.B.dylib
/bin/sh: line 1: 83323 Trace/BPT trap: 5       /Library/Java/JavaVirtualMachines/jdk1.8.0_45.jdk/Contents/Home/jre/bin/java -server -dsa -da -ea:io.netty... -XX:+AggressiveOpts -XX:+TieredCompilation -XX:+UseBiasedLocking -XX:+UseFastAccessorMethods -XX:+OptimizeStringConcat -XX:+HeapDumpOnOutOfMemoryError '-javaagent:/Users/marcus/.m2/repository/org/mortbay/jetty/alpn/jetty-alpn-agent/2.0.4/jetty-alpn-agent-2.0.4.jar=${jetty.alpnAgent.option}' -D_ -D_ -D_ -jar /Users/marcus/git/netty/transport-native-kqueue/target/surefire/surefirebooter830902336434320850.jar /Users/marcus/git/netty/transport-native-kqueue/target/surefire/surefire219842681247456731tmp /Users/marcus/git/netty/transport-native-kqueue/target/surefire/surefire_06621318527175539997tmp

Results :

Tests run: 0, Failures: 0, Errors: 0, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------

After reading https://github.com/Homebrew/homebrew-core/issues/4809 I've installed command line tools from Apple. The Symbol not found: _clock_gettime issue disappears then but now there is another problem: https://gist.github.com/marcuslinke/24ec140def4cd162cd6180aba34ca171

@marcuslinke - Can you update your JDK? There was a known issue in your JDK related to GCM ciphers that would core dump.

@Scottmitch I've updated my JDK to 1.8.0_111 and the kqueue related tests now run successfully. But still having the issue when try to use the kqueue stuff in my own project:
Tests run: 5, Failures: 1, Errors: 0, Skipped: 4, Time elapsed: 1.495 sec <<< FAILURE! - in com.github.dockerjava.netty.exec.VersionCmdExecTest
beforeTest(com.github.dockerjava.netty.exec.VersionCmdExecTest)  Time elapsed: 1.388 sec  <<< FAILURE!
java.lang.UnsatisfiedLinkError: failed to load the required native library
	at com.github.dockerjava.netty.exec.VersionCmdExecTest.beforeTest(VersionCmdExecTest.java:22)
Caused by: java.lang.NoClassDefFoundError: io/netty/channel/unix/LimitsStaticallyReferencedJniMethods
	at com.github.dockerjava.netty.exec.VersionCmdExecTest.beforeTest(VersionCmdExecTest.java:22)
Caused by: java.lang.ClassNotFoundException: io.netty.channel.unix.LimitsStaticallyReferencedJniMethods
	at com.github.dockerjava.netty.exec.VersionCmdExecTest.beforeTest(VersionCmdExecTest.java:22)

For the files: my project is compiled with Java 7 (1.7.0_80)

After adding dependency to unix-common I ended up again with:
Caused by: java.lang.NoSuchMethodError: Method io.netty.channel.unix.Socket.recvFd(I)I not found
    at java.lang.ClassLoader$NativeLibrary.load(Native Method)

@marcuslinke - Is your project open source? Is https://github.com/docker-java/docker-java the project you are attempting to use kqueue in? If so can you send me a link to a PR so I can poke around?

Sure, here it is https://github.com/docker-java/docker-java/tree/netty-native-kqueue

Just run mvn clean install -Dit.test=VersionCmdExecTest

@marcuslinke - The test fails, but I think it is because I don't have docker running (or someone opening the /var/run/docker.sock socket). I was able to accomplish this with some changes to your pom. I commented out the epoll module for now because I didn't go through the trouble of building the epoll module and copying it to my local maven repo. However when we publish artifacts we could publish epoll built from MacOS and kqueue built from Linux to make including in the pom files easier (the other alternative is to partition you project as described in https://github.com/netty/netty/pull/6000#issuecomment-260015816). Let me know if this allows to you to do testing.
Test output:
09:37:32.123 INFO  c.g.d.c.AbstractDockerClientTest - ======================= BEFORETEST =======================
09:37:32.126 INFO  c.g.d.c.AbstractDockerClientTest - Connecting to Docker server
09:37:32.498 DEBUG c.g.d.core.command.AbstrDockerCmd - Cmd: busybox,com.github.dockerjava.netty.exec.InspectImageCmdExec@2b506a79
Tests run: 5, Failures: 1, Errors: 0, Skipped: 4, Time elapsed: 1.097 sec <<< FAILURE! - in com.github.dockerjava.netty.exec.VersionCmdExecTest
beforeTest(com.github.dockerjava.netty.exec.VersionCmdExecTest)  Time elapsed: 1.02 sec  <<< FAILURE!
java.lang.RuntimeException: socket not found: /var/run/docker.sock
	at com.github.dockerjava.netty.exec.VersionCmdExecTest.beforeTest(VersionCmdExecTest.java:22)

diff --git a/pom.xml b/pom.xml
index 39f84a4..441ef78 100644
--- a/pom.xml
+++ b/pom.xml
@@ -244,35 +244,17 @@
 			<artifactId>netty-handler-proxy</artifactId>
 			<version>${netty.version}</version>
 		</dependency>
-		<dependency>
+		<!--<dependency>
 			<groupId>io.netty</groupId>
 			<artifactId>netty-transport-native-epoll</artifactId>
 			<version>${netty.version}</version>
-			<classifier>linux-x86_64</classifier>
-			<exclusions>
-				<exclusion>
-					<groupId>io.netty</groupId>
-					<artifactId>netty-transport-native-unix-common</artifactId>
-				</exclusion>
-			</exclusions>
-		</dependency>
+			<classifier>${os.detected.classifier}</classifier>
+		</dependency>-->
 		<dependency>
 			<groupId>io.netty</groupId>
 			<artifactId>netty-transport-native-kqueue</artifactId>
 			<version>${netty.version}</version>
-			<classifier>osx-x86_64</classifier>
-			<exclusions>
-				<exclusion>
-					<groupId>io.netty</groupId>
-					<artifactId>netty-transport-native-unix-common</artifactId>
-				</exclusion>
-			</exclusions>
-		</dependency>
-		<dependency>
-			<groupId>io.netty</groupId>
-			<artifactId>netty-transport-native-unix-common</artifactId>
-			<version>${netty.version}</version>
-			<classifier>osx-x86_64</classifier>
+			<classifier>${os.detected.classifier}</classifier>
 		</dependency>
 		<dependency>
 			<groupId>junit</groupId>
@@ -308,7 +290,7 @@
 			<extension>
 				<groupId>kr.motd.maven</groupId>
 				<artifactId>os-maven-plugin</artifactId>
-				<version>1.4.0.Final</version>
+				<version>1.5.0.Final</version>
 			</extension>
 		</extensions>
 		<pluginManagement>

@Scottmitch Thanks a lot! So it seems the epoll module SNAPSHOT dependency retrieved from the main 4.1 branch has caused this.

@Scottmitch Some of my tests that passed with the old epoll module now fail with throwing io.netty.util.IllegalReferenceCountException when using the kqueue module. For example:
io.netty.util.IllegalReferenceCountException: refCnt: 0, decrement: 1
	at io.netty.buffer.AbstractReferenceCountedByteBuf.release0(AbstractReferenceCountedByteBuf.java:110)
	at io.netty.buffer.AbstractReferenceCountedByteBuf.release(AbstractReferenceCountedByteBuf.java:98)
	at io.netty.buffer.AbstractPooledDerivedByteBuf.deallocate(AbstractPooledDerivedByteBuf.java:73)
	at io.netty.buffer.AbstractReferenceCountedByteBuf.release0(AbstractReferenceCountedByteBuf.java:115)
	at io.netty.buffer.AbstractReferenceCountedByteBuf.release(AbstractReferenceCountedByteBuf.java:98)
	at io.netty.util.ReferenceCountUtil.release(ReferenceCountUtil.java:84)
	at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:112)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:351)
	at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:293)
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:267)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:351)
	at com.github.dockerjava.netty.handler.HttpResponseHandler.channelRead0(HttpResponseHandler.java:72)
	at com.github.dockerjava.netty.handler.HttpResponseHandler.channelRead0(HttpResponseHandler.java:1)
	at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:351)
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:233)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:351)
	at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:435)
	at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:293)
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:267)
	at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:250)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:351)
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:233)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:351)
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359)
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926)
	at io.netty.channel.kqueue.AbstractKQueueStreamChannel$KQueueStreamUnsafe.readReady(AbstractKQueueStreamChannel.java:609)
	at io.netty.channel.kqueue.KQueueDomainSocketChannel$KQueueDomainUnsafe.readReady(KQueueDomainSocketChannel.java:127)
	at io.netty.channel.kqueue.AbstractKQueueChannel$AbstractKQueueUnsafe.readReady(AbstractKQueueChannel.java:335)
	at io.netty.channel.kqueue.KQueueEventLoop.processReady(KQueueEventLoop.java:201)
	at io.netty.channel.kqueue.KQueueEventLoop.run(KQueueEventLoop.java:273)
	at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:873)
	at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
	at java.lang.Thread.run(Thread.java:745)

Do you have an idea what could cause this?

@marcuslinke - Do you mean that running this PR with transport-native-epoll on linux doesn't report an exception, but running this PR with transport-native-kqueue on MacOS does report this exception?
Based on at io.netty.buffer.AbstractPooledDerivedByteBuf.deallocate(AbstractPooledDerivedByteBuf.java:73) being in the stack trace this exception may be caused by the same issue as https://github.com/netty/netty/issues/5999. Can you use the UnpooledByteBufAllocator until this PR is pulled in?

Do you mean that running this PR with transport-native-epoll on linux doesn't report an exception, but running this PR with transport-native-kqueue on MacOS does report this exception?

Right.

Can you use the UnpooledByteBufAllocator until the fix for #5999 is pulled in?

Not sure how to do this. My custom handlers in the pipeline only allocate buffers via Unpooled.buffer(). Running with -Dio.netty.allocator.type=unpooled makes the issue disappear.

@marcuslinke thanks for reporting back... Using the system property is for sure a good way to change the default.

@normanmaurer So I think changing the system property indicates a netty issue here. Right?

@marcuslinke yes... This is what @Scottmitch referred to with  #5999.

@normanmaurer Thank. But how to deal with this PR? @Scottmitch Will the fix for #5999 be merged into #6000?

@marcuslinke nope... once #5999 is fixed and merged we will rebase this pr and you can test again. Stay tuned, @Scottmitch is working on a fix

As @normanmaurer said I will push a PR which fixed #5999 soon :) (but this is an independent issue)

Running with -Dio.netty.allocator.type=unpooled makes the issue disappear.

Thanks for confirming! Do you have any other issues or is everything working as expected?

My custom handlers in the pipeline only allocate buffers via Unpooled.buffer()

Unrelated to this issue, but it is recommended to use the allocator configured in the Bootstrap object. This can be accessed through the Channel or the ChannelHandlerContext

@Scottmitch Beside issue #5999 everything works as expected. You did a great job. Thanks again! Hope this will be merged soon.

@marcuslinke - Yay! Thanks for your help!
Before pulling this in I will polish the netty-all deployment story first ... and hopefully get some more volunteers ( @slandelle @vietj 😉) to get a bit more confidence

@Scottmitch Whats the current state of this PR? Will it be merged in the near future?

@marcuslinke yes after 4.1.8.Final.

I cloned the PR and got Unix Domain Sockets running on my Mac via kqueue!  Thank you guys for your great work :)
I ran into an issue during shutdown though.  I'm not sure if it is because I am using Netty incorrectly, please forgive my ignorance if that is the case.
Our shutdown logic for a Unix domain socket server is:
eventLoop.shutdownGracefully();
try {
    channel.closeFuture().sync();
}
catch (InterruptedException e) {
    Thread.currentThread().interrupt();
}

where eventLoop is of type KQueueEventLoopGroup and channel is of type KQueueServerDomainSocketChannel.
This same logic worked for Epoll running on Linux.  But when I tried the above with kqueue, it hangs indefinitely on the sync() part of channel.closeFuture().sync();.

@dannyferreira - Can you try and reverse the order in which you shutdown the event loop and close the channel? The channel may need to use the event loop for shutdown operations.

build failure captured in issue https://github.com/netty/netty/issues/6577

@Scottmitch - closing the channel before shutting down the event loop seems to work well for both kqueue and epoll.  Thanks!

@normanmaurer - Any objections to merging this? I'm not sure that anyone has actually reviewed the code. We need to coordinate the release cycle and builds on the CI servers.
Release Cycle
I'm not very familiar with how the maven release process works but IIRC we need to do something like the following:

On linux: mvn clean install

either publish available artifacts directly from this machine, or copy artifacts over the machine in step (2))

On Mac: mvn clean install -Duber

If we released in step (1) we should only have to build/release transport-native-kqueue, all, and tarball.

It doesn't matter which platform is built first but the -Duber will build netty-all and depends upon all native libraries being available.  IIRC we do something similar for the release of tcnative which includes jars for all the different platforms? We should be able to do something similar here.
CI server PR builds
We need to build PRs on MacOS and Linux (to some degree). It would be nice if we could just build the kqueuebits on MacOS and build the rest on Linux (since we have more Linux CI servers). I don't have much visibility into the CI configuration so I'm not sure how to accomplish this.

@Scottmitch please see the comment: https://github.com/Scottmitch/netty/pull/1#issuecomment-290704233

@Scottmitch Managed to build and run your kqueue transport and run it with docker java but seems it fails on one of my colleagues machine (he didn't build your branch but just reused the jar):
dyld: lazy symbol binding failed: Symbol not found: _netty_unix_util_parse_package_prefix
Referenced from: /private/var/folders/dc/_328f1bn5zggz0xm2ztxxfmr0000gn/T/libnetty-transport-native-kqueue4910433518549296650.dylib
Expected in: flat namespace
Any idea ?

4.1 (3cc405296)

@lucboutier - What version of macOS was the artifact built on, and what version of macOS are you trying to use the artifact on? Now that the PR has been merged can you open an issue?

@Scottmitch Tried to use kqueue native transport under MacOS 10.11.6 and got
dyld: lazy symbol binding failed: Symbol not found: _clock_gettime
  Referenced from: /private/var/folders/y3/8mx9s87n01942z74s5wjsxjw0000hb/T/libnetty-transport-native-kqueue8841250350785055197.dylib (which was built for Mac OS X 10.12)
  Expected in: /usr/lib/libSystem.B.dylib

Any idea if this could be fixed somehow?

Created an issue for it: https://github.com/netty/netty/issues/6837


iTerm Nightly Support - AppleScript Changes - Et al.

shuttle is supporting iTerm Stable and Nightly versions. In the settings you can set:
"iTerm_version": "stable", or "iTerm_version": "nightly",
You can now set a command to open in a new terminal window or in the current terminal
 "inTerminal": "new", or  "inTerminal": "current",
The current terminal assumes that the title and theme have been set by another command. Shuttle is not passing the Title or the Theme when using "inTerminal": "current",
You can set a custom path to the JSON file. This is useful if you want to share your config with several computers.
In your home directory create ~/.shuttle.path in this file set the full path to your JSON file.
Mine currently reads/Users/thshdw/Desktop/shuttle.json
Applescripts are now pre-compiled the scripts handler is called / loaded.

@fitztrev You're Welcome! :smile:


This is too vague

(1) Is <CertificateIssuer> the CN of the authority, the full DN (e.g; CN=Subordinate,DC=domain,DC=local) or what exactly. Windows' certificate property viewer breaks this value down in several lines. 2. Is <X509Certificate> a thumbprint, a serial number, a CN, a file path, a DN. There's not matching value for that in the built-in tools. Same goes for <RegisteredIssuer>. Though most people might be able to figure it out, when variables are used in documentation examples must be given. 3. On the SharePoint side, there's no explanation on what to do when Root CAs conflict...like it's bound to happen in a domain with their local CA, e.g; if claims auth has been setup.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: e13939c6-9846-8b98-bb52-68e7d942efce
Version Independent ID: 57a4cc31-f44f-7a85-ae56-e40108d397c2
Content: Configure server-to-server authentication between Office Online Server and SharePoint Server 2016
Content Source: OfficeOnlineServer/configure-office-online-server-for-sharepoint-server-2016/configure-server-to-server-authentication-between-office-online-server-and-share.md
Product: office-online-server
GitHub Login: @santanu-wac
Microsoft Alias: samukhe

Immediately above that section is this text:

 - The URL of your top-level site collection.
 - The name of the certificate issuer. You can find this by viewing the Details tab of the certificate in IIS Manager.
 - The path and file name of the certificate file that you exported.
 - The GUID of the trusted token issuer. SharePoint Server is 67e3df25-268a-4324-a550-0de1c7f97287@bd2372e4-0a11-495c-9541-8377c6def195 and SQL Server is 67e3df25-268a-4324-a550-0de1c7f97287@ffab2d74-c6ae-4375-819a-8555d49b699a

Does that clarify what is required?


[BEAM-5445] [BEAM-4796] SpannerIO: Only batch on the current bundle. Adds streaming support

Rewrite the mechanism whereby SpannerIO batches the incoming mutations.
Previously the entire input was scanned, assigned partitions and GroupByKey before writing. This required that the input was bounded, large amounts of disk space, and the entire input had to be read before any writes started.
With this change, the input is only sorted/batched over the current Bundle (or 1000x max batch size, whichever is smaller).
This has the following benefits:

Significantly decreases latency before the first row is written (down to a few seconds after the pipeline is started)
Allows the SpannerIO writer to be used in a streaming pipeline
Reduces the amount of worker disk space required for large sources
Significantly improves the write speed for large sources

This PR also includes PR #6422 and #6409
@chamikaramj
Post-Commit Tests Status (on master branch)

Lang
SDK
Apex
Dataflow
Flink
Gearpump
Samza
Spark

Go

---
---
---
---
---
---

Java

Python

---

---
---
---

R: @nithinsujir

Thanks Niel. Added @nithinsujir for a first round of reviews for this and https://github.com/apache/beam/pull/6422. I can take a look next week.

LGTM. I suggest we wait for @biswanag comments as well.

LGTM. Thanks.
Can you fix up to one (or few) commits for merging.

Run Java PostCommit

Run Java PreCommit

Can you fix up to one (or few) commits for merging.

Done. (also updated title to reflect changes).

Hello, do know if this is going to make it to release 2.9.0?


coverage tools fail on relative source path

Originally reported by Dima Tisnek (Bitbucket: dimaqq, GitHub: dimaqq)

d = coverage.data.CoverageData(".coverage")
d.lines[FILENAME] = {7: None, 8:None, 11:None, 12: None}
d.write()

then coverage reportran from . where both .coverage and source.py are present, depending on FILENAME above, observed

source_that_doesnt_exist.py -- complains that source cannot be found, ok
/full/path/to/source.py -- no error, coverage 25%, ok
./source.py -- no error, coverage 0%, why?
source.py -- no error, coverage 0%, why?

It appears that error reporting is not consistent with coverage reporting.
same for coverage html

Bitbucket: https://bitbucket.org/ned/coveragepy/issue/320
This issue had attachments: ctest.tgz.  See the original issue for details.

for me, whatever I put in source, it completely ignores it and coveres my entire env instead (except the defined source)

@manoadamro Is your virtual environment inside your working tree? Can you show us how to reproduce what you are seeing?

@nedbat I can't even consistently reproduce it myself tbh, it just happens from time to time. Deleting all the cache files (basically everything except the actual source code files) seems fix it though.

Feel free to re-open this if you can provide more information.


fix: updated deps

:tada: This PR is included in version 2.13.1 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


add allowPrivilegedContainers to flavors

What this PR does / why we need it:
Adds allowPrivilegedContainers as an additional option in the flavor configuration. Gardener shoots default to true is this option is not set, hence you can use it to disable privileged containers for certain tests or have your tests run against both: shoots with allowPrivilegedContainers="true" and shoots with allowPrivilegedContainers="false"
Release note:
`allowPrivilegedContainers` can now be customized in the flavor configuration.

Regarding metadata, I though it would be enough to distinguish via flavor description, but you are right, a dedicated metadata is better, I'll add that, thanks for the suggestion.

added metadata support, hope I didn't miss some places, ptal

we also need to read the metadta from the testrun here https://github.com/gardener/test-infra/blob/master/pkg/testmachinery/metadata/metadata.go#L57

Added.

How about adding it also here https://github.com/gardener/test-infra/blob/master/pkg/testmachinery/metadata/metadata.go#L41 so that it will also be displayed in the tm dashboard and the slack notifications?

I actually envisioned to put it in the description of the flavor configuration, but sure, putting it directly in the code is more consistent in the end, so added it.


Ensure line endings are not modified in fixtures

Looks good. Something that occurred to me as I was reviewing other pull-requests: what do you think about adding AppVeyor integrations as we fix specs on all these repos? https://github.com/atom/ci should have a ready-to-go script to setup the integration.


Make output created after strip_value cover tx fee

The existing code looks correct to me. This PR effectively adds two fees together and ends up with an assertion error:
thread 'main' panicked at 'assertion failed: `(left == right)`
  left: `Amount(0.00000265 BTC)`,
 right: `Amount(0.00000154 BTC)`: invariant: fee estimation is correct'


argument change in GATK

Hi Brad,
I ran gatk-nextgen for on paired-end illumina data and get this error!
ERROR MESSAGE: SAM/BAM file SAMFileReader{/media/proj1/mutech/2s2ns/work/bamclean/SID38973/2ns-reorder-fixrgs.bam} is malformed: the BAM file has a read with no stored bases (i.e. it uses '*') which is not supported in the GATK; see the --filter_bases_not_stored argument. Offender: @;>
ERROR ------------------------------------------------------------------------------------------
GATK said that this error can be resolved by setting  --filter_bases_not_stored, but is it feasible with bcbio-nextgen?
http://gatkforums.broadinstitute.org/discussion/3597/did-anyone-solved-the-incompatibility-of-bam-files-by-tmap-with-gatk
Is it common for illumina platforms to ouput  reads with no stored bases?
Many thanks!
Son.

Hi,
I am struggling with the same problem. I preformed amplicon sequencing using the Illumina MiSeq. I am using the fastq files from the MiSeq, creating sam files using bwa, then converting the sam files to bam files using samtools, and attempting to realign the bam files using GATK. GATK gives me the following error:
ERROR MESSAGE: SAM/BAM file SAMFileReader{Sample1.S1.sorted.bam} is malformed: the BAM file has a read with no stored bases (i.e. it uses '*') which is not supported in the GATK; see the --filter_bases_not_stored argument. Offender: (null)
Did you find out what was the cause of this error?
Regards,
Lisa

for the record, I just had this error, issued by GATK BaseRecalibrator. I tracked it down to using bwa-mem -a, ie "-a: Output all found alignments for single-end or unpaired paired-end reads. These alignments will be flagged as secondary alignments."
i'm now hunting for tips on fixing these slightly broken BAMs

Mark;
Thanks for following up here with the solution -- great to have this discussion be useful. Much appreciated.

Hello,
I'm running into this issue myself. I tracked this down to an empty read.
I'm not really sure how can I provide bcbio the relevant flags to overcome this problematic reads.
`
GGGGGGGGGGIIIGIIIIIIGIIIIIIIIIIIIIIIIIIIIIIIGGIIIGIIIIIIIGGGGGGGIGIIGGIIIIGGIIGGGIGGIIGIIIGIIGIIGIIIG
@D00166:565:HWFZ6BCXX:1:2302:18025:97411 1:N:0:TGGTGGT
CTTGTGGGGTGACTGAGGAACCCCAGCGACTCTTTTATGGTGAGTGCTCTCAGCCTCAAGACTCCTCCCATAGAGACTGGGGGAAAAGAGGGGACTTTACC
+
AAGAGGGAAGGAGGGAAGA..AGGI.AGAGG.GGAAGGIIGGGGGGGGIGIGGGIGGGAGGIGGGG.GG<.GAAGGGGGGIGA.GGGGGGGGGGGGGGGAG
@D00166:565:HWFZ6BCXX:1:2302:18159:97423 1:N:0:TGGTGGT

@D00166:565:HWFZ6BCXX:1:2302:18080:97449 1:N:0:TGGTGGT
ATAAGCGTTAGTTCTTGAAACCAAGGCATTTGGGCAAATATTATACATTTTTATTTTATTAATTTTCCAGAACCCGTTTGAACCATGAAGCCATTTGTGC`
And this is the log
[2016-10-17T07:14Z] INFO  07:14:05,256 ProgressMeter -            done      4741.0     1.0 s       4.7 m       99.7%     1.0 s       0.0 s
[2016-10-17T07:14Z] INFO  07:14:05,256 ProgressMeter - Total runtime 1.34 secs, 0.02 min, 0.00 hours
[2016-10-17T07:14Z] INFO  07:14:05,257 MicroScheduler - 2 reads were filtered out during the traversal out of approximately 4743 total reads (0.04%)
[2016-10-17T07:14Z] INFO  07:14:05,257 MicroScheduler -   -> 0 reads (0.00% of total) failing BadCigarFilter
[2016-10-17T07:14Z] INFO  07:14:05,257 MicroScheduler -   -> 0 reads (0.00% of total) failing MalformedReadFilter
[2016-10-17T07:14Z] INFO  07:14:05,258 MicroScheduler -   -> 2 reads (0.04% of total) failing NotPrimaryAlignmentFilter
[2016-10-17T07:14Z] INFO  07:14:05,846 GATKRunReport - Uploaded run statistics report to AWS S3
[2016-10-17T07:14Z] Index BAM file: 1_2016-10-16_germline_pipeline-sort-chrUn_gl000215_0_172545-prep.bam
[2016-10-17T07:14Z] GATK: realign ('chrUn_gl000216', 0, 172294) : 224433
[2016-10-17T07:14Z] INFO  07:14:06,285 GATKRunReport - Uploaded run statistics report to AWS S3
[2016-10-17T07:14Z] ##### ERROR ------------------------------------------------------------------------------------------
[2016-10-17T07:14Z] ##### ERROR A USER ERROR has occurred (version 3.5-0-g36282e4):
[2016-10-17T07:14Z] ##### ERROR
[2016-10-17T07:14Z] ##### ERROR This means that one or more arguments or inputs in your command are incorrect.
[2016-10-17T07:14Z] ##### ERROR The error message below tells you what is the problem.
[2016-10-17T07:14Z] ##### ERROR
[2016-10-17T07:14Z] ##### ERROR If the problem is an invalid argument, please check the online documentation guide
[2016-10-17T07:14Z] ##### ERROR (or rerun your command with --help) to view allowable command-line arguments for this tool.
[2016-10-17T07:14Z] ##### ERROR
[2016-10-17T07:14Z] ##### ERROR Visit our website and forum for extensive documentation and answers to
[2016-10-17T07:14Z] ##### ERROR commonly asked questions http://www.broadinstitute.org/gatk
[2016-10-17T07:14Z] ##### ERROR
[2016-10-17T07:14Z] ##### ERROR Please do NOT post this error to the GATK forum unless you have really tried to fix it yourself.
[2016-10-17T07:14Z] ##### ERROR
[2016-10-17T07:14Z] ##### ERROR MESSAGE: SAM/BAM/CRAM file htsjdk.samtools.SamReader$PrimitiveSamReaderToSamReaderAdapter@4a8fa358 is malformed. Please see http://gatkforums.broadinstitute.org/discussion/1317/collected-faqs-about-input-files-for-sequence-read-data-bam-cramfor more information. Error details: the BAM file has a read with no stored bases (i.e. it uses '*') which is not supported in the GATK; see the --filter_bases_not_stored argument. Offender: D00155:466:HWF7YBCXX:1:2205:18159:97423
[2016-10-17T07:14Z] ##### ERROR ------------------------------------------------------------------------------------------
[2016-10-17T07:14Z] Uncaught exception occurred
Traceback (most recent call last):
File "/usr/local/share/bcbio-nextgen/anaconda/lib/python2.7/site-packages/bcbio/provenance/do.py", line 21, in run
do_run(cmd, checks, log_stdout)
File "/usr/local/share/bcbio-nextgen/anaconda/lib/python2.7/site-packages/bcbio/provenance/do.py", line 95, in do_run
raise subprocess.CalledProcessError(exitcode, error_msg)
CalledProcessError: Command 'set -o pipefail; java -Xms166m -Xmx583m -XX:+UseSerialGC -Djava.io.tmpdir=/mnt/work/tx/tmpqUEvZa -jar /mnt/work/inputs/jars/gatk/GenomeAnalysisTK.jar -T PrintReads -L chr1:109827209-142541836 -R /mnt/work/inputs/data/genomes/hg19/seq/hg19.fa -I /mnt/work/align/224433/1_2016-10-16_germline_pipeline-sort.bam -BQSR /mnt/work/align/224433/1_2016-10-16_germline_pipeline-sort.grp -U LENIENT_VCF_PROCESSING --read_filter BadCigar --read_filter NotPrimaryAlignment -o /mnt/work/bamprep/224433/chr1/tx/tmp1KuGJ/1_2016-10-16_germline_pipeline-sort-chr1_109827208_142541836-prep.bam
INFO  07:12:35,635 HelpFormatter - --------------------------------------------------------------------------------
INFO  07:12:35,637 HelpFormatter - The Genome Analysis Toolkit (GATK) v3.5-0-g36282e4, Compiled 2015/11/25 04:03:56
INFO  07:12:35,637 HelpFormatter - Copyright (c) 2010 The Broad Institute
INFO  07:12:35,637 HelpFormatter - For support and documentation go to http://www.broadinstitute.org/gatk
INFO  07:12:35,640 HelpFormatter - Program Args: -T PrintReads -L chr1:109827209-142541836 -R /mnt/work/inputs/data/genomes/hg19/seq/hg19.fa -I /mnt/work/align/224433/1_2016-10-16_germline_pipeline-sort.bam -BQSR /mnt/work/align/224433/1_2016-10-16_germline_pipeline-sort.grp -U LENIENT_VCF_PROCESSING --read_filter BadCigar --read_filter NotPrimaryAlignment -o /mnt/work/bamprep/224433/chr1/tx/tmp1KuGJ/1_2016-10-16_germline_pipeline-sort-chr1_109827208_142541836-prep.bam
INFO  07:12:35,645 HelpFormatter - Executing as ubuntu@frontend001 on Linux 3.13.0-98-generic amd64; OpenJDK 64-Bit Server VM 1.7.0_95-b00.
INFO  07:12:35,646 HelpFormatter - Date/Time: 2016/10/17 07:12:35
INFO  07:12:35,646 HelpFormatter - --------------------------------------------------------------------------------
INFO  07:12:35,646 HelpFormatter - --------------------------------------------------------------------------------
INFO  07:12:35,816 GenomeAnalysisEngine - Strictness is SILENT
INFO  07:12:36,351 ContextCovariate - 		Context sizes: base substitution model 2, indel substitution model 3
INFO  07:12:36,383 GenomeAnalysisEngine - Downsampling Settings: No downsampling
INFO  07:12:36,390 SAMDataSource$SAMReaders - Initializing SAMRecords in serial
INFO  07:12:36,420 SAMDataSource$SAMReaders - Done initializing BAM readers: total time 0.03
INFO  07:12:36,445 IntervalUtils - Processing 32714628 bp from intervals
INFO  07:12:36,664 GenomeAnalysisEngine - Preparing for traversal over 1 BAM files
INFO  07:12:36,883 GenomeAnalysisEngine - Done preparing for traversal
INFO  07:12:36,884 ProgressMeter - [INITIALIZATION COMPLETE; STARTING PROCESSING]
INFO  07:12:36,885 ProgressMeter -                 | processed |    time |    per 1M |           |   total | remaining
INFO  07:12:36,886 ProgressMeter -        Location |     reads | elapsed |     reads | completed | runtime |   runtime
INFO  07:12:36,901 ReadShardBalancer$1 - Loading BAM index data
INFO  07:12:37,149 ReadShardBalancer$1 - Done loading BAM index data
INFO  07:13:07,579 ProgressMeter -  chr1:110469275         0.0    30.0 s      50.8 w        2.0%    25.5 m      25.0 m
INFO  07:13:38,006 ProgressMeter -  chr1:111924380    200005.0    61.0 s       5.1 m        6.4%    15.9 m      14.8 m
INFO  07:14:06,285 GATKRunReport - Uploaded run statistics report to AWS S3
ERROR ------------------------------------------------------------------------------------------
ERROR A USER ERROR has occurred (version 3.5-0-g36282e4):
ERROR
ERROR This means that one or more arguments or inputs in your command are incorrect.
ERROR The error message below tells you what is the problem.
ERROR
ERROR If the problem is an invalid argument, please check the online documentation guide
ERROR (or rerun your command with --help) to view allowable command-line arguments for this tool.
ERROR
ERROR Visit our website and forum for extensive documentation and answers to
ERROR commonly asked questions http://www.broadinstitute.org/gatk
ERROR
ERROR Please do NOT post this error to the GATK forum unless you have really tried to fix it yourself.
ERROR
ERROR MESSAGE: SAM/BAM/CRAM file htsjdk.samtools.SamReader$PrimitiveSamReaderToSamReaderAdapter@4a8fa358 is malformed. Please see http://gatkforums.broadinstitute.org/discussion/1317/collected-faqs-about-input-files-for-sequence-read-data-bam-cramfor more information. Error details: the BAM file has a read with no stored bases (i.e. it uses '*') which is not supported in the GATK; see the --filter_bases_not_stored argument. Offender: D00155:466:HWF7YBCXX:1:2205:18159:97423
ERROR ------------------------------------------------------------------------------------------
' returned non-zero exit status 1
Traceback (most recent call last):
File "/usr/local/bin/bcbio_nextgen.py", line 226, in
main(**kwargs)
File "/usr/local/bin/bcbio_nextgen.py", line 43, in main
run_main(**kwargs)
File "/usr/local/share/bcbio-nextgen/anaconda/lib/python2.7/site-packages/bcbio/pipeline/main.py", line 39, in run_main
fc_dir, run_info_yaml)
File "/usr/local/share/bcbio-nextgen/anaconda/lib/python2.7/site-packages/bcbio/pipeline/main.py", line 82, in _run_toplevel
for xs in pipeline(config, run_info_yaml, parallel, dirs, samples):
File "/usr/local/share/bcbio-nextgen/anaconda/lib/python2.7/site-packages/bcbio/pipeline/main.py", line 147, in variant2pipeline
samples = region.parallel_prep_region(samples, run_parallel)
File "/usr/local/share/bcbio-nextgen/anaconda/lib/python2.7/site-packages/bcbio/pipeline/region.py", line 139, in parallel_prep_region
"piped_bamprep", _add_combine_info, file_key, ["config"])
File "/usr/local/share/bcbio-nextgen/anaconda/lib/python2.7/site-packages/bcbio/distributed/split.py", line 59, in parallel_split_combine
split_output = parallel_fn(parallel_name, split_args)
File "/usr/local/share/bcbio-nextgen/anaconda/lib/python2.7/site-packages/bcbio/distributed/multi.py", line 28, in run_parallel
return run_multicore(fn, items, config, parallel=parallel)
File "/usr/local/share/bcbio-nextgen/anaconda/lib/python2.7/site-packages/bcbio/distributed/multi.py", line 86, in run_multicore
for data in joblib.Parallel(parallel["num_jobs"])(joblib.delayed(fn)(x) for x in items):
File "/usr/local/share/bcbio-nextgen/anaconda/lib/python2.7/site-packages/joblib/parallel.py", line 812, in call
self.retrieve()
File "/usr/local/share/bcbio-nextgen/anaconda/lib/python2.7/site-packages/joblib/parallel.py", line 749, in retrieve
exception = exception_type(report)
TypeError: init() takes at least 3 arguments (2 given)
' returned non-zero exit status 1

Following up on this in #1603


Other subject columns

@josef-janousek Please describe the changes on the wiki - inside the space: https://grips.semantic-web.at/display/ADEQ/Odalic+Core

So the idea is that you can have multiple subject columns? what is the status in UI? If it is not done yet, the question is whether there is time for that (should be considered by Vasek, whether there is time to modify the subject column detection step in UI accordingly)


Support node-rdkafka

Hey,
The best supported and highest performing node kafka library at the moment seems to be https://github.com/Blizzard/node-rdkafka. Any chance we could get support for it in this sensor? I might look at doing it myself, any pointers to get started?

Hey @Tapppi,
supporting node-rdkafka sounds like a good plan. Instrumenting libraries is mostly straightforward. You could take a look at the existing kafka instrumentation [1] as well as the integration tests [2]. Looking forward to your contribution! ❤
[1] https://github.com/instana/nodejs-sensor/blob/master/src/tracing/instrumentation/kafka.js
[2] https://github.com/instana/nodejs-sensor/blob/master/test/tracing/kafka_test.js

Since we never heard back from Tapppi, I'll close this for now.


Problems setting up email account with same settings

I have two openmailbox.org accounts. I set up the first one, no problem. But I'm having problems setting up the second one. I'm using the same defs as the first one. Could this be a bug?

Please use the mailing list for support requests: http://groups.google.com/group/k-9-mail


Invalid link in CONTRIBUTING.md

Overview

In the picture above I have cricled link that does not work. Due to that I am not sure how to follow that guide exactly. I am eager to prepare translation to polish language 😄

I assume that this is that file where translated resources should be added but I would like to receive a confirmation. If yes, I could correct that link.

I would like to make the above change.

😛😛😛😛😛😟
torialay amin
On Sat, May 25, 2019, 11:34 PM Nitin Sahu<EMAIL_ADDRESS>wrote:

I would like to make the above change.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/github/training-kit/issues/675?email_source=notifications&email_token=ALWEUX2ANPNJY73XBV36XOLPXGER7A5CNFSM4HHE25R2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODWHXNWA#issuecomment-495941336,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ALWEUX63PKHDCQX5JLQDIFTPXGER7ANCNFSM4HHE25RQ
.

hey @paluchnuggets sorry for the delay. I can see in #678 that the polish translation was already added, effectively closing out this issue.
@globefire if you would like to create a new Pull Request to update https://github.com/github/training-kit/blob/master/CONTRIBUTING.md by making it link to https://github.com/github/training-kit/blob/master/index.html, that would be an awesome contribution to keep these documents up to date!


suvo/uuvo/etc modifier isn't being added

ok we are not going to use the pipeline for this after all


Private supertrait pattern

This pattern is a way to retain the ability to add/remove/modify methods of a public trait without breaking compatibility. Example from byteorder: https://github.com/BurntSushi/byteorder/issues/69.

Relevant discussion in the context of Hyper: https://github.com/hyperium/hyper/issues/1194

Added in e7d7202eceea3357ff87e1026ed9000b927fb4aa.

@dtolnay curious, why we put #[doc(hidden)] on the methods of the public trait, if we can just add these methods to the private counterpart? That is, something like this should work?
I don't think that it matters much in practice, but doc(hidden) methods are still visible, and might cause ambiguous method calls in theory....


Please release

https://github.com/cyraxx/node-pogo-protos/commit/010b5465e49b363ae54195dfa167f1420d3dc820
Please release master. The above commit contains a typings bug fix. In the previous version, Envelopes.UnknownMessage was not included.

Usually node-pogo-protos follows POGOProtos releases but there hasn't been one in a while so I went ahead and published 2.7.1.

Thank you! @cyraxx


Error at startup: "Permission denied: '/dev/input/event6'"

First time installation here.
The program is (apparently) running when launching it but I've got this error:
[INFO              ] [Logger      ] Record log in /home/gus/.kivy/logs/kivy_15-07-01_8.txt
[INFO              ] [Kivy        ] v1.9.0
[INFO              ] [Python      ] v3.4.3 (default, Mar 26 2015, 22:03:40)
[GCC 4.9.2]
[INFO              ] [Factory     ] 173 symbols loaded
[INFO              ] [Image       ] Providers: img_tex, img_dds, img_gif, img_pygame (img_pil, img_ffpyplayer ignored)
[INFO              ] [Window      ] Provider: pygame(['window_egl_rpi'] ignored)
[INFO              ] [GL          ] OpenGL version <b'3.0 Mesa 10.6.0-devel'>
[INFO              ] [GL          ] OpenGL vendor <b'X.Org'>
[INFO              ] [GL          ] OpenGL renderer <b'Gallium 0.4 on AMD KABINI'>
[INFO              ] [GL          ] OpenGL parsed version: 3, 0
[INFO              ] [GL          ] Shading version <b'1.30'>
[INFO              ] [GL          ] Texture max size <16384>
[INFO              ] [GL          ] Texture max units <16>
[INFO              ] [Window      ] virtual keyboard not allowed, single mode, not docked
[INFO              ] [Text        ] Provider: pygame
[INFO              ] [VideoGstplayer] Using Gstreamer <IP_ADDRESS>
[INFO              ] [Video       ] Provider: gstplayer
[INFO              ] [OSC         ] using <multiprocessing> for socket
[INFO              ] [ProbeSysfs  ] device match: /dev/input/event6
[INFO              ] [MTD         ] Read event from </dev/input/event6>
[INFO              ] [Base        ] Start application main loop
 Exception in thread Thread-1:
 Traceback (most recent call last):
   File "/usr/lib/python3.4/threading.py", line 920, in _bootstrap_inner
     self.run()
   File "/usr/lib/python3.4/threading.py", line 868, in run
     self._target(*self._args, **self._kwargs)
   File "/home/gus/.Envs/python3_kivy/lib/python3.4/site-packages/kivy/input/providers/mtdev.py", line 197, in _thread_run
     _device = Device(_fn)
   File "/home/gus/.Envs/python3_kivy/lib/python3.4/site-packages/kivy/lib/mtdev.py", line 131, in __init__
     self._fd = os.open(filename, os.O_NONBLOCK | os.O_RDONLY)
 PermissionError: [Errno 13] Permission denied: '/dev/input/event6'

[INFO              ] [GL          ] NPOT texture support is available
[INFO              ] [Base        ] Leaving application in progress...

Okay, apparently, it's due to my touchpad on the laptop. Not an error apparently:
http://stackoverflow.com/questions/26164723/kivy-and-buildozer-permission-denied


Cannot use end-to-end encryption / client that doesn't support OMEMO.

Hello,
When i try to enable encryption, we have the message "Cannot use end-to-end encryption / client that doesn't support OMEMO" . We are using Padé & or Xabber to connect, We use Padé 1.25 version. Possibly a regression as it was working on a previous release.

OMEMO is not working with Pade right now. See https://github.com/conversejs/converse.js/issues/1524

Please retry in version 1.3 and re-open issue if it does not work.


Migrate to EventStream in precaching

Even though it's straightforward, I hadn't have time to test it.
Bug: https://phabricator.wikimedia.org/T166046

Codecov Report

Merging #200 into master will increase coverage by 0.54%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master     #200      +/-   ##
==========================================
+ Coverage   10.97%   11.51%   +0.54%
==========================================
  Files           5        5
  Lines         319      304      -15
==========================================
  Hits           35       35
+ Misses        284      269      -15

Impacted Files
Coverage Δ

ores/tests/test_util.py
0% <0%> (ø)
:arrow_up:

ores/errors.py
75.67% <0%> (+10.55%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d7f64a8...f6a9112. Read the comment docs.

This needs a real world test before merging.

Okay, I made lots of tests and made tons of fixes. It now works but server responds with "database" is not there (and "rev_id" is not there) because change schema of event stream is different than what is change propagation change scheme (ping @ottomata and @Pchelolo, Any plans to make them look alike?)
Also this needs to send user-agent.

Ya you are reading recent changes which is s different stream. We've been waiting for someone to need revision-create in the EventStreams public API before we added it in. Looks like this is it! :)

On Jun 11, 2017, at 02:43, Amir Sarabadani<EMAIL_ADDRESS>wrote:
Okay, I made lots of tests and made tons of fixes. It now works but server responds with "database" is not there (and "rev_id" is not there) because change schema of event stream is different than what is change propagation change scheme (ping @ottomata and @Pchelolo, Any plans to make them look alike?)
Also this needs to send user-agent.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

Yeah!  please :)))

See https://phabricator.wikimedia.org/T167670

tested on my local machine.  I ran this command to set up precached daemon:
./utility precached https://stream.wikimedia.org/v2/stream/revision-create http://localhost:8080


Double sorting

I would like to Double sort using bidAmount and userCount in ascending order using following schema:
const bidsFrequencySchema = new Schema({
auctionId: {
type: Number,
hashKey: true
},
bidAmount: {
type: Number,
validate: function (v) {
return validations.validBidAmount(v);
},
rangeKey: true
},
userId: {
type: Array
},
userCount: {
type: Number,
index: true
}
},
{
timestamps: true,
}
);
Below query doesnt work:
await bidsFrequencyModel.query({ auctionId: auctionId }).where('userCount, bidAmount').ascending().limit(1).exec();
Please help.

@Nikhil2508 We strongly encourage issues and PRs to follow the template that is provided when you create an issue or PR. This is because it gives us a lot of information we need in order to help. Because you did not do that is makes it a lot more difficult for us to help. The statement “Below query doesnt work” is very broad and doesn’t give any detail about why it doesn’t work. Is it throwing an error? Is it failing silently? What do you expect to be returned? What is being returned? Do you have code for your model that you can show us? Why do you expect that “where” code with two property names separated by a comma to work? Did you find that in the documentation? It’s also helpful for us to know what environment you are working in. Are you running the latest version of Dynamoose? Is this a feature request or do you have reason to believe this is a current bug in Dynamoose or just a question?
I will be closing this issue for now. But will consider reopening if the correct details and structure is followed.


Is anyone working on implementing a CARLA dataset in OpenPCDet for use of simulation data in training?

I have been trying to create an implementation for a CARLA dataset by recording the data given by this simulator.Then I have been trying to emulate the simulator data through the KITTI dataset so that I don't have to create a new implementation of this dataset in OpenPCDet.
Has anyone tried to implement a similar dataset using the CARLA simulation data in this project?

Hi, I might work one day on implementing Carla-like datasets in the future, such as KITTI-CARLA dataset the style of KITTI. I will keep you informed if I start this.

Hi, I might work one day on implementing Carla-like datasets in the future, such as KITTI-CARLA dataset the style of KITTI. I will keep you informed if I start this.

Hi, have you started the work yet? If yes, maybe we can discuss some problems with it.

@AOOOOOA Can you please tell what problems are you facing?


Implement player name and option for fullscreen windows

add optional string 'name' to Player (removes LocalPlayer or foo12345)
add option to launch the sc2 window in fullscreen
Example usage:

run_game(maps.get("BlueshiftLE"), [
        Human(Race.Zerg, fullscreen=True, name="Human"),
        Bot(Race.Zerg, WorkerRushBot(), name="WorkerRush")],
             realtime=True)

NOTE: apparently using player_name screws up laddermanager, not sure how

it messes up the Ladder Manager because they created their own function to put the bot names into the replays because that ability didn't exist at the time.    Will be change in the future, but isn't seen as a priority.     Any bots that go on the ladder should just leave it empty for now.

I implemented the name feature by hand before noticing this PR. I'll see if I can check out the per-user-fullscreen soon. Thanks for the PR, anyway.


Omnisharp fails to start

Omnisharp fails to start.
Here is the complete output (with some portion of some paths omitted and replaced with ...):
Starting OmniSharp server at 5/22/2022, 5:18:55 PM
    Target: /Useres/.../*.sln

OmniSharp server started with Mono 6.12.0.
    Path: /Users/.../.vscode/extensions/ms-dotnettools.csharp-1.24.4/.omnisharp/1.38.2/omnisharp/OmniSharp.exe
    PID: 7276

[info]: OmniSharp.Stdio.Host
        Starting OmniSharp on MacOS 12.3.1 (x64)
[info]: OmniSharp.Services.DotNetCliService
        Checking the 'DOTNET_ROOT' environment variable to find a .NET SDK
[info]: OmniSharp.Services.DotNetCliService
        Using the 'dotnet' on the PATH.
[info]: OmniSharp.Services.DotNetCliService
        DotNetPath set to dotnet
[info]: OmniSharp.MSBuild.Discovery.MSBuildLocator
        Located 2 MSBuild instance(s)
            1: Mono 16.10.1 - "/Library/Frameworks/Mono.framework/Versions/6.12.0/lib/mono/msbuild/Current/bin"
            2: StandAlone 17.1.1 - "/Users/.../.vscode/extensions/ms-dotnettools.csharp-1.24.4/.omnisharp/1.38.2/omnisharp/.msbuild/Current/Bin"
[info]: OmniSharp.MSBuild.Discovery.MSBuildLocator
        Registered MSBuild instance: Mono 16.10.1 - "/Library/Frameworks/Mono.framework/Versions/6.12.0/lib/mono/msbuild/Current/bin"
            CscToolExe = csc.exe
            CscToolPath = /Users/.../.vscode/extensions/ms-dotnettools.csharp-1.24.4/.omnisharp/1.38.2/omnisharp/.msbuild/Current/Bin/Roslyn
System.TypeLoadException: Could not load type of field 'Roslynator.Host.Mef.MefHostServices:_compositionContext' (2) due to: Could not load file or assembly 'System.Composition.Runtime, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
  at (wrapper managed-to-native) System.RuntimeType.GetPropertiesByName_native(System.RuntimeType,intptr,System.Reflection.BindingFlags,System.RuntimeType/MemberListType)
  at System.RuntimeType.GetPropertiesByName (System.String name, System.Reflection.BindingFlags bindingAttr, System.RuntimeType+MemberListType listType, System.RuntimeType reflectedType) [0x0001b] in <45c5a879a76043448b35ece5a66e4b89>:0
  at System.RuntimeType.GetPropertyCandidates (System.String name, System.Reflection.BindingFlags bindingAttr, System.Type[] types, System.Boolean allowPrefixLookup) [0x00010] in <45c5a879a76043448b35ece5a66e4b89>:0
  at System.RuntimeType.GetProperties (System.Reflection.BindingFlags bindingAttr) [0x00000] in <45c5a879a76043448b35ece5a66e4b89>:0
  at System.Reflection.RuntimeReflectionExtensions.GetRuntimeProperties (System.Type type) [0x00014] in <45c5a879a76043448b35ece5a66e4b89>:0
  at System.Composition.TypedParts.Discovery.TypeInspector+<DiscoverPropertyExports>d__7.MoveNext () [0x00037] in <2d4bd81b7fea479cb0a52f844b6edf97>:0
  at System.Composition.TypedParts.Discovery.TypeInspector+<DiscoverExports>d__5.MoveNext () [0x000db] in <2d4bd81b7fea479cb0a52f844b6edf97>:0
  at System.Composition.TypedParts.Discovery.TypeInspector.InspectTypeForPart (System.Reflection.TypeInfo type, System.Composition.TypedParts.Discovery.DiscoveredPart& part) [0x00061] in <2d4bd81b7fea479cb0a52f844b6edf97>:0
  at System.Composition.TypedParts.TypedPartExportDescriptorProvider..ctor (System.Collections.Generic.IEnumerable`1[T] types, System.Composition.Convention.AttributedModelProvider attributeContext) [0x00037] in <2d4bd81b7fea479cb0a52f844b6edf97>:0
  at System.Composition.Hosting.ContainerConfiguration.CreateContainer () [0x00042] in <2d4bd81b7fea479cb0a52f844b6edf97>:0
  at Microsoft.CodeAnalysis.Host.Mef.MefHostServices.Create (System.Collections.Generic.IEnumerable`1[T] assemblies) [0x00031] in <f78fde32b7b244ca95ad9f75f5377ae7>:0
  at OmniSharp.HostServicesAggregator.CreateHostServices () [0x00000] in <3997e19046634a39a20d03c6385fc69b>:0
  at OmniSharp.OmniSharpWorkspace..ctor (OmniSharp.HostServicesAggregator aggregator, Microsoft.Extensions.Logging.ILoggerFactory loggerFactory, OmniSharp.FileWatching.IFileSystemWatcher fileSystemWatcher) [0x00046] in <3997e19046634a39a20d03c6385fc69b>:0
  at (wrapper dynamic-method) System.Object.lambda_method(System.Runtime.CompilerServices.Closure,System.Composition.Hosting.Core.LifetimeContext,System.Composition.Hosting.Core.CompositionOperation)
  at System.Composition.TypedParts.ActivationFeatures.DisposalFeature+<>c__DisplayClass0_0.<RewriteActivator>b__0 (System.Composition.Hosting.Core.LifetimeContext c, System.Composition.Hosting.Core.CompositionOperation o) [0x00000] in <2d4bd81b7fea479cb0a52f844b6edf97>:0
  at System.Composition.Hosting.Core.LifetimeContext.GetOrCreate (System.Int32 sharingId, System.Composition.Hosting.Core.CompositionOperation operation, System.Composition.Hosting.Core.CompositeActivator creator) [0x000a8] in <db7a9e9ce8f2483abd2dba005db8e856>:0
  at System.Composition.TypedParts.ActivationFeatures.LifetimeFeature+<>c__DisplayClass1_0.<RewriteActivator>b__0 (System.Composition.Hosting.Core.LifetimeContext c, System.Composition.Hosting.Core.CompositionOperation o) [0x00011] in <2d4bd81b7fea479cb0a52f844b6edf97>:0
  at System.Composition.Hosting.Core.CompositionOperation.Run (System.Composition.Hosting.Core.LifetimeContext outermostLifetimeContext, System.Composition.Hosting.Core.CompositeActivator compositionRootActivator) [0x00022] in <db7a9e9ce8f2483abd2dba005db8e856>:0
  at System.Composition.Hosting.Core.LifetimeContext.TryGetExport (System.Composition.Hosting.Core.CompositionContract contract, System.Object& export) [0x0001d] in <db7a9e9ce8f2483abd2dba005db8e856>:0
  at System.Composition.Hosting.CompositionHost.TryGetExport (System.Composition.Hosting.Core.CompositionContract contract, System.Object& export) [0x00000] in <db7a9e9ce8f2483abd2dba005db8e856>:0
  at System.Composition.CompositionContext.GetExport (System.Composition.Hosting.Core.CompositionContract contract) [0x00000] in <9222e906419e4f47bc57ced5ad91678c>:0
  at System.Composition.CompositionContext.GetExport (System.Type exportType, System.String contractName) [0x00008] in <9222e906419e4f47bc57ced5ad91678c>:0
  at System.Composition.CompositionContext.GetExport[TExport] (System.String contractName) [0x00000] in <9222e906419e4f47bc57ced5ad91678c>:0
  at System.Composition.CompositionContext.GetExport[TExport] () [0x00000] in <9222e906419e4f47bc57ced5ad91678c>:0
  at OmniSharp.Stdio.Host.Initialize () [0x0000d] in <b65fb505403e4a9f95ed53beadc952de>:0
  at OmniSharp.Stdio.Host..ctor (System.IO.TextReader input, OmniSharp.Services.ISharedTextWriter writer, OmniSharp.IOmniSharpEnvironment environment, System.IServiceProvider serviceProvider, OmniSharp.CompositionHostBuilder compositionHostBuilder, Microsoft.Extensions.Logging.ILoggerFactory loggerFactory, System.Threading.CancellationTokenSource cancellationTokenSource) [0x0007b] in <b65fb505403e4a9f95ed53beadc952de>:0
  at OmniSharp.Stdio.Driver.Program+<>c__DisplayClass0_1.<Main>b__1 () [0x0017d] in <529aa935fa644d439a555153e68c17a3>:0
  at OmniSharp.CommandLineApplication+<>c__DisplayClass12_0.<OnExecute>b__0 () [0x0000b] in <2c738f48401c42ab8866fe9742b40aa0>:0
  at McMaster.Extensions.CommandLineUtils.CommandLineApplication+<>c__DisplayClass146_0.<OnExecute>b__0 (System.Threading.CancellationToken _) [0x00000] in <abe0fc6797a94a73931eea5cecaffd78>:0
  at McMaster.Extensions.CommandLineUtils.CommandLineApplication.ExecuteAsync (System.String[] args, System.Threading.CancellationToken cancellationToken) [0x000bf] in <abe0fc6797a94a73931eea5cecaffd78>:0
  at McMaster.Extensions.CommandLineUtils.CommandLineApplication.Execute (System.String[] args) [0x00016] in <abe0fc6797a94a73931eea5cecaffd78>:0
  at OmniSharp.CommandLineApplication.Execute (System.String[] args) [0x00081] in <2c738f48401c42ab8866fe9742b40aa0>:0
  at OmniSharp.Stdio.Driver.Program+<>c__DisplayClass0_0.<Main>b__0 () [0x00028] in <529aa935fa644d439a555153e68c17a3>:0
  at OmniSharp.HostHelpers.Start (System.Func`1[TResult] action) [0x0001c] in <2c738f48401c42ab8866fe9742b40aa0>:0
[ERROR] Error: OmniSharp server load timed out. Use the 'omnisharp.projectLoadTimeout' setting to override the default delay (one minute).

This seems to have fixed the issue.


GD project modify for AliOS update

修正AliOS升级后，工程编译报错的问题

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


DGS-612 Added new clone endpoing EnvelopeTemplatesEndpoint::clone()

Description
Added new clone() method for envelope-template/{template}/clone endpoint
Motivation and context
for selfcare usage
How has this been tested?
by running tests & script using the method execution
Screenshots (if appropriate)
Types of changes
What types of changes does your code introduce? Put an x in all the boxes that apply:

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:
Go over all the following points, and put an x in all the boxes that apply.
Please, please, please, don't send your pull request until all of the boxes are ticked. Once your pull request is created, it will trigger a build on our continuous integration server to make sure your tests and code style pass.

[x] I have read the CONTRIBUTING document.
[x] My pull request addresses exactly one patch/feature.
[x] I have created a branch for this patch/feature.
[x] Each individual commit in the pull request is meaningful.
[x] I have added tests to cover my changes.
[x] If my change requires a change to the documentation, I have updated it accordingly.

If you're unsure about any of these, don't hesitate to ask. We're here to help!

@nedvajz pls ještě přidej úpravu do Changelogu


Revert "Fix :arm64_big_sur bottle ordering."

Should fix https://github.com/Homebrew/brew/issues/9449

Review period will end on 2020-12-09 at 12:01:10 UTC.

Review period skipped due to critical label.

Confirmed locally on ARM Big Sur that it does fix the issue.


update dial and number components

Updated number and dial components, so you can hold the shift key while pressing arrow keys to change the increment/decrement value ( x 10 )

Good idea, thanks!

Sweet, thinking about adding a new component  that would be a port of the Max nslider. Is that something you would be interested in having in the library?

Hey, sorry for the late reply, that would be welcome of course! Do you thing you could achieve it without adding dependencies?


JS Key Value Pair Challenge - Valid Solution Fails

Describe the Issue
This solution code should pass but doesn't.
Also, we should update the seed code to use const instead of let.
Affected Page
https://www.freecodecamp.org/learn/javascript-algorithms-and-data-structures/basic-data-structures/add-key-value-pairs-to-javascript-objects
Your code
const newFoods = {
  bananas: 13,
  grapes: 35,
  strawberries: 27
}

Object.keys(newFoods)
  .forEach(food => foods[food] = newFoods[food]);

Expected behavior
Valid code should pass.
Screenshots
No response
System
All
Additional context
No response

I guess the regex has to be scoped to the foods object.
Object.assign doesn't work either for the same reason.
Object.assign(foods, newFoods);

Tests: https://github.com/freeCodeCamp/freeCodeCamp/blob/main/curriculum/challenges/english/02-javascript-algorithms-and-data-structures/basic-data-structures/add-key-value-pairs-to-javascript-objects.md

That sounds right, though I'm no regex wizard so I'm probably not the best person to puzzle out how to write that regex.
Though maybe it doesn't have to be that fancy - we should check that the entire object literal should be left as given, right?

we should check that the entire object literal should be left as given, right?

Correct.
The assert message should probably be changed as well as we do not care if the properties are "set using dot or bracket notation".

Do not add properties directly to the foods object.

Or something like that.

An issue with the help wanted or first timers only label is open for contribution. The first comprehensive PR created will be reviewed and merged. We typically do not assign issues to anyone other than long-time contributors.
If you would like to contribute and have not read the contributors docs, please do so here: https://contribute.freecodecamp.org/#/
If you have any issues with contributing, be sure to join us on the contributors channel, or on the contributors sub-forum

Hi jeremylt and lasjorg, thank you for pointing out this issue and discussing possible solutions :)
I am working on a draft PR to fix this issue. (First PR in the codebase!)
I am troubleshooting some problems with setting up FreeCodeCamp, to test my changes. I will mark the draft PR as 'ready for review' after I successfully run the app and test the changes.

Thank you jeremylt for reviewing the draft PR :)
I added your suggestion, fixed the regex, and tested the changes on GitPod. I have marked the PR as 'ready for review'. I left a comment on the PR about one tiny concern with the regex.


Drop down arrow in choose course

There's a drop down arrow here but not on the previous enter provider name box

Yep, that's intentional. There's a small set of options for these courses and you can click to see them all. For providers, because there's so many, that won't be the case.
We might do some refinements to this design:

if there's fewer than X courses, show them as radio buttons
use this select only when there is more than X

We might also need to show the short description, consider two courses shown:
Primary (X100)
Primary (B345)
It's not clear what the difference is.

Has this now been addressed by #104, @fofr and @laurahermionetennant92?


league/fractal ^0.19 dependency conflicts with Element API plugin

@riasvdv Just making you aware, the recent 2.2.0 release pushed an update to the league/fractal dependency, however there is a bit of  a conflict with the Element API plugin as that is currently specifying ^0.18.
  Problem 1
    - Installation request for rias/craft-scout ^2.2 -> satisfiable by rias/craft-scout[2.2.0].
    - Conclusion: remove league/fractal 0.18.0
    - Conclusion: don't install league/fractal 0.18.0
    - rias/craft-scout 2.2.0 requires league/fractal ^0.19 -> satisfiable by league/fractal[0.19, 0.19.1, 0.19.2].
    - Can only install one of: league/fractal[0.19, 0.18.0].
    - Can only install one of: league/fractal[0.19.1, 0.18.0].
    - Can only install one of: league/fractal[0.19.2, 0.18.0].
    - Installation request for league/fractal (locked at 0.18.0) -> satisfiable by league/fractal[0.18.0].

I guess one workaround is to stay on 2.1.4 temporarily

Should be fixed in 2.2.1, thanks for bringing it to my attention!

Thanks for the quick fix!


Ineligible create table DDL is sent after it become ligible by next DDL

create table t (a int);
insert int t values (1);

alter table t add primary key (a);

insert int t values (2);
insert int t values (3);

The first one create table DDL should not be sent to the downstream, but it was observed unexpectedly.

In TiCDC, when it encounters a DDL, it grabs a schemaSnapshot based on the DDL's TableID. Then, it uses the IsIneligibleTableID function from schemaSnapshot to decide if the changes for that table should be synced.
The snag is with how the schemaWrap4Owner structure currently gets the schemaSnapshot. It's calling the GetLastSnapshot function directly. So, when it's handling a "create table" DDL, it might end up with a schemaSnapshot from after an "add index" DDL, causing things to go wrong.


Can you update vcpkg install?

Because 'https://github.com/Microsoft/vcpkg/tree/master/ports/soci' install is old.

I'm busy with preparing new release. Why don't you update yourself the URL, date, checksum to point to the latest master. Anyone can submit or update vcpkg ports.

I've pushed a branch with the latest update [1] and I intend to merge it if the CI passes. This also switches SOCI to use vcpkg_from_github() so we'll be able to automatically pick up any updates in the future :)
[1] https://github.com/Microsoft/vcpkg/tree/dev/roschuma/soci

@ras0219-msft Awesome,thank you!

Merged!
To keep with standard policy, we'll wait until 4.0.0 is truly released (with a tag that won't move) to pull it in as soci. Since there will be no future maintenance of 3.x, we won't have a soci3.

@ras0219-msft Good plan. Thank you!
Meanwhile, I'm closing as addressed.

@ras0219-msft Can't find /backends/mysql/factory.cpp alter install soci.
Because I not install mysql driver in my computer?

@IronsDu https://github.com/Microsoft/vcpkg/blob/b142632afdba938a7fc7386074c5d8446e1d7ab0/ports/soci/portfile.cmake#L27-L33 Possibly, someone will be able to help. I am not able to maintain any aspects of SOCI 3.x, so I can't help. However, I will try to help with vcpkg ports for SOCI 4, once it is released. No promises though and help will be appreciated. (Porting vcpkg packages is not difficult!)
Please, move questions and issues about vcpkg to https://github.com/Microsoft/vcpkg


400 error querying for collection view

Hi @jamalex! Thank you for making notion-py. Really like it for custom automation with Notion.
Recently, I have encountered an HTTP 400 error when trying to query for a collection. A few code snippets and their outputs might highlight what's going on.
Client instantiation works perfectly fine:
from notion.client import NotionClient
from dotenv import load_dotenv
import os

load_dotenv()
client = NotionClient(token_v2=os.getenv("NOTION_V2_APIKEY"))

Getting page works well too:
page = client.get_block("https://www.notion.so/ericmjl/Biotech-Stack-Dive-Podcast-fc136cda60fb46129eff0f746fbc94bc")
page.children  # I can see the children blocks and collections.

However, getting collections and their views fail with the same error message. To save space, I have encapsulated them in a details tag.
client.get_collection(os.getenv("NEWSLETTER_TABLE_URL"))
client.get_collection_view(os.getenv("NEWSLETTER_TABLE_URL"))

---------------------------------------------------------------------------
HTTPError                                 Traceback (most recent call last)
/var/folders/lb/fzbrctwd2klcrwsg63vzqj0r0000gn/T/ipykernel_29726/2646176100.py in
----> 1 client.get_collection(os.getenv("NEWSLETTER_TABLE_URL"))
~/anaconda/envs/dspn-automation/lib/python3.9/site-packages/notion/client.py in get_collection(self, collection_id, force_refresh)
183         Retrieve an instance of Collection that maps to the collection identified by the ID passed in.
184         """
--> 185         coll = self.get_record_data(
186             "collection", collection_id, force_refresh=force_refresh
187         )
~/anaconda/envs/dspn-automation/lib/python3.9/site-packages/notion/client.py in get_record_data(self, table, id, force_refresh)
160
161     def get_record_data(self, table, id, force_refresh=False):
--> 162         return self._store.get(table, id, force_refresh=force_refresh)
163
164     def get_block(self, url_or_id, force_refresh=False):
~/anaconda/envs/dspn-automation/lib/python3.9/site-packages/notion/store.py in get(self, table, id, force_refresh)
184                 self.call_load_page_chunk(id)
185             else:
--> 186                 self.call_get_record_values(**{table: id})
187             result = self._get(table, id)
188         return result if result is not Missing else None
~/anaconda/envs/dspn-automation/lib/python3.9/site-packages/notion/store.py in call_get_record_values(self, **kwargs)
252                 )
253             )
--> 254             results = self._client.post(
255                 "getRecordValues", {"requests": requestlist}
256             ).json()["results"]
~/anaconda/envs/dspn-automation/lib/python3.9/site-packages/notion/client.py in post(self, endpoint, data)
258                 )
259             )
--> 260             raise HTTPError(
261                 response.json().get(
262                     "message", "There was an error (400) submitting the request."
HTTPError: Invalid input.

May I ask whether you might have a hypothesis as to why getting collections and their views are failing?
Update: this issue might actually be fixed by https://github.com/jamalex/notion-py/pull/334.

Encoutered the same issue today, the #334 did not fix it to me unfortunately.

Same here, #334 didn't seem to fix it.

Same issue :/

I can confirm that the fix in #334 fixes the problem. @jamalex  would you be kind enough to chime in up there to see if there is anything else that need to be done to finish up the PR?
Btw, just wanted to also add that the Python package helped me build a hacky garage sale app backed by notion. Huge kudos to you for building notion-py, it's super useful!


Incompatible use of input

Hi @marvinhagemeister thank you for your work on ivi-jsx!
I tried to use it today with ivi 0.8.0 and ivi-* 0.1.0 (core, dom, html, etc.) but it fails on input. It looks like this is because the latest release uses inputText instead of input. This is changed in the current master branch, but that is not yet released.
Any advice? Thanks in advance!

Hi again @marvinhagemeister
Now with the latest version of ivi I get the following error in the jsx plugin:
Uncaught ReferenceError: exports is not defined
 Object.defineProperty(exports, "__esModule", {
  value: true
});

To reproduce:
git clone https://github.com/foxdonut/meiosis-examples
cd meiosis-examples/examples/setup
npm i
npm start
cd ../..
python -m SimpleHTTPServer 3000
open http://localhost:3000/examples/setup/ivi/index.html

Would appreciate your help. Thank you!


little issue with new stable js-controller 3.3.15

log gets filled up with:
2021-08-07 14:16:06.289  - info: go-e.0 (6496) State value to set for "go-e.0.temperatures.tempereatureArray" has to be stringified but received type "object"

Here is the same, please fix it, thanks

Seems to be only at Versuon 3 Hardware. This HW is currently not available to test it. Bug will be on hold.

I have a V2 hardware; i think the easy fix is to replace
await queue.add(() => this.setState("temperatures.tempereatureArray",     { val: o.tma, ack: true }));
with
await queue.add(() => this.setState("temperatures.tempereatureArray",     { val: JSON.stringify(o.tma), ack: true }));
in main.js; messages are gone…

Hi and thank you Jlegen.
Currently all people have used for their Adapter settings and may be temperatures alarms methods to act on arrays. If I now will switch to an Object or to an string as you suggest above it will be a breaking change and many scripts will fail.
This bug needs a deeper analyst why the adapter is sending here a object instead of the array.
This bug will be on hold until the hardware and firmware is in place.
Kind regards

Hi, i'm not an "iobroker adapter expert", but i honestly don't think that this fix changes anything for applications apart from the annoying log records.
As i understand, without the fix js-controller is doing the type conversion automatically (as it did this before, but silently), and version 3.3 now throws the warning to encourage explicit type conversion. Which is what the fix does, imo...
Anyway, the contents of the data point (tempereatureArray) before the fix was "[22.88,21.38,21.38,21.75]", and it is now "[22.88,21.38,21.38,21.75]" after the fix. Not sure why/what apps may break now...

Same here with V2 Hardware

It's still that issue - only the mentioned state with a wrong type
I also don't have a v3, but a v2 go-e Charger

i have the same error with my new v3 go-e Charger
so how to handle that issue in future? fix from jlegen or change the loglevel of the adapter to error or update from the go-e adapter (which in my opinion is the cleanest solution)?

Duplicate of #91. Please switch it off in the settings menu.
In der letzten Version auf dem Stable kann das Schreiben des Tempera deaktiviert werden, damit erscheinen die object<<=>string Warnungen nicht mehr.


vzbuilders -> ouryahoo

I confirm that this contribution is made under the terms of the license found in the root directory of this repository's source tree and that I have the authority necessary to make this contribution on behalf of its copyright owner.

I suggest you just remove the comment


Build error on example

Being a new to MMRecord and swift, I wanted to check out MMRecord with swift. Just download the zip from github. Wanting to learn more from the MMRecordAtlassian example, I ran into build issues (using latest Xcode v6.1.1).
Can anybody please check this out and fix the example(s)?

I'm going to go ahead and close this. We added the Swift example as a reference, and to test initial functionality of MMRecord with Swift. But Swift is changing faster than MMRecord is being updated, so maintaining that example project with the latest tools is going to be a losing battle.


how to auto unlock the windows screen when using LockWorkStation api?

I use NamedPipe to pass the username,password to the credential, and call Provider.RaiseCredentialsChanged() but nothing happened,how to auto unlock the windows screen after using LockWorkStation api?

Hi, before calling RaiseCredentialsChanged make sure you're setting Provider.DefaultCredential and setting AutoLogon to true.

got it! thank you very much!


[C#Writer]: Abstract EntityType without Key produces non-compiling c# code

For example the below edmx produces non-compiling c# code.
<?xml version="1.0" encoding="utf-8"?>
<edmx:Edmx Version="4.0" xmlns:edmx="http://docs.oasis-open.org/odata/ns/edmx">
  <edmx:DataServices>
    <Schema Namespace="mscrm" xmlns="http://docs.oasis-open.org/odata/ns/edm">
      <EntityType Name="crmbaseentity" Abstract="true" />
      <EntityType Name="customerrelationship" BaseType="mscrm.crmbaseentity">
        <Key>
          <PropertyRef Name="customerrelationshipid" />
        </Key>
		<Property Name="customerrelationshipid" Type="Edm.Guid" />
		</EntityType>
		<EntityContainer Name="System">
        <EntitySet Name="customerrelationships" EntityType="mscrm.customerrelationship"/>
		</EntityContainer>
		</Schema>
  </edmx:DataServices>
</edmx:Edmx>

The collection class for 'crmbaseentity' uses the Key property to overload the index [] operator. But since the key is missing the generated code looks like below snippet and compilation fails with error - "Indexers must have at least one parameter".
public global::mscrm.IcrmbaseentityFetcher this[] {}
According to OData CSDL spec an abstract EntityType may or may not define a Key element.

I believe this is an issue in C# code generator. ODCM model has enough information to generate an appropriate code.