UBSAN issue with sanitizers

CRAN folks noted that we have another problem that we hadn't noticed on our page of checking results for nimble:
https://cran.r-project.org/web/checks/check_results_nimble.html
Memtest notes: clang-UBSAN
See: https://www.stats.ox.ac.uk/pub/bdr/memtests/clang-UBSAN/nimble-Ex.Rout
search for "runtime error" and you'll see an issue with compilation of our nfVar() example.
This post from Dirk seems useful and I will try to run his test on 0.6-1 in his docker container and see what I discover.
https://www.r-bloggers.com/running-ubsan-tests-via-clang-with-rocker/
Note that issue #228 is also revealed in the checking of nimble's examples... Not sure if CRAN will insist we fix that.

The error reported is:
RcppUtils.cpp:1044:20: runtime error: member call on address 0x60d000002018 which does not point to an object of type 'NimArrType'
0x60d000002018: note: object is of type 'NimArr<2, double>'
But NimArrType is a base class to NimArr<2, double>, and the call is to a virtual member function.  So it is complaining about a call to a base class virtual member function via a base class pointer that actually points to a derived class, as it should.  Should be ok, and the code has worked for a long time and is used a lot in the system.
It looks like a bit of work to set up Dirk’s docker image for re-creating the AddressSanitizer result.  If you get it working I could try to exploring more.
Or, can we just remove the @example (or set it to not run) and move on?  I think this is a non-issue.

I don't know how CRAN will react if we simply remove the @example.  And I'm also not sure if they will balk at the warning related to issue #228, which also shows up in the output from our examples when run under R-devel.
I suppose I could email Kurt back and ask for the configuration they are using to try to replicate this.
More info on the address sanitizer is here
https://cran.r-project.org/doc/manuals/r-release/R-exts.html#Using-Address-Sanitizer
I tried putting this in .R/Makevars:
CC = gcc -std=gnu99 -fsanitize=address -fno-omit-frame-pointer
CXX = g++ -fsanitize=address -fno-omit-frame-pointer
F77 = gfortran -fsanitize=address
FC = gfortran -fsanitize=address
But now I get this error with the nfVar example instead:

 Cnf2 <- compileNimble(nf2)

compiling... this may take a minute. Use nimbleOptions(showCompilerOutput = TRUE) to see C++ compiler details.
Error in dyn.load(getSOName(name, dirName), local = TRUE) :
unable to load shared object '/tmp/RtmpDAEwhh/nimble_generatedCode/P_1_nfRefClass_R_GlobalEnv8109_12_19_39_52.so':
/tmp/RtmpDAEwhh/nimble_generatedCode/P_1_nfRefClass_R_GlobalEnv8109_12_19_39_52.so: undefined symbol: __asan_report_load8

 Cnf2$run()

More generally when I add the "-fsanitize=undefined" (this seems to correspond to the UBSAN checking) I also get an error in loading our .so's (the nfVar example or more generally in trying to compileNimble or even load the nimble package) with a similar error to the above:
Error in dyn.load(file, DLLpath = DLLpath, ...) :
unable to load shared object '/tmp/nim061/nimble/libs/nimble.so':
/tmp/nim061/nimble/libs/nimble.so: undefined symbol: __ubsan_handle_type_mismatch
I can't figure out Dirk's docker container. It seems to be advertized as using clang, but when I run it, it seems to use gcc. Plus it doesn't seem to be doing the address/ubsan checking - NIMBLE installs fine and the nfVar example is fine.

The undefined symbols might be because R has to be built with certain -f flags related to the address/ubsan stuff. I'll monkey around with that a bit more.

Hey @paciorek , just to double check, you are using RD to run the devel version of R on Dirk's ubsan container, and not R (which just runs the R version from r-base, not the devel version built with UBSAN)?  And are you using the check.r script from littler?
(apologies for the out-of-the-blue comment, I don't normally pay attention to any nimble issues but I do work with Dirk on rocker images; though admittedly ubsan containers are entirely his baby)

I don't know how CRAN will react if we simply remove the @example
https://github.com/example.

My experience thus far with CRAN is that they will not care: there are
simply way too many R-packages coming through CRAN at any given moment that
they cannot track anything other than what the latest automated warnings
are. Perhaps Duncan's involvement in the project may change things, but
basically they check that R CMD check --as-cran under R-devel has no errors
or warnings (on all systems) and run it through valgrind. No flags no
problem.
I've had a surprisingly high amount of support from them when there is a
problem though (which is extra incentive to not have any problems if you
have any respect for CRAN).

And I'm also not sure if they will balk at the warning related to issue
#228 https://github.com/nimble-dev/nimble/issues/228, which also shows up
in the output from our examples when run under R-devel.

Under normal circumstances, they certainly would.
On Tue, Sep 13, 2016 at 8:30 AM, Christopher Paciorek <
<EMAIL_ADDRESS>wrote:

The undefined symbols might be because R has to be built with certain -f
flags related to the address/ubsan stuff. I'll monkey around with that a
bit more.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/nimble-dev/nimble/issues/229#issuecomment-246721026,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AF64k8okEVtXh6pAUaUq9W71rGcQ-F_8ks5qpsGbgaJpZM4J6-Ip
.

The undefined symbol looks like something from the addressSanitizer that is getting lost in nimble's on-the-fly compilation.
I am trying to create, outside of R, a boiled down case of the class hierarchy and compilation scheme involved and see if I can re-create it that way.
@paciorek if you want to keep pushing on re-creating CRAN's result, I suspect we need those flags in src/Makevars and inst/make/Makevars.in, so they will be picked up in both package building and on-the-fly compilation.
Thanks @pistacliffcho.  I think it is worth giving this some further attention but then planning to just remove it from the example in the Rd.  We can include a note to CRAN.

So here is what I suspect is going on:
It may be an issue created by nimble.so and libnimble.a having some of the same code compiled into them, which is for simplicity in our build process but does create some duplication in binary libraries.  I think a pointer to an object of a derived class (NimArr<2, double>) created in code linked against libnimble.a is then being passed as a void* (since R external pointers end up cast to void*) into a function in nimble.so (not libnimble.a) that casts it back to something meaningful and uses it.  But it casts it to a base class pointer, and everything should be completely identical to what is in libnimble.a, but it may not be recognized by the address sanitizer as part of the same class hierarchy since they are compiled twice and appear in binary libraries redundantly.  We have never had a problem from this.
This guess at what's happening could be wrong.  If it's right, it involves some of the same C++ code organization issues around the CRAN system that we've had challenges with before.  I'll think about if there are easy workarounds, but just dropping the example still seems like a good path.

Thanks, @cboettig !  I hadn't realized RD was there and was very confused about the container being for clang+Rdevel but not seeing R-devel with clang there.
I'm still not sure how to run check.r such that it uses RD rather than R. Can you clarify?

@pistacliffcho when you say run through valgrind, I'm not sure how that is related to the memtest UBSAN  thing. Do you know how they do the valgrind checking?

I wasn't specifically referring to memtest UBSAN, but just in general the
testing that they appear to do.
My understanding of they do is just something like this:
R -d valgrind --vanilla < mypkg-Ex.R
R -d "valgrind --tool=memcheck --leak-check=full" --vanilla < mypkg-Ex.R
(example lifted from Writing R Extensions 4.3.2). I had a conversation with
Ripley in which he looked through the valgrind output produced by the code
above. I don't think they run it all the time, but it does seem to be run
some times. Interestingly, he found an out of bounds error that would never
be a problem; basically something like:
if(c_vector[i] > 0 && i < c_vector.size() ) {...}
// error is that you could read past the last element, but doing so always
produces false!
I'm not a valgrind expert at all, but I have heard that it runs much
better on Linux than Mac OS. And the output from the code above is much
messier on my mac than what Ripley sends me, so I suspect it is less of a
headache to use Linux and valgrind.
On Tue, Sep 13, 2016 at 11:42 AM, Christopher Paciorek <
<EMAIL_ADDRESS>wrote:

@pistacliffcho https://github.com/pistacliffcho when you say run
through valgrind, I'm not sure how that is related to the memtest UBSAN
thing. Do you know how they do the valgrind checking?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/nimble-dev/nimble/issues/229#issuecomment-246781660,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AF64k68cU-AjW85uszIrl4vKukox2u3kks5qpu6ggaJpZM4J6-Ip
.

I asked Kurt H for more info on reproducing how CRAN is generating the
memtest-UBSAN output as I can build R-devel with clang with the
-fsanitize=undefined but not with -fsanitize=address so I can't replicate
the error CRAN is seeing.
Let's see what I hear and then make a decision about simply removing ?nfVar.
On Tue, Sep 13, 2016 at 1:40 PM, perrydv<EMAIL_ADDRESS>wrote:

I haven't proven if it's the double compilation, but it's still my best
guess, and I have confirmed that a derived object created from one
compilation is being handled in code from the other, which is as expected.
If we want to address this underlying issue, I think it would take some
careful following of threads about which functions are called in which
contexts and could be separated into only libnimble and kept out of
nimble.so and called via R's dll information so that when multiple projects
are compiled we only use the utility functions built (redundantly) into
that compilation unit. This has been on the to-do list for a while and
would be a good idea, but since things haven't been breaking, it hasn't
been the top priority.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/nimble-dev/nimble/issues/229#issuecomment-246816560,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACYr3nFTSZVkuN4AmRpzNrucOfxpMeg_ks5qpwo6gaJpZM4J6-Ip
.

I think I've fixed the situation for the particular cpp function in question.  Questions are: Am I right about the underlying problem, since we haven't been able to reproduce it in the first place?  And how much pre-emptive modifications of other functions in similar compilation situations is worth it?  I'm going to start looking at some other cases.

Ok, hopefully we get more info from Kurt and then we can reproduce the original error and see if this fixes it. If we don't get more info or still can't reproduce the original error, then I think we can say we've done our due diligence with the -fsanitize=undefined and just resubmit.

here's the configuration info from Kurt, so it's basically what I've been trying but I was using an older clang.  clang-3.9 is not available via an ubuntu package so I may try to test with clang-3.8 instead.
Perry, what branch has your fix to the cpp function used in the nfVar example?
====== from Kurt H. ========================
On Debian testing I configure with
CC="clang-3.9 -fsanitize=undefined,address -fno-sanitize=float-divide-by-zero,vptr -fno-omit-frame-pointer"
and similarly for CXX.  Not sure about clang-3.5, in general you'd want
gcc-6 and clang-3.9 if possible for ASAN and UBSAN diagnostics.
The .Rout file at
https://www.stats.ox.ac.uk/pub/bdr/memtests/clang-UBSAN/nimble-Ex.Rout
is obtained by running the examples with a *SAN enhanced build, which
apparently finds
RcppUtils.cpp:1044:20: runtime error: member call on address 0x60d000002018 which does not point to an object of type 'NimArrType'
0x60d000002018: note: object is of type 'NimArr<2, double>'
00 40 8f 40  a0 66 e8 ba 73 7f 00 00  02 00 00 00 be be be be  00 00 00 00 00 00 00 00  28 20 00 00
^~~~~~~~~~~~~~~~~~~~~~~
vptr for 'NimArr<2, double>'
SUMMARY: AddressSanitizer: undefined-behavior RcppUtils.cpp:1044:20 in

compilation_overlaps
At the moment it has some other changes and it outputs some diagnostic messages.  I've made more progress isolating use of some functions just to nimble.so or libnimble.  Partly this is pre-emptive against further addressSanitizer risks, and partly it is because it is a good idea anyway.

For future ref, here's what seems to work to build R-devel with *SAN support:
./configure --prefix=/usr/local/R-devel_2016-09-12-clang38 'CC=clang-3.8 -fsanitize=undefined,address -fno-sanitize=float-divide-by-zero,vptr  -fno-omit-frame-pointer' 'MAIN_LDFLAGS=-fsanitize=undefined,address -fno-sanitize=float-divide-by-zero,vptr  -fno-omit-frame-pointer' 'CFLAGS=-pipe -std=gnu99 -Wall -pedantic -g' 'F77=gfortran' 'FFLAGS=-pipe -Wall -pedantic -g' 'CXX=clang++-3.8 -fsanitize=undefined,address -fno-sanitize=float-divide-by-zero,vptr -fno-omit-frame-pointer' 'CXXFLAGS=-pipe -Wall -pedantic -g' 'CXX1X=clang++-3.8 -fsanitize=undefined,address -fno-sanitize=float-divide-by-zero,vptr -fno-omit-frame-pointer' 'FC=gfortran' 'FCFLAGS=-pipe -Wall -pedantic -g' --with-cairo --with-jpeglib --with-readline --with-tcltk --with-lapack --enable-R-profiling --enable-R-shlib
Also note to self - need to turn vm.overcommit_memory to 2 on my Linux box or building R will fail because conftest will not run as ASAN tries to allocate a ridiculous amount of memory.

I'm getting this when I run R CMD check (without the *SAN business) on nimble_0.6-1.tar.gz from compilation_overlaps.

checking foreign function calls ... NOTE
Registration problems:
Evaluating ‘getNativeSymbolInfo("setDoublePtrFromSinglePtr", dll)’ during check gives error
‘object 'dll' not found’:
.Call(getNativeSymbolInfo("setDoublePtrFromSinglePtr", dll),
elementPtr, value)
Evaluating ‘getNativeSymbolInfo("SEXP_2_Nim", dll)’ during check gives error
‘object 'dll' not found’:
.Call(getNativeSymbolInfo("SEXP_2_Nim", dll), elementPtr, as.integer(pointDepth),
values, allowResize)
Evaluating ‘getNativeSymbolInfo("setOnePtrVectorOfPtrs", dll)’ during check gives error
‘object 'dll' not found’:
.Call(getNativeSymbolInfo("setOnePtrVectorOfPtrs", dll), accessorPtr,
as.integer(i - 1), contentsPtr)
Evaluating ‘getNativeSymbolInfo("setPtrVectorOfPtrs", dll)’ during check gives error
‘object 'dll' not found’:
.Call(getNativeSymbolInfo("setPtrVectorOfPtrs", dll), accessorPtr,
contentsPtr, as.integer(length))

Not sure if compilation_overlaps is obsolete - I was hoping to test just the nfVar fix...

Ok, well, I have R-devel built with clang-3.8 with ASAN and UBSAN checking working in Ubuntu but it's not reproducing the clang-UBSAN error for nfVar reported by CRAN.  Frustrating. Perhaps because CRAN may be using clang-3.9 but that feels a bit unlikely.

R-devel CMD check on compilation_overlaps with ASAN/UBSAN checking is giving the following.
I'm also filing a separate note on R-devel CMD check on compilation_overlaps3.
makeNimbleFxnCppCopyTypes: Error while checking: C stack usage 8011696
is too close to the limit
=================================================================
==13564==ERROR: LeakSanitizer: detected memory leaks
Direct leak of 1368 byte(s) in 3 object(s) allocated from:
#0 0x4c3bc8  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/bin/exec/R+0x4c3bc8)
#1 0x7fc447c38ca4  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/modules//internet.so+0x72ca4)
#2 0x7fc452896bed  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xdd6bed)
#3 0x7fc4523b207e  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0x8f207e)
#4 0x7fc4526dfa19  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xc1fa19)
#5 0x7fc4526afcee  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xbefcee)
...
Direct leak of 912 byte(s) in 2 object(s) allocated from:
#0 0x4c3bc8  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/bin/exec/R+0x4c3bc8)
#1 0x7fc447c2c6e4  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/modules//internet.so+0x666e4)
#2 0x7fc452893eba  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xdd3eba)
#3 0x7fc4523b2117  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0x8f2117)
#4 0x7fc4526dfa19  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xc1fa19)
#5 0x7fc4526afcee  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xbefcee)
#6 0x7fc45276f30c  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xcaf30c)
...
SUMMARY: AddressSanitizer: 3192 byte(s) leaked in 7 allocation(s).

here's what I get running R-devel with clang and *SAN checking on compilation_overlaps3:

checking for portable file names ... OK
checking for sufficient/correct file permissions ... OK
checking whether package ‘nimble’ can be installed ... ERROR
Installation failed.
See ‘/var/tmp/paciorek/nimble-dev/nimble/packages/nimble.Rcheck/00install.out’ for details.
DONE

Status: 1 ERROR, 1 NOTE
See
‘/var/tmp/paciorek/nimble-dev/nimble/packages/nimble.Rcheck/00check.log’
for details.
=================================================================
==10005==ERROR: LeakSanitizer: detected memory leaks
Direct leak of 1368 byte(s) in 3 object(s) allocated from:
#0 0x4c3bc8  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/bin/exec/R+0x4c3bc8)
#1 0x7f4e07848ca4  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/modules//internet.so+0x72ca4)
#2 0x7f4e124abbed  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xdd6bed)
#3 0x7f4e11fc707e  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0x8f207e)
#4 0x7f4e122f4a19  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xc1fa19)
#5 0x7f4e122c4cee  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xbefcee)
#6 0x7f4e1238430c  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xcaf30c)
#7 0x7f4e122f29e4  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xc1d9e4)
#8 0x7f4e122c4cee  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xbefcee)
#9 0x7f4e1237d123  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xca8123)
#10 0x7f4e123cd1e2  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xcf81e2)
#11 0x7f4e123cd702  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xcf8702)
#12 0x7f4e122db496  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xc06496)
#13 0x7f4e122c4cee  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xbefcee)
#14 0x7f4e1238430c  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xcaf30c)
#15 0x7f4e122f29e4  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xc1d9e4)
#16 0x7f4e122c4cee  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xbefcee)
#17 0x7f4e1238430c  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xcaf30c)
#18 0x7f4e123897e3  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xcb47e3)
#19 0x7f4e11d6c94d  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0x69794d)
#20 0x7f4e1257ac04  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xea5c04)
#21 0x7f4e122f5e63  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xc20e63)
#22 0x7f4e122c4cee  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xbefcee)
#23 0x7f4e1238430c  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xcaf30c)
#24 0x7f4e122f29e4  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xc1d9e4)
#25 0x7f4e122c4cee  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xbefcee)
#26 0x7f4e1237d123  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xca8123)
#27 0x7f4e123cd1e2  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xcf81e2)
#28 0x7f4e123cd702  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xcf8702)
#29 0x7f4e122db496  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xc06496)
Direct leak of 912 byte(s) in 2 object(s) allocated from:
#0 0x4c3bc8  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/bin/exec/R+0x4c3bc8)
#1 0x7f4e0783c6e4  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/modules//internet.so+0x666e4)
#2 0x7f4e124a8eba  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xdd3eba)
#3 0x7f4e11fc7117  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0x8f2117)
#4 0x7f4e122f4a19  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xc1fa19)
#5 0x7f4e122c4cee  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xbefcee)
#6 0x7f4e1238430c  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xcaf30c)
#7 0x7f4e122f29e4  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xc1d9e4)
#8 0x7f4e122c4cee  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xbefcee)
#9 0x7f4e1237d123  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xca8123)
#10 0x7f4e123cd1e2  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xcf81e2)
#11 0x7f4e123cd702  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xcf8702)
#12 0x7f4e122db496  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xc06496)
#13 0x7f4e122c4cee  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xbefcee)
#14 0x7f4e1238430c  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xcaf30c)
#15 0x7f4e122f29e4  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xc1d9e4)
#16 0x7f4e122c4cee  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xbefcee)
#17 0x7f4e1238430c  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xcaf30c)
#18 0x7f4e122f29e4  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xc1d9e4)
#19 0x7f4e122c4cee  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xbefcee)
#20 0x7f4e1238430c  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xcaf30c)
#21 0x7f4e122f29e4  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xc1d9e4)
#22 0x7f4e122c4cee  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xbefcee)
#23 0x7f4e1238430c  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xcaf30c)
#24 0x7f4e122f29e4  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xc1d9e4)
#25 0x7f4e122c4cee  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xbefcee)
#26 0x7f4e1238430c  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xcaf30c)
#27 0x7f4e122f29e4  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xc1d9e4)
#28 0x7f4e122c4cee  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xbefcee)
#29 0x7f4e1238430c  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xcaf30c)
Direct leak of 456 byte(s) in 1 object(s) allocated from:
#0 0x4c3bc8  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/bin/exec/R+0x4c3bc8)
#1 0x7f4e0783c6e4  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/modules//internet.so+0x666e4)
#2 0x7f4e124a8eba  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xdd3eba)
#3 0x7f4e11fc7117  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0x8f2117)
#4 0x7f4e122f4a19  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xc1fa19)
#5 0x7f4e122c4cee  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xbefcee)
#6 0x7f4e1238430c  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xcaf30c)
#7 0x7f4e122f29e4  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xc1d9e4)
#8 0x7f4e122c4cee  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xbefcee)
#9 0x7f4e1237d123  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xca8123)
#10 0x7f4e123cd1e2  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xcf81e2)
#11 0x7f4e123cd702  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xcf8702)
#12 0x7f4e122db496  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xc06496)
#13 0x7f4e122c4cee  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xbefcee)
#14 0x7f4e1238430c  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xcaf30c)
#15 0x7f4e122f29e4  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xc1d9e4)
#16 0x7f4e122c4cee  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xbefcee)
#17 0x7f4e1238430c  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xcaf30c)
#18 0x7f4e123897e3  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xcb47e3)
#19 0x7f4e11d6c94d  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0x69794d)
#20 0x7f4e1257ac04  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xea5c04)
#21 0x7f4e122f5e63  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xc20e63)
#22 0x7f4e122c4cee  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xbefcee)
#23 0x7f4e1238430c  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xcaf30c)
#24 0x7f4e122f29e4  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xc1d9e4)
#25 0x7f4e122c4cee  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xbefcee)
#26 0x7f4e1237d123  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xca8123)
#27 0x7f4e123cd1e2  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xcf81e2)
#28 0x7f4e123cd702  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xcf8702)
#29 0x7f4e122db496  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xc06496)
Direct leak of 456 byte(s) in 1 object(s) allocated from:
#0 0x4c3bc8  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/bin/exec/R+0x4c3bc8)
#1 0x7f4e0783c6e4  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/modules//internet.so+0x666e4)
#2 0x7f4e124a8eba  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xdd3eba)
#3 0x7f4e11fc7117  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0x8f2117)
#4 0x7f4e122f4a19  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xc1fa19)
#5 0x7f4e122c4cee  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xbefcee)
#6 0x7f4e1238430c  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xcaf30c)
#7 0x7f4e122f29e4  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xc1d9e4)
#8 0x7f4e122c4cee  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xbefcee)
#9 0x7f4e1237d123  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xca8123)
#10 0x7f4e123cd1e2  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xcf81e2)
#11 0x7f4e123cd702  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xcf8702)
#12 0x7f4e122db496  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xc06496)
#13 0x7f4e122c4cee  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xbefcee)
#14 0x7f4e1238430c  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xcaf30c)
#15 0x7f4e122f29e4  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xc1d9e4)
#16 0x7f4e122c4cee  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xbefcee)
#17 0x7f4e1238430c  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xcaf30c)
#18 0x7f4e122f29e4  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xc1d9e4)
#19 0x7f4e122c4cee  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xbefcee)
#20 0x7f4e1237d123  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xca8123)
#21 0x7f4e123cd1e2  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xcf81e2)
#22 0x7f4e123cd702  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xcf8702)
#23 0x7f4e122db496  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xc06496)
#24 0x7f4e122c4cee  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xbefcee)
#25 0x7f4e1238430c  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xcaf30c)
#26 0x7f4e122f29e4  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xc1d9e4)
#27 0x7f4e122c4cee  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xbefcee)
#28 0x7f4e1238430c  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xcaf30c)
#29 0x7f4e122f29e4  (/usr/local/R-devel_2016-09-12-clang38/lib64/R/lib/libR.so+0xc1d9e4)
SUMMARY: AddressSanitizer: 3192 byte(s) leaked in 7 allocation(s).
==========/var/tmp/paciorek/nimble-dev/nimble/packages/nimble.Rcheck/00install.out===========

installing source package ‘nimble’ ...
checking for g++... g++
checking whether the C++ compiler works... yes
checking for C++ compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C++ compiler... yes
checking whether g++ accepts -g... yes
working on eigen
checking how to run the C++ preprocessor... clang++-3.8 -fsanitize=undefined,address -fno-sanitize=float-divide-by-zero,vptr -
fno-omit-frame-pointer -E
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking Eigen/Dense usability... no
checking Eigen/Dense presence... no
checking for Eigen/Dense... no
Using the version of Eigen provided with NIMBLE
NIMBLE_DIR = /var/tmp/paciorek/nimble-dev/nimble/packages/nimble.Rcheck/nimble
ENABLE_LIB=true
configure: creating ./config.status
config.status: creating inst/make/Makevars
config.status: creating inst/make/Makevars_lib
config.status: creating R/config.R
config.status: creating src/Makevars
Building libnimble
clang++-3.8 -fsanitize=undefined,address -fno-sanitize=float-divide-by-zero,vptr -fno-omit-frame-pointer  -I/usr/local/R-devel_2016-09-12-clang38/lib64/R/include -DNDEBUG -I"../include" -I/usr/local/include    -fpic  -pipe -Wall -pedantic -g  -c RcppUtils.cpp -o RcppUtils.o
clang++-3.8 -fsanitize=undefined,address -fno-sanitize=float-divide-by-zero,vptr -fno-omit-frame-pointer  -I/usr/local/R-devel_2016-09-12-clang38/lib64/R/include -DNDEBUG -I"../include" -I/usr/local/include    -fpic  -pipe -Wall -pedantic -g  -c RcppNimbleUtils.cpp -o RcppNimbleUtils.o
clang++-3.8 -fsanitize=undefined,address -fno-sanitize=float-divide-by-zero,vptr -fno-omit-frame-pointer  -I/usr/local/R-devel_2016-09-12-clang38/lib64/R/include -DNDEBUG -I"../include" -I/usr/local/include    -fpic  -pipe -Wall -pedantic -g  -c Utils.cpp -o Utils.o
clang++-3.8 -fsanitize=undefined,address -fno-sanitize=float-divide-by-zero,vptr -fno-omit-frame-pointer  -I/usr/local/R-devel_2016-09-12-clang38/lib64/R/include -DNDEBUG -I"../include" -I/usr/local/include    -fpic  -pipe -Wall -pedantic -g  -c NamedObjects.cpp -o NamedObjects.o
clang++-3.8 -fsanitize=undefined,address -fno-sanitize=float-divide-by-zero,vptr -fno-omit-frame-pointer  -I/usr/local/R-devel_2016-09-12-clang38/lib64/R/include -DNDEBUG -I"../include" -I/usr/local/include    -fpic  -pipe -Wall -pedantic -g  -c ModelClassUtils.cpp -o ModelClassUtils.o
clang++-3.8 -fsanitize=undefined,address -fno-sanitize=float-divide-by-zero,vptr -fno-omit-frame-pointer  -I/usr/local/R-devel_2016-09-12-clang38/lib64/R/include -DNDEBUG -I"../include" -I/usr/local/include    -fpic  -pipe -Wall -pedantic -g  -c accessorClasses.cpp -o accessorClasses.o
clang++-3.8 -fsanitize=undefined,address -fno-sanitize=float-divide-by-zero,vptr -fno-omit-frame-pointer  -I/usr/local/R-devel_2016-09-12-clang38/lib64/R/include -DNDEBUG -I"../include" -I/usr/local/include    -fpic  -pipe -Wall -pedantic -g  -c accessorClasses.cpp -o accessorClasses.o
clang++-3.8 -fsanitize=undefined,address -fno-sanitize=float-divide-by-zero,vptr -fno-omit-frame-pointer  -I/usr/local/R-devel_2016-09-12-clang38/lib64/R/include -DNDEBUG -I"../include" -I/usr/local/include    -fpic  -pipe -Wall -pedantic -g  -c dists.cpp -o dists.o
clang++-3.8 -fsanitize=undefined,address -fno-sanitize=float-divide-by-zero,vptr -fno-omit-frame-pointer  -I/usr/local/R-devel_2016-09-12-clang38/lib64/R/include -DNDEBUG -I"../include" -I/usr/local/include    -fpic  -pipe -Wall -pedantic -g  -c nimDists.cpp -o nimDists.o
clang++-3.8 -fsanitize=undefined,address -fno-sanitize=float-divide-by-zero,vptr -fno-omit-frame-pointer  -I/usr/local/R-devel_2016-09-12-clang38/lib64/R/include -DNDEBUG -I"../include" -I/usr/local/include    -fpic  -pipe -Wall -pedantic -g  -c nimbleGraph.cpp -o nimbleGraph.o
linking libnimble.so
Linux
clang++-3.8 -fsanitize=undefined,address -fno-sanitize=float-divide-by-zero,vptr -fno-omit-frame-pointer  -shared -o libnimble.so  RcppUtils.o RcppNimbleUtils.o Utils.o NamedObjects.o ModelClassUtils.o accessorClasses.o dists.o nimDists.o nimbleGraph.o
accessorClasses.o: In function __cxx_global_var_init.13': /var/tmp/paciorek/nimble-dev/nimble/packages/nimble.Rcheck/00_pkg_src/nimble/inst/CppCode/accessorClasses.cpp:21: multiple definition of _NIMBLE_WHITESPACE'
RcppUtils.o:/var/tmp/paciorek/nimble-dev/nimble/packages/nimble.Rcheck/00_pkg_src/nimble/inst/CppCode/RcppUtils.cpp:10: first defined here
accessorClasses.o: In function __cxx_global_var_init.14': /var/tmp/paciorek/nimble-dev/nimble/packages/nimble.Rcheck/00_pkg_src/nimble/inst/CppCode/accessorClasses.cpp:21: multiple definition of _NIMBLE_WHITESPACEBRACKET'
RcppUtils.o:/var/tmp/paciorek/nimble-dev/nimble/packages/nimble.Rcheck/00_pkg_src/nimble/inst/CppCode/RcppUtils.cpp:10: first defined here
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [libnimble.so] Error 1
ERROR: configuration failed for package ‘nimble’
removing ‘/var/tmp/paciorek/nimble-dev/nimble/packages/nimble.Rcheck/nimble’

=====================

No, compiler_overlaps3 isn't supposed to work yet.
I'm not sure what the status of compiler_overlaps was for the version you had.
I suspect all the warnings of memory leaks from the sanitizer checks are for similar reasons that I suspect (but we can't confirm) for the first address sanitizer warning we saw:  In some cases we have relied on duplicate compilations of exactly the same code and counted on the ability to instantiate an object from one compilation unit and then pass a pointer to it to a function in the other compilation unit and have that function use the pointed-to class correctly because both were compiled identically.  It's annoying to do it this way but we've never had a problem.  It was essentially a workaround to the difficulty of building and distributing a so/dll via CRAN that our on-the-fly compilation could link to.
The message about the C stack usage being too close to the limit is from a pure R function that is not doing anything with our compiled code, so on first glance it would seem to involve R itself and not our compiled code.
Does CRAN actually run the LeakSanitizer?  That output does not easily reveal what is at issue, and I have no way to know if it can see R and objects from our compiled code interacting, such as if R has an external pointer to an object from our compiled code.

Yeah, not clear on what CRAN runs (Kurt indicated both ASAN and UBSAN but the naming of the testing that we failed was clang-UBSAN).
And if they do run LeakSanitizer, which seems to be the case, why didn't our submission fail based on that... and why can't we reproduce the nfVar error... arggh.

I did things more carefully and verified that with UBSAN (i.e., no -fsanitize=address) then R CMD check with R-devel and clang-3.9 does not give any memory leak issues, so that suggests that is what CRAN is running.
That said, I can't get the nfVar error to show up. I've just been running the nfVar example directly. I wonder if somehow running all the examples together would do it, but I presume that happens in R CMD check, and we're not seeing an error report from there.

Could it have to do with the environment in which the examples are actually run during building of the Rd pages?

On Sep 16, 2016, at 3:18 PM, Christopher Paciorek<EMAIL_ADDRESS>wrote:
I did things more carefully and verified that with UBSAN (i.e., no -fsanitize=address) then R CMD check with R-devel and clang-3.9 does not give any memory leak issues, so that suggests that is what CRAN is running.
That said, I can't get the nfVar error to show up. I've just been running the nfVar example directly. I wonder if somehow running all the examples together would do it, but I presume that happens in R CMD check, and we're not seeing an error report from there.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub https://github.com/nimble-dev/nimble/issues/229#issuecomment-247721889, or mute the thread https://github.com/notifications/unsubscribe-auth/AHpCMdCHhIAnMgBB8o7tinNxr0y13JvCks5qqxWigaJpZM4J6-Ip.

my running of R CMD check creates nimble.Rcheck/nimble-Ex.Rout that has the same format as we see in the clang-UBSAN file posted by CRAN, so that suggests it is being created as part of R CMD check.

branch compilation_overlaps3 is now ready to be tested.  specifically @paciorek please check it with your UBSAN setup.
all other testing of people's code is also welcome on this branch.  the changes represent an overdue cleanup and reorganization of which compilation unit has which built-in interface function and how the .Call()s in the R code look up the compiled functions they need.  I've run through all our testing, so the remaining risk would be any cases not represented in the test suite.
A new module for the test suite could systematically check every feature of compilation and ensuing build-in interface calls, but that will be a separate issue.  By every feature of compilation, I don't mean every keyword but rather every use of every interface functions via the situations in which they are needed.

I'm seeing the following when running R CMD check using ASAN+UBSAN.
Note that all the address/memory stuff showing up when checking examples is coming from teh ASAN checking.
www.stat.berkeley.edu/~paciorek/transfer/00check.log
Also the following was printed after the output of R CMD check:
==4029==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 1368 byte(s) in 3 object(s) allocated from:
    #0 0x4c923c  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0x4c923c)
    #1 0x7f230d1bcc44  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/modules//internet.so+0x72c44)
    #2 0xcb76c6  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xcb76c6)
    #3 0x7b23db  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0x7b23db)
    #4 0xaf468c  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xaf468c)
    #5 0xac352a  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xac352a)
    #6 0xb884ab  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xb884ab)
    #7 0xaf2551  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xaf2551)
    #8 0xac352a  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xac352a)
    #9 0xb80f49  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xb80f49)
    #10 0xbd3596  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xbd3596)
    #11 0xbd3ad4  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xbd3ad4)
    #12 0xada54f  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xada54f)
    #13 0xac352a  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xac352a)
    #14 0xb884ab  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xb884ab)
    #15 0xaf2551  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xaf2551)
    #16 0xac352a  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xac352a)
    #17 0xb884ab  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xb884ab)
    #18 0xb8dc90  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xb8dc90)
    #19 0x5464e3  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0x5464e3)
    #20 0xd946ab  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xd946ab)
    #21 0xaf5b61  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xaf5b61)
    #22 0xac352a  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xac352a)
    #23 0xb884ab  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xb884ab)
    #24 0xaf2551  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xaf2551)
    #25 0xac352a  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xac352a)
    #26 0xb80f49  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xb80f49)
    #27 0xbd3596  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xbd3596)
    #28 0xbd3ad4  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xbd3ad4)
    #29 0xada54f  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xada54f)

Direct leak of 912 byte(s) in 2 object(s) allocated from:
    #0 0x4c923c  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0x4c923c)
    #1 0x7f230d1b0684  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/modules//internet.so+0x66684)
    #2 0xcb4873  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xcb4873)
    #3 0x7b247a  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0x7b247a)
    #4 0xaf468c  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xaf468c)
    #5 0xac352a  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xac352a)
    #6 0xb884ab  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xb884ab)
    #7 0xaf2551  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xaf2551)
    #8 0xac352a  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xac352a)
    #9 0xb80f49  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xb80f49)
    #10 0xbd3596  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xbd3596)
    #11 0xbd3ad4  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xbd3ad4)
    #12 0xada54f  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xada54f)
    #13 0xac352a  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xac352a)
    #14 0xb884ab  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xb884ab)
    #15 0xaf2551  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xaf2551)
    #16 0xac352a  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xac352a)
    #17 0xb884ab  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xb884ab)
    #18 0xaf2551  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xaf2551)
    #19 0xac352a  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xac352a)
    #20 0xb884ab  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xb884ab)
    #21 0xaf2551  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xaf2551)
    #22 0xac352a  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xac352a)
    #23 0xb884ab  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xb884ab)
    #24 0xaf2551  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xaf2551)
    #25 0xac352a  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xac352a)
    #26 0xb884ab  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xb884ab)
    #27 0xaf2551  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xaf2551)
    #28 0xac352a  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xac352a)
    #29 0xb884ab  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xb884ab)

Direct leak of 456 byte(s) in 1 object(s) allocated from:
    #0 0x4c923c  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0x4c923c)
    #1 0x7f230d1b0684  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/modules//internet.so+0x66684)
    #2 0xcb4873  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xcb4873)
    #3 0x7b247a  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0x7b247a)
    #4 0xaf468c  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xaf468c)
    #5 0xac352a  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xac352a)
    #6 0xb884ab  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xb884ab)
    #7 0xaf2551  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xaf2551)
    #8 0xac352a  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xac352a)
    #9 0xb80f49  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xb80f49)
    #10 0xbd3596  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xbd3596)
    #11 0xbd3ad4  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xbd3ad4)
    #12 0xada54f  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xada54f)
    #13 0xac352a  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xac352a)
    #14 0xb884ab  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xb884ab)
    #15 0xaf2551  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xaf2551)
    #16 0xac352a  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xac352a)
    #17 0xb884ab  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xb884ab)
    #18 0xb8dc90  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xb8dc90)
    #19 0x5464e3  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0x5464e3)
    #20 0xd946ab  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xd946ab)
    #21 0xaf5b61  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xaf5b61)
    #22 0xac352a  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xac352a)
    #23 0xb884ab  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xb884ab)
    #24 0xaf2551  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xaf2551)
    #25 0xac352a  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xac352a)
    #26 0xb80f49  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xb80f49)
    #27 0xbd3596  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xbd3596)
    #28 0xbd3ad4  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xbd3ad4)
    #29 0xada54f  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xada54f)

Direct leak of 456 byte(s) in 1 object(s) allocated from:
    #0 0x4c923c  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0x4c923c)
    #1 0x7f230d1b0684  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/modules//internet.so+0x66684)
    #2 0xcb4873  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xcb4873)
    #3 0x7b247a  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0x7b247a)
    #4 0xaf468c  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xaf468c)
    #5 0xac352a  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xac352a)
    #6 0xb884ab  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xb884ab)
    #7 0xaf2551  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xaf2551)
    #8 0xac352a  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xac352a)
    #9 0xb80f49  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xb80f49)
    #10 0xbd3596  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xbd3596)
    #11 0xbd3ad4  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xbd3ad4)
    #12 0xada54f  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xada54f)
    #13 0xac352a  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xac352a)
    #14 0xb884ab  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xb884ab)
    #15 0xaf2551  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xaf2551)
    #16 0xac352a  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xac352a)
    #17 0xb884ab  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xb884ab)
    #18 0xaf2551  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xaf2551)
    #19 0xac352a  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xac352a)
    #20 0xb80f49  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xb80f49)
    #21 0xbd3596  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xbd3596)
    #22 0xbd3ad4  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xbd3ad4)
    #23 0xada54f  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xada54f)
    #24 0xac352a  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xac352a)
    #25 0xb884ab  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xb884ab)
    #26 0xaf2551  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xaf2551)
    #27 0xac352a  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xac352a)
    #28 0xb884ab  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xb884ab)
    #29 0xaf2551  (/usr/local/R-devel_2016-09-12-clang39-san/lib64/R/bin/exec/R+0xaf2551)

SUMMARY: AddressSanitizer: 3192 byte(s) leaked in 7 allocation(s).

I can't do anything with this output unless you can isolate the example or call each one comes from as specifically as possible.

Ok, now I see the output at the link you gave.  That is a little more useful.  Let's talk tomorrow.

Ok, I think this long saga is nearly dealt with.
On branch compilation_overlaps4:

Better separation of what goes into nimble.so and libnimble.a or libnimble.dylib (or other extenstion)
Better propagation of Dlls throughout the R code
The separation above leads back into problems with unloading a DLL and then having crashes upon exit or other garbage collection by R since finalizers have been unloaded. This necessitates further changes:
Adaptation of a design sketched by DTL for inserting a layer into handling of finalizers.  Now finalizer registration is driven from R, and a generic finalizer is called that actually manages a C++ map of [extptr, real finalizer function pointer] pairs, also with DLL information kept track of.  That means we can manually finalize, or R can finalize, and if managed properly we should never call an unloaded finalizer, and with further changes below we should never have unloaded a DLL without finalizing all of its C++ objects first.
Generation of a session-specific DLL at time of first compileNimble use that goes all the way to loading a DLL.  The session-specific DLL includes the C++ global object keeping track of all finalizers and the functions to use it.  This DLL should never by unloaded until exiting R.
Added a new function clearCompiled(obj) that will clear compiled content of a nimbleProject.  This includes manually finalizing C++ objects, setting any external pointers to them to NULL, and sometimes just ensuring that any attempted uses of any remaining interface objects in R will fail safely.
R CMD CHECK --as-cran assumes limited uses of .Call.  Specifically it assumes it can determine safe usages by looking at the first argument (and sometimes the PACKAGE argument)  during package checking.  This does not accommodate nimble's on-the-fly compilation of DLLs, so I had to work around it.
As of now the UBSAN seems to pass, specifically the case of compileNimble that was in an Rd example (happened to be for nfVar, but I don't think that's the pertinent point).
There are a couple of issues like something about the roxygen for runMCMC and a few other tweaks I'd like to discuss separately before going back to CRAN, but I think we are over the big hurdle (why did we have to fix it if it wasn't broken?) of this Issue.

What's the issue with the roxygen for runMCMC ?

I'm closing this now, as we think the issue is dealt with.  It's difficult to tell since we couldn't reproduce it.
(@danielturek, I think the runMCMC roxygen issue may have been that Rd was out of date with roxygen and something wasn't happy, but it seems to resolve itself when Chris rebuilt the Rd files)

Good to know.  If that was at some point assigned to me, then I had lost track of it anyway.


Bio Fuel Generator, Right Click crash

Industrial Foregoing Version:  1.15.2-2.3.0-6f57752
Titanium Version:  1.15.2-2.4.0
Forge 31.1.74
Crashlog If Applicable (Upload to pastebin/gist):
https://pastebin.com/iq8deF8K
If you right click the Bio Fuel Generator block from IndustrialForegoing the game crashes. Just place the block then click it
I reported on Titanium issue tracker too

Just had this same crash. Here's my logs.
latest.log
crash-2020-05-06_07.05.25-client.txt

Having the same issue

Should be fixed in the latest Titanium


Unboxing to Enum Key in Dictionary fails

JSON
{
"section1": [{ "id": "1"}, { "id": "2"}],
"section2": [{ "id": "1"}, { "id": "2"}]
}

Code
enum SectionType: String, UnboxableEnum {
case section1 = "section1"
case section2 = "section2"
}

struct SectionObject: Unboxable {
let id: String
...
}

let sections: [SectionType: [SectionObject]]? = unbox ....

unboxing will fail when using [SectionType: [SectionObject]] but will work when using [String: [SectionObject]]

Hi @vhosune!
Thanks for submitting this issue 👍
Unboxing into a dictionary of models with enum keys is not a supported feature. I don't think I'd like to add enum support directly, as it's a bit too specific (and can be achieved by simply unboxing into [String : T] and then transforming that dictionary's keys into enum values). However, I'm willing to accept a PR that adds support for unboxing into a dictionary with a key type conforming to UnboxableKey - this is already supported on the model level.
What do you think?

Ping @vhosune. Is this something you'd want to add to Unbox, or should I close this issue? 🙂

I don't see how I would handle failure if the key as new values as it should be backward compatible, and yet if it is malformed...

I don't know if its new but implementing UnboxableKey does the trick


[bug] Zoom level is not actually 1 when CMD+0

Describe the bug
It's difficult for a user to tell how their app looks at actual scale because of this. We should consider either going down to 1 on CMD+0, adding some for of zoom level display or an actual full-screen view.

This is ok from the new zoom input


wetransfer 0.0.5

New Pull Request Checklist
Please mark the points which you did / accept.

[x] I read the Step Development Guideline
[x] I have a test for my Step, which can be run with bitrise run test (in the step's repository)
[x] I did run bitrise run audit-this-step (in the step's repository - note, if you don't have this workflow in your bitrise.yml, you can copy it from the step template.)
[x] I read and accept the Abandoned Step policy

Come on bitrise team :) you like to make me suffer !

:D thanks for the update @kawaiseb - I swear once the initial version is accepted updates will go way faster, as we only have to check the diffs ;)

Hi @kawaiseb ,
thanks for the fixes and for this new step!
It looks great to me!

Step has been removed from steplib: https://github.com/bitrise-io/bitrise-steplib/blob/d3581bdf1de00ddbbe0f0c7113f75d855312e8b1/steps/wetransfer/step-info.yml


fix(manage-merge-queue): improve logging further

:pencil: Description

:link: Related Issues

:tada: This PR is included in version 1.62.4 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


bump jobserver to respect --jobserver-auth=fifo:PATH

What does this PR try to resolve?
tl;dr: GNU Make 4.4 switches to use named pipes instead of anonymous simple pipes. Cargo needs to respect that to prevent clients from breaking.
See https://github.com/alexcrichton/jobserver-rs/pull/49
How should we test and review this PR?

Get a GNU Make 4.4 binary.
Change lines like this pointing to your binary.
For example, on macOS I changed to gmake to invoke GNU Make from homebrew.
Run cargo t --test testsuite jobserver::

If you want to test this patch live, you can also execute cargo build from make -j2 to test the behaviour.
(I usually run procs -w rustc to observe the amount of rustc invocations.)
# Makefile
all:
	+./target/debug/cargo build --release
clean:
	cargo clean --release

Additional information
Fallback workaround: add --jobserver-style=pipe if you are using GNU Make 4.4 and this PR has yet got landed.
Some distros that already got GNU Make 4.4:

Fedora 38: https://src.fedoraproject.org/rpms/make/c/715e733cee430436b250af7d1e32d72fb7dfc8b6?branch=f38
openSUSE Tumbleweed: https://build.opensuse.org/request/show/1032805
Arch Linux, Manjaro: https://github.com/archlinux/svntogit-packages/commit/9448ab762d966b18025ec55f0a4239cbe6de8b0f
Alpine Linux (edge): https://git.alpinelinux.org/aports/commit/main?id=9852cb19b76057ec5216f238ffbf5ff8d5de010f
KaOS: https://github.com/KaOSx/core/commit/6b2939d51145fd0ca6660894b5820af5ed3b6e30

BTW, should we update rustc and other crates as well in the ecosystem, i.e. cc?

BTW, should we update rustc and other crates as well in the ecosystem, i.e. cc?

When this merges in rust-lang/rust, it should implicitly update rustc.
As for users of cc, bumping its requirement would only be helpful for users who explicitly update cc without updating any other dependencies (such as cargo update -p cc), right? I'm not sure how common that would be. Of course it shouldn't hurt, so if you want to send a PR to update it, that sounds good.
This also only really matters when cargo is being run under a relatively recent make, correct?
Is this ready to merge?

Yep. It should behave as before for any GNU Make older than v4.4.
And yes, it is ready to merge.

Thanks!
@bors r+

:pushpin: Commit b9bfda596f1506362cbfae4830d3861cca945753 has been approved by ehuss
It is now in the queue for this repository.

:hourglass: Testing commit b9bfda596f1506362cbfae4830d3861cca945753 with merge 9880b408a3af50c08fab3dbf4aa2a972df71e951...

:sunny: Test successful - checks-actions
Approved by: ehuss
Pushing 9880b408a3af50c08fab3dbf4aa2a972df71e951 to master...


Support for YAML format

I just stumbled onto RapiDoc and I'd like to check it out, but my OpenAPI definition is in YAML format, not JSON. All other swagger-related tools that I've come across work with both YAML and JSON, so it would be very nice if YAML could be added to RapiDoc.

yaml works great - you just can't load it from the top bar - here is part of my html:

this is how you load yaml.
As of now you can open yaml only if you are hosting it on a web-server
<rapi-doc spec-url="https://api.apis.guru/v2/specs/bitbucket.org/2.0/swagger.yaml">
</rapi-doc>

if you want to load a local file from disk using the top bar, thats limited to json. we will eventually be adding that support, but thats not gonna be soon

I found my problem. The RapiDoc instructions say to add the HTML to an index.html file and then just open the file. Actually, you have to run a server to serve the file; you can't just open it in a browser.
Now that I'm past that little misunderstanding, I'm enjoying looking at the RapiDoc output. Is there a way to control the order of tag presentation in the RapiDoc display? They seem to be sorted alphabetically, which is decent for a default order, but just because something starts with A doesn't mean it's the most significant thing or that I want to show it first. I admit that I'm a technical writer and not a programmer, so I'm focused on both content and presentation.

is there a way to control the order of tag presentation in the RapiDoc display

Well there is no way to specify tag order in the spec, so it is up-to the viewer how it shows up the tags. RapiDoc shows them alphabetically. This gives you an ability to present the content in a sequence you desire by following some naming conventions. I have seen people naming tags like
1. Authentication, 2. Users, 3. Employees, 4. Products and so on
I am open to ideas, If you have any suggestions

ReDoc uses x-tagGroups to group tags and control the order of presentation. You'd have to do some redesigning in the RapiDoc display to somehow show grouped tags, but I've found it very useful for large APIs.

x-tags are OpenApi extensions which are specific to viewers. At this point we do not want to introduce any extensions for RapiDoc.
I have floated the idea of tag-organization to our usability team, if something needed to be changed on this area,  my guess is, it wont be soon.
I will be closing this issue for now as the primary issue has been resolved


added intial build of news page including the news index and styles. …

…Styles include general site styling as well as specific news page styling.

This PR covers the init build, the search bar, the nav bar. I haven't truly checked responsiveness yet, so I didn't close that issue. Search bar JS functionality to come later with API integration.


Crash on train.py

The script train.py crashes in line 229 sess.run([subgraph[model]['train_op']], feed_dict=feed_dict).
The exact message is
terminate called after throwing an instance of 'std::bad_alloc'   what():  std::bad_alloc Process finished with exit code 134
I ran the script by
 python2 train.py --hypes hypes/multinet2.json

Fixed when used with tensorflow 1.0.1


Javadoc for 0.10.0

Everything is auto-generated. Also, can either of you enable branch protection for gh-pages branch?

I enabled branch protection, but I didn't make AppVeyor or Travis required, since they can't run on this branch.


Revert "Changed logs-agent logs severity from Error to Info"

What does this PR do?
This reverts commit 591a09dd428860fad1c33f301b19998e0fa3066a.
Fixed the issue changing logs format in logs-agent.
Cannot be merged before merging:
https://github.com/DataDog/datadog-log-agent/pull/10

 Make pod annotations based off default yaml files/ collect a lot more JMX metrics out of the box


fix(ipfs-coord): Updating to v6.8.4

:tada: This PR is included in version 2.0.5 :tada:
Your semantic-release bot :package::rocket:

:tada: This PR is included in version 2.0.5 :tada:
Your semantic-release bot :package::rocket:


Dynamically reclaiming resources

What type of PR is this?
/kind feature

What this PR does / why we need it:
Add the support for dynamic resources reclaim.
Which issue(s) this PR fixes:

Fixes #78
Special notes for your reviewer:
Does this PR introduce a user-facing change?

Add the support for dynamic resources reclaim.

/cc @alculquicondor

Totally LGTM.


Add license files to build

See #1610 - auto generation of license files (maven)

Build has failed on gaian/ranger.
This is due to the fact that gaian is downloaded and added into a local repo during the build.
However the license tool runs during the 'process-resources; phase, which is earlier.
Didn't cause. problem locally as once downloaded the gaian repo is left in-situ
Will need to adjust the sequencing - probably ensuring gaian plugin is downloaded earlier (process-resources or before in the lifecycle)

Migrated the download of gaian into top level pom & moved to initialise phase so that it is done very early on before on dependency, license, security checks -- otherwise other maven goals can fail/warn due to these not being present.
Additionally contained the download within a profile, activated by default, so that if an org wishes NOT to download additional files direct from the internet during the build, they can instead host on their own local artifactory (since gaian is not available publicly except in zip format) which should fix #469. While not explicitly offering a URL, the net effect is the same since additional maven repos can be configured when executing the build, and these will also be searched for artifacts - in our case gaiandb & derby

To avoid download use
mvn clean install -DskipThirdPartyLocalRepo


[AutoPR keyvault/resource-manager] Keyvault: added package tag in readme and added Java readme

Created to sync https://github.com/Azure/azure-rest-api-specs/pull/6451

This PR has been merged into https://github.com/Azure/azure-sdk-for-java/pull/4133


docs: Improve wording for labels and services policies

Based on user feedback, improve the way that this documentation reads in order
to clarify how these policy types apply to Cilium objects.

docs: Rename Labels-based endpoint policies
docs: Move services note about applicability higher

/test

/test

/test

PR is in draft, going to remove ready-to-merge label.

/test


Cr 6076

Codecov Report

Merging #96 (5c1030e) into main (28873ba) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main      #96   +/-   ##
=======================================
  Coverage   25.13%   25.13%
=======================================
  Files           1        1
  Lines         187      187
=======================================
  Hits           47       47
  Misses        134      134
  Partials        6        6

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 28873ba...5c1030e. Read the comment docs.


v1.0.0 cause Program exited with code -11 (v0.13.3 is fine)

maybe the problems caused by some sharedStaticCtor ?
Thread 1 "" received signal SIGSEGV, Segmentation fault.
0x0000555555ea28d1 in _D13fluentasserts4core6expect6Expect6__ctorMFNcNeSQBwQBk10evaluation15ValueEvaluationxAyaxmAyaZSQDqQDeQDcQCy (this=0x7fffd9638500, value=..., fileName=..., line=506, prependText="") at /.dub/packages/fluent-asserts-1.0.0/fluent-asserts/source/fluentasserts/core/expect.d:25
25	    evaluation.id = Lifecycle.instance.beginEvaluation(value);

(gdb) info threads
  Id   Target Id                                             Frame
* 1    Thread 0x7fffe722f000 (LWP 3201287) "" 0x0000555555ea28d1 in _D13fluentasserts4core6expect6Expect6__ctorMFNcNeSQBwQBk10evaluation15ValueEvaluationxAyaxmAyaZSQDqQDeQDcQCy (this=0x7fffd9638500,
    value=..., fileName=..., line=506, prependText="") at /.dub/packages/fluent-asserts-1.0.0/fluent-asserts/source/fluentasserts/core/expect.d:25
  2    Thread 0x7fffd771e000 (LWP 3201290) "ats-test-LIVE_T" __memmove_avx_unaligned_erms () at ../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S:384

(gdb) where
#0  0x0000555555ea28d1 in _D13fluentasserts4core6expect6Expect6__ctorMFNcNeSQBwQBk10evaluation15ValueEvaluationxAyaxmAyaZSQDqQDeQDcQCy (this=0x7fffd9638500, value=..., fileName=..., line=506, prependText="")
    at /.dub/packages/fluent-asserts-1.0.0/fluent-asserts/source/fluentasserts/core/expect.d:25
#1  0x00005555557884bf in _D13fluentasserts4core6expect__TQkTlZQpFNeLlxAyaxmAyaZSQCbQBpQBn6Expect (testedValue=140737488344216, file=..., line=506, prependText="")
    at /.dub/packages/fluent-asserts-1.0.0/fluent-asserts/source/fluentasserts/core/expect.d:282
#2  0x0000555555778bed in _D13fluentasserts4core4base6Assert__T10opDispatchVAyaa11_677265617465725468616eTlTiZQBvFNfliQBqxAyaxmZv (actual=10281, expected=0, reason="", file=..., line=987)
    at /.dub/packages/fluent-asserts-1.0.0/fluent-asserts/source/fluentasserts/core/base.d:506

#5  0x00005555556e9c64 in _D ... 25_sharedStaticCtor_L321_C1FZv () at
#6  0x0000555556074c79 in rt.minfo.rt_moduleCtor().__foreachbody1(ref rt.sections_elf_shared.DSO) ()
#7  0x000055555607549a in rt.sections_elf_shared.DSO.opApply(scope int(ref rt.sections_elf_shared.DSO) delegate) ()
#8  0x000055555606d0bf in rt_init ()
#9  0x000055555606d54d in rt.dmain2._d_run_main2(char[][], ulong, extern(C) int(char[][]) function).runAll() ()
#10 0x000055555606d486 in _d_run_main2 ()
#11 0x000055555606d2de in _d_run_main ()
#12 0x0000555555a22502 in main (argc=1, argv=0x7fffffffd8d8) at /ldc2-1.30.0-linux-x86_64/bin/../import/core/internal/entrypoint.d:42
#13 0x00007ffff4f04d90 in __libc_start_call_main (main=main@entry=0x555555a224e0 <main>, argc=argc@entry=1, argv=argv@entry=0x7fffffffd8d8) at ../sysdeps/nptl/libc_start_call_main.h:58
#14 0x00007ffff4f04e40 in __libc_start_main_impl (main=0x555555a224e0 <main>, argc=1, argv=0x7fffffffd8d8, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffffffd8c8)
    at ../csu/libc-start.c:392
#15 0x00005555556e80d5 in _start ()

(gdb) thread 2
[Switching to thread 2 (Thread 0x7fffd771e000 (LWP 3201290))]
#0  __memmove_avx_unaligned_erms () at ../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S:384
384	../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S: No such file or directory.
(gdb) where
#0  __memmove_avx_unaligned_erms () at ../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S:384
#1  0x000055555607312d in _d_arraycatnTX ()
#2  0x0000555555edb948 in _D13fluentasserts4core10operations8registry8Registry8registerMFAyaQdQfDFNbNfKCQCyQCm10evaluation10EvaluationZACQEfQDt7results7IResultZCQFdQErQEpQEgQEa (this=0x7fffd82e0a50, valueType="long[]",
    expectedValueType="byte[]", name="equal", operation=...) at /.dub/packages/fluent-asserts-1.0.0/fluent-asserts/source/fluentasserts/core/operations/registry.d:53
#3  0x0000555555edbaf6 in _D13fluentasserts4core10operations8registry8Registry8registerMFAyaQdQfPFNbNfKCQCyQCm10evaluation10EvaluationZACQEfQDt7results7IResultZCQFdQErQEpQEgQEa (this=0x7fffd82e0a50, valueType="long[]",
    expectedValueType="byte[]", name="equal", operation=0x555555ec7d80 <_D13fluentasserts4core10operations10arrayEqualQmFNbNfKCQCbQBp10evaluation10EvaluationZACQDiQCw7results7IResult>)
    at /.dub/packages/fluent-asserts-1.0.0/fluent-asserts/source/fluentasserts/core/operations/registry.d:63
#4  0x0000555555eb6345 in _D13fluentasserts4core9lifecycle18_staticCtor_L31_C1FZv () at /.dub/packages/fluent-asserts-1.0.0/fluent-asserts/source/fluentasserts/core/lifecycle.d:106
#5  0x0000555556074cd9 in rt.minfo.rt_moduleTlsCtor().__foreachbody1(ref rt.sections_elf_shared.DSO) ()
#6  0x000055555607549a in rt.sections_elf_shared.DSO.opApply(scope int(ref rt.sections_elf_shared.DSO) delegate) ()
#7  0x0000555556065b1c in thread_entryPoint ()
#8  0x00007ffff4f6fb43 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#9  0x00007ffff5001a00 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Would it be possible to share the test/project that triggers this error? also, the compiler version would also be helpful to know.

Actually I think I found the problem:
In one of my own sharedStaticCtor, some function called Assert.equal(), I think maybe it means the fluent-asserts has not be fully init-ed yet?
Maybe in this case, all such Assert should be just ignored.
(I will try to remove such Assert, and see what happens).

yes, to improve the integration with the Trial test runner, I had to introduce some global states. It's better not to have asserts in a module constructor. I'm currently working on a better solution to it, but I don't have enough time...

It is confirmed that this is the cause of the problem (I remove such Assert calls in the sharedStaticCtor call path, it works now).

I'm currently working on a better solution to it

As I suggested: for now, maybe just issue a warning message, and ignore such assert? (not sure if it's detectable).

Hit by this issue again. Any update?
Another thought: make Assert only do real call in version (unittest), otherwise, just NO-OP.

I only had a little time for this library. I want to spend some time on Friday on this. I hope I can fix all open issues then. Otherwise, PRs are always welcomed.


Bug Fixed & added some features

*** Bug Fixes ***
1. In make_string method:
    Newline (\n) and tab (\t) character was not working as expected.
    Like when typed "I'm fine.\nWhat's about you?", it was printing: I'm fine.nWhat about you?
    Now this problem is fixed. \n and \t is working fine.
2. In make_number method:
    Typing something like .1233 or .004 was a cause for crashing the program.
    Now this is fixed. Type .123 or 0.123 will give you same result.

*** Features added ***
1. For making string now one can start with single qoute or double qoute. Type "Hi" or 'Hi' will make same result.
2. Expoential number can be used now.
   1.23e45 or 1.23E45 will print 1.23e+45
   .00123e-89 will print 1.23e-92
3. multiline comment is enabled.
    Type #[
            This is a
            multiline
            comment
        ]#

Hey, @ad1b003 can you try out radon this and make contribution here. Because  this project is not moving. So, I thought to create a new repository and improve this language more.


Remove unused packages from the Dapper container

gcc, ca-certificates, git, vim, less, and file are all unused.
Signed-off-by: Stephen Kitt<EMAIL_ADDRESS>

Makes sense, could we keep vim & less, I sometimes use them through:
./.dapper -s -m bind
which will drop you in a shell inside the dapper container. May be we should document that.

Makes sense, could we keep vim & less, I sometimes use them through:
./.dapper -s -m bind
which will drop you in a shell inside the dapper container. May be we should document that.

You can install them when you need them, instead of requiring every build to download them. Would that be good enough?

True!

Would it be possible to keep git? I would like to do  something like:
export KUBECONFIG=$(echo $(git rev-parse --show-toplevel)/output/kind-config/local-dev/kind-config-cluster{1..3} | sed 's/ /:/g')
This is really helpful to prevent directory structure calculations. This will help further work removing dapper.

Would it be possible to keep git? I would like to do something like:

Done!


Typo in the example of: Load data from CSV into Azure SQL Database or SQL Managed Instance (flat files)

The following first command of step 3 - Load the Data of this MS Azure Doc has a typo for the switch -t for field terminator.
bcp DimDate2 in C:\Temp\DimDate2.txt -S <ServerName> -d <DatabaseName> -U <Username> -P <password> -q -c -t  ,
My Example:
bcp DimDate2 in C:\Temp\DimDate2.txt -S MyAzureSQLServer.database.windows.net -d MyAzureSQLDb -U testUser -P testpassword -q -c -t ,
Error when you run the above command without enclosing field terminator as ",":
At line:1 char:135
+ ... windows.net -d MyAzureSQLDb -U testUser  -P testpassword  -q -c -t ,
+                                                                         ~
Missing argument in parameter list.
    + CategoryInfo          : ParserError: (:) [], ParentContainsErrorRecordException
    + FullyQualifiedErrorId : MissingArgument

Remark: After I use -t "," in the above bcp command, it successfully imports data to table DimDate2 in my Azure SQLDb. Except for that typo, it is a good example provided by your team.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 2d10956a-5e04-e36e-4525-68850678a7ef
Version Independent ID: 572cdda2-f855-01e4-dfea-989dd3b0c3d0
Content: Load data from CSV file into a database (bcp) - Azure SQL
Content Source: articles/azure-sql/load-from-csv-with-bcp.md
Service: sql-database
Sub-service: data-movement
GitHub Login: @dzsquared
Microsoft Alias: drskwier

Thank you for reaching out. At this time we are reviewing the ask and will provide an update as appropriate

I'm assigning this to the author for the feedback review.


Dynamic Links in Subfolders not Working Properly with Routes

Hi. I have a problem with dynamic links in subfolders in the pages directory. I have gone through the documentation a number of times, and I have not been able to solve this, so I am posting the question here.
I have a repository with the problem here: https://github.com/d-kirkland/i18n-dynamic-link-problem
The problem: I have a link on the index.vue page that should direct to pages/customers/challenges/[slug].vue. I have set up the routes like so:
'customer/challenges/[slug]': {
        en: '/customers/challenges/[slug]',
        de: '/kunden/challenges/[slug]',
    },

And the link looks like this:
<NuxtLink :to="localePath({ name: 'customers-challenges-slug', params: { slug: currentRoute }})">               {{ t('links.challenge') }} </NuxtLink>
The problem is that when I click on the link when I am the EN locale, I am directed to /en/customers/challenges/food. That is not a problem, but when I click on the link in the DE locale, I am directed to /de/customers/challenges/lebensmittel. This is where the problem is. I would expect the link in the DE locale to link to /de/kunden/challenges/lebensmittel. NuxtLink/localePath is picking up the slug, but it is not getting the route correct. I have provided all of the content of the files, and the link to the repository is above. Is there something that I am doing wrong?
Here is the index.vue page:
script setup lang="ts">
    const { localeProperties, setLocale } = useI18n()
    const localePath = useLocalePath()
    const { t } = useI18n({ inheritLocale: true })
    const source = {
        en: {
            route: 'food'
        },
        de: {
            route: 'lebensmittel'
        }
    }
    const code = localeProperties.value.code
    const currentRoute = computed(() => {
        if (localeProperties.value.code === 'de') return source.de.route
        else return source.en.route
    })
</script>

<template>

            <NuxtLink :to="localePath({ name: 'customers-challenges-slug', params: { slug: currentRoute }})">{{ t('link.name') }}</NuxtLink>

            <button @click="setLocale('en')">en</button>
            <button @click="setLocale('de')">de</button>

</template>

<i18n lang="yaml">
    en:
        link:
            name: Challenges
    de:
        link:
            name: Herausforderungen
</i18n>

I am pulling the data in from a headless CMS, so the source object is meant as a substitute from data from the CMS.
There is then a pages/customers/challenges/[slug].vue page that looks like this:

<template>

{{ t('welcome')}}

</template>

<i18n lang="yaml">
    en:
        welcome: Welcome to the Challenge
    de:
        welcome: Willkommen zur Herausforderung
</i18n>

Then the nuxt.config.ts file:
import pages from './locales/routes'

export default defineNuxtConfig({
    compatibilityDate: '2024-04-03',
    devtools: { enabled: true },
    modules: [
        '@nuxtjs/i18n',
        '@nuxtjs/tailwindcss',
    ],
    i18n: {
        locales: [
            {
                code: 'en',
                iso: 'en-DE',
                languageCode: 'EN',
                countryCode: 'US',
                regionCode: 'EU',
                countryName: 'United States',
                currency: 'EUR',
            },
            {
                code: 'de',
                iso: 'de-DE',
                languageCode: 'DE',
                countryCode: 'DE',
                regionCode: 'EU',
                countryName: 'Deutschland',
                currency: 'EUR',
            },
        ],
        defaultLocale: 'en',
        strategy: 'prefix',
        detectBrowserLanguage: false,
        pages: pages,
        customRoutes: 'config',
    },
})

And then the locales/routes/index.js file:
const pages = {
    'index': {
        en: '/',
        de: '/',
    },
    'customer/challenges/index': {
        en: '/customers/challenges',
        de: '/kunden/challenges',
    },
    'customer/challenges/[slug]': {
        en: '/customers/challenges/[slug]',
        de: '/kunden/challenges/[slug]',
    },
}
export default pages;

package.json:
{
  "name": "nuxt-app",
  "private": true,
  "type": "module",
  "scripts": {
    "build": "nuxt build",
    "dev": "nuxt dev",
    "generate": "nuxt generate",
    "preview": "nuxt preview",
    "postinstall": "nuxt prepare"
  },
  "dependencies": {
    "@nuxtjs/i18n": "^8.5.1",
    "nuxt": "^3.13.0",
    "vue": "latest"
  },
  "packageManager"<EMAIL_ADDRESS>  "devDependencies": {
    "@nuxtjs/tailwindcss": "^6.12.1"
  }
}

The issue in your reproduction appears to be that you're not using the route names in the pages configuration object, so the custom route paths are not applied.
const pages = {
    'index': {
        en: '/',
        de: '/',
    },
    // use route name
-  'customer/challenges/index': {
+  'customers-challenges': {
        en: '/customers/challenges',
        de: '/kunden/challenges',
    },
    // use route name
-   'customer-challenges-slug': {
+   'customers-challenges-slug': {
        en: '/customers/challenges/[slug]',
        de: '/kunden/challenges/[slug]',
    },
}
export default pages;

Sorry for the late response! Perhaps you already resolved this issue on your own, but if you have any questions let me know and I'll reopen this issue 🙏


UCB: add gossip handlers for Proposal and Ballot

Motivation

3rd PR for #2902

Changes

add gossip handler for Proposal and Ballot

PRs to follow:

miner produce Proposal
layer fetch and fetcher fetch Proposal
save Proposal and Ballot in mesh

Test Plan

unit tests
DevOps Notes

[x] This PR does not require configuration changes (e.g., environment variables, GitHub secrets, VM resources)
[x] This PR does not affect public APIs
[x] This PR does not rely on a new version of external services (PoET, elasticsearch, etc.)
[x] This PR does not make changes to log messages (which monitoring infrastructure may rely on)

what is the idea behind publishing both ballots and proposals separately, but including ballots into proposals? i guess validity of the proposal is based on the ballot validity, that's why it is included. but in what case ballot can be published separately?

i was wrong. we only need to fetch Ballot independently from peers, but not gossip about Ballot.

what do you think about two different modules for proposal handler and ballot handler, it is generally easier to see dependencies and interactions between modules

without the Ballot gossip handler, do you still prefer to break them to separate modules? i find them quite closely connected and logically easier to see all code in the same module.

bors merge


Emulate on throwing exception inside method implementation

Is there any way to emulate throwing an exception inside method implementation?
Because I need the backtrace methods to determine which methods get called and the game itself send back the backtrace into the game server.
Thanks!

Yes, you can create a System.Exception (or whatever is called) object and then call its method (can't remember if it's static or not) to throw it.
Alternatively, there's also a il2cpp export (e.g. il2cpp_exception_throw).

How do I call it without return/throw keyword? Or is it possible just to invoke as normal method from exception class? Could you give some example to do it?

Or is it possible just to invoke as normal method from exception class

static System.Void ReportUnhandledException(System.Exception exception);

I cant find this method

Well, it depends on the unity version you are using. Just print the System.Exception class you will probably find one. You can dump the application and look for methods such as static System\.Void \w+\(System\.Exception \w+\)\; as well

first , i called Il2Cpp.installExceptionListener('all') to catch the exception happened in il2cpp thread

as the above shown , the expception can be catched via hook the __ctx_thrown
when we hook a function or replace a function , then we want to know who called the function , we need to backtrace , but frida's api bacltrace can not provide so much detail infomation , so i hava an idea
first , implement this function , once called the function , it create an exception , then throw this exception , so the detail backtrace can generate ?
some example below
    Il2Cpp.Domain.assembly("Assembly-CSharp").image.class("XXX").method("XXX").implementation = function (message) {
        var il2cpp_raise_exception = new NativeFunction(Module.getExportByName(null , 'il2cpp_raise_exception') , 'void' , ['pointer'])
        // var systemException = Il2Cpp.Image.corlib.class("System.Exception").new()
        var exp = Il2Cpp.Domain.assembly("Assembly-CSharp").image.class("LitJson.JsonException").new()
        Il2Cpp.installExceptionListener('all')
        il2cpp_raise_exception(exp)
    }

but only output this ,  don't cantain any infomation like backtrace .

the idea like java self backtrace

i wonder is there a way to implement this function like java backtrace ?

I had the same idea few months ago, so I tried implementing it but I couldn't make it for the same reason you pointed out.
Code you might find helpful: https://github.com/vfsfitvnm/frida-il2cpp-bridge/commit/78fd8ef78fc289458514899fa25fc39a5764718d


linalg sum method

@OXPHOS hey!
Can you quickly change the implementation of vector class based on what we had discussed? It is better to be done in the initial phase before we add too many methods (easy to change now than later). So the idea is that there will only be one class for vector, something like
class GPUVectorImpl; // this is the GPUArray thing that you have

template <typename T>
class Vector
{
public:
    Vector(SGVector<T> const &);
    Vector& operator=(SGVector<T> const &);
    operator SGVector<T>() const;
    bool onGPU() const;
    T* data();
    T const * data() const;
    index_t size() const;
private:
    bool m_onGPU;   // this variable would be assessed in every linalg call,
                    // makes sense to put this first
    index_t m_len;  // same logic
    T* m_data;      // non-owning ptr, referring to the SGVector.vector
    unique_ptr<GPUVectorImpl> m_gpu_impl;
    void transferToGPU();
    void transferToCPU();
};

So, every time a call to transferToGPU() is made (by SGLinalg class), the flag onGPU is set to be true. The main thing to keep in mind here: we need to make sure that the CPU and GPU data are in sync.

We need to ensure that we invalidate the GPU data if the CPU data is updated. Say, someone transfers the data to GPU, then (s)he assigns another SGVector to it (m_data points to a different memory now). Then the data on GPU is no more valid. Subsequent calls to linalg methods should use the CPU data then.

// vec1 and vec2 are SGVectors
auto v = sg_linalg->transferToGPU(vec1);
// do something with v
v = vec2;
// v.onGPU() == ?, should be false

To achieve this, all we have to do is the set onGPU is false every time the non-const method data() is invoked. Then as long as we implement the assignment operator with the non const data() method we're good.

Similarly, if the data is on GPU and we perform some in-place operations, then the CPU data is obsolete. We need to make sure we call the transferToCPU() method in the operator that returns a SGVector then (after all the linalg operations we need to return a SGVector finally anyway).

The linalg methods that take a Vector<T> const & as parameters (e.g. dot, sum, etc), will make calls to the const version of data so it's all good. Methods that performs in-place operations will simply pass a non-const reference. Notice that we won't pass the vector as a ptr anymore. It can live on stack (just like SGVector).

Also, think a bit about thread safety while updating the onGPU variable. Please let me know if you have any questions about this. This should be much cleaner than what we have now - hate the name "BaseVector" for a class name :D
@karlnapf @vigsterkr @lisitsyn please share your thoughts and suggestions.

Maybe it is a good idea to keep the refcount as well?


Eye color picker only changes left eye's color

thanks for your work! 😉

Colorpicker's input int's - and + button doesn't changes it's value correctly

The separate odd-eyes color picker was missing, should be fixed. The inputs too.


[BUG] Matter OTA Abnormal in Apple Home

Reproduction steps
This same issue is in Apple feedback: https://feedbackassistant.apple.com/feedback/14775639
HomePod Mini version: 17.6
iPhone8 Plus with 16.7.5
I am using Apple Home TestNet to test Matter Device OTA but found unexpected behavior after OTA successes.
Before the OTA, the Matter device version is 251689472(Dec) = 0x0F007A00(Hex), the software version string should be "<IP_ADDRESS>".

shows the New SW version and the update available for the Matter device, the new version is 268470784(Dec) = 0x10008A00(Hex) and the software version string should be "<IP_ADDRESS>".

and

show the Matter device is OTA succeeded to the OLD software version and a pop-up message for the Matter Device upgrade again a few minutes later. However, I access the device information in Apple Home, the Firmware version is already the 268470784(Dec) = 0x10008A00(Hex)
The Firmware version is the newest (268470784) when I repair the Matter Device.
When I power down and power up the HomePod Mini, the update available area in the Matter Devie disappears.
Was any similar issue reported?
I would confirm what is the problem right now, esp-matter SDK/connectedhomeip SDK/HomePod Mini/iPhone?
Bug prevalence
always
GitHub hash of the SDK that was being used
83f4163
Platform
esp32
Platform Version(s)
release/v1.1 #548ef58
Anything else?
Please let me know if other information is needed.

This does not sound like a core SDK issue.  The Apple feedback assistant ticket linked above is the right venue for this (and I prodded some folks to take a look).

@emesamwong Can you please test with iOS 18 and tvOS 18? There have been many fixes in this area since. If you still see this issue, please file another FB ticket with new set of logs.


[LOC] Invalid link in an article

There is an invalid link redirecting to a non exisitng article
Repro steps:

Go to https://docs.microsoft.com/en-us/outlook/rest/java-tutorial
Search for "Spring Tool Suite"
Click on the link

Actual results:
Link redirects to a non exising page
Expected results:
Link redirects to an existing page
GH article:
https://github.com/OfficeDev/outlook-dev-docs/blob/live/docs/rest/java-tutorial.md

New link: https://spring.io/tools/


Failed to parse JSON response: {"detail":"Not Found"}

The code im running:
lm = dspy.HFClientVLLM(model="NurtureAI/Meta-Llama-3-8B-Instruct-32k", port=38242, url="http://localhost", max_tokens=4)
test_text = "This is a test article. abc"
output_normal = lm(test_text)
print("output_normal:", output_normal)

I start the server with:
python3 -m vllm.entrypoints.openai.api_server --model NurtureAI/Meta-Llama-3-8B-Instruct-32k --tensor-parallel-size 1 --port 38242 --gpu-memory-utilization 0.8

Output:
Failed to parse JSON response: {"detail":"Not Found"}
Traceback (most recent call last):
  File "/home/tom/dspy/dsp/modules/hf_client.py", line 206, in _generate
    completions = json_response["choices"]
KeyError: 'choices'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/tom/web_ai/./vllm_logprob_test.py", line 7, in <module>
    output_normal = lm(test_text)
  File "/home/tom/dspy/dsp/modules/hf.py", line 190, in __call__
    response = self.request(prompt, **kwargs)
  File "/home/tom/dspy/dsp/modules/lm.py", line 26, in request
    return self.basic_request(prompt, **kwargs)
  File "/home/tom/dspy/dsp/modules/hf.py", line 147, in basic_request
    response = self._generate(prompt, **kwargs)
  File "/home/tom/dspy/dsp/modules/hf_client.py", line 215, in _generate
    raise Exception("Received invalid JSON response from server")
Exception: Received invalid JSON response from server

Doesn't happen when I switch the model to
astronomer/Llama-3-8B-Instruct-GPTQ-8-Bit
python3 -m vllm.entrypoints.openai.api_server --model astronomer/Llama-3-8B-Instruct-GPTQ-8-Bit --quantization gptq --tensor-parallel-size 1 --port 38242 --gpu-memory-utilization 0.8 --dtype float16

Now I'm getting a BadRequestError again. Maybe the vllm server just blocked me because I was senden that many bad request errors earlier.
Creating basic bootstrap: 1/9
  2%|▊                                   | 3/128 [00:00<00:04, 25.35it/s]
Creating basic bootstrap: 2/9
  2%|▊                                   | 3/128 [00:00<00:01, 92.33it/s]
Creating basic bootstrap: 3/9
  2%|▊                                   | 3/128 [00:00<00:01, 76.30it/s]
Creating basic bootstrap: 4/9
  2%|▊                                   | 3/128 [00:00<00:01, 94.34it/s]
Creating basic bootstrap: 5/9
  2%|▊                                   | 3/128 [00:00<00:01, 90.12it/s]
Creating basic bootstrap: 6/9
  2%|▊                                  | 3/128 [00:00<00:01, 102.07it/s]
Creating basic bootstrap: 7/9
  2%|▊                                   | 3/128 [00:00<00:01, 76.79it/s]
Creating basic bootstrap: 8/9
  2%|▊                                   | 3/128 [00:00<00:01, 77.57it/s]
Creating basic bootstrap: 9/9
  2%|▊                                   | 3/128 [00:00<00:01, 80.30it/s]
Failed to parse JSON response: {"object":"error","message":"[{'type': 'extra_forbidden', 'loc': ('body', 'do_sample'), 'msg': 'Extra inputs are not permitted', 'input': True, 'url': 'https://errors.pydantic.dev/2.5/v/extra_forbidden'}]","type":"BadRequestError","param":null,"code":400}
Traceback (most recent call last):
  File "/home/tom/dspy/dsp/modules/hf_client.py", line 206, in _generate
    completions = json_response["choices"]
KeyError: 'choices'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/tom/web_ai/./generate_emails.py", line 175, in <module>
    compiled_with_assertions_mailer = teleprompter.compile(emailer, trainset=trainset,  num_trials=100, max_bootstrapped_demos=3, max_labeled_demos=5, eval_kwargs=kwargs, requires_permission_to_run=False)
  File "/home/tom/dspy/dspy/teleprompt/mipro_optimizer.py", line 461, in compile
    instruction_candidates, _ = self._generate_first_N_candidates(
  File "/home/tom/dspy/dspy/teleprompt/mipro_optimizer.py", line 249, in _generate_first_N_candidates
    self.observations = self._observe_data(devset).replace("Observations:", "").replace("Summary:", "")
  File "/home/tom/dspy/dspy/teleprompt/mipro_optimizer.py", line 177, in _observe_data
    observation = dspy.Predict(DatasetDescriptor, n=1, temperature=1.0)(examples=(trainset[0:upper_lim].__repr__()))
  File "/home/tom/dspy/dspy/predict/predict.py", line 61, in __call__
    return self.forward(**kwargs)
  File "/home/tom/dspy/dspy/predict/predict.py", line 103, in forward
    x, C = dsp.generate(template, **config)(x, stage=self.stage)
  File "/home/tom/dspy/dsp/primitives/predict.py", line 112, in do_generate
    completions: list[dict[str, Any]] = generator(prompt, **kwargs)
  File "/home/tom/dspy/dsp/modules/hf.py", line 190, in __call__
    response = self.request(prompt, **kwargs)
  File "/home/tom/dspy/dsp/modules/lm.py", line 26, in request
    return self.basic_request(prompt, **kwargs)
  File "/home/tom/dspy/dsp/modules/hf.py", line 147, in basic_request
    response = self._generate(prompt, **kwargs)
  File "/home/tom/dspy/dsp/modules/hf_client.py", line 215, in _generate
    raise Exception("Received invalid JSON response from server")
Exception: Received invalid JSON response from server

Getting the same "Not found" error again, but only after the MIPRO bootstraping phase.

looks related to https://github.com/stanfordnlp/dspy/issues/1002

#1002 should be solved by https://github.com/stanfordnlp/dspy/pull/1043 , but yes, they do look related

same issue, a little different. I use the Qwen2-72B-Instruct-GPTQ-Int4 model.
I followed the gsm8k demo, get the same error.
completions = json_response["choices"]
KeyError: 'choices'
but when I change to do the dspy.Predict('sentence -> sentiment') demo, it works fine.

Should be fixed with DSPy 2.5 migration? But lmk if not.


Clean up default perspective

Clean up default perspective for the overview infrastructure tab. We just need to change the default perspective for "all cloud".
https://issues.redhat.com/browse/COST-2164

Codecov Report

Merging #2309 (fd5a896) into master (69a206e) will not change coverage.
The diff coverage is n/a.

:exclamation: Current head fd5a896 differs from pull request most recent head 81f569f. Consider uploading reports for the commit 81f569f to get more accurate results

@@           Coverage Diff            @@
##           master    #2309    +/-   ##
========================================
  Coverage   70.02%   70.02%
========================================
  Files         403      403
  Lines        7536     7536
  Branches     1658     1532   -126
========================================
  Hits         5277     5277
  Misses       2230     2230
  Partials       29       29

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 69a206e...81f569f. Read the comment docs.


vmlatency: Define user at Dockerfile

Currently, the vm latency Dockerfile does not specify the user running the kubevirt-vm-latency, thus it is implicitly running as root.
This required the checkup Job to specify the container's user.
Add a USER stanza to the vm latency checkup's Dockerfile.
Remove runAsUser from the container's securityContext.
Install shadow-utils in the image in order to use the adduser command.
Signed-off-by: Orel Misan<EMAIL_ADDRESS>

Change: Commit message.

Change: Use a dedicated user for the checkup.

Change: Answered a comment.

I do not understand why this is the correct action to take. Specifying with which user to run in the job seems more secure to me.
Depending for the image to have a user is less secure, because it may just miss that part in general. Could you please explain why this is consider more secure? And also explain what does not work? (alternatively just ref an issue/bug)

Please see this link to the dockerfile best practices regarding the USER stanza.
K8s API reference - securityContext:
runAsUser: The UID to run the entrypoint of the container process. Defaults to user specified in image metadata if unspecified.
runAsNonRoot: Indicates that the container must run as a non-root user. If true, the Kubelet will validate the image at runtime to ensure that it does not run as UID 0 (root) and fail to start the container if it does. If unset or false, no such validation will be performed.
In summary: we specify the user in the Dockerfile, so we are not dependent on the user specifying it in the checkup Job manifest.

I also do not like that the image adds a binary in it, I would hope to see this done as part of the build, and not leave it in the container.

The image adds the shadow-utils package, because without it, there is no adduser command.
This is a tiny package.

runAsNonRoot: Indicates that the container must run as a non-root user. If true, the Kubelet will validate the image at runtime to ensure that it does not run as UID 0 (root) and fail to start the container if it does. If unset or false, no such validation will be performed.

If we specify this, then I guess it protects against wrong usage and improper image creation.
Thanks for the explanation.

The image adds the shadow-utils package, because without it, there is no adduser command.
This is a tiny package.

Why can't you do this from outside or perform a two step build?

We need to add a user to the image we are going to run.
This package is installed only because the ubi image is tiny and very minimal.

Change: Remove the shadow-utils package after creating the user.