Enable hidden visibility by default

This commit uses the new visibility feature added in https://github.com/boostorg/build/commit/898ddfa1b6888424eb292942d1014261ac7b6183 and enables hidden visibility by default for Boost libraries.
The discussion that led to this PR is here:
http://boost.2283326.n4.nabble.com/all-Request-for-out-of-the-box-visibility-support-tt4704134.html

I see CI failures in Boost.Regex. Could be related to https://github.com/boostorg/regex/issues/49 and https://bugs.launchpad.net/ubuntu/+source/llvm-toolchain-3.8/+bug/1664321

I can only suggest adding <visibility>global to Boost.Regex jamfile right now.

To speed up the fixes I sent emails directly to the Boost.Regex and Boost.Locale maintainers.

Seems that the issues were resolved in submodules. I'm in favour of merging this PR.

@Lastique could you please rebase the PR on top of the boostorg/boost to force the CI  run on fixed submodules?

Looks like there was sufficient consensus (or insufficient dissent). We observe the effect on the develop testers.

I have test issues in boost.test example folder, which are not run by the current boost regression test (develop was green last week).
What should I do?

AMDG
On 09/12/2018 03:01 PM, Raffi Enficiaud wrote:

I have test issues in boost.test example folder, which are not run by the current boost regression test (develop was green last week).
What should I do?

a) How is that related to this issue?
b) What kind of issues are we talking about?
c) Do you think that these examples should
be run by the regression tests?
In Christ,
Steven Watanabe

a) rev 638c880c5a5896bfabc13cedc7cf83d7b65a2ba5 (master) was green on boost.test side, and it turned red without any change for GCC5.4 and clang6.
b) it is firing an test error with shared linked boost.test. The test is short:
    #define BOOST_TEST_MODULE Unit_test_example_08
    #include <boost/test/unit_test.hpp>

    BOOST_AUTO_TEST_CASE_EXPECTED_FAILURES( my_test1, 1 )
    BOOST_AUTO_TEST_CASE( my_test1 )
    {
        BOOST_TEST( 2 == 1 );
    }

but the decorator is not taken into account. BOOST_AUTO_TEST_CASE_EXPECTED_FAILURES creates a static object in a translation unit. I suspect that it gets removed. Since I am not able to reproduce from a more recent compiler, I also tend to think this is a compiler bug.
c) absolutely! Those are valid use case. I am running those on my own CI. Do you think we can integrate those to the boost CI?

AMDG
On 09/12/2018 03:32 PM, Raffi Enficiaud wrote:

a) rev 638c880c5a5896bfabc13cedc7cf83d7b65a2ba5 (master) was green on boost.test side, and it turned red without any change for GCC5.4 and clang6.
 Since I am not able to reproduce from a more recent compiler, I also tend to think this is a compiler bug.

Well, I don't have either of those compiler versions
set up either, and I'm not really an expert on
visibility issues, so I'll leave this for someone else.

c) absolutely! Those are valid use case. I am running those on my own CI. Do you think we can integrate those to the boost CI?

The test Jamfile should explicitly build the examples, then.
See https://www.boost.org/development/requirements.html#Integration
In Christ,
Steven Watanabe

@apolukhin I also think that the compiler removes the symbol erroneously. However, I am also wondering: this should be client code only? I mean, apart from that, boost.test seem to be working fine, means that the generated .so has all the necessary.
What is your opinion on that?

@apolukhin Did you have the chance to look deeper into this? As a workaround, how can I remove this option from boost.test?

@raffienficiaud You can add <visibility>global to the project requirements to force the default visibility, as was before.
However, it might be better to mark the global variable with BOOST_SYMBOL_VISIBLE or __attribute__((used)), depending on which effect is intended. I have clang 6.0 but I'm not sure when I will have a chance to take a closer look at the code.
We should probably move the discussion to a Boost.Test ticket.

@Lastique thanks, I will check with the global or explicit visibility of those symbols, even if this seems to be a compiler bug.
Concerning the move of this discussion to boost.test, I am not sure the issue will get the attention from your side if we do so.

+1 for trying out BOOST_SYMBOL_VISIBLE or __attribute__((used)).
P.S.: It took unexpectedly long to fix the visibility related test failures in Stacktrace library. I've finished a few hours ago and I had no chance to look at the issue with Test. I'll try hard to find some time at this week.

Concerning the move of this discussion to boost.test, I am not sure the issue will get the attention from your side if we do so.

You can mention me in that ticket if you want to draw my attention.

@raffienficiaud I've reproduced the issue locally and now trying to find a fix

@apolukhin Thanks for looking into this!

@raffienficiaud found it out!
Perpending all the m_tu_decorators accesses in https://github.com/boostorg/test/blob/638c880c5a5896bfabc13cedc7cf83d7b65a2ba5/include/boost/test/impl/decorator.ipp with singleton<collector>::instance(). fixes the problem.
Most important functions are collector::operator*( base const& d ) and collector::store_in( test_unit& tu ).
I'm not sure that my fix fits the Boost.Test development practice. Please take a look that it's the right thing to put singleton<collector>::instance().m_tu_decorators all around the place. You should probably also check other singleton<> usages.

@apolukhin Thanks for digging this out! Do you have an explanation for this? The singleton itself has normally public visibility... I do not understand :)

I've made a PR for fixing some of the visibility issues: https://github.com/boostorg/test/pull/166
BUT it does not work right now, but works if add visibility=global to the b2 run (without PR visibility=global does not help). Here's some creepy thing, as I understand it:
-fvisibility=hidden works as expected when building shared objects. But does a strange thing for executables. It makes all the symbols hidden (probably because exporting symbols from the binary requires -rdynamic)
So the full fix will require setting visibility=global for tests of the Boost.Test OR the creepy singleton<collector>::instance(). thing from above. @raffienficiaud what would you prefer?

@apolukhin I've seen it and my CI is testing right now, many thanks. Apart from the visibility (symbol not being visible), I do not understand why the side effects of this symbol are not handled. It looks like, the symbols being removed, all their possible effects are also removed. This looks like a huge compiler bug to me, so I tend to think that we are missing something...

The side effects are not removed. They are applied to another singleton, that is not used after applying the stuff. I've put a stacktrace printing and it showed two singletons, one in the executable, the other one in the shared library of Boost.Test.

That explains - almost - everything, indeed! many thanks!
But this also opens a new question, and the fact that Visual does the job right and not GCC or Clang is troubling.
The singleton is marked with import/export symbol (through BOOST_TEST_DECL). It involves two classes collector and this template singleton<collector>. On the client code, there is a ref to the collector class, which does not call any constructs. This ref is taken by calling collector& base::operator*. In base::operator*, we call collector::instance(), so it should be calling the singleton made visible by BOOST_TEST_DECL.
Once I am inside any collector member function, I should be inside the singleton. singleton<collector>::instance() is not explaining why I have 2 singletons. The client code itself should not have the code for generating a ctor for this.
Any other decorator either refers directly to the singleton collector::instance() or to the collector base::operator* that calls collector::instance().
So the question remains open :-)

From this https://gcc.gnu.org/onlinedocs/gcc-4.7.1/gcc/Function-Attributes.html

visibility: default:

On ELF, default visibility means that the declaration is visible to other modules and, in shared libraries, means that the declared entity may be overridden.
On Darwin, default visibility means that the declaration is visible to other modules.

visibility: protected: Protected visibility is like default visibility except that it indicates that references within the defining module will bind to the definition in that module. That is, the declared entity cannot be overridden by another module.

Would protected would be better for a singleton?

Nope, still bad. You'll apply the problem with executables to the shared objects. Here's how it would probably work: you have a.so and b.so, both have singleton instances. With default visibility linker will map both entities into one address (will make one entity). With protected visibility c.so will see singleton from a.so or b.so, but a.so and b.so will have their own instances of singletons. a.so will work with its own singleton, b.so will work with its own singleton, c.so will work with some singleton (from a.so or b.so)
With executables the problem is that default visibility stops working on executables biuld with hidden visibility and without -rdynamic like flags. For that case executable a.out has it's own singleton, a.so has it's own singleton. Linker does not unify those, so a.so uses it's own singleton, while a.out uses its own.
You may also try to hide the singleton entirely in the .so so that in the the executable it is never instantiated

@raffienficiaud I've updated the PR https://github.com/boostorg/test/pull/166
Now it works with any visibility flags (makes sure that singleton is instantiated only in ipp files).
P.S.: have not test it on Windows

@apolukhin Yes, I also fixed it like this, many thanks! but here are my findings:

visibility hidden != import/export: the DLL/MSVC mechanism as I understand it does not make a symbol "invisible". It instructs the linker on the location of the symbol, but this symbol is still visible. OTOH, the 'visibility hidden' hides completely the symbols that are coming from external translation units to the linker. The linker is free to generate one if needed, it has no means to know if the symbol exists in the global binary that is being generated.
By inspecting the symbols with nm, the only way to avoid the creation of the symbol on the client side was to hide its implementation in a .cpp/.ipp file. Means that the inlining of the static accessor to the symbol was not inheriting the visibility as we expected it. Even if I move the static member function to the derived class, I had the symbol duplication. This was in fact quite surprising.

I started getting rid of the singleton implementation, as it does not what it is supposed to do properly.
Thank you for the time you took for assisting me with this. I am still a bit worried as boost.test contains several singletons in different places. I will inspect with nm.

the DLL/MSVC mechanism as I understand it does not make a symbol "invisible". It instructs the linker on the location of the symbol, but this symbol is still visible. OTOH, the 'visibility hidden' hides completely the symbols that are coming from external translation units to the linker. The linker is free to generate one if needed, it has no means to know if the symbol exists in the global binary that is being generated.

dllexport/dllimport are indeed different from visibility, but not in the aspect you describe. By marking a symbol dllexport you instruct the compiler to generate some additional shims in the binary (which start with __imp_ prefix) and the linker to emit this symbol in the export table and add the shims to the interface library (*.lib). dllimport does the opposite - it tells the compiler to rely on that symbol being present elsewhere with the __imp_ prefix (and not define the symbol), and since the linker pulls the interface .lib library that symbol gets resolved and associated with the corresponding .dll library. Also, the linker generates the import table, which is used to resolve the symbols against the export tables of the linked .dlls at load time. So, in hindsight, dllexport/dllimport is mostly about mangling the symbols differently and putting them into export/import tables in the executable.
Visibility is different in that it is simply a markup that can be applied to any symbol that is put in the symbol table of the executable. It does not alter its mangling or causes additional code generation but it does get reflected in the symbol table of the shared library. The symbol table contains both the symbols that are implemented by the executable and the symbols it is missing, and the runtime linker will try to resolve the missing symbols as it loads shared libraries into the running process.
When you mark a symbol visible (i.e. with default or global visibility), the compiler will still emit it in every library where it's defined. But at load time only a single copy will be used (from the first library that gets loaded in the process). From the language perspective, this has pretty much the same effect as dllexport/dllimport. When a symbol is marked hidden, it is still emitted in the shared library and is also present in the symbol table of the library. But the compiler knows that the symbol cannot be referenced from outside the library, so it can apply additional optimizations, such as more efficient addressing modes. Or the compiler can simply remove the symbol if it knows it is not used in the library and such a change doesn't remove any side effects like a non-trivial constructor/destructor call. Normally, this should only be possible with LTO. Note that hidden visibility has nothing to do with visibility across translation units - a hidden symbol is still very much accessible across all translation units that comprise the executable. Multiple definitions of a symbol with hidden visibility but external linkage will still cause linker errors about duplicate definitions.
Protected visibility is a rare beast, as it is not supported on all platforms. It works as the global (default) visibility, except that if the symbol is referenced from the same executable, the reference is always resolved to the copy in that executable. I'm not entirely sure how that works in cases when multiple shared objects in a process have the same symbol; most likely this is just a recipe for trouble, so just avoid it unless you know what you're doing.

@glenfe how about merging this commit to master? Looks like all the issues were resolved and we still have time to stabilize master if something goes wrong.


make fails due to error: ‘GTEST_SKIP’ was not declared in this scope

What reproduces the bug?
Followed the steps from install.md file for fedora to build bpftrace .
'make -j8' throws the following error:
make[2]: *** [tests/CMakeFiles/bpftrace_test.dir/build.make:146: tests/CMakeFiles/bpftrace_test.dir/main.cpp.o] Error 1 make[2]: *** Waiting for unfinished jobs.... /root/ak/bpftrace/tests/clang_parser.cpp: In member function ‘virtual void bpftrace::test::clang_parser::clang_parser_btf_unresolved_typedef_Test::TestBody()’: /root/ak/bpftrace/tests/clang_parser.cpp:648:5: error: ‘GTEST_SKIP’ was not declared in this scope      GTEST_SKIP();      ^~~~~~~~~~ /root/ak/bpftrace/tests/clang_parser.cpp:648:5: note: suggested alternative: ‘GTEST_FAIL’      GTEST_SKIP();      ^~~~~~~~~~      GTEST_FAIL
bpftrace --info

System
OS: Linux 4.18.0-409.el8.ppc64le #1 SMP Tue Jul 12 00:27:15 EDT 2022
Arch: ppc64le
Build
version: v0.15.0-69-g7a81
LLVM: 14.0.6
ORC: v2
foreach_sym: yes
unsafe uprobe: no
bfd: yes
bcc_usdt_addsem: yes
bcc bpf_attach_uprobe refcount: yes
bcc library path resolution: yes
libbpf btf dump: yes
libbpf btf dump type decl: yes
libbpf bpf_prog_load: no
libbpf bpf_map_create: no
libdw (DWARF support): yes
Kernel helpers
probe_read: yes
probe_read_str: yes
probe_read_user: yes
probe_read_user_str: yes
probe_read_kernel: yes
probe_read_kernel_str: yes
get_current_cgroup_id: yes
send_signal: yes
override_return: no
get_boot_ns: yes
dpath: yes
skboutput: no
Kernel features
Instruction limit: 1000000
Loop support: yes
btf: yes
map batch: yes
uprobe refcount (depends on Build:bcc bpf_attach_uprobe refcount): no
Map types
hash: yes
percpu hash: yes
array: yes
percpu array: yes
stack_trace: yes
perf_event_array: yes
Probe types
kprobe: yes
tracepoint: yes
perf_event: yes
kfunc: yes
iter:task: yes
iter:task_file: yes
kprobe_multi: no
raw_tp_special: yes

Do you have the correct version of gtest installed? Can you try building with the -DVENDOR_GTEST=ON cmake flag?

With the above mentioned solution , the make was successful and not finding the error.

yay


Converter location

I have already installed the plugin and have the Havok SDK. Which files should I select respectively as the converter location, temporary folder, and default skeleton in blender?

You reference the skeleton.hkx on import. For people, that's in actors/character/character assets/skeleton.hkx or something similar.
You don't have to worry about the temp folder or converter location. That's handled for you.


chore: validate the printed code (fixes #71)

seems there are some issues with the parser/printer, can you take a look? @KFlash

I'm soon going to publish a huge change to Kataw that will change it all, and solve a 7y try/fail and never succeed about comments so I'm also validating locale now both through re-parse and the transformers and visitors.

@aladdin-add the issues you experienced is soon to be solved.

@aladdin-add I think your issues has been solved with https://github.com/kataw/kataw/pull/138


Veto Proximity Plugin

What is the problem / what does the code in this PR do
Similarly to what we had in XENON1T, data kind produced by this plugin will be used by the DAQVetoCut, to remove events which are too close to the start or stop of a hardware veto signal.
Can you briefly describe how it works?
For each event center (event as defined in strax) it will find and store the distance to the previous and next veto on/off.
Can you give a minimal working example (or illustrate with a figure)?
It can be illustrated (and was tested) using data obtained with the special triggered mode, where left boundary of the yellow region is "trg start" and right is "trg stop". For each event center, the plugin will find the time to the next/previous trg start/stop.

Hej, I am getting a bit confused by your example. I understand it as following:
You would like to have a plugin which computes the time difference between an "event center" and the next veto signals. But your example shows data where you used an external trigger to record the events. Hence to convert your example image according to your what you want (or at least I think you want). I have to imagine the blue dashed line as veto-start and the dashed-dotted line as veto-end, am I correct?

Hej, I am getting a bit confused by your example. I understand it as following:
You would like to have a plugin which computes the time difference between an "event center" and the next veto signals. But your example shows data where you used an external trigger to record the events. Hence to convert your example image according to your needs (or at least what I think you want), I have to imagine the blue dashed line as veto-start and the dashed-dotted line as veto-end, am I correct?

Hi,
These data were not taken with an external trigger but with the "special veto" mode that I've made for commissioning. In this mode, we periodically artificially render the digitizers busy (veto), allowing them to operate even under high rate conditions, and that is how we get the 17%, 20%, 80%, etc live time data.  https://xe1t-wiki.lngs.infn.it/doku.php?id=xenon:xenon1t:alexelykov:v1495_busy_special
It made more sense at the time to record the start and end of the periods when we have data. These times are recorded in the hev_on/off channels of the acquisition monitor, so I just used them as a proxy for real hev_on and hev_off for test purposes here. Hope this makes more sense.

Thanks for clarification and I think

It made more sense at the time to record the start and end of the periods when we have data.

goes in the direction of what I was wondering. In this case your plugin works, while I think if you revert the logic you would need an overlapping window plugin. Since in principle it would be possible  that you chunk your data at a point where you have n events and n-1 veto_regions (or vise-versa). Hence you would miss one "delta t to the next/prev veto".

Thanks for clarification and I think

It made more sense at the time to record the start and end of the periods when we have data.

goes in the direction of what I was wondering. In this case your plugin works, while I think if you revert the logic (dash-dotted start and dashed end) you would need an overlapping window plugin. Since in principle it would be possible that you chunk your data at a point where you have n events and n-1 veto_regions (or vise-versa). Hence you would miss one "delta t to the next/prev veto".
https://github.com/AxFoundation/strax/blob/a1b814cca74957f812b5a80eb90ee174a5e72d84/strax/plugin.py#L292-L302

In real background/calibration data, if dash-dotted is veto "start" and dashed is veto "end" there would be no data in between them - and they would indicate the veto bounds. To avoid associating anything potentially found in proximity to them is one of the reasons for having this plugin.

I think we are not on the same page yet, maybe better to move to slack :D

I think we are not on the same page yet, maybe better to move to slack :D

Hey. After our discussion on slack, I've made some quick tests with run 10362, where the busy was firing like crazy and the plugin seems to work as it is. I might make it an OverlapWindowPlugin just in case though.

Hi Alex, I'm for a second converting this PR to draft that makes it a little easier for me to do the bookkeeping. Does not affect anything but is just a reminder to myself that you are working on it

Hi Alex, I'm for a second converting this PR to draft that makes it a little easier for me to do the bookkeeping. Does not affect anything but is just a reminder to myself that you are working on it. Let me know when you are ready and if we can help but I think you got this covered.

No problem. I had some other things to finish up today but I have some commits ready.

Hi Alex, I did not notice before but there are quite some untracked options in acqmon processing that look like they should be tracked. Are you sure about those? Not really related to this  PR

Hi Alex, I did not notice before but there are quite some untracked options in acqmon processing that look like they should be tracked. Are you sure about those? Not really related to this  PR

Hi Alex, I did not notice before but there are quite some untracked options in acqmon processing that look like they should be tracked. Are you sure about those? Not really related to this PR

Hey, what do you mean by untracked options?

Hi Alex, I did not notice before but there are quite some untracked options in acqmon processing that look like they should be tracked. Are you sure about those? Not really related to this PR

Hey, what do you mean by untracked options?

Hi Alex, I did not notice before but there are quite some untracked options in acqmon processing that look like they should be tracked. Are you sure about those? Not really related to this PR

Hey, what do you mean by untracked options?

Options like these where track==False. Now that I think about it of course make sense since it is a logic signal and these are rather low thresholds. It just at first looks like an option that should be tracked, e.g. you don't want the same data when someone sets it to 0 instead. But okay, maybe not as relevant as I thought at first glance.
https://github.com/XENONnT/straxen/blob/4e137c98d273eb87219780062b95dfecb66d14bd/straxen/plugins/acqmon_processing.py#L17

Hi Alex, I did not notice before but there are quite some untracked options in acqmon processing that look like they should be tracked. Are you sure about those? Not really related to this PR

Hey, what do you mean by untracked options?

Options like these where track==False. Now that I think about it of course make sense since it is a logic signal and these are rather low thresholds. It just at first looks like an option that should be tracked, e.g. you don't want the same data when someone sets it to 0 instead. But okay, maybe not as relevant as I thought at first glance.
https://github.com/XENONnT/straxen/blob/4e137c98d273eb87219780062b95dfecb66d14bd/straxen/plugins/acqmon_processing.py#L17

Hi Alex, I did not notice before but there are quite some untracked options in acqmon processing that look like they should be tracked. Are you sure about those? Not really related to this PR

Hey, what do you mean by untracked options?

Options like these where track==False. Now that I think about it of course make sense since it is a logic signal and these are rather low thresholds. It just at first looks like an option that should be tracked, e.g. you don't want the same data when someone sets it to 0 instead. But okay, maybe not as relevant as I thought at first glance.
https://github.com/XENONnT/straxen/blob/4e137c98d273eb87219780062b95dfecb66d14bd/straxen/plugins/acqmon_processing.py#L17

I can change them to tracked if you think it is necessary. I was planning to submit a PR anyways (there are some bugs that I've found a while ago) so I can add it there. However, thus far I simply didn't have time to update anything cause I'm trying to concentrate on the thesis thing.

Hi Alex, I did not notice before but there are quite some untracked options in acqmon processing that look like they should be tracked. Are you sure about those? Not really related to this PR

Hey, what do you mean by untracked options?

Options like these where track==False. Now that I think about it of course make sense since it is a logic signal and these are rather low thresholds. It just at first looks like an option that should be tracked, e.g. you don't want the same data when someone sets it to 0 instead. But okay, maybe not as relevant as I thought at first glance.
https://github.com/XENONnT/straxen/blob/4e137c98d273eb87219780062b95dfecb66d14bd/straxen/plugins/acqmon_processing.py#L17

I can change them to tracked if you think it is necessary. I was planning to submit a PR anyways (there are some bugs that I've found a while ago) so I can add it there. However, thus far I simply didn't have time to update anything cause I'm trying to concentrate on the thesis thing.

Hopefully final update to the aqmon plug-in set:

I re-did the veto interval plugin, as it had issues with finding intervals of variable length - now it seems to work well in triggered mode and for vetoes in normal runs as well.
Made updates based on your inputs to the plugins, i.e. tracking of options, overlawindow plugin for proximity, fixed max veto distance to 1 hour.

Hopefully final update to the aqmon plug-in set:

I re-did the veto interval plugin, as it had issues with finding intervals of variable length - now it seems to work well in triggered mode and for vetoes in normal runs as well.
Made updates based on your inputs to the plugins, i.e. tracking of options, overlawindow plugin for proximity, fixed max veto distance to 1 hour.

Look good Alex! Thanks for the PR, made some comment that might speed it up a little but I know it's not really much of a concern for this plugin.
Do you want me to also test, technically I think it looks fine.

Hi, feel free to test it.
I made some tests on a triggered run and a normal bg run with high rate where there were some busy signals and it worked fine.
However, maybe a good idea would be to see if you can run it and then get the correct livetimes in the online monitor.

Look good Alex! Thanks for the PR, made some comment that might speed it up a little but I know it's not really much of a concern for this plugin.
Do you want me to also test, technically I think it looks fine.

Hi, feel free to test it.
I made some tests on a triggered run and a normal bg run with high rate where there were some busy signals and it worked fine.
However, maybe a good idea would be to see if you can run it and then get the correct livetimes in the online monitor.

@AlexElykov unfortunately the test was not successful:
2021-02-03T12:44:19.174546 | [INFO]     | Still processing run 011593
Process Process-1:
Traceback (most recent call last):
  File "/home/xedaq/miniconda/envs/py38/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/home/xedaq/miniconda/envs/py38/lib/python3.8/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/xedaq/software/straxen/bin/bootstrax", line 1231, in run_strax
    st_make()
  File "/home/xedaq/software/straxen/bin/bootstrax", line 1210, in st_make
    for i, c in enumerate(st.get_iter(run_id,
  File "/home/xedaq/software/strax/strax/context.py", line 956, in get_iter
    generator.throw(e)
  File "/home/xedaq/software/strax/strax/context.py", line 933, in get_iter
    for n_chunks, result in enumerate(strax.continuity_check(generator), 1):
  File "/home/xedaq/software/strax/strax/chunk.py", line 266, in continuity_check
    for s in chunk_iter:
  File "/home/xedaq/software/strax/strax/processor.py", line 270, in iter
    raise exc.with_traceback(traceback)
  File "/home/xedaq/software/strax/strax/mailbox.py", line 254, in _send_from
    x = next(iterable)
  File "/home/xedaq/software/strax/strax/plugin.py", line 583, in iter
    yield from super().iter(iters, executor=executor)
  File "/home/xedaq/software/strax/strax/plugin.py", line 373, in iter
    raise RuntimeError(
RuntimeError: VetoProximity was unable to get time-consistent inputs after ten passess. Inputs:
{'veto_intervals': [011593.veto_intervals:<PHONE_NUMBER>sec 0 ns -<PHONE_NUMBER>sec 657170360 ns, 609 items, 0.0 MB/s], 'events': [011593.events:<PHONE_NUMBER>sec 0 ns -<PHONE_NUMBER>sec 657170360 ns, 442 items, 0.0 MB/s]}
Input buffer:
{'veto_intervals': [011593.veto_intervals:<PHONE_NUMBER>sec 657170360 ns -<PHONE_NUMBER>sec 687090199 ns, 5 items, 0.0 MB/s], 'events': [011593.events:<PHONE_NUMBER>sec 657170360 ns -<PHONE_NUMBER>sec 213960090 ns, 186 items, 0.0 MB/s]}

Perhaps it should be tested from processing starting at raw_records_*.

Hmm...interesting, I'll check it out. What did you use to run this test?

bootstrax, on run 011593 located at /live_data/xenonnt_tests. Let me update the bootstrax instance on master that I needed to do this test.
Actually, should work if you use 011593 from raw_records

bootstrax, on run 011593 located at /live_data/xenonnt_tests. Let me update the bootstrax instance on master that I needed to do this test.
Actually, should work if you use 011593 from raw_records.
I'll merge #354 such that you can check:
bootstrax --process 11593 --targets veto_proximity --post_process raw_records

Interesting, meanwhile I downloaded the raw_records_aqmon run and it worked fine from aqmon_hits to vet_proximity, just in a notebook.

Can it be caused by the fact that veto_proximity needs events being available to work, or I'm missing something?

No per se, strax would make sure to make the events first.

@jorana Hi. I've just noticed that this plugin wasn't merged in the end. I thought that the rechunking changes solved the issues that it had with the test run? However, honestly I tunned out a bit so maybe I'm missing/forgetting something?

@jorana @AlexElykov I took a look at this and tested it with 30 runs (using data with both raw_records_aqmon and events on Midway). Most of them worked, but one run failed with this chunk error that I've seen in older data as well. Is it possible we made a change in raw_records chunking that did not get propagated to raw_records_aqmon?
>>> st.get_array('020346', 'veto_proximity', progress_bar=False)
# long trace...
ValueError: Attempt to create chunk [020346.veto_proximity:<PHONE_NUMBER>sec 385127957 ns -<PHONE_NUMBER>sec 407021079 ns, 13371 items, 0.0 MB/s] whose data starts early at<PHONE_NUMBER>384229430

I did a bit more debugging. I think my ignorance of internals of strax is catching up with me 😕
So if I first create veto_intervals, everything works fine. But I thought strax was supposed to do this automatically? If I clear all the strax data and try reloading veto_proximity, I can see that the veto_intervals data doesn't actually get created; instead it only has a metadata file that contains the exception for the veto_proximity processing I show above.

Ok so the above was the case for Run 20346, which I was able to get past by forcing allow_rechunk=False. But I hit the same error for Run 20392 which persists if I (a) for allow_rechunk=False and/or (b) first create the veto_intervals.

Ok it works for all runs tested if I swap the order of the depends_on attribute to make events come first. Maybe some issue of mixing different data kinds? I leave it to the real experts.
Anyway at least from my side I think if @AlexElykov can merge this PR then it should be good to merge. Assuming @jorana and @WenzDaniel are okay with it. I think this is fairly high priority for a DAQ veto cut.

@ershockley Thanks for checking it out! Let's see what @jorana will say, would be nice if this time it actually can run on ebs.

@AlexElykov @ershockley @WenzDaniel let me have a look if we can get it working

Sorry @jorana do you mean it doesn't work on ebs or it does? It definitely works when processing from raw_records_aqmon. At least it did yesterday before the most recent changes.

@ershockley correct, it did not work on the ebs, with https://github.com/AxFoundation/strax/pull/450 it does

Thanks Alex, have you had the time to sanity check the changes I made? I'm happy with the current implementation, thanks a lot for the effort and sorry that it all took so long 😉

It's fine from my side. Hope it works and we can actually use the HEV to its full potential without scaring ppl with some weird s2 relics. Thank you for checking it out, and finding the strax overlapwindow plugin issue!
Also, I like your solution with the numpy arrays better than my dict one :)


Functions called with shell=1 should be flagged

The injection_shell plugins that look for shell=True (both in subprocess functions,
and in other functions) look for either the bool True or the string "True".
They might be able to detect more cases, like shell=1.
In the example below, the first two cases are flagged as HIGH, and the second
two are flagged as LOW. The case where shell=1 is not caught:
import subprocess

command = 'pwd'
subprocess.call(command, shell=True)
subprocess.call(command, shell='True')
subprocess.call(command, shell=1)
subprocess.call(command, shell=False)

If "shell" is set to anything other than a falsey constant, it
can be flagged. It may be possible to use bandit.core.constants.FALSE_VALUES,
without the string 'False'.
I think this is the real bug behind https://bugs.launchpad.net/bandit/+bug/1505389

@ericwb what do you think about the pull request?

I think this could be closed @ericwb unless you have some objection about the test case.

Fixed with PR #298


Video

The attached video is not really good quality. Please attach one with detailed instructions.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 87e2cd41-2a63-d083-22e0-cf5f75fb4b07
Version Independent ID: e82a144d-f425-d6ed-9f7d-bd4b5f395e45
Content: Tutorial - Use Azure Key Vault with an Azure web app in .NET
Content Source: articles/key-vault/tutorial-net-create-vault-azure-web-app.md
Service: key-vault
GitHub Login: @msmbaldwin
Microsoft Alias: mbaldwin

@mdonkervoort
Thanks for your feedback! We will investigate and update as appropriate.

@mdonkervoort: Based on your feedback, we have completely rewritten this article. The revised tutorial uses the new Azure Key Vault v4 client library for .NET. See  https://docs.microsoft.com/en-us/azure/key-vault/general/tutorial-net-create-vault-azure-web-app.
Please let us know if you find any issues with the new tutorial, and thank you again for your feedback. #please-close


VID/PID are strings?

I was pleasantly surprised the enumerator subpackage supports USB VID/PID numbers so it's possible to filter serial ports on them.
However, I found that VID and PID are stored as strings? That doesn't make a lot of sense to me, as these are 16-bit numbers and we have a Go type for this: uint16. strconv.ParseUint works, but requires extra error handling and in general seems less reliable. Example code:
for _, p := range portsList {
	if !p.IsUSB {
		continue
	}
	if p.VID != "" && p.PID != "" {
		vid, vidErr := strconv.ParseUint(p.VID, 16, 16)
		pid, pidErr := strconv.ParseUint(p.PID, 16, 16)
		if vidErr == nil && pidErr == nil {
			// check for VID/PID match
		}
	}
}

Maybe this ship has sailed already (because of API breakage) but in my opinion changing this to use uint16 would make the API easier to use.
For reference: https://github.com/tinygo-org/tinygo/pull/1956

Maybe this ship has sailed already (because of API breakage)

Yes, that's a big problem... maybe in a future major release we may change them, but in the meantime, it's a nice idea to use uint16 so I'll probably add the "integer" fields together with the already existing ones...
May you want to open a PR for that?


chore: Adds note about numerical x-axis

SUMMARY
26034 fixed a problem where numeric x-axis were being treated as categorical values. As a consequence of that, the way labels are displayed might change given that ECharts has a different treatment for numerical and categorical values. This PR adds a note to UPDATING.md explaining the changes to users and guiding them on how to restore the previous behavior.
ADDITIONAL INFORMATION

[ ] Has associated issue:
[ ] Required feature flags:
[ ] Changes UI
[ ] Includes DB Migration (follow approval process in SIP-59)

[ ] Migration is atomic, supports rollback & is backwards-compatible
[ ] Confirm DB migration upgrade and downgrade tested
[ ] Runtime estimates and downtime expectations provided

[ ] Introduces new feature or API
[ ] Removes existing feature or API

@michael-s-molina This looks like a breaking change:

To revert to the old behavior, users need to manually convert numerical columns to text so that they are treated as categories.
Do you think we can make the default behavior the old behavior instead so that users won't have their existing charts change?


 How do you tell which version you're running?
As far as I know, there currently isn't a way. The string 3.0. isn't anywhere in the repo. We should have a -version for the cli interface, and if possible have the version included in an "about" dialog or window.

FWIW, this came to mind while writing the issue reporting template.

Actually there is a version number in the About dialog. It is also reported to stdout as the first line after Workcraft start.
Note that the version is converted into a string in Info.java class -- that is why the string 3.0. cannot be found.
I agree that the -version parameter would be useful.

Whoops, my bad then. Let's leave this open for the flag then.

-help would be also useful.


Five or Six examples insights

Short description:

Current state:

Difficult to comprehend featureset

Desired state:

Links to several useful / actionable insights that might encourage others to explore further

Tasks:

[ ] Research some insights (5 to 6)
[ ] Add those stateful urls as links to the left hand side

Issue https://github.com/cncf/apisnoop/issues/256 is for adding the examples to the side-bar.

Potential Examples
Show All Conformant APIs
https://apisnoop.cncf.io/?zoomed=level-stable&showUntested=false&showTested=false&showConformanceTested=true
Stable-Core Tested but Not Conformant Filter
https://apisnoop.cncf.io/?zoomed=category-stable-core&showUntested=false&showConformanceTested=false
Stable-Core Conformance Tests
https://apisnoop.cncf.io/?zoomed=category-stable-core&showUntested=false&showConformanceTested=true&showTested=false)
Stable Endpoints Hit but Not Conformant
https://apisnoop.cncf.io/?useragents=[^e2e.test]&showUntested=true&showTested=false&showConformanceTested=false&zoomed=level-stable

The fourth URL above works for me but the first 3 give image errors. I suspect you're having URL escaping errors.

That's odd. They are all working for me. Though it did feel like they took a bit longer to load than usual; Approximately 3~ seconds after opening each link.

Now working for me.

Good. What were the "image errors" you saw? Was the sunburst not loading, or something else?

None of the images loaded, including even the favicon. I suspect it was a temporary Netlify issue.

/lgtm

Thanks @zachmandeville and @devaips.
Thanks for the feedback @dankohn


[Question]  can't download model

❓ General Questions
i run mlc_llm package .
mlc-package-config.json:
{
"device": "android",
"model_list": [
    {
        "model": "HF://mlc-ai/Qwen-7B-Chat-q4f16_1-MLC",
        "model_id": "Qwen-7B-Chat-q4f16_1-MLC",
        "estimated_vram_bytes":<PHONE_NUMBER>
    }
]

}
at hugging face have this model.https://hf-mirror.com/mlc-ai/Qwen-7B-Chat-q4f16_1-MLC
i try use other model.but all same error.
i look this. but i no have  D:\Users\W9057210.ADC\AppData\Local\Temp\tmpc399rnv1\tmp is folder
[2024-06-12 18:48:57] INFO download_cache.py:56: [Git] Cloning https://huggingface.co/mlc-ai/Qwen-7B-Chat-q4f16_1-MLC.git to D:\Users\W9057210.ADC\AppData\Local\Temp\tmpc399rnv1\tmp
(mlc-panghongtao) E:\project\mlc-llm\android\MLCChat>mlc_llm package
[2024-06-12 18:48:57] INFO package.py:327: MLC LLM HOME: "E:\project\mlc-llm"
[2024-06-12 18:48:57] INFO package.py:28: Clean up all directories under "dist\bundle"
[2024-06-12 18:48:57] INFO jit.py:43: MLC_JIT_POLICY = ON. Can be one of: ON, OFF, REDO, READONLY
[2024-06-12 18:48:57] INFO download_cache.py:227: Downloading model from HuggingFace: HF://mlc-ai/Qwen-7B-Chat-q4f16_1-MLC
[2024-06-12 18:48:57] INFO download_cache.py:29: MLC_DOWNLOAD_CACHE_POLICY = ON. Can be one of: ON, OFF, REDO, READONLY
[2024-06-12 18:48:57] INFO download_cache.py:56: [Git] Cloning https://huggingface.co/mlc-ai/Qwen-7B-Chat-q4f16_1-MLC.git to D:\Users\W9057210.ADC\AppData\Local\Temp\tmpc399rnv1\tmp
Traceback (most recent call last):
File "C:\ProgramData\anaconda3\envs\mlc-panghongtao\Lib\site-packages\mlc_llm\support\download_cache.py", line 57, in git_clone
subprocess.run(
File "C:\ProgramData\anaconda3\envs\mlc-panghongtao\Lib\subprocess.py", line 571, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['git', 'clone', 'https://huggingface.co/mlc-ai/Qwen-7B-Chat-q4f16_1-MLC.git', '.tmp']' returned non-zero exit status 128.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "", line 198, in run_module_as_main
File "", line 88, in run_code
File "C:\ProgramData\anaconda3\envs\mlc-panghongtao\Scripts\mlc_llm.exe_main.py", line 7, in
File "C:\ProgramData\anaconda3\envs\mlc-panghongtao\Lib\site-packages\mlc_llm_main.py", line 53, in main
cli.main(sys.argv[2:])
File "C:\ProgramData\anaconda3\envs\mlc-panghongtao\Lib\site-packages\mlc_llm\cli\package.py", line 64, in main
package(
File "C:\ProgramData\anaconda3\envs\mlc-panghongtao\Lib\site-packages\mlc_llm\interface\package.py", line 351, in package
model_lib_path_for_prepare_libs = build_model_library(
^^^^^^^^^^^^^^^^^^^^
File "C:\ProgramData\anaconda3\envs\mlc-panghongtao\Lib\site-packages\mlc_llm\interface\package.py", line 73, in build_model_library
model_path = download_cache.get_or_download_model(model)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ProgramData\anaconda3\envs\mlc-panghongtao\Lib\site-packages\mlc_llm\support\download_cache.py", line 228, in get_or_download_model
model_path = download_and_cache_mlc_weights(model)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ProgramData\anaconda3\envs\mlc-panghongtao\Lib\site-packages\mlc_llm\support\download_cache.py", line 180, in download_and_cache_mlc_weights
git_clone(git_url, tmp_dir, ignore_lfs=True)
File "C:\ProgramData\anaconda3\envs\mlc-panghongtao\Lib\site-packages\mlc_llm\support\download_cache.py", line 70, in git_clone
raise ValueError(
ValueError: Git clone failed with return code 128: None. The command was: ['git', 'clone', 'https://huggingface.co/mlc-ai/Qwen-7B-Chat-q4f16_1-MLC.git', '.tmp']

The tmp folder will be deleted after download, in your case seems there is some issue with download, you can also try to manually download(by running the command) to a local path and pass in local path
['git', 'clone', 'https://huggingface.co/mlc-ai/Qwen-7B-Chat-q4f16_1-MLC.git', '.tmp']'
note that the default command may not pick up mirros

The tmp folder will be deleted after download, in your case seems there is some issue with download, you can also try to manually download(by running the command) to a local path and pass in local path
['git', 'clone', 'https://huggingface.co/mlc-ai/Qwen-7B-Chat-q4f16_1-MLC.git', '.tmp']'
note that the default command may not pick up mirros

Why do I download the model when I set this configuration? This action is to put the configuration information into the APP.
Select this model in my phone. The download model is operated on the phone.
So if you download the model on your phone. Do mlc_llm packages still need to be downloaded?

The tmp folder will be deleted after download, in your case seems there is some issue with download, you can also try to manually download(by running the command) to a local path and pass in local path
['git', 'clone', 'https://huggingface.co/mlc-ai/Qwen-7B-Chat-q4f16_1-MLC.git', '.tmp']'
note that the default command may not pick up mirros

I have download the https://huggingface.co/mlc-ai/Qwen-7B-Chat-q4f16_1-MLC.git to my local directory. What do I need to do then? Change the configuration file? Or execute a command?

you can change the model field that originally contains HF:// to point to your local path

you can change the model field that originally contains HF:// to point to your local path

Ok. I've started using local. Thank you very much

glad that local works, closing this for now

@panghongtao if you have suggestions to the documents (e.g. enhancing some docs to include reading from local), please feel free to send a PR,

@panghongtao if you have suggestions to the documents (e.g. enhancing some docs to include reading from local), please feel free to send a PR,
Well. I will send some suggestions for documentation. Because the company's model doesn't allow hugging face to be uploaded. So I'm using the model transformed path. You have given a speech in our company, my colleague told me. Thank you very much for responding to my question


GUI to dynamically change PWM.

It is very annoying to do thruster testing by writing each and every command on the terminal to test it. We can do it easily by dynamic reconfiguration.

the GUI has already been developed ...call saini ..... !!!!! also you can use the rqt monitor to change the values on topics....

For rqt monitor use rqt_publisher ...... all topics will appear one by one and u can change the values being published by just one click !!

But that is in the development stage, we don't know how stable that is and creating a gui using dynamic reconfigure is very easy. We will also not require any other dependency, plus it is. For using rqt, we will need to install Qt.

I am confused what exactly is wanted from the thruster testing GUI? By testing, you mean just to put PWM and see if the thruster works or not? In that case @pkhrag had made a GUI for this. It is available here.
Dynamic reconfiguration can be done but again it is used for parameters to your node instead of data to which your node subscribes to. Now, PWM would be topic messages while PID parameters are best suited for dynamic reconfiguration. So using dynamic reconfiguration is not suitable if you just want to pass PWM values. to node.

What would be better to have a joystick controller for our vehicle through which we can give motion commands like (surge/sway/heave/roll/pitch/yaw) instead of each thruster individual PWM.
Also for checking if the thruster is connected or not, we can use the status pins in motor drivers to check whether everything is fine at hardware level or not. While this was missing in Varun, I believe Nilay and Saksham are doing that in the new actuation board for the vehicle.

I think GUI which @pkhrag has made is a keyboard controller, as I can infer from here, to control the motion. So, it doesn't control individual thrusters. We need to dynamically set the PWM sent to thrusters too, for our testing purposes which we are unware of. The issue topic is misleading, sorry for that. Changing it right know.

As pointed out by @Suryansh470 there is already a ros package rqt_reconfigure can be used for dynamically publishing values on a topic. So, closing this issue.

there is another ros package of rqt to monitor that ..... just see that


[FAQ] - How to set an api key

Can you please describe how to  create a key  for this
I suppose some service should be enabled as well
provider "vyos" {
  url = "https://vyos.local"
  key = "xxxxxxxxx"
}

Read VyOS docs https://docs.vyos.io/en/latest/configuration/service/https.html


Need Dell decision on FRU EEPROM 'other areas'

Currently the firmware decodes the board info area.   The product area appears to me to be similar but has not been decoded.   It is about 8 hours of work to decode it.
Also the chassis area and internal use area - neither are decoded.
For some areas (including these 2) there is data that cannot be decoded into text (ASCII) so should those be displayed as hex bytes?

I'm not certain if reading the entire decoded FRU would be a must have.   Currently the FRU can be read by a different piece of gear.  I do think it would be a nice feature to have since reviewing FRUs is common to my work.
Quick question.  At one point I was able to use the 'eeprom' command without arguments and see the fru portion mentioned.  But lately I enter 'eeprom' I get the statement: Displays FRU EEPROM info areas if no args.
'eeprom  ' dumps  bytes @
I'm not sure what I am doing different.  Any thoughts?

Figured out to use the command EEPROM SHOW

After speaking with Jason K about this I think we can close it and keep as is.  Decoding the entire FRU would take time and the current capability is useful to demonstrate the FRU is there and working.  Worse comes to worse we can review this at a later time if the functionality is demanded.
It may be worth adding the "show" argument in the help menu.


Initial version of running build binaries via tarp

Currently a bit clunky need to do cargo tarpaulin --command Build --bin <bin_name>. I should be able to fix the --bin defaults so it's not needed. Just need to make sure I don't change anything that breaks it for tests.
So you should be able to (for a project with a single binary) run it with multiple sets or args and see how much you cover like so:
[run_help]
command = "build"
release = true
args = ["--help"]

[run_cmd_1]
command = "build"
release = true
args = ["tarpaulin", "--fail-under", "50", "--root", "test_project"]

Further testing still needed

I'm hoping that cargo will release it wouldn't have to rebuild the project for any but the first run, then for longer CLI apps they can run through some scenarios get coverage and then analyse logs or something for the pass/fail of the tests

This also coincided well with #610 so just going to merge now


List browser compatability

It would be nice if you could test and list browser compatibility somewhere. For example, this works in Chrome 16.x but not IE7.

k - I just pushed a fix for forEach and indexOf

I ran the mocha tests in IE7 (after downloading modifying remote file references) and everything appears to have passed.
When I run either _random or the todos example, the error is now:
SCRIPT438: Object doesn't support property or method 'map'
o_O.js, line 231 character 7

The _random.html sample still complains about:
SCRIPT5007: Unable to get value of the property '_id': object is null or undefined
o_O.js, line 441 character 3

ok - pushed a fix
On Fri, Feb 10, 2012 at 3:08 PM, dhm116
<EMAIL_ADDRESS>wrote:

I ran the mocha tests in IE7 (after downloading modifying remote file references) and everything appears to have passed.
When I run either _random or the todos example, the error is now:
SCRIPT438: Object doesn't support property or method 'map'
o_O.js, line 231 character 7

The _random.html sample still complains about:
SCRIPT5007: Unable to get value of the property '_id': object is null or undefined
o_O.js, line 441 character 3

Reply to this email directly or view it on GitHub:
https://github.com/weepy/o_O/issues/2#issuecomment-3907375

Perhaps using something like what is suggested here - may be a cleaner solution than your latest commit, which is explicitly overriding the functions even if they are built in.
I've also put the suggested solution in a gist here (would LOVE to know how to embed gists in comments)

Actually it's not overwriting them it's wrapping them:
"function indexOf(array, obj, start) {"
On Fri, Feb 10, 2012 at 3:27 PM, dhm116
<EMAIL_ADDRESS>wrote:

Perhaps using something like what is suggested here - may be a cleaner solution than your latest commit, which is explicitly overriding the functions even if they are built in.
I've also put the suggested solution in a gist here (would LOVE to know how to embed gists in comments)

Reply to this email directly or view it on GitHub:
https://github.com/weepy/o_O/issues/2#issuecomment-3907711

did that fix work ?

Looking much better!
I don't think the issue with '_id' or 'id' is related to the naming of the property. I still get the same error, which means that, for some reason, the object instance is null.

SCRIPT5007: Unable to get value of the property 'id': object is null or undefined
o_O.js, line 444 character 3

This happens even on IE9, so something else is going on here...

odd - I'll need to debug it properly.
On Fri, Feb 10, 2012 at 4:01 PM, dhm116
<EMAIL_ADDRESS>wrote:

Looking much better!
I don't think the issue with '_id' or 'id' is related to the naming of the property. I still get the same error, which means that, for some reason, the object instance is null.

SCRIPT5007: Unable to get value of the property 'id': object is null or undefined
o_O.js, line 444 character 3

This happens even on IE9, so something else is going on here...

Reply to this email directly or view it on GitHub:
https://github.com/weepy/o_O/issues/2#issuecomment-3908360

OK, turns out that 'object is null or undefined' issue was the result of a trailing comma:

var company = {
  people: o_O.collection([
    new Person('Bobby', 'Reado', 30 ),
    new Person('Bobby2', 'Readoo', 301, "green"),
  ])
}

as opposed to:

var company = {
  people: o_O.collection([
    new Person('Bobby', 'Reado', 30 ),
    new Person('Bobby2', 'Readoo', 301, "green")
  ])
}

Thanks for addressing the other errors! I think that probably closes this issue unless anybody else has found anything.

cool - added a section in Readme


Bump @govuk-pay/pay-js-commons from 1.4.0 to 2.2.0

I don't know why they all change, I guess I could revert the ones we don't care about, but should I be tinkering with it?

package-lock changes are mad, not worth looking 🤷♂


it takes too long before gulp.watch() returns when there are more than 4 glob descriptors

codes like below take 20s before the log comes out...
gulp.watch(["!build/**","!node_modules/**","**/*.js"], () => {});
console.log('watching');

There are about 1000 files matching the glob above.
I have to switch to gulp-watch package, and it works fine (which takes less than 1s).
I'm using the latest gulp, 3.9.1
My env is Win10, i7 cpu and 250GB SSD.
It's frustrating that the embeded API gulp.watch is so slow that we accutally created another widely used NPM package (gulp-watch) just because of the poor performance......

This is the watcher we are stuck with until gulp 4 is released.

As gulp@3 is the default version of gulp and the one of the four APIs is slow by default, why don't we just merge chokidar to gulp@3 as we do in gulp@4 and make gulp@3 more usable?
If there is a good reason for the "sticking with old watcher" strategy, could you give me a hint?
Otherwise I'm happy to send a pull request.

That completely breaks semver. We won't be doing that.


[HUDI-7107] Reused MetricsReporter fails to publish metrics in Spark streaming job

Change Logs
When shutdown all the metrics, clear the metric cache to avoid reusing already stopped MetricsReporter
Impact
No public API change.
Risk level (write none, low medium or high below)
low
Documentation Update
N/A
Contributor's checklist

[x] Read through contributor's guide
[x] Change Logs and Impact were stated clearly
[x] Adequate tests were added if applicable
[ ] CI passed

I need to fix unit tests, will update later

The test failures are as follows:
Error:  Errors:
Error:    TestHoodieInternalRowParquetWriter.setUp:61->HoodieSparkClientTestHarness.initSparkContexts:194 » Spark
Error:    TestHoodieInternalRowParquetWriter.setUp:61->HoodieSparkClientTestHarness.initSparkContexts:194 » Spark
Error:    TestHoodieRowCreateHandle.setUp:67->HoodieSparkClientTestHarness.initSparkContexts:194 » Spark
Error:    TestHoodieRowCreateHandle.setUp:67->HoodieSparkClientTestHarness.initSparkContexts:194 » Spark
Error:    TestHoodieRowCreateHandle.setUp:67->HoodieSparkClientTestHarness.initSparkContexts:194 » Spark
Error:    TestHoodieRowCreateHandle.setUp:67->HoodieSparkClientTestHarness.initSparkContexts:194 » Spark
Error:    TestHoodieRowCreateHandle.setUp:67->HoodieSparkClientTestHarness.initSparkContexts:194 » Spark

Looking at some other PRs, the tests look flaky

Thank you @danny0405 !


Add Salesforce Tutorial

Migrated from GitLab: https://gitlab.com/meltano/meltano/-/issues/341
Originally created by @jschatz1 on 2019-01-28 15:54:51

Add a Tutorial for extracting from Salesforce with Meltano and make sure that it works with the default install.

View 17 previous comments from the original issue on GitLab


cmd/go: vcs.* info can be inaccurate for Mercurial

What version of Go are you using (go version)?

$ go1.20.2 version
go version go1.20.2 windows/amd64

Does this issue reproduce with the latest release?
I didn't try, but looking at the current code, yes.
What operating system and processor architecture are you using (go env)?
go env Output
$ go1.20.2 env
set GO111MODULE=
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\Matt\AppData\Local\go-build
set GOENV=C:\Users\Matt\AppData\Roaming\go\env
set GOEXE=.exe
set GOEXPERIMENT=
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOINSECURE=
set GOMODCACHE=C:\Users\Matt\go\pkg\mod
set GONOPROXY=
set GONOSUMDB=
set GOOS=windows
set GOPATH=C:\Users\Matt\go
set GOPRIVATE=
set GOPROXY=https://proxy.golang.org,direct
set GOROOT=C:\Users\Matt\sdk\go1.20.2
set GOSUMDB=sum.golang.org
set GOTMPDIR=
set GOTOOLDIR=C:\Users\Matt\sdk\go1.20.2\pkg\tool\windows_amd64
set GOVCS=
set GOVERSION=go1.20.2
set GCCGO=gccgo
set GOAMD64=v1
set AR=ar
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=NUL
set GOWORK=
set CGO_CFLAGS=-O2 -g
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-O2 -g
set CGO_FFLAGS=-O2 -g
set CGO_LDFLAGS=-O2 -g
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=C:\Users\Matt\AppData\Local\Temp\go-build3808080945=/tmp/go-build -gno-record-gcc-switches

What did you do?
I ran go1.20.2.exe version -m on my executable.
What did you expect to see?
        build   vcs=hg
        build   vcs.revision=76cef81bcff371c88d277f17c712ecf22b8c83e7
        build   vcs.time=2023-10-12T23:09:25Z
        build   vcs.modified=true

What did you see instead?
        build   vcs=hg
        build   vcs.revision=82be5bac456b8fbde0268380e336fe2fbd1294e7
        build   vcs.time=2023-09-28T04:18:18Z
        build   vcs.modified=true

It looks like 76cef81bcff371c88d277f17c712ecf22b8c83e7 introduced a regression when it switched from identify -i (which produces information about the current checkout) to log -l1 (which prints info about whatever the topmost commit is, and may be a totally different branch). [1]    The fix here is to simply add a -r. argument to the log command to look at the current commit, and that will fix both vcs.time and vcs.revision.
Typically, the topmost commit is whatever the developer is working on, but it won't be if updating to an old commit, or pulling changes from the server.  While these might be edge cases, creating a tag creates a new commit, and CI typically updates to the commit that was tagged.  That means this info will be wrong for any tagged builds (which is how I noticed this).
There are two other correctness issues here:

The status command should have a -S argument, so that it recurses into any subrepos when checking for modified files.[2]

HGPLAIN=1 should be set in the environment when executing any commands, to prevent unexpected command output changes.[3]  For example, I have ui.tweakdefaults=True in my configuration, so at the end of hg status listing any file changes, it can print info about the current state of the repo, like this:

# The repository is in an unfinished *bisect* state.

# To mark the changeset good:    hg bisect --good
# To mark the changeset bad:     hg bisect --bad
# To abort:                      hg bisect --reset

Given the current code of checking for any status output, that would incorrectly flag the build as modified.
[1] https://github.com/golang/go/commit/76cef81bcff371c88d277f17c712ecf22b8c83e7#diff-50574a97a14d32e7cc56ad1ba6de8c554e491c48a3718e599cfc8cc0ab2858e4R166
[2] https://github.com/golang/go/blame/cc13161a0d62fc1deab019996c17a7da1ff6a7f1/src/cmd/go/internal/vcs/vcs.go#L211
[3] https://repo.mercurial-scm.org/hg/file/6.5.2/mercurial/helptext/scripting.txt#l45

@mharbison72, given that you seem to understand the problems here, would you like to send a fix?
(I'm not sure that I have a solid enough understanding of the Mercurial command line to do it properly — especially to write a regression test that would correctly reproduce these bugs.)

@mharbison72, given that you seem to understand the problems here, would you like to send a fix?
(I'm not sure that I have a solid enough understanding of the Mercurial command line to do it properly — especially to write a regression test that would correctly reproduce these bugs.)

I'd be happy too, but I haven't contributed here before.  I'm reading the contributing doc and need to get setup first.
I'm not sure where you are in the release cycle- is there a particular branch I should target to get these fixes backported?  Also, where are the tests I should add to?  I see src/cmd/go/internal/vcs/vcs_test.go and src/cmd/go/testdata/vcstest/hg which I assume is what you mean, but I'm not sure how to run them.  Is the typical workflow to just push to a PR, and let tests run on GH?

I'm not sure where you are in the release cycle- is there a particular branch I should target to get these fixes backported?

All fixes should land on the master branch first; then we can decide about backports.

Also, where are the tests I should add to?

src/cmd/go/testdata/script — see in particular src/cmd/go/testdata/script/version_buildvcs_hg.txt.
They are run as subtests of cmd/go.TestScript, so you can invoke them individually like:
$ go test cmd/go -run=TestScript/version_buildvcs_hg

Thanks for the pointers @bcmills.  I'm probably missing something obvious, but the tests seem to sail right over the changes I'm making to version_buildvcs_hg.txt when I run the test command in verbose mode, with caching disabled:
$ time GOFLAGS="-count=1" go test -v cmd/go -run=TestScript/version_buildvcs_hg

Also, if I understand how the scripts work, they aren't currently testing the revision or time value at all, other than that it's present:
 stdout '^\tbuild\tvcs.revision='
 stdout '^\tbuild\tvcs.time='
 stdout '^\tbuild\tvcs.modified=false$'

I can see why that was done- the current time and username are part of the commit hash, so they aren't fixed from run to run.  They can be set to specific values in the test environment like in the Mercurial test setup if you want to go that far (and go.exe doesn't clear them somewhere), but I don't want to leave you with a bunch of complexity.  Without it though, there's really no way to add regression tests for this.

I figured out my problem- I was running the global go instead of the local one to run the tests.
LMK if the more elaborate revision matching in the tests is what you had in mind, or if you meant something else.


E2E: Update Sidebar feature test to spec

Requirements

Sidebar only appears expanded by default in desktop
Sidebar is collapsed in mobile devices.


chore(deps): upgrade dev dependencies

Upgrades project dependencies. See details in workflow run.

Automatically created by projen via the "upgrade-dev-dependencies-k8s-29-main" workflow

⚪ Backport skipped
The pull request was not backported as there were no branches to backport to. If this is a mistake, please apply the desired version labels or run the backport tool manually.
Manual backport
To create the backport manually run:
backport --pr 4383

Questions ?
Please refer to the Backport tool documentation


Bugfix in LabelProduct

Codecov Report

Merging #111 (979f72c) into computgraph_kun (0d2cac2) will decrease coverage by 2.09%.
The diff coverage is 0.00%.

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
@@                 Coverage Diff                 @@
##           computgraph_kun     #111      +/-   ##
===================================================
- Coverage            56.63%   54.54%   -2.09%
===================================================
  Files                   54       47       -7
  Lines                 3069     2849     -220
===================================================
- Hits                  1738     1554     -184
+ Misses                1331     1295      -36

Impacted Files
Coverage Δ

src/frontend/LabelProduct.jl
0.00% <0.00%> (ø)

src/diagram_tree/common.jl
50.00% <0.00%> (-50.00%)
:arrow_down:

src/FeynmanDiagram.jl
41.66% <0.00%> (-33.34%)
:arrow_down:

src/parquet_builder/common.jl
56.45% <0.00%> (-25.64%)
:arrow_down:

src/diagram_tree/operation.jl
59.64% <0.00%> (-16.92%)
:arrow_down:

src/diagram_tree/DiagTree.jl
0.00% <0.00%> (-14.29%)
:arrow_down:

src/computational_graph/common.jl
33.33% <0.00%> (-7.85%)
:arrow_down:

src/diagram_tree/tree.jl
55.69% <0.00%> (-6.81%)
:arrow_down:

src/common.jl
60.95% <0.00%> (-3.02%)
:arrow_down:

src/frontend/GV.jl
0.00% <0.00%> (-2.64%)
:arrow_down:

... and 43 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


Enable staticcheck for this repo

Why you need it?

staticcheck is useful, it can help to do go static analysis, detect bugs, performance issues, and much more. A lot of repos use staticcheck as a code check tool.
Ref: https://github.com/dominikh/go-tools
If this makes sense, I'd like to help introduce the staticcheck.
How it could be?

Other related information

If this makes sense, I'd like to help introduce the staticcheck.

I totally support that we need this. Code quality is one of the most important things we emphasize.

OK! @allencloud
I will send a PR to deal with this. :smile:


how to define obstacle set

I'm trying to change the obstacle set.
I notice from the code that there are at least two means of defining obstacle set:
a) using several vertex
b) using a .nav file
And I have some questions.
a) is there a convenient way to generate the coordinates of vertex instead of typing it one by one ?
b) how to generate a .nav file?
c) is there any other ways of defining obstacle set as I didn't find the documentation for that.
Thanks in advance!

I agree; editing XML files is a nuisance. I almost never do that. I use a number of informal utilities to do the job. You can get access to them here (https://github.com/curds01/MengeUtils). These are currently all python 2 utilities (I need to upgrade them to python 3).
You have several options:

If you have access to a 3D modeling package, you can model your world as objects. Then you can automatically slice that scene with a plane and that will generate the definition of an ObstacleSet.
I have a utility for drawing obstacles by hand and then saving the obstacles out as an ObstacleSet.

Let me know which one you prefer and I can elaborate on that mechanism.

Thanks for your answer.
For the first option, which 3D modeling package should I install?
I'm more interested in the first one.
But the second one sounds easier to use(I don't know if it's true or not)

For option 1, if you don't already have a modeling package that you are familiar with, it's probably not going to be a very good option. Essentially, you'd have to learn how to model. I find it useful because I'm familiar with the tools. And I generally model a visualization and simulation version of the environments I'm interested in. If you're interested in this route, you can download blender -- an open-source 3D package.
Option 2 is more accessible. The problem is that the tool is quite crude. You can't do precision (for example, putting a vertex at an exact location). And you'll have to do some tricks to make sure you're creating obstacles at an appropriate scale. But it should get you up and running -- it depends on how complex your simulation environment is and how precise it needs to be.

In this case, I think I might start with the second one.
So it is included in the utilities you provided above right? Could you please show me how to use it.
Thank you very much!

So, checkout the repo of utilities outlined above.

Go to the directory you've checked it out into.
Execute python roadmapBuilder.py (assuming you've got python 2.7 in your path). You should get a window like this:

If you hold the 'h' key down, it will display context-sensitive information.

You will want to go into the "ObstacleContext" to create/edit obstacles. But first, we're going to go into the AgentContext to create an agent to help define scale.
4. Hit the 'a' key to enter the AgentContext.
5. In the middle of the window, click the mouse. A very, very small agent will be created. Zoom in on the agent (using the mouse wheel).
6. Hit escape to leave the agent context.
7. Hit 'o' to enter the obstacle context.
8. Hit the 'h' key to display the help for obstacles:

It should say "New Polygon" at the bottom and "ObstacleContext" at the top. Plan on using the 'h' key a lot. It'll explain the details.
Obstacle editing has two modes:

Drawing obstacles (reach this by hitting the '1' key.)
Edit existing obstacles (hit the '2' key). At the bottom of the screen it will tell you what mode you're in.

Drawing obstacles

Every time you click a vertex will be placed and a polygon will be gradually constructed. A vertex will be yellow. Once you have two, an obstacle will be shown (as a red line). Once you have three or more you can see the whole polygon.
A polygon will be colored red or blue, depending on whether agents should stay outside or inside the obstacle, respectively.
To finish one obstacle and start another, right click.

This should get you started, I strongly suggest you experiment with the documented features (as shown when you hit the 'h' key).

Thanks for your thorough answer which helps a lot. And the tool works really well!
And I'm learning how to model 3D world using blender right now.
If I'm not wrong, the utility you provided can slice the 3D world with a plane and generate a ObstacleSet, right?
So what I have to do is to model a 3D world using blender and then use the utility you provided to slice it into a ObstacleSet, right?

I'm glad it's working for you.
In Blender, after you've modeled your scene, export it as an .obj file (wavefront obj). Then you can pass it as input to the program ObjSlice.py. It has instructions for how to use it (python ObjSlice.py --help). It's pretty straightforward.

hi!
I'm also facing a similar situation in which I'm creating my own scenario with obstacles. so I have three questions:

you mentioned that in the roadmapBuilder.py, the red and blue polygons indicate whether agent should be inside or outside... how to change it? i drew 5 polygons and they turned in each color respectively (first polygon -blue, second polygon - red, third polygon - blue, ..etc)  how to turn them all into red so the agent doesn't enter anywhere to the obstacles?

2 - Before reading this post, I changed manually the vertices in the xml so I could have an octagon in the center. When menge_sim got executed, the agent could enter the octagon but then it couldn't leave! How should I understand this? or how to modify it so the agent can not access to the obstacles?
3- besides this Scene xml file, is there any other consideration to take into account as for the obstacles placement? what about the nav file?
Thank you very much for your kind attention in advanced!

Reversing the direction of obstacles:

Hit 'o' to enter obstacle mode.
Hit '2' to enter obstacle edit mode.
Hit 'p' to enter polygon edit mode.
Move the mouse over the obstacle (it highlights yellow)
Hit the 'r' key to reverse the winding.
Move the mouse -- color has reversed.

Manually editing an xml file has to be done in conjunction with all the various components (are agent positions still valid? Is the roadmap still valid?  etc.)

You only have to worry about a .nav file if you're using the navigation map in a velocity component. Authoring a velocity component is quite complex.


Added political geography of uk and england in csv

Please merge my file and add hacktoberfest tag to make me eligible for swags. Thanks

@ManashAnand
Congrats 🎉, your PR got merged. To be eligible for the swags from ClueLess. Follow these steps.

Star this repository. ⭐
SignIn / SignUp in [ClueLess Website](https://www.clueless.tech/) .
Go to the profile section from the top right-hand corner.

Comment your profile URL under this comment tagging the person who merged your PR.

That’s it. Follow the leaderboard!
And don’t forget to share your achievement with your friends.


Support testing with non text response bodies (e.g. images)

Hi,
I have a possibly unusual use-case for this library: testing some very complex redirect/proxying logic that is translating a complicated URL structure for one thumbnail generating service to another we want to use.
I've used pyresttest for similar things before and would really like to again to have robust suite of validation for every type of URL we must support.
The problem is that the new proxy, when it works, is returning image data. If the test passes all is OK provided we make no assertions based on the body content, but in the common case where it fails this line: https://github.com/svanoort/pyresttest/blob/ae0a1215696958726a61cd2e7e9baaf8ff08baa2/pyresttest/resttest.py#L431
    # Print response body if override is set to print all *OR* if test failed
    # (to capture maybe a stack trace)
    if test_config.print_bodies or not result.passed:
        if test_config.interactive:
            print("RESPONSE:")
        print(result.body.decode(ESCAPE_DECODING))

Causes a problem - the decode call throws exceptions trying to decode binary data.
I can think of a couple of possible solutions:

(work around) change it so that if --print-bodies is explicitly set to false rather than just unset, this line is skipped
Detect Content-Type of response and only try to print it if it's a known-text format (i.e. text/* or application/json, maybe find a white list of common ones?
Detect Content-Type of response and black-list common, known-binary types like image/* and application/octet-stream etc.
Create a new per-test config that allows you to specify this explicitly, perhaps expect_binary_body: true

Which of the above (or another alternative) is preferred? If there is a response soon enough I can probably provide a PR implementing any of those.
Thanks for the great tool!

@banks Thank you for reporting this and giving the detail.  If you can put together a PR addressing this, it would certainly be welcome and I'd be happy to review/test/merge as appropriate.  I am aiming to set aside more time in the next few weeks for PyRestTest (free time tends to come in bursts).
This bug was definitely an unintended consequence, here are my thoughts on your solutions:

I don't like this option, because it is tied to the current implementation, and the whole mechanism behind them will changing massively with #101
This one is smart, and I think the best option, given that the result object created during execution has a field for headers.  I'd probably match on text/, xml, json, and then go from there.  In cases where options would print a body but it does not match one of these, I would print a line like "Response body not printed: content-type XXX/YYY does not match patterns ['text/',  'xml', 'json'] so assumed to be binary."
Too many different MIME types to blacklist, it will just get messy.
This is a thought, but I really want to avoid adding too many configuration options here, and rely on sensible defaults.

I think the best solution is a combination:

Create a function for this, that does the printing and is invoked during execution (and is easy to unit test alone).
Define a list of regex patterns within the resttest.py class: PRINTABLE_CONTENTTYPE_PATTERNS which are matched against  content-type
Function does a comparison against pattern list, if it is in the list, print the body, otherwise print the message above about it probably being binary (and doing a string-formatted output of the patterns list).
If an exception is thrown during decoding of body for printing, catch the exception and print a message like "Failed to print body due to {Exception type}"
Remember we have to handle both raw bytes and Unicode, on Python 2.6, 2.7, and 3.x.  There are helper functions in place to simplify this already.

I think we're getting far enough along that if a PR for  this + cookie extractor are merged in, it might be time to cut another small release (and to heck with quirks in version numbering of milestones).  Potentially merging in my execution/logging refactors if I can, since otherwise future work will explode with merge conflicts.
Anyway, looking forward to hearing your thoughts (possibly in PR form).
Cheers!
Sam

Thanks for the response Sam,
Your proposal makes a lot of sense and was my preferred option too.
I will see how this week pans out - I'm in an interesting position right now where I'm actually handing this work on to another team in the next couple of weeks so it's not clear if I'll have time to work on a PR for this or not before I'm no longer working on the project.
It sounds like it's not a huge amount of work but at least a little overhead to refactor and add suitable tests etc. I will likely know in next few days if this is likely to be possible for me to do or not and let you know (unless you have a moment and inclination to work on it yourself).
Thanks for the great tool

@banks Absolutely, I know how that work gets shuffled around.  I'll probably do a quick fix for the immediate problem (catch exceptions when printing body and log that it tried to print the body and failed), so your tests don't bomb out while running.  If you can slip in a PR with content handling, that's wonderful and welcomed, if not, I'll convert this to a feature request to work on soon.
Anyway, glad you've been finding PyRestTest quite useful, and think you'll like where it's going in the future ;)


Add files via upload

It looks as if you are computing the arithmetic mean, please use the geometric mean.


CLA not signed shows build as red (red cross ❌) when an anonymous Googler uploads a PR

Problem
the CLA/google check is red if somebody whom is a Googler (working at Google) uploads a commit.

This makes it look like the build failed on first glance.
Expected behavior
"Googler" does not need to sign the CLA, build is green.

This is now fixed via a google-internal  change


(shortfin-sd) Server segfaults at startup on python3.13 free-threaded

in a python3.13t venv:
pip install shortfin --upgrade --pre -f https://github.com/nod-ai/SHARK-Platform/releases/expanded_assets/dev-wheels
pip install transformers pillow dataclasses-json
python -m shortfin_apps.sd.server --help

Using shortfin 2.9.1rc20241114 for 3.13t

Potential culprit:
Python 3.13.0 experimental free-threading build (main, Oct  8 2024, 08:51:28) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import fastapi
Segmentation fault (core dumped)


cmake: app: set zephyr_extra_modules from environment

This PR adds the possibility to specify ZEPHYR_EXTRA_MODULES from an
environmental variable. To add a custom module mymodule in
path/mymodule the variable ZEPHYR_EXTRA_MODULES can be set in .zephyrrc.
Signed-off-by: Laczen JMS<EMAIL_ADDRESS>

The code follows convention and enables a new use-case.
But it needs to be documented.

I updated the docs as well (hope I didn't break anything).

@dbkinder, thanks for the help.


hpp-fcl: 2.4.4-1 in 'humble/distribution.yaml' [bloom]

Increasing version of package(s) in repository hpp-fcl to 2.4.4-1:

upstream repository: https://github.com/humanoid-path-planner/hpp-fcl.git
release repository: https://github.com/ros2-gbp/hpp_fcl-release.git
distro file: humble/distribution.yaml
bloom version: 0.12.0
previous version for package: 2.4.0-1

Holding for Humble sync


Translated 3 page into Bengali

Thanks for your hard work nasir vai. But please refer to the following issue.
We follow those rules for contributing here. Please put a comment on that issue that you want to do these pages.
And please put one PR per file.
Thanks again for your upcoming colaboration.

Thanks for your hard work nasir vai. But please refer to the following issue. We follow those rules for contributing here. Please put a comment on that issue that you want to do these pages. And please put one PR per file. Thanks again for your upcoming colaboration.

Thank you aswell nafis bhai, i will keep that in mind. actually its my mistake, i did not read clearly th rules.

Thank you as well, Nafis Bhai. I'll keep that in mind.


side : bottom-left

Hi,
The version 2.0 was supported : position : 'bottom-left', but the v4.0 not support side : 'bottom-left'.
We get this error :
jquery.tooltipster.min.js: Uncaught TypeError: Cannot read property 'width' of undefined
Thanks

Hi, it used to be supported yes, but not anymore. Maybe someone will write a plugin for that some day, it's entirely possible. Thanks

Hi, you can use the functionPosition property to do the same thing, like this:
$tooltipable.tooltipster({
    side: 'bottom',
    functionPosition(instance, helper, position){
        const helperRight = helper.geo.origin.offset.right
        const width = position.size.width
        position.coord.left = helperRight - width
        return position
    }
})


API endpoint for available vectordbs

There is a API endpoint for getting a list of available LLMs similarly it would be helpful to get a list of accepted vectordb, to avoid referring to the scripts for the exact keyword/accepted format of the vectordb name.

The new api has been exposed


[wip] Port integration_launch

This package is very small, I did what I could now but I couldn't run the package as it's blocked by

[ ] autoware_launch
[ ] planning_simulator_launch

Now our latest code is synced. https://github.com/tier4/AutowareArchitectureProposal_launcher/commits/main


Change test case for internationalization to refer to resource name instead of absolute text values

Test cases for internationalized log messages are fragile because they check the end text value. If we check the resource name instead, the tests will be more stable.

Hi @neopragma
Please elaborate.
Preferably with examples on how to reproduce.
/Rune


add map update subscriber

I added it because I wanted to update the map while working on a multi map project.
!! You have to write the absolute path of the PCD file as msg.data.

Thanks for your PR. It look l looks good and I merge it.


refactor: annotations

Type of change:

[ ] Bugfix
[ ] New feature provided
[x] Improve performance
[ ] Backport patches

What this PR does / why we need it:

Decouple the transformation of resources and annotations.
Improved code scalability.

Pre-submission checklist:

[ ] Did you explain what problem does this PR solve? Or what new features have been added?
[ ] Have you added corresponding test cases?
[ ] Have you modified the corresponding document?
[ ] Is this PR backward compatible? If it is not backward compatible, please discuss on the mailing list first

Codecov Report

Merging #1170 (71fc302) into master (628abb9) will decrease coverage by 0.10%.
The diff coverage is 55.42%.

:exclamation: Current head 71fc302 differs from pull request most recent head c85158f. Consider uploading reports for the commit c85158f to get more accurate results

@@            Coverage Diff             @@
##           master    #1170      +/-   ##
==========================================
- Coverage   30.45%   30.34%   -0.11%
==========================================
  Files          81       81
  Lines        9839     9833       -6
==========================================
- Hits         2996     2984      -12
- Misses       6517     6522       +5
- Partials      326      327       +1

Impacted Files
Coverage Δ

...kg/kube/translation/annotations/plugins/plugins.go
0.00% <0.00%> (ø)

...g/kube/translation/annotations/plugins/redirect.go
12.50% <20.00%> (ø)

...kg/kube/translation/annotations/plugins/rewrite.go
11.11% <20.00%> (ø)

pkg/kube/translation/annotations/plugins/csrf.go
16.66% <25.00%> (ø)

...e/translation/annotations/plugins/authorization.go
22.22% <33.33%> (ø)

pkg/kube/translation/annotations.go
58.82% <72.72%> (-2.72%)
:arrow_down:

pkg/kube/translation/ingress.go
59.43% <76.19%> (-1.48%)
:arrow_down:

pkg/kube/translation/annotations/plugins/cors.go
100.00% <100.00%> (ø)

...be/translation/annotations/plugins/forward_auth.go
100.00% <100.00%> (ø)

...e/translation/annotations/plugins/iprestriction.go
100.00% <100.00%> (ø)

... and 5 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 628abb9...c85158f. Read the comment docs.

we have enterd v1.5 release window, I will put this one to v1.6, thanks


chore(Topology/LocallyClosed): move around

move definitions to Topology/Defs/Basic;
move the last lemma to Topology/LocalAtTarget.

Forgive my ignorance, but is there an obvious reason the import chain should be reversed in the case of LocallyClosed and LocalAtTarget?

I think that LocallyClosed is a basic predicate like IsOpen or IsClosed while LocalAtTarget depends on Topology.Sets.Opens which depends on a lot of stuff.

bors d+

bors merge


feature request...

Tried it, works well, I have a few suggestion ... like pure leech mode.. it would delete the torrent only once the sharing / seeding ratio hits 0.0 (qttorrent has this mode.. kind of.)
it needs anonymous mode, where it can require encryption before downloading...
But it is VERY memory friendly and most definitely not gaudy at all, it is simple, easy and just works, thanks for the app!

Thanks for the requests! I'll probably split this into two issues later on.

I've added the option to force encryption, it will be released with v0.11. If pure leech mode is something necessary, please open another issue and describe in detail what it entails.


Develop upstream sync 190429

Weekly Merge 20190429
Conflict files:
both modified:   tensorflow/compiler/xla/service/gpu/BUILD
both modified:   tensorflow/compiler/xla/service/gpu/kernel_thunk.cc
both modified:   tensorflow/core/BUILD
both modified:   tensorflow/core/kernels/BUILD
both modified:   tensorflow/core/kernels/reduction_ops.h
both modified:   tensorflow/core/kernels/stateful_random_ops_cpu_gpu.h
both modified:   tensorflow/core/util/gpu_device_functions.h
both modified:   tensorflow/core/util/gpu_kernel_helper.h
both modified:   tensorflow/core/util/gpu_launch_config.h
both modified:   tensorflow/core/util/port.cc
both modified:   tensorflow/core/util/port.h
both modified:   tensorflow/examples/speech_commands/BUILD
both modified:   tensorflow/python/kernel_tests/conv_ops_test.py
both modified:   tensorflow/python/kernel_tests/pooling_ops_test.py
both modified:   tensorflow/stream_executor/rocm/rocm_blas.cc
both modified:   tensorflow/stream_executor/rocm/rocm_diagnostics.cc
both modified:   tensorflow/stream_executor/rocm/rocm_driver.cc
both modified:   tensorflow/stream_executor/rocm/rocm_gpu_executor.cc
both modified:   tensorflow/stream_executor/rocm/rocm_platform.cc
both modified:   third_party/gpus/rocm_configure.bzl
both added:      third_party/rocprim.BUILD

Three failed cases for rocm CI, starting from those:
//tensorflow/python/kernel_tests:map_stage_op_test_gpu
//tensorflow/python/ops/parallel_for:math_test
//tensorflow/python/ops/parallel_for:math_test_gpu

jenkins: retest this please

jenkins: retest this please

After sync up with @parallelo  we believe some of the CI nodes are not behaving correctly, kick off another round of CI check.

both rocm and rocm-v2 CI are passing, the two failed rocm-xla CI check was due to timeout.
Looking into the Cuda path now.

jenkins: retest this please

Two failures in rocm-xla CI check:
//tensorflow/compiler/tests:svd_op_test_gpu

time out
//tensorflow/compiler/xla/tests:token_hlo_test_gpu
seg fault

@sunway513 token_hlo_test should be safe to ignore, please refer to this comment

Thanks, @jerryyin , I've also synced with @deven-amd  and we can put no_rocm tag on svd_op_test_gpu and look it back when investigating the similar timeout issues.
Let me commit the flags and close this weekly merge.

@sunway513 Why is it closed instead of merged?

I clicked the merged button, and the commits were merged in the develop-upstream branch:
https://github.com/ROCmSoftwarePlatform/tensorflow-upstream/commit/2114c2fb385d9223732efe7750a987e9b32b732f
However, this PR was still showing as unmerged, something was not right for github. It looks to me this repo is getting too large for github to handle, I can often see the unicorn error.

Thanks for the clarification

Sure, let's clean the idled branches and hope that can help.


Missing basic strategy charts

Most of the charts are missing for anything other than double deck. Ideally we have charts for:

1 deck DAS
1 deck DAS LS
2 deck DAS
2 deck DAS LS

Up to 4, 6, and 8 decks. Also variants for H17 and S17. This is basically a data entry task unless we find a way to compute these based on EV.

Fixed by https://github.com/mhluska/blackjack-simulator/commit/d85c055ee0d2c27817e6aea901fbb10064c98636


It seems vscode merges theme rules with same colors but different font styles

VSCode Version: 1.52.1 (user setup)
OS Version: Windows_NT x64 10.0.19041

It only happens if one of them doesn't have fontstyle.
Steps to Reproduce:

Create a theme with at least two different rules but with the same color
Set font style of one of them to italic for example
Now test the theme. The token without italic rule is now styled italic.

Does this issue occur when all extensions are disabled?: extension related

Not a bug, My mistake. It is inherited. Realized after reading this

Not a bug, My mistake. It is inherited. Realized after reading this


Fix Request Manager on Mainnet

The RequestManager is misconfigured and we need to redeploy the current Mainnet RequestManager.
Action plan

[x] Think again if only deploying RequestManager is fine @istankovic @fredo
[x] Write deployment config @fredo
[x] Deploy RequestManager @istankovic
[x] shutdown agents @istankovic
[x] Do two requsts on old deployment @fredo
[x] Withdraw one @fredo
[x] Deprecate RequestManager @istankovic
[x] Withdraw the other @fredo
[x] Try a new request on deprecated contract @fredo
[x] Update the deployment.json & affected ABI file(s) @istankovic
[x] Update frontend config @GabrielBuragev
[x] Redeploy frontend @GabrielBuragev
[x] Pull repo of agents @istankovic
[ ] Create a process of checking the values before deployment @fredo @andfletcher
[ ] Create a process to double/sanity check configured contracts after deployment @fredo @andfletcher

Also added this to this sprint @andfletcher

Regarding RequestManager deployment:

Also change MAX_VALIDITY_PEROID to 30 min (must be changed in the contract)
Add finality period of Boba via setFinalityPeriod (288, 604800)
change transfer limit to 50 USDC via updateTransferLimit(50000000)
change feeData via updateFeeData 1 cent minLpFee (value = 10000)

Might it make sense to move the last two items in the description into their own issue?

Might it make sense to move the last two items in the description into their own issue?

I was planning the same thing, as it isn't necessarily critical for mainnet

created #961, will close this issue


Bolster unit testing for iam_allowed_bindings

Current testing suites are a minimal suite for rapid development. New tests need to be created/added for a production run.

From documentation - hashicorp/google provider:
Warning:
You must specify the role field using the legacy format OWNER instead of roles/bigquery.dataOwner. The API does accept both formats but it will always return the legacy format which results in Terraform showing permanent diff on each plan and apply operation.
This is for google_bigquery_dataset_access: Legacy format must be specified for the "role" field, while CAI uses modern terminology (ie. roles/owner). See how to remedy this in rego, although it's not time sensitive (no projects use this functionality yet)


Add RTL support to Drawer

Fixes #5215

As a mostly-unrelated aside, I learned last week that MD specifies a second drawer as well, coming from the opposite side.  They call it a "right nav":
https://github.com/flutter/flutter/issues/11388

They call it a "right nav":

Yeah, we probably need to abstract that" into separate property that we flip around based on the TextDirection, like we do for scrolling.


Allow stubbed connection to receive on_shutdown

This fixes an improper test failure as a result of upgrading logstash-mixin-rabbitmq_connection . This will fix failing tests found in https://github.com/elastic/logstash/issues/5606

@andrewvc - teeny tiny omission. In the CHANGELOG the 4.0.5 version is missing for the latest entry.


verifyRequestSignature does not create a signature that's the same as slack's

Description
verifyRequestSignature does not seem to be working, maybe doesn't work very well with the body-parser package
What type of issue is this? (place an x in one of the [ ])

[x] bug
[ ] enhancement (feature request)
[ ] question
[ ] documentation related
[ ] testing related
[ ] discussion

Requirements (place an x in each of the [ ])

[x] I've read and understood the Contributing guidelines and have done my best effort to follow them.
[x] I've read and agree to the Code of Conduct.
[x] I've searched for any related issues and avoided creating a duplicate issue.

Bug Report
verifyRequestSignature does not seem to be working, either debug steps should be created or an example snippet. Here is my code:
const { verifyRequestSignature } = require('@slack/events-api');

const { SLACK_SIGNING_SECRET } = require('../config');

const receiveWebhook = (botCtrl) => async (req, res) => {
  const { body } = req;
  const ts = req.headers['x-slack-request-timestamp'];
  const signature = req.headers['x-slack-signature'];

  payload = qs.stringify(body, { format: 'RFC1738' });

  return verifyRequestSignature({
      signingSecret: SLACK_SIGNING_SECRET,
      requestSignature: signature,
      requestTimestamp: ts,
      body: payload,
    });

  // always returns false
}

My node.js express code looks like this:
const bodyParser = require('body-parser');
const compression = require('compression');
const cookieParser = require('cookie-parser');
const cors = require('cors');
const express = require('express');
const methodOverride = require('method-override');
const morgan = require('morgan');
const path = require('path');
const { errorHandler: bodyErrorHandler } = require('bodymen');

const winston = require('./lib/logger');
const { error } = require('./api/responses');

module.exports = (apiRoot, routes) => {
  const app = express();

  app.use(cors());
  app.use(compression());
  app.use(morgan('combined', { stream: winston.stream }));
  app.use(express.static(path.join(__dirname, 'public')));

  app.use(bodyParser.json());
  app.use(bodyParser.urlencoded({ extended: false }));
  app.use(cookieParser());

  app.use(
    methodOverride((req) => {
      if (req.body && typeof req.body === 'object' && '_method' in req.body) {
        const method = req.body._method;
        delete req.body._method;
        return method;
      }
    }),
  );

  app.get('/favicon.ico', (req, res) => res.sendStatus(204));
  app.use(apiRoot, routes);

  app.use(bodyErrorHandler());

  app.use((req, res, next) => {
    const err = new Error('Not Found');
    err.status = 404;
    next(err);
  });

  app.use((err, req, res, next) => {
    if (err.status) {
      next(err);
    } else {
      console.error(err);
      error(res, err.message);
    }
  });

  return app;
};

Reproducible in:
@slack/events-api version: 2.1.1
node version: v10.12.0
OS version(s): macOS 10.12.6
Steps to reproduce:

Set up Slack Events API, and node.js express server, and entrypoint for slack events http request
Import the verification code above
Install app to your slack workspace
Go to the bot in your slack workspace and send a DM
Look at node.js logs and see it fail to verify the payload

Expected result:
verifyRequestSignature should return true
Actual result:
verifyRequestSignature returns false
Attachments:
None, just code.

I found the issue. You need to take the body exactly as Slack provides (in utf-8 Buffer form) and not using qs.stringify (if you look at the raw body, it looks more like a JSON.stringify())
Creating the signature with this works.


fix(@angular-devkit/build-angular): i18n with Ivy in universal builder

Ci failure should be fixed by https://github.com/angular/angular-cli/pull/15934