stage damage messaging to user

at the moment, I do not find the errors that are found during a stage-damage compute, we need to make sure that these errors find their way to the user.

messaging again is challenged by the Framework's error levels not bubbling up....

Validation groups does not work.

ImpactAreaStageDamage is a class that has data, and we can do calculations with that data. ImpactAreaStageDamage inherits ValidationErrorLogger, which has a three methods: GetErrors, LogErrors, and ReportMessage.
ImpactAreaStageDamage also implements IContainValidationGroups, which has a property that is a list of ValidationGroup, while a ValidationGroup consists of a list of ValidationErrorLogger.
Does this make sense?

We have objects that implement validation, whose properties implement validation, whose properties of those properties implement validation. It is a confusing system that has come about of the requirement to aggregate in the way that we do.
I think we need two separate classes. One is Validation, and another is an interface IHaveValidationProperties, which will have a get errors method that will cycle through each of those properties to get the errors. If any of those properties themselves are IHaveValidationProperties then the method will continue through separate concrete implementations of the get errors method.

We've tried to use one class for different needs. Silly beans!

And, not just get errors, but all of the information provided by Validation needs to bubble up, but only from that class that truly implements validation


Let martsy submit analytics events for Eigen where applicable

We need to figure out what we need to do to track events across martsy views. If the analytics endpoint simply needs a UUID for the active user, then we can just pass that to all martsy calls and have martsy send the events to snowplow?
This would mean that martsy should also pass that parameter around when linking to other views, to ensure that this is maintained when Eigen doesn’t provide native views for those views being linked to.
@craigspaeth Does that sound like something that’s doable on your end?

@craigspaeth Any ideas on this? Would love to get underway. /cc @alexanderfabry

I can see two ways to do this:

Martsy / force checks if it's hosted in eigen, then reports that in their own analytics
We expose an analytics object to the webview, martsy/force checks for its existence then sends along it's events to our object where eigen sends all the analytics

@orta I’d really prefer the former, it would cut down on requests the app has to make (which is better from our mobile pov) and most infrastructure already exists (assuming martsy tracks these events as well).
The only thing we’d need to add is a way for martsy to track a user that is not signed in, e.g. generate a UUID and pass that along.

Based on conversations with Ani, it seemed like they're looking for more of a funnel to track a user session from Native -> Martsy -> Native and so on. We would need to provide some sort of interaction between eigen/martsy then, wouldn't we?

Ah, @alloy just said that. Ok.

This sounds reasonable on the web end. Still kinda unclear exactly how we implement this. Right now the only thing we use to check against Eigen is the header. So we could use that to pass in a in_native_app param to our analytics calls. For logged out users maybe you can pass in another header like x-eigen-session-id and we can likewise pass that along analytics calls?
Anyways feel free to log an issue in Microgravity/Force explaining the details of what needs to be implemented on the web end, bring it up before user team priorities next Weds., and maybe this is something @jtotoole can work on next sprint.

Happy to close this now as microgravity segment events do carry context user agent properties which let us tag them as Eigen events.

@anipetrov Are you able to associate those events to specific Eigen users?

That's a good point @alloy -- when they are logged in it will be fine, but not when we have trial users. The thread is still valid then.

This is fixed by #1199.


Scale based on distance.

Is it possible to scale the POI views overlaid on the camera based on distance?

Hi,
Have you found solution to this?

@torok-zoltan @Rjmaurya13 @DanijelHuis  Anyone have solution on this? I want to remove that overlapping thing using some alternative either zoom in kind of feature or any other way... but right now issue is when multiple POI are very near to each other it creates a issue and not able to view each. As everyone is overlapping on it.


Problems on mobile phone

I know navigation on mobile phones isn't optimized yet. However I wanted to note the several things which may not have been seen/thought of:

[x]  impossible to check the remember box on the login page

[x] impossible to enable the "Computer Mode" (on Chrome at least) to see the site as a computer would.

Should be fixed in upcoming release :)


Added an item in TeX.gitignore

To ignore backup files generated by vim https://www.vim.org.
Reasons for making this change:
TODO
Links to documentation supporting these rule changes:
TODO
If this is a new template:

Link to application or project’s homepage: TODO

Do we really need to consider the temp file "~"? This is a quite general one.

They are left by vim, existing and detected by git even if vim has been closed.  This is why I'd like to ignore them.

Thanks, but given Global\Vim.gitignore already exists I want to avoid duplicating these rules in other templates.


[WIP] Spec

Some observations

Putting too much intermediate layers (performing data abstraction in multiple phases) would make things more complex. (need to define a language that is not C nor mathematical spec)
Concurrency abstraction (totally eradicating YIELD) might need some more discussion.
I think we can do it mechanically by exploiting some kind of memory-irrelevance.
That is why I abstracted data & memory first, and then concurrency.
Also, to prove this, our concurrency semantics might need some notion of liveness (starvation freedom) for each cpu.

I wrote the spec assuming flat memory model.
One thing to discuss: for data abstraction, we need "private memory" reasoning and I am not sure if we can do it nicely in flat memory model.


Support reset using DTR

Could LibreTuya support programmers that can reset the modules for programming with the DTR signal, like ESPHome does?  That would make flashing WB3S modules as brain-dead easy as flashing ESP-12F modules with ESPHome.
Thanks SO MUCH for this system.  Before now, I was transplanting ESP-12F modules onto Treatlife DS02S dimmers with WB3S modules.  Even after several operations, I was still very slow, and still ripped off solder pads.
LibreTuya helped speed up integrating these switches into HASSIO by 300%.  Now I just do enough desoldering to slip a piece of paper under the EN and VCC pads, solder five wires to the programmer, and run LibreTuya.  If LibreTuya toggled DTR before upload, I wouldn't even have to wait for the right moment to ground the EN pin.  Then I just solder the EN and VCC pads back to the main board and it's done!
The other roadbump I ran into was bk7231tools breaking on my Linux system, but that one was easy to fix.

Thanks for the PR!
LibreTuya's flasher is based on ltchiptool, our custom utility for flashing chips and managing binaries. When it establishes connection to the chip, first it checks if it's available already. Then, after 2 seconds, it does a hardware reset by toggling RTS and DTR lines, and tries to connect again. Does it work after waiting 2 seconds?
After next 2 seconds, it should print a connection guide in the terminal. Does it do that?

@zultron If you weren't aware, the Treatlife DS02S is supported by Tuya-CloudCutter for OTA flashing and avoids soldering altogether.

Thanks for the tips, that helped direct me as I did a little digging.  TLDR; I've gotten this to work, but lost track of exactly what might have changed, and now can't repeat the failures.  ;(  I'm out of town next week, but I'll be back at this afterwards.  Feel free to close this out.

Notes on what I did for when I pick this up again.  Some of this is from memory, so it's possible some facts are incorrect or omitted.
I'm using a burner like this.  I'm not 100% sure the schematic is the same as my burner, but experimenting with combinations of setting/clearing RTS and DTR give the following outputs:
DTR RTS  EN RST GPIO0
1   1   1  1   0
1   0   1  1   0
0   1   0  0   0
0   0   1  1   1

Flashing the WB3S by jumpering only GND, 3V3, TX, RX and EN (i.e. the above-described method of lifting 3V3 and EN pads  from the DS02S board and soldering wires) wasn't working yesterday without pressing the burner's RST button, which pulls EN & RST low.
Today, I tried flashing a few WB3S module plugged directly into the burner (i.e. WB3S modules completely removed from the DS02S board), and same results, burning failed.  I don't remember if I tried hitting the RST button during this activity.
Then I went and cloned the ltchiptool repo.  I couldn't find where DTR/RTS were manipulated.  (The ltchiptool.soc.ambz.util.rtltool.RTLXMD.connect() method manipulates DTR/RTS, but IIUC that's N/A for the WB3s, which uses the ltchiptool.soc.bk72xx module instead.)  In any case, I installed ltchiptool into the venv from git.
Next, I found the bk7231tools.serial.protocol.BK7231Protocol.reset() method, and tried toggling only RTS (using the above truth table), with no success.
Next, I flashed using the RST button, success.  Then I lose track, and can't remember whether I got any more failures, but unexpectedly, the flashing started succeeding at some point.  I found that when the chip detection succeeds, the ltchiptool.soc.ambz.util.rtltool.RTLXMD.connect() method never runs, so that's a red herring.
When I get back to this, I'll try flashing a few more modules still attached to the DS02S PCB.  If that succeeds, I'll wipe the entire install and see if I can retrace my steps (install following the LibreTuya docs from pip, no cloned git repos).

The buttons on the burner might be wired via transistors, which change the logic levels. I'm not sure how do you differentiate EN/CEN and RST, but there's only one "chip enable" pin on BK7231.
The CEN pin, short for chip enable, resets the chip when low, and allows it to run when high. The DTR and RTS pins are commonly used on ESP8266, which also requires toggling GPIO0 to put it in download mode.
In theory, only one line is needed to reset the chip. People are used to go with DTR->CEN; I like using RTS instead, because it's kind of like RST which makes me remember "this is the one to reset the chip" 😆
LibreTuya (and ltchiptool) toggles both DTR and RTS simultaneously. The code you found in RTLXMD is not applicable here, as it's for Realtek chips. The code in bk72xx has a flash_hw_reset() method, which is called after first 2 seconds of being unable to communicate with the chip. This method calls BK7231Tools's reset(), which in turn toggles the pins.
I have used that method some time ago; it worked fine, then it didn't work, then it worked again, then I didn't use it anymore, as I was flashing LT over LT, so I had the auto-reset feature. In the end, I'd say it should still work, but it can also be broken.

Back in town now, I just burned three more modules.  One thing that was confusing me, the LibreTuya firmware doesn't need reset by pulling EN low to flash, but the original firmware does.
I submitted a pull request against the bk7231tools repo.  BK7231Protocol.hw_reset() now puts a short pause between clearing DTR and RTS (also note the changed order) for these wacky burners.  That fixes the problem for me.
https://github.com/tuya-cloudcutter/bk7231tools/pull/14

One thing that was confusing me, the LibreTuya firmware doesn't need reset by pulling EN low to flash, but the original firmware does.

And that's a feature! :smiley:
Read more here: https://docs.libretuya.ml/docs/flashing/tools/adr/
Thanks for the PR, I'll try to review it soon, but remind me if I forget.

The buttons on the burner might be wired via transistors, which change the logic levels. I'm not sure how do you differentiate EN/CEN and RST, but there's only one "chip enable" pin on BK7231.

Sorry, the "RST" and "PROG" buttons on the burner got me confused, ignore that.  The burner controls EN and GPIO0 outputs separately, depending on combination the DTR and RTS signal inputs, according to the truth table above.

In theory, only one line is needed to reset the chip. People are used to go with DTR->CEN; I like using RTS instead, because it's kind of like RST which makes me remember "this is the one to reset the chip" laughing LibreTuya (and ltchiptool) toggles both DTR and RTS simultaneously.

That's a nice mnemonic.  OTOH, consider sticking with established convention (assuming DTR is the convention) so this sort of unexpected problem doesn't crop up.  Maybe a mnemonic for DTR, "Data Terminal Ready", is when it goes low, it resets the chip, which becomes not Ready.  :D
The code you found in RTLXMD is not applicable here, as it's for Realtek chips. The code in bk72xx has a flash_hw_reset() method, which is called after first 2 seconds of being unable to communicate with the chip. This method calls BK7231Tools's reset(), which in turn toggles the pins.
Thanks for the pointer!  I think I'd already found it when you posted, but that would have saved me some grepping around.

I have used that method some time ago; it worked fine, then it didn't work, then it worked again, then I didn't use it anymore, as I was flashing LT over LT, so I had the auto-reset feature. In the end, I'd say it should still work, but it can also be broken.

Please keep it!  :D
BTW, this code is very well organized.  I spend a lot of time with a large pile of spaghetti Python code, so I really appreciate this code.  Thanks!


80/popout component

What is the context of this PR?
Adds basic Popout component. This component can accept any other component as content, and any other component trigger for toggling visiblity of that content. It also supports animating content in and out of the DOM
How to review
Inspect code, run tests, play with storybook

 Pop-Out Component (5)


Add break-word to table cells

Fixes #414
This PR adds break-word to table cells to ensure table cells to not render larger than their intended size.
It has been mentioned that this adjustment deviates from GDS, as the table cells there do no specify break-word, I am assuming this is because this issue has not been experienced by any of their implementations, or if it has, it has been overridden during its implementation.
Before;

After;

[x] Ready to be merged

Deploy preview for affectionate-nobel-f3a8ec processing.
Built with commit bd06a1d34799ed9bd7f15e1c461a9a40c9b9bef9
https://app.netlify.com/sites/affectionate-nobel-f3a8ec/deploys/5ba25531dd28ef04ed4ab199

Hah, nice one with the deploy previews Ali, that is cool!

Liking the fix, the previews and their naming!

Codecov Report

Merging #415 into master will not change coverage.
The diff coverage is n/a.

@@          Coverage Diff          @@
##           master   #415   +/-   ##
=====================================
  Coverage     100%   100%
=====================================
  Files          77     77
  Lines         358    358
  Branches       16     16
=====================================
  Hits          358    358

Impacted Files
Coverage Δ

components/table/src/atoms/Cell/index.js
100% <ø> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b9e6fc5...bd06a1d. Read the comment docs.


fix(deps): update required_provider to latest for the deployable architecture solution

This PR contains the following updates:

Package
Type
Update
Change

helm (source)
required_provider
minor
2.13.2 -> 2.14.0

kubernetes (source)
required_provider
minor
2.30.0 -> 2.31.0

Release Notes

hashicorp/terraform-provider-helm (helm)
v2.14.0
Compare Source
ENHANCEMENT:

Add support for Terraform's experimental deferred actions [GH-1377]
helm_release: add new attributes metadata.last_deployed, metadata.first_deployed, metadata.notes [GH-1380]

hashicorp/terraform-provider-kubernetes (kubernetes)
v2.31.0
Compare Source
ENHANCEMENTS:

Add support for Terraform's experimental deferred actions [GH-2510]

Configuration
📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
👻 Immortal: This PR will be recreated if closed unmerged. Get config help if that's undesired.

[ ] If you want to rebase/retry this PR, check this box

This PR has been generated by Renovate Bot.

/run pipeline

/run pipeline

/run pipeline

/run pipeline

/run pipeline

:tada: This PR is included in version 1.8.3 :tada:
The release is available on:

GitHub release
v1.8.3

Your semantic-release bot :package::rocket:


Add default keybindings for switching terminals

I didn't end up making defaults for focusNext/focusPrevious and use ctrl+shift+j/k personally, but maybe it's time to make it act more like the editor tabs when the terminal is focused since it will be much clearer what terminal is focused when tabs come (https://github.com/microsoft/vscode/issues/10546).

So you're suggesting Ctrl+Page Up/Down like normal tabs?
I like Alt+Left/Right.

Essentially yeah, also ctrl+1-9 and ctrl+(shift+)tab. Though I'm not sure I want to navigate my terminals via an MRU list like editors and I'm sure I'll have a hard time getting used to ctrl+tab not moving the editor 🤔

@Tyriar In my opinion terminal key bindings can inherit from editor to unify user experience in all aspects. For example ctrl + tab When terminal section focused to switch between opens terminals and ctrl + w for close current terminal.

@Tyriar In my opinion terminal key bindings can inherit from editor to unify user experience in all aspects. For example ctrl + tab When terminal section focused to switch between opens terminals and ctrl + w for close current terminal.
This gonna be awesome because tabs got to terminal to unify editor experience, And this 'll be a plus to unify editor shortcuts as possible. This 'll effect the user at the end so he doesn't remember many shortcuts for editor and for terminal.

Editor shortcuts:
https://github.com/Microsoft/vscode/blob/601dec633b08a5184375b127d014967e90ae1954/src/vs/workbench/browser/parts/editor/editor.contribution.ts#L324-L325
@pixieaka I'm not sure we want to touch the ctrl+(shift+)tab shortcuts because some people like me use it to move focus from the terminal to the editor, plus we don't have the concept of MRU terminal navigation currently. For ctrl+w I'm hesitant to add because it's a shell shortcut.

I went with:

Linux/Windows: Ctrl+Page Up/Down
macOS: Cmd+Shift+[/]

Mac also has cmd+alt+left/right to navigate tabs but that's already used to navigate split panes and this aligns with iterm.

Hi, so ffedback:
I'm on Version: 1.56.2 and the new "next pane" default shortcut is not working on Mac or it is so complicated that is imposible to remember in some layouts like Spanish International layout.:

Sorry if I'm overloading you- I just feel like this should be the natural "next terminal" shortcut, now that they are shown vertically:
CMD+ALT+DOWN
CMD+ALT+UP


'mask_img' instead of 'mask' in FirstLevelModel and SecondLevelModel specification

Nistats is currently not following the Nilearn convention of calling the provided mask 'mask_img'

@kchawla-pi  are you doing this ?

Ok. WIll do tomorrow.

Done in #321
Closing.


Added Czech cs translation

The translation is complete, any feedback is welcome.

This looks great! Thank you for your contribution!


#921 show directory index [WIP]

What do these changes do?

By default throws HTTP 403 - Forbidden if we access static directory
If static directory indexing is enabled - lists the static directory contents when we access it

Are there changes in behavior for the user?

UrlDispatcher.add_static got new flag - show_index, which will show index of a directory if explicitly set to True. By default, accessing directory index is forbidden (HTTP/403, show_index=False)
Related issue number

#921
Checklist

[ ] StaticRoute._directory_as_html():

[ ] Maybe it should be outside the class? Unit-testing would be easier that way..
[ ] I would want a unit-test for this method to explicitly check that HTML is as we want
[ ] Method looks cumbersome and has lots of newlines, so the output would be human readable (is this a priority?)
Example output (from a test test_access_root_of_static_handler):

<title>Index of /</title>

Index of /

my_file

[ ] There is currently missing coverage for L506 and L531
[ ] Documentation should reflect the changes, though I have build problems ATM and cannot check docs as HTML

Fixed by #1094


update to mongodb 1.4.31

Hey @damirv . Thanks for the PR. I merged #182 which updates to the next version of the driver. It's published on 0.18.2


adapt tests to new google credential ids

After https://github.com/jenkinsci/google-oauth-plugin/pull/186 , when we create a google credential, the ID is no longer the project id but a generated one. Editing tests to no longer rely on that assumption. For the CasC test, it is now entirely CasC-ified, with creating the credential through CasC as well (creating a credential on the UI, ignoring the output and applying a CasC file made before we knew it wasn't a very realistic scenario.)
### Submitter checklist
- [x] Make sure you are opening from a **topic/feature/bugfix branch** (right side) and not your main branch!
- [ ] Ensure that the pull request title represents the desired changelog entry
- [x] Please describe what you did
- [ ] Link to relevant issues in GitHub or Jira
- [x] Link to relevant pull requests, esp. upstream and downstream changes
- [x] Ensure you have provided tests - that demonstrates feature works or fixes the issue

this needs google-oauth-plugin 1.1.1 for the id field in the credentials.

I sadly cannot push to @Riliane 's PR, so I created a new one to take care of the review comments: https://github.com/jenkinsci/google-compute-engine-plugin/pull/417


⚠ Bofrost.fr has degraded performance

In 3e13fb0, Bofrost.fr (https://www.bofrost.fr) experienced degraded performance:

HTTP code: 200
Response time: 914 ms

Resolved: Bofrost.fr performance has improved in 1cf4084.


resolve problem with SSDEEP use in hashing-algorithm-ov

Eliminates problem in stix2 where an object using an ssdeep hash in 2.1 would use the lowercase variant. closes #391

Codecov Report

Merging #393 into master will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##           master     #393   +/-   ##
=======================================
  Coverage   98.44%   98.44%
=======================================
  Files         127      127
  Lines       14613    14627   +14
=======================================
+ Hits        14386    14400   +14
  Misses        227      227

Impacted Files
Coverage Δ

stix2/patterns.py
99.33% <ø> (ø)

stix2/test/v21/test_pattern_expressions.py
100.00% <ø> (ø)

stix2/properties.py
98.42% <100.00%> (+<0.01%)
:arrow_up:

stix2/test/v20/test_observed_data.py
100.00% <100.00%> (ø)

stix2/test/v21/test_observed_data.py
100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 658e70b...b4dbc41. Read the comment docs.

@clenk I rebased, and build passed now...


METRON-622: Create a Metron Docker Compose application

This PR is an initial attempt at creating a Docker development and testing environment for Metron.  Information about what's included and how to use it can be found in the README.

Also, this is great - thanks for submitting this!

Just pushed out a commit that attempts to address these comments.  Includes:

hbase config is now read from Storm classpath instead of hbase jar
removed global config push from kafkazk init script
added license to hbase/bin/init-commands.txt
improved data generator script and updated documentation

I have followed the instructions, and it looks like it all worked but I am not sure what to do next to test this out.  The README is missing a usage example or something similar to the output of the ansible install that lists the services and ports

In my experience I've found that if you run these commands:
eval $(docker-machine env metron-machine)
docker ps -a
and all containers are reported as up:
CONTAINER ID        IMAGE               COMMAND                  CREATED
STATUS              PORTS
NAMES
a2f28ac154f3        metron_storm        "/bin/bash /home/stor"   2 days ago
Up 2 days           <IP_ADDRESS>:8000->8000/tcp,
<IP_ADDRESS>:8080-8081->8080-8081/tcp
metron_storm_1
7f40995c1344        metron_hbase        "/bin/sh -c ./bin/sta"   2 days ago
Up 2 days           8080/tcp, 8085/tcp, 9090/tcp, 9095/tcp,
16000/tcp, 16201/tcp, 16301/tcp, <IP_ADDRESS>:16010->16010/tcp   metron_hbase_1
013a5b660f2d        metron_kibana       "/docker-entrypoint.s"   2 days ago
Up 2 days           <IP_ADDRESS>:5601->5601/tcp
metron_kibana_1
cdb8a20d1f2d        metron_mysql        "docker-entrypoint.sh"   2 days ago
Up 2 days           <IP_ADDRESS>:3306->3306/tcp
metron_mysql_1
32f28acd2421        elasticsearch:2.3   "/docker-entrypoint.s"   2 days ago
Up 2 days           <IP_ADDRESS>:9200->9200/tcp, <IP_ADDRESS>:9300->9300/tcp
metron_elasticsearch_1
6a8aaa5c3853        metron_kafkazk      "/bin/sh -c ./bin/sta"   2 days ago
Up 2 days           <IP_ADDRESS>:2181->2181/tcp, <IP_ADDRESS>:9092->9092/tcp
metron_kafkazk_1
It's a pretty good indication that everything is working.  If 1 or more
containers have a status of "Exited" that means something went wrong.
There are some UIs that are included as well and I agree we should add
those urls to the README.  Will get that pushed out soon.
Ryan
On Wed, Dec 14, 2016 at 3:53 PM, ottobackwards<EMAIL_ADDRESS>wrote:

I have followed the instructions, and it looks like it all worked but I am
not sure what to do next to test this out. The README is missing a usage
example or something similar to the output of the ansible install that
lists the services and ports
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/apache/incubator-metron/pull/393#issuecomment-267168563,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AATCeCXJW9x2lHv8hwP7sGFl10vh66w2ks5rIGVYgaJpZM4LJZHT
.

I suspect we should ensure that data is flowing through.  @merrimanr , maybe it'd be good to document in the README.md a worked example where you:

spin up a docker cluster
generate data using the data generation scripts
look at the indices to see that there are data
check Kibana for the data

Am I missing anything?

That is a start @cestella @merrimanr.  Enough to validate the PR.  For the intended use case - we should have as much documentation for this as the vagrant stuff.  So wiki peer entries etc.  At least I would think.

I will continue to validate this when that next push goes out.

@merrimanr I'm super excited about this one! Thanks for your work!
While kicking the tires on it, I did notice something that isn't really a bug but got me thinking... The README assumes the developer is using a Mac. I personally use Fedora for most of my development and testing of Metron.
In my case, I found that without using docker-machine, the kafkazk service did not start up properly due to the use of sed on the DOCKER_HOST variable in the Dockerfile to set the advertised.listeners parameter in kafka server.properties. I was using the docker-engine on the localhost and thought I would avoid the hassle of docker-machine. It was easy enough to solve by manually manipulating the server.properties file in the container, but I was wondering if this could be avoided somehow?
I assume that had I installed docker-machine and used the provided script it would have worked as expected even on Fedora (I'm planning to test that case as well), but was curious why the use of the virtualbox driver? Is this a Mac specific requirement?
Thanks again for the contribution! I know this will make coding/testing new parsers a lot faster.

Just pushed out changes based on this initial round of feedback.  Main changes were moving the install steps to a Maven pom and adding more detail to the documentation.

@kylerichardson thank you for reviewing it!  I updated the documentation to include Docker for Mac or Docker for Windows.  One of the primary requirements for the way I use this is that containers must be accessible from my local environment where my IDE is running so services that broadcast their host address are tricky.  The Kafka advertised listener thing was the single most challenging issue I faced so not surprised you hit it too.  The Kafka Dockerfile is wired to pull the DOCKER_HOST from an input argument and the compose file is wired to pass the local DOCKER_HOST environment variable as the DOCKER_HOST input argument to the Kafka Dockerfile.  So you should be able to set your local DOCKER_HOST environment variable as such:
$ export DOCKER_HOST="tcp://:2376"
When you run "eval $(docker-machine env metron-machine)" that's pretty much what Docker Machine is doing, setting local environment variables to match the desired host.  Then after you build the environment, the advertised.listener property should be set to  without you having to manually change it.
I only used the virtualbox drive because that's what came out of the box.

Thanks for the explanation @merrimanr. I totally agree on the need for the
local IDE to have access to the containerized services. It's an easy enough
fix for me to manipulate the DOCKER_HOST env variable on Fedora and avoid
using docker-machine (although, I do see that as a big win for ease of use
on Mac and Windows).
I haven't had much luck with vagrant myself so am super excited for a
docker alternative to quick dev. As an aside, there are some Ansible
modules for docker (
https://docs.ansible.com/ansible/list_of_cloud_modules.html#docker) that we
could look into for incorporating some of the setup scripts and image
building to make it even more user friendly. Might just be a low priority
nice to have but thought I'd throw it out there.
On Mon, Dec 19, 2016 at 4:32 PM, merrimanr<EMAIL_ADDRESS>wrote:

@kylerichardson https://github.com/kylerichardson thank you for
reviewing it! I updated the documentation to include Docker for Mac or
Docker for Windows. One of the primary requirements for the way I use this
is that containers must be accessible from my local environment where my
IDE is running so services that broadcast their host address are tricky.
The Kafka advertised listener thing was the single most challenging issue I
faced so not surprised you hit it too. The Kafka Dockerfile is wired to
pull the DOCKER_HOST from an input argument and the compose file is wired
to pass the local DOCKER_HOST environment variable as the DOCKER_HOST input
argument to the Kafka Dockerfile. So you should be able to set your local
DOCKER_HOST environment variable as such:
$ export DOCKER_HOST="tcp://:2376"
When you run "eval $(docker-machine env metron-machine)" that's pretty
much what Docker Machine is doing, setting local environment variables to
match the desired host. Then after you build the environment, the
advertised.listener property should be set to without you having to
manually change it.
I only used the virtualbox drive because that's what came out of the box.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/apache/incubator-metron/pull/393#issuecomment-268084862,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AJ6R3a4JoffRcq9etOoUAJExjJAeue-aks5rJvf_gaJpZM4LJZHT
.

So just to be clear, the end result of this is the cluster deployed, but nothing installed?  No topologies, no indices in ES etc?

I would consider the topologies installed, just not running.  But yes, no data flowing end to end by default.

If the indices are not installed, then the topologies are moot.  I am sorry, I don't mean to be difficult, I am just not sure if it is working and I don't understand what the result should be or if it did not work

There were several examples added to the README.  The last one "Run sensor data end to end" will result in data flowing into an ES index if executed properly.  If you are able to successfully follow the examples then it works.  There is also a "Usage" section that explains how to check the container status and lists the UIs.
I would argue that the topologies are not moot just because indices are not installed.  What if I'm working on a new parser class?  I will more likely be inspecting the enrichments topic to verify the new parser is working correctly rather than looking at the indices.  That would come later.
Maybe the scope of the PR is different than what you are expecting?  This originally was intended to be a light-weight development environment that allows a developer to work on or troubleshoot specific parts of the Metron flow, not an end-to-end solution.  We already have something for that (vagrant quickdev).

ok, I was not using the updated examples in the read me to verify the pr.  It was not clear to me, after starting everything up, what I should expect to be working and what should not be working.  I will go back and do that get the context.

I've run this up and successfully tested it using the examples provided in the README. It works as documented on docker-machine/boot2docker. Nice job.
One showstopper for me. I can't seem to find the topology logs in the storm container. I checked /var/log/storm and no topology specific logs were ever written. For debugging new parsers, etc. this will be important to have available.
I also want to highlight a few nice-to-haves that I would be perfectly happy submitting as separate, follow-on PRs.

Custom the elasticsearch image (1) to have the elasticsearch-head plugin installed as part of the image build, (2) copy the es_templates into the image so they are available, (3) for bonus points, deploy the templates on container start
Load zookeeper config on container start
Improve kafkazk Dockerfile for local docker-engine (Linux); current problem is that DOCKER_HOST in this case defaults to empty string and the default argument in the Dockerfile is never hit
Add an HDFS container to allow for complete testing of the indexing topology

The logs for this container are written to /usr/share/apache-storm/logs/ instead of /var/log/storm.  That's probably why you don't see them.  That's just the way they configured it in this Docker image but I'm sure it's easy enough to change.
You can also view the logs through the Storm UI.  The one catch is that once you drill into a specific log (will give you a 404 initially) you have to replace the ip address (it's an internal Docker container IP and not accessible outside of the Docker host) with the Docker machine ip.
I agree those are nice add ons.  This is a pretty significant PR as is so I tried to keep the scope as small as possible while still providing the ability to flow data end to end.  The first one isn't too bad and I think it fits nicely into a separate PR.  The 2nd one is trival, I'm fine with configs being loaded initially and will make that change.  The 3rd one might be easy, you have the environment setup so maybe you could experiment with it (I will help)?  The last one will definitely require some work and should be a separate PR.

Maybe we can create jiras for these, and someone else might help?

I'll created METRON-646 for the elasticsearch image customizations. I already have a start on those changes.

@merrimanr Thanks for pointing out the location of the storm logs. I think it would be worth adding that to the README.
I have a fix for the kafkazk image running on local docker-engine (using unix socket). It requires an additional ARG in the Dockerfile (and docker-compose.yml) and an extra sed statement when producing the advertised.listeners parameter.
docker-compose.yml
...
kafkazk:
  build:
    context: ./kafkazk
  args:
    DOCKER_HOST: $DOCKER_HOST
+   BROKER_IP_ADDR: $BROKER_IP_ADDR
    METRON_VERSION: $METRON_VERSION
...

kafkazk/Dockerfile
...
+ARG DOCKER_HOST
+ARG BROKER_IP_ADDR
ARG METRON_VERSION
...
RUN echo -n 'advertised.listeners=PLAINTEXT://' >> /opt/kafka_2.11-<IP_ADDRESS>/config/server.properties
+RUN echo $DOCKER_HOST | sed "s/^$/"$BROKER_IP_ADDR":/g" | sed "s/tcp:\\/\\///g" | sed "s/:.*/:9092/g" >> /opt/kafka_2.11-<IP_ADDRESS>/config/server.properties
...

I would also suggest a note in the documentation. In the Setup section of the README, maybe something along the lines of... If you wish to use a local docker-engine install, please set an environment variable BROKER_IP_ADDR to the IP address of your host machine. This cannot be the loopback address.

When trying to verify the Run Sensor Data End to end with two terminals I get :
ottofowler@Winterfell ~/src/apache/work/incubator-metron-pr-393/metron-docker/compose
1 pr-393 % docker-compose exec kafkazk ./bin/produce-data.sh /data/BroExampleOutput.txt bro
WARNING: The DOCKER_HOST variable is not set. Defaulting to a blank string.
Emitting data in /data/BroExampleOutput.txt to Kafka topic bro every 1 second(s)
[2017-01-06 19:28:49,385] ERROR Error when sending message to topic bro with key: null, value: 482 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
org.apache.kafka.common.errors.TimeoutException: Failed to update metadata after 60000 ms.
and
ottofowler@Winterfell:metron-docker/compose ‹pr-393›$ docker-compose exec kafkazk ./bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic bro
WARNING: The DOCKER_HOST variable is not set. Defaulting to a blank string.
No brokers found in ZK.
Is there something I'm doing wrong?

@ottobackwards you need to run this before you execute any docker or docker-compose command:
eval "$(docker-machine env metron-machine)"
This is documented in the "Setup" section of the README.  Is there a change we can make to the documentation to make this clearer?

Thanks @kylerichardson!  I will add your change in along with initially loading all configs over the weekend.

I have run that - I have been able to do the steps in the readme ( except the topology which i haven't tried ) up until that point ( so my ZK is configured etc )
ottofowler@Winterfell ~/src/apache/work/incubator-metron-pr-393/metron-docker/compose
0 pr-393 % eval "$(docker-machine env metron-machine)"
Error checking TLS connection: Host is not running
ottofowler@Winterfell ~/src/apache/work/incubator-metron-pr-393/metron-docker/compose
0 pr-393 % docker-compose exec kafkazk ./bin/produce-data.sh /data/BroExampleOutput.txt bro
WARNING: The DOCKER_HOST variable is not set. Defaulting to a blank string.
Emitting data in /data/BroExampleOutput.txt to Kafka topic bro every 1 second(s)
[2017-01-06 19:47:02,967] ERROR Error when sending message to topic bro with key: null, value: 482 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
org.apache.kafka.common.errors.TimeoutException: Failed to update metadata after 60000 ms.

Actually - the test output data stuff was failing too, I just hadn't waited long enough to see since I wasn't trying to read.

"Error checking TLS connection: Host is not running" that means it didn't work.  Can you try running "docker-machine ls" and ensure you have a machine called "metron-machine"?  It should look like this:
HW10673:compose rmerriman$ docker-machine ls
NAME             ACTIVE   DRIVER       STATE     URL                         SWARM   DOCKER    ERRORS
metron-machine   *        virtualbox   Running   tcp://<IP_ADDRESS>:2376           v1.12.5
HW10673:compose rmerriman$ eval "$(docker-machine env metron-machine)"
HW10673:compose rmerriman$ echo $DOCKER_HOST
tcp://<IP_ADDRESS>:2376

ottofowler@Winterfell ~/src/apache/work/incubator-metron-pr-393/metron-docker/compose
130 pr-393 % docker-machine ls
NAME             ACTIVE   DRIVER       STATE     URL   SWARM   DOCKER    ERRORS
metron-machine   -        virtualbox   Stopped                 Unknown

I don't know enough about docker, i can exec into bash and populate zk etc so it is 'running' or something is

0 pr-393 %  docker ps --format 'table {{.Names}}\t{{.Status}}'
NAMES                    STATUS
metron_storm_1           Up About an hour
metron_kibana_1          Up About an hour
metron_hbase_1           Up About an hour
metron_elasticsearch_1   Up About an hour
metron_mysql_1           Up About an hour
metron_kafkazk_1         Up About an hour

Your machine is stopped, that is the problem.  Can you go into Virtualbox and start it back up?  Then you should be able to run the eval statement.  You may have to build the environment again if you didn't build on "metron-machine".
It's likely that you are using MacOS as your Docker host which has networking limitations.  Can you try doing this using docker-machine?  My guess is that you followed the instructions in "Setup" since you do have "metron-machine", but when it came time to run "docker-compose up -d" you were in a new session and had not run eval "$(docker-machine env metron-machine)" so Docker was not pointed at the right machine.  In that case it would have used your local environment instead which will not work.

What step was supposed to start the virtual box machine?  How do I do this 'using docker-machine' differently than the instructions?  I followed them exactly from the top of the readme.

9028  checkout-pr 393
9029  cd incubator-metron-pr-393
9030  mvn clean install -DskipTests
9031  export METRON_HOME=.
9032  export METRON_DOCKER_HOME=$METRON_HOME/metron-docker
9033  cd $METRON_DOCKER_HOME
9034  ./scripts/create-docker-machine.sh
9035  eval "$(docker-machine env metron-machine)"
9036  cd compose
9037  docker-compose up -d

The step that starts the machine is in "Setup":
cd $METRON_DOCKER_HOME && ./scripts/create-docker-machine.sh
After you run that the machine will be created and running.  To start a stopped machine you can either go into the VirtualBox application and start it that way or you can run:
$ docker-machine start metron-machine
Or you could start over too.  Somehow you got into a bad state because your machine is not running.  That is your problem.  Maybe you are trying to do this a couple days after you initially built the environment and the machine went down between then and now?  If you want to start over, run this to remove the current metron-machine:
$ docker-machine rm metron-machine

So - should we destroy the machine when we are done?

ok - starting the machine fixed it.  I think the issue is that I had a defined, but not running machine from a previous run.  So either we need instructions to be clear on what to do with the machine when you are done... or the script maybe needs to start it if it doesn't create it?

I usually just leave it up.  My current environment has been running for 2 weeks with no problem.  The intention behind creating a Metron-specific machine is that it won't conflict with other Docker machines someone may be using (or if people are using the default machine, which I would guess is most common).
Not sure how yours went down but sometimes that happens.  Would adding a note in the "Setup" section of the README to make sure machine is running do it (right before the eval command)?  I could also add quick instructions on how to stop/start a machine.

`
value=$(VBoxManage list vms | grep -ic "metron-machine" )
if [ $value -eq 1 ]
then
VBoxManage startvm "metron-machine" --type headless
fi
`

I don't think we can assume that dev's will just leave it up and running.   The flow should be close to the vagrant flow shouldn't it?

This does bring up an interesting point: when testing something like this - it is not enough to just run through the examples, you need to do the more realistic workflow - multiple sessions etc to catch something like this

Fair enough.  Probably best to start/stop with each use.  I will update the docs to reflect that.  Vagrant is not ideal for me as a development tool so I don't use it as a guide :)  But I see your point and consistency is good.
Yeah I agree.  Testing with multiple sessions is a good idea and something we should do for tools like this.  In the future I will highlight that in the PR description if applicable.

This is a great work.  I am not sure what the right flow or scripts will end up being, but if we document it we can land it and iterate through pr's later.

Just waiting on the docs for a +1 on this, then we can get it in

I added the BROKER_IP_ADDR change from @kylerichardson and now configs are loaded on container start.  Also added the various documentation requests.  @ottobackwards let me know if the start/stop documentation is clear enough.

+1 - this is great work Ryan

Thanks!  Appreciate all the valuable feedback.

+1 Awesome contribution.
I'm already working on some follow on PRs for this :-).


The training result is poor after using the weak supervision data set

Hello, if I want to use weakly supervised data to replace the ground truth value to train the segmented network in the experiment, do I need to modify the code after replacing the dataset?
Because I tried to run this code with the pseudo mask image (10582) obtained by weak supervision, but the result was only 0.03, which actually only recognized the background. Do I need to modify the code?

Have you checked if your GT files have been loaded successfully?
I think you may also need to debug the class-wise progress for training not only the overall loss.

Is the GT file you mentioned the ground truth image? Yes, successfully loaded. And I tried to train with train (1464) to produce results. The result was 61%, lower than expected. With train_aug (10582), it could only identify the background. The scores of other categories were 0.
Do you have any other suggestions besides debugging?
Thanks for your help!

From your results, I guess your model only produces 0 for any images.
Sorry for the duplicated question. Are your loaded weakly-supervised labels decoded numbers from 0 to 20 successfully? Does it suit your task that the default setting only treats 255 as the label to be ignored in the loss computation? Since I never assumed the weakly-supervised setting to maintain this repository, I can only suggest general things.

OK, I see. I will go to debug it. Thank you!


[REQUEST] Support 8bit optimizer

Hi thanks for the library! It would be great if the AdamW cpu optimizer can have 8bit variant. For example, I would like to try adamw_8bit to full-finetune a 8B model on a 24GB GPU card (RTX4090). With deepspeed offload, the GPU memory is OK, but the CPU memory requirement is still very huge, partially because it uses normal adamw, thus needs 8x8=64GB for the optimizer itself. If we have 8bit adamw, then this part is reduced to 8x2=16GB.

If you are able please create a pull request for this enhancement and we will review it.

@jomayeri Thank you. Firstly, I would like to know whether this is reasonable (and not implemented simply because people did not have time doing that), or is it nonsense?

It is certainly reasonable and would be applicable in a finetuning scenario.


Merge Dockerfiles

merge dockerfiles, include base image branching in updated dockerfile

CI is blocked by errors related to perf_config_table changes, stalled until previous pr is merged.

Opened an issue LWPTUNA-141 for updating the ci to use new performance pipeline commands.
Ok to proceed with PR.

@cderb do any of our docs need updating for this?

@alexandraBara and @iq136boy please review the PR


Alert modal on home page about test auction

Make a modal box giving info about the test auction that has been created to test the software. Add freeze and unfreeze flags to allow various activities in the auction.


ARM template comment and multiline support

Documentation should clearly state that only selected deployment methods support extended JSON.
Afaik only AZ CLI and Powershell cmdlet are the only deployment methods provided by MS supporting extended JSON.
The following are failing:

Azure Devops tasks: https://github.com/microsoft/azure-pipelines-tasks/issues/14645 and https://github.com/microsoft/azure-pipelines-tasks/issues/14268
Deployments done in the portal
Deployments done directly with the API endpoints

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: c5d2f2e7-e025-aa47-457b-0dc8f4528511
Version Independent ID: 4b3b1147-6032-323b-327a-967ed764fd20
Content: Template structure and syntax - Azure Resource Manager
Content Source: articles/azure-resource-manager/templates/template-syntax.md
Service: azure-resource-manager
Sub-service: templates
GitHub Login: @mumian
Microsoft Alias: jgao

@jikuja Thanks for your comment! We will review and provide an update as appropriate.

@jikuja - thanks for this feedback. I am updating the doc to clearly state what tools support this syntax.

The update should get published later today.
#please-close


bump device crate version, and move examples to the rtfm! macro

@homunkulus r+

:pushpin: Commit 8bb4a94 has been approved by japaric

:hourglass: Testing commit 8bb4a94 with merge 8bb4a94...

:broken_heart: Test failed - status-travis

@homunkulus r+

:pushpin: Commit 2b7d5c5 has been approved by japaric

:hourglass: Testing commit 2b7d5c5 with merge 2b7d5c5...

:sunny: Test successful - status-travis
Approved by: japaric
Pushing 2b7d5c56b25f4efad6c7c40042f884cbecb47c0b to master...


Update get-clients.md

Removing Linux

@dstrome
PR has been copyedited and is ready for final review, could you please check and merge? Thanks!


Support for SGB

hi can you add support for SGB core and Gamate
SGB: Nint. SGB
Gamate: Bit Corp. Gamate
Gamate: UMC. Gamate
best regards
Mork

Done and credited to you. Sorry it took a while :)

bug SGB needs to be in capital letters or it wont work
SGB:  Nint. Super GB
best regaards
Mork

il only comment on cores that is downloaded for mister update all or download 1.5

Sure thing and thank you very much for contributing! :)


Addresses #1865 NTR bipolar cells

Fixes #1865 NTR bipolar cells

Thank you for the review, @bvarner-ebi!

In the associated "MIRACL" sheet, the taxon is noted to be "Homo sapiens", although many of the dbxrefs are not exclusive to or include human samples.

That is true, but as these terms are not human specific unless they have a bag of genes that differentiates them with other species, I have not added the taxon constraint. I will remove it from the MIRACL sheet.

interneuron
and ('has characteristic' some 'cortical bipolar morphology') appears in the inherited subclasses for these cell types, coming from the equivalence axiom of 'bipolar neuron'. This may not apply to many/any of these cell types.

Thank you for noticing, I will have a look.

CL has label 'OFF-bipolar cell' while the text definitions use "OFF bipolar cell" (with no hyphen). Same applies to 'ON-bipolar cell'.

I will update the text definition to align them with the CL label.

There is a likely copy error for the ORCiDs in the associated "MIRACL" sheet. It should be confirmed that the ORCiD used for these terms is correct.

The ORCID in the MIRACL are wrong, I drag from the first cell and it made consecutive numbers, instead of copying in all cells.

Thank you very much for all the review process, @bvarner-ebi! I won't submit so many cells in one ticket anymore.


Add support for Edge webdriver

Add support for Edge webdriver.
The following tests are failing:

integration tests - page test for Edge - should support links and navigation
integration tests - page test for Edge - should support switching windows
integration tests - page test for Edge - should support popup boxes
integration tests - page test for Edge - should support various mouse events
integration tests - selection test for Edge - should support retrieving element properties
integration tests - selection test for Edge - should support element actions

However, when using the webdriver on my own, I noticed that Click is not working most of the time, requiring a DoubleClick instead. Don't know yet if normal or an issue on the edge webdriver

@sclevine the tests are passing on go 1.7 but not on older version (an error changed).
The code also now handle the non-windows platform

Could you update the failing test to use MatchError with a regex so it passes on all versions of Go?

@sclevine tests are now passing

Wonderful, thanks! Sorry for the delay.


Possible bug from lane arrow update

Not been able to find a reliable way to reproduce this yet, however I think there's an interaction state bug crept in as a result of #538.
After loading a save, I was going to test the stuff from #538. I opened TMPE main menu and hovered lane arrow button to check what the shortcuts were:

Side note: It would be nice if we could show the button shortcut (in my case Shift + .) part of the tooltip in a different colour, eg. yellow, to differentiate from the tool tip text.

So far everything working well. Note: I don't recall clicking the lane arrow button (I had no roads to test with).
I was about to build some roads to do the testing, but noticed some cruft on the map that needed moving out of the way and pressed "M" to open "Move It" mod. The "Move It" mod UI appeared but I was not able to select anything (usually the default "single selection" tool will allow almost anything to be moved).

I was able to get things working again by escaping out and then reopening Move It UI.

I've not yet been able to reproduce this glitch, maybe it only occurs once on first use or something?
I checked for other mod updates but most recent was 6th/7th November (Theme Mixer 2, Environment Changer, Loading Screen Mod TEST). I've not noticed the issue described above before, despite several sessions since last external mod update. Only thing that's changed is TM:PE which I've built twice in past 48 hours (alpha9 for workshop, and a subsequent build to test #538 - it's this second build where I noticed the issue).

Interesting I don't see this at all.

The issue wasn't with the tooltip as such, but that it seemed to temporarily break Move It mod.
I'll do some more testing and see if I can make it happen again and, if so, try and document repeatable steps to help track down problem. It's possible that it was just a weird one-off transient issue that might not even be caused by TM:PE.

Its compatible  with the move it mod on my computer.

I've not been able to reproduce this so going to close this issue for now. If it happens again I'll reopen.


pasystray not detecting other software

Hello,
I am using pasystray (services.pasystray.enable = true;) and noticed that it does not detect other tools in the path. The other options are greyed out as described in https://github.com/christophgysin/pasystray/issues/23 .
I added paprefs and pavucontrol:
home.packages = with pkgs; [
  pavucontrol
  paprefs
  ];

I am using home-manager in my system-wide configuration.nix by importing it with builtins.fetchTarball.
@pltanton is listed as maintainer, so maybe he can help.
Cheers
Amarandus

Please give it a try now. It should also work even if you don't have pavucontrol and paprefs explicitly installed.


CodeChecker log does not work for 32bits binaries on 64bits platform

CodeChecker` log -b "./builder64" -o ./compile_command64.json
[INFO 2024-08-14 14:09] - Starting build...
[INFO 2024-08-14 14:09] - Using CodeChecker ld-logger.
[INFO 2024-08-14 14:09] - Build finished successfully.
(CodeChecker venv-dev) ednikru@seliiuvd02700[14:09][workspace/test_files/codechecker_loggin_test]$ cat compile_command64.json
[
        {
                "directory": "/local/workspace/test_files/codechecker_loggin_test",
                "command": "/usr/bin/gcc ./test.c",
                "file": "./test.c"
        }
](CodeChecker venv-dev) ednikru@seliiuvd02700[14:10][workspace/test_files/codechecker_loggin_test]$ CodeChecker log -b "./builder32" -o ./compile_command32.json
[INFO 2024-08-14 14:10] - Starting build...
[INFO 2024-08-14 14:10] - Using CodeChecker ld-logger.
ERROR: ld.so: object '/local/workspace/codechecker/build/CodeChecker/ld_logger/lib/x86_64/ldlogger.so' from LD_PRELOAD cannot be preloaded (wrong ELF class: ELFCLASS64): ignored.
[INFO 2024-08-14 14:10] - Build finished successfully.
`
builder.c
`
#include<stdlib.h>
int main(void){
  const char* command="gcc ./test.c";
  system(command);
  return 0;
}

gcc ./builder.c -m32 -o builder32

Looks like this PR changed the behaviour in CodeChecker:
https://github.com/Ericsson/codechecker/pull/3919/files
the ld_logger.so binary is selected based on the host OS platform instead of the executed binary...


Issue 462 - Unsubscribe topics from consumer group

[ ] Breaking change? (if so, please describe the impact and migration path for existing application instances)

What changes did you make? (Give an overview)
Added the feature to delete topics (unsubscribe from topic), assuming that the consumer has been stopped.
How Has This Been Tested? (put an "x" (case-sensitive!) next to an item)

[ ] No need to
[x] Manually (please, describe, if necessary)
[x] Unit checks
[ ] Integration checks
[ ] Covered by existing automation

Checklist (put an "x" (case-sensitive!) next to all the items, otherwise the build will fail)

[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[x] I have made corresponding changes to the documentation (e.g. ENVIRONMENT VARIABLES)
[x] My changes generate no new warnings (e.g. Sonar is happy)
[] I have added tests that prove my fix is effective or that my feature works
[x] New and existing unit tests pass locally with my changes
[x] Any dependent changes have been merged

Check out Contributing and Code of Conduct
A picture of a cute animal (not mandatory but encouraged)

Isn't this one the same as #549 ?

Isn't this one the same as #549 ?

Yeah basically, I'm taking a course right now where we choose an issue and open a PR for it. #549 was opened in between but I still have to deliver a PR for the grade.


refactor/feature: Refactor structure and set up Next.js

What the title says is completely correct.

Hey dawn, the PR looks good more or less, but would you mind fixing the workflows as well? Thanks

Hey dawn, the PR looks good more or less, but would you mind fixing the workflows as well? Thanks

I'll attempt to do so

I'm closing this PR in favor of using (Docusaurus)[https://docusaurus.io].


Benchmark utility to perform diff of output from benchmark runs, allowing for precision differences

This PR adds a convenience method to BenchUtils for comparing the results from two benchmark runs (e.g. verifying that GPU and CPU results match).
This is a simple implementation that pulls results down to the driver for comparison so will only work for data sets that can fit in the driver's memory, but this is likely adequate for most of the benchmarks we are currently running. For example, TPCxBB q5 at 10TB scale produces ~600MB of output.
If data needs sorting before comparison, this is delegated to Spark before collecting the results.
Example usage from spark-shell:
scala> val cpu = spark.read.parquet("/data/andygrove/q5-cpu")
scala> val gpu = spark.read.parquet("/data/andygrove/q5-gpu")
scala> import com.nvidia.spark.rapids.tests.common._
scala> BenchUtils.compareResults(cpu, gpu, ignoreOrdering=true, epsilon=0.0)
Collecting rows from DataFrame
Collected 989754 rows in 7.701 seconds
Collecting rows from DataFrame
Collected 989754 rows in 2.325 seconds
Results match

if this is useful by itself it might be nice to document in the readme, otherwise it can wait til more stuff is in place.

if this is useful by itself it might be nice to document in the readme, otherwise it can wait til more stuff is in place.

I agree. I filed https://github.com/NVIDIA/spark-rapids/issues/794 for writing benchmark documentation.

build

build

build

I merged this too quickly, sorry. If someone had pending concerns please let me know, I can revert, or help fix issues.


Add cron expression form validation

Using the CronExpression library, this addition ensures that a cron expression is valid in settings before allowing the user to save it.
Addresses #278
Other notes
The ValidCronExpression Laravel rule was generated using the following command
php artisan make:rule ValidCronExpression --invokable

Before
Invalid cron expressions are saved without an error.

After
On Save, invalid cron expressions are not saved and the error is reflected in a UI message.

@jocmp thanks for the PR! Can you do me a quick favor and address the code quality issues pint flagged? If you want to check locally you can run sail pint --test -v if you're using Sail.

@alexjustesen sure thing. the branch is now up-to-date with those lints.


PHP FPM slow log

Is there a way to enable PHP FPM slow log?

I think you can enable it using custom php.ini.
Please check : https://devilbox.readthedocs.io/en/latest/configuration-files/php-ini.html


No feedback to user when parsing fails

If you try the following in the reverse word question:
def reverseWords(word):
  if word == "moo cow bark dog":
    return "oom woc krab god"

Parsing fails and an error is written to console, but no feedback (or error) is given to the user.
Expected behavior: Some indication to user what went wrong.
Observed behavior: Nothing (unless you open a console and see the obscure error message written there)

I checked, and the error lies in the _jsToHumanReadable function in FeedbackGeneratorService.js, where it does not handle null inputs. We should just fix this to correctly handle all possible types once and for all, I think.


createLink for windows

Turns out that windows supports something that seems identical to unix hard links. Win32-extras System.Win32.HardLink.createHardLink exposes this functionality.
unix-compat's getFileStatus reports the right link count for hard linked files on windows.
So, suggest making System.PosixCompat.Files.createLink call createHardLink on windows, instead of throwing an error as it does now.

I'd be happy enough to add this / accept a PR if people think it's a good idea.
/cc @snoyberg @ndmitchell do you have any thoughts on this?

No ideas, I'm afraid.

@joeyh if you submit a PR and test that it works I'll be happy to merge it


⚠ ClamAV has degraded performance

In 11072e4, ClamAV (https://spamassassin.apache.org/) experienced degraded performance:

HTTP code: 200
Response time: 84 ms

Resolved: ClamAV performance has improved in bd9eeca.


[Docs] OnsetSlice and BufOnsetSlice

I aimed to simplify the descriptions here and remove the dead DAfX link that was there.
@tedmoore, do you mind have a quick squiz at this?

Performs segmentation based on the measured difference between spectral frames. I think you can take out word measures. not a big issue though.
The metric for calculating difference can be modified from a curated selection, lending the algorithm toward slicing a broad range of musical materials. The word "modified" is not accurate. It should be "chosen" or something like that.

Thanks :) Merged.


An Updates

Are you actually working on this or just keeping the actual source private?

Hey John - actively working on this. Haven't pushed updates in awhile due to lots of churn. I'm targeting 10/26/15 for an early alpha release. Interested in contributing?
Date: Sun, 11 Oct 2015 07:44:00 -0700
From<EMAIL_ADDRESS>To<EMAIL_ADDRESS>Subject: [falcor.net] An Updates (#1)
Are you actually working on this or just keeping the actual source private?
—
Reply to this email directly or view it on GitHub.

Yeah, I would love to but I have other commitments at the moment.  I'll
definitely contribute when I can.
On Sun, Oct 11, 2015 at 8:15 AM Craig Smitham<EMAIL_ADDRESS>wrote:

Hey John - actively working on this. Haven't pushed updates in awhile due
to lots of churn. I'm targeting 10/26/15 for an early alpha release.
Interested in contributing?
Date: Sun, 11 Oct 2015 07:44:00 -0700
From<EMAIL_ADDRESS>To<EMAIL_ADDRESS>Subject: [falcor.net] An Updates (#1)
Are you actually working on this or just keeping the actual source private?
—
Reply to this email directly or view it on GitHub.
—
Reply to this email directly or view it on GitHub
https://github.com/falcordotnet/falcor.net/issues/1#issuecomment-147208723
.

Pushed latest bits here: https://github.com/falcordotnet/falcor.net
I’m working on building out the example that will have all the examples from https://github.com/Netflix/falcor-router-demo/blob/master/index.js
You can see my progress here: https://github.com/falcordotnet/falcor.net/blob/master/examples/Falcor.Examples.Netflix/NetflixRouter.cs

Wooo. Nice.  I would love to use Falcor instead of REST.  I'll test this
out when I get some time.
On Wed, Oct 21, 2015 at 8:23 PM Craig Smitham<EMAIL_ADDRESS>wrote:

Pushed latest bits here: https://github.com/falcordotnet/falcor.net
I’m working on building out the example that will have all the examples
from https://github.com/Netflix/falcor-router-demo/blob/master/index.js
You can see my progress here:
https://github.com/falcordotnet/falcor.net/blob/master/examples/Falcor.Examples.Netflix/NetflixRouter.cs
—
Reply to this email directly or view it on GitHub
https://github.com/falcordotnet/falcor.net/issues/1#issuecomment-150096128
.

Hi  @craigsmitham Thanks for implementing Falcor in .net, Any progress on this one? Any plan to release a stable version?

Okay. Ignore my Question. Just learned falcor is still in developer preview https://www.npmjs.com/package/falcor

@dotnetdude thanks for checking this project out. Yes, as Falcor.js is still in developer preview, I haven't made many continued investments in Falcor.NET. I'm still really happy of the work that is done here so far, and there is a really solid foundation here for any future implementation. Personally, I've been implementing a small-ish site using Angular2, .NET Core MVC, and ngrx/store (redux). It's a bit of a different approach from Falcor. I'm still learning. I think some mix of GraphQL, Falcor, and Redux is going to be the holy grail for managing client/server state in the future. Until then, each have their own unique trade-offs. Having worked with the redux pattern (ngrx/store) for about 6 months now, I see its key benefits but also see where Falcor provides some key capabilities as well. No time to dig into observations now, but if you are interested in Falcor - I would encourage you to keep diving in, including Falcor.NET.


Can't run in alpine linux - __vfprintf_chk: symbol not found/Static binary is dynamically linked

I tried downloading the Standalone Binary v0.10 release and when I tried to run it you get odd behaviour:
bash-4.3# /usr/bin/influx
bash: /usr/bin/influx: No such file or directory
bash-4.3# ls -l /usr/bin/influx
-rwxr-xr-x 1 root root 11611184 Feb  5 19:05 /usr/bin/influx

Odd. ldd to the rescue:
bash-4.3# ldd /usr/bin/influx
        /lib64/ld-linux-x86-64.so.2 (0x55fc22b32000)
        libpthread.so.0 => /lib64/ld-linux-x86-64.so.2 (0x55fc22b32000)
        libc.so.6 => /lib64/ld-linux-x86-64.so.2 (0x55fc22b32000)
Error relocating /usr/bin/influx: __vfprintf_chk: symbol not found

I'm not sure if this is intended, but the static binary still links against libc?
This from my OSX machine:
(themisto tmp/influx :*+)% curl -fsSL https://s3.amazonaws.com/influxdb/influxdb-0.10.0-1_linux_amd64.tar.gz | tar x
(themisto tmp/influx :*+)% file influxdb-0.10.0-1/usr/bin/influx
influxdb-0.10.0-1/usr/bin/influx: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), not stripped

A quick google reveal Chromium Issue 572417 – PPAPI not compatible with musl libc on linux which is the same libc that Alpine Linux is based on. No fix, but other people have reported similar problems.
I think I remember reading something about go 1.4 not really producing static binaries -- could this be the cause?

I found this post http://blog.hashbangbash.com/2014/04/linking-golang-statically/ which says that if anything is using cgo (which seems plausible) then you have to specify other flags to actually get a static binary out (reproduced here):
$> go build ./code-cgo.go
$> file ./code-cgo
./code-cgo: ELF 64-bit LSB  executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), not stripped

vs
$> go build --ldflags '-extldflags "-static"' ./code-cgo.go
$> file ./code-cgo
./code-cgo: ELF 64-bit LSB  executable, x86-64, version 1 (GNU/Linux), statically linked, not stripped
$> ldd ./code-cgo
        not a dynamic executable
$> ./code-cgo
hello, world!

Any news on that? Would like to use the static binary, without any linked dependencies.

So I manged to get it work:
diff --git a/build.py b/build.py
index 144bbe9..abba133 100755
--- a/build.py
+++ b/build.py
@@ -423,9 +423,10 @@ def build(version=None,
             build_command += "-race "
         go_version = get_go_version()
         if "1.4" in go_version:
-            build_command += "-ldflags=\"-X main.version {} -X main.branch {} -X main.commit {}\" ".format(version,
-                                                                                get_current_branch(),
-                                                                                get_current_commit())
+            build_command += "-a -installsuffix nocgo -ldflags=\"-s -extldflags \"--static\" -X main.version {} -X main.branch {} -X main.commit {}\" ".format(version,
+                                                                                get_current_branch(),
+                                                                                get_current_commit())
+            build_command = "CGO_ENABLED=0 " + build_command
         else:
             # With Go 1.5, the linker flag arguments changed to 'name=value' from 'name value'
             build_command += "-ldflags=\"-X main.version={} -X main.branch={} -X main.commit={}\" ".format(version,

The thing that worries me is that CGO_ENABLE=1 -- I have no idea if this has an impact on performance. Do we have an easy way to test it?

@ashb We have not tested the performance impact of static binaries, which is one of the reasons why we have not been offering them. I am working on a few changes to the build script to enable static binaries, so we are going to start testing on them more over the next few weeks. I'll send out updates on this thread if we encounter any noticeable performance issues, but for now I would say use at your own risk.

Test build https://hub.docker.com/r/niteoweb/docker-influxdb/ (alpine+nightly), works without problems atm


Lambda is given additional permissions than required

Lambda is given the permissions via the AWSLambdaExecute Policy which gives access to all the S3 buckets in the current AWS Account. The Lambda simply needs s3:GetObject permissions on the input S3 bucket and the s3:PutObject permissions on the output S3 bucket.
I have given this page as a reference to a lot many and they might be blindly following the documentation to give the unnecessary permissions to the Lambda. So, is this Issue.

The changes have been made/merged to the documentation with the below pull request.
https://github.com/awsdocs/aws-lambda-developer-guide/pull/217


Fix sectorMu deadlocks

This fixes the two issues I noticed in #50

Fixes a deadlock on startup when calling loadSectorLocations from newContractManager.
Fixes a deadlock in managedLockSector when waiting to acquire a sector lock.

Closes #52

Looks alright to me, but I'd feel more confident if @ChrisSchinnerl review this as well.

@lukechampine look good? Looks like only the flaky renter tests failed.


Upgrading helm release leads to crash

Describe the bug
cannot reload values if no data
To Reproduce
Steps to reproduce the behavior:

Go to Helm Releases
Choose to upgrade release
Close tab

Go to Helm Releases
Choose to upgrade release
Refresh page when tab is open

Screenshots
https://user-images.githubusercontent.com/3083083/179744125-1f2d34a9-d126-400e-9900-9eafb056bced.mov
https://user-images.githubusercontent.com/3083083/179744148-c3a12eb8-7a90-49f4-98ac-c1145f9abbde.mov
Environment (please complete the following information):

Lens Version: 5.6.0-alpha.6.2
OS: OSX

Duplicate of https://github.com/lensapp/lens/issues/5678


[Question] Integrate Timeout and Retry with an API ?

I tried to access the slack space to do this question, but I couldn't login there. seems that we need an invite.
I'm using Failsafe's Retry, Timeout and Fallback for some time, and they are working great for the majority of my team's use cases.
But I have a use case were they are not fitting as we need.
We have some process chains that depends on another team's jobs that send us (using ssh) some files which we need to use.
Sometimes those files are not sent in the proper time, so a failure occurs, and retried until a limited time.  But sometimes the resolution on the origin takes longer than the values we set for retries and timeout.
I'm thinking that one solution would be to increase the timeout and the delay between retries, or even cancel the retries (to use FallBack), but at runtime.
It would be possible to notify our monitoring tool through its API when an error have occurred, and also implement a rest to retrieve parameters, but how could we have Timeout and Retry policies modified and used after they have started?
Ideas and alternatives are welcome too :)

Without knowing the details: You can use listeners to notify your monitoring tool of Failsafe events. You can provide a computed delay function to a RetryPolicy that calculates the next delay based on runtime state. You can't change the duration of a Timeout, but you could compose a Timeout within a RetryPolicy to get something like the effect of a dynamically configurable Timeout.
Or you could consider standalone execution, just using Failsafe to record attempts, but controlling the flow yourself.


Error when building APK: "An asset is marked with HideFlags.DontSave ..." (on Unity 2023.2.6f1)

Hi!
I'm exploring this unity package and I'm having this issue when trying to build my APK file

Any ideas on how to solve this?
Thanks in advance!

Delete that image (photonvrlogo.png), its the simplest way

Delete PhotonVRLogo.png

Delete PhotonVRLogo.png. Does anyone know why that solution works? Is there something wrong with the picture?

Delete the logo yeah. I'm unsure why this issue is happening though. It could be because PhotonVR is made for Unity 2021 and you've been running it in 2023. Let me know if this works

fchb1239,
I think you are right that it's caused by the Unity version. I tested PhotonVR in 2022.3.X and 2023.X.X and rhapsodia's error occurs.

Thank you!


Request signature validation fixes

Passing null when the header is missing causes a TypeError, as the signature must be a string.
$_SERVER params were being accessed using undefined constants auto-converted to strings. Fix to use regular strings.
HTTP headers are usually keyed with HTTP_ prefix, and using all uppercase characters and underscores. The existing check for simply MessageBird-Signature-JWT did not pick up the header when using Apache. Maybe that works on other web servers, so I've left it as a fallback.

@ErikBooijMB is there anything I need to add to get this PR approved? I'm not sure how anyone could use signature validation currently as it seems to be entirely broken for us, and blocking using v3 of this package.


Support preview repository creation template API

GitHub Developer API announcement:
https://developer.github.com/changes/2019-07-16-repository-templates-api/
This would be a great PR for any new contributor to this repo or a new Go developer.
All contributions are greatly appreciated!
Feel free to volunteer for any issue and the issue can be assigned to you so that others don't attempt to duplicate the work.
Please check out our CONTRIBUTING.md guide to get started.
Thank you!

I'd like to take this up @gmlewis

@waseem18 I started this as a learning exercise, feel free to use my changes or redo it. I did not want to claim it in case it was over my head. I've linked the PR above. But I appear to need more code coverage. I will look into it tomorrow if you haven't finished your own. Take care!

Thank you, @waseem18, for your offer, and thanks also for your understanding about @shanempope putting together a PR.
Thanks, @shanempope, for your PR... I reviewed it and made some comments.  In the future, please place a note on the issue that you are working on it so others don't waste their time working on it too.  Thanks!


Added exceptionhandling for QSOs with no Call

Sometimes a Logbook-Tool called "HamOffice" exports wrong ADIFs.
There are QSOs without a partner. This one fixes the issue (QSOs will be skipped during import)

ADIF importer works with this fix. Can't really test the rest.


Allow robots to crawl search results

The search results pages will be prevented from being indexed by a meta
tag on the pages themselves.
See:

https://github.com/alphagov/digitalmarketplace-buyer-frontend/pull/189
https://github.com/alphagov/digitalmarketplace-aws/pull/162

:+1:


errors() method to return true if errors

It looks like errors() returns true if there are no errors and false if there are errors. This is counter-intuitive. But it may make MANY THINGS?

I'm strongly in favour of changing the documentation instead of changing the return value - otherwise you're going to break previous implementations.


Use yarn in Drone

Since yarn can use our lockfile this avoids the 1min setup npm does, which brings our total build time down to 5min.

Yeah, that was done because I was lazy and wanted to see how fast the build could potentially be.
I think it's likely that I'll merge the other PR first anyway.


[DOCS] Updates documentation version

Related to https://github.com/elastic/docs/issues/930
This PR updates the documentation version attributes from 6.8.0 to 6.8.1

@lcawl - I added a suggested to bump the SNAPSHOT version too.
@jbaiera - Can you review too ?


Error detected while processing function UnBundle

After running :PluginUpdate, I'm getting an error when starting Vim:
Error detected while processing function UnBundle:
line    2:
E121: Undefined variable: g:bundles
E116: Invalid arguments for function filter

Looks like a Vundle update broke the UnBundle function.

Seems they changed g:bundles to g:vundle#bundles. I added this to the top of my .vimrc.local to solve the problem temporarily:
if !exists("g:bundles")
    let g:bundles = g:vundle#bundles
endif

@ptzz Thanks for the fix.

Temporary fix is to comment out all the "UnBundle" in your .vimrc.local file.

Thank you for the fix; definitely an issue here too.


Add a useful, read-only admin interface

For figuring out where definitions/wordforms come from and how they are linked, I’ve found it very useful to have an internal read-only admin-only view where you can point-and-click to see how things are connected.

Codecov Report

Merging #630 (d61305d) into master (f6c3447) will decrease coverage by 0.59%.
The diff coverage is 81.48%.

@@            Coverage Diff             @@
##           master     #630      +/-   ##
==========================================
- Coverage   89.27%   88.68%   -0.60%
==========================================
  Files          50       51       +1
  Lines        1986     2076      +90
  Branches      305      321      +16
==========================================
+ Hits         1773     1841      +68
- Misses        153      174      +21
- Partials       60       61       +1

Impacted Files
Coverage Δ

CreeDictionary/API/admin.py
81.25% <81.25%> (ø)

CreeDictionary/CreeDictionary/urls.py
100.00% <100.00%> (ø)

CreeDictionary/API/search.py
89.66% <0.00%> (-0.74%)
:arrow_down:

CreeDictionary/DatabaseManager/xml_importer.py
95.45% <0.00%> (-0.63%)
:arrow_down:

CreeDictionary/DatabaseManager/__init__.py

CreeDictionary/DatabaseManager/__main__.py

...eDictionary/management/commands/wipedefinitions.py
50.00% <0.00%> (ø)

CreeDictionary/DatabaseManager/main.py
68.96% <0.00%> (ø)

CreeDictionary/DatabaseManager/log.py
100.00% <0.00%> (+3.03%)
:arrow_up:

... and 1 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f6c3447...d61305d. Read the comment docs.

Codecov Report

Merging #630 (d61305d) into master (f6c3447) will decrease coverage by 0.59%.
The diff coverage is 81.48%.

@@            Coverage Diff             @@
##           master     #630      +/-   ##
==========================================
- Coverage   89.27%   88.68%   -0.60%
==========================================
  Files          50       51       +1
  Lines        1986     2076      +90
  Branches      305      321      +16
==========================================
+ Hits         1773     1841      +68
- Misses        153      174      +21
- Partials       60       61       +1

Impacted Files
Coverage Δ

CreeDictionary/API/admin.py
81.25% <81.25%> (ø)

CreeDictionary/CreeDictionary/urls.py
100.00% <100.00%> (ø)

CreeDictionary/API/search.py
89.66% <0.00%> (-0.74%)
:arrow_down:

CreeDictionary/DatabaseManager/xml_importer.py
95.45% <0.00%> (-0.63%)
:arrow_down:

CreeDictionary/DatabaseManager/__init__.py

CreeDictionary/DatabaseManager/__main__.py

...eDictionary/management/commands/wipedefinitions.py
50.00% <0.00%> (ø)

CreeDictionary/DatabaseManager/main.py
68.96% <0.00%> (ø)

CreeDictionary/DatabaseManager/log.py
100.00% <0.00%> (+3.03%)
:arrow_up:

... and 1 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f6c3447...d61305d. Read the comment docs.


test_gluon_rnn.test_rnn_cells hang

For bugs or installation issues, please provide the following information.
The more information you provide, the more likely people will be able to help you.
Environment info
Operating System: DL AMI
Compiler:
Package used (Python/R/Scala/Julia):
MXNet version:
Or if installed from source:
MXNet commit hash (git rev-parse HEAD):  861e929cec8fa8fbab06884d9605debb74cd7217
If you are using python package, please provide
Python version and distribution:
If you are using R package, please provide
R sessionInfo():
Error Message:
Please paste the full error message, including stack trace.
....
test_executor.test_dot ... ok
test_executor.test_reshape ... ok
test_gluon_data.test_array_dataset ... ok
test_gluon_data.test_recordimage_dataset ... --2017-08-01 18:07:26--  http://data.mxnet.io/data/test_images.tar.gz
Resolving data.mxnet.io (data.mxnet.io)... <IP_ADDRESS>
Connecting to data.mxnet.io (data.mxnet.io)|<IP_ADDRESS>|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 2211939 (2.1M) [application/x-gzip]
Saving to: ‘data/test_images.tar.gz’

100%[=================================================>] 2,211,939   2.57MB/s   in 0.8s

2017-08-01 18:07:27 (2.57 MB/s) - ‘data/test_images.tar.gz’ saved [2211939/2211939]

ok
test_gluon_data.test_sampler ... ok
test_gluon_model_zoo.test_concurrent ... ok
test_gluon_model_zoo.test_identity ... ok
test_gluon_model_zoo.test_models ... ok
test_gluon_rnn.test_rnn ... ok
test_gluon_rnn.test_lstm ... ok
test_gluon_rnn.test_lstm_forget_bias ... ok
test_gluon_rnn.test_gru ... ok
test_gluon_rnn.test_residual ... ok
test_gluon_rnn.test_residual_bidirectional ... ok
test_gluon_rnn.test_stack ... ok
test_gluon_rnn.test_bidirectional ... ok
test_gluon_rnn.test_zoneout ... ok
test_gluon_rnn.test_rnn_cells ...

Minimum reproducible example
if you are using your own code, please provide a short script that reproduces the error.
Steps to reproduce
or if you are running standard examples, please provide the commands you have run that lead to the error.

run nosetests-2.7 -v ./tests/python/unittest/, it hangs when it comes to test_gluon_rnn.test_rnn_cells.

What have you tried to solve it?

run nosetests-2.7 -v ./tests/python/unittest/test_gluon_rnn.py, but it successfully passes the test

It indeed takes a lot longer for nosetests -v unittest to run through test_gluon_rnn.test_rnn_cells than for nosetests -v unittest/test_gluon_rnn.py. I have the same experience running it on my laptop.
The codes that this test is stuck at is the first invocation of the following function:
def check_rnn_forward(layer, inputs):
    layer.collect_params().initialize()
    with mx.autograd.record():
        layer.unroll(3, inputs, merge_outputs=True)[0].backward()
        mx.autograd.backward(layer.unroll(3, inputs, merge_outputs=False)[0])
    mx.nd.waitall()

By printing my way through the codes, it seems to be stuck at mx.nd.waitall() after it successfully exits the autograd.record(). I'm at a loss. @piiswrong what's your guess?

It didn't help changing mx.nd.waitall() to
for param in layer.collect_params().values():
    param.grad().wait_to_read()

You can fix this by deleting test_gluon_model_zoo.py
so I think it's because test_gluon_model_zoo is slow and didn't wait for it's computation to finish

You're right. I will add wait in model zoo tests.


Can't disable the 3D transform gizmo in the editor

Tested versions
v4.3.beta.custom_build [82cedc83c]
System information
Windows 11 - Vulkan (Forward+)
Issue description
There is an option to disable all gizmos in the editor's 3D view at Perspective > View Gizmos. However, the transform gizmo is the only one that doesn't hide.
Also, there is no option to hide only the transform gizmo in the scene's toolbar View > Gizmos. For reference, the 2D view has options to disable all or specific transformation gizmos.
Steps to reproduce

Create or select any Node3D.
3D scene view: Perspective > View Gizmos

Minimal reproduction project (MRP)
N/A

This is more of a missing feature than a bug, and is being implemented in:

https://github.com/godotengine/godot/pull/87793


problem running on CentOS Linux

I am having problems getting Parliament2 running in a Docker container on CentOS. I'll spare you the longer story. If I spin up a vanilla CentOS 7.4 VM on AWS, install docker, pull down dnanexus/Parliament2:0.1.9 and run it with this command:
sudo docker run -v /mnt/bucket00/projects/cnv/input/:/home/dnanexus/in -v /mnt/bucket00/projects/cnv/output/:/home/dnanexus/out dnanexus/parliament2:0.1.9 --bam test.bam --bai test.bam.bai --fai genome.fa.fai -r genome.fa --breakdancer --manta --cnvnator --genotype
I get this output:
Set up and index BAM/CRAM
BAM file input, index exists
mv: cannot move '/home/dnanexus/in/test.bam' to '/home/dnanexus/in/input.bam': Permission denied
mv: cannot move '/home/dnanexus/in/test.bam.bai' to '/home/dnanexus/in/input.bam.bai': Permission denied
touch: cannot touch '/home/dnanexus/in/test.bam': Permission denied
touch: cannot touch '/home/dnanexus/in/done.txt': Permission denied
Generate contigs
[E::hts_open_format] Failed to open file input.bam
samtools view: failed to open "input.bam" for reading: No such file or directory
mkdir: cannot create directory '/home/dnanexus/out/log_files/': Permission denied
mkdir: cannot create directory '/home/dnanexus/out/log_files': Permission denied
Launching jobs that cannot be parallelized by contig
Manta
parliament2.sh: line 167: /home/dnanexus/out/log_files/manta_logs/test.manta.stdout.log: No such file or directory
Processing bam input.bam
open: No such file or directory
Failed to open samfile input.bam
mkdir: cannot create directory '/home/dnanexus/out/log_files': Permission denied
Launching jobs parallelized by contig
mkdir: cannot create directory '/home/dnanexus/out/log_files': Permission denied
mkdir: cannot create directory '/home/dnanexus/out/log_files': Permission denied
mkdir: cannot create directory '/home/dnanexus/out/log_files': Permission denied
mkdir: cannot create directory '/home/dnanexus/out/log_files': Permission denied
mkdir: cannot create directory '/home/dnanexus/out/log_files': Permission denied
mkdir: cannot create directory '/home/dnanexus/out/log_files': Permission denied
mkdir: cannot create directory '/home/dnanexus/out/log_files': Permission denied
mkdir: cannot create directory '/home/dnanexus/out/sv_caller_results/': Permission denied
Converting results to VCF format
Convert Manta results to VCF format
No outputs of Manta found. Continuing.
Traceback (most recent call last):
File "/miniconda/bin/pip", line 4, in
import pip
File "/miniconda/lib/python2.7/site-packages/pip/init.py", line 14, in
from pip.utils import get_installed_distributions, get_prog
File "/miniconda/lib/python2.7/site-packages/pip/utils/init.py", line 27, in
from pip._vendor import pkg_resources
File "/miniconda/lib/python2.7/site-packages/pip/_vendor/pkg_resources/init.py", line 3130, in
@_call_aside
File "/miniconda/lib/python2.7/site-packages/pip/_vendor/pkg_resources/init.py", line 3116, in _call_aside
f(*args, **kwargs)
File "/miniconda/lib/python2.7/site-packages/pip/_vendor/pkg_resources/init.py", line 3143, in _initialize_master_working_set
working_set = WorkingSet._build_master()
File "/miniconda/lib/python2.7/site-packages/pip/_vendor/pkg_resources/init.py", line 633, in _build_master
ws = cls()
File "/miniconda/lib/python2.7/site-packages/pip/_vendor/pkg_resources/init.py", line 626, in init
self.add_entry(entry)
File "/miniconda/lib/python2.7/site-packages/pip/_vendor/pkg_resources/init.py", line 682, in add_entry
for dist in find_distributions(entry, True):
File "/miniconda/lib/python2.7/site-packages/pip/_vendor/pkg_resources/init.py", line 2110, in find_on_path
for entry in os.listdir(path_item):
OSError: [Errno 13] Permission denied: '/miniconda/bin'
ERROR: SVTyper requested, but candidate VCF files required to genotype. No VCF files found.
Traceback (most recent call last):
File "/home/dnanexus/parliament2.py", line 80, in
main()
File "/home/dnanexus/parliament2.py", line 76, in main
run_parliament(args.bam, args.bai, ref_genome_name, args.fai, prefix, args.filter_short_contigs, args.breakdancer, args.breakseq, args.manta, args.cnvnator, args.lumpy, args.delly_deletion, args.delly_insertion, args.delly_inversion, args.delly_duplication, args.genotype, args.svviz, args.svviz_only_validated_candidates, args.dnanexus)
File "/home/dnanexus/parliament2.py", line 49, in run_parliament
subprocess.check_call(['bash', 'parliament2.sh', bam, bai, ref_genome, fai, prefix, str(filter_short_contigs), str(breakdancer), str(breakseq), str(manta), str(cnvnator), str(lumpy), str(delly_deletion), str(delly_insertion), str(delly_inversion), str(delly_duplication), str(genotype), str(svviz), str(svviz_only_validated_candidates), str(dnanexus)])
File "/miniconda/lib/python2.7/subprocess.py", line 190, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['bash', 'parliament2.sh', '/home/dnanexus/in/test.bam', '/home/dnanexus/in/test.bam.bai', '/home/dnanexus/in/genome.fa', '/home/dnanexus/in/genome.fa.fai', 'test', 'False', 'True', 'False', 'True', 'True', 'False', 'False', 'False', 'False', 'False', 'True', 'False', 'False', 'False']' returned non-zero exit status 1
As you can see, the errors start with:
mv: cannot move '/home/dnanexus/in/test.bam' to '/home/dnanexus/in/input.bam': Permission denied
mv: cannot move '/home/dnanexus/in/test.bam.bai' to '/home/dnanexus/in/input.bam.bai': Permission denied
I think the inability to create the input.bam and input.bam.bai  files leads to all the subsequent errors. I've tried this on two other CentOS machines and generated similar errors. The /mnt/bucket00/projects/cnv/input and /mnt/bucket00/projects/cnv/output directories are 0777, so they should be writable by anyone.
However, if I run the exact same command on an AWS Ubuntu 18 LTS VM it runs to completion with no unexpected errors. Has anyone else seen this behavior? Is there an easy fix without forking and heavily customizing the code?
Thanks.

I have not tested this on a CentOS machine, unfortunately. This may be a naive attempt at a fix, but could you try using sudo docker run?
Otherwise, the Docker container is using Ubuntu 14.04, so I would look into running Ubuntu Docker containers on CentOS machines in general.

Thanks for answering. I got this working by not installing the system version of docker and following the directions on this page to install a more recent version: https://linuxize.com/post/how-to-install-and-use-docker-on-centos-7/ .