Protobuf

At long last, here is actual protobuf serialization and deserialization.
It's a lot of code because translating to and from the ProtoBuf generated classes is just repetitive and verbose.
Some decisions I made and the justification

No support for WIR

I see it as a follow on, this is enough to get Chisel -> FIRRTL via protobuf working which is my main goal

Using Java Protobuf instead of ScalaPB

ScalaPB provides nice Scala-y datastructures from the protobuf. It doesn't provide much if anything over the default Java implementation because we're just converting to and from it anyway (note this code would be just as verbose). The Java implementation is supported by the protobuf team so I didn't see sufficient reason to use ScalaPB

I still need to add support for protobuf input and output but I figured I should PR this now since it is ready for overall code review and this is enough for some performance benchmarking (which I haven't done yet).

Preliminary results but for a 420MB .fir file (460MB in .pb)
Parser
~90 seconds
51.5 GB

Protobuf
~7 seconds
13GB

Those memory use numbers include the FIRRTL AST which is ~4.5GB

@chick Something I was wondering specifically is what do you think about the change to the command-line API? It used to be if you gave an input file that didn't end in .fir, it would add .fir to the end. I changed it to no longer do that, using "fileNameOverride" exactly as given for the filename and then treating .pb files as protobuf and anything else as firrtl files (to remain similar to how it used to work).
This is similar to how tools like g++ work where it will treat a .c as C, .cpp as c++, but if it doesn't recognize the extension, it actually just errors (to be precise, if it doesn't recognize the extension, it passes it to the linker which will then error if it doesn't recognize the extension).

@jackkoenig This sounds fine to me. As things move forward some little convenience items like that can't always work the way they did before. I'd like to think no too many people rely on this and that those that have used this side effect have not built up too many scripts or Makefiles that rely on it.

I've also added an Apache 2.0 license for my contributions as an employee of SiFive. I'm following the pattern used in rocket-chip where the files I'm creating and modifying also get the notification.
We should probably clean up the current license as it's like the BSD 3-clause but phrased in an atypical order (and it also misspells license as "licence"), and rename it to LICENSE.Berkeley in the root directory as done in rocket-chip. However, I think that's a separate, self-contained PR.

@jackkoenig: possibly out of scope of this PR, but I'm commenting since you reference the license here...
Are contributions licensed individually or are files jointly licensed once touched? Do other organizations have the ability to choose how their contributions are licensed (BSD vs. Apache v2)? I expect that most non-academic organizations would prefer Apache due to the patent clause. That then brings up the question of how contributions from external organizations are licensed by default.
I expect that this whole situation could be remedied by bulk migrating to Apache v2 or clarifying with a DCO, etc (e.g., all contributions must be signed and will be under the Apache v2 license).

@seldridge I think it is beyond the scope of this PR, but I am the one who brought it up 😬
I'm going to remove all of that from this PR, but I think proper discussion of licensing and contributions is long overdue. I think it's relevant to the Free Chips Porject at large although there are probably small differences between how different projects are handled (for example, GCC clarifies that you can compile non-GPL programs, since we are not copy-left we don't have quite that big of a problem, but we might want to clarify that you don't need to include the Chisel and FIRRTL copyright notices in the product of Chisel+FIRRTL compilation).
In any case, I think this is a discussion for the Chisel Governance meeting and not for a PR, my apologies for bringing it up.

All good, Jack! I'm in total agreement with your comments: this is long overdue, an issue to be brought up with Chisel governance, and can be a model for how the Free Chips Project (or whatever the final name is) will operate. Obviously, if there's a business need and/or some specific worry on the SiFive side about the need to get a patent clause in there then it makes sense to do it now. Otherwise, I assume it can wait for a broader discussion.
I think this only hit me oddly as Chisel/FIRRTL is much more decoupled from SiFive than Rocket Chip (i.e., I fully expect SiFive to run the licensing of Rocket Chip however they want and to protect themselves there especially since Rocket Chip is, at this point, a fully SiFive artifact).
Yes, the notion of this generating circuits should likely be clarified, as well. I have no idea how that works or if there's some motivation in investigating how the patent clause would extend to artifacts produced.

There's nothing specific, this was just a large contribution and got me thinking about licensing issues. As you said, I think most non-academics organizations (like SiFive) would be more comfortable with an Apache license. In any case this issue can wait until we can talk about this in a governance context and come to a consensus!

I think this is ready to go. @chick has looked at the bulk of it. The only big change since then is the is the addition of the ToProto.writeToStream* methods. I'm ready to merge this, and hope people will take a/another look.
ToProto.writeToStreamFast is perhaps the main thing that could be contentious so I should explain. The normal ProtoBuf serialization looks something like FIRRTL -> ProtoBuf -> serialized, the intermediate ProtoBuf data structure is about the same size as the FIRRTL representation. For small designs this is irrelevant, but for large designs it's a substantial increase to memory use. Because the intermediate structure is large, it survives several small GCs and ultimately gets promoted to the Tenured heap (which is expensive). I wrote ToProto.writeToStreamFast to improve serialization to do the FIRRTL -> ProtoBuf -> serialized on a Module at a time instead of the whole circuit. This allows the intermediate ProtoBuf objects to be garbage collected during serialization while in the Young generation rather than afterwards while [frequently] Tenured.
That doesn't explain the "deconstructed" Circuit function arguments. Fundamentally, moving from a custom written Emitter in Chisel to multi-step serialization is going to be slower. When I used just the naive way of translating the whole circuit one step at a time, the slowdown for Chisel serialization wasn't great. For Chisel, ProtoBuf serialization has an additional step: ChiselIR -> FIRRTL -> ProtoBuf -> serialized. By deconstructing the Circuit and accepting () => DefModule, the intermediate FIRRTL objects can be garbage collected in the same way as the ProtoBuf objects. While ProtoBuf serialization is still slower than the existing Chisel emitter, ToProto.writeToStreamFast still has a nice benefit in serialization times:
I'll let FastProto be Chisel serialization using writeToStreamFast and SlowProto be serialization using naive full Circuit -> FIRRTL -> ProtoBuf -> serialized

Config
Chisel Elaboration
Chisel Emitter
SlowProto
FastProto

Medium
10 s
0.85 s
1.9 s
1.4 s

Large
100 s
5.6 s
16 s
7.2 s

Huge
1060s
40 s

19s

Small (<5%) performance hit for most Chisel designs, small improvement for huge designs, and a nice gain for FIRRTL:
These result are for full compilation (Parser was/is memory bottleneck):

Config
w/ Parser Time
w/ Parser Memory
w/ Proto Time
w/ Proto Memory

Medium
55 s
800 MB
50 s
700 MB

Large
280 s
7 GB
260 s
4 GB

Huge
422 s
40 GB
400 s
15 GB

I've done another once over and it still looks good to me. I few nits and comments

IntelliJ did not seem to know how to get started with this branch, but it cleared itself up when ran sbt and did clean and test:compile
FromProto style violation

Line 143 def convert does not declare it's return type

ToProto style violations

Lines 135, 140, 145 Suspicious shadowing of variable padding
In same section as above, tpe is set in pattern match several times but not used
There's a few more nits that IntelliJ's checker complains about but I don't think are necessarily what we consider bad style (though it would be nice to get theirs matched up with ours)
Tests all passed.
Otherwise, as before, LGTM

Alright I've integrated @chick's/IntelliJ's suggestions (I even used IntelliJ to do it!)
As @chick noted, this adds an additional code generation task and unfortunately IntelliJ seems to struggle with them. You either need to run compile on the command-line before trying to build with IntelliJ or you can run the code gen tasks manually:
$ sbt
> protobufGenerate
> antlr4:antlr4Generate

Obnoxiously, protobufGenerate deletes the src_managed directory so it needs to be run first, otherwise it'll delete the ANTLR-generated files.

Barring objection I'm going to merge this once Travis passes


Add hover effect to play button

Trying to get into sfml by applying a slight hover effect to the play button in the start menu. This also removes the hardcoded <= 130 which at least on my machine was a bit off. Now the hover/click only fire when the mouse is actually inside the button.

Regarding the compilation errors: Yeah, looking at https://packages.ubuntu.com/trusty/libsfml-dev it seems trusty comes with sfml-dev 2.1. I don't know what we require, but it a bit behind the most recent release which is 2.4.

I'll have a look at changing to a different distro which supports 2.4


FasterTransformer deberta example run failed

Branch/Tag/Commit
release v5.3
Docker Image Version
nvcr.io/nvidia/tensorflow:22.11-tf2-py3, tensorflow version:2.10.0
GPU name
A30
CUDA Driver
Driver Version: 470.57.02    CUDA Version: 11.8
Reproduced Steps
python deberta_example.py with following errors:

Traceback (most recent call last):
  File "deberta_example.py", line 116, in <module>
    main(vars(args))
  File "deberta_example.py", line 54, in main
    ft_deberta = FTDebertaModel(lib_path, ft_deberta_weight)
  File "/workspace/FasterTransformer/examples/tensorflow/deberta/../../../examples/tensorflow/deberta/utils/ft_deberta.py", line 154, in __init__
    self.transformer_op_module = tf.load_op_library(lib_path)
  File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/load_library.py", line 54, in load_op_library
    lib_handle = py_tf.TF_LoadLibrary(library_filename)
tensorflow.python.framework.errors_impl.NotFoundError: /workspace/FasterTransformer/examples/tensorflow/deberta/../../.././build/lib/libtf_deberta.so: undefined symbol: _ZN10tensorflow11GetNodeAttrERKNS_9AttrSliceEN4absl12lts_2022062311string_viewEPb

It should be because TF change the CXX_STD from 14 to 17 in tf-2.10.
You can try tf-2.09 or change https://github.com/NVIDIA/FasterTransformer/blob/main/CMakeLists.txt#L104 to 17.


Fix: Unwanted 'by' text in '/blog'

Signed-off-by: toth2000<EMAIL_ADDRESS>Description
Update: if condition for rendering by {{page.author}} span.
This PR fixes #275
Notes for Reviewers
Signed commits

[x] Yes, I signed my commits.

LGTM!


logger: Report item sizes for fetch, mutation, and eviction watchers

This change proposes the addition of a new tag size (int) for watchers on fetchers, mutations, and evictions that expresses the value size of the associated item in bytes.
This has been quite useful, for example, for building histograms of item value sizes partitioned by key prefix for traffic into a large shared-tenancy cluster.
If there's interest in this change, I'm also happy to add a test to t/watcher.t if desired.
Some sample representative output from watch fetchers mutations:
ts=1621616365.760769 gid=13491372 type=item_store key=<scrubbed> status=stored cmd=set ttl=360 clsid=8 cfd=134 size=350
ts=1621616365.763121 gid=13491375 type=item_get key=<scrubbed> status=not_found clsid=0 cfd=134 size=0
ts=1621616365.763121 gid=13491376 type=item_store key=<scrubbed> status=stored cmd=set ttl=360 clsid=8 cfd=134 size=350
ts=1621616365.766095 gid=13491377 type=item_get key=<scrubbed> status=found clsid=8 cfd=144 size=350
ts=1621616365.766215 gid=13491378 type=item_get key=<scrubbed> status=found clsid=8 cfd=99 size=350
ts=1621616365.770232 gid=13491379 type=item_get key=<scrubbed> status=not_found clsid=0 cfd=40 size=0
ts=1621616365.781125 gid=13491380 type=item_get key=<scrubbed> status=found clsid=8 cfd=111 size=350
ts=1621616365.781126 gid=13491381 type=item_store key=<scrubbed> status=stored cmd=set ttl=360 clsid=8 cfd=111 size=350
ts=1621616365.781809 gid=13491382 type=item_get key=<scrubbed> status=found clsid=8 cfd=99 size=350
ts=1621616365.781810 gid=13491383 type=item_store key=<scrubbed> status=stored cmd=set ttl=360 clsid=8 cfd=99 size=350
ts=1621616365.791614 gid=13491384 type=item_get key=<scrubbed> status=not_found clsid=0 cfd=78 size=0

Thanks! I'm a bit behind right now but I'll get to PR's soon.
I feel like there was some reason why I didn't add the actual size at the time, but if I can't remember why I wouldn't say no to this. :)

Think this is fine. Anything else you want to do? any squashing/cleanup?
Thanks!

Thanks! I think the change is ready to go. I rebased against next and squashed it down.


GH-1268 Fix mosaic layout for daily rollup

Closes #1268

Not fully working for me. For instance:
* Browser is maximized. I load rollup - layout looks good. I switch to full screen (F11). A scroll bar appears even if there is more realestate. I exit full screen mode - scroll bar remains

* Browser is maximized. I load rollup - layout looks good. I resize the browser. Scroll bar appears. I maximize again and it remains.

@vassil Which browser/OS? It's working correctly for me…


Adding some tests for is_valid_signature

[X] issue #113
[X] follows contribution guide
[X] code change includes tests
[ ] breaking change

This PR is adding some tests for the is_valid_signature function. Still work in progress.

For the empty signature case. I'm getting this as a value when I run the test:
[FAIL] tests::test_account::test_empty_signature

Failure data:
    0x696e76616c6964207369676e6174757265 ('invalid signature')

And in the test I'm comparing the value to 'invalid signature' but it seems that is not catching it. I dont know how that works.

The test_is_not_valid_signature is working fine. But I dont know how to do a test that checks that the signature is working fine. Or the function is working as expected. Is there any resource that could be shared with me to see how to do that?

Contributor

thats fine


FreeCodeCamp course

Is your feature request related to a problem? Please describe.
Can I add a FreeCodeCamp resource to the README.md file.
Describe the solution you'd like
I would like to add this YouTube video: https://www.youtube.com/watch?v=gyMwXuJrbJQ.
It's a 32 hour course about Blockchain, Solidity, and Full Stack Web3 Development with JavaScript.
Describe alternatives you've considered
No response
Additional context
This will be me first PR in an Open Source repo. So any help woudl be appreciated.

we are not focused on a single platform, but rather on tutorials on different blockchains. that video is already in the available resources


Failed to establish a new connection: [Errno 101] Network is unreachable

Thanks for sharing your great work. When I use modeling.py for pharmacophore modeling, the following error is reported:
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='drive.google.com', port=443): Max retries exceeded with url: /uc?id=1gzjdM7bD3jPm23LBcDXtkSk18nETL04p (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x2b0755d5eed0>: Failed to establish a new connection: [Errno 101] Network is unreachable'))
In fact, the computing cluster I am using now is not connected to the Internet. Could I run your code without an Internet connection?
Many thanks

The internet is required only once to download the trained deep learning model file.
I'll update the code so that you can download and load the model manually in 3-4 days.

The internet is required only once to download the trained deep learning model file. I'll update the code so that you can download and load the model manually in 3-4 days.

Sorry I just saw the reply. Could you please explain it in the code after you update the code? So I can download and use it according to the instructions. Thank you very much for your help.

The internet is required only once to download the trained deep learning model file.
I'll update the code so that you can download and load the model manually in 3-4 days.

I apologize for the delay.
I am currently participating the NeurIPS conference, and cannot access my school internal network due to VPN issue.
I will update as soon as possible after the conference ends.

The internet is required only once to download the trained deep learning model file.
I'll update the code so that you can download and load the model manually in 3-4 days.

I apologize for the delay. I am currently participating the NeurIPS conference, and cannot access my school internal network due to VPN issue. I will update as soon as possible after the conference ends.

Hi,
I recently downloaded the following from a link I found in the code, but I don't know whether these are all the files I need and how to use them.

If you have been quite busy recently, could you please let me know how to use these files? This would allow me to do some simple trials in the meantime. I would appreciate your reply and look forward to successfully using your code.

Sorry. Now I'm fine.
I updated the code for pharmaconet, but I feel what you need is the offline setup for reward function. Right? (e.g., TacoGFN or RxnFlow)
It would be helpful for what function do you need.

Sorry. Now I'm fine.
I updated the code for pharmaconet, but I feel what you need is the offline setup for reward function. Right? (e.g., TacoGFN or RxnFlow)
It would be helpful for what function do you need.

Thank you very much for your help, I ran screening.py successfully and got the results. The screening speed is very impressive, which is very helpful to me.
Thank you again for your patience and help

I'm glad to hear that you solved the problem well.


Convert the project into typescript

Description
This attempted to convert the entire project to typescript. This is a big change but I think it will improve the code quality of the library. It also is an essential part of my goal to convert this project to AssemblyScript so that I can use the (currently extremely alpha) Rosetta compiler to convert this into Kotlin so that there are bindings for android. That change itself is most likely out of scope of this library, although it might be beneficial to the library's performance, regardless that change won't be complete for a good amount of time because there are a lot of huge issues converting a dynamically typed javascript-y library into a strictly typed language with a similar but not identical standard library. Back on topic, this is just part 1 of that change, and simply intends to convert the project into typescript.
This currently is not finished in the slightest.
Type of change

[X] New feature (non-breaking change which adds functionality)
[X] This change requires a documentation update

Checklist:

[ ] My code follows the style guidelines of this project
[ ] I have performed a self-review of my own code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[ ] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] New and existing unit tests pass locally with my changes
[ ] I have checked my code and corrected any misspellings

This will probably be helpful to you: https://gist.github.com/Wykerd/6a97a810340f5382d9c38360bd12182c
It's the original TypeScript files I wrote for the NToken and Signature deciphering classes.

Sweet! Thanks!


Add promsocatc.net - Collège Technique "Aumôniers du Travail"

Official website:
http://www.promsocatc.com
Bachelor in IT program:
http://www.promsocatc.com/formation/bachelier-en-informatique-de-gestion/
The public website is in .com, but students/teachers mails are in .net
See http://www.promsocatc.com/plateforme/Plateforme_promsocatc.pdf p. 2
(point 1.3)

@dereckson Pull request merged. Thank you.


Add more tests

Also fix nasty bug with looping over non-reference values and trying to modify them.

Codecov Report

Merging #78 into master will increase coverage by 1.19%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master      #78      +/-   ##
==========================================
+ Coverage   81.54%   82.74%   +1.19%
==========================================
  Files          19       19
  Lines        1252     1252
==========================================
+ Hits         1021     1036      +15
+ Misses        165      147      -18
- Partials       66       69       +3

Impacted Files
Coverage Δ

client/reconciler.go
69.79% <100%> (+15.62%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f888928...8f32f8c. Read the comment docs.


sensor.kwh_to_home shows zero

Hi,
sensor.kwh_to_home shows 0 for me. Isn't this sensor supposed to report today's energy? On the website, I can see 10.1kWh for today.

Try looking at the data your system reports directly:
curl http://<IP_ADDRESS>/cgi-bin/dl_cgi?Command=DeviceList (or whatever host you are using)
Look for
"neg_ltea_3phsum_kwh" which is "KWH To Grid"
"pos_ltea_3phsum_kwh" which is "KWH To Home"
My Consumption meter ( PVS6M0400c ) reports
"neg_ltea_3phsum_kwh":	"0",
"pos_ltea_3phsum_kwh":	"12619.08",
This is the consumption of my house.  I never see kwh to grid because the clamp is installed on the feed to my panel and my solar backfeeds directly into the meter (via a different breaker panel).  I do see data on my 'p' meter for production though.

The number from the website, is that produced power or consumed power? (or do you have both)
I know some systems don't get installed with all of the current clamps so sometimes this data is zero
check the 'P' meter it has a 'lifetime power' entity which should be the production from your panels
Feel free to attach the output from above and we might be able to see whats happening (it does have your serial numbers in it you might want to remove)


NEP: Improve Pricing Structure

In order to solve some underlying issues with the current gas pricing structure, I've proposed adjusting the gas ratio and dropping the free gas cap, in addition to changing how Storage.Put cost is derived as mentioned in https://github.com/neo-project/neo/issues/159,.
A consensus mechanism for dynamic gas ratio would be nice, but the implementation for such an endeavour is unclear at the moment.
Other individual op costs can also be further simplified or tweaked based on what we now know, as compared to when NEO was launched.

Just wanted to provide some additional details on the matter. Under the current fee model, NEP5 token transfers take about 2.4~ gas per transfer (1 CheckWitness, 2 Get commands to pull balance of each account, 2 Put commands to update balances of each account; less if the from account balance becomes 0, then the total gas is 1.5~). So we can safely batch 3 transfer commands into a single transaction.
At the current limit of 500 transactions per block, we are limited to a max of 1500 NEP5 transfers per block. Depending on the block times, this leads to a max transfers per second of 75-60 (20-25 second block times).
If we were to update the fee model as suggested, retaining the gas_free amount, updating the min Put cost to 0.1 gas, and assuming that a simple Put of the account token balance falls within this amount. We can possibly expect a the cost per NEP5 transfer decreased to 0.6~. Which will allow batches of roughly 16 transfers per transaction. Resulting in a new max of 8000 NEP5 transfers per block, a 533% increase, and a new transfers per second of 400-320 (20-25 second block times).
This will help to reduce the load during airdrops and bulk KYC registrations. For KYC registrations, the increase would be from 200-160 to 825-660 registrations per second.
This figures are all assuming the transactions are all within the gas_fee, and that it is kept in place at 10 gas per transaction. However, if it is removed, these numbers become irrelevant since the the cost for making such transactions won't change whether they are batched in much higher numbers or not at all.

In NEO 3.0, prices will be significantly reduced.

Pricing structure will be modified in NEO 3.0. I will close this PR.


slack: init at 2.0.1

Things done:

[ ] Tested using sandboxing (nix-build --option build-use-chroot true or nix.useChroot on NixOS)
Built on platform(s)

[ X ] NixOS
[ ] OS X
[ ] Linux

[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[ X ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Fits CONTRIBUTING.md.

More
Fixes issue #
cc @

Please note, that points are not mandatory, but rather desired.

This is unrelated, but I tried building and got:
curl: (22) The requested URL returned error: 503 Service Unavailable
/nix/store/wfh61l85g730j6l1qg58i9bfmsy5jdlh-xz-5.2.2/bin/xz: (stdin): File format not recognized
error: unexpected end-of-file
download of ‘https://cache.nixos.org/nar/1l0vpnwg7bjwbj7ghgd0v52w2d5bfagdjrdbh101dfyzvbmkdyyq.nar.xz’ failed: No such file or directory
could not download ‘/nix/store/y5j91bndalrgf7idifrzcqw24pjd45f5-spidermonkey-17.0.0’ from any binary cache
fetching path ‘/nix/store/y5j91bndalrgf7idifrzcqw24pjd45f5-spidermonkey-17.0.0’ failed with exit code 1
killing process 12590
killing process 12589
error: some substitutes for the outputs of derivation ‘/nix/store/cf01mahy0a10xjzkj2b7ix9p71cif24f-gconf-2.32.4.drv’ failed (usually happens due to networking issues); try ‘--fallback’ to build derivation from source

When I cherry-picked to 16.03 (where I want to use it) and rebuilt, I get:
rmdir: failed to remove '/nix/store/191wxvaimw85glsky9xpyh6ddlaxwz5a-slack-2.0.1/usr': Directory not empty
builder for ‘/nix/store/z617zbw317qaqvyi1y0m321alkgm9pcc-slack-2.0.1.drv’ failed with exit code 1
error: build of ‘/nix/store/z617zbw317qaqvyi1y0m321alkgm9pcc-slack-2.0.1.drv’ failed

cc @volhovM

Pushed in https://github.com/NixOS/nixpkgs/commit/7fe888e521fc523b77b1a4208b12c858d127e457

Yay! Thank you. Since this is so small but many users will want it, can you backport to 16.03?

On Mar 23, 2016, at 3:30 AM, Arseniy Seroka<EMAIL_ADDRESS>wrote:
Pushed in 7fe888e
—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub

@grahamc AFAIK we do backport updates to pkgs that were already in 16.03 there. If you need to use it then use unstable channel.

My understanding is the policies aren't so well defined or strictly enforced. I've counted many people in #nixos on Freenode wanting to use and bundle slack, and until recently was broken. Other than policy, many users will be happy for this. Other than policy, is there a reason to not backport it to the next release? This isn't a small program few people use.

That will make no difference between unstable and 16.03 then.
On 23 March 2016 15:32:16 Graham Christensen<EMAIL_ADDRESS>wrote:

My understanding is the policies aren't so well defined or strictly
enforced. I've counted many people in #nixos on Freenode wanting to use and
bundle slack, and until recently was broken. Other than policy, many users
will be happy for this. Other than policy, is there a reason to not
backport it to the next release? This isn't a small program few people use.

You are receiving this because you modified the open/close state.
Reply to this email directly or view it on GitHub:
https://github.com/NixOS/nixpkgs/pull/14124#issuecomment-200328729

Well OK, but note I disagree. I would prefer a user have the common packages they need much closer to them, than require them learn how to get unstable but retain a strict difference between 16.03 and master. I can understand the spirit of what you say, though and will let it drop.

So, sorry I didn't get back before now (eastern time, had work).  I had switched cp to mv but I think I accidentally undid that just before committing.  Should I resubmit a PR with that rebased on, submit a PR with only that change (how to comment, then), or just leave it as is?

@greyson I've done everything.


Failed to load model for eachadea/ggml-vicuna-7b-1.1

After I downloaded eachadea/ggml-vicuna-7b-1.1's ggml-vicuna-7b-1.1-q4_0.bin
model from https://huggingface.co/eachadea/ggml-vicuna-7b-1.1/tree/main, I was able to add Chat Source successfully.
However, during the conversation, an error "Failed to load model" occurred.
I also tried llama.cpp, and I could load the model only after updating to the latest llama.cpp. The llama.cpp from 5 days ago would also fail to load the model. I'm not sure if the ggml model in llama.cpp has been modified in any way.

Hey @fakechris, I know there have been some changes to llama.cpp in the last week, I'm working on updating the bindings so that these are now supported. I haven't tested Vicuna support specifically either, and that's coming

Vicuna works with the same sort of parameters as plain llama, but requires the "User:" prompt to be used AFAIK


Recursive directory walks and unit tests

Hey Josh, this was super handy! I ran into a weird issue where I exceeded MacOS' argument size limit:
❯ python3 s3logparse.py toptalkers ./
Failed to execute process '/opt/homebrew/bin/python3'. Reason:
The total size of the argument and environment lists 9.8MB exceeds the operating system limit of 1MB.
Try running the command again with fewer arguments.

¯\(ツ)/¯
So I ended up implementing a directory walk to build the argument list. I also added some unit tests and set up GH Actions while I was at it (I wanted to make sure I didn't break anything).
Feel free to pick up as much or as little as you like =]

You're going to make people think I'm a professional developer.¯(ツ)/¯ Thank  you!


Does not load in Edge browser

Perhaps a polyfill is needed. Might be related to #40.

I've added multiple polyfills:
import "react-app-polyfill/ie11";
import "react-app-polyfill/stable";
// https://developer.mozilla.org/en-US/docs/Web/API/TextEncoder
import "./polyfill/textencoder";

It works!


MM-12574 Track events for interactive messages and dialogs

Summary
Track events for interactive messages and dialogs.
Ticket Link
https://mattermost.atlassian.net/browse/MM-12574

@grundleborg updated


⚠ ClamAV has degraded performance

In d9605e5, ClamAV (https://spamassassin.apache.org/) experienced degraded performance:

HTTP code: 200
Response time: 214 ms

Resolved: ClamAV performance has improved in 45f061c.


northd, controller: Commit flows dropped by ACLs in a separate zone

Each logical port is assigned a new zone for committing dropped flows.
The zone is loaded into register MFF_LOG_ACL_DROP_ZONE.
A new lflow action ct_commit_drop is introduced that commits the flow
to connection tracking table in a zone identified by
MFF_LOG_ACL_DROP_ZONE register.
An ACL with "drop" action and label is translated to ct_commit_drop
instead of silently dropping the packet.

LG, please go ahead and share the changes with upstream folks.


added str_which, str_ends, and str_starts

Take your time! Docstrings always have tricks up their sleeves. Thank you

I got them to work. There was a problem with the spacing


Add thumbnail image

Add a thumbnail image to the dataset detail page and possibly search result list as well.

I need more information. What image, is it being generated somewhere?

@gxtchen They were talking about the thumbnail image that shows up when you do a search in PO.DAAC now:
https://podaac.jpl.nasa.gov/datasetlist?search=wind
Maybe you can scrape it straight from PO.DAAC?

@fgreg @gxtchen Actually, the image URL of each dataset is contained in the metadata attribute "Dataset-ImageUrl". For instance, "Dataset-ImageUrl": "https://podaac.jpl.nasa.gov/Podaac/thumbnails/CCMP_MEASURES_ATLAS_L3_OW_L2_5_SSMI_F8_WIND_VECTORS_FLK.jpg"


Consider exposing named colors map

There's a case for me that I need to convert RGB value to named color back (if it's available), but this crate doesn't expose that named colors map so I can't do it. Is there a plan that considering exposing it?

I added a new Color's method name() that will return color name if it's available.
assert_eq!(Color::from_rgba8(255, 0, 0, 255).name().unwrap(), "red");

Is it sufficient for your needs?

I've checked the commit: https://github.com/mazznoer/csscolorparser-rs/commit/20ef912ba1bd452af2eb951abb310ad87b55d83e . Would it be better that using Option instead of Result? For my opinion, it means "whether there's a name for such color or not", not an error about missing name.

I agree with you.


What happens if user tries to import from ledger twice

I don't have ledger to try this, so here are only first two steps (have two ledger windows open):

Open import from ledger, but don't complete the flow
Open another import from ledger
(have two ledger windows open)
Complete the flow to import from ledger
It seems to me that listener would get triggered twice: https://github.com/oasisprotocol/oasis-wallet-ext/blob/f912f132cc22bec5d70541ddf46b90d7d28c6c6d/src/utils/ledger.js#L12-L24

I will investigate this one

The behavior that I observed in trying to reproduce this seems ok--it allows you to import each address only once.


Neovim configuration

As it seems the language server does not call spago sources when using Neovim.
Steps to reproduce:
git clone https://github.com/mkohlhaas/qqqq
cd qqqq
nvim src/Main.purs

Then using goto definition for log works fine and jumps to source code in the .spago directory. But then I get the following error:
Output directory does not exist at '/home/schmidh/Gitrepos/qqqq/.spago/console/v5.0.0/output'. Ensure project is built, or check configuration of output directory and build command..

Ensure project is built with the same purs version as the IDE server is using

Request Actions:
1. Build project
Type number and <Enter> or click with the mouse (q or empty cancels):

and further code navigation is not possible.
Language server configuration:
        → purescript.addNpmPath               default: false
        → purescript.addPscPackageSources     default: false
        → purescript.addSpagoSources          default: true
        → purescript.autoStartPscIde          default: true
        → purescript.autocompleteAddImport    default: true
        → purescript.autocompleteAllModules   default: true
        → purescript.autocompleteGrouped      default: true
        → purescript.autocompleteLimit        default: null
        → purescript.buildCommand             default: "spago build --purs-args --json-errors"
        → purescript.buildOpenedFiles         default: false
        → purescript.censorWarnings           default: []
        → purescript.codegenTargets           default: null
        → purescript.declarationTypeCodeLens  default: true
        → purescript.exportsCodeLens          default: true
        → purescript.fastRebuild              default: true
        → purescript.formatter                default: "purty"
        → purescript.fullBuildOnSave          default: false
        → purescript.importsPreferredModules  default: ["Prelude"]
        → purescript.outputDirectory          default: "output\/"
        → purescript.packagePath              default: ""
        → purescript.preludeModule            default: "Prelude"
        → purescript.pscIdePort               default: null
        → purescript.pscIdelogLevel           default: ""
        → purescript.pursExe                  default: "purs"
        → purescript.sourcePath               default: "src"
        → purescript.trace.server             default: "off"

Code navigation with the same project in VS Code works perfectly fine!

It looks like your editor is incorrectly configured regarding the workspace root, are you using nvim-lspconfig per the README? In particular the root_dir should be set correctly
I don't use neovim so I don't know how this works, but regardless of editing a file nested inside .spago, you should connect to the language server with a workspace folder of the actual purescript project (the folder with output/ in).
(Edited the title, this has nothing to do with spago sources, which you have actually confirmed was working correctly)

Thanks Nicolas,
I was investigating the problem a bit more and it seems to be a problem with nvim-lspconfig. I will open an issue over there.
As it seems I am one of the few who work with PureScript under Neovim. Visual Studio Code is much more popular these days.
Thanks a lot for your language server. Makes working with PureScript a lot easier.
Purerl is also an amazing project!

Nicolas,
Michael from the nvim-lspconfig project is asking if setting the root directory to where the output directory is, would solve the problem. I use PureScript only with Spago, so I'm not sure how it would work with other project schemes (Bower, ...).
So the question is: Would setting the root directory to output be the solution to the problem?
Thanks for your help!

That's not a general solution, because output is configurable, but for a particular user, if you do not reconfigure output, it's a pretty good indicator. However that will only be present if the project is already built, which will be a difference even in the working case today.
Otherwise you can have some random file that your projects use but is never checked into git for a dependency


Adds a blog post for forecasting timeseries data in VSCode Jupyter Notebooks.

Leverages blob storage (and logs derived from app insights) as persistance to demonstrate ARIMA prediction and result persistance.
The actual notebook can be found within the public URL within the blog itself; PRing notebooks has always been an "interesting" concept so it seems fitting to review the markdown generated off of the notebook instead.

Closing this in leiu of the properly formatted version here. (Thanks Adrian++)


After Restart got Error: "Failed to setup INTx fd: Operation not Permitted"

With Ubuntu everything went smooth.
MacOS installed correctly after the restart got the error.

If an Interrupt Remapping option is available in your BIOS, please enable it.

@Ronsor No there isn't option like this. These are the only options. I have attached screenshot.

Same error here

CPU not supported?


How to get the records on which the scheduled DLRS failed to process

Hi,
we are getting below exception when schedule job runs :
"Apex script unhandled exception by user/organization: 005b0000000mPx4/00Db0000000IrDI
Failed to process batch for class 'dlrs.RollupJob' for job id '7070J00005zEVc3'
caused by: System.LimitException: dlrs:Too many query rows: 50001"
I want to know which all records actually failed to recalculate in a chunk.
Is there any way to identify these records?

@aheber this soooooo needs to go into a wiki page! Nice explanations!

Hi there - I've encountered this issue and have adjusted the batch size and record scope numbers way down, and I still can't get passed this error. I'm getting the error in dataloader when trying to load new records (the new records are children in DLRS)...is it possible DLRS isn't the problem (we don't have much installed in sfdc and don't have devs creating custom stuff in there)? Or if it is, am I missing something in troubleshooting?

@jmiz that is indeed possible. Have you enabled Debug Logs and counted the queries?

I just did that - thank you! But I don't know how to read the logs. Here is what I think is a meaningful excerpt. I will say that currently our data load batch size is 10 and record scope is 10, even though our top 10 parent records together have far fewer than 50k child records (and we don't have layered/cascading DLRS). So like, it should be working, in theory?
08:49:30.761<PHONE_NUMBER>)|FATAL_ERROR|System.LimitException: dlrs:Too many query rows: 50001
Class.dlrs.LREngine.QueryExecutor.query: line 659, column 1
Class.dlrs.LREngine.WithSharingQueryExecutor.query: line 668, column 1
Class.dlrs.LREngine.rollUp: line 238, column 1
Class.dlrs.LREngine.rollUp: line 117, column 1
Class.dlrs.RollupService.updateMasterRollupsTrigger: line 989, column 1
Class.dlrs.RollupService.handleRollups: line 846, column 1
Class.dlrs.RollupService.triggerHandler: line 301, column 1
Trigger.dlrs_CatalogTrigger: line 7, column 1
08:49:30.761<PHONE_NUMBER>)|FATAL_ERROR|System.LimitException: dlrs:Too many query rows: 50001
Class.dlrs.LREngine.QueryExecutor.query: line 659, column 1
Class.dlrs.LREngine.WithSharingQueryExecutor.query: line 668, column 1
Class.dlrs.LREngine.rollUp: line 238, column 1
Class.dlrs.LREngine.rollUp: line 117, column 1
Class.dlrs.RollupService.updateMasterRollupsTrigger: line 989, column 1
Class.dlrs.RollupService.handleRollups: line 846, column 1
Class.dlrs.RollupService.triggerHandler: line 301, column 1
Trigger.dlrs_CatalogTrigger: line 7, column 1
08:49:31.26<PHONE_NUMBER>)|CUMULATIVE_LIMIT_USAGE
08:49:31.26<PHONE_NUMBER>)|LIMIT_USAGE_FOR_NS|(default)|
Number of SOQL queries: 0 out of 100
Number of query rows: 0 out of 50000
Number of SOSL queries: 0 out of 20
Number of DML statements: 0 out of 150
Number of DML rows: 0 out of 10000
Maximum CPU time: 4599 out of 10000
Maximum heap size: 0 out of 6000000
Number of callouts: 0 out of 100
Number of Email Invocations: 0 out of 10
Number of future calls: 0 out of 50
Number of queueable jobs added to the queue: 0 out of 50
Number of Mobile Apex push calls: 0 out of 10
08:49:31.26<PHONE_NUMBER>)|LIMIT_USAGE_FOR_NS|dlrs|
Number of SOQL queries: 7 out of 100
Number of query rows: 54339 out of 50000 ******* CLOSE TO LIMIT
Number of SOSL queries: 0 out of 20
Number of DML statements: 1 out of 150
Number of DML rows: 1170 out of 10000
Maximum CPU time: 0 out of 10000
Maximum heap size: 0 out of 6000000
Number of callouts: 0 out of 100
Number of Email Invocations: 0 out of 10
Number of future calls: 0 out of 50
Number of queueable jobs added to the queue: 0 out of 50
Number of Mobile Apex push calls: 0 out of 10

So you appear to be using rollups configured as Calculation Mode = Realtime, thus the batch settings will have no benefit at all, these only effect schedule mode (see wiki article on schedule modes). This limit is a platform limit and defined in the known issues section of the README. It looks like it has only done 7 queries before hitting the issue, so its likely just the rollup code (please check if there are other triggers on the child or parent objects though). To confirm this can you look for "SOQL" references in the debug log and share those?

sure! here it is. appreciate your help. calculation mode doesn't need to happen in real time per se (can happen once/day), so if that would solve the problem it seems like an easy fix
Number of SOQL queries: 0 out of 100
08:49:22.8 (206787559)|SOQL_EXECUTE_BEGIN|[160]|Aggregations:0|SELECT Name, dlrs__Active__c, dlrs__AggregateAllRows__c, dlrs__AggregateOperation__c, dlrs__AggregateResultField__c, dlrs__CalculationMode__c, dlrs__CalculationSharingMode__c, dlrs__ChildObject__c, dlrs__ConcatenateDelimiter__c, dlrs__FieldToAggregate__c, dlrs__FieldToOrderBy__c, dlrs__ParentObject__c, dlrs__RelationshipCriteriaFields__c, dlrs__RelationshipCriteria__c, dlrs__RelationshipField__c, dlrs__RowLimit__c, dlrs__TestCodeSeeAllData__c, dlrs__TestCode__c, dlrs__UniqueName__c FROM dlrs__LookupRollupSummary__c WHERE CalculationMode__c in :calculationModeNames and ChildObject__c in :childObjectNames and Active__c = true ORDER BY Name ASC NULLS FIRST , dlrs__ParentObject__c ASC NULLS FIRST , dlrs__RelationshipField__c ASC NULLS FIRST
08:49:22.8 (223830554)|SOQL_EXECUTE_END|[160]|Rows:62
08:49:22.8 (401578334)|SOQL_EXECUTE_BEGIN|[274]|Aggregations:0|SELECT DeveloperName, Id, Label, dlrs__Active__c, dlrs__AggregateAllRows__c, dlrs__AggregateOperation__c, dlrs__AggregateResultField__c, dlrs__CalculationMode__c, dlrs__CalculationSharingMode__c, dlrs__ChildObject__c, dlrs__ConcatenateDelimiter__c, dlrs__Description__c, dlrs__FieldToAggregate__c, dlrs__FieldToOrderBy__c, dlrs__ParentObject__c, dlrs__RelationshipCriteriaFields__c, dlrs__RelationshipCriteria__c, dlrs__RelationshipField__c, dlrs__RowLimit__c, dlrs__TestCodeSeeAllData__c, dlrs__TestCode__c FROM dlrs__LookupRollupSummary2__mdt WHERE CalculationMode__c in :calculationModeNames and ChildObject__c in :childObjectNames and Active__c = true ORDER BY DeveloperName ASC NULLS FIRST , dlrs__ParentObject__c ASC NULLS FIRST , dlrs__RelationshipField__c ASC NULLS FIRST
08:49:22.8 (407969529)|SOQL_EXECUTE_END|[274]|Rows:4
08:49:25.977<PHONE_NUMBER>)|SOQL_EXECUTE_BEGIN|[659]|Aggregations:0|SELECT Account__c, Catalog_ID__c FROM Catalog__c WHERE Account__c in :masterIds AND Catalog_Status__c = 'done' ORDER BY Account__c,Catalog_Sale_Date_Time__c DESC
08:49:27.195<PHONE_NUMBER>)|SOQL_EXECUTE_END|[659]|Rows:21912
08:49:29.761<PHONE_NUMBER>)|SOQL_EXECUTE_BEGIN|[659]|Aggregations:0|SELECT Account__c, Catalog_ID__c FROM Catalog__c WHERE Account__c in :masterIds AND Catalog_Status__c = 'online' ORDER BY Account__c,Catalog_Sale_Date_Time__c ASC
08:49:29.761<PHONE_NUMBER>)|SOQL_EXECUTE_END|[659]|Rows:429
08:49:29.761<PHONE_NUMBER>)|SOQL_EXECUTE_BEGIN|[659]|Aggregations:0|SELECT Account__c, Has_Promoted_Listings__c FROM Catalog__c WHERE Account__c in :masterIds AND Catalog_Status__c = 'online' ORDER BY Account__c,Catalog_Sale_Date_Time__c DESC
08:49:29.761<PHONE_NUMBER>)|SOQL_EXECUTE_END|[659]|Rows:429
08:49:29.761<PHONE_NUMBER>)|SOQL_EXECUTE_BEGIN|[659]|Aggregations:0|SELECT Account__c, Avg(Lots_Sell_Thru__c) lre0, Sum(Bids_Total__c) lre1, Sum(Catalog_Pageviews__c) lre2, Sum(GMS_Disputed_Total__c) lre3, Sum(Total_LA_Revenue__c) lre4, Sum(Lots_Count_Total__c) lre5, Sum(NMS_Total__c) lre6 FROM Catalog__c WHERE Account__c in :masterIds AND Catalog_Sale_Date_Time__c = LAST_YEAR and Catalog_Status__c = 'done' GROUP BY Account__c
08:49:29.761<PHONE_NUMBER>)|SOQL_EXECUTE_END|[659]|Rows:44
08:49:29.761<PHONE_NUMBER>)|SOQL_EXECUTE_BEGIN|[659]|Aggregations:0|SELECT Account__c, Avg(Lots_Sell_Thru__c) lre0, Max(Total_LA_Revenue__c) lre1, Min(Total_LA_Revenue__c) lre2, Count(Catalog_Sale_Date_Time__c) lre3, Count_Distinct(Catalog_Month__c) lre4, Sum(Bids_Total__c) lre5, Sum(Catalog_Pageviews__c) lre6, Sum(GMS_Disputed_Total__c) lre7, Sum(Total_LA_Revenue__c) lre8, Sum(Lots_Count_Total__c) lre9, Sum(NMS_Total__c) lre10 FROM Catalog__c WHERE Account__c in :masterIds AND Catalog_Sale_Date_Time__c = THIS_YEAR and Catalog_Status__c = 'done' GROUP BY Account__c
08:49:29.761<PHONE_NUMBER>)|SOQL_EXECUTE_END|[659]|Rows:45
08:49:30.761<PHONE_NUMBER>)|SOQL_EXECUTE_BEGIN|[659]|Aggregations:0|SELECT Account__c, Min(Catalog_Sale_Date_Time__c) lre0, Max(Catalog_Sale_Date_Time__c) lre1 FROM Catalog__c WHERE Account__c in :masterIds AND Catalog_Status__c = 'done' GROUP BY Account__c
08:49:30.761<PHONE_NUMBER>)|SOQL_EXECUTE_END|[659]|Rows:45
Number of SOQL queries: 0 out of 100
Number of SOQL queries: 7 out of 100

So yes it looks like your Catalog__c object has a lot of records in it, hence the 21k records queried above. Its not clear how it got past 50k though? I see each query completes with rows counted below. Was there another query that it eventually failed on? Anyway its well worth reading this and the bullet on this platform restriction https://github.com/afawcett/declarative-lookup-rollup-summaries#usage-information-and-known-issues. And yes moving to scheduled will help you query less child records by constraining the number of master/parent records in scope at any given time. Please also check out the scheduled mode wiki page as well! :-)

thanks for your help! will check those resources out.

hi! me again...
so this issue resolved once i moved all our DLRS to scheduled mode, but it's reappeared again, suddenly. we're getting this error for many rows in an automated insert process we run for a custom object called "Catalogs"
"CANNOT_INSERT_UPDATE_ACTIVATE_ENTITY:dlrs_CatalogTrigger: System.LimitException: dlrs:Too many query rows: 50001:--"
if we're already using schedule mode, is there something else we can do to resolve this error?
thank you!

@jmiz did you make sure to change the calculation mode on the rollup record? Also if you enable debug log you can search for "SOQL" and see what queries are being made that sometimes helps.


The value before on_load event is displayed for a moment

rx.text shows "A" briefly before showing "C"
Is it possible to make rx.text stop displaying "A" briefly and display "C" immediately?
To Reproduce
import reflex as rx

class TestState(rx.State):
    test_temp: str = "A"
    def test_init(self):
        self.test_temp = "C"

def test():
    return rx.vstack(
        rx.text(TestState.test_temp),
    )

app = rx.App(state=TestState)
app.add_page(test, route="/test", on_load= TestState.test_init)
app.compile()

Specifics (please complete the following information):

Python Version: 3.11.4
Reflex Version: 0.2.6
OS: Windows 11
Browser (Optional): Chrome

    return rx.vstack(
        rx.cond(TestState.is_hydrated, rx.text(TestState.test_temp)),
    )

if you avoid showing the value until is_hydrated gets set, then it shouldn't flicker.

Thank you! masenf


No bulbs seen

Hi there,
I tried using the test code in the readme but it returns 0 bulbs even though I have one LIFX bulb up and running and pingable from the prompt. I confirmed that I can control it from the app as well.
This is the code I used ( I printed bulbs to make sure it really is empty in this case):
import lazylights
import time
bulbs = lazylights.find_bulbs(expected_bulbs=1)
lazylights.set_power(bulbs, True)
time.sleep(1)
lazylights.set_power(bulbs, False)
print bulbs

running it on my windows dos prompt:
>python fraktest.py
set([])

What am I missing?
I also tried "python test_lazylights.py" but nothing happened there either.
Thank you
-I am using Python 2.7.5 on windows 7 64bit with a LIFX bulb running latest 2.0 firmware
-I installed lazylights2.0 by downloading the zip file and running "python setup.py install" . The github way didn't work for some reason and gave me a " error: invalid command 'egg_info'" error.

Hey, thanks for filing an issue! I'll see what I can do to help.
Everything in your code snippet looks good to me.
One thing you could try, that's hopefully quick, is to call find_bulbs with the timeout parameter. The default timeout is 1 second; I've noticed with mine that once in a while, for whatever reason, I don't always get a response within that time. Can you try something like the following?
import lazylights
print lazylights.find_bulbs(expected_bubs=1, timeout=60)

If it still doesn't work with the exaggerated timeout, we can dig in a little deeper.
I will say I don't currently have access to a Windows machine, and I've done no real testing on Windows. It might be the case that the some of the socket options in use don't work quite the same way in Windows -- that's something I could look into.

I also tried "python test_lazylights.py" but nothing happened there either.

As an aside on this, the test suite is meant to be run through Nose -- so you'd pip install nose and then run nosetests, rather than running test_lazylights.py directly.

I installed lazylights2.0 by downloading the zip file and running "python setup.py install"

Just to be super extra sure, you got the zip file from https://github.com/mpapi/lazylights/archive/2.0.zip (and not master.zip), right?

The github way didn't work for some reason and gave me a " error: invalid command 'egg_info'" error.

What do you get when you run pip --version? I may not have set it up to work with older (or much newer) versions than what I have on my machine.

Hi Matt,
Thank you for replying and offering to help.
1- I tried the extended timeout and there was no change. I  made sure I could ping and control the pulb first via my iphone app and http API. See output:
C:\Python27>python
Python 2.7.5 (default, May 15 2013, 22:43:36) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import lazylights
>>> print lazylights.find_bulbs(expected_bulbs=1, timeout=60)
set([])

2- I had already installed nosetests since the tesT_lazylights wouldn't work without it. I ran nosetests and it just sat there with a "EE" in the output until I cancelled it with a Control-C. It looks like it was running some other python scripts that I wrote in the past. I have no idea what it is supposed to be used for. I would have thought that a test python script would be the thing to run when attempting a test. Maybe I am misunderstanding something. See output below:
C:\Python27>nosetests
EE
======================================================================
ERROR: docsis_signal_testv23.run_full_test
----------------------------------------------------------------------
Traceback (most recent call last):
  File "c:\Python27\lib\site-packages\nose\case.py", line 197, in runTest
    self.test(*self.arg)
  File "C:\Python27\docsis_signal_testv23.py", line 448, in run_full_test
    load_settop_objects()
  File "C:\Python27\docsis_signal_testv23.py", line 286, in load_settop_objects
    sys.exit()
SystemExit:
-------------------- >> begin captured stdout << ---------------------
<dst_started nosetests 2.3 Thu Jun 04 13:18:50 2015>
<dst_cannot_open_ip_file docsis_signal_list_of_ip_or_mac.txt>
<dst_ip_file_error_end_of_test at Thu Jun 04 13:18:50 2015 elapsed 0 seconds>

--------------------- >> end captured stdout << ----------------------

======================================================================
ERROR: Failure: SyntaxError (EOL while scanning string literal (testip.py, line 22))
----------------------------------------------------------------------
Traceback (most recent call last):
  File "c:\Python27\lib\site-packages\nose\loader.py", line 418, in loadTestsFromName
    addr.filename, addr.module)
  File "c:\Python27\lib\site-packages\nose\importer.py", line 47, in importFromPath
    return self.importFromDir(dir_path, fqname)
  File "c:\Python27\lib\site-packages\nose\importer.py", line 94, in importFromDir
    mod = load_module(part_fqname, fh, filename, desc)
  File "C:\Python27\testip.py", line 22
    os.system("cmd2000.exe " + HOST + " osdiag SilentDiag" ")
                                                            ^
SyntaxError: EOL while scanning string literal

----------------------------------------------------------------------
Ran 2 tests in 15.627s

FAILED (errors=2)

3- I didn't use the link you provided. I did the only thing I thought I could do and that was go to the repository at: https://github.com/mpapi/lazylights and then click on "Download zip" and used that to do the setup from. How else would I do it? Where would I get your link from?
4-
C:\Python27>pip --version
pip 6.0.2 from c:\Python27\lib\site-packages (python 2.7)

Are you sure your code supports the new v2.0 firmware? In your credits, you reference the "lifxjs Protocol wiki page " project and I see that it is based on the v1.x firmware. Could this be the issue?

Okay, I think I understand what might be happening.
The support for the 2.0 firmware is only on the 2.0 branch, and it sounds like you're actually running code from the master branch. To get 2.0, you'd either have to use the pip install command in the readme (which you said didn't work for you, possibly because I messed up the setup.py), or select the 2.0 branch from the branch selection dropdown (to get to this page) before you hit "Download ZIP" -- that's how I got the link to 2.0.zip that I referenced above. So, try using that link, and see if that works for you.
The master branch still only has 1.x support, mostly because I hadn't done much testing of the 2.0 code. But, yes, the 2.0 branch supports the 2.0 protocol, which I ended up reverse engineering using tcpdump the week the new firmware came out (so it's not in fact using what was on that wiki page anymore). I've been running that code with my 2.0 bulbs since the update, on an application that I use daily, without any major issues.
Now that 2.0 has been out for a while and I've done quite a bit of real-world testing, I can merge the 2.0 branch into master the next time I have a chance, to make things clearer going forward. The repository is definitely in need of some love. :)
Thanks again for these reports, by the way -- it's super useful for me to see what's confusing to users, and I'm always glad to see that other people are interested in the project!

Thanks for clearing that up. I didn't even know about the other branch.
So I tried downloading the new zip and running the test script also failed. Thinking it had something to do with having installed the original/master scripts, I deleted all references to lazylights on my computer and started again. This time, I upgraded pip and tried the github method. That didn't work either unfortunately. See output below:
C:\Python27>pip install git+https://github.com/mpapi/lazylights
Collecting git+https://github.com/mpapi/lazylights
  Cloning https://github.com/mpapi/lazylights to c:\temp\pip-8aged5-build
    c:\Python27\lib\distutils\dist.py:267: UserWarning: Unknown distribution option: 'install_requires'
      warnings.warn(msg)
    usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
       or: -c --help [cmd1 cmd2 ...]
       or: -c --help-commands
       or: -c cmd --help
    error: invalid command 'egg_info'
    Complete output from command python setup.py egg_info:
    c:\Python27\lib\distutils\dist.py:267: UserWarning: Unknown distribution option: 'install_requires'

      warnings.warn(msg)

    usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]

       or: -c --help [cmd1 cmd2 ...]

       or: -c --help-commands

       or: -c cmd --help

    error: invalid command 'egg_info'

    ----------------------------------------
    Command "python setup.py egg_info" failed with error code 1 in c:\temp\pip-8aged5-build

C:\Python27>pip --version
pip 6.0.2 from c:\Python27\lib\site-packages (python 2.7)

C:\Python27>python get-pip.py
c:\temp\tmp6ilko2\pip.zip\pip\_vendor\requests\packages\urllib3\util\ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
c:\temp\tmp6ilko2\pip.zip\pip\_vendor\requests\packages\urllib3\util\ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
Collecting pip
  Downloading pip-7.0.3-py2.py3-none-any.whl (1.1MB)
    100% |################################| 1.1MB 401kB/s
Collecting wheel
  Downloading wheel-0.24.0-py2.py3-none-any.whl (63kB)
    100% |################################| 65kB 2.7MB/s
Installing collected packages: pip, wheel
  Found existing installation: pip 6.0.2
    Uninstalling pip-6.0.2:
      Successfully uninstalled pip-6.0.2
Successfully installed pip-7.0.3 wheel-0.24.0

C:\Python27>pip --version
pip 7.0.3 from C:\Python27\lib\site-packages (python 2.7)

C:\Python27>pip install git+https://github.com/mpapi/lazylights
Collecting git+https://github.com/mpapi/lazylights
  Cloning https://github.com/mpapi/lazylights to c:\temp\pip-scwem8-build
    Complete output from command python setup.py egg_info:
    C:\Python27\lib\distutils\dist.py:267: UserWarning: Unknown distribution option: 'install_requires'
      warnings.warn(msg)
    usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
       or: -c --help [cmd1 cmd2 ...]
       or: -c --help-commands
       or: -c cmd --help

    error: invalid command 'egg_info'

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in c:\temp\pip-scwem8-build

So then I just manually downloaded the 2.0 branch's zip file and tested with that instead. I ran "python setup.py install" and ran the test script with the longer timeout and still got the empty set...
C:\Python27\lazylights-2.0 (1)\lazylights-2.0>python setup.py install
c:\python27\lib\distutils\dist.py:267: UserWarning: Unknown distribution option: 'install_requires'
  warnings.warn(msg)
running install
running build
running build_py
creating build
creating build\lib
copying lazylights.py -> build\lib
running install_lib
copying build\lib\lazylights.py -> c:\python27\Lib\site-packages
byte-compiling c:\python27\Lib\site-packages\lazylights.py to lazylights.pyc
running install_egg_info
Writing c:\python27\Lib\site-packages\lazylights-2.0.0-py2.7.egg-info

C:\Python27\lazylights-2.0 (1)\lazylights-2.0>python fraktest.py
set([])

As an aside, I created a simple python script that accesses my LIFX bulbs using curl through the LIFX cloud network using a user-unique token ID. It matches the colour of the LIFX bulb to that of your computer screen. If you are interested, you can check it out on: https://github.com/frakman1/lifxscreen
The official API is here: http://developer.lifx.com/

If I know the IP and MAC addresses of the bulbs, can't I leverage that to make it work better? I know I can ping the bulbs so maybe the code can be improved to allow that to be specified then it wouldn't have to try (and in my case fail) to find the bulb(s).

@frakman1 Try changing the broadcast address in lazylights.py to your personal network broadcast address. If your network is <IP_ADDRESS> with netmask <IP_ADDRESS> then set it to <IP_ADDRESS>. I tested this on my Raspberry Pi (with Python 2.7) and it works. Also tested on Windows (with Python 3.4 (via 2to3 script) but then i get socket errors.
btw, sending state (or power) commands are not reliable with this script. You have to check if the given command were really executed on the bulb by checking the get_state against the executed command.
After playing around a bit i created a function that does just that: (only power on/off)
def setBulbPower(bulbs, power):
	for bulb in bulbs:
		lazylights.set_power([bulb],power)
		x = 0
		while True:
			print('Retrieving bulbstate from ' + bulb.addr[0])
			bulbState = lazylights.get_state([bulb])
			if len(bulbState) > 0:
				if bulbState[0].power==0 and power == False:
					print(bulbState[0].label + ' is powered OFF!')
					break
				elif bulbState[0].power>0 and power == True:
					print(bulbState[0].label + ' is powered ON!')
					break;
				else:
					if x==0:
						print("Sending command....")
					else:
						print("Re-sending command....")
					lazylights.set_power([bulb],power)
					if x>0:
						time.sleep(1)
			else:
				print('The bulb did not respond yet...')
			x = x+1
	return

I've read somewhere that Lifx Firmware 2 also works with TCP instead of UDP. packets for controlling the bulbs. that would result in a reliable connection...

okay, tcp is only for cloud, which i don't want. So we stick with the UDP protocol.
Just saw a python 3 script forked from Lazylights2.0. (by georgwaechter). Tested on windows and has the same, non reliable, functionality.
Played a whole day with this script figuring out why powering off (+Status check) a bulb is taking so long.
Powering ON 2 bulbs takes 0.02 seconds. (almost every time)
Powering OFF 2 bulbs takes 0.3 seconds and can take up to 3 seconds!
After executing the set_power function. The get_state still receives a powered ON bulb for a while...
Is this a bug in the lazylights script or in the Lifx bulb firmware? Can we send a transition time with a power command?
@frakman1
You can easily create your own bulbs if you know the mac, IP address and port number. If i'm not mistaken, portnumber is always 56700. Have a look at my testscript, more specifically the function createBulb()
my testscript:
"""
Created by VinzzB on 2015/08/23
-------------------------------
Argument:
1 = power all bulbs
0 = set all bulbs off
-------------------------------
"""
import lazylights
import time
import struct
import sys, string
import binascii

if sys.version_info[0] > 2:
    PY3K = True
else:
    PY3K = False

def setBulbPower(bulbs, power, timeout=5):
	try:
		starttime = time.time()
		for x in range(timeout): #iterate for timeout seconds max. Stop executing if not all bulbs were found within this time
			print("Sending power command to " + str(len(bulbs)) + " bulbs")
			lazylights.set_power(bulbs, power)
			print("Retrieving State from " + str(len(bulbs)) + " bulbs")
			for y in range(20): # ~1sec	max
				bulbState = lazylights.get_state(bulbs) #Get state from all bulbs gvien in the args.
				if len(bulbState) == len(bulbs): #Did we find all the bulbs? If not, check again...
					if powerState(bulbState) == power: #What prowerState do we have? Did the command exceute on all bulbs?
						print(getLabels(bulbState) +  ' bulbs powered '+ ("ON" if power else "OFF")  + '!')
						return #exit function
				else:
					print('Not all bulbs responded yet... Found {0} bulb(s) - Retry {1}'.format(len(bulbState), str(y+x+1)))
				time.sleep(0.05)
		print("The command timed-out. (" + (time.time() - starttime) + " secondes)" )
	finally:
		print("setBulbPower() was executed in %1.2f secondes" % (time.time() - starttime))
	return

#Check if all bulbs are powerd on or off
def powerState(bulbs):
	power = False #Off by default.
	for bulb in bulbs:
		power |= bulb.power>0 #once True, always True!
		print("bulb " + getLabels([bulb]) + " is " + ("ON" if bulb.power>0 else "OFF"))
	return power

def getLabels(bulbs):
	r =""
	for bulb in bulbs:
		if PY3K:
			r += ("" if r=="" else ", ") + str(bulb.label,'ASCII').split('\x00')[0]
		else:
			r += ("" if r=="" else ", ") + str(bulb.label).split('\x00')[0]
	return r

def createBulb(ip, port, macString):
	return lazylights.Bulb(b'LIFXV2', binascii.unhexlify(macString.replace(':', '')), (ip,port))

if __name__ == "__main__":
	#Create our own Bulbs
	myBulb1 = createBulb('<IP_ADDRESS>',56700,'00:00:00:00:00:00') #FILL IN
	myBulb2 = createBulb('<IP_ADDRESS>',56700,'00:00:00:00:00:00')  #FILL IN
	print('MyBulb1: ' + str(myBulb1))
	print('MyBulb2: ' + str(myBulb2))
	#append to bulbs list
	bulbs = [myBulb1, myBulb2]

	#Or discover bulbs
	# bulbs = lazylights.find_bulbs(2,0.075,5)
	#print('done searching. Found ' + str(len(bulbs)) + ' bulbs')
	#for bulb in bulbs: print(bulb)

	setBulbPower(bulbs,sys.argv[1] == '1')

Thank you for the insight and sample code. I will try them out and let you know.
Two quick comments:
1- I ran this script on my jail broken iPhone's command line/Python prompt and althoug it works sometimes, it was unreliable. I've had varying degrees of success with windows laptops (Win7 and 10)
2- Regarding powering on and off. I doubt it I the firmware because I wrote an iOS program that uses their/LIFX SDK and it works perfectly and is very responsive. I imagine they use the same functions at the lower level.
3- Did you know that they have release the documentation for their bulbs? Full IP packet protocol so no need to reverse engineer everything.

If you create your bulbs yourself then this script is actually pretty reliable (even without checking the state). But discovering the bulbs isn't working perfectly. That's why i was also looking for a way to create the bulbs manually (and to store / re-create the bulb list)
I saw the API documentation yesterday. Seems that the power command also has a transition time. But it is not implemented in lazylights. This would fix my problem. I have to dig deeper in the lazylights code to add this feature.
jep, i know / i saw / i've rapidly read it :-)

What i'm looking for now is a way to see changes in the lightbulbs. If i dim the lights with my phone, my script should see these changes almost immediately. The Lifx app handles that quite well. Just have to figure out how they do it (Do they use polling or just listen to broadcasts/multicasts messages)

@VinzzB and @frakman1: thanks for the updates!
Can you both recap/summarize which specific things are working for you reliably at this point and which aren't? Is it just bulb discovery, or other things as well?
The best next step seems to be updating lazylights according to what's in the protocol docs, and I'd love to make sure I understand which things are not reliable to figure out what to focus on with the limited time I have. Trying to reproduce these kinds of issues in my local environment (where everything's working great) tends to be a lot of work for little benefit.

@VinzzB: for power transition, one thing you could do today is set the bulb state (not power) to change brightness to 0 with a transition/fade time, and then set power to off when that's done. That's pretty much what I do now without support for a transition time for power.
Seeing changes is a little harder: I believe the app listens to the broadcast messages, and that's what the _recv function does as well. I've used _recv directly to listen to state changes or other message types as well -- I didn't make that first class in the library, though, because in many cases you'd need threading or greenlets or some other form of concurrency for it to be useful, and those come with a bunch of complexity and tradeoffs for different use cases. I'm not opposed to adding it eventually, though.

Also, please feel free to add some comments on #4 with anything else you find in the protocol docs that you'd like to see (transition time for powering on/off being a great example).

@mpapi,
I figured out that the your discovery implementation isn't working well. Once you manually add the bulbs (like i did in my testscript) your script actually is reliable. Although UDP isn't reliable by nature, i did not noticed any flaws yet. Luckily i have a stable home network. :-)
I just tried an other python script form smarthall. (https://github.com/smarthall/python-lifx-sdk) He actually has a good discovery implementation and his script also listens to incoming messages for bulb changes. (+ it has transition time argument in power method)
What i'm missing there is a method to re-create the bulbs from a db store. as i did in my testscript.
Still have to figure out what script i'm going to use. Both has there pro / cons...
About the power transition:
I tried setting the brightness to 0 first. But the power state remains on. And when i power it down after setting brightness to zero, the power state is still going down in steps.

@VinzzB
Thank you for your test script. I tried it with two manually defined bulbs and although the power turned on and off correctly, it never heard back from the bulbs and was stuck in a neverending loop of these messages:
Not all bulbs responded yet... Found 0 bulb(s) - Retry 1
By the way, I've used smarthall's library too and was able to get it to work.


Mempool Management

In the MempoolBehavior class what is your strategy of sending out the GetDataPayload to the node?
https://github.com/stratisproject/StratisBitcoinFullNode/blob/fe218bb1df7a041562d3b53c7d8fde1668912272/src/Stratis.Bitcoin.Features.MemoryPool/MempoolBehavior.cs#L310-L331
As I see you guys reply with GetDataPayload to every INV, except if you're in blocksonly mode or you already have the tx in orphans, which I'm not sure what that is.
Now, if you already proceed the transaction, because another node already sent it to you (every node has its own MempoolBehavior instance) then you ask for that transaction again, right?
Pinging @molnard.

Orphans are trasnactions that we don't know the input for, if a trx is spending an unconfirmed input and for some reason the child arrived before the parent it will go in to orphans, when the parent arrives it will then from orphans in to the mempool.
If you mean relay then yeah we relay all trx we received and validated to all peers, this should be the same logic applied in core

I created a PR, I hope that explains what I think I've found.

Actually the orphans do the job, they aren't just orphans: https://github.com/stratisproject/StratisBitcoinFullNode/pull/3958#discussion_r326110458


backupccl: Backup Table Statistics

PR: https://github.com/cockroachdb/cockroach/pull/39644
From release notes:

Table statistics for the tables that are being backed up are now also backed up. [#39644][#39644] {% comment %}doc{% endcomment %}

No docs needed


Update base jest config to handle nested package.json files

This PR updates the base-jest-config to work in a monorepo where there may be more than one package.json file. It first attempts to read the package file in the cwd and falls back to the package file in the git root directory.
This works for the following setup:
my-service/package.json has a test script that looks like:
fusion test --configPath ../../node_modules/fusion-cli/build/jest/jest-config.js
And the monorepo structure is as follows:
monorepo
├── package.json
└── services
    └── my-service
        └── package.json

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


implement msgSend example

implement example snippet code
The goal is to enhance the first-time developer experience with InitiaJS.

on second thought, maybe no need to create an example folder
only need to update the readme.md to the latest condition
https://github.com/initia-labs/initia.js/pull/64

closing this as you mentioned https://github.com/initia-labs/initia.js/pull/64


Error 400 bad request

Pulled latest with the following error. Generated a new UUID with no change.  there is more error logs upon request
info: Requesting vehicles
error: Main function error. {"error":{"message":"Request Failed with status 400 - Bad Request","request":{"_closed":true,"_contentLength":"457","_defaultKeepAlive":true,"_ended":true,"_events":{},"_eventsCount":7,"_hasBody":true,"_header":"POST /api/v1/oauth/token HTTP/1.1\r\nAccept: application/json\r\nContent-Type: text/plain\r\nAccept-Language: en\r\nHost: api.gm.com\r\nConnection: keep-alive\r\nAccept-Encoding: br, gzip, deflate\r\nUser-Agent: myChevrolet/118 CFNetwork/1408.0.4 Darwin/22.5.0\r\nContent-Length: 457\r\n\r\n","_headerSent":true,"_keepAliveTimeout":0,"_last":true,"_redirectable":{"_currentRequest":"[Circular]","_currentUrl":"https://api.gm.com/api/v1/oauth/token","_ended":true,"_ending":true,"_events":{},"_eventsCount":3,"_options":{"agents":{},"beforeRedirects":{},"headers":{"Accept":"application/json","Accept-Encoding":"br, gzip, deflate","Accept-Language":"en","Connection":"keep-alive","Content-Length":"457","Content-Type":"text/plain","Host":"api.gm.com","User-Agent":"myChevrolet/118 CFNetwork/1408.0.4 Darwin/22.5.0"},"hostname":"api.gm.com","maxBodyLength":null,"maxRedirects":21,"method":"POST","nativeProtocols":{"http:":{"ME

It looks like I'm getting the same error after pulling and starting. After inspecting the large json object that it returns, the only clue I can find is this section in the response portion:
"data":{     "error":"invalid_client" },
It looks like it's an oauth issue, but I'm no oauth expert so I'm not sure.

Pushed a version bump. I'll have to switch the docker build from ci so the weekly build will evergreen the onstarjs version.

Pulled latest and getting the following error again.
error: Main function error. {"error":{"message":"No response","request":{"_currentRequest":{"_closed":false,"_contentLength":"457","_defaultKeepAlive":true,"_ended":false,"_events":{},"_eventsCount":7,"_hasBody":true,"_header":"POST /api/v1/oauth/token HTTP/1.1\r\nAccept: application/json\r\nContent-Type: text/plain\r\nAccept-Language: en\r\nHost: api.gm.com\r\nConnection: keep-alive\r\nAccept-Encoding: br, gzip, deflate\r\nUser-Agent: myChevrolet/118 CFNetwork/1408.0.4 Darwin/22.5.0\r\nContent-Length: 457\r\n\r\n","_headerSent":true,"_keepAliveTimeout":0,"_last":true,"_redirectable":"[Circular]","_removedConnection":false,"_removedContLen":false,"_removedTE":false,"_trailer":"","aborted":false,"agent":{"_events":{},"_eventsCount":2,"_sessionCache":{"list":[],"map":{}},"defaultPort":443,"freeSockets":{},"keepAlive":false,"keepAliveMsecs":1000,"maxCachedSessions":100,"maxFreeSockets":256,"maxSockets":null,"maxTotalSockets":null,"options":{"noDelay":true,"path":null},"protocol":"https:","requests":{},"scheduling":"lifo","sockets":{"api.gm.com:443:::::::::::::::::::::":[{"_SNICallback":null,"_closeAfterHandlingError":false,"_controlReleased":true,"_events":{"close":[null,null,null,null,null],"connect":[null,null,null],"end":[null,null]},"_eventsCount":11,"_hadError":true,"_host":"api.gm.com","_httpMessage":"[Circular]","_newSessionPending":false,"_parent":null,"_pendingData":[{"chunk":"POST /api/v1/oauth/token HTTP/1.1\r\nAccept: application/json\r\nContent-Type: text/plain\r\nAccept-Language: en\r\nHost: api.gm.com\r\nConnection: keep-alive\r\nAccept-Encoding: br, gzip, deflate\r\nUser-Agent: myChevrolet/118 CFNetwork/1408.0.4 Darwin/22.5.0\r\nContent-Length: 457\r\n\r\n","encoding":"latin


Clicking user only searches their name?

Sometimes for random users when clicking on their name in the player joined/left logs it switches tabs to the search and searches them. Am I missing a reason on why this happens?

Because VRChat doesn't log userId's to gameLog when clicking on a user it has to search their displayName to get their userId and because VRChats user search is hot garbage it fails a lot of the time sending you to the user search page so you can try find them manually. You can feed VRCX most userIds of players in your instance by adding this VRChat launch parameter --log-debug-levels=API.

I see, that sucks but thank you for the work around!


[UIKIT-16] Feature: Add the option to save the Figma API key as a global config

UIKIT-16
This is a WIP not yet fully functioning, currently fails when trying run script.

Trying to figure out the best way to handle both local and global .figma2themerc files, currently if there is a local one the global one is ignore which isn't great if you want to use the global one for the API key and the local one for the file URL.


Update Request - GSD-2021-1000000

Automated Edit Request
For: "GSD-2021-1000000"
{
  "GSD": {
    "vendor_name": "GPAC",
    "product_name": "GPAC",
    "product_version": "0.8.0",
    "vulnerability_type": "Denial of service(DoS)",
    "affected_component": "GPAC 0.8.0-e10d39d-master branch",
    "attack_vector": "a malicious image file",
    "impact": "DoS. An attacker can exploit this vulnerability by submitting a malicious media file that exploits this issue. This will result in a Denial of Service (DoS) when the application attempts to process the file.",
    "credit": "Rui Yang, Riddle Ben",
    "references": [
      {
        "type": "REPORT",
        "url": "https://github.com/gpac/gpac/issues/1485"
      }
    ],
    "reporter": "DuckRui",
    "notes": "Here are crash files.https://github.com/DuckRui/vul-/blob/master/isom_store163. https://github.com/DuckRui/vul-/blob/master/isom_store165. https://github.com/DuckRui/vul-/blob/master/isom_store173.",
    "description": "In GPAC GPAC version 0.8.0 a Denial of service(DoS) exists in the GPAC 0.8.0-e10d39d-master branch that can be attacked via a malicious image file resulting in DoS. An attacker can exploit this vulnerability by submitting a malicious media file that exploits this issue. This will result in a Denial of Service (DoS) when the application attempts to process the file.."
  },
  "OSV": {
    "id": "GSD-2021-1000000",
    "summary": "Denial of service(DoS) in GPAC version 0.8.0",
    "details": "In GPAC GPAC version 0.8.0 a Denial of service(DoS) exists in the GPAC 0.8.0-e10d39d-master branch that can be attacked via a malicious image file resulting in DoS. An attacker can exploit this vulnerability by submitting a malicious media file that exploits this issue. This will result in a Denial of Service (DoS) when the application attempts to process the file..",
    "modified": "2021-06-24T22:55:38.996515Z",
    "published": "2021-05-31T15:39:45.031586Z",
    "references": [
      {
        "type": "WEB",
        "url": "https://github.com/gpac/gpac/issues/1485"
      }
    ],
    "affected": [
      {
        "package": {
          "name": "GPAC",
          "ecosystem": "DWF"
        },
        "versions": [
          "0.8.0"
        ]
      }
    ]
  }
}

Testing editing references with new UI, works!


Delete multiple blobs

IIUC, az storage blob delete will delete just one blob.
Similar to #998, it would be nice to be able to delete a whole
"directory".  It looks like the only way to do that currently is to get
a list of all blobs using --prefix then iterate over each one of them,
which would overall be extremely slow compared to a single
delete-batch.

@tjprescott Is there any chance that this will get implemented?  We currently have a script that garbage-collects old build artifacts and since we added documentation pages the process of a single version deletion takes about 30 minutes...

Sounds like a very reasonable request, although if the service doesn't support batch deletion then anything we do would basically be the same loop you are doing behind but behind the scenes. @troydai, @mayurid for technical feasibility and priority.

Well, one thing that would be different is that if the cli is doing the deletions in one call, then the implementation could do the iteration over each blob and issue a delete, all on a single connection --- and that would be a huge saving over what can be done now since doing the loop outside the cli means about 2 seconds for each request which is probably mostly spent on establishing a connection.
In other words, it would make it as fast deleting a "directory" with the storage explorer, which is still way faster than spending two seconds over each deleted blob...

https://docs.microsoft.com/en-us/rest/api/storageservices/delete-blob
Currently, there isn't a REST API call for batch deletion. We can implement batch delete for your convenience, but we basically do the same thing as looping blob in a script.
I'm not sure what optimization the storage explorer has done. I'll look into that.

Another option is multi-threading. Then the main topic will be how to do throttling and prevent racing situation.

@troydai, I don't know what it actually does, but I think that a lot of
speedup can happen if you keep the connection alive -- and it seems to
me that this could lead to the speedup that happens when deleting a
subtree in the storage explorer.  Assuming that most of the ~2 second
per deletion via the cli is spent on establishing a connection, this
would be the boost that will make it a practical operation.

@elibarzilay agrees. Thanks for the advice. I'll see what we can do to improve the performance.

Thanks!

Is this going to be implemented? I have a pyspark job generating tons of files in blob storage, but since sc.saveAsTextFile does not allow overwriting, I'm stuck having to delete the files (or leave them and generate new directories all the time, or deleting the entire container). This is pretty inconvenient.
I would actually love a feature where you don't need to list files at all, but rather specify a folder, prefix or maybe a regex pattern of files to be deleted.

@phdowling thank you for "up voting" this issue :). Looks like popular concern. I'll add this issue to the next sprint. We generally ship every two to three weeks.

@williexu -- Thanks!!  What's the time frame for this feature to be part of the available version?

@elibarzilay  you're welcome, sorry it took so long.
Here's a link to our schedule: https://github.com/Azure/azure-cli/milestones
Our next version release will be on 11/15

@williexu -- thanks, I'll follow it (it's very relevant for me, since I do have these long deletion sequences that are very slow).


RAP-4149 Integration tests - Reflex for Plugin API

For PluginApiImpl, there is this statement in the for loop:
System.out.println(cfg.toString());
Shouldn't it be log.info(cfg.toString()); ?

That shouldn't have been checked in. I've removed it. Thanks.


Using slf4j-api instead of slf4j-simple

slf4j-simple is an implementation. slf4j-api should be used instead, otherwise this will conflict, if you use this as a library in other projects, i.e. our OSGi-less trusted connector (which uses logback by default). We should not impose a choice of logging framework here.
Please re-publish the artefact after merging this.

Any objections against merging this @milux @leonbeckmann ?

Just released as 0.4.1 on maven central


testAvgPoolSamePadding fails on ppc64le. Attempting to root cause, have some info need some help

Please go to Stack Overflow for help and support.
http://stackoverflow.com/questions/tagged/tensorflow
If you open a GitHub issue, here is our policy:

It must be a bug or a feature request.
The form below must be filled out.

Here's why we have that policy: TensorFlow developers respond to issues.
We want to focus on work that benefits the whole community, e.g., fixing
bugs and adding features. Support only helps individuals. GitHub also notifies
thousands of people when issues are filed. We want them to see you communicating
an interesting problem, rather than being redirected to Stack Overflow.

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow):
No
OS Platform and Distribution (e.g., Linux Ubuntu 16.04):
Ubuntu 16.04 ppc64le
TensorFlow installed from (source or binary):
From source
TensorFlow version (use command below):
'v1.0.1-0-ge895d5c-dirty'
Bazel version (if compiling from source):
Build label: 0.4.4-2017-04-13 (@80a07b5)
CUDA/cuDNN version:
No GPU
GPU model and memory:
No GPU
Exact command to reproduce:
bazel test //tensorflow/compiler/tests:pooling_ops_test_cpu
You can collect some of this information using our environment capture script:
https://github.com/tensorflow/tensorflow/tree/master/tools/tf_env_collect.sh

You can collect the TensorFlow version with
python -c "import tensorflow as tf; print (tf.GIT_VERSION, tf.VERSION)"

Describe the problem
pooling_ops_test_cpu fails on ppc64le, the test failing is testAvgPoolSamePadding
I am making an attempt to root cause the issue, need some help with understanding the code flow (i am new to tensorflow code but interested/would-like-to debug & help :smile: )
First i tried to understand the input parameters being passed to the avg_pool function (this is after understanding what avg_pool does)
The input tensor for this test case is
[[[1 2 3],[4 5 6],[7 8 9],[10 11 12],[13 14 15],[16 17 18]]]   (4-d tensor)
The test expects avg_pool o/p as [  7. ,   8. ,   9. ,  11.5,  12.5,  13.5] but gets [  7. ,   8. ,   9. ,  11.5,   9. ,  13.5] (4th entry is different)
Then i tried to trace the code to see why the entry differs (only for ppc64le, x86 the test passes)
Test case calls avg_pool in python/ops/nn_ops.py, which ultimately drops into core/kernels/avgpooling_op.cc. The AvgPoolingOp class "compute" func calls SpatialAvgPool which i tried to find/grep, could find it in core/kernels/eigen_pooling.h.
This is where i am getting lost. I am suspecting an int overflow or sign type issue and my goal is to pinpoint the exact location where this could be happening, however seeing the code in avgpooling_op.cc (or eigen_pooling.h) i cant seem to find any good places where i can debug further
Any help or pointers where i can go from here (or if i am following the right trail) would help
Thanks!
Vaibhav
Source code / logs
Include any logs or source code that would be helpful to diagnose the problem.
If including tracebacks, please include the full traceback. Large logs and files
should be attached. Try to provide a reproducible test case that is the minimum
necessary to generate the problem.
Failure assertion snippet
exec ${PAGER:-/usr/bin/less} "$0" || exit 1
I tensorflow/compiler/xla/service/platform_util.cc:58] platform Host present with 16 visible devices
I tensorflow/compiler/xla/service/service.cc:180] XLA service executing computations on platform Host. Devices:
I tensorflow/compiler/xla/service/service.cc:187]   StreamExecutor device (0): ,
F.
FAIL: testAvgPoolSamePadding (main.PoolingTest)
File "/root/.cache/bazel/_bazel_root/68a62076e91007a7908bc42a32e4cff9/execroot/tensorflow/bazel-out/local-opt/bin/tensorflow/compiler/tests/pooling_ops_test_cpu.runfiles/org_tensorflow/tensorflow/compiler/tests/pooling_ops_test.py", line 288, in testAvgPoolSamePadding
expected=expected_output)
File "/root/.cache/bazel/_bazel_root/68a62076e91007a7908bc42a32e4cff9/execroot/tensorflow/bazel-out/local-opt/bin/tensorflow/compiler/tests/pooling_ops_test_cpu.runfiles/org_tensorflow/tensorflow/compiler/tests/pooling_ops_test.py", line 125, in _VerifyValues
data_format, expected)
File "/root/.cache/bazel/_bazel_root/68a62076e91007a7908bc42a32e4cff9/execroot/tensorflow/bazel-out/local-opt/bin/tensorflow/compiler/tests/pooling_ops_test_cpu.runfiles/org_tensorflow/tensorflow/compiler/tests/pooling_ops_test.py", line 108, in _VerifyOneTest
self.assertAllClose(expected, actual.flatten(), rtol=1e-5, atol=1e-6)
File "/root/.cache/bazel/_bazel_root/68a62076e91007a7908bc42a32e4cff9/execroot/tensorflow/bazel-out/local-opt/bin/tensorflow/compiler/tests/pooling_ops_test_cpu.runfiles/org_tensorflow/tensorflow/python/framework/test_util.py", line 485, in assertAllClose
np.testing.assert_allclose(a, b, rtol=rtol, atol=atol)
File "/usr/lib64/python2.7/site-packages/numpy/testing/utils.py", line 1411, in assert_allclose
verbose=verbose, header=header, equal_nan=equal_nan)
File "/usr/lib64/python2.7/site-packages/numpy/testing/utils.py", line 796, in assert_array_compare
raise AssertionError(msg)
AssertionError:
Not equal to tolerance rtol=1e-05, atol=1e-06
(mismatch 16.6666666667%)
x: array([  7. ,   8. ,   9. ,  11.5,  12.5,  13.5])
y: array([  7. ,   8. ,   9. ,  11.5,   9. ,  13.5], dtype=float32)

Ran 2 tests in 0.514s

The definition is here: https://github.com/tensorflow/tensorflow/blob/2098b9abcf20d2c9694055bbfd6997bc00b73578/tensorflow/core/kernels/pooling_ops_common.h#L258

Thanks!
With a little more investigation found that this fails on ppc64le only for the data_format "NCHW", for the other data format ("NHWC") it passes (data_formats as passed as input to tf.nn.avg_pool in test case file ./compiler/tests/pooling_ops_test.py). Investigating this further, if could get any suggestions/pointers to why this may be happening that would be great!

I would look at the numbers coming in and out of the operators to see where the difference is.

Closing due to lack of recent activity. Please reopen if this is still an issue.


Add signal::connect_raw() working on raw c_char pointers for the sign…

…al name
This can safe one allocation compared to s.to_glib_none().0

@sdroege Thanks
👍

@GuillaumeGomez all green :)

Thanks!


添加f1c100s和m484到CI里

[ ] f1c100s
[ ] m484

https://github.com/vsfteam/vsf/pull/147

not necessary now


As a observing player, I want to see the barrier question as well, but without the option to answer it / click anything. I want to see a message saying “Wait for the player to answer the barrier question”.

Resolved with commit: player who waits gets a message


Work in progress to add line wrapping

Should close #7, it's not even close right now 😄 The line wrapping fail because there are space before and after the leading string.

[ ] Line wrapping after certain line length
[ ] make line length configurable, search for settings of other tools (black, pylint)
[ ] Break up multiple sentences on first line into summary and main block (see PEP 256)
[ ] Handle numpy/google style docstrings gracefully with line wrapping

I like how it auto-formatted our own test examples 😄

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

6 of 6 (100.0%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 100.0%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
359

Relevant Lines:
359

💛  - Coveralls

@Pierre-Sassoulas I have started working some more on this as it is a nice little job I can do in between other (personal) stuff.
This is still WIP, but in order for this to remain manageable I'd like to focus on wrapping summaries for now. In this PR we should add that basic functionality with the line length as ordered by PEP 8 (which is 72 😮).
Before releasing this we should probably also allow setting the max line length in some way.
One thing I haven't yet fixed is how to handle docstrings that haven't been split. For example:
"""My beautiful docstring
This function will do a number of things:
bla di bla
"""

Should be
"""My beautiful docstring

This function will do a number of things:
bla di bla
"""

That makes it much easier for this to line wrap. However since the \n\n is missing, this current iteration will now wrap this to:
"""My beautiful docstring This function will do a number of things: bla di bla."""

in order for this to remain manageable I'd like to focus on wrapping summaries for now

Yes, let's not do too much at once.

add that basic functionality with the line length as ordered by PEP 8 (which is 72 😮).

I think this is unreasonable and we should use 88 by default (black's default).

this current iteration will now wrap this to: """My beautiful docstring This function will do a number of things: bla di bla."""

Is it really problematic ? I mean if we encounter an end of sentence punctuation we could cut easily after it then line wrap. Maybe we should first add end of sentence punctuation if there's a \n then line wrap ? If there's no \n at all in the docstring we do not add a ., and we line wrap ?

add that basic functionality with the line length as ordered by PEP 8 (which is 72 😮).

I think this is unreasonable and we should use 88 by default (black's default).

Personally I'd like to strictly adhere to the PEP's and make it very clear how to avoid this (awful) default. Something like "we recommend running with pydocstringformatter --line-length=88" in the first sections of the README.
Adhering strictly to the PEP's avoids any future discussion about defaults etc, which is something that's likely going to come up with a strict tool like this.

this current iteration will now wrap this to: """My beautiful docstring This function will do a number of things: bla di bla."""

Is it really problematic ? I mean if we encounter an end of sentence punctuation we could cut easily after it then line wrap. Maybe we should first add end of sentence punctuation if there's a \n then line wrap ? If there's no \n at all in the docstring we do not add a ., and we line wrap ?

We do cut in SplitSummaryAndDocstringFormatter. So pydocstringformatter  --linewrap-full-docstring --split-summary-body . works best. However, that formatter is still optional as I'm not fully satisfied with how we determine what is a summary and what is a description. I think it's a good design philosophy not to depend too much on other checkers being activated. That's also why I use quotes here instead of """. If people really want to use ''' or " then this formatter won't break that.
We could also think: let's format everything and if the user doesn't want us to they should clearly indicate what is a summary and what isn't. However, I think that's not the most user-friendly approach.
I'll think about this some more and hopefully come up with something elegant.

make it very clear how to avoid this (awful) default

Hmm, why put the bad default value in the first place if we think everyone need to change it ? black is pretty successful with a "sane default" and the possibility to change it to the value from pep8 if you're a purist (and 79 is more reasonable than 72 too). Having good default values is pretty important imo.

avoids any future discussion

If the default is sane it's going make the discussion less likely. I think that if black default was 79, the first thing to do when adopting black would be to discuss between 79, 80, 90, 100, 110, 120 for line length. As it's reasonable, I think discussions happen less often.
Also it would not be a problem if we go check the line length for black/pylint/flake8 ourselves in the configuration files later on.

We could also think: let's wrap everything and if the user doesn't want us to they should clearly indicate what is a summary and what isn't

Pretty much agree with that, if the docstring is bad we cannot make it good magically. crap in, crap out !

@Pierre-Sassoulas Could you review this 😄


Merge static build fixes into release-staging for testing

…334)

disable sparse functions when building a static library

explicitly disable

add include guards as failsafe

remove reference to unused variable

build without sparse by default

Forcing merge since these changes were already approved for the release branch as discussed with Juan.


Implements Volunteer Page Posting

This branch implements a button on the volunteer page that allows posting a client request to the donor page. After posting the request is deleted

This branch also implements deleting from the volunteer page