[Bug] Exception when running Get-DbaRegServer and have Azure Data Studio connections

Environmental information

#### PowerShell version:

Name                           Value
----                           -----
PSVersion                      5.1.17763.771
PSEdition                      Desktop
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   10.0.17763.771
CLRVersion                     4.0.30319.42000
WSManStackVersion              3.0
PSRemotingProtocolVersion      2.3
SerializationVersion           <IP_ADDRESS>

#### dbatools Module version:

Name    : dbatools
Path    : C:\Users\xxxxx\Documents\WindowsPowerShell\Modules\dbatools\1.0.20\dbatools.psd1
Version : 1.0.20

Name    : dbatools
Path    : C:\Program Files\WindowsPowerShell\Modules\dbatools\1.0.82\dbatools.psd1
Version : 1.0.82

Name    : dbatools
Path    : C:\Program Files\WindowsPowerShell\Modules\dbatools\1.0.72\dbatools.psd1
Version : 1.0.72

<!-- Only if using non-English versions of Windows -->
<!-- Paste output of `Get-Culture` -->
```powershell
# Replace with output of Get-Culture

SQL Server:

/* REPLACE WITH output of @@VERSION */

/* REPLACE WITH output of @@LANGUAGE */

Report

Host used

[ ] powershell.exe
[x ] ISE
[ ] VS Code
[ ] Other (please specify)

If anything other than powershell.exe was used, please confirm that you can duplicate the issue with powershell.exe

[ ] Still buggy in powershell.exe

Errors Received

PSMessageDetails      :
Exception             : System.Management.Automation.SetValueInvocationException: Exception setting "ConnectionString": "Format of the initialization string does not conform to
                        specification starting at index 107." ---> System.ArgumentException: Format of the initialization string does not conform to specification starting at index 107.
                           at System.Data.Common.DbConnectionOptions.GetKeyValuePair(String connectionString, Int32 currentPosition, StringBuilder buffer, Boolean useOdbcRules, String&
                        keyname, String& keyvalue)
                           at System.Data.Common.DbConnectionOptions.ParseInternal(Hashtable parsetable, String connectionString, Boolean buildChain, Hashtable synonyms, Boolean firstKey)
                           at System.Data.Common.DbConnectionOptions..ctor(String connectionString, Hashtable synonyms, Boolean useOdbcRules)
                           at System.Data.Common.DbConnectionStringBuilder.set_ConnectionString(String value)
                           at Microsoft.SqlServer.Management.RegisteredServers.RegisteredServer.GetServerNameFromConnectionString(String connString)
                           at Microsoft.SqlServer.Management.RegisteredServers.RegisteredServer.set_ConnectionString(String value)
                           at CallSite.Target(Closure , CallSite , Object , Object )
                           --- End of inner exception stack trace ---
                           at System.Management.Automation.ExceptionHandlingOps.ConvertToMethodInvocationException(Exception exception, Type typeToThrow, String methodName, Int32 numArgs,
                        MemberInfo memberInfo)
                           at CallSite.Target(Closure , CallSite , Object , Object )
                           at System.Management.Automation.Interpreter.DynamicInstruction`3.Run(InterpretedFrame frame)
                           at System.Management.Automation.Interpreter.EnterTryCatchFinallyInstruction.Run(InterpretedFrame frame)
TargetObject          :
CategoryInfo          : NotSpecified: (:) [], SetValueInvocationException
FullyQualifiedErrorId : ExceptionWhenSetting
ErrorDetails          :
InvocationInfo        : System.Management.Automation.InvocationInfo
ScriptStackTrace      : at Get-DbaRegServer<Process>, C:\Users\xxx\Documents\WindowsPowerShell\Modules\dbatools\1.0.20\allcommands.ps1: line 31486
                        at <ScriptBlock>, <No file>: line 1
PipelineIterationInfo : {}

Steps to Reproduce

When a connection using Azure Data Studio is saved and run Get-DbaRegServer it generates the error in question...
Expected Behavior

No
Actual Behavior

Exception

i will need to go and reopen all of these auto closures because there's been no movement and they're all legit. @wsmelton can you disable stale bot for now?

I changed it to stretch the days between stale and closure to 120 days. We can't really disable it the way it is setup.
I can go back and look to see if the Action works better now because that we can disable.

This is still an issue with 1.0.102
This is the part causing the issues - I'm not sure what it's used for, it's not returned as a default property - perhaps we could just remove it? I believe it is only relating to ADS connections but I don't have a CMS setup to test against right now - perhaps a goal for the weekend.
https://github.com/sqlcollaborative/dbatools/blob/6752767e1019a3150c2b7f35f66e22f966c2fa05/functions/Get-DbaRegServer.ps1#L178-L180

ADS CMS <> SSMS CMS

@wsmelton - not sure what you are implying there?
I mentioned CMS setup for testing if we removed that section - do you know what that 3 lines I highlighted is for and is it needed for ADS CMS servers?

The issue is that multiple connection strings are being returned for each server. I've added a PR to filter the results from Get-ADSConnection that should resolve the issue


Provide node context in ClassMapping > ConvertFrom

Motivation
This feature would help us maintain relationships between content items that are being converted from pages to reusable content. For example, we could use the GUID of the node being converted to a reusable item, to look up child nodes in the tree and get additional data to map to the parent node.
Proposed solution
Having full TreeNode context in the ConvertFrom converter function.

@tkrch I appreciate it! I'm only asking because it's blocking a part of the migration work I am doing - do you know when we could expect this to be implemented?


Bayes by Backdrop improvements

Added illustrations
Improved current implementation of Bayes by Backprop
Added gluon implementation of Bayes by Backprop

Great work, thanks!


[REEF-535] Triage TODO notes in reef-wake

JIRA:
REEF-535
Pull request:
This closes #

@tcNickolas This looks good. I will test and merge.


Update create-a-custom-sensitive-information-type-in-scc-powershell.md

@chrfox  Can we add -ReadCount 0 , after -Encoding Byte) in the [New-DlpSensitiveInformationTypeRulePackage] example here.  If not, you will receive an error regarding the object size is not allowed.  This is shown in the Powershell article for [New-DlpSensitiveInformationTypeRulePackage].
Thanks

merged


feat(esp8266): Add script to make esp8266 using easily

"aos make XXXX@YYYY ESP8266_DWN=1" to download all firmware(boot, blank, init_data, app)
"aos make XXXX@YYYY ESP8266_DWN_APP=1" to download only app
"aos make XXXX@YYYY ESP8266_DBG=1" open printf info debug window
"aos make XXXX@YYYY ESP8266_DWN=1 ESP8266_DBG=1" download and then open printf info debug window
"ESPPORT" can select the board download & debug UART, just add "ESPPORT=/dev/ttyUSBxx" to the command
"MONITORBAUD" can select the debug UART baudrate, just add "MONITORBAUD=xxxx" to the command

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.Dong Heng seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.

@donghengqaz
Thanks for the contribution! Would you please sign the CLA above?
BTW, is this copy of esptool.py necessary? aos-cube will install it as a dependency.


Remove floating point code from cdef_dist

The floating point computations in cdef_dist take a decent amount of time and make it hard to implement assembly covering large blocks (> 8x8).

I guess just the output (although might have to scale earlier?) https://github.com/xiph/rav1e/blob/master/src/rdo.rs#L199


Allow user to specify a commit message to be used

Describe what problem your feature request solves:
"Created by OWASP Threat Dragon" and "Updated by OWASP Threat Dragon" are fine for playing around, but are not enough for a real production use.
Describe the solution you'd like:
I would like to be able to write a commit message myself while saving changes in TD. The feature should be disabled by default but has 2 modi if activated via ENV: enabled (you can write your own message) and forced (you have to write your own message).

this is a good suggestion @fdellwing , are you able to provide the implementation? I can assign it to you if you like

I have no idea where I would even start with that. Like, I know where these strings are hardcoded. But I have no idea how to create a popup message field or pass the content around. I'm not a developer and even less a JS developer.


Make the authorizer and registry authorizer configurable

Fixes: #12584
This change makes the authorizer and registryAuthorizer of the registry client configurable via options. This allows Go SDK users to override the authentication behavior of the client.
This PR makes both the authorizer and registryAuthorizer configurable because depending on the exact scenario that may be needed. The default registryAuthorizer only supports a specific implementation of the authorizer.
What this PR does / why we need it:
This PR ports #12588 to the dev-v3 branch. I'm opening this per-request after it was merged to v4/main https://github.com/helm/helm/pull/12588#issuecomment-2492867470
Special notes for your reviewer:
If applicable:

[ ] this PR contains user facing changes (the docs needed label should be applied if so)
[ ] this PR contains unit tests
[ ] this PR has been tested for backwards compatibility

At this point I think we should revisit this after oras v2 merges

This is just a backport for v3 of the PR that was already merged into main (now that it's for v4). At this point there is no significant divergence.


Class list: Expand fix, sort columns, and revised inline profile design

Who is this PR for?
K8 teaching teams
What problem does this PR fix?
The "expand" button wouldn't work without interaction (maybe react-beautiful-dnd bug from 6.x >
11.x?).  This tries to work around.
What does this PR do?
It adds a "sort" button to each column for sorting by name.
It revises the design for the inline profile to match "equity check" categories, and to emphasize notes and humanize more.
Screenshot (if adding a client-side feature)
before

after

sorting

Checklists
Which features or pages does this PR touch?

[x] Class lists

Does this PR use tests to help verify we can deploy these changes quickly and confidently?

[x] Included specs for changes
[x] Manual testing made more sense here

selfie


Add inverse_beta_cdf scalar function

Add scalar function inverse_beta_cdf mapped to presto inverse_beta_cdf.
The underlying math library being used is different from Presto, boost::math::quantile calculates inversed cdf for C++.
Presto wiki reference: https://prestodb.io/docs/current/functions/math.html#probability-functions-inverse-cdf
Presto code implementation reference:

https://github.com/prestodb/presto/blob/master/presto-main/src/main/java/com/facebook/presto/operator/scalar/MathFunctions.java#L826
https://github.com/prestodb/presto/blob/master/presto-main/src/test/java/com/facebook/presto/operator/scalar/TestMathFunctions.java#L1389

I think the work has already been done in https://github.com/facebookincubator/velox/pull/5238 :(

I think the work has already been done in #5238 :(

Hmm I was seeing it in the recent query run, didn't realize there was an existing fix, thanks for referring it!


Move pdfmake from devDependencies to dependencies

... since that appears to be what it is :smiley:

Thanks for your pull request!
pdfmake is not required to run the module. It is only needed to build the minified pdfmake-browserified.min.js in the build directory. Building seems to be a part of development.
But I am not sure if there is an (unofficial) node convention for these kind of things, I am not a node expert :-)
Is this an inconvenience or a blocker for you?
Please let me now, I am willing to change it, if it is an improvement.

It is not possible to use pdfmake-browserified without building it. The same is true for pdfmake.
As used PDF library needs to be processed with brfs. Which is only possible during development, not production.

Hey @xErik, no prob.
We have an app that's built with webpack, and wanted to try this module out during an investigation into using pdfmake. We didn't want to use the prebuilt version during the investigation.
With pdfmake being a dev dependency, it doesn't get installed when we install this module into our app, which means we can't build it easily with our webpack setup.
In saying that, I can see the intent of this module, given it's named -browserified, and it's main points to the built file :smile: So I can understand if you'd rather close this PR.
Incidentally, we have pdfmake working with our webpack setup. It's not running from the pre-built files, but we're building it via our own webpack setup. It just means we have to tweak our own webpack config to do things similar to pdfmake's webpack.config.js. I've also created a fork which allows us to access the browser-extensions used during the build process.

Building your own version sounds good, I needed to do the same and created pdfmake-browserified.
It would be more convenient if one simply could use pdfmake "out of the box" :-)


Flaky test list

CI has been improved with the last few PRs, but there are still sources of flakyness.
Floodgates are open here: #3189
This ticket is to list them all, and get to the bottom of it and fix them all.
Feel free to add here any transient failure you see in the comments, with a copy of the logs and a link to the failing build (first failure in the run ONLY).
Fixed by #3192:

[x] TestCreateWithMACAddress - fails regularly - no idea -     container_create_linux_test.go:121: assertion failed: res.ExitCode is not 0: Command should have succeeded%!(EXTRA string=time="2024-07-05T16:36:12Z" level=fatal msg="container \"e06fb05e3174c1be8afedfc8d58981cdf90f14507b5d7040e6e6f2448823b7ca\" in namespace \"nerdctl-test\": not found"       ) see https://github.com/containerd/nerdctl/actions/runs/9811180349/job/27092899863?pr=3178
[x] TestExecEnv on windows -  container_exec_test.go:79: assertion failed: res.ExitCode is not exitCode: time="2024-07-05T16:54:47Z" level=fatal msg="container \"30bbda4628649e094fee7f172c3aa7c16b1ddb032685f46bc297c9126fa9cefd\" in namespace \"nerdctl-test\": not found" - looks strangely similar - not sure what is going on here - is there some concurrent prune happening? https://github.com/containerd/nerdctl/actions/runs/9811363482/job/27093391801?pr=3181
[x] TestRunUserGID 2024-07-05T17:26:39Z" level=fatal msg="container \"d5eddc278faf40a910638e34c9186d5d1b4027c77be4e08f38192850bb194190\" in namespace \"nerdctl-test\": not found (https://github.com/containerd/nerdctl/actions/runs/9811583654/job/27093967186?pr=3178)

These issues look a lot like #3092
Something really weird is going on.

Latest "no-retry" canary PR:

[ ] TestTailFollowRotateLogs: 2 counts - this is very likely fixed by changes in #3178
[ ] TestIPFSAddress: 1 count - probably fixed by #3181
[ ] TestNetworkWithNamespace: 4 counts - task not found 3 times - possibly another variant of #3167 - last count gives container not created: not found which is a bit more worrisome as this is from containerd
[ ] TestLogin: 1 count - TLS handshake timeout - either some weird connectivity issue, a crash in the auth container, or ?
[ ] TestImageEncryptJWE: 1 count - unclear - image_encrypt_linux_test.go:127: assertion failed: res.ExitCode is 0
[ ] TestComposeUp: 1 count - unclear - OCI runtime start failed: cannot start a container that has stopped: unknown

Build link:
https://github.com/containerd/nerdctl/actions/runs/9815148442?pr=3189

Canary PR: 7 failures (3 counts TestTailFollowRotateLogs, 1 count TestIPFSAddress).
3 "new":

[ ] TestBuildAttestation 2 counts - log is very weak - error: failed to solve: not found sounds like a buildkit message to me
[ ] TestRunRestart - OCI runtime start failed: cannot start a container that has stopped: unknown - a bit worrisome

Build link:
https://github.com/containerd/nerdctl/actions/runs/9815520719?pr=3189

Canary: 7 failures

TestRunWithSystemdTrueEnabled
TestBuildAttestation
TestImageEncryptJWE
TestTailFollowRotateLogs
TestIPFSComposeUp (2 counts)
TestVolumeLsFilter

TestBuildAttestation & TestTailFollowRotateLogs are now top two.

As of today, a lot of the issues got PR merged-in.
Latest Canary build (with extra fixes for TestBuildAttestation) failed 4 times:

[ ] TestCopyToContainer
[ ] TestTailFollowRotateLogs is still failing (3x count)

Latest (unmerged yet) version of the test tooling will introduce a way to mark flaky tests in code - allowing to have separate runs for flaky (with retry) / not flaky.
This ticket here is getting outdated and is unwildly.
Closing for the better solution of in-code marking.


[BUG] Accessible/Tagged PDFs not generated when running on Lambda

Environment

chromium Version: HeadlessChrome/112.0.5614.0
puppeteer / puppeteer-core Version: 19.8.0
Node.js Version: 18.x
Lambda / GCF Runtime: nodejs18.x

Expected Behavior
This chromium version should produce tagged PDFs by default.
Current Behavior
When running the code locally, tagged PDFs are created, but when deployed on Lambda it just creates a regular PDF with no accessibility attributes.
Steps to Reproduce
Generate a PDF and verify if it has accessibility attributes - I use VoiceOver to check if h1 tags are announced correctly.
Possible Solution
Really just looking for advice on what the issue might be - what differences would there be running this locally vs in Lambda?

My testing so far indicates the problem is caused by the included Chromium binary. When I use the default executablePath and get the default browser, it does not generate accessibility tags - if I install chromium through yum and point the executablePath there it works fine.
Is there supposed to be any difference between the two?

@dom-jones-1 The chromium binary is compiled with ansible as described here. It's likely coming from the settings used to compile chromium.

I believe it's probably because we are using the headless_shell example code. Likely this is the reason for all the bugs.

I generated an accessible PDF by changing _/ansible/plays/chromium.yml. In the Creating Headless Chromium Configuration play I removed
enable_basic_print_dialog = false

and added the following
enable_basic_printing = true
enable_pdf = true
enable_tagged_pdf = true

NOTE: This will dramatically increase the time required to generate a PDF.

I generated an accessible PDF by changing _/ansible/plays/chromium.yml. In the Creating Headless Chromium Configuration play I removed
enable_basic_print_dialog = false

and added the following
enable_basic_printing = true
enable_pdf = true
enable_tagged_pdf = true

NOTE: This will increase the time required to generate a PDF.

Sorry for crowding the issue but would you mind sharing the chromium binary you've created? I've been getting build errors with my attempts :(


Proposal of restructuring code (RFC)

As the project grows, in order to better manage the code, expectations and dependencies, I hope we can re-organize the repository by creating multiple modules. With that, we no longer have to put all ideas into one single lib, trying to make it both a JDBC driver and Java client for ClickHouse. Instead, we'll follow JDBC specification in clickhouse-jdbc module for standard update/delete operation, synchronous execution, and escape syntax like {d '2020-02-17'}(which is same as toDate('2021-02-17') by the way) etc. On the other hand, we'll stick with the "clickhouse-way" when implementing clickhouse-client module, so that it behaves like any other clickhouse clients.

optionally new package name(and maven groupId): ru.yandex.clickhouse -> tech.clickhouse
new directories(and maybe a separate repo like clickhouse-java later)|
|-- clickhouse-client
|-- clickhouse-http-client
|-- clickhouse-grpc-client
|-- clickhouse-native-client
|-- clickhouse-jdbc
|-- clickhouse-jdbc-ext (tentative)
|-- clickhouse-gql (tentative)
|-- clickhouse-r2dbc (tentative)
|...

modules and API layers

how to approach?
I hope we can implement the new structure starting from 0.3.0, but it's primarily about moving code from src to clickhouse-jdbc/src and pile up code into other modules. This ensures we can produce binary compatible package in 0.3 series. And then, starting from 0.4.0, I hope we can optionally change package name and refactor clickhouse-jdbc module by leveraging code from clickhouse-client module etc., which will introduce different binary packages.

Please feel free to comment and I'll start to create PR if no concern.

I am OK with all of this.

I am totally Ok with all the proposed changes.

While I do see a benefit in having clearer module structures, I think there is other stuff to do for 0.3.0 before trying to break things up this way, see Roadmap 2021 #537

Update:

added clickhouse-mysql-client module, but it's just for testing ClickHouse server using 3rd party JDBC driver for comparison
will add clickhouse-gql, clickhouse-r2dbc, and clickhouse-jdbc-ext 3 modules in the future when needed


Make .prettierrc JSON compliant

Same issue here :)

After merging the "official" CodeSandbox linked to this repository https://codesandbox.io/s/github/andybelldesign/hylia should work.

Cheers!


Can Popper use constants?

Simple question as the header says.
I would like to learn rules that have constants in them.

Thank you! magicpopper worked right away :)
I will try to use popper as well.
The performance should be similar?

Thank you! magicpopper worked right away :)
Great!

The performance should be similar?

Popper should be much faster than MagicPopper (mayeb 10x+ faster) because it contains many recent improvements that are not in MagicPopper. MagicPopper will be merged into Popper soon, but I have simply not found the time to do it ...
Kind regards,
Andrew

On 18 Apr 2024, at 09:47, Nelson Higuera @.***> wrote:
Thank you! magicpopper worked right away :)
I will try to use popper as well.
The performance should be similar?
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you commented.Message ID: @.***>

Thank you for the prompt reply.
I am having trouble running it on Popper.
############# bias ##############
head_pred(hard, 1).
body_pred(contains, 2).
body_pred(direction, 2).
body_pred(shape, 2).
body_pred(size, 2).
constant(small, szid).
constant(large, szid).
constant(airliner, shid).
constant(biplane, shid).
constant(jet, shid).
constant(fighter, shid).
constant(utility, shid).
constant(tandem, shid).
constant(road, shid).
constant(mountain, shid).
constant(articulated, shid).
constant(double, shid).
constant(regular, shid).
constant(school, shid).
constant(truck, shid).
constant(suv, shid).
constant(minivan, shid).
constant(sedan, shid).
constant(wagon, shid).
constant(chopper, shid).
constant(scooter, shid).
constant(cruiser, shid).
constant(dirtbike, shid).
constant(south, did).
constant(southwest, did).
constant(west, did).
constant(northeast, did).
constant(north, did).
constant(northwest, did).
constant(west, did).
constant(southeast, did).
type(hard, (sid,)).
type(contains, (sid, oid)).
type(direction, (oid, did)).
type(shape, (oid, shid)).
type(size, (oid, szid)).
body_pred(P,1):-
constant(P,_).
type(P,(T,)):-
constant(P,T).
####### bk ##############
:-style_check(-discontiguous).
contains(s1, o1_0).
direction(o1_0, southeast).
shape(o1_0, double).
size(o1_0, small).
...
examples
pos(hard(s1)).
pos(hard(s2)).
neg(hard(s4)).
neg(hard(s8)).
...
It throws the error:
hard(V0):- small(V2),size(V1,V2),contains(V0,V1).
Traceback (most recent call last):
File "/home/nelson/Research/ILP4SD/Popper/popper.py", line 8, in
prog, score, stats = learn_solution(settings)
File "/home/nelson/Research/ILP4SD/Popper/popper/loop.py", line 1729, in learn_solution
timeout(settings, popper, (settings, tester, bkcons), timeout_duration=int(settings.timeout-time_so_far),)
File "/home/nelson/Research/ILP4SD/Popper/popper/util.py", line 81, in timeout
result = func(*args, **kwargs)
File "/home/nelson/Research/ILP4SD/Popper/popper/loop.py", line 1658, in popper
Popper(settings, tester, bkcons).run()
File "/home/nelson/Research/ILP4SD/Popper/popper/loop.py", line 247, in run
pos_covered = tester.test_prog_pos(prog)
File "/home/nelson/Research/ILP4SD/Popper/popper/tester.py", line 113, in test_prog_pos
return frozenset(query_once(q)['S'])
File "/home/nelson/miniconda3/lib/python3.10/site-packages/janus_swi/janus.py", line 243, in query_once
return _swipl.call(query, inputs, keep)
janus.PrologError: ''/1: Unknown procedure: small/1
If possible could you help me spot the error?
Thank you again.

For each constant, you need to define the corresponding relation in the BK file, such as:
airliner(airliner).
small(small).

It is silly, but I have not yet implemented the code to automatically add them to the BK (I will move it up my TODO list),

It worked!
Thank you so much for your help!


Improve WassersteinDistanceResult

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 100.00%. Comparing base (e49dc5e) to head (26a9626).
Report is 2 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff            @@
##              main      #484   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files          114       114
  Lines         2956      2961    +5
  Branches       178       178
=========================================
+ Hits          2956      2961    +5

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


[CPT-1489] Add data-testid to ValidationErrors in QueryBuilder

CPT-1489
Description
This PR introduces data test id to the validation errors component in the query builder
How to test

Verify that there is JSDoc to the attribute testIds in the Query Builder.
Verify it includes validationErrors.

Screenshots
Development checks

[ ] Add changeset according to guidelines (if needed)
[ ] Read CONTRIBUTING.md and Component API principles
[ ] Make sure that additions and changes on the design follow Toptal's BASE design, and it's been already discussed with designers at #-base-core
[ ] Annotate all props in component with documentation
[ ] Create examples for component
[ ] Ensure that deployed demo has expected results and good examples
[ ] Ensure the changed/created components have not caused accessibility issues. How to use accessibility plugin in storybook.
[ ] Self reviewed
[ ] Covered with tests (visual tests included)

Breaking change

[ ] codemod is created and showcased in the changeset
[ ] test alpha package of Picasso in StaffPortal

All development checks should be done and set checked to pass the
GitHub Bot: TODOLess action

PR commands

List of available commands:

@toptal-bot run package:alpha-release - Release alpha version
@toptal-anvil ping reviewers - Ping FX team for review

PR Review Guidelines

When to approve? ✅
You are OK with merging this PR and

You have no extra requests.
You have optional requests.

Add nit: to your comment. (ex. nit: I'd rename this variable from makeCircle to getCircle)

When to request changes? ❌
You are not OK with merging this PR because

Something is broken after the changes.
Acceptance criteria is not reached.
Code is dirty.

When to comment (neither ✅ nor ❌)
You want your comments to be addressed before merging this PR in cases like:

There are leftovers like unnecessary logs, comments, etc.
You have an opinionated comment regarding the code that requires a discussion.
You have questions.

How to handle the comments?

An owner of a comment is the only one who can resolve it.
An owner of a comment must resolve it when it's addressed.
A PR owner must reply with ✅ when a comment is addressed.

@toptal-bot run package:alpha-release

@toptal-bot run package:alpha-release

@toptal-bot run package:alpha-release


Don't use the volatile qualifier for multi-threaded values

Reading through the code I noticed the use of volatile values to communicate between threads, like this:
https://github.com/facil-io/cstl/blob/835bd33990d4d388366f44a295757ab9f798a8ed/fio-stl/102 queue.h#L523
which is defined here:
https://github.com/facil-io/cstl/blob/835bd33990d4d388366f44a295757ab9f798a8ed/fio-stl/102 queue.h#L79
That is, most definitely, not safe.
See, for example, this site with a rather cute name:

https://isvolatileusefulwiththreads.com/c/

The second link is broken, but I found another copy of the article here. Here's an excerpt:

For multi-threaded programming, there are two key issues that volatile is often mistakenly thought to address:

atomicity
memory consistency, i.e. the order of a thread's operations as seen by another thread.

Let's deal with (1) first. Volatile does not guarantee atomic reads or writes. ...
Now consider issue (2). Sometimes programmers think of volatile as turning off optimization of volatile accesses. That's largely true in practice. But that's only the volatile accesses, not the non-volatile ones. ...

The StackOverflow link also has some good content, e.g.:

The problem with volatile in a multithreaded context is that it doesn't provide all the guarantees we need. It does have a few properties we need, but not all of them, so we can't rely on volatile alone.
However, the primitives we'd have to use for the remaining properties also provide the ones that volatile does, so it is effectively unnecessary.

and:

C programmers have often taken volatile to mean that the variable could be changed outside of the current thread of execution; as a result, they are sometimes tempted to use it in kernel code when shared data structures are being used. In other words, they have been known to treat volatile types as a sort of easy atomic variable, which they are not. The use of volatile in kernel code is almost never correct; this document describes why.

Finally, here is a short answer given by Herb Sutter, a C/C++ super-expert:

I am implementing multi producer single consumer problem. I have shared variables like m_currentRecordsetSize which tells the current size of the buffer. I am using m_currentRecordsetSize in a critical section do i need to declare it as volatile.

If you’re in C or C++, and the variable is not already being protected by a mutex or similar, then you need to declare it atomic (e.g., if it’s an int, then atomic_int in C or atomic in C++. Not volatile.

I thought that your use of volatile would be considered; that's why I went a little over the top with links and quotes. :-) It's also why I called out a specific example. That same example is used in the linked articles to explain why such usage of volatile is wrong!
I specifically tried to point to some particularly authoritative — i.e., "highly reliable" — sources. I'm not aware of any authoritative source that carves out the kind of exception you're describing for volatile?

(similar to what one would do in a signal handler)

Yes, signal handlers are one of three "existing portable uses" of volatile given by Hans Boehm (yet another authoritative source). But there's a critical caveat: you have to use it with a specific type, a type that's guaranteed to be atomic on the given platform. Here's the way the libc manual explains it:

To avoid uncertainty about interrupting access to a variable, you can use a particular data type for which access is always atomic: sig_atomic_t. Reading and writing this data type is guaranteed to happen in a single instruction, so there’s no way for a handler to run “in the middle” of an access.
The type sig_atomic_t is always an integer data type, but which one it is, and how many bits it contains, may vary from machine to machine.

Btw, Boehm's post also includes this quote from David Butenhof:

the use of volatile accomplishes nothing but to prevent the compiler from making useful and desirable optimizations, providing no help whatsoever in making code 'thread safe'

Butenhof is the author of Programming with POSIX Threads.

Thank you for reaching out to me with your concerns and for opening this issue. The debate is important and I fixed some missing atomics in the FIO_QUEUE module.
However, I will close the issue for now, please free to open a new issue (or re-open this one) if you have specific points in the code you think I should fix.
Also, note:
The while (!grp->stop)  use of volatile as a flag was not the mistake in the example code here. The issue in the example concerned the instruction re-ordering by the compiler (the missing instruction barrier that would be provided by atomics) and not the use of volatile...


Update load_compounds command to incorporate synonyms

FEATURE DESCRIPTION
Feature Inspiration
Various compounds may have synonyms we will encounter new synonyms for existing compound records.
Feature Description
Have load_compounds accept an additional column that contains a synonym record which will be associated with compound records indicated by the name and HMDB id on that line
Alternatives Considered
A separate command to load synonyms is an option, but it seems that would complicate the interface (another command) without providing much benefit (that I can think of right now).
Comment
Add any other context or screenshots about the feature request here.

ISSUE OWNER SECTION
Assumptions

1 row per compound
synonyms delimited by semi-colon (by default), or override synonym_delimiter by new argument
new example data file
NOTE: this new file changes the previous compound names of C16:0 to palmitic acid , C18:1 to oleic acid, C18:2, to linoleic acid.  These new names also differ from the *ate style of other compounds, if that is important.
-NOTE: the new file also has (probably) new compounds dihydroxy-acetone-phosphate , fumarate, glucose-6-phosphate , glycerol-3-phosphate , stearic acid (C18:0) and taurine

Requirements

Compounds (as indicated by the name) that don't exist should be added
Existing compounds should have synonyms added (unless the synonym is already associated with that compound)
Synonyms in the file that already exist, but as associated with a different compound should throw an error

Limitations

Affected/Changed Components

DataRepo/load_compounds.py
DataRepo/utils.py

DESIGN
GUI Change description
Describe changes the user will see.
Code Change Description (Pseudocode optional)
Describe code changes planned for the feature.
Tests
A test should be planned for each requirement (above), where possible.

@lparsons @hepcat72 @fkang-pu  Looking at the new file (see notes above), it did appear that ~6 compounds are probably new to the database .  However there are 3 rows that match existing compounds (C16:0 == palmitic acid , C18:1 == oleic acid, C18:2 == linoleic acid, ordered as  previous_name == new_name)
As an implementation detail, would you prefer:

don't touch the existing compound record (name), but just add the list of all names as compound_synonyms
update the compound.name to presumably the newly preferred designation in the latest file, and also add the list of all names as compound_synonyms

I say first past the post is the winner, so: 1.  We can tackle changes if they're requested.

My preference would be to throw an exception on rows that attempt to "change the primary name". Otherwise, option 1 is OK.

OK, maybe if we meet with Michael tomorrow, we can discuss.  Fan mentioned (in office discussions) kicking the file back to the lab for clarifications (presumably the same result with the "throw an exception" route).  However, to my mind, at some point if your are taking researcher's files as source material, you may eventually have to update the previous records with the "new preferred names" [i.e. Michael may reply with "yes, I meant it when I want wrote linoleic acid instead of C18:2 as the preferred name"; followed in 6 months by "how about linoleate, instead"]

yes, for mapping synonym(s) to compound name, I prefer to generate a list during loading validation process and ask the lab for clarification.  I use the following example to share how I wish it would be handled:
assume a case with "glucose"
in TraceBase:
compound.name=glucose,  HMDB_ID=HMDB0000122
synnonyms=Glucose, GLUCOSE

then peakgroup="D-Glucose" in loading file which has no match with any compound name or synonym. The question is how to handle it during loading?
I would prefer:

raise exception during loading validation and suggest compound/synonym mapping and kick back to the user for verification:

suggestion: D-Glucose => compound.name=glucose,  HMDB_ID=HMDB0000122

how to record detected peakgroup in TraceBase for this case?
I would like to add a new column named pg_name_in_file (or call it user_preferred_name, pg_display_name, or a better name)
after loading, the record in TraceBase: peakgroup.name="glucose", peakgroup.pg_name_in_file="D-Glucose".

In this way, we would have consistent name for peakgroup.name without losing user selected synonym for a detected peak.
p.s. I hope we can work with the lab to compile and curate a long list of compounds with synonyms, so missing compounds/synonyms would not occur frequently during loading.  We should also remind the lab to choose which synonym as compound.name carefully and try not to change unless they have to.


OldKanban._id Master

Coverage remained the same at 78.419% when pulling 723b1d951d1388abd85fce9c2a5336bdc037506a on jackyruslymoonlay:master into 0b5798bf5b6ab5a4cef84b56d9cc5728953567d9 on danliris:master.


Scrollable FlatList within _renderContent

Has anyone been able to get a FlatList successfully scrolling within a section that gets generated from _renderContent? I can get the list items to render and start to scroll but it refuses to continue to scroll "endlessly/infinitely" if there are more items that can fit on the screen.

Hey @Tzinov15 can you create a simple example on https://snack.expo.io/. Just so we can see it in action and debug the problem

Can confirm. Am running into the same issue trying to scroll a flatlist from within renderContent.

Closing this as inactive as there's been no activity for over 30 days. If the problem re-occurs on the latest version of react-native-collapsible then ping me to reopen or create a new issue.

I'm having this exact issue.

Expand project team and the list ends at demo test checkbox
Collapse section and open it again
This time it renders all of the fields in the section plus a big white space at the bottom

renderAccordionHeader = section => <Text style={styles.headerText}>{capitalizeFirstLetter(section.key.toLowerCase())}</Text>;
renderAccordionContent = section => <FlatList data={section.data} renderItem={this.renderItem} keyExtractor={(item, index) => index} />;
renderItem = item => <FieldDispatcher field={item.item} parentRecordId={this.props.id} dataSource={recordDataSource.endpoint} />;
<ScrollView     style={{ flex: 1 }}     refreshControl={<RefreshControl refreshing={this.state.refreshing} onRefresh={() => this.refreshComponent()} />}           >             <Accordion               sections={this.getOriginalRecordData()}               renderHeader={this.renderAccordionHeader}               renderContent={this.renderAccordionContent}               renderSectionTitle={() => null}               initiallyActiveSection={0}             />           </ScrollView>

@CarlosJJR Please create a new issue, and try to replicate it in https://snack.expo.io


set() should be able to set the zone

Should be identical to fromObject() in form

Decided against this. It's too weird to set units at the same time as setting the zone. Are the units in the old zone or the new one? What about the units you don't set?


Run 'limits' tests in travis.

These tests were never run. It should though!
My changes affect the following components

[X] Tests

Types of changes

[X] Bug fix (generally a non-breaking change which closes an issue).

Checklist:

[X] I signed an Apache CLA.
[X] I reviewed the style guides and followed the recommendations (Travis CI will check :).

Codecov Report

Merging #3933 into master will decrease coverage by 4.69%.
The diff coverage is n/a.

@@            Coverage Diff            @@
##           master    #3933     +/-   ##
=========================================
- Coverage   85.35%   80.66%   -4.7%
=========================================
  Files         146      146
  Lines        7054     7054
  Branches      422      422
=========================================
- Hits         6021     5690    -331
- Misses       1033     1364    +331

Impacted Files
Coverage Δ

...core/database/cosmosdb/RxObservableImplicits.scala
0% <0%> (-100%)
:arrow_down:

...core/database/cosmosdb/CosmosDBArtifactStore.scala
0% <0%> (-95.1%)
:arrow_down:

...sk/core/database/cosmosdb/CosmosDBViewMapper.scala
0% <0%> (-92.6%)
:arrow_down:

...whisk/core/database/cosmosdb/CosmosDBSupport.scala
0% <0%> (-81.82%)
:arrow_down:

...abase/cosmosdb/CosmosDBArtifactStoreProvider.scala
0% <0%> (-58.83%)
:arrow_down:

...la/whisk/core/database/cosmosdb/CosmosDBUtil.scala
92% <0%> (-4%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d7e9ce6...120daef. Read the comment docs.

Could the limit and max duration (only remaining suite now excluded?) be run as seperate layers?


JOSS paper

In the process of wrapping up the JOSS revisions, let's bring this into the main branch before creating a new release.

Indeed, it does add a APA/BibTex ref. link on the right, but this will make it more obvious :)


installation issue. CAMERA type not defined. Please update 'home/pi/allsky/config.config.sh'

Hi,
I have done a fresh install many time over the past couple of days and always got this following error when trying to access web interface.
CAMERA type not defined. Please update '/home/pi/allsky/config.config.sh'

@pimmoon we'll get this fixed.  We've been looking at ways to do it.
@linuxkidd We can fix this by prompting for CAMERA during the allsky/install.sh, before the reboot prompt (probably should do it at the beginning), then editing config.sh as soon as we create it.  CAMERA definitely needs to be set prior to the install.sh finishing cause it turns out people like to get into the WebUI right away.


Selected-route is not selecting the specified route.

Expected behavior and actual behavior:
Tried to preselect a route using the "selected-route" attribute, but the route doesn't seem to get pre-selected as expected.
Environment:
CB: Redesign branch - today's pull
App-helpers: Master - today's pull
Screenshot:

Code example:
        <px-context-browser
            selected-route='["Sy1DFzRs_ENTERPRISE","r1TutzRs_SITE"]'
            items="{{data}}"
            selected="{{selectedItem}}"
            has-filter
            open-arrow
            has-favorites
            loading-timeout=0
            enable-warnings
            >
        </px-context-browser>

/cc @davidrleonard

Fixed in:

px-context-browser b9b55e8d74ff6e74880699d0b051b5520b585e0b
https://github.com/PredixDev/px-app-helpers/commit/0671ec15ff82d7fbe08b44b1ad7e1432d649c271


Extend v4 Active till v4.4 release.

Version: 4
Stage: LTS
Date: v4.4 release date
Notes: Several features were planned from v4.4, which were missed in the planning for #2. We're proposing to backtrack slightly to sneak them in under the wire.
Approval:

[ ] @bardiharborow
[ ] @xhmikosr
[ ] @johann-s
[ ] @mdo

Agreed, we need to release 4.4 and then adapt the dates.


Fix task bug

Can one of the admins verify this patch?

Current coverage is 94.91% (diff: 100%)

Merging #43 into master will increase coverage by 0.38%

@@             master        #43   diff @@
==========================================
  Files            30         30
  Lines           987        984     -3
  Methods           0          0
  Messages          0          0
  Branches         42         42
==========================================
+ Hits            933        934     +1
+ Misses           46         42     -4
  Partials          8          8

Powered by Codecov. Last update 09fc180...6e3e876


Add CONTRIBUTING.md

Please add a CONTRIBUTING.md see https://help.github.com/articles/setting-guidelines-for-repository-contributors/
It is not clear how to contribute to the project.

are there any questions you'd like answered in a CONTRIBUTING.md specifically?

Why yes @postmodern , there are a few. :grin:

Are contributions welcome?
What does the project want in Issues?
Are Pull Requests welcome?

I hadn't looked at ore in a few years. Then I saw it had awoken. Personally, I'm curious if the current contributors have a direction for the future of ore and a desire to accept third-party contributions.

from the Github help document:

For the repository owner, contribution guidelines are a way to quickly communicate how people should contribute.

For contributors, these guidelines help them verify that they're submitting well-formed pull requests and opening useful issues.

For both owners and contributors, contribution guidelines save time and hassle generated by improperly created pull requests or issues that have to be rejected and re-submitted.

Bugs, issues, improvements, feature requests, the usual. If your issue is with a specific template, then probably submit it to the appropriate GitHub repo. Pull Requests to fix minor things are welcomed. Although, I kind of want to rewrite portions of Ore to be more Object Orientated. Even thought about working on writing a generic template/generation library that Ore and other projects with generators could benefit from. Also thought of renaming the project (the whole ore != mine thing bothers me), or turning it into a rubygem plugin.
Despite those things, Ore satisfies my needs to generate new gems just how I want them initially setup.


line edits to docs in the language directory

Signed-off-by: Christine Rose<EMAIL_ADDRESS>

Thank you for that oldish PR! I’m porting your suggestions to the current state of the documentation. I left a question about the punctuation of a list. I’ve seen a couple of edits (in that PR or the others) changing something like “the contract of a function” into “a function’s contract”. For my education, is that just a difference of style or the first one is not grammatically correct? When the possessor is not a person, I never know which one to choose.

It is a matter of style, but I find that too many prepositional phrases make for a clunky, sometimes awkward sentence, so I try to minimise them. Feel free to ignore if you prefer or if it changes the meaning/makes it more confusing.

Thank you for the clarifications and for the update with the punctuation!


Allow callable types

This allows to define the type as callable that is invoked later. I see that it is partially implemented eg. https://github.com/webonyx/graphql-php/blob/master/src/Type/Definition/NonNull.php#L19

Coverage decreased (-0.07%) to 92.768% when pulling e2732ad486220f521ec4f76d26d00d1ce7d18925 on simPod:allow-callable-types into 94525c00252696b47673db1ca56fe0436c5949be on webonyx:master.

This functionality existed in one of the first versions of the library but was deprecated in favor of callables in fields (and yes, some traces of it still exist in the code). It caused more troubles (with maintenance and performance) than gave benefits. I don't see any reason why we should restore it.
You are the first who needs it in 2.5 years of library existence for a quite specific reason (trying to avoid registry/factory of types). But I'll keep this PR open to collect more evidence that this feature is needed by other library users as well.

I consider callables and their __invoke as something great when you want something to be lazy / performance sensitive.
Not sure why I'm first tho. I guess it's because people gave up on circular schemas (just a thought 🤔 )
I have a registry of types. Created a compiler pass that takes all type services from DI container and puts them into registry from where I can request them.
That doesn't solve the problem tho. When there's a type A in a registry that requires type B as a field's type and also there's type B that requires type A as its field's type, there's no other way how to create that. Type registry have no role it this case.
The only way to solve it (I might not have found 'em all) is to have eg. type A that has field with type B within a callback so it doesn't reference the type B directly. And vice versa.
I, for example, set the type B of A type field as:
Type::listOf(
        function () : Type {
            return $this->typeRegistry->getTypeB();
        }
)

So type B is resolved only when needed and the infinite circular reference cannot happen.

I've posted you a link on how to deal with circular types.
class User extends ObjectType
{
    public function __construct(TypeRegistry $types)
    {
        parent::__construct([
            'fields' => function() use ($types) {
                return [
                    'lastPost' => [
                        'type' => $types->post(),
                    ]
                ];
            }
        ]);
    }
}

class BlogPost extends ObjectType
{
    public function __construct(TypeRegistry $types)
    {
        parent::__construct([
            'fields' => function() use ($types) {
                return [
                    'author' => [
                        'type' => $types->user(),
                    ]
                ];
            }
        ]);
    }
}

Why doesn't this work for you?

Because TypeRegistry would look like this:

class TypeRegistry
{
    private $user;

    private $blogPost;

    public function user()
    {
        return $this->user ?: ($this->user = new User($this));
    }

    public function post()
    {
        return $this->blogPost ?: ($this->blogPost = new BlogPost($this));
    }
}

So then it goes like this

User is instantiated from TypeRegistry (new User called)
Withing User __construct $types->post() is called
In $types->post() new BlogPost is called
Withing BlogPost __construct $types->user() is called
It all repeats over and over

2. Withing User __construct $types->post() is called

No, it is not called within constructor, because fields is a closure. It is called later during execution. The whole point to make fields (and interfaces) a closure to break a cycle.

Oh, missed that one. Thanks, that works. Going to refactor my schema a bit but it seems to be a way and this PR is not necessary then I guess.


NRF_CLOCK tinyusb function signature change

Relatively recent tinyusb upstream changes seem to have broken the mynewt-core build?:
https://github.com/hathach/tinyusb/commit/4008f0d1e66dff6bb42a0fb0c682bf4bfe8e6c4a#commitcomment-36285182

Perhaps related/dependent on PR https://github.com/apache/mynewt-core/pull/2107?

Hi Roman,
Yes, that is why, we are working to merge that PR in order to fix this
issue, since nrfx 2.0.0 breaks some drivers and features.
Miguel Azevedo
On Thu, Dec 5, 2019 at 2:16 PM Roman Valls Guimera<EMAIL_ADDRESS>wrote:

Perhaps related/dependent on PR #2107
https://github.com/apache/mynewt-core/pull/2107?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/apache/mynewt-core/issues/2125?email_source=notifications&email_token=AA2QJ2RZJ23VNZMQKA45VGTQXEEKFA5CNFSM4JV2CCKKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEGA25YA#issuecomment-562147040,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AA2QJ2WOFUOURNWHNSOLB7TQXEEKFANCNFSM4JV2CCKA
.

Hi,    nrfx was updated to 2.1.0 in https://github.com/apache/mynewt-core/pull/2206
Please test if this works for you

@sjanc is the Homebrew formula updated accordingly to bump up to 2.1.0 (assuming that I installed mynewt with brew install mynewt-newt --HEAD?
$ brew reinstall mynewt-newt
==> Cloning https://github.com/apache/mynewt-newt.git
Updating /Users/romanvg/Library/Caches/Homebrew/mynewt-newt--git
==> Checking out branch master
Already on 'master'
Your branch is up to date with 'origin/master'.
HEAD is now at 9e6fc3d Emit `size` in mfg manifest target entries
==> ./build.sh
🍺  /usr/local/Cellar/mynewt-newt/HEAD-9e6fc3d: 3 files, 9.5MB, built in 15 seconds

$ newt version
Apache Newt 1.8.0-dev / 9e6fc3d-dirty / 2020-02-29_22:02

@sjanc is the Homebrew formula updated accordingly to bump up to 2.1.0 (assuming that I installed mynewt with brew install mynewt-newt --HEAD?
$ brew reinstall mynewt-newt
==> Cloning https://github.com/apache/mynewt-newt.git
Updating /Users/romanvg/Library/Caches/Homebrew/mynewt-newt--git
==> Checking out branch master
Already on 'master'
Your branch is up to date with 'origin/master'.
HEAD is now at 9e6fc3d Emit `size` in mfg manifest target entries
==> ./build.sh
🍺  /usr/local/Cellar/mynewt-newt/HEAD-9e6fc3d: 3 files, 9.5MB, built in 15 seconds

$ newt version
Apache Newt 1.8.0-dev / 9e6fc3d-dirty / 2020-02-29_22:02

I am getting the same compilation fail/results.

Upgrading newt only updates to tool, it does not automatically update the sources, you have to run newt upgrade to get the latest versions.

Fair point, but still seems stuck in 1.x versions?:
$ newt upgrade
Downloading repository tinyusb (commit: master) from https://github.com/hathach/tinyusb.git
Skipping "mynewt_nrf52840_mdk": already upgraded (0.0.0)
Making the following changes to the project:
    upgrade apache-mynewt-core (8615b0d01d1cfbf808554fef252ae85f56098475 --> 1.7.0)
    upgrade apache-mynewt-nimble (0.0.0 --> 1.2.0)
    upgrade mcuboot (0.0.0 --> 1.3.1)
apache-mynewt-core successfully upgraded to version 1.7.0
apache-mynewt-nimble successfully upgraded to version 1.2.0
mcuboot successfully upgraded to version 1.3.1
tinyusb successfully upgraded to version 0.0.0

Fair point, but still seems stuck in 1.x versions?:
$ newt upgrade
Downloading repository tinyusb (commit: master) from https://github.com/hathach/tinyusb.git
Skipping "mynewt_nrf52840_mdk": already upgraded (0.0.0)
Making the following changes to the project:
    upgrade apache-mynewt-core (8615b0d01d1cfbf808554fef252ae85f56098475 --> 1.7.0)
    upgrade apache-mynewt-nimble (0.0.0 --> 1.2.0)
    upgrade mcuboot (0.0.0 --> 1.3.1)
apache-mynewt-core successfully upgraded to version 1.7.0
apache-mynewt-nimble successfully upgraded to version 1.2.0
mcuboot successfully upgraded to version 1.3.1
tinyusb successfully upgraded to version 0.0.0

You have to update your project.yml to use vers: 0-dev instead of vers: 1-latest

Thanks, makes sense, here's the result:
$ newt upgrade
Skipping "apache-mynewt-mcumgr": already upgraded (0.0.0)
Skipping "mynewt_nrf52840_mdk": already upgraded (0.0.0)
Skipping "tinyusb": already upgraded (0.0.0)
Making the following changes to the project:
    upgrade apache-mynewt-core (1.7.0 --> 0.0.0)
    upgrade apache-mynewt-nimble (1.2.0 --> 0.0.0)
    upgrade mcuboot (1.3.1 --> 0.0.0)
apache-mynewt-core successfully upgraded to version 0.0.0
apache-mynewt-nimble successfully upgraded to version 0.0.0
mcuboot successfully upgraded to version 0.0.0

$  newt build nrf52_mdp_link
WARNING: apache-mynewt-core: Repo version missing from compatibility map
Building target targets/nrf52_mdp_link
Compiling repos/mynewt_nrf52840_mdk/hw/bsp/nrf52840_mdk/src/sbrk.c
Compiling repos/mynewt_nrf52840_mdk/hw/bsp/nrf52840_mdk/src/hal_bsp.c
Compiling apps/mdp-link/src/main.c
Compiling apps/mdp-link/src/usb_descriptors.c
Assembling repos/mynewt_nrf52840_mdk/hw/bsp/nrf52840_mdk/src/arch/cortex_m4/gcc_startup_nrf52840.s
Assembling repos/mynewt_nrf52840_mdk/hw/bsp/nrf52840_mdk/src/arch/cortex_m4/gcc_startup_nrf52_split.s
Error: apps/mdp-link/src/main.c: In function 'usb_hardware_init':
apps/mdp-link/src/main.c:230:9: error: passing argument 1 of 'nrf_power_int_enable' makes pointer from integer without a cast [-Werror=int-conversion]
         NRF_POWER_INT_USBDETECTED_MASK |
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
         NRF_POWER_INT_USBREMOVED_MASK  |
         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
         NRF_POWER_INT_USBPWRRDY_MASK);
         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from apps/mdp-link/src/main.c:36:
repos/apache-mynewt-core/hw/mcu/nordic/src/ext/nrfx/hal/nrf_power.h:899:62: note: expected 'NRF_POWER_Type *' {aka 'struct <anonymous> *'} but argument is of type 'int'
 NRF_STATIC_INLINE void nrf_power_int_enable(NRF_POWER_Type * p_reg, uint32_t mask)
                                             ~~~~~~~~~~~~~~~~~^~~~~
apps/mdp-link/src/main.c:229:3: error: too few arguments to function 'nrf_power_int_enable'
   nrf_power_int_enable(
   ^~~~~~~~~~~~~~~~~~~~
In file included from apps/mdp-link/src/main.c:36:
repos/apache-mynewt-core/hw/mcu/nordic/src/ext/nrfx/hal/nrf_power.h:899:24: note: declared here
 NRF_STATIC_INLINE void nrf_power_int_enable(NRF_POWER_Type * p_reg, uint32_t mask)
                        ^~~~~~~~~~~~~~~~~~~~
apps/mdp-link/src/main.c: In function 'POWER_CLOCK_IRQHandler':
apps/mdp-link/src/main.c:254:22: error: too few arguments to function 'nrf_power_int_enable_get'
   uint32_t enabled = nrf_power_int_enable_get();
                      ^~~~~~~~~~~~~~~~~~~~~~~~
In file included from apps/mdp-link/src/main.c:36:
repos/apache-mynewt-core/hw/mcu/nordic/src/ext/nrfx/hal/nrf_power.h:909:28: note: declared here
 NRF_STATIC_INLINE uint32_t nrf_power_int_enable_get(NRF_POWER_Type const * p_reg)
                            ^~~~~~~~~~~~~~~~~~~~~~~~
apps/mdp-link/src/main.c:257:37: error: passing argument 1 of 'nrf_power_event_get_and_clear' makes pointer from integer without a cast [-Werror=int-conversion]
       nrf_power_event_get_and_clear(NRF_POWER_EVENT_USBDETECTED))
                                     ^~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from apps/mdp-link/src/main.c:36:
repos/apache-mynewt-core/hw/mcu/nordic/src/ext/nrfx/hal/nrf_power.h:882:72: note: expected 'NRF_POWER_Type *' {aka 'struct <anonymous> *'} but argument is of type 'int'
 NRF_STATIC_INLINE bool nrf_power_event_get_and_clear(NRF_POWER_Type *  p_reg,
                                                      ~~~~~~~~~~~~~~~~~~^~~~~
apps/mdp-link/src/main.c:257:7: error: too few arguments to function 'nrf_power_event_get_and_clear'
       nrf_power_event_get_and_clear(NRF_POWER_EVENT_USBDETECTED))
       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from apps/mdp-link/src/main.c:36:
repos/apache-mynewt-core/hw/mcu/nordic/src/ext/nrfx/hal/nrf_power.h:882:24: note: declared here
 NRF_STATIC_INLINE bool nrf_power_event_get_and_clear(NRF_POWER_Type *  p_reg,
                        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
apps/mdp-link/src/main.c:263:37: error: passing argument 1 of 'nrf_power_event_get_and_clear' makes pointer from integer without a cast [-Werror=int-conversion]
       nrf_power_event_get_and_clear(NRF_POWER_EVENT_USBREMOVED))
                                     ^~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from apps/mdp-link/src/main.c:36:
repos/apache-mynewt-core/hw/mcu/nordic/src/ext/nrfx/hal/nrf_power.h:882:72: note: expected 'NRF_POWER_Type *' {aka 'struct <anonymous> *'} but argument is of type 'int'
 NRF_STATIC_INLINE bool nrf_power_event_get_and_clear(NRF_POWER_Type *  p_reg,
                                                      ~~~~~~~~~~~~~~~~~~^~~~~
apps/mdp-link/src/main.c:263:7: error: too few arguments to function 'nrf_power_event_get_and_clear'
       nrf_power_event_get_and_clear(NRF_POWER_EVENT_USBREMOVED))
       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from apps/mdp-link/src/main.c:36:
repos/apache-mynewt-core/hw/mcu/nordic/src/ext/nrfx/hal/nrf_power.h:882:24: note: declared here
 NRF_STATIC_INLINE bool nrf_power_event_get_and_clear(NRF_POWER_Type *  p_reg,
                        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
apps/mdp-link/src/main.c:269:37: error: passing argument 1 of 'nrf_power_event_get_and_clear' makes pointer from integer without a cast [-Werror=int-conversion]
       nrf_power_event_get_and_clear(NRF_POWER_EVENT_USBPWRRDY))
                                     ^~~~~~~~~~~~~~~~~~~~~~~~~
In file included from apps/mdp-link/src/main.c:36:
repos/apache-mynewt-core/hw/mcu/nordic/src/ext/nrfx/hal/nrf_power.h:882:72: note: expected 'NRF_POWER_Type *' {aka 'struct <anonymous> *'} but argument is of type 'int'
 NRF_STATIC_INLINE bool nrf_power_event_get_and_clear(NRF_POWER_Type *  p_reg,
                                                      ~~~~~~~~~~~~~~~~~~^~~~~
apps/mdp-link/src/main.c:269:7: error: too few arguments to function 'nrf_power_event_get_and_clear'
       nrf_power_event_get_and_clear(NRF_POWER_EVENT_USBPWRRDY))
       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from apps/mdp-link/src/main.c:36:
repos/apache-mynewt-core/hw/mcu/nordic/src/ext/nrfx/hal/nrf_power.h:882:24: note: declared here
 NRF_STATIC_INLINE bool nrf_power_event_get_and_clear(NRF_POWER_Type *  p_reg,
                        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
cc1: all warnings being treated as errors


How about combining object detection and image classification in a single network?

In addition to do object detection using SqueezeDet, I tried adding another branch to do image classification, as follows,
#########################################################
# One more fire module that is not trained before
fire10_2 = self._fire_layer(
'fire10_2', fire9, s1x1=96, e1x1=384, e3x3=384, freeze=False)
pool10_2 = self._pooling_layer('pool10_2', fire10_2, size=3, stride=2, padding='SAME')
dropout10_2 = tf.nn.dropout(pool10_2, self.keep_prob, name='drop10_2')
num_output_2 = mc.GLOBAL_CLASSES # number of classes in image classification
conv11_2 = self._conv_layer(
    'conv11_2', dropout10_2, filters=num_output_2, size=3, stride=1,
    padding='SAME', freeze=False, relu=True, stddev=0.0001)
self.preds_2 = self._pooling_layer_avg('pool11_2', conv11_2, conv11_2.get_shape().as_list()[1], conv11_2.get_shape().as_list()[2])
#########################################################

However, when training, the branch for object detection converges, but the one for image classification diverges terribly.
My friends, what's your opinion?

What do you get out of doing detection+classification in a single net? The
detection net will already classify the objects in the picture...
On Fri, 22 Sep 2017 at 10:16 AM, centosrhel<EMAIL_ADDRESS>wrote:

In addition to do object detection using SqueezeDet, I tried adding
another branch to do image classification, as follows,
#########################################################
One more fire module that is not trained before
fire10_2 = self._fire_layer(
'fire10_2', fire9, s1x1=96, e1x1=384, e3x3=384, freeze=False)
pool10_2 = self._pooling_layer('pool10_2', fire10_2, size=3, stride=2,
padding='SAME')
dropout10_2 = tf.nn.dropout(pool10_2, self.keep_prob, name='drop10_2')
num_output_2 = mc.GLOBAL_CLASSES # number of classes in image classification
conv11_2 = self._conv_layer(
'conv11_2', dropout10_2, filters=num_output_2, size=3, stride=1,
padding='SAME', freeze=False, relu=True, stddev=0.0001)
self.preds_2 = self._pooling_layer_avg('pool11_2', conv11_2, conv11_2.get_shape().as_list()[1], conv11_2.get_shape().as_list()[2])
#########################################################
However, when training, the branch for object detection converges, but the
one for image classification diverges terribly.
My friends, what's your opinion?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/BichenWuUCB/squeezeDet/issues/79, or mute the thread
https://github.com/notifications/unsubscribe-auth/AN_wJtl7SycJDYH61w0IpRaAp8sVurbuks5skxiVgaJpZM4PgJGR
.


Options: Document data format

Looking through LeanData.cs, it appears that the on disk format for options should be pretty simple. I'm interested in adding daily options quotes from somewhere that isn't AlgoSeek and it appears that it should be a matter of:

To get the price of the underlying, putting a ZIP file at Data/equity/usa/daily/spx.zip containing a file named spx.csv, containing lines of the form YYYYMMDD HH:MM,open_price,high_price,low_price,close_price,volume. This actually seems to work fine, I have a test algorithm and this works without issue.

Putting another ZIP file at Data/option/usa/daily/spx_quote_american.zip containing files of the form spx_quote_american_<call/put>_<strike price x 10000>_<expiry YYYYMMDD>.csv, containing lines of the form YYYYMMDD HH:MM,ask_open_price,ask_high_price,ask_low_price,ask_close_price,ask_volume,bid_open_price,bid_high_price,bid_low_price,bid_close_price,bid_volume

That should as far as I can tell, be it. It is however not, as my test algorithm, though receiving equity prices without issues, is not receiving a single option chain or displaying any indication of one being loaded. It is for some reason attempting to load SPY (which I don't have) and minute resolution trade data (which I also don't have, or want, or need).

Ok, this is just mean :)
Has no one ever heard of vectors?
Also, Twitter has done some work on smooth derivatives of code. Refer : Torch code
Don't know integration ability ( pun intended ). That would be nice indeed!
Good luck and thank you for open source this!

@bobobo1618 -  Do you see any errors in log.txt regarding reading the data from file? You may also try setting a breakpoint in DefaultDataProvider.cs and ZipDataCacheProvider.cs to make sure that the data is actually being read off disk correctly. Is your code making it to the QuoteBar.Reader() method successfully?

Do you see any errors in log.txt

None regarding options data, only a bunch about trade data (seems to be related to its odd insistence on reading SPY, despite my never asking it to).

You may also try setting a breakpoint

I've set breakpoints on both DefaultDataProvider.cs and ZipDataCacheProvider.cs and I see the calls for the SPX equity data and for a bunch of SPY stuff but there are definitely no calls for SPX.

Is your code making it to the QuoteBar.Reader() method successfully?

QuoteBar.Reader() is never called.
I've also added the algorithm I'm testing here, my log file here and a list of the files in my Data folder here.

We've never done daily options backtesting yet; so far it all built to work around minute. There are probably many things to think about including the universe updating which is configured to run each minute at the moment. For now I think this "isn't supported" out of the box in LEAN; but if you're willing to keep tinkering you could probably figure it out.

@jaredbroad I've created a new issue for that but I'll add that I'm able to run an options strategy with daily resolution on QuantConnect.com so it seems odd that I can't run one locally.

I eventually seem to have found my problem: I used american in the filename but european in the Zip entries.
During my debugging, I found a new fun problem:
When I look at the value of symbol.Underlying in the debugger here, it shows symbol.Underlying.HasUnderlying = false and symbol.Underlying.Value = "". As such, it attempts to open ../../../Data/option/usa/daily/_quote_american.zip.
The same is the reason for the The specified file was not found: ../../../Data/equity/usa/daily/.zip line in the logs. The SPX equity symbol doesn't have a value present when it reaches here, so empty string is formatted in.
Now I just get OOM killed.

We were planning on making a line somewhere which specifically substituted
SPX underlying request for SPY as a its 99% parallel and easily available
(minutely SPX index data is expensive). We hadn't planned where this would
be yet.
Daily & minute data are different in the mechanics of LEAN IMHO though
(e.g. the strike filters we scan will only be updated daily). I'm not sure
it will even work without careful review as mentioned before. We do not
support daily in the web IDE // there is literally no daily data there :)
If you run a daily backtest you may be simply looking at the underlying
asset.
JB
On Thu, May 25, 2017 at 12:41 AM, bobobo1618<EMAIL_ADDRESS>wrote:

I eventually seem to have found my problem: I used american in the
filename but european in the Zip entries.
During my debugging, I found a new fun problem:
When I look at the value of symbol.Underlying in the debugger here
https://github.com/QuantConnect/Lean/blob/c8e756200c8f87f68daa63142a424f968765ee97/Common/Util/LeanData.cs#L495,
it shows symbol.Underlying.HasUnderlying = false and symbol.Underlying.Value
= "". As such, it attempts to open ../../../Data/option/usa/
daily/_quote_american.zip.
The same is the reason for the The specified file was not found:
../../../Data/equity/usa/daily/.zip line in the logs. The SPX equity
symbol doesn't have a value present when it reaches here
https://github.com/QuantConnect/Lean/blob/c8e756200c8f87f68daa63142a424f968765ee97/Common/Util/LeanData.cs#L495,
so empty string is formatted in.
Now I just get OOM killed.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/QuantConnect/Lean/issues/939#issuecomment-303920690,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACI6mRth_5NLSB6leU3r3XnWU74p7xMTks5r9QZ5gaJpZM4NkfYO
.

--
Jared Broad

This is fairly confusing for folks new to the platform, mainly because the data library documentation page claims support for Daily, Hourly and Minute resolutions. We spent many hours spinning on this, assuming we were doing something wrong. :)
I suggest a you note this in the documentation and that you correct the data page (https://www.quantconnect.com/data#options/usa/price).


Add a way to get the minified bundle js

Hi,
I am trying to get the minified bundle js (the final result js) using:
` ....
Private Sub IBundleTransform_Process(context As BundleContext, response As BundleResponse) Implements IBundleTransform.Process
Dim current As IReactEnvironment = ReactEnvironment.Current
response.Content = current.Babel.Transform(response.Content, "unknown")
.....`
But current.Babel.Transform() always returns the original unmodified JS, is there a way to get the actual result of this call?

Never mind, I wasn't understanding how babel.Transform() works


How to run on MOT17/20 test split

Hi! I wanted to reproduce table 4 and table 5 of the manuscript however the current scripts, e.g. data/tools/convert_mot20_to_coco.py, dont produce test samples to run the validation.

I see MOT test gt is not available and to be able to get test results it should be uploaded! Thanks anyways.
The validation results of boosttrack+ are considerably lower than test results MOT20 HOTA ~62, compared to 66 for test split report in the paper. and I guess that is normal


Tracking: Lists Feature MVP

List creation flow MVP

[ ] https://github.com/open-sauced/app/issues/1672
[ ] https://github.com/open-sauced/app/issues/1671
[ ] https://github.com/open-sauced/app/issues/1670
[ ] https://github.com/open-sauced/app/issues/1669
[ ] https://github.com/open-sauced/app/issues/1668
[ ] https://github.com/open-sauced/app/issues/1671

List Page MVP (only overview, activity and contributors table tabs)

[ ] https://github.com/open-sauced/app/pull/1569
[ ] https://github.com/open-sauced/app/pull/1594

@isabensusan I updated the OP (original post) to include all the open issues. Creating a video to update the community in the discord and thought it might be easier to see what is still in progress and what is merged.


Post URL of model results from Quick TUV-x calculator

User will see a directory of NetCDF files to download:
modeling2:/home/drews/TUV-x/stage/20230410-144427--029087> ls -lh
total 129M
lrwxrwxrwx 1 drews support  45 Apr 10 14:44 data -> /home/drews/TUV-x/tuvx-build/tuv-x/build/data/
-rw-r--r-- 1 drews support 64M Apr 10 14:44 dose_rates.nc
-rw-r--r-- 1 drews support 65M Apr 10 14:44 photolysis_rate_constants.nc
-rw-r--r-- 1 drews support 67K Apr 10 14:44 tuvx_config.json
modeling2:/home/drews/TUV-x/stage/20230410-144427--029087>

I need Garth to set up for me a mount point (or something) by which web3 can access modeling2:
web3:/net> ls -l
total 0
dr-xr-xr-x 11 root root 0 Mar  9 04:20 modeling1/
dr-xr-xr-x  6 root root 0 Mar  9 04:45 nitrogen.acom.ucar.edu/
dr-xr-xr-x  3 root root 0 Mar 11 12:46 otserver/
web3:/net>


Cannot pick BatchTransfer  resources in blueprint namespace to reconcile

I am struggling trying to enable the implicit copy in a multi-cluster environment. By specifying the workload and asset in different cluster with different region, now the read module and copy module got deployed in different cluster and BatchTransfer resource is created. But unfortunately the data-movement-operator dos not pick the resource to reconcile at all.
After some investigation I found my BatchTransfer is created in the fybrik-blueprints namespace as below,
apiVersion: v1
items:
- apiVersion: motion.fybrik.io/v1alpha1
  kind: BatchTransfer
  metadata:
....................
    name: my-notebook-fybrik-notebook-sample-implicit-cop-a9308
    namespace: fybrik-blueprints
.....................
  spec:
    destination:
      description: s3://my-notebook-fybrik-notebook-sample12b73371af/Synthetic Financial
        Datasets For Fraud Detection4afecbad70

But the data-movement-operator is deployed in fybrik-system namespace and checked there's no appropriate cluster role for manager.
I think we need create correct RBAC rules for data-movement-operator, especially after we make the module namespace configurable now.

Seems there's a role created and bind in the  fybrik-blueprints namespace. Need investigate more.
(base) ➜  ~ k get rolebinding -n fybrik-blueprints
NAME                               ROLE                                      AGE
batchtransfer-rb                   Role/batchtransfer-editor                 26h
data-movement-operator-motion-rb   Role/data-movement-operator-motion-role   26h
fybrik-blueprints-rb               Role/fybrik-blueprints-role               3d
streamtransfer-rb                  Role/streamtransfer-editor                26h

Do we have guide on how to use this module? I still cannot make my implicit copy module work...

Strangely when I copy that BatchTransfer definition and created manually then it's OK and job is created. Need more investigation.

@revit13 I see you are expert in this repo, do you help some insight or instructions on this issue?

@pugangxa thanks, I'm new to this repo... :-)  @shlomitk1 helped me to debug a problem which seems to be similar... we continue looking into it today and update....

@pugangxa thanks, I'm new to this repo... :-) @shlomitk1 helped me to debug a problem which seems to be similar... we continue looking into it today and update....

Thanks @revit13 for your information. To reproduce this issue, just install data-movement-operator as in README:
export DOCKER_HOSTNAME=localhost:5000
export DOCKER_NAMESPACE=fybrik-system
export VALUES_FILE=charts/data-movement-operator/integration-tests.values.yaml
make docker-build docker-push
make deploy

Then create any batchtransfer resource manually, then you can see in log there's information that webhook get called, but the reconcile never being triggered.
If install with manager.enabled to be false and debug locally then it's fine.

@pugangxa Local setup has the "all-mighty" permissions of the cluster admin, it is not using a service account "dmo-manager".
What is weird, that roles seem to be correct, i.e. kubectl auth can-i list batchtransfer --as=system:serviceaccount:fybrik-system:dmp-manager  returns yes

@pugangxa can you please check that https://github.com/fybrik/data-movement-operator/pull/30 works for you? Thanks

@pugangxa can you please check that #30 works for you? Thanks

Thanks very much and it works. BTW, I just did a slight change to make LeaderElectionID to be data-movement-operator-leader-election.
Thanks all for your help! @revit13 @shlomitk1

Great! Thanks for testing. I'll do the change in the name as well. Thanks to @roee88 for his help as well!


Update whats-new-in-version-1802.md

A couple of typos.

D'oh! Niall beat you to this change with PR #442. Sorry :( Thanks for the contribution, look forward to your next!


Added a little more extensive tests for the new queries browser.

The CI did not break so I would merge and squash this :)


fix local demo

readme.md refers to many demos on jsfiddle.net. They are all dead, because they try to load js over insecure HTTP. This PR places the demos under /demo folder, and changed to refer to the local copy of roofpig_and_three.min.js at the root of the repo. So it is easier to view the behaviour against different versions of js under test.
Fixes #18

Sorry my reviews are slow to arrive. I appreciate the PRs a lot, I've just been having a busy time!
Should get to it within 1-2 weeks.


Timeout option

Closes #1
To control the timeout, I changed the sleep strategy used in the generator to a new one that uses a promise over listeners on data, error, and end/close.

The listener promise, to avoid an event listener memory leak, removes the error and end listeners once the data event is called. Also, this promise always returns undefined or throws an error;
The timeout awaiter returns an internal symbol when the deadline is reached;
A race is made over those promises if the timeout is activated, if it's not, the race runs only over the listener promise;
If the timeout is activated, every time a value is obtained from the data event, the timeout deadline is updated;

The timeout is controlled in the wrapper returned by the function in timeout.ts. It uses the sleep method to wait until the moment the timeout must be checked out. When this moment comes, the deadline may be updated because some value was obtained from the data event, in that case, a new sleep is executed until the next deadline moment.

Hey, thank you for taking the time on this. In terms of the requirements, timeout should be the max time in-between each event (My bad I did not state them!). I don't think we should throw Event timed out until at least 1 event has yield What do you think?

There is an important bit of code that you removed,

/* We do not want to block the process!
    This call allows other processes
    a chance to execute.
*/
await sleep(0);

@danstarns I think it's fair enough! I'll change the code so it complies with the in-between requirement :)

@danstarns done! I created an option inBetweenTimeout with default value as true. I think that maybe there some case of use where one need for the timeout to be thrown even before the first event.

@danstarns about the sleep(0), I changed it with a new strategy with a promise that resolves when an event is emitted, so I can race the timeout with it.
The event emission may be synchronous, so I still keep the buffer array you created to get those values and that promise is only used when the array is empty, no error has been thrown and the event emitter is still active

@danstarns I apologize. I created a new test case to guarantee that the for await ran over the generated iterable is nonblocking and I could see the importance of this piece of code. I put it again in the code with the new case

@Farenheith I like the idea of making it configurable with inBetweenTimeout although I don't believe its working as intended.

With the below code I expect it to throw Event timed out

const forEmitOf = require("./dist");
const fs = require("fs");
const path = require("path");
const sleep = require("util").promisify(setTimeout);

async function main() {
  const readStream = fs.createReadStream(
    path.join(__dirname, "./package.json")
  );

  const iterator = forEmitOf(readStream, {
    transform: (buff) => buff.toString(),
    timeout: 1000, // TIMEOUT less than 10 seconds
    inBetweenTimeout: false,
  });

  await sleep(10000); // 10 seconds longer than TIMEOUT

  for await (const chunk of iterator) {
    console.log(chunk);
  }
}

main();

I'm thinking we could have 2 parameters & this may make it easier to reason about.

We wont need timeout with this

inBetweenTimeout => max time allowed in between each event
firstEventTimeout => max time allowed from construction() to first event.

interface Options<T = any> {
  inBetweenTimeout?: number;
  firstEventTimeout?: number;
  ...
}

The problem is that the timeout is applied, by the implementation, to the event emission, not the iteration. The event emission starts at the first chance node has to fulfill the fs promises. I put some console logs to illustrate it here, on index.ts:
  const eventListener = <T>(event: T) => {
    console.log("chunk received");
    return events.push(event);
  };
  const endListener = () => {
    console.log("ended");
    active = false;
  };

Here is the code I used to test:
const forEmitOf = require("./dist");
const fs = require("fs");
const path = require("path");
const sleep = require("util").promisify(setTimeout);

async function main() {
  const readStream = fs.createReadStream(
    path.join(__dirname, "./package.json")
  );

  const iterator = forEmitOf(readStream, {
    transform: (buff) => buff.toString(),
    timeout: 1000, // TIMEOUT less than 10 seconds
    inBetweenTimeout: false,
  });
  console.log("before sleep");

  await sleep(2000); // 10 seconds longer than TIMEOUT

  console.log("before for");
  for await (const chunk of iterator) {
    console.log(chunk.length);
  }
  console.log("after for");
}

main();

The console log result looked like this:
before sleep
chunk received
ended
ended
before for
1779
after for

See that, chunk received was print before the start of the iteration, and it was pushed to the events array. Even before the iteration, the emitter ended, so, when the iteration started, all it had to do was to iterate over the events array.
Also, even if you have an emitter that emits more than one time and if you do something like this:
  for await (const chunk of iterator) {
   await sleep(2000);
    console.log(chunk.length);
  }

Still, the timeout will not be thrown, unless the event emission takes more than 1 second.
Don't you think this is a desirable behavior? Think about a situation where the user of this library may do some slow operations, which will take longer than the timeout. In this situation, I think the user will expect that the timeout defines a deadline to the event emission, but his code is not limited by it.
About the changes in the timeout parameters, I think your proposal makes much sense as it brings more flexibility for the lib user! I'll make the changes and do the PR :).

@danstarns I changed the timeout options as you suggested :)
Let me know what you think about the timeout behavior in regard of my last comment.
Thanks for yours considerations!

@danstarns just one more important thing. There is another piece of code I removed as I found a situation which could generates an error. This one:
      if (!event) {
        continue;
      }

The problem I found is that, if an event emitter e emits an event with a falsy argument, like undefined, 0 or false, this value would not be yielded. Example:
myEmitter.emit('data', false);
myEmitter.emit('data', 0);
myEmitter.emit('data', '');
myEmitter.emit('data', 'not empty');

Before removing the quoted if, this could would only yield the last value. As I changed it, I think I should create a test case to validate such behavior. I'll create it

@Farenheith Hey! Thanks for changing it soo quickly 🐱👤 I'm still struggling with firstEventTimeout from your comment It seems to me that we can't guarantee the execution of firstEventTimeout. Could it cause more issues than it's trying to solve?
I've been playing around with firstEventTimeout and I would expect the below code to throw Event timed out.
const forEmitOf = require("./dist");
const { EventEmitter } = require("events");

async function main() {
  const emitter = new EventEmitter();

  const iterator = forEmitOf(emitter, {
    firstEventTimeout: 1000,
  });

  setTimeout(() => {
    emitter.emit("data", "hi");
  }, 2000);

  for await (const msg of iterator) {
    console.log(msg);
  }
}

main();

In response to your bug you found about falsy values with the check maybe we can check for 'nullish' values null & undefined and allow all others. I think there is a new language feature that could help here 🤔

@danstarns I'm actually with time to spend due to this pandemic scenario lol. Such boredom, dude...
But you got a bug in my implementation, but it's actually simple to fix. I was validating firstEventTimeout only when inBetweenTimeout was informed, my bad.
I fixed it and created a new test case to cover it. Take a look when you can in the last commit!
Also, I'm creating a library more for fun that works quite like rxjs, but over Iterables and AsyncIterable instead of Observables. I'm quite enthusiastic about it and I want to make it works also with EventEmitters, and I loved the solution you find with for-emit-of, it fits very well in what I was looking for

@Farenheith Nice, Glad it works :) Send a link I would love to check it out!
Your commit fixed my issue after I remembered to run npm run build 😂

Getting undesired behavior, expected/actual

const forEmitOf = require("./dist");
const { EventEmitter } = require("events");

async function main() {
  const emitter = new EventEmitter();

  const iterator = forEmitOf(emitter, {
    firstEventTimeout: 1000,
  });

  setTimeout(() => {
    emitter.emit("data", "hi");
  }, 100);

  for await (const msg of iterator) {
    console.log(msg);
  }
}

main();

expected
$ node test
hi

actual
$ node test
hi
(node:25028) UnhandledPromiseRejectionWarning: Error: Event timed out

It's this project I talked about.
I just sent a PR fixing this scenario you found. Thank you for the care you're having with my implementation as some of these scenarios I was really not covering it. I have a special desire for this timeout to work for the first emission, as you may guess lol.

@Farenheith Nice it's looking good, I can see you have put a lot of time into it

199 commits ahead of kataik:master 😲

I've tested all your updates & I feel they are all working as intended. I'm going to merge this & commit some examples plus additions to the README.  v1.1.0 will be released very shortly, thank you for your help 😃

The number of commits is not that much lol... I just a compulsive commuter lol

@Farenheith me too :) Its the best way :)


[INSTALL]: fortranformat Python package

Package name
fortranformat (py-fortranformat)
Package version/tag
tbd
Build options
tbd
Installation timeframe
spack-stack-1.8.0
Other information
See https://github.com/JCSDA-internal/pyiri-jedi/issues/22 if you have access. In short, this package is required for pyiri-jedi (high altitude physics module for NEPTUNE with JEDI DA).

@eap I believe you want this in spack-stack-1.8.0, or?


[WIP] Ethrpc api compatibility with web3

Codecov Report

Merging #34 into master will decrease coverage by 0.15%.
The diff coverage is 0%.

@@            Coverage Diff             @@
##           master      #34      +/-   ##
==========================================
- Coverage   63.43%   63.27%   -0.16%
==========================================
  Files          20       20
  Lines         402      403       +1
  Branches       62       62
==========================================
  Hits          255      255
- Misses        132      133       +1
  Partials       15       15

Impacted Files
Coverage Δ

src/rpc/ethrpc.js
42.42% <0%> (-1.33%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3e4cfa9...078568f. Read the comment docs.


server BUGFIX fix partial processing of tcp params changes

When deleting an enpoint, neither nc_server_endpt_set_keepalives nor nc_server_ch_client_endpt_set_keepalives are called. rc remains with its current value which is necessarily SR_ERR_NOT_FOUND (meaning "end of changes").
This causes np2srv_tcp_keepalives to return SR_ERR_INTERNAL which causes np2srv_endpt_tcp_params_cb to stop processing the changes and return an error. No error message is printed.
Reset the value of rc to fix the issue.
If one of nc_server_ch_client_endpt_set_keepalives or nc_server_endpt_set_keepalives actually failed, display an explicit error message.

Hi,
right, missed this one, thanks.
Regards,
Michal