Feature Request: Support Mismatch Retrieval Via FFI

When writing the pact-net FFI integration, it's currently not possible to get any logging out of the verifier. The consumer side has had features added so that you can redirect logging to an in-memory sink and then retrieve the logs afterwards for inclusion in test output, but the verifier still logs directly to stdout as if it was invoked via a CLI.
The feature request is to extend the consumer logging support into the provider tests so that the verifier output (with mismatches etc) can also be included in test output.
This is currently a blocker for the new pact-net v4.0.0 release using the Rust core library instead of the Ruby integration.

Pact-js needs the output on standard out, so supporting both would be ideal.
Is retrieving the logs the right way to implement this, though? It might be better for integration with reporting tools if the verifier result returned was structured.

Yeah if it could somehow return the individual checks so you can enumerate and run them individually (with proper output) that would be even more amazing. I'd have to look into how to do that with popular test frameworks in .Net because I've only done that sort of thing in xUnit.
At the moment in pact-net you just get one test per pact file which is very coarse grained.
The key thing is just getting those mismatches back though so you can include them when the provider test fails. At the moment it fails but you have no idea why because there's no output at all.

We have the same in pact-js, one test runs one pact.
I haven’t looked into it very seriously yet, but I think there are potentially options like talking to the reporting frameworks directly, or autogenerating a set of tests to fool the test framework into printing the output for us. Or, using the output modules from the test framework so that it looks like it’s a first class run (the jest test framework, for example, has a separate module that can be used to print diffs that look like they came from jest). I don’t know if a similar approach could work for .NET.
Anyway, as you said, that’s not exactly the core issue here :)

Oh, so the log level is different for the verifier you're right. I don't think the FFI log methods fix this now that I think about it.
On mobile, but from memory it's controlled by the flag --loglevel.

Yeah the log level controls that, but it still only prints to stdout (which xUnit doesn't capture). I'm pretty sure it also prints all the colour escape sequences and stuff as well, so even if you could capture it I think it would look garbled.
What we really need is the log approach used for the mock server to be unified with the verifier now that they're in a common FFI library.

Ah I forgot about that limitation in xUnit.
Agreed, I'll see if that's something that can be easily navigated but at some point we'll need to change the verifier interface as discussed above.

Is there any update on this? We are using .Net and we're looking at moving from a custom NuGet package we have been maintaining to v4.0.0, but the lack of output is a blocker for us. I am unable to troubleshoot why tests are failing. (The tests should pass, so I'm guessing it is a config issue, but no way of knowing right now).

FWIW, after some further testing, I am able to get output from my consumer creation tests, but not my verification tests. The NativeInterop.Very method does not provide any output details.

@martinisaksenprogleasing you should be able to get output if you run from your terminal in the meantime. Because it logs to stdout. I've tested that whilst running https://github.com/DiUS/pact-workshop-dotnet-core-v3/ on my Mac. I assume it's the same.
It looks like 42be9eb8 landed last week. @uglyog do we need to migrate to the newer provider verifier methods for this to work, or is the second log extraction function (pactffi_verifier_logs_for_provider) that just uses the name of the provider sufficient?

I am on a Windows and I am not getting the output when running the Provider tests using that repo. (I have the preview version of .Net 6 installed. But I doubt that is the problem)
`PS C:\Code\pact-workshop-dotnet-core-v3\Provider> dotnet test
Determining projects to restore...
All projects are up-to-date for restore.
You are using a preview version of .NET. See: https://aka.ms/dotnet-core-preview
You are using a preview version of .NET. See: https://aka.ms/dotnet-core-preview
provider -> C:\Code\pact-workshop-dotnet-core-v3\Provider\src\bin\Debug\netcoreapp3.1\provider.dll
tests -> C:\Code\pact-workshop-dotnet-core-v3\Provider\tests\bin\Debug\net5.0\tests.dll
Test run for C:\Code\pact-workshop-dotnet-core-v3\Provider\tests\bin\Debug\net5.0\tests.dll (.NETCoreApp,Version=v5.0)
Microsoft (R) Test Execution Command Line Tool Version 17.0.0-preview-20210817-02
Copyright (c) Microsoft Corporation.  All rights reserved.
Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
[xUnit.net 00:00:01.84]     tests.ProductTest.EnsureProviderApiHonoursPactWithConsumer [FAIL]
Failed tests.ProductTest.EnsureProviderApiHonoursPactWithConsumer [468 ms]
Error Message:
PactNet.PactFailureException : The verification process failed, see output for errors
Stack Trace:
at PactNet.Native.NativePactVerifier.Verify(String args) in /Users/erikdanielsen/work/dius/pact-net/src/PactNet.Native/NativePactVerifier.cs:line 34
at PactNet.Native.PactVerifier.Verify() in /Users/erikdanielsen/work/dius/pact-net/src/PactNet.Native/PactVerifier.cs:line 240
at tests.ProductTest.EnsureProviderApiHonoursPactWithConsumer() in C:\Code\pact-workshop-dotnet-core-v3\Provider\tests\ProductTest.cs:line 46
Standard Output Messages:
Invoking the pact verifier with args:
--file
C:\Code\pact-workshop-dotnet-core-v3\pacts\ApiClient-ProductService.json
--state-change-url
http://<IP_ADDRESS>:9001/provider-states
--provider-name
ProductService
--hostname
<IP_ADDRESS>
--port
9001
--filter-consumer
ApiClient
--loglevel
trace
Failed!  - Failed:     1, Passed:     0, Skipped:     0, Total:     1, Duration: < 1 ms - tests.dll (net5.0)`

pactffi_verifier_logs_for_provider will return all the logs that occurred when verifying a particular provider by name. It won't return the verifier output which shows the results, because that is not written via a logging framework.
Still need an API to get the results so the test framework can output it in an appropriate way. But it will need to use the Handle API, not the call where you pass in the command line.

@mefellows There seems to be compatibility issue with non-UTF-8 byte sequences in Rust's standard output on Windows. Could this be part of the issue?
https://doc.rust-lang.org/std/io/fn.stdout.html

No the issue is that currently it's not implemented. We'll add support to
PactNet once it's stabilised in the FFI
On Tue, 5 Oct 2021, 17:24 Martin Isaksen, @.***> wrote:

@mefellows https://github.com/mefellows There seems to be compatibility
issue with non-UTF-8 byte sequences in Rust's standard output on Windows.
Could this be part of the issue?
https://doc.rust-lang.org/std/io/fn.stdout.html
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/pact-foundation/pact-reference/issues/132#issuecomment-934563412,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAD4FKTL4CPNTSX7I7ETMC3UFMRCLANCNFSM5CB43G2A
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

@adamrodger My comment was in regards to the verification output working on Macs with the https://github.com/DiUS/pact-workshop-dotnet-core-v3 example. It shows the output during verification in the terminal, but I am unable to replicate that behavior locally on my Windows machine.
How were they able to get the verification output in the examples for that repo if it has not been implemented? (See step 4)


GLC Scan Pipeline

Signed-off-by: Avishrantsh<EMAIL_ADDRESS>

Is this what are you talking about @tdruez? For using the tool on a single file?
https://scancodeio.readthedocs.io/en/latest/scanpipe-tutorial-2.html

@AvishrantsSh I was talking about the ability to run ScanCode-toolkit on the whole codebase (similar to a command line usage) or to call the internal API directly from Python.
See https://github.com/nexB/scancode.io/blob/main/scanpipe/pipes/scancode.py#L143 and https://github.com/nexB/scancode.io/blob/main/scanpipe/pipes/scancode.py#L159 for calling the API on 1 file.
And https://github.com/nexB/scancode.io/blob/main/scanpipe/pipes/scancode.py#L301 for running on a directory.
In any case, it's an example of various ways to run the toolkit, not meant to be duplicated on your side.
For now, it'll be great to have a new pipe function in your new module that take a location of a single file and return the scan results.

For now, it'll be great to have a new pipe function in your new module that take a location of a single file and return the scan results.
Right now the module only supports json file as output. I am working on returning the result as a dictionary type object to Python. Once done, I'll do the necessary amendments in the pipes as you said.

@AvishrantsSh I'm having my Python process killed when calling glc.run_glc on some directories.
At first, it seems to be random, but after some investigation, it appears that the Killed is not directly related to a file specific content but instead happens when the size of the files in that directory is above a certain sum. In my case that threshold is around 18MB but I'm guessing it may vary depending on the machine.
If I remove a file from that directory, the run_glc completes fine, but let's add any other file of ~1MB and the process dies.
This looks a lot like a memory related issue.
To reproduce locally:
import tempfile
from scanpipe.pipes import glc

glc.run_glc(
    location="<debug directory location>",
    output_file=tempfile.mkstemp()[1],
)

Can you share the memory stats while execution of the program?

This looks a lot like a memory related issue.

It is possible because JSON is generated only when all the data is available. So it kind of waits until the scan is complete.
Also can you point me to some resources for testing the module, because I've been able to successfully run it on files as big as 100MB.

Can you share the memory stats while execution of the program?

It'll be easier if you can reproduce the issue locally so you can debug from there.

Also can you point me to some resources for testing the module

Pretty much any large input fails on my side. For example: docker://centos:8

For now, it'll be great to have a new pipe function in your new module that take a location of a single file and return the scan results.

I've implemented a scan_file function in glc pipe. Hope it meets the requirement.

Pretty much any large input fails on my side.

I've also added a maxRoutine parameter in analyze function to reduce the number of concurrent goroutines. By default it is set to 10. Also I've made some changes to the main module to limit memory usage. Still if it doesn't help, try reducing the maxRoutines to 1.

@AvishrantsSh same issue with maxRoutines=10 and maxRoutines=1
>>> classifier.analyze("codebase/", output=tempfile.mkstemp()[1], maxRoutines=1)
Killed

The glc.scan_file appears to work fine on most files but also get "Killed" on large one:
>>> glc.scan_file("codebase/centos_8.tar")
Killed

I think I've figured out the issue. The module currently tries to read the file in its entirety before analysing it. That being said, it will take me some time to modify the module to make it work with larger packages.

For now, I think we should work on developing some kind of framework to support newer pipelines. We can work on the module part in parallel.

@tdruez, I've been looking at the issue and it seems there is not much that I can do. Google Licenseclassifier requires the file to be in byte array format and doesn't support data streams.
func (c *Classifier) match(in []byte) Matches {
	id := c.createTargetIndexedDocument(in)

So until changes are made in the upstream, it will be difficult.
One thing that we can do is skip the files which are too big to analyze.
Thoughts?

One thing that we can do is skip the files which are too big to analyze.

Yes, but instead of skipping, it'd be better to catch the error when it occur (so we can log it) and move on the next file.

@AvishrantsSh the code we worked out together was fine IMHO https://github.com/nexB/scancode.io/blob/a4a451f809c4e9cabd353c7e036fac8b822e7861/scanpipe/pipelines/glc_scan.py
The two commits you made afterwards make no sense to me.

we do not want to scan a whole directory at once for now at all
wrt. #231 I think you should not get hung up on the format. Rather try to update a single thing for now which would be the license_expressions field and nothing else directly in https://github.com/nexB/scancode.io/blob/a4a451f809c4e9cabd353c7e036fac8b822e7861/scanpipe/pipelines/glc_scan.py#L60 and do notr save all the data you get for now.

So IMHO you should revert https://github.com/nexB/scancode.io/pull/226/commits/9041af6ebc25aee12f2ed5a459d1dec1cce8759f entirely
https://github.com/nexB/scancode.io/pull/226/files#diff-0297e88d813e7ee05472fe75fd201ca5277f7e8f71085456558416c43230b39fR49 does not make sense at if you scan 10,000 files you would get 100,000,000 loops done.
Please do not try to work in batch on a full directory for now. Focus on correctness, one file at a time. And not on any optimization.

@pombredanne, I have reverted the changes as you mentioned. Now the pipeline scans individual files and saves them in a linear fashion, just as we designed the other day. The pipeline is now able to store all the parameters in scancode.io, including copyright info and holders.
Also, I have made some modifications in the module itself to only scan files of size 50MB or less for now (@tdruez mentioned high memory usage with some files). Anything above that, a scan_error will be generated.

@tdruez, I have updated the tests. Can you have a look?

Also, I have made some modifications in the module itself to only scan files of size 50MB or less for now (@tdruez mentioned high memory usage with some files). Anything above that, a scan_error will be generated.

Pipelines are now able to complete without crashing, much better :)

Closing the PR, the code is now available at https://github.com/nexB/scancodeio.io-pipeline-glc_scan


⚠ Adacta-fintech.com has degraded performance

In dacb19a, Adacta-fintech.com (https://www.adacta-fintech.com) experienced degraded performance:

HTTP code: 200
Response time: 1047 ms

Resolved: Adacta-fintech.com performance has improved in 121d3af after 7 minutes.


How to calculate the x, y coordinates in the relationship graph to determine the node position

What problem does this feature solve?
What algorithm is used to calculate the x and y coordinates in the relationship diagram？
What does the proposed API look like?
Use a demo to illustrate the calculation method of x and y coordinates

同问，有解决吗？关系图中的xy坐标是如何得来的呢


Add mautic and recent CVEs

I have created a PR according to the instructions to add Mautic and included three recent CVEs, however I am unable to have it pass the validation due to the naming of our main branch (features).
Would appreciate some guidance on how to proceed!

Just remove the branch, it's not needed (we only track release branches).

Just remove the branch, it's not needed (we only track release branches).

Great, thank you, should be good to go!

Great, thank you, should be good to go!


Hooking Callback List

I need to hook ALL windows callback list(ProcessCreate Callback)
How I can do that?
Exemple:
Lets supose there is 6 callbacks registered by kernel, I traverse the linked list, get the addresses of All callbacks, and redirect the execution flow to one function -> (void hkHandleAllSixCallbacks())
There is any way to do that using easyhook?

In theory you can do this within the kernel because the callbacks are not core kernel so I believe you are able to modify them without PatchGuard issues (i.e. hook each callback). I don't have a suitable test environment for EasyHook kernel mode currently so can't answer definitively.

@spazzarama I have tested, It works, but the problem is how to know which function called hkHandleAllSixCallbacks, it seems there is no information in return address

Take a look at LhBarrierGetCallback, this is used to retrieve context about the currently executing hook (must be called from your hook handler, and before trying to call the original).

@spazzarama I am trying to use LhBarrierGetCallback but it does not store nothing at pointer passed as argument, and yes, I am calling it inside hkHandleAllSixCallbacks


Selection by Form

I found the ability to select by form really useful when moving things around or selecting code to evaluate in the REPL.
I remember Nightcode doing this a while back (in the paredit days); I believe using Alt plus an arrow key to select the current form, then the form containing it, etc.

Yep this should definitely be doable. Technically this would be a paren-soup feature but I'll leave it here since this project is more publicly facing. I'll look into it soon.


UI suggestions 🖍

Okay, so the first thing that we need to agree on is what is the scope of the app. Because, according to the project description, it should have two features:

The quick navigation should make it possible to quickly find the article you are looking for by simply entering an article (text and/or speech) and to display and navigate the way to it from your current location.

On the other hand, with the optimal route, the shopping basket can be filled before shopping. So that an optimal route through the store can be created and the app can display your current location and the route which you have to follow for the shopping basket.

I would suggest to focus on one feature first. I would say that we should start with 1. What do you think? @Hossam47 and @kiranvasudev?

1- we can get the current location of user by the nearest QR Code and build the path depending on the distance between those two points

For feature 1, I have a certain idea of how to app flow could look like:

Start screen: User is instructed to scan QR code.

Supermarket overview screen: Layout of supermarket and current location (retrieved from QR code) is displayed. User has button to find an item in the supermarket (which triggers screen 3).

Article selection view: User can search and select article they are looking for. User can type in the textbox and suggestions are displayed below. Selecting an item triggers screen 4.

Shortest path view: Location of item and shortest path to item are displayed on navigation view.

In screen number 2 : add item will be Textfield ?

In screen number 2 : add item will be Textfield ?

addItem would be a button that brings you to screen 3.
And maybe it should be called "searchItem" because you can only search for one item.

Okay

I would suggest to focus on one feature first. I would say that we should start with 1.

I would suggest that we also focus on 2. I feel like it is a more realistic use case.

I would suggest to focus on one feature first. I would say that we should start with 1.

I would suggest that we also focus on 2. I feel like it is a more realistic use case.

Yes, it is definitely the more exciting use case! I just found it a lot harder to make it work on the app (UI wise). We have it working in the backend already though, so I was thinking about showing it inside a Jupyter Notebook (we could even deploy it on binder and make it available to everyone). What do you think @kiranvasudev?

I just found it a lot harder to make it work on the app (UI wise).

Why do you say this? If we can get the path to one item(in the UI), it should be more or less similar for more items right?
@Hossam47 could you give us some input to this?

I just found it a lot harder to make it work on the app (UI wise).

Why do you say this? If we can get the path to one item(in the UI), it should be more or less similar for more items right?

I guess one thing I found hard is to show which item to collect at which section in the supermarket. You would need some sort of hover for that.
Besides the app flow would be a bit different, because you wouldn't start with a QR code. They said you would use this feature before going to the supermarket. So I guess the first thing in the app flow would be to select the supermarket you want to go to. Then instead of just selecting one item, you could select multiple. I would be okay with drawing some mockups for that as well, but it really depends on what @Hossam47 thinks and how much time he has. :)

So I guess the first thing in the app flow would be to select the supermarket you want to go to. Then instead of just selecting one item, you could select multiple.
I was thinking the same. And to me, instead of focusing on just one product and getting a route for it, we should do a multiple item path navigator. Because, realistically, I would say that a person would calculate the path to a single item if he/she is a tourist/person(?) who goes to a new supermarket.

For the regular visitors, they would mostly just use Feature 2.

I would be okay with drawing some mockups for that as well, but it really depends on what @Hossam47 thinks and how much time he has. :)
Agree with this. I have no idea how difficult this would be to build on the app.

@kiranvasudev @janjagusch it will take a long time , i think we can work on the first feature and planing to the second


Drop support of Django-1.5

Django-1.5 doesn't support Python-3.5. It leads to errors like this:
AttributeError: module 'html.parser' has no attribute 'HTMLParseError'


Sprint Epic 8th Dec - 12th January

For 0.7.3 (to land asap):

[x] {{tags}} and {{foreach}} helper improvements (#6205)
[x] Publish date bug fix (#6184)
[x] Reduce no. async helpers to reduce the no. cases that cause aSyNcId's to be output (#5850)
[x] Editor bug where certain chars can't be typed (#6192)
[x] PR review

Post 0.7.3:

[x] Data migration for hidden -> hashtag (#6165)
[ ] Addition of client permissions + any others (maybe themes) (#4176)
[ ] First pass at hash tags (#6165)
[x] Remaining {{tags}} and {{foreach}} helper improvements

The remainder of the Sprint / Holiday period will be used to spike/experiment with the following things:

new editor
advanced image features (better upload, optimisation, management, and more)
post filtering in the admin

The next Sprint hangout meeting will be on the 5th January.

Closing this in favour of https://github.com/TryGhost/Ghost/issues/6328

Also... we got 2 more bug fix releases done than we expected... not bad :cookie: :doughnut:


Error parsing curl command: expected a 'command' AST node, got redirected_statement instead

curl -H "X-Requested-With: curl" -H "Content-Type: text/xml" -u "USER:PASS" -X "POST" --data-binary @- "https://example.com/api/2.0/fo/auth/unix/?action=create&title=UnixRecord&username=root&password=abc123&ips=<IP_ADDRESS>" < add_params.xml

Your command pipes input from a file < add_params.xml, if you remove it, it'll convert it to
import requests

headers = {
    'X-Requested-With': 'curl',
    'Content-Type': 'text/xml',
}

params = (
    ('action', 'create'),
    ('title', 'UnixRecord'),
    ('username', 'root'),
    ('password', 'abc123'),
    ('ips', '<IP_ADDRESS>'),
)

data = open('-', 'rb').read()
response = requests.post('https://example.com/api/2.0/fo/auth/unix/', headers=headers, params=params, data=data, auth=('USER', 'PASS'))

and then I think you just need to manually change
data = open('-', 'rb').read()

to
data = open('add_params.xml', 'rb').read()

Thanks, that manual code edit works.

You could also edit your command to pass the filename directly to curl:
--data-binary @add_params.xml

instead of telling it read from stdin and piping the file's contents
--data-binary @- < add_params.xml

and that gets converted the way you'd expect.


Fix ModalComposite closeTimeout prop

This prop exists on ModalComposite's underlying Modal component, but it wasn't propagated making it unusable.
Also removed JF from the code owners so he doesn't get flooded with reviews while he's not here.

Codecov Report

Merging #648 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #648   +/-   ##
=======================================
  Coverage   99.72%   99.72%
=======================================
  Files         261      261
  Lines        6635     6635
  Branches     1541     1541
=======================================
  Hits         6617     6617
  Misses         18       18

Impacted Files
Coverage Δ

src/components/modal/ModalComposite.tsx
100% <ø> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cd39a5b...311e3b2. Read the comment docs.


initial patch for worker node data update

fixes #8279
Still need to do

update correct value
agree on default value
agree on property name and format

test this please

It gets the host name but not Job features and machine features. Valentin, @vkuznet  Valentin, do you know which site this env value is set?

Seangchan, I think we should have left this one out of the current WMAgent release. The last jenkins test was done in the beginning of November and we have no idea how the job and machine features data looks like.
We should probably agree on the data format and exactly which data we need to fetch and publish to WMArchive.

Hi, yes, MJF is defined in, at least, this document:
http://hepsoftwarefoundation.org/notes/HSF-TN-2016-02.pdf
As a test site, you can use PIC, for example. Any job should have access to these files and variables, if you want to check.

Does PIC allow ssh_to_job (global or itb pool?)? Such that I can play with these metrics and gather all info we need?

I'm not sure ssh_to_job works right now, but looking at one job currently running, I can provide the following info:
[root@td707 jobfeatures-43250153.pbs04.pic.es]# pwd
/tmp/mjf-cmpilot033/jobfeatures-43250153.pbs04.pic.es
[root@td707 jobfeatures-43250153.pbs04.pic.es]# ls -l
total 28
-rw-r--r-- 1 cmpilot033 cmpilot  1 Feb 18 22:23 allocated_cpu
-rw-r--r-- 1 cmpilot033 cmpilot  5 Feb 18 22:23 hs06_job
-rw-r--r-- 1 cmpilot033 cmpilot 21 Feb 18 22:23 job_id
-rw-r--r-- 1 cmpilot033 cmpilot 10 Feb 18 22:23 jobstart_secs
-rw-r--r-- 1 cmpilot033 cmpilot 11 Feb 18 22:23 max_swap_bytes
-rw-r--r-- 1 cmpilot033 cmpilot 12 Feb 18 22:23 scratch_limit_bytes
-rw-r--r-- 1 cmpilot033 cmpilot  6 Feb 18 22:23 wall_limit_secs
[root@td707 jobfeatures-43250153.pbs04.pic.es]# for i in ls; do echo $i; cat $i; done
allocated_cpu 8
hs06_job 83.80
job_id 43250153.pbs04.pic.es
jobstart_secs<PHONE_NUMBER>
max_swap_bytes<PHONE_NUMBER>2
scratch_limit_bytes<PHONE_NUMBER>52
wall_limit_secs 385200
Each variable is stored in a file named after the variable, which has the variable value as content.

Thanks, Antonio!
I managed to connect to a T2_DE_DESY worker node (batch0670.desy.de) and this is the information I extracted.
[cmsplt003@batch0670 /var/lib/condor/execute/dir_5653/glide_RaR3wG/execute/dir_16494]$ echo $JOBFEATURES

[cmsplt003@batch0670 /var/lib/condor/execute/dir_5653/glide_RaR3wG/execute/dir_16494]$
[cmsplt003@batch0670 /var/lib/condor/execute/dir_5653/glide_RaR3wG/execute/dir_16494]$ echo $MACHINEFEATURES
/etc/batch/machinefeatures/
[cmsplt003@batch0670 /var/lib/condor/execute/dir_5653/glide_RaR3wG/execute/dir_16494]$ ll $MACHINEFEATURES
total 12
-rw-r--r-- 1 root root 3 Dec 11 14:14 hs06
-rw-r--r-- 1 root root 4 Sep 21  2016 hsW
-rw-r--r-- 1 root root 2 Sep 21  2016 total_cpu
[cmsplt003@batch0670 /var/lib/condor/execute/dir_5653/glide_RaR3wG/execute/dir_16494]$ for i in `ls $MACHINEFEATURES`; do echo $i; cat $MACHINEFEATURES/$i; echo ""; done
hs06
369
hsW
1.48
total_cpu
32

where one can see the worker node does not provide JOBFEATURES var/dir

Would you know the precise htcondor classads? BTW, didn't I have you added in my gtalk? If you can find me there, it's probably easier to talk there then we can update a summary here

Not sure how you'd access $JOBFEATURES directly as the user. As job = pilot in this context, wouldn't you need to do ssh_to_job from the pilot factory, which is the schedd for the glidein, rather than from the payload schedd, to access the proper environment?

I checked a job that ran at CERN and I saw both environment variables, though I couldn't list their content because I cannot connect to CERN worker nodes.
I created a new issue to fix these information and properly feed the monitoring tools. For now, this patch will always have an empty information (empty string?) for Job and Machine features.
I also ran a couple of tests just to make sure it was not going to break somewhere else.

@ticoann, Seangchan it seems like you already resolved this issue without me.
But I don't really know which site to look at. I saw that Alan already provided
some info.
On  0, ticoann<EMAIL_ADDRESS>wrote:

It gets the host name but not Job features and machine features. Valentin, @vkuznet  Valentin, do you know which site this env value is set?
--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
https://github.com/dmwm/WMCore/pull/8304#issuecomment-366873877

@vkuznet, Valentin, I discussed with Alan about the data structure.
Is it OK to have worker node info in metadata top level and separate metrics for worker node (job/machine features).
For now we can just push host info and we can push job/machine info when we know more about what is need to be posted. (I will create the separate patch). Would that be OK?

Yes, I think all of these metrics should be at top level since they remain
constant for entire document which represent workflow/steps running on a node.
On  0, ticoann<EMAIL_ADDRESS>wrote:

@vkuznet, Valentin, I discussed with Alan about the data structure.
Is it OK to have worker node info in metadata top level and separate metrics for worker node (job/machine features).
For now we can just push host info and we can push job/machine info when we know more about what is need to be posted. (I will create the separate patch). Would that be OK?
--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
https://github.com/dmwm/WMCore/pull/8304#issuecomment-367022859

Hi folks,
Do we ever expect this to be useful for more than 20% of jobs?  Certainly the adoption in the US is zero and will always be that way for non-WLCG resources.
Seems to be a more widely applicable to pull the information from the HTCondor machine ad. SI always has some benchmark info there and also takes care of interacting with the batch environment.  We should let them handle this, rather than both teams parsing the info.
Brian

Sorry I wasn't aware there was other effort propagating this info. My understanding was this info was needed in WMArchive. Certainly, we don't have to duplicate the effort.

Brian, Seangchan, I'll not close the case, since we should clarify what
end-users need. It's most likely that some information is present in one source
but not another and we need to have a common key to match the info in
two sources. If I recall correctly that was the case with DESY people
request, but we better clarify it with Manuel Giffels. Here what he asked me
originally in a series of emails (I only extract relevant parts):
I have one additional question concerning the data, because a computer science student recently started to work on a workload balancing simulation. The idea is to find the optimal balance between I/O-intense tasks and CPU-intense tasks running at GridKa. Therefore, we would be interested in performance data of WMAgent jobs.
...
We are interested in anything stored under the 'performance‘ key inside your FWJR Schema. In addition, we like to get also information about the WnHostName, JobType (production, processing, etc.) and the task (e.g. mcremone_ACDC0_task_TOP-RunIIFall17wmLHEGS-00056__v1_T_180122_144320_4391).
...
Do you have any hint on how one could match the jobs inside job monitoring with the fwjr’s in wmarchive?
Best,
Valentin.
On  0, ticoann<EMAIL_ADDRESS>wrote:

Sorry I wasn't aware there was other effort propagating this info. My understanding was this info was needed in WMArchive. Certainly, we don't have to duplicate the effort.
--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
https://github.com/dmwm/WMCore/pull/8304#issuecomment-367363887

Wait, let's recap; what information is needed and what is the intended source? Can you guys check what's already being collected by the pilots and propagated to job classads?
Then we may consider if we need anything that's only provided via de Machine & Job Features mechanism, as the deployment of is is certainly scarce (because it's not requested by CMS).
If it's indeed needed, then we have to make mandatory for CMS sites, deploy, etc.

Hi,
Indeed - I think this is already in the job ClassAds (and has been for many months).  Let's bring it up with Manuel maybe?
In addition, the SI pilots include DB12-based benchmarking far more uniformly than HS06 is deployed.  It's less accurate than HS06, but might be a reasonable tradeoff they want to consider.
Brian


SILGen: Handle emitting borrowed lvalues from value bindings.

Replace the "must be an address" assertion with an implementation that emits and borrows the base value. Fixes #71598.

@swift-ci Please test


[Code health] Simplify LocalDataStore hierarchy

Proposals:

[x] Minor renames in https://github.com/google/ground-android/issues/1492
[ ] Move mutation methods into new repository, MutationRepository
[ ] Remove LocalDataStore, inject individual stores directly, simplifying how stores are injected.
[ ] Remove LocalStore and LocalMutationStore base classes: stores don't always implement the same operations, and we may want to migrated methods to coroutines incrementally.
[ ] Move room under stores since it's an impl of stores.
[ ] Move classes in room.stores up one level to room
[ ] Remove domain object name from methods since clear from context

@scolsen Wdyt?

One challenge I encountered while doing the cleanup is that we uninstall the entire LocalDataStoreModule module in tests. Since we don't inject a fake for each individual datastore, we're required to mock any uses of injected substores. I'll do that temporarily if there aren't too many instances, then return to cleanup #1471 to simplify usage in tests.

Most work done in #1521. Closing this for now.


Add template debugging functions for variables

Template developers frequently need to show variable values while
debugging templates. Issue #1009 requests better support for this.
There are already ways to display information about template
variables. This patch makes it easier for users that don't have
experience working with Go templates.
The printf "%#v" idiom is powerful but requires some knowledge
of Hugo's internal representation of Go value types. Not all
template designers will have that knowledge. The two functions
inspect and typeOf provide a path towards easier to use and
understand printing of variables and values.
The inspect function replaces the printf call. This version
relies on the default string conversion for the result. This
provides a path towards pretty-printing variables and values.
Templates may use typeOf to determine the best formatting
for variables and values. This replaces printf %T.
References #1009

This is a big maybe in my book. Both of these could be replaced with printf. I don't think we should duplicate existing functionality.
If some of that is hard to use, we should document it. I think people would be really happy to know that the powers of printf is available.

I think the correct way to do this is:

Add a new DebugEnabled flag (default false)
Add an internal partial that, if DebugEnabled, dumps the context into a HTML table

This way the users can chose where to put this info.

I'm closing this now. The thought behind this one is excellent -- but I'm not seeing anything here that isn't possible with printf.


:racehorse: wrap loop of config.setDefaults() in transact()

When config.setDefaults() is called "did-change" event emitted per one config key.
This hits performance when package has large set of default configs (like atom-beautify.
Without transact:

With transact:

Thanks!

Thanks for noticing this @ypresto!

I noticed this overlaps with #12226 because setDefaults is only called by setSchema. Anyway thanks..!


Update Namespaces Docs

This is a...

[x] Feature Request
[ ] Bug Report

For this topic: Share a Cluster with Namespaces:

Put the content into the Task template.
Edit the text so that is complies with the style guide.
Test the step in the topic. Make sure the steps can actually be performed by the reader.

Study the following topics, and consolidate, if possible, into a smaller set of topics:
Namespaces
Namespaces Walkthrough
Federated Namespaces
Share a Cluster with Namespaces

/remove-lifecycle stale
/lifecycle frozen

/language en

/priority backlog

/remove-label team/google
/triage accepted

I'm going to close this. It's actionable, but things have moved on since the issue was opened.
/close


fix: inferring args types without as const

Previously, we were forced to define the args object with as const even when the field was defined with defineField. This PR fixes it.
The args will now be inferred for the resolver without using as const if defined with defineField.

:tada: This PR is included in version graphql-gene-v1.1.0-beta.2 :tada:
The release is available on:

npm package (@beta dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


ipsec: Simplify XFRM FWD policies

This pull request implements various simplifications of our XFRM FWD policies and the code that installs them. See commits for details.

/test

/test

All required tests are passed, mark ready to review.


Fix Accumulo hanging when TLS enabled on ZooKeeper

This PR addresses the issue described in #1578. The code changes has been successfully tested against TLS & non-TLS configured ZK setup. Please review & let me know for any questions/suggestions.
Like I mentioned in #1578 regarding the warning message during start/stop of services I'll create a separate issue to investigate.

Hi @karthick-rn ; thanks for the pull request. Here are some tips to improve your git log messages: https://chris.beams.io/posts/git-commit/

I feel like the three affected tools could simply have avoided using ZooReaderWriter and ZooSession and instead just used ZooKeeper directly.

This may lead to a lot of duplication and/or require a large refactoring of code to reuse things like the authentication code for ZK.   Not sure.
Another way to make this change cleaner would be to call SingletonManager.setMode(CLOSED) in ServerContext.close().  However doing this may destabilize existing code using ServerContext.  To avoid the destabilization, could create an AdminUtilContext that extends ServerContext AND calls SingletonManager.setMode(CLOSED) in its close method.   Then admin util like the ones changed in this PR could do something like the following.
try(AdminUtilContext ctx = new AdminUtilContext()) {
   // do admin stuff
}

Just sharing my thoughts, not suggesting we do any of this at this point.

Another way to make this change cleaner would be to call SingletonManager.setMode(CLOSED) in ServerContext.close(). However doing this may destabilize existing code using ServerContext. To avoid the destabilization, could create an AdminUtilContext that extends ServerContext AND calls SingletonManager.setMode(CLOSED) in its close method. Then admin util like the ones changed in this PR could do something like the following.

We've talked about creating a Context for utilities before. I think it probably makes sense to do that for this reason as well.
Another way to simplify this is to let the context explicitly manage the ZooKeeper object, rather than use the SingletonManager stuff at all. In my view, the SingletonManager was a short-term hack to manage static object lifecycles, when the long-term fix is to avoid static objects. The context object is the primary lifecycle object, so we really should only be getting our ZooKeeper object from that. The context itself should close the ZooKeeper object when it is closed.

Just sharing my thoughts, not suggesting we do any of this at this point.

Yeah, all these ideas can be left for future changes/improvements. The approach taken by this PR is fine with me for now.

I'm preparing to merge this.

Hi @karthick-rn ; thanks for the pull request. Here are some tips to improve your git log messages: https://chris.beams.io/posts/git-commit/

Thanks @ctubbsii for sharing the link. Going forward I'll ensure the git log messages are as described on the link.

Merged. Thanks for the PR @karthick-rn . I know you've contributed a few times, but I didn't see you listed on https://accumulo.apache.org/people/, and didn't see a previous invitation to add yourself there. So apologies if you've received this information before:
If you wish to be added as a contributor to https://accumulo.apache.org/people/ , please open a pull request to add yourself at https://github.com/apache/accumulo-website/edit/master/pages/people.md and leave a reference to apache/accumulo#1586 in your commit log.
If you intend to be a regular contributor to Accumulo projects, please consider subscribing to our developer mailing list (https://accumulo.apache.org/contact-us/) and introducing yourself. :smiley_cat:


Docs: Update diagnostics-distributed.rst [skip ci]

This PR updates diagnostics-distributed.rst with a couple punctuation improvements (minor things)

Thanks @farrajota


Voting

Nice work! Support a method to let us vote for the best banging tracks on the sample page!

Thanks for the idea, this probably won't happen though, the sample page is just a simple static HTML :)


wide-resnet-keras error

you don't write this line :sess.run(tf.global_variables_initializer()),this will cause error .

What's more ,your initializer should be glorot_uniform rather than he_normal which will cause this model didn't convergence .


HTTPS links don't work when adding new torrent

Yes, it doesn't support it right now, will fix it soon (here is the issue). I didn't know that lostfilm supports https.
If it's is critical you can fix it here (path/to/monitorrent/plugins/trackers/lostfilm.py line 133):
replace
_regex = re.compile(u'http://www\.lostfilm\.tv/browse\.php\?cat=(?P<cat>\d+)')

with
_regex = re.compile(u'https?://www\.lostfilm\.tv/browse\.php\?cat=(?P<cat>\d+)')

@werwolfby here you are. Thanks ;)


Ensure uniform sample is in range [min, max]

See #60

#60 fixed the upper bound. Now we have to ensure we have a closed-closed interval instead of closed-open.
There's a Closed01 wrapper in the rand crate. I'll take a look into it, maybe we can use it.

I updated #60 to sample from a closed-closed range [min, max].

Resolved wit h 5d07c04. Thanks @mp4096


Cannot find module '../appindex.js'

Followed instructions from: https://github.com/postlight/lux/wiki/Getting-Started
After creating a new app, 'lux serve' fails:
~/projects/ember/lux-test-2 (master) $ lux serve
Error: Cannot find module '../appindex.js' from 'c:/Users/xxxxx/projects/ember/lux-test-2/dist'
    at C:\Users\xxxx\projects\ember\lux-test-2\node_modules\resolve\lib\async.js:55:21
    at load (C:\Users\xxxx\projects\ember\lux-test-2\node_modules\resolve\lib\async.js:69:43)
    at onex (C:\Users\xxxx\projects\ember\lux-test-2\node_modules\resolve\lib\async.js:92:31)
    at C:\Users\xxxx\projects\ember\lux-test-2\node_modules\resolve\lib\async.js:22:47
    at FSReqWrap.oncomplete (fs.js:117:15)
~/projects/ember/lux-test-2 (master) $

😞  Currently we do not support Windows. This is most likely an issue with Windows paths using \ and UNIX based systems using /. I'm going to close this issue and open up a new one to add Windows support.
😄  Thanks for bringing this up!


LWC retrive issue with sfdx

I am not able to retrieve my LWC components from org to local on windows machine . I ran below commands but both failed
sfdx force:source:retrieve -m LightningComponentBundle
or
sfdx force:source:retrieve -m LightningComponentBundle:helloWorld

This will be fixed in the next patch release of the salesforcedx plugin (2/21/19).  Thanks for your patience!

This is not fixed in this release. I am still facing the same issue.

I tried today after running sfdx update (salesforcedx 45.3.4 (core)), and it's now working fine :)

*.js-meta.xml file not retrieved when retrieving component for the first time. I've updated to salesforcedx plugin version 45.3.4 and tried to retrieve an lwc component that was created on a different computer. I was unable to retrieve the lwc component by its name from my Sandbox environment. I was able to retrieve the .html and .js files using a package.xml file to retrieve all LightningComponentBundles. I can also retrieve all lwc components using 'sfdx force:source:retrieve -m LightningComponentBundle'. In both cases, this does retrieve the *.js-meta.xml files from other lwc components that were already on this computer, but not the .js-meta.xml file for the component I was trying to retrieve to this computer for the first time. I had to get the LightningComponentBundle using Workbench (with the exact same package.xml) to also retrieve the .js-meta.xml file for this component.
Additionally, I am not able to retrieve any component using 'sfdx force:source:retrieve -m LightningComponentBundle:' for any of my lwc components. I am able to retrieve the lwc component by right-clicking the folder in VS Code and selecting 'SFDX: Retrieve Source from Org', but only after I have added the .js-meta.xml file to the folder (and possibly after deploying the folder with the .js-meta.xml file in the folder).

I reproduced the issue and am working on a fix now.  I'm hopeful to get this into the next patch release this Thursday.  My apologies for the incomplete fix!

This should be fixed with the patch release tomorrow.  Closing this issue.  Thanks for reporting and for patiently awaiting the fixes.

i have this issue with AuraDefinitionBundle
1)sfdx -v:
sfdx-cli/7.19.0-c5be43915c win32-x64 node-v10.15.3
2)sfdx plugins:
no plugins installed

SfdxError: You must specify the Aura bundle folder, not an individual component.

I still same error while retrieving AuraDefinitionBundle
sfdx-cli 7.82.0 (core)

I still same error while retrieving AuraDefinitionBundle
sfdx-cli 7.82.0 (core)


Make Android MapRenderer.NativeMap overridable

This is useful when someone wants to combine this library with a closed source library that
provides its own GoogleMap instance.
I'm currently happily using this library, but have to combine it with a third party library that provides a SupportMapFragment subclass. I've got it working by overriding the MapRenderer.OnElementChanged() method and doing a lot of reflection as a substitute for calling the base method. By making the property virtual it allows a hook to provide a substitute GoogleMap instance.

Thanks 👍


State machine executor doesn't recognize new SMs immediately and gets stuck

It appears that the SM executor

Doesn't always recognize new SMs immediately while others are running
Blocks if too many run at the same time

See the logs attached below.
For 1. I'll rely on @dpc for help since maybe that's benign and expected after the recent refactor. We should think about cases where this could lead to stuck SMs that spawn another SM and then wait for it to finish.
My initial theory for 2 isn't supported by the logs, there should be warnings about autocommit failing.
~For 2. I have an idea what could cause it: since even state transitions run in parallel now there could be enormous amounts or retrying on DB commit failure. In the worst case this roughly scales quadratically with the number of state machines contending for the same resource afaik.~
~Every round the executor runs all remaining state transitions in parallel, so consuming n units of CPU time the first round, but only committing 1 of n results to DB. Next round consumes n-1 CPU, the one after n-2 and so on.~

@elsirion Awaiting any state transitions to become ready is a message from the pre-#4230 codebase. One reason why I rewrote it is because it was prone to problems like this.
After #4230 everything should be much more snappy and smooth. However there is currently no bound on number of futures that can be running in parallel. I figured that under normal circumstances it should stay below 10-thousands-like order of magnitude and I expect tokio to handle it just fine.

That sounds like there is a reasonable chance this isn't a problem anymore. TODO: reproduce on 0.2 and test that it doesn't happen on 0.3.

Not actionable imo, we'll keep improving the executor in #5006 and improve testing over time.


The Docker containers do not have network access to the internet.

I added driver: bridge in the defining the networks section.  This is the default network driver.  It not only allows the containers to communicate with each other via a bridge network but makes the service accessible to external networks (e.g., the Internet) via the exposed ports.

Pass the AWS_SESSION_TOKEN environment variable to the containers.


Broken BIOS at Orin AGX after flashing

First, thank you so much for your dedication to bringing Nix/OS to the Jetson ecosystem!
Given the README instructions, I've effectively (I think) flashed an Orin AGX with an output tail:
[ 453.5772 ] Writing partition A_MEM_BCT with mem_coldboot_sigheader.bct.encrypt [ 243712 bytes ]
[ 453.5776 ] [................................................] 100%
[ 456.5801 ] tegradevflash_v2 --write B_MEM_BCT mem_coldboot_sigheader.bct.encrypt
[ 456.5807 ] Bootloader version 01.00.0000
[ 456.6825 ] Writing partition B_MEM_BCT with mem_coldboot_sigheader.bct.encrypt [ 243712 bytes ]
[ 456.6829 ] [................................................] 100%
[ 459.6847 ] Flashing completed

[ 459.6848 ] Coldbooting the device
[ 459.6856 ] tegrarcm_v2 --chip 0x23 0 --ismb2
[ 459.6861 ] MB2 version 01.00.0000
[ 459.7876 ] Coldbooting the device
[ 459.7884 ] tegrarcm_v2 --chip 0x23 0 --reboot coldboot
[ 459.7889 ] MB2 version 01.00.0000
*** The target t186ref has been flashed successfully. ***
Reset the board to boot from internal eMMC.

Now, I can't access the BIOS (by pressing Esc), and the boot no longer has an Nvidia or any other logo. Is it expected?
Curiously, if I put the built minimal ISO, it boots NixOS with working video, peripherals, and ethernet (I didn't test WiFi).
So, I've successfully installed NixOS, as it's traditionally done, while following README specifics for this SOM:

I'm using F2FS with a low-capacity Samsung 960 EVO NVMe (PCI Gen 3) disk that the OS correctly recognizes.

However, a boot loop after flashing makes the new NixOS installation on the SSD unbootable: it defaults to the live ISO.
I tried to change the boot order through efibootmgr to other options, but the BIOS seems to ignore it completely.
Note: I've bought this Jetson from a reseller, so I wonder if they flashed it at all previously. It came with Ubuntu.
Any help is greatly appreciated, as I look forward to documenting the process. Thank you!

I just tried to re-flash this Orin AGX but now using the initrd package w/o success:
./result/bin/initrd-flash-orin-agx-devkit

################################
# L4T BSP Information:
# R35 , REVISION: 4.1
###############################################################################
# Target Board Information:
# Name: jetson-agx-orin-devkit, Board Family: t186ref, SoC: Tegra 234,
# OpMode: production, Boot Authentication: NS,
# Disk encryption: disabled ,
###############################################################################
copying emc_fuse_dev_params(/tmp/tmp.V7ximevfZF/bootloader/t186ref/BCT/tegra234-br-bct-diag-boot.dts)... done.
copying device_config(/tmp/tmp.V7ximevfZF/bootloader/t186ref/BCT/tegra234-mb1-bct-device-p3701-0000.dts)... done.
copying misc_config(/tmp/tmp.V7ximevfZF/bootloader/t186ref/BCT/tegra234-mb1-bct-misc-p3701-0000.dts)... done.
./tegraflash.py --chip 0x23 --applet "/tmp/tmp.V7ximevfZF/bootloader/mb1_t234_prod.bin" --skipuid --cfg readinfo_t234_min_prod.xml --dev_params tegra234-br-bct-diag-boot.dts --device_config tegra234-mb1-bct-device-p3701-0000.dts --misc_config tegra234-mb1-bct-misc-p3701-0000.dts --bins "mb2_applet applet_t234.bin" --cmd "dump eeprom cvm cvm.bin; dump custinfo custinfo_out.bin; reboot recovery"
Welcome to Tegra Flash
version 1.0.0
Type ? or help for help and q or quit to exit
Use ! to execute system commands

[   0.0280 ] tegrarcm_v2 --chip 0x23 0 --ismb2applet
[   0.0283 ] File rcm_state open failed
[   0.0287 ] ERROR: failed to read rcm_state
[   0.0287 ]
[   0.0351 ] tegrasign_v3.py --key None --getmode mode.txt
[   0.0352 ] Assuming zero filled SBK key
[   0.0290 ] Pre-processing config: tegra234-mb1-bct-device-p3701-0000.dts
[   0.0427 ] Pre-processing config: tegra234-mb1-bct-misc-p3701-0000.dts
[   0.0587 ] Parsing partition layout
[   0.0591 ] tegraparser_v2 --pt readinfo_t234_min_prod.xml.tmp
[   0.0599 ] Kernel DTB used: None
[   0.0599 ] WARNING: dce base dtb is not provided

[   0.0599 ] Parsing partition layout
[   0.0601 ] tegraparser_v2 --pt readinfo_t234_min_prod.xml.tmp
[   0.0607 ] Creating list of images to be signed
[   0.0609 ] tegrahost_v2 --chip 0x23 0 --partitionlayout readinfo_t234_min_prod.xml.bin --list images_list.xml zerosbk
[   0.0611 ] MB1: Nvheader already present is mb1_t234_prod_aligned.bin
[   0.0621 ] Header already present for mb1_t234_prod_aligned_sigheader.bin
[   0.0624 ] MB1: Nvheader already present is mb1_t234_prod_aligned.bin
[   0.0664 ] Header already present for mb1_t234_prod_aligned_sigheader.bin
[   0.0668 ] MB1: Nvheader already present is psc_bl1_t234_prod_aligned.bin
[   0.0705 ] Header already present for psc_bl1_t234_prod_aligned_sigheader.bin
[   0.0708 ] adding BCH for mb2_t234_aligned.bin
[   0.0732 ] MB1: Nvheader already present is psc_bl1_t234_prod_aligned.bin
[   0.0834 ] Header already present for psc_bl1_t234_prod_aligned_sigheader.bin
[   0.0836 ] adding BCH for mb2_t234_aligned.bin
[   0.0961 ] Filling MB1 storage info
[   0.0961 ] Parsing dev params for multi chains
[   0.1087 ] Generating br-bct
[   0.1089 ] Updating dev and MSS params in BR BCT
[   0.1089 ] tegrabct_v2 --dev_param tegra234-br-bct-diag-boot_cpp.dtb --brbct br_bct.cfg --chip 0x23 0
[   0.1094 ] Updating bl info
[   0.1096 ] tegrabct_v2 --brbct br_bct_BR.bct --chip 0x23 0 --updateblinfo readinfo_t234_min_prod.xml.bin
[   0.1098 ] WARNING: boot chain is not completed. set to 0
[   0.1105 ] Generating signatures
[   0.1169 ] tegrasign_v3.py --key None --list images_list.xml --pubkeyhash pub_key.key --sha sha512
[   0.1170 ] Assuming zero filled SBK key
[   0.1230 ] Warning: pub_key.key is not found
[   0.1168 ] Parsing dev params for multi chains
[   0.1168 ] Generating br-bct
[   0.1171 ] Updating dev and MSS params in BR BCT
[   0.1171 ] tegrabct_v2 --dev_param tegra234-br-bct-diag-boot_cpp.dtb --brbct br_bct.cfg --chip 0x23 0
[   0.1176 ] Updating bl info
[   0.1179 ] tegrabct_v2 --brbct br_bct_BR.bct --chip 0x23 0 --updateblinfo readinfo_t234_min_prod.xml.bin --updatesig images_list_signed.xml
[   0.1181 ] WARNING: boot chain is not completed. set to 0
[   0.1189 ] Get Signed section of bct
[   0.1192 ] tegrabct_v2 --brbct br_bct_BR.bct --chip 0x23 0 --listbct bct_list.xml
[   0.1197 ] Signing BCT
[   0.1261 ] tegrasign_v3.py --key None --list bct_list.xml --pubkeyhash pub_key.key --sha sha512
[   0.1262 ] Assuming zero filled SBK key
[   0.1275 ] Sha saved in br_bct_BR.sha
[   0.1277 ] Warning: pub_key.key is not found
[   0.1214 ] Updating BCT with signature
[   0.1217 ] tegrabct_v2 --brbct br_bct_BR.bct --chip 0x23 0 --updatesig bct_list_signed.xml
[   0.1220 ] Offset :4608 Len :3584
[   0.1224 ] Generating SHA2 Hash
[   0.1288 ] tegrasign_v3.py --key None --list bct_list.xml --sha sha512
[   0.1289 ] Assuming zero filled SBK key
[   0.1289 ] Assuming zero filled SBK key
[   0.1305 ] Sha saved in br_bct_BR.sha
[   0.1243 ] Updating BCT with SHA2 Hash
[   0.1246 ] tegrabct_v2 --brbct br_bct_BR.bct --chip 0x23 0 --updatesha bct_list_signed.xml
[   0.1249 ] Offset :4608 Len :3584
[   0.1252 ] Offset :68 Len :8124
[   0.1254 ] Generating coldboot mb1-bct
[   0.1257 ] tegrabct_v2 --chip 0x23 0 --mb1bct mb1_cold_boot_bct.cfg --misc tegra234-mb1-bct-misc-p3701-0000_cpp.dtb --device tegra234-mb1-bct-device-p3701-0000_cpp.dtb
[   0.1260 ] MB1-BCT version: 0.10
[   0.1262 ] ERROR: carveout /misc/carveout/aux_info@CARVEOUT_UNUSED1/ is not supported
[   0.1265 ] ERROR: carveout /misc/carveout/aux_info@CARVEOUT_UNUSED1/ is not supported
[   0.1267 ] ERROR: carveout /misc/carveout/aux_info@CARVEOUT_UNUSED1/ is not supported
[   0.1270 ] ERROR: /misc/tsc_controls/tsc_locking_config is not supported
[   0.1272 ] ERROR: /misc/tsc_controls/tsc_locking_diff_configuration is not supported
[   0.1272 ] ERROR: /misc/tsc_controls/tsc_locking_ref_frequency_configuration is not supported
[   0.1272 ] ERROR: /misc/tsc_controls/tsc_locking_control is not supported
[   0.1272 ] ERROR: /misc/tsc_controls/tsc_locking_adjust_configuration is not supported
[   0.1272 ] ERROR: /misc/tsc_controls/tsc_locking_fast_adjust_configuration is not supported
[   0.1272 ] ERROR: /misc/tsc_controls/tsc_locking_adjust_delta_control is not supported
[   0.1272 ] ERROR: /misc/tsc_controls/tsc_capture_control_ptx is not supported
[   0.1272 ] ERROR: /misc/tsc_controls/tsc_capture_config_ptx is not supported
[   0.1272 ] ERROR: /misc/tsc_controls/tsc_stscrsr is not supported
[   0.1272 ] ERROR: /misc/tsc_controls/tsc_locking_adjust_num_control is not supported
[   0.1272 ]
[   0.1272 ] Parsing config file :tegra234-mb1-bct-device-p3701-0000_cpp.dtb
[   0.1272 ] Added Platform Config 9 data with size :- 100
[   0.1272 ]
[   0.1273 ] Updating mb1-bct with firmware information
[   0.1275 ] tegrabct_v2 --chip 0x23 0 --mb1bct mb1_cold_boot_bct_MB1.bct --updatefwinfo readinfo_t234_min_prod.xml.bin
[   0.1283 ] tegrahost_v2 --chip 0x23 0 --align mb1_cold_boot_bct_MB1_aligned.bct
[   0.1288 ] tegrahost_v2 --chip 0x23 0 --magicid MBCT --appendsigheader mb1_cold_boot_bct_MB1_aligned.bct zerosbk
[   0.1290 ] adding BCH for mb1_cold_boot_bct_MB1_aligned.bct
[   0.1360 ] tegrasign_v3.py --key None --list mb1_cold_boot_bct_MB1_aligned_sigheader.bct_list.xml --pubkeyhash pub_key.key --sha sha512
[   0.1361 ] Assuming zero filled SBK key
[   0.1371 ] Warning: pub_key.key is not found
[   0.1312 ] tegrahost_v2 --chip 0x23 0 --updatesigheader mb1_cold_boot_bct_MB1_aligned_sigheader.bct.encrypt mb1_cold_boot_bct_MB1_aligned_sigheader.bct.hash zerosbk
[   0.1321 ] Generating recovery mb1-bct
[   0.1324 ] tegrabct_v2 --chip 0x23 0 --mb1bct mb1_bct.cfg --misc tegra234-mb1-bct-misc-p3701-0000_cpp.dtb --device tegra234-mb1-bct-device-p3701-0000_cpp.dtb
[   0.1326 ] MB1-BCT version: 0.10
[   0.1328 ] ERROR: carveout /misc/carveout/aux_info@CARVEOUT_UNUSED1/ is not supported
[   0.1333 ] ERROR: carveout /misc/carveout/aux_info@CARVEOUT_UNUSED1/ is not supported
[   0.1337 ] ERROR: carveout /misc/carveout/aux_info@CARVEOUT_UNUSED1/ is not supported
[   0.1339 ] ERROR: /misc/tsc_controls/tsc_locking_config is not supported
[   0.1339 ] ERROR: /misc/tsc_controls/tsc_locking_diff_configuration is not supported
[   0.1339 ] ERROR: /misc/tsc_controls/tsc_locking_ref_frequency_configuration is not supported
[   0.1339 ] ERROR: /misc/tsc_controls/tsc_locking_control is not supported
[   0.1339 ] ERROR: /misc/tsc_controls/tsc_locking_adjust_configuration is not supported
[   0.1339 ] ERROR: /misc/tsc_controls/tsc_locking_fast_adjust_configuration is not supported
[   0.1339 ] ERROR: /misc/tsc_controls/tsc_locking_adjust_delta_control is not supported
[   0.1339 ] ERROR: /misc/tsc_controls/tsc_capture_control_ptx is not supported
[   0.1339 ] ERROR: /misc/tsc_controls/tsc_capture_config_ptx is not supported
[   0.1339 ] ERROR: /misc/tsc_controls/tsc_stscrsr is not supported
[   0.1339 ] ERROR: /misc/tsc_controls/tsc_locking_adjust_num_control is not supported
[   0.1339 ]
[   0.1340 ] Parsing config file :tegra234-mb1-bct-device-p3701-0000_cpp.dtb
[   0.1340 ] Added Platform Config 9 data with size :- 100
[   0.1340 ]
[   0.1340 ] Updating mb1-bct with firmware information
[   0.1343 ] tegrabct_v2 --chip 0x23 0 --mb1bct mb1_bct_MB1.bct --recov --updatefwinfo readinfo_t234_min_prod.xml.bin
[   0.1352 ] tegrahost_v2 --chip 0x23 0 --align mb1_bct_MB1_aligned.bct
[   0.1358 ] tegrahost_v2 --chip 0x23 0 --magicid MBCT --appendsigheader mb1_bct_MB1_aligned.bct zerosbk
[   0.1361 ] adding BCH for mb1_bct_MB1_aligned.bct
[   0.1431 ] tegrasign_v3.py --key None --list mb1_bct_MB1_aligned_sigheader.bct_list.xml --pubkeyhash pub_key.key --sha sha512
[   0.1431 ] Assuming zero filled SBK key
[   0.1440 ] Warning: pub_key.key is not found
[   0.1380 ] tegrahost_v2 --chip 0x23 0 --updatesigheader mb1_bct_MB1_aligned_sigheader.bct.encrypt mb1_bct_MB1_aligned_sigheader.bct.hash zerosbk
[   0.1388 ] Error: Skip generating mem_bct because sdram_config is not defined
[   0.1388 ] Error: Skip generating mem_bct because sdram_config is not defined
[   0.1388 ] Copying signatures
[   0.1391 ] tegrahost_v2 --chip 0x23 0 --partitionlayout readinfo_t234_min_prod.xml.bin --updatesig images_list_signed.xml
[   0.1432 ] mb1_t234_prod_aligned_sigheader.bin.encrypt filename is from images_list
[   0.1434 ] psc_bl1_t234_prod_aligned_sigheader.bin.encrypt filename is from images_list
[   0.1434 ] Boot Rom communication
[   0.1437 ] tegrarcm_v2 --new_session --chip 0x23 0 --uid --download bct_br br_bct_BR.bct --download mb1 mb1_t234_prod_aligned_sigheader.bin.encrypt --download psc_bl1 psc_bl1_t234_prod_aligned_sigheader.bin.encrypt --download bct_mb1 mb1_bct_MB1_sigheader.bct.encrypt
[   0.1440 ] BR_CID: 0x80012344705DD6E17800000014FE8080
[   0.4815 ] Sending bct_br
[   0.7888 ] ERROR: might be timeout in USB write.
Error: Return value 3
Command tegrarcm_v2 --new_session --chip 0x23 0 --uid --download bct_br br_bct_BR.bct --download mb1 mb1_t234_prod_aligned_sigheader.bin.encrypt --download psc_bl1 psc_bl1_t234_prod_aligned_sigheader.bin.encrypt --download bct_mb1 mb1_bct_MB1_sigheader.bct.encrypt
Reading board information failed.

I tried to re-run the initial flash package flash-orin-agx-devkit (the one that worked) with the same output as above.
Then, I switched from this USB-A-to-USB-C cable to a USB-C-to-USB-C cable.
Now it seems that the initrd-flash-orin-agx-devkit package succeeded this time, and surprisingly fast:

It took 6 secods plus other 3 seconds of RCM-boot in contrast to the nearly 7 and a half minutes of the initial flash.

Here's a gist with the complete output of this build. If it helps in anything, some errors may be of interest.
After hooking into the serial console, I see the same Ubuntu 20.04.6 that it came with.
Interestingly, the output of efibootmgr from within this Ubuntu has a different order now:
BootCurrent: 0002
Timeout: 5 seconds
BootOrder: 0001,0002,0000,0003,0004,0005,0006,0007,0008
Boot0000* Enter Setup
Boot0001* UEFI Samsung SSD 970 EVO Plus 250GB S4P3NF0M410365Y 1
Boot0002* UEFI eMMC Device
Boot0003* UEFI PXEv4 (MAC:48B02DA4A494)
Boot0004* UEFI PXEv6 (MAC:48B02DA4A494)
Boot0005* UEFI HTTPv4 (MAC:48B02DA4A494)
Boot0006* UEFI HTTPv6 (MAC:48B02DA4A494)
Boot0007* BootManagerMenuApp
Boot0008* UEFI Shell

I've reinstalled NixOS on the SSD using ext4 instead of f2fs and setting efibootmgr for the new boot option.
After that, once the device powers on, I only see a kernel boot log and a black screen with no video.
I've attached it to the serial console, but the connection was to the Ubuntu on the eMMC storage.

I've decided to close this issue and start a more concise one here: https://github.com/anduril/jetpack-nixos/issues/230
I hope these comments help anyone going through the same thing. If not, please feel free to delete it. Thanks!


Update and complete - de.json

should be all of it... you wellcome ;)
nice Mod btw

Thank you for your contribution. I manually merged the file because of some issues I'm having with GitHub's merge.


Number display - small number handling

This issue is taking a piece out of issue #727.
Overview
we would like to update the way small numbers are displayed.
currently, we show 6 decimals and if the number is smaller we show "<0.000001"
the updated approach would expose the small numbers to the user in an updated manner.
Exceptions

Percentage numbers ->display logic for percentage numbers remains unchanged, and percentages below 0.01% will continue to be displayed as they are currently, i.e., "<0.01%."
Input fields->both user input and automatically prefilled values will continue to  display full numbers without any changes.
When mouse over a subscribe number and the tooltip is displayed, display full number in the tooltip without applying the subscribe format

Mockup
Designs
Specifications
IF {integer digits}=0 AND {number of first decimals that are 0}>=3 do the following:

Display first decimal as 0
Count the number of 0 decimals from the first decimal up to the first decimal that is not 0
Display the number as a subscript next to the first 0 decimal (if counted 6 zeros, the number needs to be 6 and the first zero is visible)
Take 5 decimals starting from the first decimal that is not 0 and display them after the first 0 decimal

Examples(prices displayed in USD):

After discussing with @Costa769 we decided to :

Apply full value for tooltip except the one on the strategy card's graph
Prices should follow this pattern :

111 -> 111.00
11 -> 11.00
1 -> 1.00
0 -> 0.00
0.111 -> 0.11
0.0111 -> 0.011
0.00111 -> 0.0011
0.000111 -> 0.0₃111

After discussing with @Costa769 we decided to :

Apply full value for tooltip except the one on the strategy card's graph
Prices should follow this pattern :

111 -> 111.00
11 -> 11.00
1 -> 1.00
0 -> 0.00
0.111 -> 0.11
0.0111 -> 0.011
0.00111 -> 0.0011
0.000111 -> 0.0₃111

I don't understand 0.111 -> 0.11  - why remove the last '1'? Same in: 0.0111 -> 0.011

You need to round it somehow. Maybe this example makes more sense
123.4567 -> 123.45
12.34567 -> 12.34
1.234567 -> 1.23
0 -> 0.00
0.1234567 -> 0.12
0.01234567 -> 0.012
0.001234567 -> 0.0012
0.0001234567 -> 0.0₃12345

OK, this makes more sense now - but I don't get why @ashachaf wanted to make such an exception for subscript.
why it gets 5 more decimals after the subscript but any other small number gets two decimals after the last 0.
e.g. why:
0.001234567 -> 0.0012 BUT 0.0001234567 -> 0.0₃12345
I'd understand
0.001234567 -> 0.0012 AND 0.0001234567 -> 0.0₃12
or
0.001234567 -> 0.0012345 AND 0.0001234567 -> 0.0₃12345

I assume the subscript logic was copied from other application.
Also the 2 digits logic has been decided by @Costa769 during our conversations after I show the initial result

Please implement the following logic with 6 decimals in case 0< value <1
123.4567 -> 123.45
12.34567 -> 12.34
1.234567 -> 1.23
1 -> 1.00
0 -> 0.00
0.1234567 -> 0.123456
0.01234567 -> 0.012345
0.001234567 -> 0.001234
0.0001234567 -> 0.0₃12345

If I understand correctly, this is the rule ?
x >= 0:

minimumDigits: 2
maximumDigits: 2

0 > x >= 0.001

minimumDigits: 2
maximumDigits: 6

0.001 > x (subscript)

minimumDigits (after subscript): 1
maximumDigits (after subscript): 5


Email address is missing in account settings

Subject: Email address is missing in account settings
Description:
Go to account settings. It currently says "Your email address is currently .."
Results:
Backend version: develop-3cdcce6b
Frontend version: develop-3cdcce6b
User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:95.0) Gecko/20100101 Firefox/95.0
Page: https://app.couchers.org/account-settings
User: aapeli (1)

I've debugged this a bit, it's here:
https://github.com/Couchers-org/couchers/blob/d4312c3e2641783ab345ca1991dcf876775af14e/app/web/src/features/auth/email/ChangeEmail.tsx#L60-L64
props.email is indeed set to the right thing, but somehow <Trans> isn't working right, it seems.

OK, I figured it, out the translation has <2>{{email}}</2>, but that should be <1>{{email}}</1>. I'll do a PR in a bit.


Migrating from StateNotifier and from ChangeNotifier take 3

Continues #2888.
Aims to fix #2630.
There's no TODOs left, so I'll just await for your corrections / review.

I'm sorry but I'm unsure what causes the CI tool to fail this time.
Let me know how I can help.

Could you update all links to documents to use the <Link documentID="essentials/my_document" /> component instead of [foo](/essentials/my_document)?
And at the same time reword the test around it to match?
For example instead of mutations and [side-effects](/essentials/side-effect), do mutations and side-effects. See also <Link documentID="essentials/side-effect" />
which renders are "mutations and side-effect. See also "Performing Side effects".

Content wise, LGTM.
Outside of the minor but unfortunately tedious organisation changes, it's all good!
I'll merge this when those are sorted~
Thanks for this!

Thank you Remi, you have no idea how happy I am as of now. I'll fix everything ASAP and ping you again. Thanks again.

Could you update all links to documents to use the  component instead of foo?

While doing this refactor, I can see that this component throws if you write links with # anchors, e.g. writing
<Link documentID="essentials/side_effects#defining-a-notifier" />

Throws the following:
Document title not found for documentID essentials/side_effects#defining-a-notifier

I get the same errors with links such as this one.
Removing the # anchor let Link work, but that's not what I wanted, I need to reference a specific paragraph.
Therefore, I'll substitute only "whole" pages.

Anchors should be defined as a separate parameter (<Link documentID="..." hash="..." />)

It still won't work for external links, right? Such as https://pub.dev/packages/flutter_riverpod/changelog

Alrite, I'm done now (: Thank you again

You overwrote the generated files once again :p
Make sure to run pub upgrade everywhere and run generation once again. Otherwise you're using an outdated version of Freezed

Otherwise you're using an outdated version of Freezed

oooohhhhhhhhhhhhhh that's why it kept happening

@rrousselGit there we go

Alright thanks!