Deploy Checklist: New Expensify 2022-09-26

Release Version: 1.2.7-2
Compare Changes: https://github.com/Expensify/App/compare/production...staging
This release contains changes from the following pull requests:

[x] https://github.com/Expensify/App/pull/10811
[x] https://github.com/Expensify/App/pull/11035
[x] https://github.com/Expensify/App/pull/11071
[x] https://github.com/Expensify/App/pull/11162
[x] https://github.com/Expensify/App/pull/11165
[x] https://github.com/Expensify/App/pull/11177
[x] https://github.com/Expensify/App/pull/11211
[x] https://github.com/Expensify/App/pull/11218
[x] https://github.com/Expensify/App/pull/11224
[x] https://github.com/Expensify/App/pull/11225
[x] https://github.com/Expensify/App/pull/11256
[x] https://github.com/Expensify/App/pull/11287
[x] https://github.com/Expensify/App/pull/11303

Internal QA:

[ ] https://github.com/Expensify/App/pull/11254 - @techievivek @roryabraham
[x] https://github.com/Expensify/App/pull/10899 - @puneetlath @roryabraham

Deploy Blockers:

[ ] https://github.com/Expensify/App/issues/11301
[x] https://github.com/Expensify/App/issues/11305

Deployer verifications:

[ ] I checked the App Timing Dashboard and verified this release does not cause a noticeable performance regression.
[ ] I checked Firebase Crashlytics and verified that this release does not introduce any new crashes.

cc @Expensify/applauseleads

:rocket: All staging deploys are complete, @Expensify/applauseleads please begin QA on version https://github.com/Expensify/App/releases/tag/1.2.7-1 :rocket:

https://github.com/Expensify/App/pull/10899 passed internalQA, checking it off

:clap: Heads up @Expensify/applauseleads :clap:
A new pull request has been :cherries: cherry-picked :cherries: to staging, and will be deployed to staging in version 1.2.7-2 :rocket:

:tada: All set?…You bet! @Expensify/applauseleads https://github.com/Expensify/App/releases/tag/1.2.7-2 has been deployed to staging :tada:

@roryabraham
Regression is completed
PR  #10811 is failing due to https://github.com/Expensify/App/issues/11205. Happening is PROD too.
PR https://github.com/Expensify/App/pull/11165 is failing due to https://github.com/Expensify/App/issues/11186
PR https://github.com/Expensify/App/pull/11218 is failing due to https://github.com/Expensify/App/issues/11330 & https://github.com/Expensify/App/issues/11332 & https://github.com/Expensify/App/issues/11333

Checking off https://github.com/Expensify/App/pull/11165. While it did not fix the issue (and the test steps fail), it's still a clean up for this PR.

Checking off https://github.com/Expensify/App/pull/11211. The reported bug is pretty minor, and may or may not be the expected behavior. Left some additional thoughts here. Not going to treat as a blocker.

@mvtglobally https://github.com/Expensify/App/issues/11301 should be fixed by https://github.com/Expensify/App/pull/11303 in 1.2.7-2, can you please retest?

https://github.com/Expensify/App/issues/11305 is fixed, checking it off

https://github.com/Expensify/App/pull/11303 is PASS. Let me double check the https://github.com/Expensify/App/issues/11301

I was able to verify that https://github.com/Expensify/App/issues/11301 is fixed, checking it off

Going to work on QA of https://github.com/Expensify/App/pull/11254

https://github.com/Expensify/App/pull/11254 seems to be a pass. Performance metrics look good. The only thing is that there's a new crash: https://console.firebase.google.com/u/0/project/expensify-chat/crashlytics/app/android:com.expensify.chat/issues/193757d9bc1c3c9fab13ef0737f6db59?time=last-seven-days&sessionEventKey=6333322C01C900010EA52FBC6C7CF095_1726557865424712073

The crash seems like a memory leak problem in React Native itself. Going to ignore for now because I don't think there's much we can do about it, but we'll monitor it and hopefully see an issue created if it persists.

:shipit:


Accessibility of file pane (feature request)

Brackets 1.8.0 Windows 7:
The file pane and menus look cool, but are very poor for accessibility. I would prefer:

High contrast (not grey-on-grey)
Larger font
On-hover indicators around clickable labels (file names, expand/collapse markers, and menu items)
Visible sliders
Visible panel separator between open files and files

I tried selecting different themes, but they only seem to change the edit pane.
Apologies if this is already changeable and I just missed how to do it.

@rockyallen Thanks for using Brackets and reporting this particular accessibility problem.
In it's current form Brackets has two variations in the view. One can be categorized as UI theme and the other one is Code theme. Essentially, customization is supported only in code theme. Rest of the UI ( Files pane, toolbar, status bar are all part of core UI and can't be customized by the user.
We took a note of the accessibility concerns and will review the same.


Footer capital axis

Modifications:

Layout:

Move all filters/controls to the top.
Set border as light color to make it look transparent

Capital letters in axis and label (based on TA Daniel's feedback)
Footer (requirement of milestone2)

Merge modifications in layout


Replace use of 'Poll:Pending' and 'Poll:Ready' with use or absence of 'blocked' label

I liked them but as we discussed in a previous gardening meeting, the meaning pending/ready labels may not be obvious to new contributors. I'll do this now.

Thanks for picking this up! 🙏


Auto Delete Zip/Rar when extracting

Is your feature request related to a problem? Please describe.
No
Describe the solution you'd like
When right clicking a zip/rar file to extract, could we have an option in the settings to trash the original zip/rar file once its extracted.

a checkbox could be added in the extraction dialog to choose if we want the file to be deleted when the extraction is finished

Merging with #45.


Fcrepo-2923 - Allow modification of premis:hasSize property

JIRA Ticket: https://jira.duraspace.org/browse/FCREPO-2923
What does this Pull Request do?
Allows clients to delete or set premis:hasSize property.
What's new?

Removed premis:hasSize from set of server managed triples
Added integration test that verifies the expected behavior of the four server generated triples for binaries: premis:hasSize, ebucore:filename, ebucore:hasMimeType, iana:describedby
Removes duplicate testLdpContains test (it is already being tested in testLdpNamespace)

How should this be tested?
New integration test.
Create a binary, then issue a PATCH or PUT request to replace the premis:hasSize property. This can be done with a request similar to:
echo 'INSERT { <> <http://www.loc.gov/premis/rdf/v1#hasSize> "99"^^<http://www.w3.org/2001/XMLSchema#long> } WHERE { }' | curl -XPATCH -ufedoraAdmin:fedoraAdmin -H "Content-Type: application/sparql-update" --data-binary "@-" "http://localhost:8080/rest/test_bin"

Attempt to retrieve the description and perform head requests against the binary to see that the new size is being used.
Additional Notes:
While clients can set the premis:hasSize property, it must be of type "http://www.w3.org/2001/XMLSchema#long" and there can only be one property of this predicate at a time (adding a new one will replace the original). If the client replaces the binary, the property will be overwritten with the size of the new file. If no premis:hasSize is present, then 0 will be returned as the Content-Length header.
iana:describedby is added during requests to retrieve non-rdf resource descriptions, so it cannot be truly removed by clients. Clients can, however, add additional iana:describedby relations and delete those. This is not a new behavior in this PR.
This change is intended to align
Interested parties
Tag (@ mention) interested parties or, if unsure, @fcrepo4/committers

@bbpennel : As a note, it looks like your PR message was truncated:

This change is intended to align...

Unlike what you note above in your PR comment, my local testing shows that the additional Datatype information (e.g. ""99"^^http://www.w3.org/2001/XMLSchema#long") is not necessary when setting premis:hasSize.
Am I missing something?

@awoods the type restriction on premis:hasSize only appears to apply when setting the property on a non-rdf description. For a container you can put whatever you want in there. For a binary I get org.modeshape.jcr.value.ValueFormatException: Error converting "what" from String to a Long when providing a non-numeric type.
Finished that incomplete sentence and pushed a commit for that constant.


3D Electronics Box Buttons

Replace the 2D electronics box in the controls popup with a 3D version. 3D files are in /public of demo repo. If screen size is large, load electronics-box-optimized.gltf for a nicer view. If screen size is small, load electronics-box-optimized-no-logo.gltf for better performance.

Will generate optimized files for Express another day


Update to version 0.10.0

+1

Any suggestion on how to update to 0.10.0?

@ahimsauzi This could be caused by a number of things. I'm going to run down the most likely reasons, but if they don't suit, do please provide a screen shot of the command(s) being run and the definitions of any aliases you are using.
If you are running composer update and it gave you these lines in your update log:

Then your updated installation may not be what your alias is pointing to (if you are using one). Try using locate terminus or cd / ; find . -name terminus to see all the places on your system at which files named "terminus" are installed to see if any of those are what was updated.
If this did not appear, then Composer did not update it. You'll need to edit your composer.json file and change the version number next to 'pantheon-systems/cli'. If you want to always keep Terminus up-to-date, change the version to '@stable'. But beware: this may break your scripts. Please keep an eye on CHANGELOG.md for changes before running composer update.

@TeslaDethray looks like I have more than one version but I am not sure which to delete. Obviously, pantheon-systems/cli needs to stay but not sure about the rest.
BTW thank you for the locate suggestion, a great tool!

Your /usr/local/bin/terminus is (unless you have an alias called "terminus" too) the one you are getting when you use "terminus" on the command line. Here's how to narrow this down quickly:
/usr/local/bin/terminus cli info
~/vendor/bin/terminus cli info
~/vendor/pantheon-systems/cli/bin/terminus cli info
I think the last one is the likeliest to give you version 0.10.0. To run this one, you can create an alias to it by adding one to your .bash_profile
echo 'alias "terminus=~/vendor/pantheon-systems/cli/bin/terminus"' >> ~/.bash_profile ; source ~/.bash_profile
(That dir is only right provided I am correct that the last one is your up-to-date Terminus installation.)

+1


Add support of adding custom conditions out of library

Signed-off-by: Mikhail Pronyakin<EMAIL_ADDRESS>

I really like the idea! I think there are some things here that need consideration:

We could simply make conditionFactories public by writing it upper case
Rename AddCustomCondition to AddConditionFactory

What do you think?

I've changed conditionFactories to ConditionFactories and AddCustomCondition to AddConditionFactory.
I've also updated README.md to let users know about this option.

Thank you, I think you can remove AddConditionFactory as it is easy enough to do:
ladon.conditionFactories["foo"] = bar

Do you agree?

Yes, that's enough. Fixed this.

Awesome, thank you for your contribution!


Example to append a path and get a file?

Can you provide an example of using WolframClientForPython to append to a path and get a file? I have to file, I know how to work the same in Mathematica it is like
AppendTo[$Path, "/path”] and  Get["filename"] but with wolframclient I'm not sure.

This library provide convenient tools to interact with a kernel it has started, (e.g WolframLanguageSession). As such anything that works in Mathematica will work the same when done from python (and if it's not the case it's certainly a bug 😉).
The code below update the $Path variable and in a second call returns its value:
from wolframclient.evaluation import WolframLanguageSession
with WolframLanguageSession() as s:
    s.evaluate('AppendTo[$Path, "some/path"]')
    path = s.evaluate('$Path')
    print(path[-1])

@monica1131 keep in mind that issues on github are for bug report only.
Programming questions should be posted on https://mathematica.stackexchange.com/
Thanks.


Test user-provided gamma value to CLI

Background
I’m running the latest version of TreeKnit (v0.5.5) with Julia 1.9.3 on an ARM64 Mac and I get the following error when I try to specify a gamma value:
$ ~/.julia/bin/treeknit rooted_tree_raw_ha.nwk rooted_tree_raw_na.nwk --gamma 2.5
ERROR: LoadError: MethodError: no method matching tryparse(::Type{Real}, ::String)

Closest candidates are:
  tryparse(::Type{T}, ::AbstractString) where T<:Dates.TimeType
   @ Dates /opt/homebrew/Cellar/julia/1.9.3/share/julia/stdlib/v1.9/Dates/src/parse.jl:290
  tryparse(::Type{T}, ::AbstractString, ::Dates.DateFormat) where T<:Dates.TimeType
   @ Dates /opt/homebrew/Cellar/julia/1.9.3/share/julia/stdlib/v1.9/Dates/src/parse.jl:290
  tryparse(::Type{Complex{S}}, ::AbstractString) where S<:Real
   @ Base parse.jl:389
  ...

Stacktrace:
 [1] command_main(ARGS::Vector{String})
   @ TreeKnit ~/.julia/scratchspaces/03f602be-98a1-4bb2-9504-fbd3406624a7/sysimg/libtreeknit.dylib:-1
 [2] command_main()
   @ TreeKnit ~/.julia/packages/Comonicon/AXDxW/src/codegen/julia.jl:90
 [3] top-level scope
   @ ~/.julia/bin/treeknit:15
 [4] include_string(mapexpr::typeof(identity), mod::Module, code::String, filename::String)
   @ Base ~/.julia/scratchspaces/03f602be-98a1-4bb2-9504-fbd3406624a7/sysimg/libtreeknit.dylib:-1
 [5] _include(mapexpr::Function, mod::Module, _path::String)
   @ Base ~/.julia/scratchspaces/03f602be-98a1-4bb2-9504-fbd3406624a7/sysimg/libtreeknit.dylib:-1
 [6] include(mod::Module, _path::String)
   @ Base ~/.julia/scratchspaces/03f602be-98a1-4bb2-9504-fbd3406624a7/sysimg/libtreeknit.dylib:-1
 [7] exec_options(opts::Base.JLOptions)
   @ Base ~/.julia/scratchspaces/03f602be-98a1-4bb2-9504-fbd3406624a7/sysimg/libtreeknit.dylib:-1
 [8] _start()
   @ Base ~/.julia/scratchspaces/03f602be-98a1-4bb2-9504-fbd3406624a7/sysimg/libtreeknit.dylib:-1
in expression starting at /Users/jhuddlesfredhutch.org/.julia/bin/treeknit:15

I get the same error when I pass an integer value for gamma.
I confirmed that the built-in tryparse method fails on the Julia REPL with:
tryparse(Real, "2")

The following works as expected, though:
julia> tryparse(Float64, "2")
2.0

The error in TreeKnit seemed to trace back to this type change in version 0.5.5 (after version 0.5.0). I tried downgrading TreeKnit to 0.5.0, but this did not fix the error.
Proposed changes
I forked the TreeKnit repo (here), modified the CLI test to run with the gamma argument, and confirmed that the CI fails with the same error. Then I tried reverting the type from Real to Float64 and confirmed that this change fixed the error. I also compiled TreeKnit locally and confirmed that the type reversion fixed my issue, allowing me to pass integer and floating point values.
Testing

[x] Modified CLI test to run with --gamma, confirmed test failed prior to fix and passed after fix

Hi John,
Thanks for reporting this. tryparse requires a concrete type like Float64, hence the issue. Your solution to fix this is good.
The only downside is that when calling the treeknit function from julia's REPL, integers values will fail for gamma, i.e. treeknit(...; gamma=1) will fail but treeknit(...; gamma=1.) will work. This is why I changed to gamma::Real. But since Comonicon apparently tries to call tryparse(Real, gamma), the easiest is to revert to Float64 as you did.

Thanks, @PierreBarrat! I created a proper PR for this change in the main TreeKnit repo.


bump version of node-elm-compiler to one that supports the new debug …

…flag

Was just about to propose this PR as well

Was just about to propose this PR as well

You probably should also update the ELM_VERSION in .travis.yml to 0.18.0

what's the status on this?

Nice, thanks @svoynow!

Published as 3.1.0 since it adds a new feature (--debug support).


iOS 9.x Cookies are not sent (CORS)

I've read the documentation on how to send cookies, what I'm doing is:
let requestConfig: RequestInit = {
            method: "GET",
            headers: {
                'Accept': 'application/json',
                'Content-Type': 'application/json'
            },
            mode: "cors",
            cache: "no-cache",
            credentials: "include"
        };

        let getToken = async () => {
            try {
                let csrfResponse = await window.fetch(App.BASE_URL + App.CSRF_PATH, requestConfig);
                let data = await csrfResponse.json();

                return data['_csrfToken'];
            } catch (err) {
                console.error('Can not load CSRF token...', err)
            }
        };

        let initSession = async () => {
            try {
                let data = await window.fetch(App.BASE_URL + App.DATA_PATH, requestConfig);
                return await data.json();
            } catch (err) {
                console.error('Data is not available');
            }
        };

but on iOS 9.x (I'm using Simulator) seems cookies are not sent...

Sorry guys, my fault... It was just cookies were not always allowed on iOS Simulator by default... Turned to "Always accept" and works fine.


Sp gtfs

I believe this could be good to go, pending checks by others --- there's a slight change,
You need to download the GTFS geopackage (dated 27 august), and the gtfs_config.py and locate these in data/GTFS/...
Then, if you run "python setup_config.py" this generates config files (for sample points, these are python scripts now - hope that's okay) --- and then you could use
bash ./process_regions.sh city city city
to iterate over your cities
I haven't checked the gpkg outputs yet -- but it seems to all work (following interactive checks); it runs successfully.  I'll check outputs tomorrow for my cities.  Fingers crossed, all good !

Just an update --- had a quick look at walkability for Auckland --- looks plausible.  I'll check out the outputs tomorrow properly, but for now - it looks like its run, and is quick to run.  I've uploaded my cities to the data/input/September 2020 folder on CloudStor.

Hi @shiqin-liu  and @gboeing  --- I just made a commit updating th agg.py and setup scripts to work with the new variables when outputing to city and hex aggregations.   I've run this with one city (Auckland) as a minimal test and can confirm that it now outputs the hex250, and all cities geopackages, with output apparently as intended.  The study region name in both hex and all cities layers is now title case with space replacing underscore, which might help with plotting; and I also explicitly ordered the hex output columns.  Otherwise - the code should be basically as before.

@gboeing sure, I try to test run city of Vic from this branch, what I did is:

Download GTFS .gpkg and gtfs_config from Cloudstor in the data/GTFS folder
Re-download the input .gpkg from Cloudstor input folder
Run python setup_config.py
Run python sp.py vic

But I got an error:
(base) root@b27971c88234:/home/jovyan/work/process# python sp.py vic

Global indicators project 2020-09-04

Process city: Vic

Read network from disk.
Study region sample point output file exists
Read poplulation and intersection density from local file.
Time taken to calculate or load city local neighbourhood statistics: 0.000550469 mins

Calculate assessbility to POIs.
Generating contraction hierarchies with 8 threads.
Setting CH node vector of size 4844
Setting CH edge vector of size 14080
Range graph removed 14202 edges of 28160
. 10% . 20% . 30% . 40% . 50% . 60% . 70% . 80% . 90% . 100%

Calculating nearest node analyses ...

	- Open street map destinations
		['fresh_food_market', 'convenience', 'pt_osm_any']

	- Public open space
		['public_open_space_any']
		['public_open_space_large']

	- Public transport (GTFS)
Create hex_id for sample points
Creating long form working dataset of sample points to evaluate respective node distances and densities
	 - create long form dataset
	 - join POIs results from nodes to sample points
	 - calculate proximity-weighted average of densitiy statistics for each sample point
Calculating sample point specific analyses ...
	 - Best PT (any) access score
Traceback (most recent call last):
  File "sp.py", line 289, in <module>
    samplePointsData[var] = samplePointsData[columns].max(axis=axis)
  File "/opt/conda/lib/python3.7/site-packages/geopandas/geodataframe.py", line 576, in __getitem__
    result = super(GeoDataFrame, self).__getitem__(key)
  File "/opt/conda/lib/python3.7/site-packages/pandas/core/frame.py", line 2806, in __getitem__
    indexer = self.loc._get_listlike_indexer(key, axis=1, raise_missing=True)[1]
  File "/opt/conda/lib/python3.7/site-packages/pandas/core/indexing.py", line 1553, in _get_listlike_indexer
    keyarr, indexer, o._get_axis_number(axis), raise_missing=raise_missing
  File "/opt/conda/lib/python3.7/site-packages/pandas/core/indexing.py", line 1646, in _validate_read_indexer
    raise KeyError(f"{not_found} not in index")
KeyError: "['sp_access_pt_gtfs_any_binary'] not in index"

I have not figured out why this occurs... @carlhiggs could you help me take a look if you get a chance? Thanks in advance!

Hi,
Thanks for having a look at things. That's curious about the error with Vic
--- perhaps it may arise in the case where a city didn't have a gtfs feed
defined (Vic doesn't); in theory it should be fine though and just yield a
null.  I ran it for six cities without issue - but I'll see if I can get a
chance to check out Vic later today.
Regarding time, it doesn't take days to calculate access - should take
under 10 mins for most cities (Auckland was under 6 from memory, I didn't
pay close attention to others - but the longer processing was taken in the
last round with local neighbourhood statistics --- assuming we all left
things running last week?).
In any case - we have to run it at some point to finish our analyses really
... And the sooner we attempt to run it for all cities, if there are
exceptions that arise, the sooner we'll find them.
On Sat, 5 Sep. 2020, 04:54 Shirley Liu<EMAIL_ADDRESS>wrote:

@gboeing https://github.com/gboeing sure, I try to test run city of Vic
from this branch, what I did is:

Download GTFS .gpkg and gtfs_config from Cloudstor in the data/GTFS
folder
Re-download the input .gpkg from Cloudstor input folder
Run python setup_config.py
Run python sp.py vic

But I got an error:
(base) root@b27971c88234:/home/jovyan/work/process# python sp.py vic
Global indicators project 2020-09-04
Process city: Vic
Read network from disk.
Study region sample point output file exists
Read poplulation and intersection density from local file.
Time taken to calculate or load city local neighbourhood statistics: 0.000550469 mins
Calculate assessbility to POIs.
Generating contraction hierarchies with 8 threads.
Setting CH node vector of size 4844
Setting CH edge vector of size 14080
Range graph removed 14202 edges of 28160
. 10% . 20% . 30% . 40% . 50% . 60% . 70% . 80% . 90% . 100%
Calculating nearest node analyses ...

Open street map destinations
['fresh_food_market', 'convenience', 'pt_osm_any']

Public open space
['public_open_space_any']
['public_open_space_large']

Public transport (GTFS)
Create hex_id for sample points
Creating long form working dataset of sample points to evaluate respective node distances and densities

create long form dataset

join POIs results from nodes to sample points

calculate proximity-weighted average of densitiy statistics for each sample point
Calculating sample point specific analyses ...

Best PT (any) access score
Traceback (most recent call last):
File "sp.py", line 289, in
samplePointsData[var] = samplePointsData[columns].max(axis=axis)
File "/opt/conda/lib/python3.7/site-packages/geopandas/geodataframe.py", line 576, in getitem
result = super(GeoDataFrame, self).getitem(key)
File "/opt/conda/lib/python3.7/site-packages/pandas/core/frame.py", line 2806, in getitem
indexer = self.loc._get_listlike_indexer(key, axis=1, raise_missing=True)[1]
File "/opt/conda/lib/python3.7/site-packages/pandas/core/indexing.py", line 1553, in _get_listlike_indexer
keyarr, indexer, o._get_axis_number(axis), raise_missing=raise_missing
File "/opt/conda/lib/python3.7/site-packages/pandas/core/indexing.py", line 1646, in _validate_read_indexer
raise KeyError(f"{not_found} not in index")
KeyError: "['sp_access_pt_gtfs_any_binary'] not in index"

I have not figured out why this occurs... @carlhiggs
https://github.com/carlhiggs could you help me take a look if you get a
chance? Thanks in advance!
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/gboeing/global-indicators/pull/88#issuecomment-687322863,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADDCCMSEO4WDYFIXVOLVPALSEEZWDANCNFSM4QUVUPLQ
.

Hi, Thanks for having a look at things. That's curious about the error with Vic --- perhaps it may arise in the case where a city didn't have a gtfs feed defined (Vic doesn't); in theory it should be fine though and just yield a null. I ran it for six cities without issue - but I'll see if I can get a chance to check out Vic later today. Regarding time, it doesn't take days to calculate access - should take under 10 mins for most cities (Auckland was under 6 from memory, I didn't pay close attention to others - but the longer processing was taken in the last round with local neighbourhood statistics --- assuming we all left things running last week?). In any case - we have to run it at some point to finish our analyses really ... And the sooner we attempt to run it for all cities, if there are exceptions that arise, the sooner we'll find them.
…
On Sat, 5 Sep. 2020, 04:54 Shirley Liu, @.> wrote: @gboeing https://github.com/gboeing sure, I try to test run city of Vic from this branch, what I did is: - Download GTFS .gpkg and gtfs_config from Cloudstor in the data/GTFS folder - Re-download the input .gpkg from Cloudstor input folder - Run python setup_config.py - Run python sp.py vic But I got an error: (base) @.:/home/jovyan/work/process# python sp.py vic Global indicators project 2020-09-04 Process city: Vic Read network from disk. Study region sample point output file exists Read poplulation and intersection density from local file. Time taken to calculate or load city local neighbourhood statistics: 0.000550469 mins Calculate assessbility to POIs. Generating contraction hierarchies with 8 threads. Setting CH node vector of size 4844 Setting CH edge vector of size 14080 Range graph removed 14202 edges of 28160 . 10% . 20% . 30% . 40% . 50% . 60% . 70% . 80% . 90% . 100% Calculating nearest node analyses ... - Open street map destinations ['fresh_food_market', 'convenience', 'pt_osm_any'] - Public open space ['public_open_space_any'] ['public_open_space_large'] - Public transport (GTFS) Create hex_id for sample points Creating long form working dataset of sample points to evaluate respective node distances and densities - create long form dataset - join POIs results from nodes to sample points - calculate proximity-weighted average of densitiy statistics for each sample point Calculating sample point specific analyses ... - Best PT (any) access score Traceback (most recent call last): File "sp.py", line 289, in  samplePointsData[var] = samplePointsData[columns].max(axis=axis) File "/opt/conda/lib/python3.7/site-packages/geopandas/geodataframe.py", line 576, in getitem result = super(GeoDataFrame, self).getitem(key) File "/opt/conda/lib/python3.7/site-packages/pandas/core/frame.py", line 2806, in getitem indexer = self.loc._get_listlike_indexer(key, axis=1, raise_missing=True)[1] File "/opt/conda/lib/python3.7/site-packages/pandas/core/indexing.py", line 1553, in _get_listlike_indexer keyarr, indexer, o._get_axis_number(axis), raise_missing=raise_missing File "/opt/conda/lib/python3.7/site-packages/pandas/core/indexing.py", line 1646, in _validate_read_indexer raise KeyError(f"{not_found} not in index") KeyError: "['sp_access_pt_gtfs_any_binary'] not in index" I have not figured out why this occurs... @carlhiggs https://github.com/carlhiggs could you help me take a look if you get a chance? Thanks in advance! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#88 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADDCCMSEO4WDYFIXVOLVPALSEEZWDANCNFSM4QUVUPLQ .

In any case - we have to run it at some point to finish our analyses really ... And the sooner we attempt to run it for all cities, if there are exceptions that arise, the sooner we'll find them.

@carlhiggs Agreed! I misspoke earlier, I've got 6 CPUs available: it's my RAM that's maxed out (95% r.n.) with the current process runs. It should be finished this weekend but in the meantime I'm risk-averse with loading anything else at the same time that could wipe me out (and lose all my current runtime progress). I can run this when that's finished, time permitting (got full time childcare duties the next 2 days)...

Sorry, I accidentally close the PR.. so reopen again
@carlhiggs I try to run Olomouc, and the same error occur, maybe it is because of the null GTFS?
I run Odense, there is another error:
(base) root@b27971c88234:/home/jovyan/work/process# python sp.py odense

Global indicators project 2020-09-05

Process city: Odense

Reproject network, and save the projected network to disk
Create study region sample point output file
Read poplulation and intersection density from local file.
Time taken to calculate or load city local neighbourhood statistics: 0.000598888 mins

Calculate assessbility to POIs.
Generating contraction hierarchies with 8 threads.
Setting CH node vector of size 29523
Setting CH edge vector of size 78638
Range graph removed 80002 edges of 157276
. 10% . 20% . 30% . 40% . 50% . 60% . 70% . 80% . 90% . 100%

Calculating nearest node analyses ...

	- Open street map destinations
		['fresh_food_market', 'convenience', 'pt_osm_any']

	- Public open space
		['public_open_space_any']
		['public_open_space_large']

	- Public transport (GTFS)
		['pt_gtfs_any', 'pt_gtfs_freq_30', 'pt_gtfs_freq_20']
Traceback (most recent call last):
  File "sp.py", line 234, in <module>
    output_prefix = 'sp_nearest_node_')
  File "/home/jovyan/work/process/setup_sp.py", line 356, in cal_dist_node_to_nearest_pois
    gdf_poi_filtered["y"],
  File "/opt/conda/lib/python3.7/site-packages/pandana/network.py", line 551, in set_pois
    node_ids = self.get_node_ids(x_col, y_col)
  File "/opt/conda/lib/python3.7/site-packages/pandana/network.py", line 388, in get_node_ids
    distances, indexes = self.kdtree.query(xys.values)
  File "sklearn/neighbors/_binary_tree.pxi", line 1331, in sklearn.neighbors._kd_tree.BinaryTree.query
  File "/opt/conda/lib/python3.7/site-packages/sklearn/utils/validation.py", line 73, in inner_f
    return f(**kwargs)
  File "/opt/conda/lib/python3.7/site-packages/sklearn/utils/validation.py", line 646, in check_array
    allow_nan=force_all_finite == 'allow-nan')
  File "/opt/conda/lib/python3.7/site-packages/sklearn/utils/validation.py", line 100, in _assert_all_finite
    msg_dtype if msg_dtype is not None else X.dtype)
ValueError: Input contains NaN, infinity or a value too large for dtype('float64').

I finished my cities and uploaded them to the output folder. Carl, could you try these cities, if the issue does not occur at your end, I will re-check if I have missed anything...

Hi Geoff and Shirley --- thanks again for raising this issue, now fixed.  For the case where a city doesn't have an 'nearest node' indicator, when the null variable was created and named, i missed adding the 'sp_nearest_node_' prefix.  I added that in and all good --- Vic processes in 20 seconds successfully, apparently (haven't looked at output for this city though):
(```
base) root@docker-desktop:/home/jovyan/work/process# python sp.py vic
Global indicators project 2020-09-05
Process city: Vic
Read network from disk.
Study region sample point output file exists
Read poplulation and intersection density from local file.
Time taken to calculate or load city local neighbourhood statistics: 0.000304103 mins
Calculate assessbility to POIs.
Generating contraction hierarchies with 6 threads.
Setting CH node vector of size 4844
Setting CH edge vector of size 14080
Range graph removed 14202 edges of 28160
. 10% . 20% . 30% . 40% . 50% . 60% . 70% . 80% . 90% . 100%
Calculating nearest node analyses ...
    - Open street map destinations
            ['fresh_food_market', 'convenience', 'pt_osm_any']

    - Public open space
            ['public_open_space_any']
            ['public_open_space_large']

    - Public transport (GTFS)

Create hex_id for sample points
Creating long form working dataset of sample points to evaluate respective node distances and densities
- create long form dataset
- join POIs results from nodes to sample points
- calculate proximity-weighted average of densitiy statistics for each sample point
Calculating sample point specific analyses ...
- Best PT (any) access score
- Daily living score
- Walkability index
Save to geopackage...
Total time is : 0.20 minutes
(base) root@docker-desktop:/home/jovyan/work/process#

Fair point regarding the processes Geoff ---  given its the weekend (soon for you I think!) no worries leaving this until next week anyway; I expect your cities should be complete in an hour or so then, and I completely understand not wanting to compromise what you're otherwise doing; all good.

That amendment has now been pushed, and I expect should deal with the Olomouc issue.

Catch you all next week, Carl

Thanks Shirley and Geoff - I only saw your comment re: uploading cities
after i posted my update above; i just downloaded all the cities that were
uploaded and will leave these running --- fingers crossed all good, and
we'll see where we're at on Monday!
Have a great weekend,
carl
On Sat, 5 Sep 2020 at 11:23, Shirley Liu<EMAIL_ADDRESS>wrote:

Sorry, I accidentally close the PR.. so reopen again
@carlhiggs https://github.com/carlhiggs I try to run Olomouc, and the
same error occur, maybe it is because of the null GTFS?
I run Odense, there is another error:
(base) root@b27971c88234:/home/jovyan/work/process# python sp.py odense
Global indicators project 2020-09-05
Process city: Odense
Reproject network, and save the projected network to disk
Create study region sample point output file
Read poplulation and intersection density from local file.
Time taken to calculate or load city local neighbourhood statistics: 0.000598888 mins
Calculate assessbility to POIs.
Generating contraction hierarchies with 8 threads.
Setting CH node vector of size 29523
Setting CH edge vector of size 78638
Range graph removed 80002 edges of 157276
. 10% . 20% . 30% . 40% . 50% . 60% . 70% . 80% . 90% . 100%
Calculating nearest node analyses ...

Open street map destinations
['fresh_food_market', 'convenience', 'pt_osm_any']

Public open space
['public_open_space_any']
['public_open_space_large']

Public transport (GTFS)
['pt_gtfs_any', 'pt_gtfs_freq_30', 'pt_gtfs_freq_20']
Traceback (most recent call last):
File "sp.py", line 234, in
output_prefix = 'sp_nearest_node_')
File "/home/jovyan/work/process/setup_sp.py", line 356, in cal_dist_node_to_nearest_pois
gdf_poi_filtered["y"],
File "/opt/conda/lib/python3.7/site-packages/pandana/network.py", line 551, in set_pois
node_ids = self.get_node_ids(x_col, y_col)
File "/opt/conda/lib/python3.7/site-packages/pandana/network.py", line 388, in get_node_ids
distances, indexes = self.kdtree.query(xys.values)
File "sklearn/neighbors/_binary_tree.pxi", line 1331, in sklearn.neighbors._kd_tree.BinaryTree.query
File "/opt/conda/lib/python3.7/site-packages/sklearn/utils/validation.py", line 73, in inner_f
return f(**kwargs)
File "/opt/conda/lib/python3.7/site-packages/sklearn/utils/validation.py", line 646, in check_array
allow_nan=force_all_finite == 'allow-nan')
File "/opt/conda/lib/python3.7/site-packages/sklearn/utils/validation.py", line 100, in _assert_all_finite
msg_dtype if msg_dtype is not None else X.dtype)
ValueError: Input contains NaN, infinity or a value too large for dtype('float64').

I finished my cities and uploaded them to the output folder. Carl, could
you try these cities, if the issue does not occur at your end, I will
re-check if I have missed anything...
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/gboeing/global-indicators/pull/88#issuecomment-687508569,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADDCCMQPEVJ5L7H7SYFQHL3SEGHKDANCNFSM4QUVUPLQ
.

and then you could use bash ./process_regions.sh city city city to iterate over your cities

I don't see any script called process_regions.sh in this branch.

I ran it all manually one city at a time. Appears to work. I append my terminal output below.
I don't see documentation for the changes made in this branch... we'll want to make sure all documentation and instructions are updated prior to merging into master. @nicholas-david can you coordinate with @carlhiggs to add/update the docs for new functionality and usage this coming week?
In the meantime, I'm unclear where the results are being saved and if we should be (re-)uploading data outputs at this point after running the process in this branch? I'll check back Monday if action items remain.
(base) root@0004c2bb7f95:/home/jovyan/work# python setup_config.py
Generate study region configuration json file
Generate cities aggregation configuration json file
All study region configuration file were generated, total time is : 0.00 hours or 0.00 seconds
(base) root@0004c2bb7f95:/home/jovyan/work# python sp.py phoenix true

Global indicators project 2020-09-05

Process city: Phoenix

Read network from disk.
Study region sample point output file exists
Read poplulation and intersection density from local file.
Time taken to calculate or load city local neighbourhood statistics: 0.00121052 mins

Calculate assessbility to POIs.
Generating contraction hierarchies with 12 threads.
Setting CH node vector of size 172056
Setting CH edge vector of size 492562
Range graph removed 506940 edges of 985124
. 10% . 20% . 30% . 40% . 50% . 60% . 70% . 80% . 90% . 100%

Calculating nearest node analyses ...

	- Open street map destinations
		['fresh_food_market', 'convenience', 'pt_osm_any']

	- Public open space
		['public_open_space_any']
		['public_open_space_large']

	- Public transport (GTFS)
		['pt_gtfs_any', 'pt_gtfs_freq_30', 'pt_gtfs_freq_20']
Create hex_id for sample points
Creating long form working dataset of sample points to evaluate respective node distances and densities
	 - create long form dataset
	 - join POIs results from nodes to sample points
	 - calculate proximity-weighted average of densitiy statistics for each sample point
Calculating sample point specific analyses ...
	 - Best PT (any) access score
	 - Daily living score
	 - Walkability index
Save to geopackage...
Total time is : 5.12 minutes
(base) root@0004c2bb7f95:/home/jovyan/work# python sp.py seattle true

Global indicators project 2020-09-05

Process city: Seattle

Read network from disk.
Study region sample point output file exists
Read poplulation and intersection density from local file.
Time taken to calculate or load city local neighbourhood statistics: 0.00243959 mins

Calculate assessbility to POIs.
Generating contraction hierarchies with 12 threads.
Setting CH node vector of size 359111
Setting CH edge vector of size 986596
Range graph removed 1010544 edges of 1973192
. 10% . 20% . 30% . 40% . 50% . 60% . 70% . 80% . 90% . 100%

Calculating nearest node analyses ...

	- Open street map destinations
		['fresh_food_market', 'convenience', 'pt_osm_any']

	- Public open space
		['public_open_space_any']
		['public_open_space_large']

	- Public transport (GTFS)
		['pt_gtfs_any', 'pt_gtfs_freq_30', 'pt_gtfs_freq_20']
Create hex_id for sample points
Creating long form working dataset of sample points to evaluate respective node distances and densities
	 - create long form dataset
	 - join POIs results from nodes to sample points
	 - calculate proximity-weighted average of densitiy statistics for each sample point
Calculating sample point specific analyses ...
	 - Best PT (any) access score
	 - Daily living score
	 - Walkability index
Save to geopackage...
Total time is : 11.37 minutes
(base) root@0004c2bb7f95:/home/jovyan/work# python sp.py mexico_city true

Global indicators project 2020-09-05

Process city: Mexico_City

Read network from disk.
Study region sample point output file exists
Read poplulation and intersection density from local file.
Time taken to calculate or load city local neighbourhood statistics: 0.00295682 mins

Calculate assessbility to POIs.
Generating contraction hierarchies with 12 threads.
Setting CH node vector of size 447873
Setting CH edge vector of size 1225468
Range graph removed 1234136 edges of 2450936
. 10% . 20% . 30% . 40% . 50% . 60% . 70% . 80% . 90% . 100%

Calculating nearest node analyses ...

	- Open street map destinations
		['fresh_food_market', 'convenience', 'pt_osm_any']

	- Public open space
		['public_open_space_any']
		['public_open_space_large']

	- Public transport (GTFS)
		['pt_gtfs_any', 'pt_gtfs_freq_30', 'pt_gtfs_freq_20']
Create hex_id for sample points
Creating long form working dataset of sample points to evaluate respective node distances and densities
	 - create long form dataset
	 - join POIs results from nodes to sample points
	 - calculate proximity-weighted average of densitiy statistics for each sample point
Calculating sample point specific analyses ...
	 - Best PT (any) access score
	 - Daily living score
	 - Walkability index
Save to geopackage...
Total time is : 15.23 minutes
(base) root@0004c2bb7f95:/home/jovyan/work# python sp.py melbourne true

Global indicators project 2020-09-05

Process city: Melbourne

Read network from disk.
Study region sample point output file exists
Read poplulation and intersection density from local file.
Time taken to calculate or load city local neighbourhood statistics: 0.0022413 mins

Calculate assessbility to POIs.
Generating contraction hierarchies with 12 threads.
Setting CH node vector of size 375303
Setting CH edge vector of size 1070182
Range graph removed 1082596 edges of 2140364
. 10% . 20% . 30% . 40% . 50% . 60% . 70% . 80% . 90% . 100%

Calculating nearest node analyses ...

	- Open street map destinations
		['fresh_food_market', 'convenience', 'pt_osm_any']

	- Public open space
		['public_open_space_any']
		['public_open_space_large']

	- Public transport (GTFS)
		['pt_gtfs_any', 'pt_gtfs_freq_30', 'pt_gtfs_freq_20']
Create hex_id for sample points
Creating long form working dataset of sample points to evaluate respective node distances and densities
	 - create long form dataset
	 - join POIs results from nodes to sample points
	 - calculate proximity-weighted average of densitiy statistics for each sample point
Calculating sample point specific analyses ...
	 - Best PT (any) access score
	 - Daily living score
	 - Walkability index
Save to geopackage...
Total time is : 10.68 minutes
(base) root@0004c2bb7f95:/home/jovyan/work# python sp.py sydney true

Global indicators project 2020-09-05

Process city: Sydney

Read network from disk.
Study region sample point output file exists
Read poplulation and intersection density from local file.
Time taken to calculate or load city local neighbourhood statistics: 0.00148019 mins

Calculate assessbility to POIs.
Generating contraction hierarchies with 12 threads.
Setting CH node vector of size 253304
Setting CH edge vector of size 703742
Range graph removed 715770 edges of 1407484
. 10% . 20% . 30% . 40% . 50% . 60% . 70% . 80% . 90% . 100%

Calculating nearest node analyses ...

	- Open street map destinations
		['fresh_food_market', 'convenience', 'pt_osm_any']

	- Public open space
		['public_open_space_any']
		['public_open_space_large']

	- Public transport (GTFS)
		['pt_gtfs_any', 'pt_gtfs_freq_30', 'pt_gtfs_freq_20']
Create hex_id for sample points
Creating long form working dataset of sample points to evaluate respective node distances and densities
	 - create long form dataset
	 - join POIs results from nodes to sample points
	 - calculate proximity-weighted average of densitiy statistics for each sample point
Calculating sample point specific analyses ...
	 - Best PT (any) access score
	 - Daily living score
	 - Walkability index
Save to geopackage...
Total time is : 7.51 minutes
(base) root@0004c2bb7f95:/home/jovyan/work#

Hi Shirley and Geoff -- thanks for running the code also, and for picking up on this error.  This will relate to some bug (yet to fix) in the GTFS analysis code; for some reason, the Odense GTFS stop layer does not have geometries it seems -- and a lot of Nulls --- for example, in the below screenshot, the upper table shoes Odense GTFS stops (nulls for lat and lon, amongst other fields), and the lower shows a more usual output which does have the coordinates required to generate geometries...... so, I'll go back and look at that code which will have given rise to this.  In short, this is a bug outside this code -- however, ideally a more informative error message could be thrown by this code in future (e.g. if no valid geometries were found in input data) which could lead us to find the root cause more rapidly.    I'll try to have a look at this today --- but it seems good news that the process has worked for most other cities

I believe I have found the issue, and its to do with the data not the code per se (although, both code bases could raise better errors, perhaps).  What has happened is, the stop_id in the Danish GTFS stop_times and stops files don't match --- both are interpreted as strings when parsed by UrbanAccess (which is fine, and appropriate), however, because the stop_times text contains a series of zeroes as a prefix, when the data which has been analysed for headways is to be joined with locations defined stops.txt no matches are found --- as per image below.  I think the fix to this is manual cleaning of stop_times.txt to remove the prefix --- or alternately, perhaps they could both be treated as integers which would have the same effect (but that isn't a reliable operation to use on GTFS feeds which may have non-numeric IDs).  I'll update later once I've addressed this, and re-upload the GTFS gpkg

I have now created a fix to the Odense GTFS input data (saving a copy of the original, with a note explaining this manual cleaning) and this should resolve the Odense issue described above.  The GTFS stops for Odense now look like this, which looks right (previously, it lacked geometry due to the mismatch on id described above -- which was of course wrong):

The note I left on the fix, in a file names 'stops_cleaned_readme.txt'  was this:

Stops.txt was modified to fix an issue with zero prefixes to IDs which were not present in stop_times.txt and which prevented match up on ID to join locations to stop_times.
To address this issue, it was first verified that linkage based on interpreting stop_id as a string representation of an integer (which stripped zeroes) worked successfully.  The actual file was modified by opening as a csv file in Microsoft Excel (which implicitly stripped the zeroes from the integers), and then saving, closing, and renaming to .txt.  A copy of the original stops.txt was saved as stops_original.txt.
For example
000008600512 became 8600512, so that it could be joined with stop_id 8600512 in stops_times.txt

Revised data (gpkg and feed) is up on CloudStor, and this appears to work now:

Just confirming that I believe this branch is appropriate for merging into the master --- it contains all the changes that the latest all cities comparisons were run with


Compile GDAL with OpenJPEG driver

Overview
Adds support for JPEG2000 datasets by compiling GDAL with OpenJPEG
driver. Commands are based on these installation instructions:

https://github.com/uclouvain/openjpeg/blob/master/INSTALL.md
https://wiki.orfeo-toolbox.org/index.php/JPEG2000_with_GDAL_OpenJpeg_plugin

Testing
Build Docker image:
$ docker build -t geographica/gdal2:2.2.2 .

Verify that OpenJPEG driver is enabled:
$ docker run --rm -it geographica/gdal2:2.2.2 | grep JPEG
  JPEG -raster- (rwv): JPEG JFIF
  JP2OpenJPEG -raster,vector- (rwv): JPEG-2000 driver based on OpenJPEG library
  JP2OpenJPEG -raster,vector- (rwv): JPEG-2000 driver based on OpenJPEG library

Thanks @kdeloach . I'm going to test it.

It works properly.


how make open world

Hi, i'm using your path tracer and made some changes like move the light by keyboard, and other things, im trying to create a open world, to see how the algorithm (path tracer) change and improves the scene, i remove the walls and ceiling from the depth shader but all the scene turns black and only see the spheres, light and floor. Can you help me please? thx
PD: sorry for my bad english, im from Chile (south america)

Thanks for checking out my path tracer. Unfortunately path tracers don't do so great on open space, they have to have something to bounce off of in order trace light back to the eye.
A larger scene might use a ground plane for the earth and a large hemisphere as the 'sky'. Having every ray hit something greatly increases the chance the eye will reach a light and be able to see. This is why the example scene is a fully enclosed box.

Thanks for the tip. I'm doing a research for path tracing, and i want to do some test scenarios for your path tracer (closed box, open scene, etc). Your path tracer works very well.
I have another doubt, how i can put meshes for create 3d images (for a open world) like mountains, caves , etc. In this example you create the scene in the depth.glsl with a Map Function, but i dont know how to load the meshes from your code. Hope you can resolve my doubt, thanks!

This path tracer is designed to only use signed distance functions for defining the scene. If you wish to use triangles to define your scene then you will want to use some sort of bounding volume hierarchy. An example would be to use an octree to subdivide the triangles in your scene. This allows you to compute ray-triangle intersections at a log base 8 complexity rather than one linear with the number of triangles in your scene.
I recommend checking out this blog for more info: https://www.scratchapixel.com/index.php?redirect

There are ways to make open worlds with signed distance functions, however. There are several good examples on shadertoy, like this one: https://www.shadertoy.com/view/MdX3Rr

Thanks for the link and hello again, i have follow the procedural way. But i have another doubt of your path tracer, you define a map.txt file, for properties material, but i dont know which means each value of the file, for example in the first line we have properties of the line, the 3 first element are RGB , and the others values?, that's my doubt, hope you can help me to know what means each value in the txt file

The material system could be improved. The current format is a vec4 for reflectance, then a vec4 for emittance, on each line.
The reflectance determines what colors are absorbed / reflected when reflecting across the material. For example, if you emit white light at a red surface, all non-red color will be absorbed and turned into heat while red will be reflected and be the visible color of the object.
Emittance determines what spectrum of light an object emits, and what intensity. I keep all these values between 0 and 1 to attempt some conservation of energy.
The w channel in emittance is used as a roughness parameter that determines how coherent a ray is when reflecting off of that material's surface. Higher values will cause the reflected ray to bounce in more random directions while lower values will be more like a pure reflection across a perfect mirror.
https://github.com/gheshu/gputracer/blob/master/src/depth.glsl#L22
https://github.com/gheshu/gputracer/blob/master/src/depth.glsl#L202
https://github.com/gheshu/gputracer/blob/master/src/depth.glsl#L242


Exception on start (connect-cassandra)

What version of the Stream Rector are you reporting this issue for?
kafka-connect-cassandra-1.1.0-1.1.0
Are you running the correct version of Kafka/Confluent for the Stream reactor release?
kafka_2.11-1.1.0
After I added kafka-connect-cassandra-1.1.0-1.1.0-all.jar in libs folder - I get exception:
Exception in thread "main" java.lang.InstantiationError: com.typesafe.scalalogging.Logger 	at com.typesafe.scalalogging.Logger$.apply(Logger.scala:42) 	at kafka.utils.Log4jControllerRegistration$.<init>(Logging.scala:25) 	at kafka.utils.Log4jControllerRegistration$.<clinit>(Logging.scala) 	at kafka.utils.Logging$class.$init$(Logging.scala:47) 	at kafka.Kafka$.<init>(Kafka.scala:32) 	at kafka.Kafka$.<clinit>(Kafka.scala) 	at kafka.Kafka.main(Kafka.scala)

Hi @yrsh.
Can you please clarify where the libs folder sits? (a connector has a specific location it should be added to and in its own folder and set class.path classpath isolation) are you running standalone or distributed Connect? Have you set any classpath upfront running the Connect?

Hi @stheppi!
The lib folder is just 'libs' in the root of kafka i.e. 'kafka_2.11-1.1.0/libs/'. I'm running standalone connect and kafka. No, I didn't set any classpath, just added jar and run kafka.

@yrsh Please use the classloader plugin.path you can find details here Please remember to remove the jar from the other location

@andrewstevenson Thanks

@yrsh If it works please close the issue


Annex sub-clause titles not in the right style

Annex sub-clause headings C.18.1 should be in the correct styling.
P.S. this is from ISO 690.

Introduce noTOC class into ToC generation to allow the option of excluding headers from tables of contents.

Need to define h3Annex, h4Annex, h5Annex, h6Annex in all Word stylesheets

No, they are defined.
What actually needs to happen is: h2, .h2Annex are defined in the stylesheet as having font-weight:bold. h3, .h3Annex are not. Word HTML implicitly adds font-weight:bold to all headers; this needs to be done explicitly to all other heading styles as well, since Word HTML is not doing so for paragraphs styled to look like headings.

Thanks @opoudjis !


switch build to dune

Summary:
remove ocamlbuild in favor of dune
Changelog: [internal]
Reviewed By: SamChou19815
Differential Revision: D41538767

 Duplicate of #


Xcode 16.1 报错无法运行

It is recommended to refer to the following process before submitting issues：https://github.com/didi/DoraemonKit/issues/745
If you still cannot solve your problem, you can submit your issue according to the following template
Please complete the following informations.

Android、iOS? OS version? Brand?
Expected behavior and actual behavior.
Steps to reproduce the problem.
More informations such as error messages and stack traces are welcomed.

建议提issues之前可以参考一下DoKit社区答疑流程：https://github.com/didi/DoraemonKit/issues/745
报错无法编译
请补充如下信息。
Xcode 16.1
Dokit version 3.1.3

:git => 'https://github.com/m8282823/DoKit.git', :commit => 'b8281299b3b59fe1d5cda63bfa3753ac16241aea' 好像没人维护，用我这个吧

优秀 感谢

把DoraemonFileSyncManager.m里的#import <FMDB/FMDB.h>改成#import <FMDB/fmdb/FMDatabase.h>就行了


Fix ReadinessCheckAction

ReadinessCheckAction was incorrectly looking for a list of Service
provided by multibindings. It now looks for a service manager and gets
the services from there.
Also fixes LivenessCheckAction and StatusAction.

Closes #1056

BUILD SUCCESSFUL in 6m 10s
107 actionable tasks: 68 executed, 39 up-to-date


Is worth adding a few lines to handle different image sizes?

Hi,
Not really an issue but when calling Test_cLapIRN.py for inference, if the user gives an input other than the input size you used to train your model it will rise an error.
From your conference paper "We resample all MR scans with isotropic voxel sizes of  $1\ mm^3$ and center-cropped all the pre-processed image scans to 144×192×160".
I added a few lines to reshape and interpolate the deformation field in the test() function. It will preserve the Laplacian of the vector field since I used harmonic interpolation. Let me know if you think is worth it and I will submit a PR.
Thanks!

Hi @ignaciobartol,
Sounds good! Feel free to submit a PR.
I will give it a try.
Thanks a lot.

Thanks! Just opened the PR, feel free to adapt it or not. I have been using it so far to downscale 512x512xM CT scans and it worked pretty well on the registration. Will test on chest CT scans after retraining with the appropriate dataset and will follow up.

Hello,
Just checking in if you had time to review the PR. Thanks.
Best,
Ignacio

It is approved. Sorry for the delay. I overlooked the notification. Thanks for your PR!

Thank you! I see the PR approved.


TST: close mat file, avoiding ResourceWarning

https://circleci.com/gh/scipy/scipy/11263 has a warning that a file is not being closed properly. The warning was supposed to be filtered (but it's not always the case, as shown in the CI output). This PR closes the file properly.

@mattip do you have any ideas of what's going on the pypy3 run?
+++++++++++++++++++++++++++++++++++ Timeout ++++++++++++++++++++++++++++++++++++

~~~~~~~~~~~~~~~~~~~~ Stack of MainThread<PHONE_NUMBER>13696) ~~~~~~~~~~~~~~~~~~~~~
  File "?", line 0, in ?
  File "/home/circleci/project/pypy3.6-latest/site-packages/_pytest/python.py", line 166, in pytest_pyfunc_call
    testfunction(**testargs)
  File "/home/circleci/project/build/testenv/site-packages/scipy/optimize/tests/test__differential_evolution.py", line 535, in test_parallel
    solver.solve()
  File "/home/circleci/project/build/testenv/site-packages/scipy/optimize/_differentialevolution.py", line 683, in solve
    next(self)
  File "/home/circleci/project/build/testenv/site-packages/scipy/optimize/_differentialevolution.py", line 878, in __next__
    trial_energies = self._calculate_population_energies(trial_pop)
  File "/home/circleci/project/build/testenv/site-packages/scipy/optimize/_differentialevolution.py", line 777, in _calculate_population_energies
    parameters_pop[0:nfevs]))
  File "/home/circleci/project/build/testenv/site-packages/scipy/_lib/_util.py", line 414, in __call__
    return self._mapfunc(func, iterable)
  File "/home/circleci/project/pypy3.6-latest/lib-python/3/multiprocessing/pool.py", line 260, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/home/circleci/project/pypy3.6-latest/lib-python/3/multiprocessing/pool.py", line 602, in get
    self.wait(timeout)
  File "/home/circleci/project/pypy3.6-latest/lib-python/3/multiprocessing/pool.py", line 599, in wait
    self._event.wait(timeout)
  File "/home/circleci/project/pypy3.6-latest/lib-python/3/threading.py", line 551, in wait
    signaled = self._cond.wait(timeout)
  File "/home/circleci/project/pypy3.6-latest/lib-python/3/threading.py", line 295, in wait
    waiter.acquire()

~~~~~~~~~~~~~~~~~~~~~ Stack of <unknown><PHONE_NUMBER>89792) ~~~~~~~~~~~~~~~~~~~~~
  File "/home/circleci/project/pypy3.6-latest/site-packages/execnet/gateway_base.py", line 277, in _perform_spawn
    reply.run()
  File "/home/circleci/project/pypy3.6-latest/site-packages/execnet/gateway_base.py", line 213, in run
    self._result = func(*args, **kwargs)
  File "/home/circleci/project/pypy3.6-latest/site-packages/execnet/gateway_base.py", line 954, in _thread_receiver
    msg = Message.from_io(io)
  File "/home/circleci/project/pypy3.6-latest/site-packages/execnet/gateway_base.py", line 418, in from_io
    header = io.read(9)  # type 1, channel 4, payload 4
  File "/home/circleci/project/pypy3.6-latest/site-packages/execnet/gateway_base.py", line 386, in read
    data = self._read(numbytes-len(buf))

~~~~~~~~~~~~~~~~~~~~ Stack of Thread-1254<PHONE_NUMBER>52576) ~~~~~~~~~~~~~~~~~~~~
  File "/home/circleci/project/pypy3.6-latest/lib-python/3/threading.py", line 884, in _bootstrap
    self._bootstrap_inner()
  File "/home/circleci/project/pypy3.6-latest/lib-python/3/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "/home/circleci/project/pypy3.6-latest/lib-python/3/threading.py", line 864, in run
    self._target(*self._args, **self._kwargs)
  File "/home/circleci/project/pypy3.6-latest/lib-python/3/multiprocessing/pool.py", line 367, in _handle_workers
    time.sleep(0.1)

~~~~~~~~~~~~~~~~~~~~ Stack of Thread-1255<PHONE_NUMBER>78400) ~~~~~~~~~~~~~~~~~~~~
  File "/home/circleci/project/pypy3.6-latest/lib-python/3/threading.py", line 884, in _bootstrap
    self._bootstrap_inner()
  File "/home/circleci/project/pypy3.6-latest/lib-python/3/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "/home/circleci/project/pypy3.6-latest/lib-python/3/threading.py", line 864, in run
    self._target(*self._args, **self._kwargs)
  File "/home/circleci/project/pypy3.6-latest/lib-python/3/multiprocessing/pool.py", line 376, in _handle_tasks
    for taskseq, set_length in iter(taskqueue.get, None):
  File "<builtin>/__builtin__/operation.py", line 7, in iter_generator
  File "/home/circleci/project/pypy3.6-latest/lib-python/3/queue.py", line 164, in get
    self.not_empty.wait()
  File "/home/circleci/project/pypy3.6-latest/lib-python/3/threading.py", line 295, in wait
    waiter.acquire()

~~~~~~~~~~~~~~~~~~~~ Stack of Thread-1256<PHONE_NUMBER>85600) ~~~~~~~~~~~~~~~~~~~~
  File "/home/circleci/project/pypy3.6-latest/lib-python/3/threading.py", line 884, in _bootstrap
    self._bootstrap_inner()
  File "/home/circleci/project/pypy3.6-latest/lib-python/3/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "/home/circleci/project/pypy3.6-latest/lib-python/3/threading.py", line 864, in run
    self._target(*self._args, **self._kwargs)
  File "/home/circleci/project/pypy3.6-latest/lib-python/3/multiprocessing/pool.py", line 429, in _handle_results
    task = get()
  File "/home/circleci/project/pypy3.6-latest/lib-python/3/multiprocessing/connection.py", line 250, in recv
    buf = self._recv_bytes()
  File "/home/circleci/project/pypy3.6-latest/lib-python/3/multiprocessing/connection.py", line 407, in _recv_bytes
    buf = self._recv(4)
  File "/home/circleci/project/pypy3.6-latest/lib-python/3/multiprocessing/connection.py", line 379, in _recv
    chunk = read(handle, remaining)

+++++++++++++++++++++++++++++++++++ Timeout ++++++++++++++++++++++++++++++++++++
.[gw1] node down: Not properly terminated
fReplacing crashed worker gw1
gw0 [15606] / gw3 C / gw2 [15606]      .........................ss..... [ 24%]
gw0 [15606] / gw3 ok / gw2 [15606]......................... [ 24%]
........................................................................ [ 24%]

do you have any ideas of what's going on the pypy3 run?

@andyfaff  seems OK now on PyPy too

It's more than a CI thing, the changes ensure that loadmat/savemat/whosmat will never leave a file-handle open.

Yes I've gotten these ResourceWarning's before and had to add an entry to my setup.cfg/pytest.ini file to ignore them (at the risk of missing some internal ones), so I'm excited for this.


Programming Exercise/Enable tutor dashboard for programming exercises

Is your feature request related to a problem?

The tutor course and tutor exercise dashboards can't be used atm for programming exercises.
Describe the solution you'd like

Enable the tutor dashboards for programming exercises.
Related to https://github.com/ls1intum/Artemis/issues/932.
Acceptance criteria:

Show programming exercises in dashboards with setting semi automatic enabled
Tutor exercise dashboard: Get random submission for assessment (compare modeling / text submission)

Out of scope for first iteration:

more feedback, complaints
tutor leaderboard
Your Status column
Assessment Due Date column

Describe alternatives you've considered

Additional context

Dependent of https://github.com/ls1intum/Artemis/pull/930.

Out of scope for first iteration:
more feedback, complaints

Can you give an estimate on when the complaint feature for programming exercises will be implemented and eventually available in the working instance of Artemis?
This has an impact on whether we need to provide some other means for the students to complain about  their tutor assessments.
Related to this, will it be possible for students to complain about those exercise assessments that were done before the feature release?

Complaints by students are implemented. @maxr96 is currently working on the implementation that tutors can respond to these complaints


Documentation update

Can you update this document to reflect the OMS portal through Azure?  The counters are not available in the Azure OMS portal.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: f0549300-a7dd-8325-65a0-fbd27f92d2e8
Version Independent ID: ddaf2e7b-5f5c-82d0-9d46-df8ad699cb93
Content: Monitor Cloud Connector using Operations Management Suite (OMS)
Content Source: Skype/SfbServer/skype-for-business-hybrid-solutions/plan-your-phone-system-cloud-pbx-solution/monitor-cloud-connector-using-operations-management-suite-oms.md
Product: skype-for-business-itpro
GitHub Login: @CarolynRowe
Microsoft Alias: crowe

Thank you for your feedback.  Your request has been forwarded to the product team.


Extract zip/archive files in GCS download

/kind bug
What steps did you take and what happened:
Created InferenceService with storageUri as a zip archive, e.g. gs://bucket/folder/archive.zip - no model files found.
Created InferenceService with storageUri as a folder containing a single archive, e.g. gs://bucket/folder - archive found and downloaded, but not extracted.
Used kserve.storage.storage.Storage._download_gcs directly, with either zip archive or gcs folder containing only a single archive - either downloads nothing, or downloads the zip but doesn't extract it.
What did you expect to happen:
I expected the zip to be downloaded and extracted.
What's the InferenceService yaml:
[To help us debug please run kubectl get isvc $name -n $namespace -oyaml and paste the output]
Anything else you would like to add:
https://github.com/kserve/kserve/pull/1836/files added code which should do this, but there are a couple of problems:

The uri is always used as a prefix, with a trailing slash added - so specifying the uri as gs://bucket/folder/archive.zip results in any blobs with the prefix folder/archive.zip/ being downloaded - there are none.
The folder itself is counted as a blob (I'm not sure if this is specific to the google cloud storage version I'm using - I created a fresh poetry project and just installed kserve (^0.11.2), and it chose google-cloud-storage 1.44.0), so the file_count is incremented for the folder as well which results in file_count of 2, so it doesn't extract the archive. I'm not sure including the folder in the count is the desired behaviour here - possibly this is an indentation issue?

Environment:

Istio Version:
Knative Version:
KServe Version: 0.11.2
Kubeflow version:
Cloud Environment:
Minikube/Kind version:
Kubernetes version: (use kubectl version):
OS (e.g. from /etc/os-release): Debian GNU/Linux 10 (buster)

I think if only files which are actually downloaded (e.g. not 'folders') are counted, it would fix this and not break anything else, so change (from here):
if subdir_object_key.strip() != "" and not subdir_object_key.endswith("/"):
        dest_path = os.path.join(temp_dir, subdir_object_key)
        logging.info("Downloading: %s", dest_path)
        blob.download_to_filename(dest_path)
    file_count += 1

to
if subdir_object_key.strip() != "" and not subdir_object_key.endswith("/"):
        dest_path = os.path.join(temp_dir, subdir_object_key)
        logging.info("Downloading: %s", dest_path)
        blob.download_to_filename(dest_path)
        file_count += 1

As far as I can see, that would be safe.

Hi @gfkeith , feel free to create a PR to fix it, you can refer to S3 as well.
so far from y understanding the behaviour in KServe is gs://bucket/folder/archive.zip will be downloaded and unzipped (single zip file).


The Failed tests should be red

This was brought up during a review with the #SML crew.

Anyone downloading the PBI template could change the colors. We should consider the accessibility issues with Red/Green


modify k8s sample to expect injected env vars

Description
Modify the hello-k8s sample to expect only injected env vars for Dapr ports.
Added a simple step in the README to illustrate the same.
This is a breaking change to this sample.
Merge only after merging https://github.com/dapr/dapr/pull/1712
Issue reference
We strive to have all PRs being opened based on an issue, where the problem or feature have been discussed prior to implementation.
Please reference the issue this PR will close: https://github.com/dapr/dapr/issues/1637
Checklist
Please make sure you've completed the relevant tasks for this PR, out of the following list:

[x] The sample code compiles correctly
[x] You've tested new builds of the sample if you changed sample code
[x] You've updated the sample's README if necessary

@orizohar Can you please review this pr ?

@yaron2 @amanbha Can you please review this PR? It is good to show how we're populating env vars but only documentation would be enough to explain it.

@yaron2 @amanbha Can you please review this PR? It is good to show how we're populating env vars but only documentation would be enough to explain it.

but I am fine with having the info in both in samples and in docs repos


Won't recognize that messages aren't backed up

I got SMS Backup+ working using a custom email address run through my web host, since the Gmail steps quit working and I wasn't willing to use the security settings required these days.
It has been working great, until.... apparently I had limited space in that email address and hadn't noticed. I fixed the limited space problem, but for the past month it hasn't been receiving my backup attempts.
Expected behaviour
SMS Backup+ should recognize that the messages are not stored in the email and back them up, if not automatically, at least on a manual backup.
Actual behaviour
It didn't.  I reset and tried a backup from only the most important contact I wanted to back up, it said it was backing up some 20k messages, but apparently decided they'd all been backed up as it went through them in only a minute or two and no new messages are stored in my email.
Steps to reproduce the behaviour
Try to back up messages on a full email account. Make the email account not full. Try again.
Please specify the following:

5.0.1
Samsung Galaxy S4 / SCH-I545PP
SMS Backup+ version installed: 1.5.11
Messaging app: Verizon Message +

Try "clear app data", which will reset the app's idea of what the last-backed-up message was.
(Sometimes this gets out of synch, especially if the clock in your device is wrong.)

Thanks for the suggestion. That phone is unfortunately no longer functional so I won't be able to try it, my new phone is iOS.

Thanks for letting us know.


Allow running changed packages on target branches

I'm not sure whether this fits with where the maintainers see Circletron going, but I thought I'd open a PR to see what you all think.
Right now, every change to a target branch triggers all jobs across all packages. My feeling is that as a user of the orb, I should be able to set a configuration flag to only run jobs against changed packages.

I'm not sure whether this fits with where the maintainers see Circletron going, but I thought I'd open a PR to see what you all think.
Right now, every change to a target branch triggers all jobs across all packages. My feeling is that as a user of the orb, I should be able to set a configuration flag to only run jobs against changed packages.

The problem here is this solution gets the "commit before last", but unless the merge to the target branch was only one commit, it will miss changes from the branch that was merged. Definitely for merges via rebase, and I think maybe this command also reaches into the commits of the merge rather than getting the commit before the branch.

The problem here is this solution gets the "commit before last", but unless the merge to the target branch was only one commit, it will miss changes from the branch that was merged. Definitely for merges via rebase, and I think maybe this command also reaches into the commits of the merge rather than getting the commit before the branch.

Very good point. I've replaced the git command in getLastCommitOnBranch with git log HEAD@{1} -1, which should cover merge via rebase. I also added some error handling so that if no previous commit is found, all pipelines are run - I believe this is necessary to handle the first commit on a target branch.

@Josh-ES git log HEAD@{1} -1 doesn't work for me, at least not on any test I ran on my local git repository. Is it something specific to do with how circleci performs the checkout?

@insidewhy I was thinking about this a bit more deeply, and I've realised that we don't just want to get the last commit before the merge. We probably want to get the last commit before the merge that built successfully in Circle.
I've updated this implementation to use the Circle API (disappointingly, this is a little convoluted with Circle CI's API, requiring multiple calls) to try to find the git revision corresponding to the last successful build on the current branch. I think this should satisfy both of our concerns about finding the commit before a merge and excluding commits where the build failed.
Let me know your thoughts!

@Josh-ES Thanks for this work, it's really interesting. I'm gonna get a few more of my colleagues to review it and then merge it if everything looks good.

@Josh-ES Also, do you want me to push an orb release with the dev:next tag so that you can test this out?

@Josh-ES I've merged this now and released circletron 3.0.3 with the changes. Please try it out and let me know if it's working.

@insidewhy many thanks for merging and releasing, I appreciate it and will give it a test when I next get the chance in a few hours. Would you like me to circle back and address some of the above comments that appear to be unresolved?

@Josh-ES I think we fixed them all.

@insidewhy Great! S Sounds good. I've tested this, both prior to the change with my own published version of the orb and with the new version and everything's working as intended.


Ability to add arbitrary data to Widget Definition Objects

Hello,
I think my problem is simple. I have added a property to my Widget Definitions called 'gadgetConfigured'.
I simply want to expose 'widget.gadgetConfigured' to the main Dashboard.html template using the ng-if directive as follows :

which should then either "remove or recreate the portion of the DOM" as per Angular (https://docs.angularjs.org/api/ng/directive/ngIf)
However the behavior is inconsistent. More specifically, when gadgetConfigured is set to true, then my widget content is NOT rendered. It's totally blank.
Here's the 'chart_area' widget def in the WidgetDefinitions factory:
    { // either one will work: template (directly embed the html) or templateUrl (embed the html file path)
          name: 'chart_area',
          title: 'Area Chart',
          templateUrl: 'app/shared/gadget-templates/chart-area.html',
          size: size,
          dataModelType: ColumnChartDataModel,
          initImage: 'chart_area.jpg',
          gadgetConfigured: true,
      }

and here's an image showing how I am debugging in my gadgetModel.js, within the init() function. To the right you can see gadgetConfigured is true.

Perhaps I'm missing something, or my use of ng-if is wrong; but I'm banging my head wondering where I'm going wrong.
thanks in advance,
Bob

Hey @robertmazzo,
Figured out your issue, and am considering making some changes to the tool because of it. Currently, widgets do not inherit arbitrary data from their widget definition objects, only  whitelisted properties. But that code is messy and could be cleaned up significantly.
I think your use-case is valid enough for me to change the library and support arbitrary data on widget definition objects. Hang tight, I'll try and churn something out this weekend.

@robertmazzo I have created a new milestone called v1.0.0, and I'll try to address this issue there. I am going to rename this if you dont mind so that it works better as a milestone item.

Relates to #62

Thank you Andy for your attention to this.
Best regards
Bob

Hi Andy,
You mentioned that only whitelisted properties would get inherited by the widgets.
So if I add a couple of properties to the WidgetModel as such, should this work ?
 function WidgetModel(Class, overrides) {
      var defaults = {
          title: 'Dashboard Rept',
          name: Class.name,
          attrs: Class.attrs,
          dataAttrName: Class.dataAttrName,
          dataModelType: Class.dataModelType,
          dataModelArgs: Class.dataModelArgs, // used in data model constructor, not serialized
          //AW Need deep copy of options to support widget options editing
          dataModelOptions: Class.dataModelOptions,
          settingsModalOptions: Class.settingsModalOptions,
          onSettingsClose: Class.onSettingsClose,
          onSettingsDismiss: Class.onSettingsDismiss,
          style: Class.style || {},
          size: Class.size || {},
          enableVerticalResize: (Class.enableVerticalResize === false) ? false : true,
          initImage: Class.initImage,    // added new initial image to display - BM:
          gadgetConfigured: Class.gadgetConfigured
      };

(sorry for the delayed response) Yea this should work

Andy,
I've had problems trying to show/hide the widget-content div using ng-if, however I'll keep experimenting with this:

I'll close the ticket in the meantime.
thanks

Hm, not sure what could be going wrong here. Again, this is something I will (hopefully) be taking up very soon here.

actually reopening so this remains an issue to address for the v1.0.0 milestone

realized this was duplicate of #62


Get TRT running in Conda L4T

With the success of TensorRT optimisation, and significantly less overhead than the PyTorch requirements, we can try getting TRT using a conda env from the dockerfile.base and install Pycuda + scipy (the only TRT requirements)

Forgot that this L4T runs python3.6 not python3.8 which conda runs in rip
might need more digging eventually

https://forums.developer.nvidia.com/t/how-to-support-python3-8/168813
https://arcanesciencelab.wordpress.com/2020/05/24/building-python-3-8-3-on-jetpack-4-4-dp-and-the-jetson-xavier-nx/

This done?

No its cooked, proprietary nvidia container stuff for python tensorrt. can just pip install tensorrt alongside pycuda. might have to run in it's container not a conda env


Updating of Developer Guide

see #58 changes for (partial) implementation of analyses feature.

Done!


[BUG] verifyPresentation doesn't verify VCs inside VP

Current Behavior
verifyPresentation verifies the outer VP but not the VCs inside
Expected Behavior
My understanding is that VP verification includes verification of VP plus VCs contained in it. The VC-DATA-MODEL is not clear on this, but this is the convention that we've established based on common interpretation/expectation. Documented here: https://github.com/w3c-ccg/vc-http-api/blob/main/verification.md
Failure Information
verifyPresentation succeeds when VP is wrapping a known failing VC
Steps to Reproduce
Please provide detailed steps for reproducing the issue.

Created an expired VC, e.g.
const expiredVc = 'eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2MjYyMDgzNTIsInZjIjp7IkBjb250ZXh0IjpbImh0dHBzOi8vd3d3LnczLm9yZy8yMDE4L2NyZWRlbnRpYWxzL3YxIl0sInR5cGUiOlsiVmVyaWZpYWJsZUNyZWRlbnRpYWwiXSwiY3JlZGVudGlhbFN1YmplY3QiOnsiZGVncmVlIjp7InR5cGUiOiJCYWNoZWxvckRlZ3JlZSIsIm5hbWUiOiJCYWNjYWxhdXLDqWF0IGVuIG11c2lxdWVzIG51bcOpcmlxdWVzIn19fSwic3ViIjoiZGlkOmV0aHI6MHg0MzVkZjNlZGE1NzE1NGNmOGNmNzkyNjA3OTg4MWYyOTEyZjU0ZGI0IiwibmJmIjoxNjI2MjA4MzQyLCJpc3MiOiJkaWQ6a2V5Ono2TWtzR0toMjNtSFp6MkZwZU5ENld4SnR0ZDhUV2hrVGdhN210Yk0xeDF6TTY1bSJ9.n0Cko-LZtZjrVHMjzlMUUxB6GGkx9MlNy68nALEeh_Doj42UDZkCwF872N4pVzyqKEexAX8PxAgtqote2rHMAA'

Confirm it fails verifyCredential, as expected

Wrap that in a VP and sign it with createVerifiablePresentationJwt, e.g. as in index.test.ts

const presentationPayload = {
  vp: {
    '@context': [DEFAULT_CONTEXT, EXTRA_CONTEXT_A],
    type: [DEFAULT_VP_TYPE],
    verifiableCredential: [ expiredVc],
  },
}

Note that verifyPresentation succeeds whereas verifyCredential failed

Workarounds Considered
Callers can call verifyCredential on each VC in the presentation, but I wanted to discuss and document this, since this may be a common expectation, and would lead to unexpected behavior (possible security issues)

Thanks for reporting this.
This is somewhat intentional. The main reason for this behavior is that the embedded credentials in a VP are not limited to JWT format (that I'm aware of). Since this library doesn't have the ability to verify credentials in other kinds of formats it is left to the application to do the verification.
You are right that this aspect should be better documented.


Having an issue deploying to DO Droplet

I'm having an issue getting the production Admin page working on a Digital Ocean Droplet. Docker seems to be building it fine, and I can access /strapi without issue, but when I try to access /strapi/admin I either get a blank white screen, or a darker screen with a toast notification stating there was an issue communicating with the API.
Any ideas?

I have the same issue, anyone?


Why finetune can done on one DGX

In pretrain, paper said use 512 V100, but finetune can be done on single DGX?
Do some pruning or distillation made,
What makes this difference?
And what parallelization you use in training?

The pretraining is totally data parallel, so you can decrease batch size for finetuning.

thank you


Parametrize websocket api tests for docker

Without this the two tests (parametrized via this PR), fail when running the tests with docker.

Codecov Report

Merging #1593 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #1593   +/-   ##
=======================================
  Coverage   98.44%   98.44%
=======================================
  Files          58       58
  Lines        2828     2828
=======================================
  Hits         2784     2784
  Misses         44       44

You can add a "9985" over here too to expose the websocket port.

Well, since we use the default port, I thought we did not need to include it.


Add a dtrace probe that sends FileSystemRead,Write events

Closes https://github.com/tonarino/acoustic_profiler/issues/1

Thank you @goodhoko. I did one more iteration on this PR to do necessary prep work for adding timestamp-support for dtrace events. Can you please take another look?

The probe crashes very often withError:
   0: OperationError("Abort due to drop")
Location:
   crates/dtrace_probe/src/main.rs:33

I haven't seen this error myself, but I will keep an eye 👀

I haven't seen this error myself, but I will keep an eye 👀

Yeah, let's see. I only hit it when probing some more complex programs like eg. Notion. I'll try to experiment some more with the latest version of this code.


pc console show count of query results

add total records count
should fix https://github.com/openreview/openreview-web/issues/582

I like the idea of showing the count in the Export button, but could you attach a screenshot of how it looks?

I like the idea of showing the count in the Export button, but could you attach a screenshot of how it looks?

the change has been made to the pc console of neurips, so you can use neurips pc console to see it


Add more information about BrowserLaunchMode.SystemPreferred falling back to BrowserLaunchMode.External

Type of issue
Missing information
Description
This page says:

It's recommended to use the SystemPreferred as it is the default and gracefully falls back if needed.

Please add information about when/why this happens or how one might determine why it's happened when using the following in an app.
Uri uri = new Uri("https://www.microsoft.com");
await Browser.Default.OpenAsync(uri, BrowserLaunchMode.SystemPreferred);

Page URL
https://learn.microsoft.com/en-us/dotnet/api/microsoft.maui.applicationmodel.browserlaunchmode?view=net-maui-8.0
Content source URL
https://github.com/dotnet/dotnet-maui-api-docs/blob/live/docs/xml/Microsoft.Maui.ApplicationModel/BrowserLaunchMode.xml
Document Version Independent Id
d8afd574-a0b6-86bc-eaeb-70696d79c5e4
Article author
@dotnet-bot
Metadata

ID: d8afd574-a0b6-86bc-eaeb-70696d79c5e4
Service: dotnet-mobile
Sub-service: dotnet-maui

I'm not really sure what you are asking? You mean how to detect that it didn't use the system preferred launch mode and how to handle that?
In either case, I think this is more something that should be described in the conceptual docs, not the API docs.

Hi @jfversluis, the purpose of this issue is to be able to find out why it falls back. Currently, there is no information provided when it happens, at least that I can find.
I opened the issue on this page since it's the only place that mentions the fall-back behavior. If it needs to be opened in the MAUI docs instead, is it possible for this issue to be migrated there?

Ah OK! Looking at the code I actually can't really see what is meant by this statement. I don't see any evidence of .NET MAUI doing anything special in case it fails. If that even is the case, then that probably happens on the operating system level which is just echoed by our documentation here.
Sorry that I can't really clear it up for you. I will see if I can make the API docs here a bit more precise by either clarifying the why or how or remove this statement. Thanks for raising this!


Rework vendoring of vectorscan to use patchfiles

vectorscan is currently vendored, built from source as part of Nosey Parker. For the sake of expediency I implemented that by adding a copy of all vectorscan 5.4.8 in a subdirectory, and then modifying those in-place.
It would be better for maintainability if the vendored build process instead extracted a pristine shasum-verified release tarball and applied patchfiles to that prior to building. This will make it clearer in the future what modifications were made to the Vectorscan sources.
Additionally, being able to get rid of the enormous vectorscan and boost source trees from this repository would make continued development of Nosey Parker a bit nicer, as things like fzf-based fuzzy search and whole-source-tree textual search wouldn't get confounded by all the stuff in there (which we mostly don't care about).

Using patchfiles would be cleaner, but at this point is a bunch of work for little payoff. Won't fix. Can re-evaluate if it becomes an issue in the future.


Add base balance checks to exchange tests.

Changes

Added base balance and incoming capacity checks to the funding test
Tests assume that the fee reported to the user is a maximum that could be spent.

A fundamental problem in ethereum makes it hard to calculate what will be spent on fees up front.

Added a baseLayerBalance function to the top level api.

Issues

Funding test is big and unpleasant. Tried refactoring but couldn't come up a with a satisfactory solution without spending a lot of time.

Rebased. Also, I implemented getBaseBalance for the lnd settler by using walletbalance lnd grpc method. Even though we aren't using it now, seems better to have it from an interface perspective.

Codecov Report

Merging #20 into master will decrease coverage by 1.22%.
The diff coverage is 79.16%.

@@            Coverage Diff             @@
##           master      #20      +/-   ##
==========================================
- Coverage   91.14%   89.92%   -1.23%
==========================================
  Files          16       16
  Lines         610      625      +15
  Branches       29       30       +1
==========================================
+ Hits          556      562       +6
- Misses         46       54       +8
- Partials        8        9       +1

Impacted Files
Coverage Δ

src/index.ts
93.44% <ø> (ø)
:arrow_up:

src/settlement/machinomy.ts
96.77% <100%> (+4.83%)
:arrow_up:

src/settlement/xrp-paychan.ts
94.66% <100%> (+0.3%)
:arrow_up:

src/settlement/lnd.ts
90.56% <25%> (-5.36%)
:arrow_down:

src/uplink.ts
89.28% <83.33%> (-1.2%)
:arrow_down:

src/services/switch.ts
81.33% <0%> (-8%)
:arrow_down:

src/utils/middlewares.ts
88.33% <0%> (-1.67%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cbb8c37...8e66f5c. Read the comment docs.

LGTM


feat(keychain)!: remove atom factory

Removes atom factory from keychain feature package. This now lives in @exodus/keychain-atoms. Reasons explained in https://github.com/ExodusMovement/exodus-hydra/pull/5486 as well:

If keychain ends up being extracted to a "wallet process", this makes sense to be extracted so the application side does not need to import keychain (import still required for keys, but that might also be convenient to change)
Keychain is the only feature that holds atom factories. We separated fusion ones out of fusion on @exodus/fusion-atoms, so this follows the same pattern
All atom factories close to each other

@mvayngrib CI fails for not found actions. Are you able to merge in the meantime?

@diegomura please rebase

@mvayngrib done!


[Question/Feedback]:  Notification assets naming

Check for previous/existing GitHub issues

[X] I have checked for previous/existing GitHub issues

Description
Hello.
The new naming pattern for notification assets seems to be :

[concat('ag-AMBA-', subscription().displayName, '-001')]
[concat('ag-AMBA-SH-', subscription().displayName, '-001')]
[concat('apr-AMBA-',subscription().displayName, '-001')]

Is this something we can assume to be used in the future? What is the "001" suffix for? Right now it is hard coded... Is it simply to ensure uniqueness of the asset name at the time of the first deployment or will it be auto-incremented in the future based on some condition(s)?
The reason I am asking is because I am thinking of creating a bicep that re-deploys these groups and adds email recipients to them based on which subscription the group is located in. My expectation is that I will end up with the same action groups containing only the recipients that are relevant for each particular landing zone. I believe this should work, but I must be sure that the action groups I deploy and the action groups AMBA deploys match, hence this question 😊|.
Thank you!

Hello @alexeyzolotukhin ,
thanks for your feedback. The reason why we started using that naming convention is to align with the Abbreviation recommendations for Azure resources and the '-001' at the end is to make sure we can create additional resources with similar name maintaining, as you well spotted, the uniqueness of the asset name. For instance, we now have a suppression alert processing rule which has a similar name but ends with '-002'.
Thanks,
Bruno.

Hello @alexeyzolotukhin ,
I haven't heard anything back from you regarding this issue. Assuming that it can be considered resolved, I am gonna close it as completed. Feel free to reopen it or to create a new one should it be the case.
Thanks,
Bruno.


Errors on Fennel files; need to update queries

I'm getting invalid node type errors from nvim-treesitter-textobjects after opening Fennel files. I think this is because the new queries from https://github.com/nvim-treesitter/nvim-treesitter/pull/6132, which changed the Fennel grammar, need to be pulled in. I could potentially do this-- is it just a matter of merging in the new queries?

Actually, never mind-- it turns out NixOS just has a slightly older version of the plugin without commit 3532f60.


[MRG] Fixing serialization of SleepPhysionet

This PR fixes two minor bugs with the SleepPhysionet class that I came across with while running some tests for #325. First, the exclude attribute of the Raw object was a dict_keys object, which prevented the dataset from being serialized. Second, channels were renamed using mne.rename_channels instead of using Raw's rename_channels method, which led to .ch_names and ._orig_units having different channel names. This created an issue when applying the filterbank preprocessor.

great