Ubuntu based podvm image creation failed in gh-action

Following is the error from the job
#13 [podvm_builder 7/9] RUN tar xvf /src/cloud-api-adaptor/podvm/files/pause-bundle.tar.gz -C /src/cloud-api-adaptor/podvm/files
#0 0.050 pause_bundle/
#0 0.051 pause_bundle/umoci.json
#0 0.052 pause_bundle/config.json
#0 0.052 pause_bundle/sha256_253878249d7aef127ed0e137781059d2973f4037d9385de3b87ea82d224c3089.mtree
#0 0.053 pause_bundle/rootfs/
#0 0.053 pause_bundle/rootfs/pause
#13 DONE 0.1s

#14 [podvm_builder 8/9] COPY .dummy files/*.crt /src/cloud-api-adaptor/podvm/files/
#14 ERROR: lstat /tmp/buildkit-mount25[273](https://github.com/confidential-containers/cloud-api-adaptor/actions/runs/5005607649/jobs/8971106443#step:5:277)41817/files: no such file or directory
------
 > [podvm_builder 8/9] COPY .dummy files/*.crt /src/cloud-api-adaptor/podvm/files/:
------
Dockerfile.podvm:61
--------------------
  59 |     # Use one .dummy file at here to avoid `COPY failed: no source files were specified` for non-SE build
  60 |     # https://forums.docker.com/t/copy-only-if-file-exist/3781
  61 | >>> COPY .dummy files/*.crt /src/cloud-api-adaptor/podvm/files/
  62 |
  63 |     RUN cd cloud-api-adaptor/podvm && \
--------------------
ERROR: failed to solve: lstat /tmp/buildkit-mount2527341817/files: no such file or directory
Error: buildx failed with: ERROR: failed to solve: lstat /tmp/buildkit-mount2527341817/files: no such file or directory

@liudalibj can you please take a look

podvm image build in  github action is using docker buildx so we meet this issue.
Try use one dummy.crt file to avoid the build break. Test locally with docker buildx:
 root@peerpod-se-image-vsi:~/cloud-api-adaptor/podvm# /usr/bin/docker buildx build --build-arg CLOUD_PROVIDER=generic --build-arg ARCH=s390x --build-arg UBUNTU_IMAGE_URL= --build-arg UBUNTU_IMAGE_CHECKSUM= --build-arg BINARIES_IMG=quay.io/liudalibj/podvm-binaries-ubuntu-s390x --build-arg BUILDER_IMG=quay.io/liudalibj/podvm-builder-ubuntu --file Dockerfile.podvm --platform linux/amd64 --provenance false --tag quay.io/liudalibj/podvm-generic-ubuntu-s390x:latest --push .
[+] Building 687.7s (20/20) FINISHED
 => [internal] load .dockerignore                                                                                0.0s
 => => transferring context: 2B                                                                                0.0s
 => [internal] load build definition from Dockerfile.podvm                                                                                0.0s
 => => transferring dockerfile: 2.58kB                                                                                0.0s
 => [internal] load metadata for quay.io/liudalibj/podvm-builder-ubuntu:latest                                                                                1.2s
 => [internal] load metadata for quay.io/liudalibj/podvm-binaries-ubuntu-s390x:latest                                                                                1.2s
 => [auth] liudalibj/podvm-builder-ubuntu:pull token for quay.io                                                                                0.0s
 => [auth] liudalibj/podvm-binaries-ubuntu-s390x:pull token for quay.io                                                                                0.0s
 => [internal] load build context                                                                                0.0s
 => => transferring context: 89B                                                                                0.0s
 => [podvm_binaries 1/1] FROM quay.io/liudalibj/podvm-binaries-ubuntu-s390x@sha256:d27235e27581a4e5c42ddd598945265e812b633e19cca9e42a4c161f80bb4b92                                                      0.0s
 => => resolve quay.io/liudalibj/podvm-binaries-ubuntu-s390x@sha256:d27235e27581a4e5c42ddd598945265e812b633e19cca9e42a4c161f80bb4b92                                                                     0.0s
 => CACHED [podvm_builder 1/9] FROM quay.io/liudalibj/podvm-builder-ubuntu@sha256:235bce82e5898217c1bf982631b5953dd31bf411712bbe7855cdd45e9e3e5295                                                       0.0s
 => => resolve quay.io/liudalibj/podvm-builder-ubuntu@sha256:235bce82e5898217c1bf982631b5953dd31bf411712bbe7855cdd45e9e3e5295                                                                            0.0s
 => [podvm_builder 2/9] RUN if [ -n "" ]; then       rm -rf cloud-api-adaptor &&       git clone  cloud-api-adaptor;    fi &&     if [ -n "" ]; then       cd cloud-api-adaptor &&       git fetch orig  0.2s
 => [podvm_builder 3/9] RUN ./cloud-api-adaptor/podvm/hack/cross-build-extras.sh                                                                                15.4s
 => [podvm_builder 4/9] COPY --from=podvm_binaries /podvm-binaries.tar.gz /src/cloud-api-adaptor/podvm/files                                                                                0.5s
 => [podvm_builder 5/9] RUN tar xvf /src/cloud-api-adaptor/podvm/files/podvm-binaries.tar.gz -C /src/cloud-api-adaptor/podvm/files                                                                       1.4s
 => [podvm_builder 6/9] COPY --from=podvm_binaries /pause-bundle.tar.gz /src/cloud-api-adaptor/podvm/files                                                                                0.0s
 => [podvm_builder 7/9] RUN tar xvf /src/cloud-api-adaptor/podvm/files/pause-bundle.tar.gz -C /src/cloud-api-adaptor/podvm/files                                                                         0.1s
 => [podvm_builder 8/9] COPY files/*.crt /src/cloud-api-adaptor/podvm/files/                                                                                0.0s
 => [podvm_builder 9/9] RUN cd cloud-api-adaptor/podvm &&      make BINARIES= PAUSE_BUNDLE= image                                                                                583.7s
 => [stage-2 1/1] COPY --from=podvm_builder /src/cloud-api-adaptor/podvm/output/*.qcow2 /                                                                                0.5s
 => exporting to image                                                                                81.3s
 => => exporting layers                                                                                31.0s
 => => exporting manifest sha256:d2322a730ef22b1e8a8a262d663d0394baacb3fb594fc0ca215ee58665698901                                                                                0.0s
 => => exporting config sha256:5805dd2c0fdc0017243e2af5100c55f954fadf617c778da67d0d7cdb4081ce62                                                                                0.0s
 => => pushing layers                                                                                49.9s
 => => pushing manifest for quay.io/liudalibj/podvm-generic-ubuntu-s390x:latest@sha256:d2322a730ef22b1e8a8a262d663d0394baacb3fb594fc0ca215ee58665698901                                                  0.4s
 => [auth] liudalibj/podvm-generic-ubuntu-s390x:pull,push token for quay.io                                                                                0.0s
root@peerpod-se-image-vsi:~/cloud-api-adaptor/podvm#

@bpradipt I tried to re-run the failed job but they are failed again, as the GHA using old commit,  would you like help to re-tag base on the fix code and verify the fix?

Issue is fixed now.


Pup/5033

Any idea why it's failing Windows CI? It wasn't obvious to me from a quick glance.

The reason AppVeyor failed the pops/validator tests is that they introduces an order dependent problem with the with_app_management(true) call in a before(:all) declaration. It needs a corresponding after(:all) that turns app management off again.
This problem is not specific to Windows. I can reproduce it both on windows and on Fedora. Can be tested by running:
bundle exec rspec spec/unit/pops/validator/validator_spec.rb # Works OK
bundle exec rspec spec/unit/pops # Fails!

This PR should contain a test that verifies that a new Resource::Type declared with consumes/produces end up with blueprints in their respective consumes/produces arrays.
I think would be good to also add a test to check that the "produces/consumes clause references nonexistent type XXX" error is triggered correctly.

Here's what I've changed

addressed the smaller comments around the handling of with_app_management
added a test spec/unit/capability_spec.rb to the last commit that does the various checks against PopsBridge suggested above
rebased to the latest master

Changes in the latest patch series:

Fixed with_app_management to reach into E4ParserAdapter
Dropped the patch that changes $produces to $makes in evaluating_parser_spec as that's not needed as long as app_management is feature-flagged
Rebased to latest HEAD on master


Can the code developed by d435 be directly replaced by d455

When developing, it uses d435. Now I need to change it to d455. Do I need to change any code?

Hi @liu-jiaming  If you use the minimum FPS speed of 6 FPS then you should change it to 5 FPS, as 6 is not supported on D455.  Apart from that, you should not need to change any code.

Hi @liu-jiaming  Do you require further assistance with this case, please?  Thanks!

@liu-jiaming Also keep in mind the the RGB camera of the D455 has a maximum resolution of 1280x720 and does not support 1920x1080.

Case closed due to no further comments received.


fix: remove matchedDN and diagnosticMessage from SearchResponseEntry

Hi,
Firstly, thanks for this library !!
This PR removes matchedDN and diagnosticMessage from SearchEntryResponse because they should not be found here.

SearchResultEntry ::= [APPLICATION 4] SEQUENCE {
     objectName      LDAPDN,
     attributes      PartialAttributeList }

PartialAttributeList ::= SEQUENCE OF
       partialAttribute PartialAttribute

LDAPDN ::= LDAPString
           -- Constrained to  [RFC4514]

LDAPString ::= OCTET STRING -- UTF-8 encoded,
              -- [ISO10646] characters

PartialAttribute ::= SEQUENCE {
     type       AttributeDescription,
     vals       SET OF value AttributeValue }

AttributeDescription ::= LDAPString
           -- Constrained to
           -- [RFC4512]

AttributeValue ::= OCTET STRING

source: https://ldap.com/ldapv3-wire-protocol-reference-search/,

If there's no rush and you don't mind: I'll take a look after the holiday.

Yes, of course! 😄
Happy holidays and Merry Christmas!

@xunleii I know things are always busy after the holidays, but I'm just following up about rebasing this so we can merge it.

@xunleii I know things are always busy after the holidays, but I'm just following up about rebasing this so we can merge it.

Hi, sorry, I've seen your message but totally forgot just after.
I'll try to do that as soon as possible (this night or tomorrow)
Thanks for the reminder 😄


NTR to HP: Barking cough

https://github.com/obophenotype/human-phenotype-ontology/issues/5894

This term has been assigned the ID HP:0033121.


refactor: tests

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 70.31%. Comparing base (6ef9b32) to head (47d310e).

@@                    Coverage Diff                     @@
##           chungquantin/feat-nfts     #405      +/-   ##
==========================================================
- Coverage                   70.44%   70.31%   -0.13%
==========================================================
  Files                          70       70
  Lines                       13109    13055      -54
  Branches                    13109    13055      -54
==========================================================
- Hits                         9234     9179      -55
  Misses                       3603     3603
- Partials                      272      273       +1

Files with missing lines
Coverage Δ

pallets/nfts/src/tests.rs
99.90% <100.00%> (-0.01%)
:arrow_down:

... and 1 file with indirect coverage changes


DynamoDB update item with condition

If one tries to update an item that is not already present with condition expression, it will mostly likely fail with a ValidationException. This is due to bad comparison, for example between:
'<' not supported between instances of 'NoneType' and 'int'

This seems to be related to a bad check of attribute_not_exists or the right hand part of the OR being executed even though the left hand part was true. For reference my condition:
(attribute_not_exists(LastUpdate) OR LastUpdate < :minDate)

You can test with:
from datetime import datetime, timezone

table = dynamo_resource.Table(table_name)
table.update_item(
                Key={"ID": "56789"},
                UpdateExpression=(
                    "SET "
                    "OwnerID = :ownerId, "
                    "LastUpdate = :now "
                ),
                ConditionExpression=(
                    "(attribute_not_exists(OwnerID) OR OwnerID = :ownerId) OR "
                    "(attribute_not_exists(LastUpdate) OR LastUpdate < :minDate)"
                ),
                ExpressionAttributeValues={
                    ":ownerId": "12345",
                    ":now": int(datetime.now(tz=timezone.utc).timestamp() * 1e9),
                    ":minDate":<PHONE_NUMBER>239243008,
                },
                ReturnValues="ALL_NEW",
            )

I identified the bug in the OR comparison
class OpOr(Op):
    OP = 'OR'

    def expr(self, item):
        lhs = self.lhs.expr(item)
        rhs = self.rhs.expr(item) # HERE
        return lhs or rhs

We should not try to interpret rhs if lhs is true. I will make the change.

Please close when the release is done, thanks for the work!

We have a continuous release process so you should be able to use this now: https://pypi.org/project/moto/#history


Julia packages build errors while following the install_env_gail_full.md

I used julia for the first time and had problems when go through the install_env_gail_full.md.
My system is ubuntu 16.04.
I simply went down the procedures and was not able to build the packages after running the following command.
julia ngsim_env/julia/deps/build.jl

The errors are
=======================[ ERROR: AutomotiveDrivingModels ]=======================

LoadError: ArgumentError: Module Pkg not found in current path.
Run `Pkg.add("Pkg")` to install the Pkg package.
while loading /home/yychen/.julia/v0.6/AutomotiveDrivingModels/deps/build.jl, in expression starting on line 1

================================================================================

================================[ BUILD ERRORS ]================================

WARNING: AutomotiveDrivingModels had build errors.

 - packages with build errors remain installed in /home/yychen/.julia/v0.6
 - build the package(s) and all dependencies with `Pkg.build("AutomotiveDrivingModels")`
 - build a single package by running its `deps/build.jl` script

================================================================================
INFO: Building AutoViz
===============================[ ERROR: AutoViz ]===============================

LoadError: ArgumentError: Module Pkg not found in current path.
Run `Pkg.add("Pkg")` to install the Pkg package.
while loading /home/yychen/.julia/v0.6/AutoViz/deps/build.jl, in expression starting on line 1

================================================================================

================================[ BUILD ERRORS ]================================

WARNING: AutoViz had build errors.

 - packages with build errors remain installed in /home/yychen/.julia/v0.6
 - build the package(s) and all dependencies with `Pkg.build("AutoViz")`
 - build a single package by running its `deps/build.jl` script

================================================================================
ERROR: LoadError: LightGraphs is not an installed package

Also there are warnings:
julia is fixed at 0.6.2 conflicting with requirement for BayesNets: [0.7.0,∞)

Exception when cloning https://github.com/sisl/BayesNets.jl.git: Base.Pkg.PkgError("fixed packages introduce conflicting requirements for LightGraphs:  BayesNets requires versions [1.0.0,∞) [none of the available versions can satisfy this requirement] ")

Exception when cloning https://github.com/sisl/NGSIM.jl.git: Base.Pkg.PkgError("fixed packages introduce conflicting requirements for LightGraphs:  BayesNets requires versions [1.0.0,∞) [none of the available versions can satisfy this requirement] ")

Exception when cloning https://github.com/sisl/AutoRisk.jl.git: Base.Pkg.PkgError("fixed packages introduce conflicting requirements for LightGraphs:  BayesNets requires versions [1.0.0,∞) [none of the available versions can satisfy this requirement] ")

After I tried every method, I finally reached the error:
ERROR: LoadError: DataArrays is not an installed package

If I add the package through Pkg.add("DataArrays"), I can see several warnings that  showed that DataArrays.jl, BayesNets, LightGraphs and  Inflate only support the julia version [0.7.0, ∞)
and error showed that
Inflate can't be installed because it has no versions that support 0.6.2 of julia.

I guess it is due to the update of the related packages. Can anyone give any suggestions, or details like the version of the BayesNets and other related packages? Or can anyone solve the problem from the beginning?

I also saw the BayesNets warning and had to install v0.7.0. After that things started working but there were a lot other changes I had to make to install_env_gail_full.md to finish.
I'm on Mac OS 10.14 and I've never used Julia before today so very possible not all of this is necessary or is specific to my system but it worked for me until I got to some compilation issues with the actual code in AutomotiveDrivingModels (see my next comment).
Note that v0.7.0 doesn't use juliarc.jl for setup but looks in ~/.julia/config/startup.
Here are a bunch of things I had to do that were different or not written in install_env_gail_full.md, maybe some of them apply to your case.
Get AutomotiveDrivingModels into ~/.julia/dev

git clone https://github.com/sisl/AutomotiveDrivingModels.jl.git
cd AutomotiveDrivingModels.jl
julia deps/build.jl

Run julia interpreter and run Pkg.add("Iterators"); don't add Iterators to startup.jl, that caused a lot of problems for me until I removed it
On Mac libz.so is libz.dylib so find it and add to LD_PRELOAD
Weird code changes that fixed compilation errors

remove semi-colon in src/2d/roadway/roadways.jl:591
remove comment block in src/2d/roadway/roadway_generation.jl:111-123

I had a lot of weird issues with Pkg.build("HDF5"), don't remember them all but follow the errors.
Pkg.add("***") followed by using *** for each of the following packages (I'm sure there is proper dependencies way of doing this but I'm very new to Julia and couldn't figure out how, adding them or their git repo's to REQUIRE or build.jl didn't work)

DataStructures
Parameters
StatsBase

This fixed a lot of build issues I was having but now I'm stuck on compilation error in AutomotiveDrivingModels, see next comment.

We have also been encountering related errors this Fall - it seems ever since julia v1.0 came out, so its not too surprising - but we are not necessarily sure about the best way to fix them besides just updating everything to v1.0 which we frankly do not have the bandwidth for at the moment. Another alternative would be to host a docker image or something like that somewhere, with the versions we use.
It seems like, from that error, some of the packages dropped support for julia0.6, which is the one we first were developing on.  Any help is appreciated :)

@djp42, thanks for the response. I am adapting the current installation procedure for one that works with julia v0.7.0. A docker image of a working installation is a good idea, we may go that route and then will share.
I managed to get everything installed and was on the NGSIM data part but I get the following error when runningconvert_raw_ngsim_to_trajdatas(), any thoughts?
ERROR: MethodError: no method matching abs2(::Vec.VecSE2{Float64})
Closest candidates are:
  abs2(::Missing) at missing.jl:79
  abs2(::Bool) at bool.jl:92
  abs2(::Real) at number.jl:157
  ...
Stacktrace:
 [1] index_closest_to_point(::Array{AutomotiveDrivingModels.CurvePt,1}, ::Vec.VecSE2{Float64}) at /Users/sb/.julia/dev/AutomotiveDrivingModels/src/2d/roadway/curves.jl:73

I've managed to get through all of the ngsim_env package installations and the last step using AutoEnvs now works.
Now I'm stuck on the NGSIM data part. When I run convert_raw_ngsim_to_trajdatas() I get this:
converting ~.julia/dev/NGSIM/data/i101_trajectories-0750am-0805am.txt
loading from file: elapsed time: 9.651778409 seconds
filtering:         elapsed time: 15.076234596 seconds

But it just sits at a blank line after filtering while running CPU @ 99.9%. I waited 1.5h and no change, no output. It enters load_ngsim_trajdata() because shows "filtering" output but print statements inside the or carid in carid_set(tdraw) never print.
Is this expected?

Hmm, never come across that issue before. Have you checked your memory usage while running it? We had other issues with out of memory errors at some points in the past.
Glad to see you got the packages working though, and thank you for recording the steps!

It seems to be iterating through the for loop in Base.convert() now, just taking very long running locally on my machine. Seems to have been related to package load in julia (I still don't understand it well). Once I restarted the interpreter it got to the convert() call for the first file at least.
I jumped ahead to the python tests and am getting segfaults. I don't expect them to pass but pretty sure the segfaults are not related to needing the trajectory data? I tried removing the PyCall.jl file as recommended by the instructions but the only one I have is in ~/.julia/packages/PyCall/.../src and when I removed that I just get PyCall not found errors. I also tried removing ~/.julia/compiled/v0.7/PyCall/***.jl but that didn't help. Maybe I need to remove something else? Or the entire PyCall package? Keep in mind I'm using julia v0.7.0 so slightly different directory structure than v0.6.2 that the instructions assume.
Thanks very much for responses, I get you guys don't have a lot of bandwidth for this right now. We are planning a Docker container once we get can get to the end and train and validate a policy. Any help is greatly appreciated though!

Hmm, I do remember it taking a good bit of time but not an absurd amount. And the good news is you only have to do it once.
For the segfaults, do you have the log for what the error looked like? I remember once encountering non-pycall segfaults, but it wasn’t too complicated to fix.
Also regarding pycall, if the cache doesn’t exist or whatever, just remove that PyCall.jl and try to reinstall it, i believe through another Pkg.add() command. Alternatively, just remove and reinstall the actual PyCall package.
This may help with clearing the cache, although as a disclaimer I have never run the code there so I do not know that it works. https://discourse.julialang.org/t/how-do-you-clear-cache-from-a-precompile/7821/2
Let me know if that helps, and I will do my best to keep up-to-date with this thread, I appreciate your patience, and look forward to the docker image :). I am hopeful we will get a v1.0 up and running in the near future as well..

I deleted the directories ~/.julia/compiled/v0.7/PyCall and ~/.julia/packages/PyCall and then ran Pkg.add("PyCall"); it re-created and ran build on it but still getting the segfaults. Here's the first one (there is one for each test, all similar):
(rllab3) (anaconda2-5.2.0) reba:tests sb$ python runtests.py
test_julia_env (test_julia_env.TestJuliaEnv) ... WARNING: `JULIA_HOME` environment variable is renamed to `JULIA_BINDIR`

signal (11): Segmentation fault: 11
in expression starting at no file:0
jl_apply_tuple_type_v_ at /Users/osx/buildbot/slave/package_osx64/build/src/jltypes.c:1326 [inlined]
jl_apply_tuple_type_v at /Users/osx/buildbot/slave/package_osx64/build/src/jltypes.c:1341
arg_type_tuple at /Users/osx/buildbot/slave/package_osx64/build/src/gf.c:1654
jl_lookup_generic_ at /Users/osx/buildbot/slave/package_osx64/build/src/gf.c:2149
jl_apply_generic at /Users/osx/buildbot/slave/package_osx64/build/src/gf.c:2179
do_call at /Users/osx/buildbot/slave/package_osx64/build/src/interpreter.c:324
eval_stmt_value at /Users/osx/buildbot/slave/package_osx64/build/src/interpreter.c:363 [inlined]
eval_body at /Users/osx/buildbot/slave/package_osx64/build/src/interpreter.c:686
jl_interpret_toplevel_thunk_callback at /Users/osx/buildbot/slave/package_osx64/build/src/interpreter.c:799
unknown function (ip: 0xfffffffffffffffe)
unknown function (ip: 0x1149ed0ef)
unknown function (ip: 0xffffffffffffffff)
jl_interpret_toplevel_thunk at /Users/osx/buildbot/slave/package_osx64/build/src/interpreter.c:808
jl_toplevel_eval_flex at /Users/osx/buildbot/slave/package_osx64/build/src/toplevel.c:831
jl_toplevel_eval_flex at /Users/osx/buildbot/slave/package_osx64/build/src/toplevel.c:778
jl_toplevel_eval_in at /Users/osx/buildbot/slave/package_osx64/build/src/builtins.c:633
eval at ./boot.jl:319
exec_options at ./logging.jl:317
_start at ./client.jl:432
true_main at /usr/local/bin/julia (unknown line)
main at /usr/local/bin/julia (unknown line)
Allocations: 4642 (Pool: 4632; Big: 10); GC: 0
ERROR
test_multiagent_ngsim_env (test_multiagent_ngsim_env.TestMultiagentNGSIMEnv) ... WARNING: `JULIA_HOME` environment variable is renamed to `JULIA_BINDIR`

Never seen that segfault before. It might be something related to macOS, we have done all the development exclusively in Ubuntu.
My suggestion would be to look for others with issues of julia segfaulting on macOS.
This thread was one of the few I saw that included jlytpes in the segfault: https://groups.google.com/forum/#!topic/julia-users/UxaeV1YhshQ
It might not be related to OS, just a thought.

@djp42 Thanks for keeping along. I'm running into some serious version conflicts between v0.7.0 julia which I started along and some non SISL packages (currently wit JLD). Lot's of packages ngsim depends on have moved along in development and broke compatibility with v0.6 julia at some point. It's kind of whack0-a-mol.
Is there any chance you can share the Manifest and Project toml files for your ngsim installation that works? Doesn't even matter what julia version - with the Manifest I could get all the exact git commits that worked then and exactly re-create the installation.

@sblinz  no problem, I want to get this to work too :).
This is the only toml file I have:
Project.toml.zip. I will update tomorrow with if someone else's repo has others.
What Manifest files do you want? These are what shows up for a manifest search:
./miniconda3/pkgs/qt-5.6.2-5/mkspecs/common/winrt_winphone/manifests/10.0/AppxManifest.xml.in
./miniconda3/pkgs/qt-5.6.2-5/mkspecs/common/winrt_winphone/manifests/8.1/AppxManifest.xml.in
./miniconda3/pkgs/qt-5.6.2-5/mkspecs/common/winrt_winphone/manifests/8.1_wp/AppxManifest.xml.in
./miniconda3/pkgs/qt-5.6.2-h974d657_12/mkspecs/common/winrt_winphone/manifests/10.0/AppxManifest.xml.in
./miniconda3/pkgs/qt-5.6.2-h974d657_12/mkspecs/common/winrt_winphone/manifests/8.1/AppxManifest.xml.in
./miniconda3/pkgs/qt-5.6.2-h974d657_12/mkspecs/common/winrt_winphone/manifests/8.1_wp/AppxManifest.xml.in
./miniconda3/envs/rllab3/mkspecs/common/winrt_winphone/manifests/10.0/AppxManifest.xml.in
./miniconda3/envs/rllab3/mkspecs/common/winrt_winphone/manifests/8.1/AppxManifest.xml.in
./miniconda3/envs/rllab3/mkspecs/common/winrt_winphone/manifests/8.1_wp/AppxManifest.xml.in
./.julia/v0.6/Conda/deps/usr/pkgs/qt-5.6.2-h974d657_12/mkspecs/common/winrt_winphone/manifests/10.0/AppxManifest.xml.in
./.julia/v0.6/Conda/deps/usr/pkgs/qt-5.6.2-h974d657_12/mkspecs/common/winrt_winphone/manifests/8.1/AppxManifest.xml.in
./.julia/v0.6/Conda/deps/usr/pkgs/qt-5.6.2-h974d657_12/mkspecs/common/winrt_winphone/manifests/8.1_wp/AppxManifest.xml.in
./.julia/v0.6/Conda/deps/usr/mkspecs/common/winrt_winphone/manifests/10.0/AppxManifest.xml.in
./.julia/v0.6/Conda/deps/usr/mkspecs/common/winrt_winphone/manifests/8.1/AppxManifest.xml.in
./.julia/v0.6/Conda/deps/usr/mkspecs/common/winrt_winphone/manifests/8.1_wp/AppxManifest.xml.in

@djp42 The Manifest I mean is part of the Julia installation, it's a manifest for the different Julia packages that are installed. For v0.6 it should be in something like ~/.julia/environments/v0.6/Manifest.toml (mine is in this path with v0.7). Your manifest files are for specific packages.
Here is a snippet of my ~/Users/.../.julia/enviroments/v0.7/Manifest.toml
[[AssetRegistry]]
deps = ["Distributed", "JSON", "Pidfile", "Pkg", "SHA", "Test"]
git-tree-sha1 = "b25e88db7944f98789130d7b503276bc34bc098e"
uuid = "bf4720bc-e11a-5d0c-854e-bdca1663c893"
version = "0.1.0"

[[AutoRisk]]
deps = ["Compat", "DataArrays", "DataFrames", "Discretizers", "Distributions", "HDF5", "Reexport"]
path = "/Users/.../.julia/dev/AutoRisk"
uuid = "e0b28f5a-3ea4-5d34-bf3f-495f78123ff5"
version = "0.0.0"

The key thing is that ordinary installed packages (e.g., the first one here) list the git commit that that package is checked out at (for ordinary packages this would have been the most recent commit at the time it was added; by now it would be an old commit). For development packages (like AutoRisk above), there is no git commit listed but if you go to the path (e.g., ~/Users/.../.julia/dev/AutoRisk) and run git status you can get the commit. These are packages that you essentially move to the dev/ folder so you can change them (e.g., check out other commits or just modify the code). The ngsim_env installation has a few like this.
If I can get my hands on such a manifest file from a working installation of ngsim_dev I can at least get all the non-dev package commits that work together (e.g., BayesNets, JLD, etc). For the dev one's I can probably figure them out myself as the main one's are already listed in the instructions.
Thanks

I checked with my other teammate and he also does not have the Manifest file. It might be something only for Julia 0.7+ (I saw something similar when Googling it). I'm beginning to think we may want to try another approach... We will potentially be able to spend some more time updating and/or docker filing after Nov 12 or so, but no promises there. Currently we are trying to copy the .julia of a working install to other computers and see if it works (we have had issues doing so over the past week or two). But even if it does, it might not for you because of MacOS vs Linux.

Thanks for checking. Yeah, the Mac + v0.7 julia approach is not going to happen, too many conflicting parts, so I'm going to try it from scratch in a Docker Linux container with v0.6. I'll let you know of my progress, hopefully I can get it working before mid November.

Development: I have successfully gotten past the first few phases of issues on a blank Ubuntu 16.04 LTS.
Steps: download and extract the zip files for .julia (very large. Has all of the packages).
Follow the install steps, with some exceptions:

Before the ngsim_env steps, there is a super hacky step, to avoid path issues. Basically, we create a symbolic link in /home/ that has my username on the working install dphillips and links it to your home directory. I didn't have enough time to find where to actually fix the paths, but on using PyCall and using Conda there were path issues. Fixing the Conda issue was just to run this command: export CONDA_JL_HOME="/<path/to>/rllab3", but didnt have time to figure out the other one. The hacky solution is: sudo ln -s /home/<your_username>/ /home/dphillips
I don't think we need to revert the previous versions.
I didn't get further than the using PyCall call, but I think that's pretty good for now.

Not sure if that is applicable since you're on Mac, but I wanted to share with you in case you wanted to try your hand at it.
Thanks!

@djp42 Thanks for the notes. I started from scratch inside a Docker container running Ubuntu 16.04. I was able to get through all the setup instructions through the julia tests julia ngsim_env/julia/test/runtests.jl.
Note: I had to manually check out some older versions of several julia packages to get through all the steps inside julia ngsim_env/julia/deps/build.jl but I was able to get to the end of it and manually build AutomotiveDrivingModels, AutoViz, and BayesNets. I also got through using PyCall, using PyPlot, using AutoEnvs and already had the trajectory data converted and the Julia tests ran without error (though a few warnings).
But, I'm getting some weird issues trying to run the Python tests (this is where I got stuck on mac btw). When I run python runtests.py I get this error which I cannot figure out. It happens from the call self.j = julia.Julia() inside ngsim_env/python/julia_env/julia_env.py. In fact, I get the same error if I just go into the ordinary python interpreter and execute import julia and then julia.Julia(). From the error it looks like maybe some path issue with the Python julia module trying to find python inside the julia Conda package (Python running julia running wrong python...). I tried your path changes above and also tried going into the julia interpreter first and setting ENV["PYTHON"] = <path to my conda rllab3 python> (the same python I call the test with since I have the conda rllab3 environment activated when running the test).
Any thoughts?
Fatal Python error: initsite: Failed to import the site module
Traceback (most recent call last):
  File "/root/.julia/v0.6/Conda/deps/usr/lib/python3.7/site.py", line 168, in addpackage
    exec(line)
  File "<string>", line 1, in <module>
  File "/root/.julia/v0.6/Conda/deps/usr/lib/python3.7/importlib/util.py", line 14, in <module>
    from contextlib import contextmanager
  File "/root/.julia/v0.6/Conda/deps/usr/lib/python3.7/contextlib.py", line 5, in <module>
    from collections import deque
  File "/root/.julia/v0.6/Conda/deps/usr/lib/python3.7/collections/__init__.py", line 24, in <module>
    import heapq as _heapq
  File "/root/.julia/v0.6/Conda/deps/usr/lib/python3.7/heapq.py", line 587, in <module>
    from _heapq import *
SystemError: initialization of _heapq did not return an extension module

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/root/.julia/v0.6/Conda/deps/usr/lib/python3.7/site.py", line 570, in <module>
    main()
  File "/root/.julia/v0.6/Conda/deps/usr/lib/python3.7/site.py", line 557, in main
    known_paths = addsitepackages(known_paths)
  File "/root/.julia/v0.6/Conda/deps/usr/lib/python3.7/site.py", line 349, in addsitepackages
    addsitedir(sitedir, known_paths)
  File "/root/.julia/v0.6/Conda/deps/usr/lib/python3.7/site.py", line 207, in addsitedir
    addpackage(sitedir, name, known_paths)
  File "/root/.julia/v0.6/Conda/deps/usr/lib/python3.7/site.py", line 178, in addpackage
    import traceback
  File "/root/.julia/v0.6/Conda/deps/usr/lib/python3.7/traceback.py", line 3, in <module>
    import collections
  File "/root/.julia/v0.6/Conda/deps/usr/lib/python3.7/collections/__init__.py", line 24, in <module>
    import heapq as _heapq
  File "/root/.julia/v0.6/Conda/deps/usr/lib/python3.7/heapq.py", line 587, in <module>
    from _heapq import *
SystemError: initialization of _heapq did not return an extension module

Turns out that error is solved by removing ~/.julia/lib/v0.6/PyCall.jl - missed that at first because it wasn't a segfault. Now all Python tests are passing except test_vectorized_ngsim_env because I haven't installed hgail yet.

@djp42 I got to the HGAIL tests and had 1 failure out of 43 tests. Any thoughts on this?
Btw, I am writing a Dockerfile as I go but also have the container so can commit an image at the end with a working version. Almost there it seems.
======================================================================
FAIL: test_train_domain_matters (recognition.test_domain_adversarial_recognition_model.TestRecognitionModel)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/hgail/tests/recognition/test_domain_adversarial_recognition_model.py", line 172, in test_train_domain_matters
    self.assertTrue(np.abs(domain_probs[n_samples][0] - .5) < .1)
AssertionError: False is not true

----------------------------------------------------------------------

I made it through to the end of install_env_gail_full.md!
Oddly, the hgail tests hgail/tests/runtests.py have different outcomes each time I run them. I had 1 failure when I ran it the 1st time but when I reran they all passed. I was curious so reran a few more times and it kept alternating passing and failing 1 test. The failed runs only reported 1 failed test but strangely it wasn't always the same test.
Here's one test that failed though the first run it was a different test that failed.
======================================================================
FAIL: test_hgail_two_round_stochastic_env (algos.test_hgail_impl.TestHGAIL)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/hgail/tests/algos/test_hgail_impl.py", line 206, in test_hgail_two_round_stochastic_env
    np.testing.assert_array_almost_equal(l0_dist_2, [[1,0]], 1)
  File "/root/miniconda3/envs/rllab3/lib/python3.5/site-packages/numpy/testing/utils.py", line 962, in assert_array_almost_equal
    precision=decimal)
  File "/root/miniconda3/envs/rllab3/lib/python3.5/site-packages/numpy/testing/utils.py", line 778, in assert_array_compare
    raise AssertionError(msg)
AssertionError:
Arrays are not almost equal to 1 decimals

(mismatch 100.0%)
 x: array([[ 0.8,  0.2]], dtype=float32)
 y: array([[1, 0]])

----------------------------------------------------------------------

AWESOME!! We also have made it to the end on our new install! That is interesting about the non-deterministic hgail tests. I will ask someone that is more familiar with those than I am. It might just be stochasticity, or some package having a different precision, etc. Flaky tests are generally bad anyway...
TL;DR, let me get back to you.

That's great. I'm getting a Dockerfile ready so hopefully will share that soon. I seem to be stuck on the very last part of the installation:
julia extract_ngsim_demonstrations.jl runs for a long time and keeps showing counts like frame **** / 9500.
But, it gets killed for some reason each time. For example it'll show frame 5737 / 9452Killed and return me back to my terminal prompt. It may be a Docker thing but I can't figure it out so far.
Since this step looks like it's supposed to create a file ngsim_env/data/trajectories/ngsim_all.h5 but that folder is still empty for me I'm guessing I can't go on to "Train and run a single agent GAIL policy".
Would it be enough if you just gave me a copy of ngsim_all.h5 or does this script to more than that?
I'm anxious to get on to training and running the gail algorithm.

That is interesting that it dies. It is fine if it takes a long time to run, that is expected.
I put the .tar.gz in this google drive folder. See if that helps, and let me know of any more problems.

Thanks, that's really helpful. Nice to have the julia packages as well. One question: trajectories folder has ngsim_all.h5 (14.19GB) and ngsim.h5 (1.15GB) but last line of instructions has cp ngsim_all.h5 ngsim.h5...are they supposed to be the same or different?
Update on Dockerfile progress: I was able to get julia packages working by manually cloning and checking out different versions through an interactive terminal to a running docker container. As I went I wrote out corresponding Dockerfile commands but when I tried to build that Dockerfile there were package conflicts. The order of julia package cloning and building is really sensitive. So, either I mixed up steps when writing the Dockerfile or the way Docker builds is a little different than manually calling the commands (I know there are some things around shells and sub-processes and layers when you execute RUN commands in Dockerfile, maybe related to that).

Regarding the ngsim_all vs ngsim, I believe the ngsim_all contains trajectories from different ngsim datasets that we do not really care about. When I was looking up for how they were used it seemed that they were used pretty similarly. To that end, I would suggest just using them as they appear, but if you are interested in testing the differences, order of operations would be to copy the small one and rename it to ngsim_all, but we have done this before and so no difference. The ngsim_all file is a relic of an older setup process.
Regarding Docker, I have never used it successfully, and I understand it has a very steep learning curve.   Sorry I can't be more help there.

Great, thank you. I'm now running the example training python imitate.py --exp_name NGSIM-gail --n_itr 1000 --policy_recurrent True from the README. It's running inside a Docker container which is nice, averaging about 30s / iteration so looking like ~8 hours.
I'm planning to put together a repo that will have some of the data that takes a really long time to produce (NGSIM trajdata, ngsim_env trajectories) and a Dockerfile so that others can go strait to training and running policies. Will keep you posted.

Great, definitely keep me updated. I think I'll keep this issue open until I have time to go through and update the README with new instructions / link to files or the docker.

@djp42 I was able to successfully run python imitate.py --exp_name NGSIM-gail --n_itr 1000 --policy_recurrent True and ngsim_env/data/experiments/NGSIM-gail/imitate/log has the last 100 itr_***.npz files as well as args.npz and log.txt.
I was also able to generate a simulation video from this trained policy based on these instructions.
But, when I went to run python validate.py --n_proc 5 --exp_dir ../../data/experiments/NGSIM-gail/ --params_filename itr_1000.npz --random_seed 42 I get this odd error. It happens even if I include --debug true (which calls validate.py::single_process_collect_trajectories() instead of parallel_collect_trajectories()).
Any thoughts?
pid: 1 traj: 0 / 2062Traceback (most recent call last):
  File "validate.py", line 381, in <module>
    random_seed=run_args.random_seed
  File "validate.py", line 266, in collect
    random_seed=random_seed
  File "validate.py", line 226, in single_process_collect_trajectories
    random_seed
  File "validate.py", line 135, in collect_trajectories
    env_kwargs=dict(egoid=egoid, start=starts[egoid])
  File "validate.py", line 31, in simulate
    a, a_info = policy.get_action(x)
  File "/home/hgail/hgail/policies/gaussian_latent_var_gru_policy.py", line 193, in get_action
    return actions[0], {k: v[0] for k, v in agent_infos.items()}
  File "/home/hgail/hgail/policies/gaussian_latent_var_gru_policy.py", line 193, in <dictcomp>
    return actions[0], {k: v[0] for k, v in agent_infos.items()}
KeyError: 0

@sblinz I had the same error before. The error was because the agent _infos dictionary contains the 'latent' key. You can fix it by replacing the get_action function in  gaussian_latent_var_gru_policy.py by following codes:
    def get_action(self, observation):         actions, agent_infos = self.get_actions([observation])         infos = dict()         for k, v in agent_infos.items():             if k is 'latent_info':                 infos[k] = dict()                 infos[k]['latent'] = v['latent'][0]             else:                 infos[k] = v[0]         # return actions[0], {k: v[0] for k, v in agent_infos.items()}         return actions[0], infos

I'm definitely encountering the same problem about the version conflict of Julia. Is the Dockerfiles for ngsim ready?

Code is now working with julia 1.1. Has not been merged into master yet as some tests are pending. Please see this issue to make the code work with Julia1.1.

Thanks. But it seems to fail to add the local package AutoEnvs to Julia environment by
]
~/ngsim_env/Julia
it will produce the following feedback
Updating registry at ~/.julia/registries/General  Updating git-repohttps://github.com/JuliaRegistries/General.git`
Cloning git-repo ~/ngsim_env/julia
ERROR: failed to clone from ~/ngsim_env/julia, error: GitError(Code:ERROR, Class:Net, unsupported URL protocol)`
By the way, in the branch 07_fixes_in_progress, there's no folder "AutoEnvs" in "ngsim_env/Julia


Handle all download errors in the asset library

Closes #31115.
Also, reordered the cases in that switch to better match the order defined in HTTPRequest.

Thanks!

Should the error messages be more descriptive?
For example:
error_text = TTR("Cannot save response to") + " " + download->get_download_file();
doesn't point towards how to fix the error. Maybe something along the lines of:
error_text = TTR("Cannot save response to") + " " + download->get_download_file() + "\n" + TTR("Make sure the cache folder is writable.")
This might give users a chance of finding the cause of the problem without having to open an issue.

@nleehone most Linux users should be able to recognize write errors as permission problems without too much additional hints. If you feel like it would be a quality addition, though, feel free to open a PR! :smiley:


fix: dependencies

Description
Streamlines dependencies.

delete setup.py - only use pyproject.toml for easier maintainability
add pip install embedchain[opensource] for the open source app
raise custom error

Potential problems:

the open source dependencies are huge. Since we run poetry install --all-extras in the CI script, this might make CI slow, on my local machine it takes around 20 minutes, in the worst case it might be too much for the free tier of github. In this case we might have to change the CI script.

Fixes #392
Fixes #335
Type of change
Please delete options that are not relevant.

[X] Bug fix (non-breaking change which fixes an issue)
[x] Refactor (does not change functionality, e.g. code style improvements, linting)

How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration
Please delete options that are not relevant.

[X] Unit Test
[X] Test Script (please provide)

from embedchain import OpenSourceApp

app = OpenSourceApp()
app.add("text", "lorem ipsum")

Checklist:

[X] My code follows the style guidelines of this project
[X] I have performed a self-review of my own code
[X] I have commented my code, particularly in hard-to-understand areas
[X] I have made corresponding changes to the documentation
[X] My changes generate no new warnings
[X] I have added tests that prove my fix is effective or that my feature works
[X] New and existing unit tests pass locally with my changes
[X] Any dependent changes have been merged and published in downstream modules
[X] I have checked my code and corrected any misspellings

Maintainer Checklist

[ ] closes #xxxx (Replace xxxx with the GitHub issue number)
[ ] Made sure Checks passed

Looks good to me. @cachho : can you fix the conflict? can merge post that.


Exposing Map of field renames in Transformer

Codecov Report

Merging #29 (9537cff) into master (b28c448) will decrease coverage by 0.11%.
The diff coverage is 98.18%.

:exclamation: Current head 9537cff differs from pull request most recent head a0b9190. Consider uploading reports for the commit a0b9190 to get more accurate results

@@            Coverage Diff             @@
##           master      #29      +/-   ##
==========================================
- Coverage   99.60%   99.48%   -0.12%
==========================================
  Files          26       27       +1
  Lines         752      773      +21
  Branches       81       80       -1
==========================================
+ Hits          749      769      +20
- Misses          3        4       +1

Impacted Files
Coverage Δ

.../main/scala/io/scalaland/chimney/Transformer.scala
66.66% <0.00%> (-33.34%)
:arrow_down:

.../scalaland/chimney/internal/utils/MacroUtils.scala
100.00% <ø> (ø)

.../scala/io/scalaland/chimney/TransformerUtils.scala
100.00% <100.00%> (ø)

...a/io/scalaland/chimney/internal/macros/Model.scala
100.00% <100.00%> (ø)

...laland/chimney/internal/macros/PatcherMacros.scala
96.15% <100.00%> (+0.15%)
:arrow_up:

...nd/chimney/internal/macros/TransformerMacros.scala
99.39% <100.00%> (+0.04%)
:arrow_up:

...ain/scala/io/scalaland/chimney/ErrorPathNode.scala
100.00% <0.00%> (ø)

...laland/chimney/internal/macros/MappingMacros.scala
100.00% <0.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b28c448...a0b9190. Read the comment docs.


Error: Current branch (tests/bumpversion) does not track anything. Cannot push.

I am getting the following error
Error: Current branch (tests/bumpversion) does not track anything. Cannot push.

Which makes very little sense IMHO. I was tracking issues from bump2version which led me to this project, and I never had to set the git tracking for the branch to the remote just to bump the version and add a tag.
That said, am I doing something wrong?
Here is the config file (edited) after running tbump init 0.12.2
[tool.tbump.version]
current = "0.12.2"

regex = '''
  (?P<major>\d+)
  \.
  (?P<minor>\d+)
  \.
  (?P<patch>\d+)
  (?:\-(?P<rc>[\S]+))?
  '''

[tool.tbump.git]
message_template = "Bump to {new_version}"
tag_template = "v{new_version}"

[[tool.tbump.file]]
src = "setup.py"

[[tool.tbump.file]]
src = "ecs_composex/__init__.py"

[[tool.tbump.before_commit]]
name = "Files format"
cmd = "make conform"

Only way I get it to work as expected is via tbump --only-patch
Also, your default behavior is to want to push to the branch, which again is rather different from bump2version which only creates the tag and commit message for it but is not trying to push something for me.
Maybe instead of --no-push as an option, have --push as an option for people wishing to push as they create the tag.
Many thanks for your help and consideration :)
PS: So much easier way to define the version than bump2version ..... :100: (minus the above details)

Only way I get it to work as expected is via tbump --only-patch

This is weird. --no-push should have been enough.

The default behavior is to want to push to the branch

Yup. The intention is that once the commit and the tag are created, you want to push them both. Before using tbump, I often ended up pushing the tag without the branch or vice-versa :)
The problem is that we can't just run git push --tags because the behavior would depend on the git configuration -
see https://git-scm.com/docs/git-config#Documentation/git-config.txt-pushdefault for details.
So instead we ask git for the tracking ref of the local branch and extract both the remote branch and the remote name, which allows us to run git push <origin> <tag> and git push <origin> <remote-branch> which only pushes the tag and commit we just created.

Maybe instead of --no-push as an option, have --push as an option for people wishing to push as they create the tag.

Well - I don't think we'll manage to have a default behavior that pleases everyone :)
Also, not that we don't check if the local branch tracks something when --no-push is used.

PS: So much easier way to define the version than bump2version ..... 100 (minus the above details)

Thanks - that was the goal all along :)

This is weird. --no-push should have been enough.

I believe this bug has been fixed. Closing.

This seems to happen if you:

create a new git repo (init)
push it to a remote
try to tbump it

If you start with a "git clone", it's not an issue.
git config -l output for non working repo (init + push):
branch.main.remote=git@gitlabs.com:domain/project.git
branch.main.merge=refs/heads/main

git config -l for repo where tbump works:
remote.origin.url=git@gitlabs.com:domain/project.git
remote.origin.fetch=+refs/heads/*:refs/remotes/origin/*
branch.main.remote=origin
branch.main.merge=refs/heads/main

@mathieulongtin : indeed, this is one scenario where the current branch would not track anything. There are more.


Imagery Offset Database

@bhousel would need help in figuring out certain things.

What do we keep the padding of imagery points ?
Where should we invoke the check for imagery offset ? Should it happen automatically or be shown in the imagery adjustment UI, like click to apply saved imagery offset.

(closes #1124 )

This is a great start - just a reminder to use 4 space indent, like we do everywhere else in iD.  (we should probably have an .editorconfig file in our project root that enforces this.)

What do we keep the padding of imagery points ?

Ideally we don't want to pad the lookup and store in a location cache - it would be better to cache the results of each tile url that we lookup.  I'm not sure whether the service could handle the load of all iD users asking for offset for every tile.  What do you think @Zverik ?

Where should we invoke the check for imagery offset ? Should it happen automatically or be shown in the imagery adjustment UI, like click to apply saved imagery offset.

Ideally automatically.  We store the user supplied offset here in background_source.  We could add a bool variable there to say whether the offset is automatically fetched, and we could add a corresponding checkbox on the Background ui screen.

@kepta As a next step, can you add a test here:  https://github.com/openstreetmap/iD/tree/master/test/spec/services

Thanks for the pull request. Automatically checking the offset for every tile is an overkill and will quickly put the server offline. I suggest doing that when opening the editor, and after that when a user scrolls the map more than a kilometer (at least 500 m) off the last offset checking point.

I have added some simple tests.
Next actions would be, to change the way search works. Currently the backend gives the whole list of imagery's without the provided radius. We would like to cache all of it but return back only the imagery offset matching with the active imagery.
The backend gives the data_url of the imagery, I guess that can be used to map the active imagery to the one in the cache?.

My bad forgot to check the indentation, I have fixed it in the latest commit. @bhousel should I rebase and squash the commits, to make it look better?

No problem.  I don't care much about how the commit history looks.

I have added some simple tests and also added a padding of 1km to each imagery pos as per @Zverik 's recommendation. I am still not sure about the radius to search in.

You could probably use a large radius, since we're caching the results  (so that would mean fewer calls) - and the results packets are small, and there probably aren't too many.
I guess try it with a few different search radii and see?

I guess try it with a few different search radii and see?
That makes sense, I will go for an increased radius. Any thoughts on how to match the active imagery with the one in cache?

That makes sense, I will go for an increased radius. @bhousel Any thoughts on how to match the active imagery with the one in cache?

Yes, I think we can do this similarly to how I sample imagery vintage in the Background panel.
The background layer is a bunch of <img> elements, and one of them which is closest to the center of the screen will be classed with .tile-center.
Here is some code that you can stuff into the background panel.  There is already a getVintage function, so this would be how you might write a getOffset function.
var tile = d3.select('.layer-background img.tile-center');   // tile near viewport center
if (tile.empty()) return;

var d = tile.datum();   // the data for that tile
if (!d || d.length < 4) return;

var zoom = d[2];
var url = d[3];
var center = context.map().center();

// service.getOffset will work somewhat like our nominatim service reverse():
// 1. use magic to turn the url, center, and zoom level into an identifier, per
//     https://wiki.openstreetmap.org/wiki/Imagery_Offset_Database/API#TMS
// 2. query the offset database with those parameters (or return cached result)
// 3. cache result and call the callback
service.getOffset(url, center, zoom, function(err, result) {
   // console.log something?
   // or update a div someplace with the numbers
});

@bhousel do we support WMS type of imageries in iD, since I couldn't find one in imagery.json?

do we support WMS type of imageries in iD, since I couldn't find one in imagery.json?

Not yet.. that's #1141

do we support WMS type of imageries in iD, since I couldn't find one in imagery.json?
Not yet.. that's #1141

I wouldn't worry about adding any WMS code here. #1141 is probably a long way off, and there's no way to currently test any WMS code you write.

I wouldn't worry about adding any WMS code here. #1141 is probably a long way off, and there's no way to currently test any WMS code you write.

I'd prefer to leave it in actually.  This isn't WMS code, but rather code for generating an identifier to query the the Offset Database.

Hey @Zverik , do you have test cases for this code


Merge/upstream changes

Picking up the upstream changes in seek-style-guide.
We'll want to do this regularly; even while we'll be adding a large amount of branded content, we want to keep as close to the parent repo as possible so we can share features.

so many changes, @mengtzu do we have any efficient way to review this kind of changes.

@ZhenYongSeekAsia, by being more vigilant with respect to pulling changes back from the upstream master, we can avoid these mega-merges in the future.

@ZhenYongSeekAsia if it helps I approved most of those changes myself :)  The only easy way to do it is stay on top of developments in the seek-style-guide channel and merge in often.

How do we ensure that those changes won't impact any of our (future) front-ends? Do we need to start thinking of functional testing and screenshot testing now?

@nicolassenechal Yeah, we need to worry about that regardless if it's an upstream change from seek-style-guide or a direct change to seek-asia-style-guide.
The likely workflow would be change to seek-asia-style-guide (passing its internal tests) --> PR to srp-react picking up the new npm package (which must pass manual/functional/screenshot tests)

I'd like to merge this, and hopefully make it the last megamerge.  Can I get an approval? :)


On Import, the modified url only gets updated in AutoroutePart and not as new Alias (Friendly Url)

Fixes #6316
@sebastienros i have used a property to get the aliasMaps. In that property im using lock to make sure no other call is updating it at the same moment. If the count is returned as 0 we call IAliasHolderUpdater.Refresh to refresh the map. Please let me know if this approach is not correct and we need to make any adjustments.
The other shape file had an extra column in the Admin/IndexManaged which has now been removed.

@sebastienros required changes are done. Please let me know if more changes are required.


Add signoutSilent support (Fixes #613)

Closes/fixes #613
oidc-client-ts added support for the signoutSilent() method in v2.2.0.  This PR updates the peer dependency to v2.2.1 (latest) and adds the appropriate tie-ins to expose that method from the auth context.
I opted not to add a onSignoutSilent() callback since the equivalent callbacks for the other methods are marked as "deprecated".  However, if that is actually needed, I'm happy to add it in.
Checklist

[X] This PR makes changes to the public API
[X] I have included links for closing relevant issue numbers

nice work, thanks for contributing!


[Extension] Fix Bugs in Extension Points

Description
This PR fixes bugs with extension points.

[x] Fixes inability to switch between full_page extension points

used redux store variable to track full_page extension point and any UI switches between them

[x] Fixes inability to click on extension points buttons

used CSS

https://user-images.githubusercontent.com/73700530/196007754-e733c9e5-621f-4b99-8cea-7b2df5cb7cea.mov
This PR fixes #
Notes for Reviewers
Signed commits

[x] Yes, I signed my commits.

@theBeginner86 a merge conflict came up here.

@leecalcote
fixed merge conflict.


passing the renew event type to create the correct context

Which Issue(s) This PR Fixes

Fixes #7044
Brief Description

How Did You Test This Change?

Codecov Report

Merging #7045 (91fba44) into develop (440be1e) will increase coverage by 0.04%.
The diff coverage is 52.17%.

@@              Coverage Diff              @@
##             develop    #7045      +/-   ##
=============================================
+ Coverage      42.62%   42.66%   +0.04%
- Complexity      9230     9244      +14
=============================================
  Files           1136     1136
  Lines          80652    80668      +16
  Branches       10542    10545       +3
=============================================
+ Hits           34375    34420      +45
+ Misses         41981    41954      -27
+ Partials        4296     4294       -2

Impacted Files
Coverage Δ

...rocketmq/broker/processor/PopMessageProcessor.java
45.36% <0.00%> (+0.61%)
:arrow_up:

...cketmq/proxy/processor/ReceiptHandleProcessor.java
35.89% <0.00%> (ø)

.../java/org/apache/rocketmq/common/BrokerConfig.java
27.28% <25.00%> (-0.02%)
:arrow_down:

.../rocketmq/common/utils/ConcurrentHashMapUtils.java
50.00% <42.85%> (-8.34%)
:arrow_down:

...y/service/receipt/DefaultReceiptHandleManager.java
71.52% <66.66%> (ø)

...a/org/apache/rocketmq/proxy/common/RenewEvent.java
92.85% <85.71%> (-7.15%)
:arrow_down:

... and 25 files with indirect coverage changes
:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


adding cookiecutter-django-aws-eb template to django list

https://github.com/dolphinkiss/cookiecutter-django-aws-eb

Current coverage is 99.63%

Merging #626 into master will not affect coverage as of 02de823

@@            master    #626   diff @@
======================================
  Files           12      12
  Stmts          546     546
  Branches         0       0
  Methods          0       0
======================================
  Hit            544     544
  Partial          0       0
  Missed           2       2

Review entire Coverage Diff as of 02de823

Uncovered Suggestions

+0.37% via ...ookiecutter/utils.py#27...28

Powered by Codecov. Updated on successful CI builds.

@peterlauri Merged, thanks! And sorry for the delay.


renaming tables in superset

Make sure these boxes are checked before submitting your issue - thank you!

[x] I have checked the superset logs for python stacktraces and included it here as text if any
[x] I have reproduced the issue with at least the latest released version of superset
[ ] I have checked the issue tracker for the same issue and I haven't found one similar

Superset version
0.25
Expected results
Should rename all the table name of superset
Actual results
Steps to reproduce
In our project we are planning to integrate superset, we are using oracle as backend.
Functionality everything working as expected. we are following a naming convention to create tables in our db. Is there any way to rename all the table name to "App_name_table name" without braking the code base

I think that it's a great idea, allow automation to superset


Enum / Type shorter dot syntax

Search Terms
enum type short dot syntax inference swift
Suggestion
Typescript could use a shorter dot syntax (Enum.value -> .value)
TS type inference seems perfectly fit to handle this kind of syntax
Use Cases
Thanks to TS, we can write a lot a code based on types, enums, or even trees of constants without worrying too much about making types mistakes.
But the full type/namespace is written/duplicated everywhere. And most of the time, it feels way too dense without a real reason to be, as the language is capable of inferring the type correctly without it
The short dot syntax is making the code lighter, and greatly easier to read. And I have to say that it also looks and feel a lot more modern
Examples

A good example is the Swift short dot syntax. See here for enumerations: https://docs.swift.org/swift-book/LanguageGuide/Enumerations.html

This code:
const config: MainConfig = {
    theme: {
        primaryColor: ApiTypes.Theme.Colors.blue,
    },
}

const red: ApiTypes.Theme.Colors = ApiTypes.Theme.Colors.red
const orange = mixColors(red, ApiTypes.Theme.Colors.yellow)

switch (config.theme.primaryColor) {
    case ApiTypes.Theme.Colors.red:
        return 'red'
    case ApiTypes.Theme.Colors.blue:
        return 'blue'
    case ApiTypes.Theme.Colors.orange:
        return 'orange'
    [...]
}

could be replaced by:
const config: MainConfig = {
    theme: {
        primaryColor: .blue, // MainConfig.theme.primaryColor: ApiTypes.Theme.Colors
    },
}

const red: ApiTypes.Theme.Colors = .red
const orange = mixColors(red, .yellow) // based on parameters types

switch (config.theme.primaryColor) {
    case .red:
        return 'red'
    case .blue:
        return 'blue'
    case .orange:
        return 'orange'
    [...]
}

Checklist
My suggestion meets these guidelines:

[X] This wouldn't be a breaking change in existing TypeScript/JavaScript code
[X] This wouldn't change the runtime behavior of existing JavaScript code
[X] This could be implemented without emitting different JS based on the types of the expressions
[X] This isn't a runtime feature (e.g. library functionality, non-ECMAScript syntax with JavaScript output, etc.)
[X] This feature would agree with the rest of TypeScript's Design Goals.

Duplicate of #1713.

Oh sorry, I did not find that one.
It has been 4 years since #1713 was closed though. Is it still the current policy to avoid this kind of features ? as it feels like Typescript is actually a good place to do exactly that (as it would be not impact runtime, or JS).
Also I don't know of any ES standard that could collide with this at this time (so, for years ahead).
I would argue that the positive effect of this feature could outweigh the (I believe unlikely) risk of collision. But that is mainly a subjective point of view.
Is it at least enough to consider letting this issue live for a while. Maybe it could be introduced as an experimental flag ?

Yeah, we can't do this sort of thing - it's very close in spirit to why we can't implicitly resolve foo to this.foo in some cases. The stance of not using type-based resolution to determine runtime behavior is actually stronger than it was 4 years ago.

I might be fighting for a lost cause here 😄, but I don't see how this particular feature would determine runtime behaviour.
The this.foo / foo example is clear in the sense that both would represent a different statement/variable in JS. So Typescript would carry a different meaning than JS for the exact same code.
It also means implicitly accessing a property of an object (at runtime), even though it might not be obvious to a first reading.
Many languages allow that, and it actually makes some sense in a strongly OOP language, but probably not in TS.
Yet I can't think of any of those issues with the short dot syntax, as the syntax itself simply does not exists in JS (and I would argue, cannot exist). And it does not seem like it is actually determining any runtime behaviour, as it would only be using the already known type of a parameter/variable to statically access the right JS constant, effectively filling the gaps in behalf of the developper at compilation level. It could likely do so without ever making any assumptions about the type (and could abort the compilation if there is a case where it would need to).
Based on your comment @DanielRosenwasser, I imagine there are other issues with this that I do not see (and I would actually be curious to learn a bit about that).
But are we sure that the rationale actually applies to this specific feature ?

only be using the already known type of a parameter/variable to statically access the right JS constant

The template forbids this:

This could be implemented without emitting different JS based on the types of the expressions

An alternate way to think of this is:

Can a developer, seeing a type error they don't like, always add as any to silence it without downstream problems?

The answer is "no":
switch (config.theme.primaryColor as any) {
    case .red: <-- .red of what?
        return 'red'
    case .blue:
        return 'blue'
    case .orange:
        return 'orange'
    [...]
}

There are a bunch of reasons not to do type-based emit:

Other tools like Babel would be unable to correctly transpile TS files
TS tweaks type inference over time, which could lead to your program failing to run correctly due to an inference change
Type inference can be a very nonlocal process, so type-only changes in one file might lead to runtime correctness problems in another file
Resolving types before being able to do emit slows performance in --watch scenarios
Type errors can be silenced by things like // @ts-ignore, making the runtime characteristics of programs using that directive unknownable
None of this works with generics, since TS uses erasure instead of template instantiation

Thanks @RyanCavanaugh, these are really good points.
I would still hope to argue that the short dot syntax is optional in TS, and non-existant in JS.

In this case, being able to silence the type error with any or ts-ignore feels less relevant, as you would usually do that in a situation where you are transitioning from JS (so the syntax is not present unless you want it to be, and actually do your part to make it work).
If dealing with a TS code, a type error here would mean that the code is actually wrong, or incomplete, so ignoring it should not be permitted in the first place in my opinion. TS can only go so far, we still except the developer to use the full syntax if the type isn't known, and I think that's OK.

It also wouldn't work with generics, or maybe it could work but a very limited set of cases. Yet that would just be a known limitation that makes a lot of sense, one that we could live with.
And there is this:

TS tweaks type inference over time, which could lead to your program failing to run correctly due to an inference change

That's probably the biggest problem, and even though I'd like to say that only working with fully known types could mitigate this in almost any case, that's just theory, I have no evidence to back this up 😔.
Type-based emits would make the language a lot more strict when using its related features. And I can't argue with the Babel point (a babel plugin maybe ?), or the fact that it would slow down performance. But I still believe it could be worth the cost.
As unlikely as it is, I can only hope that this kind of features can be considered in the future, as type-based emits could introduce very interesting features, that could greatly help developers.

If you add something to a language, people expect it to work. Like, really work. The idea of a "known limitation" just doesn't have a lot of traction because how do you "know" the limitation exists? No one reads the docs past the first few pages; they're just out there trying to get stuff done, as they should be.
It's extremely frustrating to write some code using some new language feature, then try to apply very standard programming techniques like refactoring common code into a generic function, only to find that you can't anymore because the feature you relied on is broken inside generics. The experience is always that it's broken, not a "known limitation" - we get frequent feedback here on some features that were less thought-through that they should be removed from the language because people ran into the "known limitations" around them and were frustrated that they were allowed to go 90% of the way without the ability go to the last 10% toward their optimal solution.


Add pre-commit autoupdate workflow

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 100.00%. Comparing base (2f021c6) to head (c26d35d).
Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff            @@
##              main      #562   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files          139       139
  Lines         4287      4287
  Branches       288       288
=========================================
  Hits          4287      4287

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


pDOP divide by 100 https://github.com/ketan/paragliding-meshmap/issue…

https://github.com/ketan/paragliding-meshmap/issues/58

You probably want to do this inside the sanitizeNodeProperties function. Give it a go? That way we update the data as soon as it is loaded, rather than worrying about each attribute which may be rendered in different place.


fix: store resolved root specifiers

Basically carry the "rootness" around until the specifier is fully resolved. Once it's resolved, store the resolved specifier in a separate set for later comparison.
This fixes an issue where we would error on JSON roots.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Delay for touch events

Is there a way to set delay for touch events, so that dragging started after N ms of touching (to differentiate from tapping and scrolling)?

you can get lastX and lastY from as the second argument in the onDrag and calculate the difference and set a value in state to disable drag after that movement enable the drag

Is there any plan for this issue?
I think it would be great to have this feature embeded in component instead of manually having to compare onDrag arguments on touchDevices

this worked for me
https://codesandbox.io/s/eager-darkness-j97pd

You can use something like useLongPress to enable dragging. You can't use the disabled prop but you can return false in the onDrag handler.

这个问题有解决方案吗？使用 @carloquilala 的答案可以在onStep函数内部拦截到点击事件，但是并不会出发节点本身的onClick事件

这个问题有解决方案吗？使用 @carloquilala 的答案可以在 onStop 函数内部拦截到点击事件，但是并不会出发节点本身的onClick事件


Error when running training step in tutorial

Hello,
On this page of the tutorial: https://dvc.org/doc/get-started/pipeline, when running the command
dvc run -d src/train.py -d data/features \
              -o model.pkl \
              python src/train.py -d data/features model.pkl

I get the following:
Running command:
	python src/train.py -d data/features model.pkl
Arguments error. Usage:
	python train.py features model

Looking at train.py, it looks like the -d causes the mistake and is not necessary.
It also looks like the command is missing a -f train.dvc to be consistent with the instructions that follow.
edit: just submitted a PR to fix https://github.com/iterative/dvc.org/pull/221

Fixed by #221 . Kudos @rpelliard !


Testing PR: Not intended to be merged

Description

Checklist:

[ ] I have tested my changes. No regression in existing tests.
[ ] My code is Linted.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

/bot run checks

/bot run checks


fix: disable COBRApy and RBApy deploy tests until new Gurobi license

What new features does this PR implement?
The Gurobi license has expired at the HPC facility which is hosting simulations for dev and production sites.  Two solvers, COBRApy and RBApy need Gurobi to execute properly.
This PR removes the COBRApy and RBApy simulations from the 'dev' site acceptance tests, which is currently blocking production deployments.  These tests are executed via .github/workflows/deploy.yml in the job entitled "Test the development deployment" during the continuous deployment Github Action.
Note: Once the new Gurobi 10.0 license is obtained, and the biosimulators_cobrapy and biosimulators_rbapy container images are rebuilt and submitted as new versions
**added manual checkpoint - uncommented github environment "review_dev_deployment_tests_environment" to enforce production deployment approval by Bilal, Jonathan, Ion or Jim.
What bugs does this PR fix?
the "Test the development deployment" job in the continuous deployment GitHub action is broken due to a Gurobi license error - this PR will temporarily bypass that failure.

added Jim Schaff and Ion Moraru to allowable 'approvers' for checkpoints for review_dev_deployment_tests_environment and prod_environment.  In hindsight - we can put back the comment for review_dev_deployment_tests_environment environment in deploy.yml, as the prod_environment is sufficient as a checkpoint.

reverted use of redundant review_dev_deployment_tests_environment environment (is once again commented out in deploy.yml).  The prod_environment environment is a sufficient manual approval checkpoint for production deployments, and is currently configured to be approved by Bilal, Jonathan, Ion, or Jim.

force push make commit message compliant.

:tada: This PR is included in version 9.51.3 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Error - Could not open NCS device.
mvncStatus value: -2

I've edited the /workspace/ncsdk/api/src/usb_boot.c
Device works from host

Ideas ?

Hey @think-free ,
Sorry for the late response.
I'm the process of recreating the installation steps on a fresh box. I think the one thing I left out by accident was copying the $SDK_DIR/udev/97-usbboot.rules file on the host machine to /etc/udev/rules.d/ and then issuing the commands:
sudo udevadm control --reload-rules
sudo udevadm trigger

If you follow those steps, can you get it work?
Cheers,
Kevin

Hi,
I have the same problem as @think-free. I've added the two commands, but docker b  to return an error code 2 on sudo udevadm control --reload-rules during the build process. I can manually execute the 2 commands after i'm inside the container, but it doesn't seem to solve the problem


Factor out common constraint in type signatures.

Yes, I initially thought that was similar too, but squinting closely, the return type doesn't match the m type pattern of the previous two (i.e. the first is m [Block] and the second is m Block, but the third is String -> m Block).

ah right i see. cool well looks good, thanks again!

Well, thanks for writing up this neat implementation, very instructive!


Blank to None legend issue fix

fixed pie, line and bar does not cast empty values to none
fixed none color is not using default color
test

@dorsha can you review?

Need to go over those PRs together


Fix didkit paths in CI

Fix #111 caching of Rust build artifacts in CI.
Full run, no cache: https://github.com/spruceid/didkit/runs/2142681378 45m 49s
Subsequent run, with caching: https://github.com/spruceid/didkit/runs/2143040544: failed for disk space reasons. Looks like we have to free up more space to make room for the larger cache file.
Cache size which previously was ~263 MB (probably containing just the cargo registry data); is now up to ~2141 MB (step:4:23).

Hi @clehner, is this ready for review?

@wyc I'm not sure how to proceed here. I believe the change does what it is supposed to do, enabling caching the build artifacts (target/), however it was causing test failure due to running out of space because of the size of the data to cache. I experimented with deleting existing files from the CI environment, to free up space (in order to hold the cache data), in #109, but that added more time to the run to delete all those files. I recall that the time deleting the files to free up space was close to or greater than the time saved by caching the build artifacts, so I stopped pursuing it after other changes (#110, #116) stopped the out-of-space issue (#108) that we were seeing around that time. The CI run here currently takes an hour, so reducing that could be good; I could look into these changes and #109 again if that would be useful; otherwise I think this PR could be closed or changed to simply remove the incorrectly-referenced paths.


TSP needs more usage info

Description
Most TSP problems I try fail to produce a minimal route. The parameters and working range of solvable problems should be outlined.
To Reproduce
The following alternative weights to the 5-node problem, EdgeView([(0, 1), (0, 2), (0, 3), (1, 2), (1, 3), (2, 3)]), all fail:

[{'weight': 22.36},  {'weight': 10.0},  {'weight': 14.14},  {'weight': 20.0},  {'weight': 22.36}, {'weight': 22.36}]
produces route: [3, 1, 0] (non-Hamiltonian route)
[{'weight': 2.24}, {'weight': 1.0}, {'weight': 1.41}, {'weight': 2.0}, {'weight': 2.24}, {'weight': 2.24}]
produces route: [1, 0, 3, 2] (a better answer is [1 2 0 3])
[{'weight': 2}, {'weight': 4}, {'weight': 1}, {'weight': 2}, {'weight': 4}, {'weight': 3}]
produces route: [3, 2, 1, 0] (a better answer is [2 1 0 3])

Expected behavior
More info on range of problem inputs, how to set lagrange, etc
Environment:

OS: Windows
Python version: 2.7

Additional context
@vgoliber

Choosing a value for a Lagrange parameter is a topic that is relevant for all of our algorithm tools (not just TSP).  I think it would be more appropriate to provide higher level guidance and/or references on this subject as a whole rather than guidance specific to each algorithm.

Agree, @vgoliber, you mentioned having references and some guidance. Please add that here for that general update. Thanks!


Adding options for telemetry and transport policy

Fixes: https://github.com/Azure/azure-sdk-for-cpp/issues/819

@vhvb1989 Given this change is non-breaking to storage and is somewhat time critical.  Please merge the change.
//cc @vinjiang @katmsft


Added the Makefile from the Trust and Safety Guide

To make it much easier to install Jekyll and use for local testing (especially since the README mentions it ;) ) , I added the Makefile from the Trust and Safety guide (with appropriate name changes).

Weird.  I'd intended to grab all that stuff.  Thanks!


documentation navbar

Stays on top when scrolling:

Hover color:

Small screens:

Issue: the navbar hides the header when following an anchor link.
Expected:

Current:

This would be solved by moving the navbar into the table of contents like Erik suggested, but it won't stay at the top when using a small-screen device like a phone. (Although, how much do we want to support phones anyways?)


fix:main yml codecoverage

This commit fixes the codecoverage for miv-os. Name of the directory for
coverage was wrong.

Codecov Report

:exclamation: No coverage uploaded for pull request base (update-0.0.2@e6f6840). Click here to learn what that means.
The diff coverage is n/a.

@@              Coverage Diff               @@
##             update-0.0.2     #12   +/-   ##
==============================================
  Coverage                ?   0.67%
==============================================
  Files                   ?       7
  Lines                   ?     148
  Branches                ?      32
==============================================
  Hits                    ?       1
  Misses                  ?     147
  Partials                ?       0

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e6f6840...4aa15ba. Read the comment docs.


chore: use same vs2017 appveyor image for master and develop

What kind of change does this PR introduce? (Bug fix, feature, docs update, ...)
not sure if this is actually hit at the moment, but might make a difference.
What is the current behavior? (You can also link to an open issue here)
Use a preview appveyor image
What is the new behavior (if this is a feature change)?
Use a non-preview appveyor image (matching the non-master build)
What might this PR break?
Please check if the PR fulfills these requirements

[ ] Tests for the changes have been added (for bug fixes / features)
[ ] Docs have been added / updated (for bug fixes / features)

Other information:

Thanks @ghuntley! This seems to have fixed the develop builds (they're flagged as pending because of #1422 I think)


HDDS-5797. Support setting Datanode Reserved Space in MiniOzoneCluster.

What changes were proposed in this pull request?
Support setting org.apache.hadoop.hdds.scm.ScmConfigKeys#HDDS_DATANODE_DIR_DU_RESERVED for each volume in each Datanode in MiniOzoneCluster (sets the same reserved space for each volume).
What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-5797
How was this patch tested?
Passed TestMiniOzoneCluster

@adoroszlai Thanks for the review. I've updated the PR.

Thanks @siddhantsangwan for the patch, @JacksonYao287 for the review.


Proposed change to broker (not MDS) Event Document layout

This proposes a change affecting the presentation of the data, not the on-disk representation.
Watching the broker API in use today, @dchabot and I think databroker's Event Documents should be changed from a dictionary of tuples to pair of dictionaries with identical keys.
Proposed API:
    event.data['pe1_image_lightfield']  # the image array
    event.timestamps['pe1_image_lightfield']  # the associated timestamps

This does not require a change of the underlying MDS Document.
For the majority of use cases, we care about measured values much more than timestamps. Slicing into the (value, timestamp) tuple, as in
    images = [event.data['pe1_image_lightfield'][0] for event in events]

adds another mental step to the long unpacking.

Provisional :+1: to this.  There is no reason we can't have both data_stream and timestamp_stream vs data?

And an in-person :+1: from @arkilic.

:+1: in principle.  It would be nice to shorten the word timestamps though, especially since tab-complete isn't necessarily a thing that people are familiar with.  Any major issues with something like ts?

I slightly prefer timestamps but I am OK with ts.


gwyddion: support darwin

Motivation for this change
Gwyddion will build on Darwin, but gnome2.gtkglext is broken there,
and so I am disabling it as an optional dependency here when building
on Darwin.
Things done
Note that I personally have only tested that this will continue to build and work on NixOS.  Someone else tested that the previous Gwyddion derivation builds, all binaries work, and the Python 2.7 module loads, so long as gnome2.gtkglext is not used (it fails to build on OS X); that is what this change is meant to ensure.

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

@GrahamcOfBorg eval
@GrahamcOfBorg build gwyddion

It looks like the libunique derivation does not have darwin set as supported, though it will build if allowUnsupportedSystem is set, and that's causing the ofborg build to fail.

Thank you!


Exception in Tkinter callback

when trying to use program (ver. 1.6.1) I get this error
Exception in Tkinter callback Traceback (most recent call last):     File "tkinter\__init__.py", line 1921, in __call__     File "structura.py", line 158, in runFromGui     File "structura_core.py", line 79, in generate_with_nametags     File "structure_reader.py", line 25, in __init__     File "nbtlib\nbt.py", line 131, in load     File "nbtlib\nbt.py", line 259, in from_fileobj     File "nbtlib\nbt.py", line 225, in parse   TypeError: Non-Compound root tags is not supported: <class 'nbtlib.tag.End'>

I get the .mcstructure file in the output but not the texture pack

your structure is empy. 0kB file. The reason this occurs is due to a minecraft bug. that comes up if you export into a one drive folder.


7078 BigInt.toInt returns a long

Just a silly typo.

I don't think I'm ever going to get this beta out tonight.

I don't think I'm ever going to get this beta out tonight.

Hey, I wasn't expecting this to go into the release! (Nor the previous pull request, that Andrei folded in). It's not a regression.
 They don't distinguish 'this is ready to be pulled' from 'please pull this now!' Would be good to develop a way to flag urgent vs non-urgent pull requests.

The trouble happened because I needed to push std/traits.d, and it wouldn't let
me do a push until after I'd done a pull. Doing a pull brought in the bigint
changes.
On 12/8/2011 2:19 AM, Don Clugston wrote:

I don't think I'm ever going to get this beta out tonight.
Hey, I wasn't expecting this to go into the release! (Nor the previous pull request, that Andrei folded in). It's not a regression.

 They don't distinguish 'this is ready to be pulled' from 'please pull this now!' Would be good to develop a way to flag urgent vs non-urgent pull requests.


trainging problem

1:I get the KITTI_odometry_grondtruth_poses data from tinghuiz/SfMLearner which only consists of 10 txt files,but there are 21 sequences in KITTI odometry dataset ,do i get the wrong groundtruth pose data?
2:in the process of training flow,when the iteration is 15200,the problem is :
Iteration: [  14700] | Time: 0.3787s/iter | Loss: 1.504
2018-07-01 17:51:40.590857: W tensorflow/core/framework/op_kernel.cc:1192] Invalid argument: Expected image (JPEG, PNG, or GIF), got empty file
[[Node: DecodeJpeg = DecodeJpegacceptable_fraction=1, channels=0, dct_method="", fancy_upscaling=true, ratio=1, try_recover_truncated=false, _device="/job:localhost/replica:0/task:0/device:CPU:0"]]
Iteration: [  14800] | Time: 0.3839s/iter | Loss: 1.462
Iteration: [  14900] | Time: 0.3801s/iter | Loss: 1.380
Iteration: [  15000] | Time: 0.3811s/iter | Loss: 1.538
Iteration: [  15100] | Time: 0.4323s/iter | Loss: 1.279
Iteration: [  15200] | Time: 0.3785s/iter | Loss: 1.250
2018-07-01 17:55:13.338006: W tensorflow/core/framework/op_kernel.cc:1192] Out of range: RandomShuffleQueue '_2_shuffle_batch/random_shuffle_queue' is closed and has insufficient elements (requested 4, current size 0)
[[Node: shuffle_batch = QueueDequeueManyV2[component_types=[DT_UINT8, DT_UINT8, DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](shuffle_batch/random_shuffle_queue, shuffle_batch/n)]]
2018-07-01 17:55:13.338261: W tensorflow/core/framework/op_kernel.cc:1192] Out of range: RandomShuffleQueue '_2_shuffle_batch/random_shuffle_queue' is closed and has insufficient elements (requested 4, current size 0)
[[Node: shuffle_batch = QueueDequeueManyV2[component_types=[DT_UINT8, DT_UINT8, DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](shuffle_batch/random_shuffle_queue, shuffle_batch/n)]]
2018-07-01 17:55:13.338448: W tensorflow/core/framework/op_kernel.cc:1192] Out of range: RandomShuffleQueue '_2_shuffle_batch/random_shuffle_queue' is closed and has insufficient elements (requested 4, current size 0)
[[Node: shuffle_batch = QueueDequeueManyV2[component_types=[DT_UINT8, DT_UINT8, DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](shuffle_batch/random_shuffle_queue, shuffle_batch/n)]]
Traceback (most recent call last):
File "geonet_main.py", line 166, in
tf.app.run()
File "/home/ubuntu/.local/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 48, in run
_sys.exit(main(_sys.argv[:1] + flags_passthrough))
File "geonet_main.py", line 157, in main
train()
File "geonet_main.py", line 143, in train
saver.save(sess, os.path.join(opt.checkpoint_dir, 'model'), global_step=step)
File "/usr/lib/python2.7/contextlib.py", line 35, in exit
self.gen.throw(type, value, traceback)
File "/home/ubuntu/.local/lib/python2.7/site-packages/tensorflow/python/training/supervisor.py", line 964, in managed_session
self.stop(close_summary_writer=close_summary_writer)
File "/home/ubuntu/.local/lib/python2.7/site-packages/tensorflow/python/training/supervisor.py", line 792, in stop
stop_grace_period_secs=self._stop_grace_secs)
File "/home/ubuntu/.local/lib/python2.7/site-packages/tensorflow/python/training/coordinator.py", line 389, in join
six.reraise(*self._exc_info_to_raise)
File "/home/ubuntu/.local/lib/python2.7/site-packages/tensorflow/python/training/queue_runner_impl.py", line 238, in _run
enqueue_callable()
File "/home/ubuntu/.local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1231, in _single_operation_run
target_list_as_strings, status, None)
File "/home/ubuntu/.local/lib/python2.7/site-packages/tensorflow/python/framework/errors_impl.py", line 473, in exit
c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.InvalidArgumentError: Expected image (JPEG, PNG, or GIF), got empty file
[[Node: DecodeJpeg = DecodeJpegacceptable_fraction=1, channels=0, dct_method="", fancy_upscaling=true, ratio=1, try_recover_truncated=false, _device="/job:localhost/replica:0/task:0/device:CPU:0"]]
where prepare_training_flow:--dataset_dir=raw_data_KITTI/ --dataset_name=kitti_raw_stereo(if I set it as kitti_raw_eigen,there is nothing wrong)
and train_flow:--dataset_dir=resulting/formatted/data_flow/   -flownet_type=direct
i don't know why

i change the num_threads and other parameter,but it doesn't work .the prepare_training_flow  is normal,i don't know why...can you help me?

KITTI has only released the gt poses for seq. 00~10;
Seemingly you have missing images. You can check if all the images in the training list are normal.

ok ,thank you very much ,i want to know what's the difference between kitti_raw_eigen and kitti_raw_stereo ,from the code ,their different test_scenes files make the training list different,and is there anything else that's important?can i only use the kitti_raw_eigen when prepare training?

The splits are for conducting experiments on different tasks. Please refer to the experiment part of our paper for more details.

i have solved it ,thank you


Can you add support for ERC-721 tokens

Can you add support for ERC-721 tokens

Do you have a reference ABI and some desired functionality? I can plan it for 1.3+ version

From: MengLiMing<EMAIL_ADDRESS>Sent: Tuesday, August 7, 2018 6:21:14 AM
To: BANKEX/web3swift
Cc: Subscribed
Subject: [BANKEX/web3swift] Can you add support for ERC-721 tokens (#209)
Can you add support for ERC-721 tokens
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHubhttps://github.com/BANKEX/web3swift/issues/209, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AGKv9w3_5J9h8z-fWc8MrPTLkB38VXD8ks5uOQeqgaJpZM4Vxayd.

This issue was moved to matterinc/web3swift#7

Added ERC721 support in v2.0


Database connection issue

after creating entity i am getting this error.

I have a similar error, but with sqlite.

Did anyone solve this problem?


[Playground] [Infrastructure] Deleting the Cloud Storage Client

fixes #22865
@olehborysevych

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[x] Choose reviewer(s) and mention them in a comment (R: @username).
[x] Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
[ ] Update CHANGES.md with noteworthy changes.
[x] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.
To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md
GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

R: @pabloem

can you please rebase? thanks!

can you please rebase? thanks!

Sure, i've merged with master

lgtm thanks!

Run Java PreCommit

Run SQL PreCommit

Run SQL_Java17 PreCommit

lgtm