Updated instructions

Also minor style changes

Hmm unsure why this is coupled with some of the loading animation stuff; will merge that branch, pull from master to clear this up before emerging this. If you want you can hold off on reviewing this then.
Actual changes in this file are:

Smaller loader
Rotating instead of expanding "close button"
Changed instructions from S/D to Down/Up


Json metadata format

Hey!
I'm using SparkMagics, so I need to manually generate the metadata JSON file to be used by the sql-language-server.
Could you provide an example of the JSON format?
Thanks in advance!

Amazing. Immediately after posting, I found it: https://github.com/joe-re/sql-language-server/blob/release/example/jupyterlab/sparkdb.schema.json.
Leaving here for future reference.


RFI: Insert if missing, otherwise do nothing.

Is there a way to only insert new rows based on the PK otherwise do nothing?

You can use BulkRead to get those that already exists in Db and exclude them from Insert.

@borisdj but using 2 round-trips would not guard against the data changing between the BulkRead and the BulkInsert.  It would be helpful to have a BulkInsertNotExists or similar method that used a MERGE statement to INSERT WHEN MATCHED and do nothing WHEN NOT MATCHED.

What is your UseCase scenario for this?
Usually InsertOrUpdate is enough for most cases, and even if doing 2 queries to DB as mentioned it is still done relatively fast so small chance for conflict. Since you do not want to update existing only issue could be if some row would be deleted in between queries and then it not being inserted again.
Is this case likely to happen?

In my case I insert the record with a Created_DT column with the update it overwrites the date each time.

Even then BulkRead can be used to set Created_DT column to value from DB so that it remains the same.

Take the scenario where a process outputs a number of items which it persists to the database, these items can then be modified by the user.  Then, when the process is re-run it generates some of the old items again and also generates new items.  In order to create the new items and not overwrite/update the existing items (which may have user edits), I need to persist the new set, ignoring any rows that already exist.
As you say this could be achieved by doing a BulkRead followed by a BulkInsert of only the items that were not returned by the BulkRead.  However, any new items added by another process between the calls to BulkRead and BulkInsert would cause the BulkInsert to fail due to a duplicate key violation, if the item happened to be a duplicate of one in the set to update.  The only solution would be to re-run the whole BulkRead, BulkInsert process again; which with large data sets would be costly.
If I were solving this problem directly in SQL, I would implement it using a MERGE statement or using several INSERT statements, each with a where not exists check.

I would generalize the issue to include control over the merge operation "WHEN NOT MATCHED BY TARGET," "WHEN MATCHED," and "WHEN NOT MATCHED BY SOURCE." In my case I want to soft delete rows by updating a database column instead of actually deleting on the merge command "WHEN NOT MATCHED BY SOURCE." A fluent API for strongly typed objects that controls the merge operation would be excellent.

Just chiming in that @ralefire proposed approach would be perfect in my opinion.

Agree with @ralefire , too bad this is not there. Could have used this library. I have a situation where I need this exact control.

The similar I proposed some time ago in #155
It would be really helpfull option.

This can now be configued with separate PropertiesTo-Include/Exclude-OnUpdate.
More info in the README.


302 errors

The DataCite create DOI test API returns a 302 redirect to https://test.ec2.datacite.org:8443/mds/login.  This seems to be an error with DataCite service, and have reported to their tech team.

Fixed encoding error


Added a course to free-courses-en.md

What does this PR do?
Added a resource in free-courses-en.md
For resources
Description
This resource has great DSA content and the programming language used is JAVA.
Why is this valuable (or not)?
The instructor(Kunal Kushwaha)) is a great teacher and himself an Open Source contributor explains difficult topics in an easy way. He has great experience and his guidance is very valuable.
How do we know it's really free?
It's available on YouTube and is charged at $0
For book lists, is it a book? For course lists, is it a course? etc.
Checklist:

[x] Read our contributing guidelines.
[x] Search for duplicates.
[x] Include author(s) and platform where appropriate.
[x] Put lists in alphabetical order, correct spacing.
[x] Add needed indications (PDF, access notes, under construction).
[x] Used an informative name for this pull request.

Follow-up

Check the status of GitHub Actions and resolve any reported warnings!

Looks like you have deleted a file from the repo. If you have any free resources you would like to add, please add them to the proper file and raise a PR. Read Contributing guidelines for more information

Sir, I am failing to find out the problem here. I did not delete any file, instead I added some material. Can you please help me out with this?


Cache: Adds size management to FSCache [WIP]

Solves #1183

close it via https://github.com/pallets/werkzeug/issues/1249


macOS build error fatal error: 'boost/beast/core/string.hpp' file not found

[  0%] Running C++ protocol buffer compiler on /app/ripple/rippled/src/ripple/proto/ripple.proto
Scanning dependencies of target rippled
[  0%] Building CXX object CMakeFiles/rippled.dir/src/ripple/beast/unity/beast_insight_unity.cpp.o
[  0%] Building CXX object CMakeFiles/rippled.dir/src/ripple/beast/unity/beast_net_unity.cpp.o
[  0%] Building CXX object CMakeFiles/rippled.dir/src/ripple/beast/unity/beast_utility_unity.cpp.o
[  0%] Building CXX object CMakeFiles/rippled.dir/src/ripple/unity/app_consensus.cpp.o
[  9%] Building CXX object CMakeFiles/rippled.dir/src/ripple/unity/app_ledger.cpp.o
[  9%] Building CXX object CMakeFiles/rippled.dir/src/ripple/unity/app_ledger_impl.cpp.o
In file included from /app/ripple/rippled/src/ripple/unity/app_consensus.cpp:21:
In file included from /app/ripple/rippled/src/ripple/app/consensus/RCLConsensus.cpp:21:
In file included from /app/ripple/rippled/src/ripple/app/consensus/RCLConsensus.h:24:
In file included from /app/ripple/rippled/src/ripple/app/consensus/RCLCxLedger.h:23:
In file included from /app/ripple/rippled/src/ripple/app/ledger/Ledger.h:24:
In file included from /app/ripple/rippled/src/ripple/ledger/View.h:34:
In file included from /app/ripple/rippled/src/ripple/core/Config.h:23:
/app/ripple/rippled/src/ripple/basics/BasicConfig.h:25:10: fatal error: 'boost/beast/core/string.hpp' file not found
#include <boost/beast/core/string.hpp>
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /app/ripple/rippled/src/ripple/unity/app_ledger_impl.cpp:22:
In file included from /app/ripple/rippled/src/ripple/app/ledger/impl/InboundLedger.cpp:21:
In file included from /app/ripple/rippled/src/ripple/app/ledger/InboundLedger.h:23:
In file included from /app/ripple/rippled/src/ripple/app/main/Application.h:24:
In file included from /app/ripple/rippled/src/ripple/shamap/TreeNodeCache.h:24:
In file included from /app/ripple/rippled/src/ripple/shamap/SHAMapTreeNode.h:25:
In file included from /app/ripple/rippled/src/ripple/basics/TaggedCache.h:24:
/app/ripple/rippled/src/ripple/basics/Log.h:24:10: fatal error: 'boost/beast/core/string.hpp' file not found
#include <boost/beast/core/string.hpp>
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /app/ripple/rippled/src/ripple/unity/app_ledger.cpp:22:
In file included from /app/ripple/rippled/src/ripple/app/ledger/AcceptedLedger.cpp:21:
In file included from /app/ripple/rippled/src/ripple/app/ledger/AcceptedLedger.h:23:
In file included from /app/ripple/rippled/src/ripple/app/ledger/AcceptedLedgerTx.h:23:
In file included from /app/ripple/rippled/src/ripple/app/ledger/Ledger.h:24:
In file included from /app/ripple/rippled/src/ripple/ledger/View.h:34:
In file included from /app/ripple/rippled/src/ripple/core/Config.h:23:
/app/ripple/rippled/src/ripple/basics/BasicConfig.h:25:10: fatal error: 'boost/beast/core/string.hpp' file not found
#include <boost/beast/core/string.hpp>
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[  9%] Building CXX object CMakeFiles/rippled.dir/src/ripple/unity/app_main1.cpp.o
1 error generated.
make[2]: *** [CMakeFiles/rippled.dir/src/ripple/unity/app_consensus.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
1 error generated.
make[2]: *** [CMakeFiles/rippled.dir/src/ripple/unity/app_ledger.cpp.o] Error 1
In file included from /app/ripple/rippled/src/ripple/unity/app_main1.cpp:22:
In file included from /app/ripple/rippled/src/ripple/app/main/Application.cpp:21:
In file included from /app/ripple/rippled/src/ripple/app/main/Application.h:24:
In file included from /app/ripple/rippled/src/ripple/shamap/TreeNodeCache.h:24:
In file included from /app/ripple/rippled/src/ripple/shamap/SHAMapTreeNode.h:25:
In file included from /app/ripple/rippled/src/ripple/basics/TaggedCache.h:24:
/app/ripple/rippled/src/ripple/basics/Log.h:24:10: fatal error: 'boost/beast/core/string.hpp' file not found
#include <boost/beast/core/string.hpp>
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
make[2]: *** [CMakeFiles/rippled.dir/src/ripple/unity/app_ledger_impl.cpp.o] Error 1
1 error generated.
make[2]: *** [CMakeFiles/rippled.dir/src/ripple/unity/app_main1.cpp.o] Error 1
make[1]: *** [CMakeFiles/rippled.dir/all] Error 2
make: *** [all] Error 2

hi @bachase
no I use boost_1_65_1,building the latest from the develop branch.

@wangdar rippled 0.90.0-b5 requires boost 1.66. The reason for this is the beast library, which was originally part of rippled, is now part of boost. In order to get important bug fixes we need to use boost 1.66 (or back-port the changes ourselves, which is less attractive).
Upgrading to boost 1.66 should fix this issue for you (assuming you are also building rippled 0.90.0-b5).

hi @seelabs
I build for  boost 1.66.0, show another error [ error: invalid operands to binary expression]

-- The C compiler identification is AppleClang <IP_ADDRESS>00039
-- The CXX compiler identification is AppleClang <IP_ADDRESS>00039
-- Check for working C compiler: /usr/bin/clang
-- Check for working C compiler: /usr/bin/clang -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/clang++
-- Check for working CXX compiler: /usr/bin/clang++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Warning .....

-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - found
-- Found Threads: TRUE
-- Boost version: 1.66.0
-- Found the following Boost libraries:
--   chrono
--   coroutine
--   context
--   date_time
--   filesystem
--   program_options
--   regex
--   serialization
--   system
--   thread
--   atomic
-- Found OpenSSL: /usr/local/opt/openssl/lib/libcrypto.dylib (found version "1.0.2m")
-- Found ZLIB: /usr/lib/libz.dylib (found version "1.2.11")
-- Found Protobuf: /usr/local/lib/libprotobuf.dylib (found version "2.6.1")
CMake Warning at CMakeLists.txt:490 (message):
  Boost b2 executable not found.  docs target will not be buildable

-- Configuring done
-- Generating done
-- Build files have been written to: /app/ripple/rippled/build

In file included from /app/ripple/rippled/src/ripple/unity/basics.cpp:29:
/app/ripple/rippled/src/ripple/basics/impl/ResolverAsio.cpp:245:16: error: invalid operands to binary expression
      ('const boost::system::error_code' and 'int')
        if (ec == false)
            ~~ ^  ~~~~~
1 error generated.
make[2]: *** [CMakeFiles/rippled.dir/src/ripple/unity/basics.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
make[1]: *** [CMakeFiles/rippled.dir/all] Error 2
make: *** [all] Error 2

@wangdar Thanks for the report. I'm not a mac user, but I'll flag the error for the other mac users in the group.

What does the command printenv BOOST_ROOT output?

hi @HowardHinnant
the boost_1_66_0 is download from
https://dl.bintray.com/boostorg/release/1.66.0/source/boost_1_66_0.tar.bz2
and build , install to /app/boost_1_66_0
[wangdar@wangdar-MacBook-Pro rippled]$ printenv BOOST_ROOT
/app/boost_1_66_0
[wangdar@wangdar-MacBook-Pro rippled]$

Ok, thanks.  And within [wangdar@wangdar-MacBook-Pro boost_1_66_0] did you:
./bootstrap.sh
./b2 cxxflags="-std=c++14"

And did that look successful?

yes I will build boost with below
./bootstrap.sh
./b2 cxxflags="-std=c++14"

[boost::system::error_code const& ec]
ec is boost::system::error_code and compare to bool,
I don't know this is ok in c++?

checking...
In the meantime, what does clang++ --version output for you?

clang++ --version
Apple LLVM version 9.0.0 (clang-9<IP_ADDRESS>)
Target: x86_64-apple-darwin17.3.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

No, it is not valid to compare boost::system::error_code and false.
From [wangdar@wangdar-MacBook-Pro rippled]$, what does git log -1 output?

$ git log -1
commit 1c44c4a43eb9e46d8270892a9e49466ec7cc3efb (HEAD -> develop, origin/develop, origin/HEAD)
Author: seelabs<EMAIL_ADDRESS>Date:   Mon Jan 29 20:28:33 2018 -0500

    Set version to 0.90.0-b5

What is the cmake command you used?  For example I often use:
cmake -Dtarget=clang.debug.nounity -Dassert=On ../rippled/

@wangdar That line of code should be ec == 0 (see: https://github.com/seelabs/rippled/blob/develop/src/ripple/basics/impl/ResolverAsio.cpp#L245). I'm not sure how it was changed on your system, but if you change it back it should fix the issue.

  I'm saying this because src/ripple/basics/impl/ResolverAsio.cpp:245 in this repository does not look like:
if (ec == false)

It instead looks like:
if (ec == 0)

hi @bachase ,@HowardHinnant
I rm ripple folder, and check out again,
now it resolved.
I am sorry and apologize to you for the inconvenience.

No worries!  Glad you're up and running!


Implement ratelimits handling

This pull request adds support for handling and prevention of ratelimits.

[x] Delaying requests on hitting 429s
[x] Global ratelimit handling
[x] Delay requests when a ratelimit bucket is exhausted.
[x] Use X-Ratelimit-Bucket (ratelimit key, as referred in code) for precise handling of ratelimits between routes that share same ratelimit.
[ ] Add hooks for ratelimits.
[ ] Extensive testing.

What do you suggest using other than locks?

Well i would recommend using Futures instead, they will give you more flexibility to unlock requests, especially concurrently and maybe try to use __aenter__ aswell?

I don't quite like nor understand the statement of futures being better than locks for this situation. Both serve entirely different purpose and I don't know how you are thinking of using them. I'm fairly positive that locks are the most convenient option to use for this use case.


APK version

Hi, kindly, provide an Android version to download. That seems awesome. 🤩

Thanks for your interest! I will get to that ASAP.


proposed fix for json file reading issue

This was just a simple patch for #2.
We probably need to change .endsWith to something else, since that method is not supported in some browsers.

Maybe something like this is better:
var ext = file.name.split('.').pop();
if (file.type == "application/json" || (ext === 'json')) {


[framework] Integrate the ASG transformation module to the framework

Implement the repair module to call the patch generation java components.

You can find a package with the code repair jar and inputs in the storage: exchange\jasy\CodeRepair\CodeRepairTests.zip


Package updates

[One line description of your change]
Motivation:
[Explain here the context, and why you're making that change. What is the problem you're trying to solve.]
Modifications:
[Describe the modifications you've done.]
Result:
[After your change, what will change.]

@rkelly2001 can you please adjust the title and provide descriptions since the changes do not correspond to the PR purpose


Add a "replay" command

I'd like to be able to take one or more requests and send them (replay) to a gRPC server.
Examples
Replay one specific call:
$ binlog replay <my.binlog --call-id 123 --target localhost:1234

Replay all calls:
$ binlog replay <my.binlog --target localhost:1234

Replay all calls and send them to the peer addresses recorded in the binary log:
$ binlog replay <my.binlog

The input file is a binary log file, that includes both request messages and response messages. The replay command should ignore the response messages (if present) in the input file, and only send out request messages to the target gRPC service.

implemented for unary calls:
$ binlog replay /tmp/simple.bin --target localhost:1122  --expand --proto ~/tmp/gorpc/helloworld/helloworld.proto --call-id 2
ID      When    Method
2       2022/05/19 16:55:03.931803      /helloworld.Greeter/SayHello
<-      {
  "message":  "Hello foo from mynewserver"
}
<-{s}   OK

streaming calls also work now . closing


the log config param level in helm chart is wrong

What happened?

When I debugged clusterpedia locally, I found that "3" passed in by level could not make clusterpedia start normally.
And it is wrong in the `charts/clusterpedia/values.yaml` file which results in it will not work properly if opened.
What did you expect to happen?
The  log config param level should be fixed to the correct available value.
How can we reproduce it (as minimally and precisely as possible)?
Install by helm chart and set the log.enabled=true in the charts/clusterpedia/values.yaml
Anything else we need to know?
No response
Clusterpedia Version

# replace the pod name of clusterpedia apiserver
$ kubectl -n clusterpedia-system exec {clusterpedia-apiserver-pod-name} -- apiserver --version=raw
version.Info{GitVersion:"v0.4.1", GitCommit:"d1652e852b5c1f102b8a69c6b176f9055a502e1c", GitTreeState:"clean", BuildDate:"2022-08-04T05:41:46Z", GoVersion:"go1.17.13", Compiler:"gc", Platform:"linux/amd64", KubeVersion:"v1.24.1"}

# replace the pod name of clusterpedia clustersynchro-manager
$ kubectl -n clusterpedia-system exec {clustersynchro-manager-pod-name} -- clustersynchro-manager --version=raw
# paste output here

Host Kubernetes version

$ kubectl version
Client Version: version.Info{Major:"1", Minor:"25", GitVersion:"v1.25.0", GitCommit:"a866cbe2e5bbaa01cfd5e969aa3e033f3282a8a2", GitTreeState:"clean", BuildDate:"2022-08-23T17:44:59Z", GoVersion:"go1.19", Compiler:"gc", Platform:"linux/amd64"}
Kustomize Version: v4.5.7
Server Version: version.Info{Major:"1", Minor:"25", GitVersion:"v1.25.0", GitCommit:"a866cbe2e5bbaa01cfd5e969aa3e033f3282a8a2", GitTreeState:"clean", BuildDate:"2022-08-23T17:38:15Z", GoVersion:"go1.19", Compiler:"gc", Platform:"linux/amd64"}

log level must be one of [Silent, Error, Warn, Info], Default should be is 'Warn'.

/good-first-issue

/assign

@ONE7live hi, if you have any problems with this issue, we can discuss it together.

@ONE7live hi, if you have any problems with this issue, we can discuss it together.

@Iceber Sorry, the daily work is a bit busy. Just submitted a PR( #346 ) on this issue, please review


feat: add rocks info command

This feature is so uninspiring that I stopped working on it halfway, thus it's a draft :smile:
I'd like to first implement rocks doc and have a general interface for grabbing documentation, so that it can be showed to the user when invoking rocks info.
I renamed rocks show to rocks info because show has always felt incredibly unintuitive to me. We may even want to make rocks info work for any rock, both remote and local, and display whether it's installed or not in the output.

I realized that I don't actually need rocks doc as a prerequisite for this so I made this as-is :)


Commented piece of code for Golden creation

Hi,
It looks to me as an issue.
Or is there any reason why this context_quality metric value is excluded when creating golden in async way while it is considered for the sync one?
https://github.com/confident-ai/deepeval/blob/dd11c22b93d008ce0e386c88c86c96a38ad7c4f4/deepeval/synthesizer/synthesizer.py#L474
            # Synthesize Golden
            golden = Golden(
                input=evolved_input,
                context=context,
                expected_output=expected_output,
                source_file=(
                    source_files[index] if source_files is not None else None
                ),
                additional_metadata={
                    "evolutions": evolutions_used,
                    "synthetic_input_quality": scores[i],
                    # "context_quality": (
                    #     context_scores[i]
                    #     if context_scores is not None
                    #     else None
                    # ),
                },
            )
            goldens.append(golden)

@kritinv can provide more info on this?


Started from v0.5.0 the most of my glyphs (~90%) are not generated correctly!

Artem (@sapegin) - your answer seems to be too agressive... unreasonably. (https://github.com/sapegin/grunt-webfont/issues/231)
But I apologize if my description was confused...
I meant that fonts genarated using webfonts v0.5.0 are not displayed correctly while everything works just fine with previous versions.
From that I could assume that here is the BUG that affects to CORRECT font generation :)
That is the FACT! (http://en.wikipedia.org/wiki/Fact)
Also - I just ask: is it mean breaking support for Windows users or no.
Screenshots:

There was some magic involved that makes all icon the same size. But I don’t know how to make that magic works the same for both engines, so I disabled it by default. There are still an undocumented normalize options that could make generation as it was in previous versions.
But. It’s better to make sure that all your icons are of the same size (~1000px). In that case it should work fine in new version with default options.
There are two related issues: #222 and #221.

options.normalize = true did the trick!
Thanks a lot Artem!


luci-app-attendedsysupgrade: show updated packages

since the update server now checks packages for updates the web view
will show all changed package versions. this PR is also important to
support future package transformations of the server
Signed-off-by: Paul Spooren<EMAIL_ADDRESS>

If possible I'd like to request commit access as there will be many more updates on the web interface.


minor: add encoding and decoding for policy rule name

Describe your changes:
This PR will add support for encoding and decoding the policy rule name when editing the rule. This ensures that characters like "/" in the rule name are not treated as part of the route.

Type of change:

[x] Improvement

Checklist:

[x] I have read the CONTRIBUTING document.
[ ] My PR title is Fixes <issue-number>: <short explanation>
[ ] I have commented on my code, particularly in hard-to-understand areas.
[ ] For JSON Schema changes: I updated the migration scripts or explained why it is not needed.

@chirag-madlani @Sachin-chaurasiya are we checking everywhere where we may not have added the urlencode?


Consider how to avoid recalculating constraint diffs for everything when making a change

So if you have a scenario where you have a complicated view and one thing changes.

Do we want to do the work of recalculating things for all components under that viewlayout?
Is there a way to only reload certain components without potentially causing layout issues because of the rest of the layout not changing? (programmer errors)
If they explicitly reload certain parts, does it defeat the purpose of the library by not making it completely declarative? Is there a way to still make this declarative?

this is involved with https://github.com/HotCocoaTouch/DeclarativeLayout/issues/39


Bug: error TS2344: Type 'NonNullable<TStateValue[K]>' does not satisfy the constraint 'StateValue'

XState version
XState version 5
Description
When I transpile my project, I get an error regarding the xstate types:
`
node_modules/xstate/dist/declarations/src/State.d.ts:5:49 - error TS2344: Type 'NonNullable<TStateValue[K]>' does not satisfy the constraint 'StateValue'.
Type 'NonNullable<TStateValue[K]>' is not assignable to type 'StateValueMap'.
Index signature for type 'string' is missing in type '{}'.
  [K in keyof TStateValue]?: ToTestStateValue<NonNullable<TStateValue[K]>>;

`
Expected result
no type errors in transpilation
Actual result
type errors in transpilation
Reproduction
https://stackblitz.com/edit/github-9ms4pa?file=tsconfig.json
Additional context
No response

TypeScript's strict mode is required for XState v5 to typecheck correctly.

See here for all the TypeScript requirements: https://stately.ai/docs/typescript

I still get following messages when I import
"node_modules/xstate/dist/declarations/src/State.d.ts:5:49 - error TS2344: Type 'NonNullable<TStateValue[K]>' does not satisfy the constraint 'StateValue'.
Type 'NonNullable<TStateValue[K]>' is not assignable to type 'StateValueMap'.
Index signature for type 'string' is missing in type '{}'.
5     [K in keyof TStateValue]?: ToTestStateValue<NonNullable<TStateValue[K]>>;
~~~~~~~~~~~~~~~~~~~~~~~~~~~
Found 1 error in node_modules/xstate/dist/declarations/src/State.d.ts:5
Can I run it as node app as backend console


AGPUSH-1276 : make the exception message shorter

@matzew @edewit Can you take a look ?

:+1:

:+1:

landed on master -> 27aa85b47cdf93a67993692b923bc7ab6ede00ab


install instructions?

Hi there, I wanted to try this, but I dont see any instructions. I tried
mvn  clean install
[ERROR] Failed to execute goal on project common: Could not resolve dependencies for project com.hortonworks.streamline:common:jar:0.1.0-SNAPSHOT: Could not find artifact com.hortonworks.registries:registry-common:jar:0.3.0-SNAPSHOT in clojars (https://repository.apache.org/content/repositories/snapshots/) -> [Help 1]

@asanchezdelc was this with latest master?

@hmcl yeah

@asanchezdelc
Sorry for the inconvenience, seems like we miss to update instruction to build the project.
Development branches of Streamline rely on SNAPSHOT version of Registry right now, so you need to build & install Registry artifacts locally via mvn install before building Streamline.
The link for Registry project is here: https://github.com/hortonworks/registry

I've filed a new issue to Registry project to release 0.3.0: https://github.com/hortonworks/registry/issues/170
It would be ideal instead of guiding contributors or early adopters to build Registry project first.

@HeartSaVioR Thanks for the details. I was able to get a successful build. Although I have to specify a version for<EMAIL_ADDRESS>artifact and skip tests.
Now what? I checked README.md history and there were some instructions to run the webservice, but it doesnt seem to work.
java -cp target/webservice-0.1.0-SNAPSHOT.jar com.hortonworks.streamline.webservice.StreamlineApplication server conf/streamline-dev.yaml
Error: Could not find or load main class com.hortonworks.streamline.webservice.StreamlineApplication

Yes I think documentation is totally missing. We've documented Streamline (Streaming analytics manager) for HDF 3.0.0 but missed for users/contributors who want to build and use Streamline directly.
https://docs.hortonworks.com/HDPDocuments/HDF3/HDF-3.0.0/bk_getting-started-with-stream-analytics/content/ch_building-end-to-end-app.html
https://docs.hortonworks.com/HDPDocuments/HDF3/HDF-3.0.0/bk_streaming-analytics-manager-user-guide/content/ch_sam-manage.html
Will handle this. Maybe still need to refer how to use from HDF document for some time.

So can I use streamline directly without HDF？


add clue id tracking

We did some additional work on top of this for the map clues and merged it into Crowdsourcing/clues. Thanks!


Jukebox

Description
Adds a functional Jukebox to the stations! Capable of playing music, being paused, queueing up songs, and a whole load of other stuff the engine doesn't actually allow for yet ' v ,

Tasks

[x] Functional system
[x] Fix the next song not playing automatically
[x] Clean UI
[ ] Sync music over the network for new joins Will probably require a pretty heavy rewrite, this is fine for now.
[x] More songs
[x] Construction
[x] Research Uses stock parts, that counts
[x] Craftable boards
[ ] Larger control icons
[x] Animate sprite

Media

Youtube Demonstration
https://github.com/Simple-Station/Parkstation/assets/81725545/4332b071-1c13-48f4-948d-5e3dcc854ab3

Changelog

:cl: Pspritechologist & DEATHB4DEFEAT

add: Nanotrasen have heard that employees function better when allowed to listen to music. They trapped some elves in a wooden box- And wouldn't you know it, it's more than just generated digital notes!*

Also note; The system needs to react to the unpaused event so that the saved time is set correctly.

@DEATHB4DEFEAT You gotta make nicer UI icons, and increase the size of the UI a bit. It needs a once over (is all of the text in it good to keep?), and then this is ready for prime time baybeeee


Possible dead lock between DefaultKafkaProducerFactory.expire and DefaultKafkaProducerFactory.removeProducer

Back-port issue for: https://github.com/spring-projects/spring-kafka/issues/2744

Fixed via: https://github.com/spring-projects/spring-kafka/commit/bd570991e8c860a6c8b47c1a15d6c49eac69bea3


Unexpected habitat result in LNIC

From @nickw-CWF

I was looking at some systems in the Lower Nicola, and noticed that there are some stream segments that seem like they should be flagged as ST habitat that aren’t. I know we had discussed removing the MAD parameter requirements for all species habitat in the Lower Nicola, but even if this hasn’t been implemented yet there are still segments that seem to meet the parameter requirements that aren’t being flagged as habitat.

For example, linear_feature_id = 701760481
gradient = 0.0333
mad_m3s = 0.13296

Which are within the rearing thresholds of < 0.074 and 0.02-60, respectively.

Not sure what the issue is at this point.

That is an interesting issue.
For that segment, the discharge and gradient do not meet spawning model thresholds - there is not enough discharge. But FISS indicates there is spawning at that location, so we code it as spawning (50m on each side of the FISS point).
As noted, the stream does meet rearing criteria and rearing can be coded on spawning segments.
But... the rearing model currently doesn't recognize spawning that comes from data/user_habitat_classification.csv, it only looks at modelled spawning.
I'd call this a bug, we should probably model rearing on all spawning.

Thanks for looking into this Simon. I would agree that we want to model connectivity based on all spawning data.


Update 22-to-30.md

Fixes #18113
@macux  please review

@macux  thanks for bringing this to our attention.


ci: fix failure

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 83.647%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
445

Relevant Lines:
532

💛  - Coveralls

Fixed by https://github.com/sermojohn/gitmono/pull/41.


Cherrypick PR-1105 into 0.21 release

Description:
Cherry-pick into 0.21 release for PR https://github.com/hashgraph/hedera-json-rpc-relay/pull/1105

Codecov Report
Patch coverage: 5.55% and project coverage change: -0.61 :warning:

Comparison is base (f9e3b90) 74.74% compared to head (08d6202) 74.14%.

:exclamation: Current head 08d6202 differs from pull request most recent head 4879121. Consider uploading reports for the commit 4879121 to get more accurate results

Additional details and impacted files
@@               Coverage Diff                @@
##           release/0.21    #1111      +/-   ##
================================================
- Coverage         74.74%   74.14%   -0.61%
================================================
  Files                30       30
  Lines              2083     2100      +17
  Branches            406      409       +3
================================================
  Hits               1557     1557
- Misses              417      434      +17
  Partials            109      109

Impacted Files
Coverage Δ

packages/relay/src/lib/clients/sdkClient.ts
19.73% <0.00%> (-1.53%)
:arrow_down:

packages/relay/src/lib/errors/SDKClientError.ts
55.55% <0.00%> (-3.27%)
:arrow_down:

packages/relay/src/lib/eth.ts
83.78% <100.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


Update Google Analytics Tracker ID

New features and functionality require a new tracker id, need to update this so that additional metrics can be captured.

Enabling advertising Collection Id.


Implement Construction API

Need to implement the various construction API endpoints:

[ ] combine
[ ] derive
[ ] hash
[ ] metadata
[ ] parse
[ ] payloads
[ ] preprocess
[ ] submit

Time Estimate: 2 hours


spider engine

https://realpython.com/blog/python/web-scraping-with-scrapy-and-mongodb/

https://github.com/geekan/scrapy-examples


Authentication provider is still present on Authentication settings page after extension is uninstalled

After removing Red Hat SSO extension using Extensions Settings page it still shows up on Authentication settings page
.

I would add that with extension disabled, it also, at least visually, keeps user logged in.


Bump version 1.1.5 -> 2.0.0

There have been a couple of recent breaking changes
so bumping the major version is required.

This change depends on #92 and #94 but I wanted to start documenting the upgrade path.

Ok, as soon as this commit wil be accepted, we'll change the version.


Add an easy util for getting the local from accept-language header

I noticed that this package is a dependency of @shopify/react-network. Does this cover our intended use case or are we looking for something else?

Or are we just looking to wrap that functionality and export it as a separate utility?

I'm think this is what @cartogram intended but it might be worth double-checking with him.

Closed in https://github.com/Shopify/quilt/pull/909


Split out TokensController and CollectiblesController from AssetsController

Addresses part 1 of #516
Objective
Split AssetsController into two new controllers (Tokens and Collectibles) that manage state these separate asset classes without introducing any functional changes.
Tested in mobile for regressions with changes in metamask-mobile  #2881

@Gudahtt thanks for the quick review! Pushed requested changes!


[BUG] State node not cloned for value changes

getRefForWriting() clones state nodes for node addition but not for value changes.

Fixed with #298 .


double-check BLEU computation

Make sure the BLEU (and wordF) scores reported at http://workbench.qtleap.eu/evaluation/ agree exactly with the scores reported at http://qtleap.eu/wp-content/uploads/2015/04/QTLEAP-2015-D2.4.pdf#page=36

I think, that I fixed the problem. I compared the BLEU scores with compute_bleu_for_all_pilots.pl and everything is correct. The problem was in normalization, tokenization and UTF-8 support. Please see ba51d3b38f24769ac49b7b18f2be56aaecd7a1ac and 417c7d3f412b119ec25387977a2f95868d1e9f64.
I also changed smoothing in precision and recall computation to be compatible with mteval-13a.pl. Please see 63cf3dfe8f2475191429d31b0f3101d6a4bdaffd.

Great. Thanks. I am about to close this issue, just waiting while @lefterav updates http://workbench.qtleap.eu/evaluation/.
(GitHub does not allow to assign an issue to multiple people, as you can see from my unsuccessful attempts:-).)

The BLEU scores seem to be correct, but the original BLEU (mteval-13a.pl without -c switch) is case insensitive, while in MT-CompareEval BLEU means case sensitive BLEU. The alternative variant is here called BLEU-cis and it is shown only in the Statistics tab. I think we will need to change this (but that's not related to this issue).

Closed with #51


Increment version for formrecognizer releases

Increment version for formrecognizer releases

Can one of the admins verify this patch?


Handle reconnect to Z-Wave JS server

The client holds a driver object based on the initial received state. This object has listeners attached from Home Assistant.
When we reconnect, the client now holds a driver object that no longer represents the latest state. We don't deal with this case.
I can think of two options:

We throw away the driver object. Consumers of this lib will just have to re-initialize their stack + attach listeners. In the case of HA we would reload the config entry.
We teach each model to update based on the latest state and add new events to indicate the consumer that they have to re-scan each model for latest data/values.

makes most sense to me since the whole state is new. There's no value then to update models individually.

makes most sense to me since the whole state is new. There's no value then to update models individually.

I agree. That way we know for sure everything is initialized correctly.
For Home Assistant, we should add generic logic that if an entry is removed from the state machine and it is still registered in the entity registry, we write unavailable. That way this will go seamless.

I agree. That way we know for sure everything is initialized correctly.
For Home Assistant, we should add generic logic that if an entry is removed from the state machine and it is still registered in the entity registry, we write unavailable. That way this will go seamless.

Generic logic for HA: https://github.com/home-assistant/core/pull/45528

Generic logic for HA: https://github.com/home-assistant/core/pull/45528

Hmmm, this seems a bit complex when only a short hiccup of the connection occurs. I agree that we need some way to ensure we have the latest state but this feels a bit overly complex and, besides... why are we solving the reconnecting HA and not in the library ?
How I would do it:
If the connection is dropped and reconnected, simply ask the server for the full state again and loop through the values.
We have the full state stored in the lib so we can very easily detect any changes.
Logic to mark the entity as unavailable when the connection is down is already down.

Hmmm, this seems a bit complex when only a short hiccup of the connection occurs. I agree that we need some way to ensure we have the latest state but this feels a bit overly complex and, besides... why are we solving the reconnecting HA and not in the library ?
How I would do it:
If the connection is dropped and reconnected, simply ask the server for the full state again and loop through the values.
We have the full state stored in the lib so we can very easily detect any changes.
Logic to mark the entity as unavailable when the connection is down is already down.

But ofcourse there are two possible scenarios:

Connection to the WS server got dropped somehow and is reconnected. Only a short period of time and there are possible no or very few changes to process.
The whole WS server got reconnected. As the whole mesh is starting and interviewed there are many changes.
The whole WS server reconnects to a different (or factory resetted) stick, huge changes to process.

But ofcourse there are two possible scenarios:

Connection to the WS server got dropped somehow and is reconnected. Only a short period of time and there are possible no or very few changes to process.
The whole WS server got reconnected. As the whole mesh is starting and interviewed there are many changes.
The whole WS server reconnects to a different (or factory resetted) stick, huge changes to process.

In all situations we already mark the integration as unavailable, restarting the integration wouldn't change anything for the user experience once https://github.com/home-assistant/core/pull/45528 is merged.

In all situations we already mark the integration as unavailable, restarting the integration wouldn't change anything for the user experience once https://github.com/home-assistant/core/pull/45528 is merged.

This will be fixed by https://github.com/home-assistant-libs/zwave-js-server-python/pull/78 and connected changes in Home Assistant.


Blur-up: Copy style attribute from original  tag

We render each img tag server-side with a style attribute, specifying object-position: xx% yy% which is specific to each individual image. Since the object position is specific to each single image, it cannot be specified in css, but we need to do it in style attributes.
However, this does not work together with blur-up. The blurred image is positioned differently since its img tag is rendered without the object-position style attribute.
Right now I am using a js hack for this to copy the styles, but I would like to avoid this.
The blur-up plugin should copy the style attribute of the original img tag. I am happy to prepare a pull-request.
Thank you for an awesome package!
Mikael

Right now I am using a js hack for this to copy the styles, but I would like to avoid this.

Sorry for not answering. Can you show me your hack?

Below is my js code. I am using Initialize.js for it.
$(function () {

    /* work-around for Lazysizes.blurup :
     * copy style tag from original img tag to blur-up img tag
     *
     * Uses Initialize library
     * */
    $.initialize(".ls-blur-up-img", function () {
        $(this).attr("style", $(this).prev("img").attr("style"));
    });
});

The change to ls.blur-up.js does not work as intended. Line 87:
blurImg.style = img.style;
needs to be replaced with with
blurImg.style.cssText = img.style.cssText;
then it works as expected.

what an embarrassment ;-). [1,2] + [3,4] = ???

:-) Unfortunately, I am capable of stuff like this every day...


Scrolling creates new region on firefox

Hi,
On firefox browser, when I scroll from left to right the new region created. This can also reproduced on sample example given on website.
http://wavesurfer-js.org/example/annotation/
Any one has solution of it?

Solved in ##851.


Check ophyd_async.core.SignalMetaData vs event model for NotRequired vs Optional fields

          Do we not still need to set this equal to `None` if we want it in the dict with the same behaviour? If not it'd be `NotRequired` right?

Originally posted by @evalott100 in https://github.com/bluesky/ophyd-async/pull/580#discussion_r1763241704

Not needed now we use the event model dicts in ophyd-async


precise, recall, fscore 全为0%

I'm submitting a ...  (check one with "x")
[x] bug report
[ ] help wanted
[ ] feature request

Current behavior
precise, recall, fscore 全为0%
定位率 NAN
请选择一项操作:2
Begin to test the easypr accuracy!
------------------
图片:冀A74110
No ground truth found!
蓝牌:冀A74110 (d)
Recall:0%, Precise:0%, Fscore:0%.
valOstu: 0.96752
valAdap: 0.922377
------------------
图片:黑B99999
No ground truth found!
蓝牌:黑B99999 (d)
Recall:0%, Precise:0%, Fscore:0%.
------------------
Easypr accuracy test end!
------------------

Reproduction of the problem
100%
Please tell us about your environment:

System: OS X 10.12

Compiler version/IDE: gcc4.8

CMake version: 3.8.0

OpenCV version: [ 3.2 ]

自己的测试图片需要自己写groundTruth，可以参考EasyPR自带的groundTruth格式


Lower case

hello ,I want to make lowercase .
I dont know what is happening
FOLDER -
$code            = Hashids::connection('code')->encode(2);

 'code' => [
            'salt' => 'qwertyiopasdfghjkllzxcvbnm123456789',
            'length' => 8,
        ],

It is like this the result -
xpoNeP90

with some words uppercase . how could I make it all lowercase ? thanks

Try changing the alphabet instead of the salt: https://github.com/vinkla/hashids#using-a-custom-alphabet


hashtag without a space or line below is not converted to tag

Describe the bug
In my memo notes if i document as below. Please see #note is not having any space or line below the hashtag, its not getting converted to tag and not displayed in tags.
This is hashtag note
#note
Steps to reproduce
In my memo notes if i document as below. Please see #note is not having any space or line below the hashtag, its not getting converted to tag and not displayed in tags.
This is hashtag note
#note
Screenshots or additional context

https://github.com/usememos/memos/wiki/Best-practices-to-write-a-TAG

Noted. Thank you. Even if its on the last line and there is no space after the hashtag, its not getting displayed or considered as a Tag.


Generalize lib interface to return context objects

Description
Generalizes the KNNLibrary interface to return an object for both search and indexing so that the plugin can search/index against them. The names of the methods are "getKNNLibraryIndexBuildContext" and "getKNNLibrarySearchContext". In the future, I could see these methods changing into something like getKNNLibraryIndexBuilder and getKNNLibrarySearcher so that the engine actually encapsulates the interactions with the libraries, but that needs some more thought. For now, this change makes it more clear what one of the central roles of the KNNLibrary interface is - telling the plugin how to index/search.
For the quantization framework, making the interface return objects will allow us to properly configure both quantization as well as whatever index we want to use with it. See the following locations for more details where we would do this:

https://github.com/jmazanec15/k-NN-1/blob/method-stage/src/main/java/org/opensearch/knn/index/mapper/MethodFieldMapper.java#L63
https://github.com/jmazanec15/k-NN-1/blob/method-stage/src/main/java/org/opensearch/knn/index/query/KNNQueryBuilder.java#L376

Again, no functionality change - just more refactoring to better support the quantization framework.
One thing I have realized from these changes is that he KNNMethodContext class needs to be sured-up a bit to be more robust. I will create a separate issue for this.
Check List

[X] Commits are signed per the DCO using --signoff.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

How about getKNNLibraryIndexingContext?

Addressed @heemin32 !


Unstructured paid API stuck again

My current environment is:
PyMuPDF==1.24.4
unstructured[all-docs]==0.14.0
nltk==3.8.1
pandas==2.2.2
llama-index==0.10.38
llama-parse==0.4.3
PyPDF2==3.0.1
fastapi==0.111.0
minio==7.2.7
uvicorn==0.29.0
psutil==5.9.8
boto3==1.34.109
botocore==1.34.109
My retry config parameters are:

My unstructured client object is:

This is the logs from the API:

Its just stuck from the last half and hour.
Please assist with this. Its urgent as the servers are stuck because of this

Apologies for all the issues here. We've stabilized the service and it should be working again. For transparency, we're very close to a brand new release of the hosted api. We've revamped the architecture to avoid these problems happening again. I'd recommend jumping into our community slack for updates.


Ensuring images in the user and listing cards link to the user

…tore
closes #918
closes #913

Please check #931 before merging


add gdsfactory and sax

Hi! If I didn't get it wrongly, gdsfactory is not a simulator, right? If it isn't, it should e placed elsewhere.


[best practices] Processing by orbit

Hello, I want to ask a question regarding best practices around processing tiles and specifically focusing on tile-orbits.
We made a small run of MAJA and processed some L1C products and got great results. Processing a different tile we got less accurate results. We went back to see what had happened and what came to our mind is the mix of the orbits. The first experiment was on a tile that happened to always have products be on the same orbit, but the second tile had mixed orbits.
Sen4CAP, using MAJA as the processor to produce L2A products, internally separates tiles per orbit and uses a tile of the same orbit as the previous product.
With those observation, would it be better to separate processing per tile and orbit or let Maja pick any tile as a previous product, regardless of the tile orbit?
We are running a test case for that second experiment, but this time with separate L1C products per orbit. I will come back with some results and new observations, but your experience and observations would certainly be valuable here.
PS: I am using MAJA v4.2.1 through StartMaja
On v4.2.1, StartMaja will just get the closest product to be used as the previous. This seems like a good place to implement a better heuristic that can pick a previous product based on some strategy (ie, the closest of the last 4 with the least cloud-coverage.)
https://github.com/CNES/MAJA/blob/4.2.1/StartMaja/Chain/Workplan.py#L259-L266

Hi Ivan,
thanks for sharing this feedback, which makes sense.
I am just wondering if your appreciation is related to the atmospheric correction of to cloud detection. Our own experience is that merging different swaths improves the cloud detection in cloudy regions because of the enhanced revisit, improves the aerosol detection in summer, but degrades it in winter, because our directional correction model is quite bad when the sun is low. We are implementing a better directional correction in MAJA 4.3.
Your own validation results will be very welcome in this regard, thanks a lot !
It is quite easy to run start-maja after separating the swaths in different folders before launching start-maja, but we should think of implementing what you suggest as an option. Our development pipeline is quite full, so it might take a while.
Best regards,
Olivier

Hi Oliver and thanks for the swift reply,
you are right, I should have been more specific. Our remarks refer to cloud detection accuracy.

merging different swaths improves the cloud detection in cloudy regions because of the enhanced revisit

This is reasonable and we want to take advantage of this. However, this case is about a tile where most of the time the different orbits come one after the other when sorted by date-time (R050 > R093 > R050 > R093 > R050 > R050 > R093 > etc). The important part is that one of the orbits has half the area as NODATA and the other is full.
Given such a case, when MAJA runs in NOMINAL mode and is processing a tile in the orbit that has full image coverage, the previous product that it will use, will be one with half the image as NODATA. Wouldn't this be a problem?
(I see that MAJA uses a single previous product. Could it use more?)

It is quite easy to run start-maja after separating the swaths in different folders

We already have an L1C set separated in different ways (year, tile, etc) but now we added the orbit, too. We are now running MAJA for each orbit separately and will come back with results.
PS: The specific tile we are experimenting with is 34SFJ and the two orbits are R050 and R093.

To answer your question, MAJA uses only one previous level 2 product, but this product conveys information from a lot of previous images. We use a composite image at 240m resolution, which for each pixel contains the most recent cloud free observation, and it is used as a reference to compare with the new image to classify. This composite is updated at each new image with the new cloud free pixels, and stored in the level 2 (the private part). As a result, if each half of the image is observed on a different swath, it should not be an issue.

Dear Olivier,
catching up to my colleague @c00kiemon5ter,  we would like to inform you that our experiments on tile 34SFJ (for year 2019), indicated that using both orbits at the same MAJA run, indeed gave better results for most scenes, concerning cloud detection.
However the first remark that Ivan pointed out, referring to much better cloud detection results on another tile (34SEG) is still valid and after further investigation we think that this can be directly associated to differences of the image's landscape.
In particular 34SFJ includes the largest agricultural plain of Greece, i.e., great heterogeneity in dark/bright values throughout the year and frequent alternation between those in just a few meters. MAJA 4.2.1 gives us a lot of false positive pixels classified as clouds, that are actually clear (cloud-free) observations of crop parcels, even in (almost) cloud-free images.
e.g. True color composite of T34SFJ_20191016 & detected clouds in red (cloud mask >0) from CLM_R1 product of MAJA 4.2.1.

What is also interesting to point out is that the false-positive issue concerning agricultural areas is not observed in that extent in some results we produced using MAJA 3.2.2 version (via Sen4Cap). It seems that older versions produce cloud mask at a coarser resolution and thought this comes with a worse and expanded delineation result of the cloud figures, at the same time there are significantly fewer issues concerning these false positives (salt & pepper) pixels detected as clouds.
e.g. Cloud masks from versions 4.2.1 & 3.2.2 on T34SFJ_20190718.

We read here (that https://labo.obs-mip.fr/multitemp/maja-4-2-is-available-and-open-source/) that 4.2 version gives the possibility of processing cloud detection at 120m (instead of 240m), is this the case when using ''start-maja'' for 4.2.1 version ? We have also observed a significant difference in the size of CLM products between 4.2 and 3.2 versions.
Thank you in advance.
Best Regards,
Christina

Thanks Christina.
Very interesting, it is a case we had not observed so far. It might be indeed a consequence of the increase or resolution in MAJA 4.2 cloud mask, but we will also need to check if all the parameters are OK. Maybe the change of resolution requires a different tuning of parameters.
Could you please provide the command line used so that we can try to reproduce that case and analyze what happens ?
Our analysis, with the help of @jerome-colin, will take some time, don't expect an answer before 2021.
Best regards,
Olivier

We run MAJA v4.2.1 (official binary release) on Ubuntu 18.04 (a Vagrant VM), in a dedicated miniconda environment setup with python=3.6.8 gdal=2.3.3 and scipy=1.3.0.
The command we use after having activated the conda environment is the following:
$ cat run.sh
#!/bin/sh

set -e

PATH="/opt/maja/v4.2.1/bin:/opt/miniconda/bin:${PATH}"
. "/opt/miniconda/etc/profile.d/conda.sh"

conda activate maja
startmaja --type_dem eudem -y -f config.ini -t 34SFJ -d 2018-12-01 -e 2020-01-31

$ cat config.ini
[Maja_Inputs]
exeMaja=/opt/maja/v4.2.1/bin/maja
repWork=/data/work
repGipp=/data/gipp
repMNT=/data/dtm
repL1=/data/L1C
repL2=/data/L2A
#repCAMS=/data/CAMS

[DTM_Creation]
repRAW=/data/dtm/raw
repGSW=/data/dtm/gsw

I will later upload the vagrant recipe and let you know, so that you can replicate the exact environment.

It might be indeed a consequence of the increase or resolution in MAJA 4.2 cloud mask,

Does this mean that MAJA v4.2 uses 120m resolution to create the cloud masks by default (while previous releases -v3.2.2 in particular- used 240m)?

but we will also need to check if all the parameters are OK.

If there is something we can help with, let us know and we can have a run with different params/settings.

Dear @c00kiemon5ter
I'm not sure it's related to your specific environment. I just launched a run for 34SFJ on our own hpc config. @olivierhagolle and I will investigate and let you know asap.
Thanks for your help, it's really useful for us to get user feedback !

Olivier and Jerome Greetings from Athens,
and wishes for a prosperous 2021!
May I ask if you had any news regarding the false positives overestimation for the higher resolution Cloud Mask for v 4.2 ?
Thank you once again,
Christina

Olivier and Jerome Greetings from Athens,
and wishes for a prosperous 2021!
May I ask if you had any news regarding the false positives overestimation for the higher resolution Cloud Mask for v 4.2 ?
Thank you once again,
Christina

Dear Christina,
we made some test, in particular on the tiles your mentioned, and found that adjusting the minimum threshold on reflectance variance with time in the blue (B2) could reduce the number of false positives for cloud detected in multitemporal mode at 120m. You can find this parameter in the GIPP directory, in the files names L2COMM for S2A and S2B. At line 234:
<Min_Threshold_Var_Blue>0.016</Min_Threshold_Var_Blue>
A value of 0.018 seems more adequate for simulations at 120m in your area of interest, but I refrain from making it a generality.
In the meantime, we found that some other parameters in the GIPPs are not well adapted to the 120m resolution (which affect the reflectances rather than the cloud mask). For now, we'd recommend you to keep processing products at 240m, and we'll reset this 240 resolution as the default one until the release of Maja 4.4 (we will have fixed all this issues until then).
Hope this help,
With our best wishes for this New Year !
Jerome

Hello @jerome-colin
(also, following up from https://github.com/CNES/MAJA/issues/82#issuecomment-766671932)
from what I see in our installation of MAJA-4.2.1 (manually built), we already have the L2CoarseResolution set to 240.
$ pwd
[reducted]/tools/maja/v4.2.1/install/maja/4.2.1

$ grep -RF "L2CoarseResolution" ./lib/python/StartMaja/userconf
./lib/python/StartMaja/userconf/MAJAUserConfig_SENTINEL2_MUSCATE.xml:	    <L2CoarseResolution>240</L2CoarseResolution>
./lib/python/StartMaja/userconf/MAJAUserConfig_LANDSAT8_MUSCATE.xml:	    <L2CoarseResolution>240</L2CoarseResolution>
./lib/python/StartMaja/userconf/MAJAUserConfig_VENUS_MUSCATE.xml:    <L2CoarseResolution>100</L2CoarseResolution>
./lib/python/StartMaja/userconf/MAJAUserConfig_SENTINEL2_TM.xml:	    <L2CoarseResolution>240</L2CoarseResolution>
./lib/python/StartMaja/userconf/MAJAUserConfig_LANDSAT8.xml:	    <L2CoarseResolution>240</L2CoarseResolution>
./lib/python/StartMaja/userconf/MAJAUserConfig_VENUS.xml:	    <L2CoarseResolution>100</L2CoarseResolution>
./lib/python/StartMaja/userconf/MAJAUserConfig_SENTINEL2.xml:	    <L2CoarseResolution>240</L2CoarseResolution>

$ grep -RF L2CoarseResolution ./etc/conf/
./etc/conf/user/MAJAUserConfig_SENTINEL2_MUSCATE.xml:	    <L2CoarseResolution>240</L2CoarseResolution>
./etc/conf/user/MAJAUserConfig_LANDSAT_MUSCATE.xml:	    <L2CoarseResolution>240</L2CoarseResolution>
./etc/conf/user/MAJAUserConfig_LANDSAT8_MUSCATE.xml:	    <L2CoarseResolution>240</L2CoarseResolution>
./etc/conf/user/MAJAUserConfig_VENUS_MUSCATE.xml:	    <L2CoarseResolution>100</L2CoarseResolution>
./etc/conf/user/MAJAUserConfig_SENTINEL2_TM.xml:	    <L2CoarseResolution>240</L2CoarseResolution>
./etc/conf/user/MAJAUserConfig_LANDSAT8.xml:	    <L2CoarseResolution>240</L2CoarseResolution>
./etc/conf/user/MAJAUserConfig_VENUS.xml:	    <L2CoarseResolution>100</L2CoarseResolution>
./etc/conf/user/MAJAUserConfig_SENTINEL2.xml:	    <L2CoarseResolution>240</L2CoarseResolution>

Only MAJAUserConfig_VENUS.xml and MAJAUserConfig_VENUS_MUSCATE.xml have the L2CoarseResolution set to 100 (not 120).
The same holds for this repository, searching for L2CoarseResolution in XML files:
https://github.com/CNES/MAJA/search?l=XML&q=L2CoarseResolution
So, I think we are already processing at 240m resolution. Are we missing some other configuration option?

Hi @c00kiemon5ter,
Actually, the L2CoarseResolution is set in S2product.py and the userconf L2CoarseResolution is no longer in use since the release 4.2.1. This wasn't clear to me until I tried to jump from 240m to 120m for comparison purposes, and opened the
following issue.
You may still guess the resolution of your DTM from the size of some files. As an example, for your 34SFJ tile, the file  S2__TEST_AUX_REFDE2_34SFJ_3001_ALC.TIF is around 1.6M at 120m, and 411K at 240m.
That's obviously not satisfying, and we'll modify the code to let the user set this coarse resolution explicitly (eg. add a --L2CoarseResolution option to startmaja), probably for release 4.4.
Jerome


Refactor to allow for adding copies of cells to design?

Currently, adding a cell to the design seems to require that it be a full PHIDL Device with a name property. I think this slows down the code because it means all the sub-Devices must be rerun if you want a copy of the cell. Particularly for code with a lot of boolean operations (e.g. outline, invert -- any CPW-style design), this can take a while.
In my normal workflow I "copy" devices by adding multiple Device References of them to the overall Device. This is not possible within the current structure.
In terms of refactoring, I imagine this referencing or copying could occur either (1) at the cell level, as a function parameter (e.g. "num_copies = 2" just adds 2 references for everything to the Device returned by the cell function) or (2) at the Design level during the place_on_chip call (e.g. add a function parameter "num_copies=2" adds 2 references of the passed-in cell to the Design.)
Probably place_on_chip/place_remaining_devices the better level to do this at, as it means less work to add a cell and less work for the current refactor.

This sounds good, I would think you could just have it so that place_on_chip accepts references instead of just devices so that you could have copies of cells (without actually having to copy them). I would think this would also be good to make sure the reference carries over to the generated GDS so the final GDS file isn't huge.

This is resolved for place_on_chip with the latest pull request, but we should consider whether it makes sense for place_remaining_devices to be able to create multiple reference/copies or not. I think maybe not, since it's more about filling in empty spaces with leftover cells, but could be persuaded otherwise


Use websockets instead of HTTP tunnel

🗣 Description
This pull request updates the version so that we can rebuild and pick up the changes from cisagov/ansible-role-guacamole#11.
💭 Motivation and Context
Using websockets instead of HTTP to stream data to the client results in a noticeable performance improvement.
🧪 Testing
These changes have been successfully applied to env1 in staging.
🚥 Types of Changes

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (causes existing functionality to change)

✅ Checklist

[x] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[x] I have read the CONTRIBUTING document.
[ ] I have added tests to cover my changes.
[x] All new and existing tests passed.

@dav3r - once you deploy env4 and are happy with it, let me know so I can mark this PR as ready for review.

@dav3r - once you deploy env4 and are happy with it, let me know so I can mark this PR as ready for review.

As discussed in chat, we are continuing to troubleshoot why this latest AMI is randomly locking up and requiring redeployment of the instance.

As discussed in chat, we are continuing to troubleshoot why this latest AMI is randomly locking up and requiring redeployment of the instance.

The reason for the AMI randomly locking turned out to be a linux kernel bug.  (The fact that we're running Docker on that instance is probably what is triggering the bug, which involves cgroups.)  I modified cisagov/ansible-role-upgrade to tweak the grub configuration to boot into an older kernel in Debian 10.  Once the kernel bug is fixed upstream and is available in Debian 10 we can remove those changes from cisagov/ansible-role-upgrade.


[Feature] Pressing TAB should select the currently selected option in the Add New Relic menu

Motivation

Currently in this view, pressing TAB skips the option instead of selecting it. Users now have to search for "Crit Rate" then press ENTER then TAB before they can go to the next input box
Objective

Pressing tab should select the currently hovered/selected option before moving to the next input box

This is not supported by antd


Removed deprecated template provider

what

Removes references to the hashicorp/template provider which is deprecated.

why

So we can use module on M1 Macs.

references

closes #40

/test all

/rebuild-readme

/test readme

@nadenf could you run the following commands locally and commit:
make init
make readme
terraform fmt

We're having some problems with our CI on this repo for some reason.

@Gowiem .. I don't see any checkbox to allow access. And when I run make/pr-autoformat the following error message:
Starting cloudposse/build-harness:slim-sha-bb66c2f
docker run --name build-harness \
		--rm -it \
		-e PACKAGES_PREFER_HOST=true \
		-e AWS_ACCESS_KEY_ID -e AWS_SECRET_ACCESS_KEY -e AWS_SESSION_TOKEN -e TERM -e AWS_PROFILE -e AWS_REGION -e AWS_DEFAULT_PROFILE -e AWS_DEFAULT_REGION -e AWS_CONFIG_FILE -e AWS_SHARED_CREDENTIALS_FILE \
		 \
		-v /Volumes/Docker/Developer/harana-oss/terraform-aws-multi-az-subnets:/opt \
		--workdir /opt \
		--entrypoint /usr/bin/make \
		cloudposse/build-harness:slim-sha-bb66c2f terraform/fmt readme
Unable to find image 'cloudposse/build-harness:slim-sha-bb66c2f' locally
docker: Error response from daemon: manifest for cloudposse/build-harness:slim-sha-bb66c2f not found: manifest unknown: manifest unknown.
See 'docker run --help'.
make: *** [build-harness/runner] Error 125

@nadenf Sorry, you hit a bug in build-harness which is now fixed. Please run
make init
make pr/auto-format

again


engine: rename some methods name of Master/Worker definitions

What problem does this PR solve?

Issue Number: ref #5853
What is changed and how it works?
This PR changes some method's names.

Use Init instead of InitImpl.
Use Close instead of CloseImpl.
Use Stop instead of StopImpl.

Check List
Tests

Unit test
Integration test
Manual test (add detailed scripts or steps below)
No code

Questions
Will it cause performance regression or break compatibility?
Do you need to update user documentation, design documentation or monitoring documentation?
Release note
None

I will close this PR because the call stack of Worker/WokerImpl/BaseWorker are coupled and need further refactor.


Create store Fixed

Nice work @marvelken
Deployment failed due to the error:
Failed to compile.
./app/(Home)/components/MainSection.tsx
Module not found: Can't resolve '../../../styles/MainSection.css'
https://nextjs.org/docs/messages/module-not-found

please fix.


facilitate ability to css animate styles of top and bottom shadows

Right now the top and bottom shadow elements are only in the DOM when the table has been scrolled to a point where they are needed. This makes them impossible to style when the table is at the top and/or bottom and also makes it difficult to animate their styles with CSS. Ideally these would work like the shadow on column groups where .public/fixedDataTableRow/columnsShadow is added to an existing element (.public/fixedDataTableRow/fixedColumnsDivider) when needed.

or they are left in the dom and a class is used to show or hide these shadows.


Server path does not use public paths override after bump to 2.5.x

Description
The default Brunch server does not use the paths.public override, and instead throws an ENOENT: no such file or directory error.  However, the files are correctly compiled into the directory specified in overrides[setting].paths.public.
Environment

Brunch: 2.5.2
Node: 5.8.0
NPM: 3.7.3
Operating system: OS X 10.10.5

package.json contents
{
  "devDependencies": {
    "brunch": "2.5.2",
    "handlebars-brunch": "2.0.1",
    "javascript-brunch": "2.0.0",
    "ng-annotate-brunch": "1.7.2",
    "sass-brunch": "2.0.0",
    "clean-css-brunch": "2.0.0",
    "uglify-js-brunch": "2.0.1"
  },
  "dependencies": {
    "dotenv": "2.0.0",
    "npid": "0.4.0"
  }
}

brunch config contents
preview override in brunch-config.js:
module.exports = {
  config: {
    overrides: {
      preview: {
        files: {
          stylesheets: {
            joinTo: {
              'style.css': /^app\/sass\/preview/
            }
          }
        },

        paths: {
          public: 'build/preview'
        }
      }
    },

    npm: {
      enabled: false
    }
  }
}

Additional information
This happens after bumping up from 2.4.2.
brunch w -e preview -s -j 2 output:
22 Mar 18:45:27 - warn: config.files.templates.defaultPaths was removed
22 Mar 18:45:27 - info: application started on http://localhost:3333/
22 Mar 18:45:30 - info: compiling
22 Mar 18:45:34 - info: compiling.
22 Mar 18:45:38 - info: compiling..
22 Mar 18:45:42 - info: compiled 143 files into 4 files, copied 450 in 15.3 sec
Error: ENOENT: no such file or directory, stat '/PATH/public/index.html'
  at Error (native)

localhost:3333 should be pointing to '/PATH/build/preview/index.html' instead of '/PATH/public/index.html'.

https://github.com/brunch/brunch/pull/1255 should fix this. Thanks for your report!


Enable WriteCore feature for healthcareapis

We need to enable WriteCore feature to healthcareapis. Therefore add use-write-core: true and do a regen.

API change check
APIView has identified API level changes in this PR and created following API reviews.
Azure.ResourceManager.HealthcareApis


chore: bump AKS Windows CSE script package to v0.0.32

What type of PR is this?
/kind chore

What this PR does / why we need it:
bump AKS Windows CSE script package to v0.0.32
Which issue(s) this PR fixes:

Fixes #
Requirements:

[x] uses conventional commit messages

[x] includes documentation
[ ] adds unit tests
[ ] tested upgrade from previous version

Special notes for your reviewer:
Release note:
none

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 79.186%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
2237

Relevant Lines:
2825

💛  - Coveralls


OS X - cannot make connections: constantly loading, "cannot read property 'connection' of undefined"

When I opened it today, all of the connections were stuck in a perpetual loading state, and the Locading contexts... bit is at the top of all notebooks. If I try to add a new connection, the connectiontype dialog doesn't open and I get a spinning wheel forever. Can't close the connection page. Updated to latest version, didn't help.
Error:
  ERR Cannot read property 'connection' of undefined: TypeError: Cannot read property 'connection' of undefined
    at E.get uiController [as uiController] (file:///Applications/Azure Data Studio.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:2789:68703)
    at E.handleShowUiComponent (file:///Applications/Azure Data Studio.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:2789:69644)
    at file:///Applications/Azure Data Studio.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:2789:72443
    at i.fire (file:///Applications/Azure Data Studio.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:129:1835)
    at F.onProviderTypeSelected (file:///Applications/Azure Data Studio.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:2789:57855)
    at F.render (file:///Applications/Azure Data Studio.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:2789:56956)
    at E.doShowDialog (file:///Applications/Azure Data Studio.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:2789:72731)
    at E.showDialogWithModel (file:///Applications/Azure Data Studio.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:2789:71146)
    at E.showDialog (file:///Applications/Azure Data Studio.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:2789:71700)
    at As.showConnectionDialog (file:///Applications/Azure Data Studio.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:2554:16711)
    at file:///Applications/Azure Data Studio.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:1633:50296
    at i.fire (file:///Applications/Azure Data Studio.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:129:1835)
    at HTMLAnchorElement.<anonymous> (file:///Applications/Azure Data Studio.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:401:4374)
    at ZoneDelegate.invokeTask (/Applications/Azure Data Studio.app/Contents/Resources/app/node_modules.asar/zone.js/dist/zone.js:421:31)
    at Zone.runTask (/Applications/Azure Data Studio.app/Contents/Resources/app/node_modules.asar/zone.js/dist/zone.js:188:47)
    at ZoneTask.invokeTask [as invoke] (/Applications/Azure Data Studio.app/Contents/Resources/app/node_modules.asar/zone.js/dist/zone.js:496:34)
    at invokeTask (/Applications/Azure Data Studio.app/Contents/Resources/app/node_modules.asar/zone.js/dist/zone.js:1540:14)
    at HTMLAnchorElement.globalZoneAwareCallback (/Applications/Azure Data Studio.app/Contents/Resources/app/node_modules.asar/zone.js/dist/zone.js:1566:17)

Closing as a duplicate of https://github.com/microsoft/azuredatastudio/issues/17535.  Please leave any additional comments on that issue.  Thanks!


Implement SQL requests construction

It does not cover all request, but most standard covers.

Two objects Device and User have two update requests, implemented @QueryAdditional to split fields on standard and additional
Decided not to add new requests cache, just add them to Context.properties if absent.
Renamed "server" table to "servers" for consistence

More comments in code

Done, also fix travis-ci build, do not know why my Eclipse decided that putIfAbsent present in Java 7

Merged, thanks.
I forgot about one important thing here. In some cases we don't want some queries. For example, OpenGTS integration only requires two queries:
https://www.traccar.org/opengts/
My suggestion is to add a config parameter database.generateQueries which will be false by default (for backward compatibility), and we set it to true in the official release. What do you think?

Yes, good solution.
Also want to note, to fix it in our memory (need to change documentation) we renamed some requests:

database.select[Object]sAll -> database.select[Object]s
database.updateUserPassword -> database.updateUserExtended
database.updateDeviceStatus -> database.updateDeviceExtended

Yes, the first is the most important one because everyone is using at least selectDevicesAll.


Update #error docs

The Syntax sections in this directory are quite inconsistent.

Thanks for clarification @corob-msft. I agree that formatted blocks help distinguish different elements.


init the mixin opt with privatekey Buffer error

Try to config mixin node like this
{
    client_id: mixin.MIXIN_CLIENT_ID,
    aeskey: mixin.MIXIN_AESKEY,
    pin: mixin.MIXIN_PIN,
    session_id: mixin.MIXIN_SESSION_ID,
    secret: mixin.MIXIN_CLIENT_SECTET,
    privatekey: Buffer.from('-----BEGIN RSA PRIVATE KEY-----  ....   -----END RSA PRIVATE KEY-----'),
  }

got error
index.js:112
Error: error:0906D06C:PEM routines:PEM_read_bio:no start line
crypto.js:331
    at Sign.sign (crypto.js:331:26)
    at Object.sign (/Users/caoshuai/develop/workspace/temp/xexServer/node_modules/jwa/index.js:55:45)
    at Object.jwsSign [as sign] (/Users/caoshuai/develop/workspace/temp/xexServer/node_modules/jws/lib/sign-stream.js:23:24)
    at Object.module.exports [as sign] (/Users/caoshuai/develop/workspace/temp/xexServer/node_modules/jsonwebtoken/sign.js:189:16)
    at MIXINNODE.self.jwtToken (/Users/caoshuai/develop/workspace/temp/xexServer/node_modules/mixin-node/index.js:127:23)
    at MIXINNODE.self.tokenGET (/Users/caoshuai/develop/workspace/temp/xexServer/node_modules/mixin-node/index.js:132:17)
    at MIXINNODE.authTokenGET (/Users/caoshuai/develop/workspace/temp/xexServer/node_modules/mixin-node/index.js:223:15)
    at initWs (/Users/caoshuai/develop/workspace/temp/xexServer/app.js:113:17)
    at Object.<anonymous> (/Users/caoshuai/develop/workspace/temp/xexServer/app.js:122:1)
    at Module._compile (module.js:635:30)

may be I use the wrong Buffer.from API to convert the  String value.
could you tell the how to config with PrivateKey(Buffer) in this way.

You have to keep the newline character, try ` instead ' or add the '\n' character explicitly.
let key =  Buffer.from(-----BEGIN RSA PRIVATE KEY----- ..... -----END RSA PRIVATE KEY-----);

Thats work , Thx


Unexpected Line Breaks

Version 3.3
Sometimes the desc starts a new line without '\n'. It gets normal lines and a single (2 bytes) character (or a word in single byte language?) per line alternating show.
Most serious situation like that:

Fixed the problem. please try new release
https://github.com/matanki-saito/CK2dll/releases/tag/Build-20200208.1


Add support for postgres as vector database (with pgvector)

🚀 The feature
add postgres support with pgvector https://github.com/pgvector/pgvector
Motivation, pitch
This gives more flexibility to the user to use any vector database of their own choice.
pgvector has a Python library (https://github.com/pgvector/pgvector-python). It's easy to integrate with a Django website using the same database and leveraging new field type VectorField.

Thanks @hleroy for creating the issue. We will prioritize this and get it done soon.

Follow issue# 899 to track this.


Magellan 1100i barcode reader

I downloaded your Hid Library project and in the examples folder I saw that there is a sample for Magtek card reader, currently I'm working on a similar product called Magellan 1100i barcode reader. So what I did was to copy the code from magtek card reader sample and changed the GetCardData method, but I'm not able to make it work, could you please help.
Magtek card reader sample
if (data != null && data.Length == 337)
 {
     if (data[0] == 1 || data[1] == 1 || data[2] == 1)
     {
         Error = true;
         _errorMessage = "Error reading data";
         return null;
     }
     var cardData = new byte[data[3] + data[4] + data[5]];
     Array.Copy(data, 7, cardData, 0, data[3]);
     Array.Copy(data, 117, cardData, data[3], data[4]);
     Array.Copy(data, 227, cardData, data[3] + data[4], data[5]);
     return cardData;
 }

Datalogic Magellan 1100i
ProductId: 0x2602
VendorId: 0x05F9
The changed code, since the magellan 1100i has the data length of 8
if (data != null && data.Length == 8)
{
    if (data[0] == 1 || data[1] == 1 || data[2] == 1)
    {
        Error = true;
        _errorMessage = "Error reading data";
        return null;
    }
    var cardData = new byte[data[2] + data[3] + data[4]];
    Array.Copy(data, 1, cardData, 1, data[3]);
    Array.Copy(data, 2, cardData, data[2], data[3]);
    Array.Copy(data, 3, cardData, data[3] + data[4], data[5]);
    return cardData;
}

Thanks
Jay

First, if the data length is 8 bytes, why are you allocating the cardData array based on the data array? You only have 8 bytes, right? Just do:
var cardData =  new byte[8];

Similarly, these copy statements don't make any sense to me. If you actually had a variable amount of data, it would make sense to index into your destination based on the section lengths provided by the device. However, you know your data only has 8 bytes, so why not copy them all?
Array.copy(data, 0, cardData, 0, 8);

Really, it would be better to fill a custom structure that would make this data meaningful to your users, but that could be handled in another method. It looks like the goal of this method is to just get the data from the device into a buffer you own.

Thanks Austin, by reading the full length it breaks on the callback method, OnReport,
if (cardData.CardData.Length > 0)
{
Console.WriteLine(Encoding.ASCII.GetString(cardData.CardData));
SystemSounds.Asterisk.Play();
}
Since the length is always greater than 0, it keeps writing to the console.

What behavior would you prefer?

So when I scan my barcode, I'm expecting a string to be returned from cardData.CardData, the format of the string is some what like this,
"POND DRDAISTOCKBRIDGEDAJGADAK30281-5192 DAQ055869536DAU064 inDAYBLUDBA11142009DBB03291960DBC2DBD09302009DCACMDCBADCDNONEDCF029600000370033759DCGUSADCHNONEDCSPERRYDCTLAURIE MITCHELDCE2
ZGZGANZGBRev06222009ZGCNZGENZGFNZGG120ZGH5-04ZGIHENRYZGK02960000017ZGL0043475709821ZGM038
|"

We're getting closer to the real issue now. That string is a lot longer than 8 characters. When you say the 1100i has a data length of 8, what does that mean really? Are you getting 8 characters of this string at a time? Do you need to write your application so that it concatenates these strings?

So on the GetCardData(byte[] data) method when I see the length of the data variable, it shows me 8 (0-7), and when I scan my barcode, I see the long string, basically it's the driver license, and I need to parse the string to get the info.

Ok. My question is what are those bytes? You know that you get 8 of them at a time. Are they 8 characters from this string? Do some of the bytes represent other information?
Your application probably needs to build a string from multiple data reads since your data is coming in 8 bytes at a time and your high-level read event needs a longer string. So, in your handler for the HidDevice read report event, you need to append some data to your barcode result string, check to see if you have a full barcode, and if so, fire an event of your own to signal to the application that it's time to write out to the console (or parse the string to fill a custom data object, etc).

I don't know what those bytes are, because they all have a value of 0, and when I scan, the string which I get is not stored anywhere, instead it writes wherever my cursor is positioned. Hope I'm clear, sorry.

Oh. That means Windows grabbed the barcode scanner as a keyboard. Once it does that, HidLibrary can't access it. Keyboards are a whole different API separate from HID. I'm not sure if there's a way to avoid that. It would probably involve installing a custom kernel-mode driver. HidLibrary only works in user-mode.

Thanks Austin, appreciate your help. I will let you know if I resolve this issue.


ARROW-3726: [Rust] Add CSV reader with example

This adds a CSV reader and an example. The CSV reader supports all primitive types + string (List<u8>).
The code in the example for reading strings is ugly and required calling unsafe. I don't know if I'm missing something or whether we need to improve the API here.

@paddyhoran @sunchao I would appreciate a review of this. The code in the example for reading strings is ugly and required calling unsafe. I don't know if I'm missing something or whether we need to improve the API here?

@paddyhoran I added documentation as requested. I also renamed CsvFile to CsvReader. I will be adding CsvWriter in separate PR once this one is merged (I already have working code).

Thanks @andygrove . I'll take a look at this today too.

@andygrove the windows CI needs to be updated to run the new example also.


Binary Search on a 2D matrix

Is your feature request related to a problem? Please describe.
Searching an element in a 2d matrix in logarithmic time.
Describe the solution you'd like
Apply binary search for 2d matrix
Additional context
@Kumar-laxmi Please assign me this issue in C++.

I can provide detailed solution with images and language in c++ with standard template library

Please assign me the task I can provide detailed solution in c++ with proper use of dynamic memory and reference variables which will be  compatible for all systems.

Assign this to me in C.

Can I be assigned this in Java?

Is your feature request related to a problem? Please describe.
Normally, many of the people apply linear search on a 2D matrix or an array which increases the time complexity of that code.
O(N*M) is the time complexity of linear search.
But in binary search, each search is divide into half which saves a lot of time with large amount of data.
O(N+M) is the time complexity of binary search.
Point to be noted that binary search only works when the matrix is row wise or column wise sorted. Otherwise it will have time complexity of O((M * N) log(M * N)) + O(N + M) which is far worse than linear search.
Describe the solution you'd like
Here we apply binary search on a 2D array in which every row is increasingly sorted from left to right, and the last number in each row is not greater than the first number of the next row.
But, as we can see each row in the matrix is sorted and the first element of a row is greater than or equal to the last number of the preceding row; therefore, the matrix can be viewed as a sorted one dimensional array. If all rows in the input matrix are concatenated in top down order, it forms a sorted one dimensional array. And, in that case binary search algorithm is suitable for this 2D array.
Additional context
@Kumar-laxmi Please assign me this issue in C++. and under SWOC S3

Assigned!
@Arbtrage : C++
@MaYaNkKashyap681 : C
@VishalS-HK : Java
@Gaurav-8604 : Python

I want  to work on this issue under SWOC S3. please assign it to  me.


transform function changes the embedding_

Hi,
I have noticed a weird thing, I am not sure if its the expected behavior of UMAP so I am posting this.
I have realized that when I use the .transform method using a new dataset that has the same dimensions as the initial data that I used to fit UMAP, the embedding_ values change. But the embedding_ values won't change if I transform new data with different dimensions.

import umap
import numpy as np

data = np.random.rand(100,50)
fitter = umap.UMAP().fit(data)

print(fitter.embedding_[:5,])
print()
transform_different = np.random.rand(200,50)
transform_same =np.random.rand(100,50)

fitter.transform(transform_different)
print(fitter.embedding_[:5,])
print()
fitter.transform(transform_same)
print(fitter.embedding_[:5,])

The output is
[[ 3.01767   -6.116551 ]
 [ 5.664277  -4.0695806]
 [ 4.170628  -5.1638904]
 [ 6.079433  -6.3256063]
 [ 5.794976  -4.5939784]]

[[ 3.01767   -6.116551 ]
 [ 5.664277  -4.0695806]
 [ 4.170628  -5.1638904]
 [ 6.079433  -6.3256063]
 [ 5.794976  -4.5939784]]

[[ 2.6579268e+00 -7.8308420e+00]
 [ 1.5021657e+00 -6.3121238e+00]
 [ 3.2839913e+00 -8.0168781e+00]
 [-3.5597345e-01 -6.4087682e+00]
 [ 6.9772257e-03 -8.1144466e+00]]

As you see the first two outputs contain the same embedding values. but when I transform using a dataset (100, 50), the same dimensions as the initial one, the embedding values change.
Isn't the embedding supposed to be maintained always? if the embedding changes the ability to transform new data will be deteriorated right?
Hope you understand.
Thanks in advance for your time!

Right now the problem occurs if you transform exactly the same number of points as the initial training data. Ideally the transform is meant for a small set of new samples, so you shouldn't be using it for the same amount of data as the original training set. As long as you don't have the numbers line up the embedding should remain unchanged.

I am interested in this reply.  Could you please elaborate on the reason(s) one shouldn't use transform to embed new data on the order of the size of the training data?  I am thinking of using the transform function in a scRNA-seq setting, wherein a UMAP embedding has been fit for a set of reference cells and I have other sets of cells  that I wish to 'project' onto the embedding of the reference cells--and in some cases the number of query cells may be on the same order as the initial set of cells.

More generally, I read the "How it works" and think I got the basic idea of how the embedding of the initial dataset is done--thanks... Is there a description  like that available for the transform process?

@DanSchnell I don't know if there is a description of the transform process anywhere, but it's similar to how the initial dataset is created:

For each point in the new data, find the k nearest neighbors in the original data.
Calculate the smooth knn distances between the the test point and its training set neighbors, which returns the sigma and rho values.
Compute the membership strengths using sigma and rho.
Initialize the output layout coordinates.
Optimize the layout so the output distances resemble the membership strengths.

The only real difference in implementation specifics is in step 1, where neighbors are found between the new data and the original data, i.e. points in the new data cannot be considered neighbors of each other; and step 4, where the initial layout for the new data is based on the average of the output coordinates of the neighbors in the original data (as found in step 1).

Thanks very much James!!


search in multi root: strange behavior with exclude folder

Verifying #126003
Create a multi root workspace
{
	"folders": [
		{
			"path": "$inner"
		},
		{
			"path": "foo"
		}
	],
	"settings": {}
}

with files in the folder
in the files to exclude field enter ./bar

Search field has error: Workspace folder does not exist

Not clear

why that syntax would match on a workspace folder name
why no search results are shown for the folders that are not affected by the excluding pattern
(I would understand if it were a include pattern)

./bar matches a workspace folder named bar in a multiroot workspace. If the pattern is invalid, then I just don't search rather than show search results which might be not what the user intended