Add MSSQL driver

Fixes #797.
I tested the output like so (might be worth adding some integration tests for this too, since only Postgres is covered  under lib/test/integration:
localhost> docker run --rm -it -e ACCEPT_EULA=Y -e SA_PASSWORD=ins4n3lyStrongP4ssword -p1433:1433 microsoft/mssql-server-linux

container> /opt/mssql-tools/bin/sqlcmd -l 30 -S localhost -h-1 -V1 -U sa -P ins4n3lyStrongP4ssword -Q "create table footable(data varchar(50) not null);"

input:
  stdin:
    codec: lines
output:
  sql:
    driver: mssql
    data_source_name: "sqlserver://sa:ins4n3lyStrongP4ssword@localhost:1433?database=master"
    query: "insert into footable(data) values(?);"
    args:
      - ${! json("data") }

localhost> ./target/bin/benthos -c sql.yaml
{"data":"foo"}

container> /opt/mssql-tools/bin/sqlcmd -l 30 -S localhost -h-1 -V1 -U sa -P ins4n3lyStrongP4ssword -Q "select * from footable;"
foo

(1 rows affected)

Awesome, thanks @mihaitodor, can you do a quick check that this import doesn't break wasm builds? Just running @GOOS=js GOARCH=wasm go build -ldflags="-s -w" ./cmd/benthos should throw an error if it's broken. If it does break them then you'll need to add the import to sql_extra.go instead.

@Jeffail I think I'm doing something wrong, because I can't get that to work even on the master branch. I tried both on OSX (go v1.16.5) and in the golang:latest Docker container and it keeps spewing a bunch of errors saying that the "non-pc-relative relocation address for ... is too big". I saw that some of them were caused by Pulsar and Google Storage, so I hid those behind some build tags, but that doesn't seem to be enough:

Details
root@bc330741a172:/workspace# GOOS=js GOARCH=wasm go build -ldflags="-s -w" ./cmd/benthos
# github.com/Jeffail/benthos/v3/cmd/benthos
type.github.com/Jeffail/benthos/v3/lib/service/test.ContentEqualsCondition: non-pc-relative relocation address for github.com/Jeffail/benthos/v3/lib/service/test.(*ContentEqualsCondition).Check is too big: 0x101580000
type.github.com/Jeffail/benthos/v3/lib/service/test.ContentEqualsCondition: non-pc-relative relocation address for github.com/Jeffail/benthos/v3/lib/service/test.ContentEqualsCondition.Check is too big: 0x101320000
type.github.com/Jeffail/benthos/v3/lib/service/test.ContentJSONContainsCondition: non-pc-relative relocation address for github.com/Jeffail/benthos/v3/lib/service/test.(*ContentJSONContainsCondition).Check is too big: 0x101590000
type.github.com/Jeffail/benthos/v3/lib/service/test.ContentJSONContainsCondition: non-pc-relative relocation address for github.com/Jeffail/benthos/v3/lib/service/test.ContentJSONContainsCondition.Check is too big: 0x101350000
type.github.com/Jeffail/benthos/v3/lib/service/test.ContentJSONEqualsCondition: non-pc-relative relocation address for github.com/Jeffail/benthos/v3/lib/service/test.(*ContentJSONEqualsCondition).Check is too big: 0x1015a0000
type.github.com/Jeffail/benthos/v3/lib/service/test.ContentJSONEqualsCondition: non-pc-relative relocation address for github.com/Jeffail/benthos/v3/lib/service/test.ContentJSONEqualsCondition.Check is too big: 0x101340000
type.github.com/Jeffail/benthos/v3/lib/service/test.ContentMatchesCondition: non-pc-relative relocation address for github.com/Jeffail/benthos/v3/lib/service/test.(*ContentMatchesCondition).Check is too big: 0x1015b0000
type.github.com/Jeffail/benthos/v3/lib/service/test.ContentMatchesCondition: non-pc-relative relocation address for github.com/Jeffail/benthos/v3/lib/service/test.ContentMatchesCondition.Check is too big: 0x101330000
type.*github.com/Jeffail/benthos/v3/internal/template.FieldConfig: non-pc-relative relocation address for github.com/Jeffail/benthos/v3/internal/template.(*FieldConfig).FieldSpec is too big: 0x101930000
type.*github.com/Jeffail/benthos/v3/internal/template.FieldConfig: non-pc-relative relocation address for github.com/Jeffail/benthos/v3/internal/template.(*FieldConfig).FieldSpec is too big: 0x101930000
type.*github.com/Jeffail/benthos/v3/internal/template.compiled: non-pc-relative relocation address for github.com/Jeffail/benthos/v3/internal/template.(*compiled).ExpandToNode is too big: 0x101850000
type.*github.com/Jeffail/benthos/v3/internal/template.compiled: non-pc-relative relocation address for github.com/Jeffail/benthos/v3/internal/template.(*compiled).ExpandToNode is too big: 0x101850000
type.*github.com/Jeffail/benthos/v3/lib/service/blobl.execCache: non-pc-relative relocation address for github.com/Jeffail/benthos/v3/lib/service/blobl.(*execCache).executeMapping is too big: 0x101a40000
type.*github.com/Jeffail/benthos/v3/lib/service/blobl.execCache: non-pc-relative relocation address for github.com/Jeffail/benthos/v3/lib/service/blobl.(*execCache).executeMapping is too big: 0x101a40000
type.*github.com/Jeffail/benthos/v3/lib/service/test.CaseFailure: non-pc-relative relocation address for github.com/Jeffail/benthos/v3/lib/service/test.(*CaseFailure).String is too big: 0x101550000
type.*github.com/Jeffail/benthos/v3/lib/service/test.CaseFailure: non-pc-relative relocation address for github.com/Jeffail/benthos/v3/lib/service/test.(*CaseFailure).String is too big: 0x101550000
type.*github.com/Jeffail/benthos/v3/lib/service/test.ContentEqualsCondition: non-pc-relative relocation address for github.com/Jeffail/benthos/v3/lib/service/test.(*ContentEqualsCondition).Check is too big: 0x101580000
type.*github.com/Jeffail/benthos/v3/lib/service/test.ContentEqualsCondition: non-pc-relative relocation address for github.com/Jeffail/benthos/v3/lib/service/test.(*ContentEqualsCondition).Check is too big: 0x101580000
type.*github.com/Jeffail/benthos/v3/lib/service/test.ContentJSONContainsCondition: non-pc-relative relocation address for github.com/Jeffail/benthos/v3/lib/service/test.(*ContentJSONContainsCondition).Check is too big: 0x101590000
type.*github.com/Jeffail/benthos/v3/lib/service/test.ContentJSONContainsCondition: non-pc-relative relocation address for github.com/Jeffail/benthos/v3/lib/service/test.(*ContentJSONContainsCondition).Check is too big: 0x101590000
type.*github.com/Jeffail/benthos/v3/lib/service/test.ContentJSONEqualsCondition: non-pc-relative relocation address for github.com/Jeffail/benthos/v3/lib/service/test.(*ContentJSONEqualsCondition).Check is too big: 0x1015a0000
/usr/local/go/pkg/tool/linux_amd64/link: too many errors

This is the fist time I tried to mess with wasm and I'm not really sure what the problem is. I read the docs that are linked in this StackOverflow question but they didn't enlighten me much.
Please let me know if you have any idea. Otherwise, I'll do some more digging.

Oh dang, yeah just tried it myself and something's clearly broken. Don't worry about it for now, it's only used for benthos labs so I can sort this out separately.

OK, cool. I'll do a bit more reading too. Maybe it's not a huge issue to fix.

@Jeffail I did a bisect and at least part of the issue is caused by 35072219. There are a bunch of updated libs in there, so I'll try to narrow it down.

@mihaitodor  does this work with windows authentication?

@brandongilzean I haven't tested that, but please test it if you have the necessary setup and let us know if you hit any issues. https://github.com/denisenkom/go-mssqldb is the library which handles the connection and I see they mention something about that here.

Right @mihaitodor , i think we may need to allow for slashes in the query sqlserver://domain\some_user:StrongPass123@host:1433?database=test-db. I think since it's a string, the slash throws it off and creates extra slashes?

@brandongilzean I'm not sure I get what you mean. Are you saying that the query parameter is rejecting valid SQL queries which contain slashes? If possible, please provide an example config which has issues.

Yeah sure, my apologies. I may be wrong but here is an example:

sql:
driver: mssql
result_codec: json_array
data_source_name: sqlserver://DOMAIN\test_user:Strongpass123@server:1433?database=test-db

{"@timestamp":"2021-06-29T11:01:50-04:00","@service":"benthos","component":"benthos","level":"ERROR","message":"Service closing due to: failed to create processor 'branch': failed to init processor 0: failed to prepare query: parse "sqlserver://DOMAIN\\test_user:Strongpass123@server:1433?database=test-db": net/url: invalid userinfo"}

@brandongilzean No worries! I wasn't sure what issue you were bumping in. Unfortunately, I don't have an environment to test this in so I want to make sure this is not an issue on the Bethos side.
I think the backslash somehow gets escaped to \\ and then it gets into some code go-mssqldb which barfs because \ isn't a valid character in an URL according to this which mentions RFC 3986.
I did some tests here which make me think that URL escaping that backslash character might work.
Could you please do me a favour and test replacing / with %09? So something like data_source_name: sqlserver://DOMAIN%09test_user:Strongpass123@server:1433?database=test-db.

Yeah that definitely works :) Thanks so kindly! Anything else you need me to do? I can also update the docs to just explain that as well or you can do it.

Awesome, thank you very much for testing this! I'll raise a PR to document it.

My pleasure!

@brandongilzean I sent a fix for this issue in #811, so you should be able to use sqlserver://DOMAIN\test_user:Strongpass123@server:1433?database=test-db in the next release. Please open a new issue if you encounter any problems.

Thanks @mihaitodor! :heart:


Optimize GitHub actions

Only run checks for MR updates and push to main and lws2 branches.

CI is green 🟢


Feature/71 subheading

Addresses #71
Reading the information available, the hgroup tag is a valid use here to connect the paragraph with the heading.
Unfortunately, support of role="doc-subtitle" is poor still.  I tested this on iOS yesterday and it still has the issue of being announced as a heading with no level - which is misleading and confusing.
For now I've suggested using the hgroup tag alone as this at least has no detrimental effects and so will be a bit of progressive enhancement for any screenreaders that may begin to support it.
The key takeaway for this pattern is still that heading tags should not be used for taglines, so as long as that's the case it's worthwhile listing.

p.s.  one of the aricles in the original ticket suggested using aria-roledescription without the role.
This actually worked quite nicely in NVDA, and iOS just ignored it without ill effects - so that was a potential add, but it's not technically valid on it's own.  It was being flagged as an AX fail against the tests, so I removed that option.


Switch over to using new service account for LDAP

After we have a new service account for tiger data ldap, switch over to using it instead of using our temporary stopgap.
Acceptance Criteria:

[ ] Update princeton-ansible vaults to include the new service account
[ ] Run the playbook on QA & staging mflux servers

Hey team! Please add your planning poker estimate with Zenhub @bess @hectorcorrea @JaymeeH @leefaisonr


Move mouse toward diagonal directions are too fast.

Using joypad to move mouse toward diagonal directions are faster than move it Horizontally or vertically. (double the speed I guess.)

I seem same speed Horizontal and Vertical, using joypad to move mouse.
not double speed.

The problem still not being solved. In order to make the problem clearer, you can set options like this:
J2M cursor speed up Ratio: up stop
Joy to Mouse/Keyboard mapping: mouse
Then, use D-pad to control the mouse cursor while holding R1(press R1 before D-pad). You will find that to the horizontal and vertical directions would not move, and the diagonal direction can move at the speed of x1.

You can also use the same setting and compare the cursor movement of diagonal with and without R1(still press R1 before D-pad), the latter will be slower. So in my opinion, there are two problems.

If the diagonal speed is already x1 in default(without R1), it shouldn’t be slowed down by R1. So As I have said before, diagonal speed is doubled.
The behavior of the R1 button is a bit strange. It should not slow down the pointer(but speed up) or affected by which key pressed first.


Specifying Client Address

Hello, first of all thanks for making and maintaining the library. I would like to connect to a specific BLE beacon. Currently I am using this command and it's working perfectly:
pClient = NimBLEDevice::createClient(NimBLEAddress("49:42:06:00:20:4e"));
However, if I substitute it with a variable I get the error: no matching function for call to 'NimBLEAddress::NimBLEAddress(String)'.
This is the code:
pClient = NimBLEDevice::createClient(NimBLEAddress(String(value)));
Can the command accept such values or must it be specified like in the first case?
Any help would be appreciated!
Kind regards,
Mia

I've managed to resolve the error by using an integer: pClient = NimBLEDevice::createClient(NimBLEAddress(value.toInt()));, however, it also truncates the address from 49:42:06:00:20:4e to 49. Do you know what could be the solution?

Hi @miag676, the NimBLEAddress constructor takes a std::string parameter. If you change the code in the first post to std::string(value) instead of String(value) it should work.

Thanks for your response! I tried using it like this: pClient = NimBLEDevice::createClient(NimBLEAddress(std::string(value)));, but get the following error: no matching function for call to 'std::__cxx11::basic_string::basic_string(String&)'. Did I write anything wrong in this line of code?

Try this:
NimBLEDevice::createClient(NimBLEAddress(std::string(value.c_str())));
A std::string needs a char* for initialization

Thank you very much. This solved the issue!


fix: default to PickFirstLoadBalancerProvider

The services/META-INF/io.grpc.LoadBalancerProvider file included by default in the jar with dependencies did not correspond with the default pick_first policy of AutoConfiguredLoadBalancerFactory.
Fixes #2061

@skuruppu Yes, I tried with the exact steps up until and including the command mentioned above (and in the Getting Started documentation).


Thavnair - Always going to gaja area

Its the same problem as previous, I thought it was just an issue with coordinates but...
If you have hunts in thavnair, it will move to the location of the hunt, dismount, re-mount and move to patrol around the gaja.
Also, is there anyway to add the weekly/elite Hunts? im not sure if we can solo them, but would be awesome if we could get it to afk at the bosses after we've done the daily hunts for example. <3

Found the issue...
Clear the maps grind spots/markers


[Image_picker] XCode 14 Beta 2 build issue.

Steps to Reproduce

I getting this error msg when build Flutter app:

Swift Compiler Error (Xcode): Stored properties cannot be marked potentially unavailable with '@available'
/Users/name/Desktop/app-repo/applications/my-app/ios/Pods/DKImagePickerController/Sources/DKImagePickerController/DKImagePickerController.swift:67:5

Could not build the application for the simulator.
Error launching application on iPhone 13 Pro Max.

Specs:
image_picker: ^0.8.5+3
XCode Version 14.0 beta 2 (14A5229c)
Flutter 3.0.2 • channel stable • https://github.com/flutter/flutter.git
Framework • revision cd41fdd495 (4 weeks ago) • 2022-06-08 09:52:13 -0700
Engine • revision f15f824b57
Tools • Dart 2.17.3 • DevTools 2.12.2

This error comes from a class called DKImagePickerController, which is not part of the image_picker plugin.
You'll need to report this to the developers of whatever other image picker plugin you are using.


[WIP] Script to remove an article reply on production

WIP

Coverage increased (+0.02%) to 85.963% when pulling 3169147d6f349f126f8afd831f4feb06728576b8 on delete-script into aa7e7f89f84d9321f18d73ee3f505166a73ffdbb on master.


Added HTML code for an unsafe web page with vulnerabilities

Type
enhancement

Description

Added HTML code for an unsafe web page
Included vulnerabilities in the code:

Cross-Site Scripting (XSS)
Sensitive Data Transmission Over Unencrypted Channels
Clickjacking attacks
Including outdated JavaScript libraries

Changes walkthrough
Relevant files                                                                                Enhancement

test.html

        test.html
**- Added HTML code for an unsafe web page

Included vulnerabilities in the code:

Cross-Site Scripting (XSS)
 - Sensitive Data Transmission Over Unencrypted Channels
Clickjacking attacks
Including outdated JavaScript libraries**

 +31/-0

/improve

/describe


Bump march hare to 2.12.0

Upgrade from march hare 2.11.0 which had broken file permissions on a jar ( https://github.com/elastic/logstash/issues/3781 )
Companion fix to https://github.com/logstash-plugins/logstash-input-rabbitmq/pull/37

LGTM


#5226 correct call details in readme

Signed-off-by: David Radley<EMAIL_ADDRESS>Updated because:

the time is wrong for the community call for UK.
the developers call incorrectly references the zoom call
for the community calls, there is no pointer to the definitive call schedule which is the Egeria LF&AI calendar

Note: The code scan is jumping in with many issues probably caused by a large merge last night. I can't see why they are attributed to this PR, and I note the check is optional. I suggest we ignore them for now for this particular proposal


prefect: init at 2.14.2

Description of changes
Adding Prefect library and new dependencies. Unit tests need work before merging.
@bharatsikka
Things done

Built on platform(s)

[X] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.11 Release Notes (or backporting 23.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

@fabaff - thank you  so much for reviewing this PR. I realize it was a bit rough, and I'm trying to clean up. The dependencies coolname and asgi-lifespan now build with checks and importChecks, but most of the tests for prefect fail. The logs show two common errors:
...
FileNotFoundError: [Errno 2] No such file or directory: '/homeless-shelter/.prefect'
...

and
...
AttributeError: partially initialized module 'prefect' has no attribute 'settings' (most likely due to a circular import)
...

Any ideas on what I'm missing? I'd really like to get most of the tests passing before I open the PR. Thanks again!

@fabaff - thanks so much for your review. I tried to clean up for your feedback. The packages asgi-lifespan and coolname are building with tests, and I am working through tests for prefect.

@fabaff - could you advise on what I have missed below? I was able to fix some of the tests, but many are still failing on these "fixture" errors.
...
_ ERROR at setup of TestLinkStateToResult.test_link_state_to_result_with_untrackables[-5] _
[gw5] darwin -- Python 3.10.12 /nix/store/zmk12p33784bpcan0nfnji3agxy2z8wg-python3-3.10.12/bin/python3.10
file /private/tmp/nix-build-python3.10-prefect-2.13.5.drv-0/prefect-2.13.5/tests/test_engine.py, line 1999
      @pytest.mark.parametrize(
          "test_input", [True, False, -5, 0, 1, 256, ..., None, NotImplemented]
      )
      async def test_link_state_to_result_with_untrackables(
          self, test_input, get_flow_run_context, state
      ):
          with await get_flow_run_context() as ctx:
              link_state_to_result(state=state, result=test_input)
              assert ctx.task_run_results == {}
file /private/tmp/nix-build-python3.10-prefect-2.13.5.drv-0/prefect-2.13.5/tests/test_engine.py, line 110
  @pytest.fixture
  async def get_flow_run_context(prefect_client, result_factory, local_filesystem):
      partial_ctx = PartialModel(FlowRunContext)

      @flow
      def foo():
E       fixture 'prefect_client' not found
>       available fixtures: anyio_backend, anyio_backend_name, anyio_backend_options, benchmark, benchmark_weave, cache, capfd, capfdbinary, caplog, capsys, capsysbinary, cov, doctest_namespace, event_loop, get_flow_run_context, insert_assert, insert_assert_add_to_builtins, insert_assert_maybe_fail, insert_assert_session, monkeypatch, no_cover, parameterized_flow, patch_manifest_load, pytestconfig, record_property, record_testsuite_property, record_xml_attribute, recwarn, respx_mock, result_factory, state, testrun_uid, tmp_path, tmp_path_factory, tmpdir, tmpdir_factory, unused_tcp_port, unused_tcp_port_factory, unused_udp_port, unused_udp_port_factory, worker_id
>       use 'pytest --fixtures [testpath]' for help on them.
...

Similarly, I see other tests failing on  fixture 'session' not found

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/prs-ready-for-review/3032/2855

Result of nixpkgs-review pr 260496 run on x86_64-linux 1

4 packages failed to build:

python310Packages.prefect
python310Packages.prefect.dist
python311Packages.prefect
python311Packages.prefect.dist

8 packages built:

python310Packages.asgi-lifespan
python310Packages.asgi-lifespan.dist
python310Packages.coolname
python310Packages.coolname.dist
python311Packages.asgi-lifespan
python311Packages.asgi-lifespan.dist
python311Packages.coolname
python311Packages.coolname.dist

Result of nixpkgs-review pr 260496 run on x86_64-linux 1

4 packages failed to build:

python310Packages.prefect
python310Packages.prefect.dist
python311Packages.prefect
python311Packages.prefect.dist

8 packages built:

python310Packages.asgi-lifespan
python310Packages.asgi-lifespan.dist
python310Packages.coolname
python310Packages.coolname.dist
python311Packages.asgi-lifespan
python311Packages.asgi-lifespan.dist
python311Packages.coolname
python311Packages.coolname.dist

I believe nixpkgs-review builds against the latest master branch, and I was building slightly dated one. Investigating...

Added a small patch to build with anyio 4.0.0

Result of nixpkgs-review pr 260496 run on x86_64-linux 1

12 packages built:

python310Packages.asgi-lifespan
python310Packages.asgi-lifespan.dist
python310Packages.coolname
python310Packages.coolname.dist
python310Packages.prefect
python310Packages.prefect.dist
python311Packages.asgi-lifespan
python311Packages.asgi-lifespan.dist
python311Packages.coolname
python311Packages.coolname.dist
python311Packages.prefect
python311Packages.prefect.dist

Are you still working on this?

Hi @tpwrules - the package builds and core functionality works, but I stalled on the unit tests. Perhaps the current status is good enough for a first PR. What do you think?

I'm having another look at the unit tests and will update later today.

@tpwrules, @fabaff - I believe this is ready for another review. Thanks in advance.

Please rebase onto the latest nixpkgs master and clean up any merge conflicts. Please also prefix your commit messages with python3Packages..

does this work?

It looks like the coolname commit got dropped and the commits are in the wrong order? Each commit should evaluate independently so the prefect commit should be last.

Tried to tidy up

There's still something funky going on with pkgs/top-level/python-packages.nix across the commits. Each commit should only add its lines to that file.
I tried to get this going but it turns out that Prefect likes to shell back out to a new Python interpreter to do stuff like run the server. This means that e.g. /nix/store/<x>-prefect-2.4.13/bin/prefect server start dies with an error. It has to be run in a shell environment so all the propagatedBuildInputs are available. This is an unfortunate problem and I'm not sure I would support merging with this flaw.
The derivation probably needs to be split up so that the module and the CLI are separate things. This and/or this will possibly help. You could remove bin/prefect from the module build (and possibly unneeded dependencies, though maintaining that list could be annoying) and then add a flag to keep it like here.
The good news is that once I figured out it needed to be run in a shell, I was able to start the server, have it connect to SQLite (I did get some "database locked" errors which seemed harmless), start a worker, then run a simple flow and have everything work. So we're pretty close!

Thanks for the feedback and references. I agree about delaying the merge until the CLI and module are resolved. My initial goal was limited to the module only, and I hadn't given any thought to the CLI in the bin path. If you have a branch going with any of the advanced work, feel free to update this one or supersede my work with something new. Just let me know. I'll try to get back to the cleanup later in the week.

I don't have any other branch or anything. I had a theory that maybe we could just wrap the binary and avoid having two derivations. I'll play around a bit with that and post a patch here if I find anything.

I'll have another go at this tonight. Does the binary have calls to system stuff? We may need to patch it.

The problem is stanzas like this. The get_sys_executable() returns the Python used, which I believe the build process sets as a simple python3, which does not have any of the dependencies, so the uvicorn module can't be found and the server start fails. If you run it in a shell, then the dependencies are in the environment and it works.
Maybe we can just wrap bin/prefect with a Python that has the dependencies if that all makes sense?

I discussed some with the nice folks on Matrix in the NixOS Python room.
The root problem is that bin/prefect just calls site.addsitedir with a fixed list of paths generated from the dependencies through whatever mechanism. So none of those get propagated to anything that it calls.
A quick fix for the server: substituteInPlace src/prefect/cli/server.py --replace 'server_env = os.environ.copy()' 'server_env = os.environ.copy(); import sys; server_env["PYTHONPATH"] = ":".join(sys.path)'. This should probably be extended to an actual .patch file, and applied to the other submodules of the CLI.
I'm also not sure what implications this has for the environment of a worker, maybe we should append to the PYTHONPATH in that case rather than setting it outright.

Finally managed to clean up the commit history, and I added your patch for the server. The server started fine, and I was able to run and observe some basic prefect examples in the UI. Thanks for your suggestion to patch the server - great to see it working!

The commit history looks good.
Did you plan to expand on the patch? If not I probably will.

Looks like server_env only appears twice. I added something for src/prefect/cli/dev.py. Any additions are welcome if you want to switch to a patch or add something more for PYTHONPATH.


yaml: unmarshal errors

https://github.com/aquasecurity/aqua-helm/blob/2022.4/enforcer/templates/enforcer-daemonset.yaml#L47
https://github.com/aquasecurity/aqua-helm/blob/2022.4/enforcer/templates/enforcer-daemonset.yaml#L136
running the helm chart through Kustomize it gets yaml unmarshall errors due to the duplicate key in the above lines. Set statically in L47 and via the values.yaml file on L136

I had sent this to @JakeMeloche .   Our kustomization is:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

helmCharts:
  - name: enforcer
    version: 6.5.14
    repo: https://helm.aquasec.com
    namespace: aqua
    releaseName: aqua-enforcer
    valuesFile: values.yaml

Where the only thing we set in the values file is platform: "eks"
The issue is that the hostPID key is duplicated in the enforcer-daemonset.yaml and, while Helm appears to handle this, Kustomize does not.

Just tested and changing the version to 6.2.8 allows Kustomize to render properly

fixed in https://github.com/aquasecurity/aqua-helm/pull/584, released enforcer 2022.4.1 with the fix.


Issue-ci: overhauls ci and release worklows

Automated Review Comments
Issues in .github/workflows/ci.yaml:

(6, 'Hardcoded string found')
(8, 'Hardcoded string found')
(10, 'Hardcoded string found')
(26, 'Line exceeds 80 characters')
(32, 'Hardcoded string found')
(43, 'Hardcoded string found')
Issues in .github/workflows/release.yaml:
(65, 'Hardcoded string found')

I got it.


perf(scrolling): optimize indent line refresh during scrolling

Hello!
Big fan of the plugin, but there's something that has always bothered me, and that is the performance in big files.
I looked around in the code and noticed that, as far as I'm aware, the indent lines get refreshed every time the screen scrolls, which in my slow computer leads to really choppy scrolling, whether while holding down j and k or with plugins like Neoscroll.
Here's a proposed optimization: when you scroll through a file the indents won't have to be updated, because the content itself hasn't changed, so why not draw the indents on zones where there's no indent yet, and only re-draw the indents when you encounter another empty zone.
I called the empty zones "ranges", and what this PR does is keep track of the parts of the file that already have indents, and expand those ranges any time we draw more intent lines on the screen.
I wanted to know if you're interested in implementing this, or if you have better ideas as to how to optimize this. I have tested some common scenarios, but of course there are probably going to be some edge cases which I haven't seen.
Here's a profile that I ran with the built in :profile file * and :profile func *, on both of these I scrolled through the file src/nvim/eval/funcs.c in the neovim repo, holding down the j key from line 0 to line 2000:
Current implementation:

After the PR:

It's roughly twice as fast, and it feels considerably smoother.

Thanks
I'm currently on vacation, I'll take a look when I'm back

Sure, enjoy your time off!


doc: net: Add networking changes to 2.6 release note

Initial set of networking related changes in 2.6 release.
Signed-off-by: Jukka Rissanen<EMAIL_ADDRESS>

Updated according to comments.


[VS Code] Starting collaboration session: Cannot read property 'positionTracker' of undefined

Error:
Starting collaboration session: Cannot read property 'positionTracker' of undefined
Steps to Reproduce:
1.
2.

Version Data

extensionName
VSLS

extensionVersion
1.0.379

protocolVersion
2.2

applicationName
VSCode

applicationVersion
1.35.1

platformName
Windows

platformVersion
10.0.17763

Hi @martianmartian  , thank you for reporting the issue. Would you give us some more information to investigate ?  What were the exact steps to repro the issue ? Also , your logs may provide us with helpful information. You can access them via the "Live Share: Export Logs" command from the command palette (ctrl + shift + p) and attach them to the issue.

Faced this issue on Mac recently. Got past it by deleting .vscode/extensions/ms-liveshare-* files.
Reinstalling Live Share fixed the issue.

Closing as duplicate of #2254


1.this way may be more preciseness.  2.replace clock_gettime with gettimeofday(because the Mac OSX does not support the syscall.)

not applicable


Suggestion: A Disclaimer that the lib uses anime.js internally

Before submitting...

[X] I have searched for duplicate or closed issues.
[X] I have read the CONTRIBUTING document and my issue is following the guidelines.
[X] I have read the template completely before filling it in.

Context
I just opened the js file using visual studio and noticed that it includes anime.js. I love and use anime.js and I guess I would have included it in my app if I didnt check this out.
This is not an issue though, if you dont mind me asking, the included anime.js is the full or an adopted chopped and changed version to suit your needs?
Current Behavior
See Context
Expected behavior
See Context
Possible Solutions or Causes
See Context
Your Environment

Version used: Checking out the alpha
Browser Name and version: Brave
Operating System and version (desktop or mobile): Windows 10
Additional information you want to tell us:

Hello @Mashiane!
Thanks for pointing it out. I think in the nearer future we should remove the dependency completely,
because it is only using 4 functions or so, which could be done with CSS Animations i guess.
It would be awesome if someone has time to fix this stuff and we do not have to rely on this library.
AnimeJs is great and I use it for other projects too, but in the sense of a fast Web-UI I think it is overpowered.
Also: As far as I know, CSS Animations use the GPU for rendering and Animejs would use Computation-Power  via CPU.
So I would prefer the more performant GPU-Path.

In the latest release 2.0.3 animejs is not used anymore.


build error

Following guide in documentation get the error on build time.
$ yarn next build
yarn run v1.22.10
warning package.json: No license field
$ /user/project/node_modules/.bin/next build
info  - Using webpack 4. Reason: custom webpack configuration in next.config.js https://nextjs.org/docs/messages/webpack5
info  - Checking validity of types
info  - Creating an optimized production build ..node:internal/process/promises:245
          triggerUncaughtException(err, true /* fromPromise */);
          ^

[Error: ENOENT: no such file or directory, open '/user/project/public/index-default.toml'] {
  errno: -2,
  code: 'ENOENT',
  syscall: 'open',
  path: '/user/project/public/index-default.toml'
}
error Command failed with exit code 1.

after this I added new file "index-default.toml" in public folder.
and then i got this:
$ yarn next build
yarn run v1.22.10
warning package.json: No license field
$ /user/project/node_modules/.bin/next build
info  - Using webpack 4. Reason: custom webpack configuration in next.config.js https://nextjs.org/docs/messages/webpack5
info  - Checking validity of types
info  - Creating an optimized production build ...node:internal/errors:415
    ErrorCaptureStackTrace(err);
    ^

Error: spawn stork ENOENT
    at Process.ChildProcess._handle.onexit (node:internal/child_process:282:19)
    at onErrorNT (node:internal/child_process:480:16)
    at processTicksAndRejections (node:internal/process/task_queues:81:21) {
  errno: -2,
  code: 'ENOENT',
  syscall: 'spawn stork',
  path: 'stork',
  spawnargs: [
    '--build',
    '/user/project/public/index-default.toml'
  ],
  cmd: 'stork --build /user/project/public/index-default.toml',
  stdout: '',
  stderr: ''
}
error Command failed with exit code 1.

also there is an update in index-default.toml
[output]
filename = "/user/project/public/index-default.st"
excerpts_per_result = 1

There is no problem with older version from packages that used in master branch
Any sugestions on how to fix this?
Thanks!

FYI: It looks like you have stork: true, which is currently undocumented and the behavior might change.

@wotori do you still gave an error and needs help?

@wotori do you still gave an error and needs help?

Nope, I just switched to docusaurus


Debug anagram.py and fix errors

This PR solves #99
Please let me know in case any changes or modifications are required in the commit.
Thanks!

@reeba212 Can you please add the hacktoberfest-accepted labels, so that this contribution will be eligible.
Thank You!


Upgrade Electron.app to v0.29.1

Thank you for the contribution. It was merged directly as 1442e0a932ec35248ef2fda30b66a0f6cf912a48 to keep commit history cleaner. Your contribution is still credited to you.


Equality improve with System level coherent

[Equality improve with System level coherent]

hipblaslt-test pass in local test


Not getting error when select different file.

When i am going to select different file but not an images it not return any error. Please consider this issue as enhancement.
We can set onActivityResult  as Cancel or Error method so it will finish our current activity.

I don't understand what you mean, you can't select non-image file...

Checkout below reproducible step that may help .
GIF

I'm curious, how common is this scenario? How it can happen in normal flow without using super user apps like file explorer?

My personal device Moto 2Generation have  Camera,Gallery and Photo default android application if I  will remove super users apps.
Also it should a more option depend on the installed application.

Sure, but camera, photos and gallery apps allow only valid image files.
You can't select an apk to load...

Yes of course but user can select file by super user app (i.e File explorer) .
Can we set any error message if file is not an images?
If Yes:
We can fix this issue in BitmapCroppingWorkerTask.java file at onPostExecute by  check result.bitmap==null that can set message like "Invalid images or choose image file only".

Any fix on the same ?

Any fix underway? It's quite likely in a normal flow: Google Drive for instance can't restrict types of files shown, it will display everything.

Super user apps like File Explorer are built-in in most phones nowadays. Could you consider fixing this issue? Thank you very much.


Control-Space shortcut no longer launches quicksilver on 1.6.0

After updating to 1.6.0 from 1.5.9 using the in-app updater, I can no longer start the Quicksilver shortcut using Control-Space.  The app shows as running in Activity Monitor but I cannot access it it anyway.  Force Quitting it and re-launching has no effect, similarly with rebooting and re-launching.  Reverting to 1.5.9 fixes this issue.  I'm running 10.10.5, on a 2010 Mac Mini with 16GB RAM.

Nothing related to keyboard shortcuts or activation was changed in this version.
First, go to System Preferences → Users & Groups → You → Login Items and make sure “Hide” isn’t checked next to Quicksilver.
You could also try enabling the Dock icon temporarily and clicking it to see if it activates QS.

Hi Rob, the Hide setting is/was not enabled in System Prefs. I am unable to enable the Dock icon because I am unable to even get to the launcher interface to get into Quicksilver preferences. Clearly something between 1.6.0 and MacOS 10.10.5 is not compatible.

On Sep 24, 2018, at 10:54 AM, Rob McBroom<EMAIL_ADDRESS>wrote:
Nothing related to keyboard shortcuts or activation was changed in this version.
First, go to System Preferences → Users & Groups → You → Login Items and make sure “Hide” isn’t checked next to Quicksilver.
You could also try enabling the Dock icon temporarily and clicking it to see if it activates QS.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.

I have a similar issue, though I might suggest this topic would be better written as "Quicksilver fails to fully launch".
I'm on macOS 10.11.6 on a Mac Pro (early 2008) with 14GB RAM. It appears both myself and @amancalledsun are on older versions of macOS, maybe it's an issue with legacy systems?
After updating from build 4024 to 4025 (using the in-app updater), I noticed a failure to launch the interface using ctrl-space.  Looking up the pid in Terminal and killing it is my only way to quit. So it appears to be (semi-)running, but I get no system menu icon (nor dock icon, though I have that disabled), nor response to the keyboard shortcut. Further evidence of the "running" state is an attempt to delete /Applications/Quicksilver.app via Finder fails with "can’t be moved to the Trash because it’s open".
The following log lines appear in Console when the 4025 build is re-launched after update, or launched from a stopped state (sans the "re-launch" line in the latter case):
9/24/18 10:30:14.913 PM Quicksilver[2422]: Relaunch from path /Applications/Quicksilver.app/Contents/MacOS/Quicksilver
9/24/18 10:30:19.452 PM Quicksilver[3547]: -[QSApp setAutomaticCustomizeTouchBarMenuItemEnabled:]: unrecognized selector sent to instance 0x7fdac0502ce0
9/24/18 10:30:19.452 PM Quicksilver[3547]: -[QSApp setAutomaticCustomizeTouchBarMenuItemEnabled:]: unrecognized selector sent to instance 0x7fdac0502ce0
Reverting to build 4024 restores functionality. No log lines appear in Console upon successful launch of this version.

@srsanborn Thanks for finding those log messages. That’s definitely Touch Bar junk. I’ll try to prevent it from doing that. In the meantime, I’ve changed 1.6.0 to require 10.12 or higher.

Another pointless "me too" maybe. But just to confirm I'm getting exactly the same issue on OSX 10,11,6 and the same messages in the terminal:
$: /Applications/Quicksilver.app/Contents/MacOS/Quicksilver
2018-10-01 19:00:27.024 Quicksilver[1012:9907] -[QSApp setAutomaticCustomizeTouchBarMenuItemEnabled:]: unrecognized selector sent to instance 0x7fe42150b3c0
2018-10-01 19:00:27.024 Quicksilver[1012:9907] -[QSApp setAutomaticCustomizeTouchBarMenuItemEnabled:]: unrecognized selector sent to instance 0x7fe42150b3c0

Rolling back to 15,9 here too.

I think Touch Bar support was added in 10.12, so that makes sense. It’ll be fixed in 1.6.1.


Implement JWT's

TODO:

use environment variable for JWT key
phase out auth.rs
add integration tests


Improve type checker performance

While performance isn't the top priority for us, Acton does need to be "fast enough". I don't think it's ok to spend multiple seconds of type checking for a relatively small program like the test_tcp_server (granted it might be the largest test program we have in this repo).
kll@ThinkYoga:~/terastream/acton/test$ /home/kll/terastream/acton/dist/bin/actonc --timing --always-build --dev rts_db/test_tcp_server.act
Building file rts_db/test_tcp_server.act
Reading file rts_db/test_tcp_server.act:  0.001 s
Parsing file rts_db/test_tcp_server.act:  0.058 s
  Compiling test_tcp_server.act for development
    Pass: Make environment:  0.001 s
    Pass: Kinds check     :  0.000 s
    Pass: Type check      :  6.026 s
    Pass: Normalizer      :  0.000 s
    Pass: Deactorizer     :  0.000 s
    Pass: CPS             :  0.000 s
    Pass: Lambda Lifting  :  0.108 s
    Pass: Generating code :  0.000 s
    Pass: Writing code    :  0.000 s
    Pass: C compilation   :  1.383 s
   Finished compilation in   7.518 s
Building executable /home/kll/terastream/acton/test/rts_db/test_tcp_server
kll@ThinkYoga:~/terastream/acton/test$

With my power cable unplugged so CPU is running in more of a battery saving mode it gets even worse, taking twice the above.
is this haskell lazy eval getting us int some bind? I believe we need to think enough about performance to avoid this at least...
My experience is that it's quite simple to improve the situation by adding some explicit type signatures somewhere but it's not as easy knowing where to do that. Could we add simple metrics to easily figure that out?

This is likely caused by some accidental loss of sharing that results in exponential complexity. I encountered this problem once in the previous Acton type-checker, was quite easy to fix.
Akternatively, it could be the solver being fooled into searching some unnecessarily large solution space. Here as well the fix should be easy, once the root cause is found.
In any case I’ll look into it.
31 okt. 2022 kl. 14:47 skrev Kristian Larsson @.***>:

While performance isn't the top priority for us, Acton does need to be "fast enough". I don't think it's ok to spend multiple seconds of type checking for a relatively small program like the test_tcp_server (granted it might be the largest test program we have in this repo).
@.:~/terastream/acton/test$ /home/kll/terastream/acton/dist/bin/actonc --timing --always-build --dev rts_db/test_tcp_server.act
Building file rts_db/test_tcp_server.act
Reading file rts_db/test_tcp_server.act:  0.001 s
Parsing file rts_db/test_tcp_server.act:  0.058 s
Compiling test_tcp_server.act for development
Pass: Make environment:  0.001 s
Pass: Kinds check     :  0.000 s
Pass: Type check      :  6.026 s
Pass: Normalizer      :  0.000 s
Pass: Deactorizer     :  0.000 s
Pass: CPS             :  0.000 s
Pass: Lambda Lifting  :  0.108 s
Pass: Generating code :  0.000 s
Pass: Writing code    :  0.000 s
Pass: C compilation   :  1.383 s
Finished compilation in   7.518 s
Building executable /home/kll/terastream/acton/test/rts_db/test_tcp_server
@.:~/terastream/acton/test$
With my power cable unplugged so CPU is running in more of a battery saving mode it gets even worse, taking twice the above.
is this haskell lazy eval getting us int some bind? I believe we need to think enough about performance to avoid this at least...
My experience is that it's quite simple to improve the situation by adding some explicit type signatures somewhere but it's not as easy knowing where to do that. Could we add simple metrics to easily figure that out?
—
Reply to this email directly, view it on GitHubhttps://github.com/actonlang/acton/issues/1025, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ABCAPCBUCX5XPSJTCC7PCBTWF7EV3ANCNFSM6AAAAAARTC7ZLE.
You are receiving this because you are subscribed to this thread.Message ID: @.***>

Turns out that a major reason for the bad performance is that the solver subroutine 'simplify' doesn't split its input into non-overlapping sets like the solver proper does, which causes a lot of repeated traversals of constraint sets that are unaffected by the current improvement step. An easy thing to fix, and the solution I've pushed seems to bring down the type-checker execution time to less than half on the problematic example.
But I've also noted some other wasteful behavior of the solver which I intend to understand better in order to correct. Keeping the issue open for this purpose!

@nordlander is there anything besides the solver that could take time during the type checking pass in the compiler? I'm thinking of this line:
                      (iface,tchecked,typeEnv) <- Acton.Types.reconstruct outbase env kchecked

which is what I've called "type checking" (if compiler is run with --timings) even though the function is called something else.
We have some programs that are rather slow to type check and I assume it is type inference making bad guesses / running through lots of potential solutions. Or should we have a counter or something to indicate how many potential solutions we've run through as an indicator?
The slow program is acton-streamer, which you have access to... just build the thing. Some files are > 30 seconds for me.

A quick glimpse of what to expect from the improved solver: I just managed to compile nsoapi/schema.act in 1 min 55 sec, compared to 10 min 41 sec when using the current main branch.
I still think 1:55 is way too much, but at least this shows that my current line of attack is bearing fruit!

We are now down to a few seconds. I'm going to close this issue since the fixes are in that brings performance to a reasonable and acceptable level. There are no currently planned improvements and nothing immediately actionable, so closing.
We can open other more specific issues in the future to further improve performance if the need arise.


Lost NFT??

I started an auction. Sent the NFT then sent the initial bid. However, The initial bid didn't go through or something. So I send the funds again about 10 mins later. About 5 mins or so I get the popup stating that my funds are being returned. However, I don't know who now owns my NFT as it didn't get sent back to me and I know I sent it to the right address as I clicked the address to copy it in the auction house when I was creating the auction. Does it take a while for it to return my NFT? I want to attempt the auction again.
Here is the NFT token ID: 3f89667d6d818d6eaa48b47404de7003be81ed3a502443b74804a0b057980cdf
What can I do now?
Thanks.

This can be resolved.
RESOLUTION: After 4+ hours the token was returned to my wallet.

This can be resolved.
RESOLUTION: After 4+ hours the token was returned to my wallet.

This can be resolved.
RESOLUTION: After 4+ hours the token was returned to my wallet.


MSW - Auto Fixture Libraries for responses

Q: Do you know of any libs that work well with msw? E.g. So that responses are not hard coded. In the past I've used efate to be a fixture builder

Hey!
So the model I tend to go with is one where I create mocks using this kind of pattern: https://github.com/citypaul/fullstack-react-tdd-example/blob/main/packages/mocks/src/product.ts
The idea is that you create a function that takes an optional overrides argument, and you return a default set of values from the function, but override any of these values with whatever is passed into overrides. It works very well when combined with Partial in typescript.
Outside of that, the MSW team themselves provide this library: https://github.com/mswjs/data
This may be quite inline with what you're looking for, but I've never used it myself.


[SPARK-8607] SparkR -- jars not being added to application classpath correctly

Add getStaticClass method in SparkR's RBackendHandler
This is a fix for the problem referenced in SPARK-5185.
cc @shivaram

JIRA issue open here: https://issues.apache.org/jira/browse/SPARK-8607

Can one of the admins verify this patch?

Jenkins, add to whitelist

Jenkins, ok to test

Merged build triggered.

Merged build started.

Test build #35732 has started for   PR 7001 at commit 7c6bd0c.

Test build #35732 has finished for   PR 7001 at commit 7c6bd0c.

This patch fails Scala style tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

@shivaram Fixed style nits.

Merged build triggered.

Merged build started.

Test build #35738 has started for   PR 7001 at commit 5a80844.

@cafreeman Code change looks good -- I'm wondering if there is way to add a test for this in SparkR. We could add a dummy jar file and then try and see if we can call into that ?  @davies Do we have a dummy jar file we use for similar tests in Python / Java etc.

Test build #35738 has finished for   PR 7001 at commit 5a80844.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Jenkins, retest this please

Merged build triggered.

Merged build started.

Test build #35742 has started for   PR 7001 at commit 5a80844.

Test build #35742 has finished for   PR 7001 at commit 5a80844.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

@shivaram where would we store test data? I have a JAR we could probably use.

NAVER - http://www.naver.com/
<EMAIL_ADDRESS>님께 보내신 메일 <Re: [spark] [SPARK-8607] SparkR -- jars not being added to application classpath correctly (#7001)> 이 다음과 같은 이유로 전송 실패했습니다.

받는 사람이 회원님의 메일을 수신차단 하였습니다.

We could put it in in R/pkg/inst/test_support or something like that. It would be similar to the python files at https://github.com/apache/spark/tree/master/python/test_support

@shivaram Just committed a first draft of a test, but there are two issues:

The current build appears to be broken. I'm getting issues about sparkPackages being an unused argument and "object 'sparkSubmitBinName' not found". Is this branch currently in a state of flux?
I'm using a relative path in the test to point to the JAR (which has been placed in test_support), and I'm not sure how that will play. I can't actually run it right now though because I can't launch a spark context due to the first issue.

Merged build triggered.

Merged build started.

Test build #35798 has started for   PR 7001 at commit 9a5c362.

Sorry the first one was a bug we introduced yesterday. Let me send a hotfix for it right now

@cafreeman The worrying thing is that the tests seem to pass right now on Jenkins and on my machine if I run ./run-tests.sh -- If you get a chance it'll be good to figure out why that is happening

@cafreeman Created https://github.com/apache/spark/pull/7022 -- Could you take a look at it ?

@cafreeman Regarding the test itself, a couple of things

Could we have a smaller JAR file ? This one looks like its around 5MB which is pretty big :) Instead of using sbt assembly to generate it, could you try to use sbt package ? That should make it smaller
Regarding the paths, you can use SPARK_HOME and refer things relative to that. See https://github.com/apache/spark/blob/master/python/pyspark/tests.py#L400 for an example in Python

Test build #35798 has finished for   PR 7001 at commit 9a5c362.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Merged build triggered.

Merged build started.

@shivaram Alright, I've uploaded a smaller JAR and switched to an absolute filepath using SPARK_HOME. So far this test runs successfully in both the sparkR shell and in regular R, but fails when triggered along with the entire test suite, but I'm not sure why.  I added a sparkR.stop() so we could reboot the Spark Context and add the JAR, but it's still having trouble when it's run with run-tests.sh

Test build #35810 has started for   PR 7001 at commit 0bea809.

Test build #35810 has finished for   PR 7001 at commit 0bea809.

This patch fails SparkR unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Thanks @cafreeman for the update. I'll try this out today and see why run-tests.sh is not picking it.

So I took a look at this and I managed to figure out why the tests don't work but why it works from sparkR shell. Its due to the fact we have two different ways in which things are initialized

For the sparkR shell (and RStudio), the R process comes up first and then the JVM is launched when the spark context is created. In this case if we pass in sparkJars they are added to the spark-submit command and things work fine.

For the unit tests (and any script run with spark-submit) the JVM comes up first and forks the R process. In this case adding sparkJars to the context has no effect as the JVM is already running. In this case the user needs to run sparkR --jars <jarFile> to get things to work correctly.

For the unit tests we can't test the first case as it is a batch script. But we can test the second case by actually running the sparkR script from the unit test. This would be something similar to https://github.com/apache/spark/blob/master/python/pyspark/tests.py#L1641

@shivaram Took your advice and changed things so that the test runs another script using R's system2. The tests work now :)

Merged build triggered.

Merged build started.

Test build #35876 has started for   PR 7001 at commit 8f81194.

Test build #35876 has finished for   PR 7001 at commit 8f81194.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Hmm - weirdly the Scala spark-submit test failed. Lets give this another go and see if it was just a flaky test.
Jenkins, retest this please

Jenkins, retest this please

Merged build triggered.

Merged build started.

Test build #35884 has started for   PR 7001 at commit 8f81194.

Test build #35884 has finished for   PR 7001 at commit 8f81194.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

LGTM. That test case looks great. Thanks @cafreeman

No problem!


[bext-ut] Update to 1.1.9

Describe the pull request

What does your PR fix?
Updates bext-ut to 1.1.9

Which triplets are supported/not supported? Have you updated the CI baseline?
As before, Yes

Does your PR follow the maintainer guide?

Yes

CI fails on all platforms with
-- The CXX compiler identification is MSVC 19.31.31106.2
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files/Microsoft Visual Studio/2022/Enterprise/VC/Tools/MSVC/14.31.31103/bin/Hostx64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Downloading CPM.cmake to D:/buildtrees/bext-ut/x64-windows-rel/cmake/CPM_0.31.1.cmake
-- CPM: adding package<EMAIL_ADDRESS>(v1.6.0)
CMake Error at D:/downloads/tools/cmake-3.22.2-windows/cmake-3.22.2-windows-i386/share/cmake-3.22/Modules/ExternalProject.cmake:2666 (message):
  error: could not find git for clone of packageproject.cmake-populate
Call Stack (most recent call first):
  D:/downloads/tools/cmake-3.22.2-windows/cmake-3.22.2-windows-i386/share/cmake-3.22/Modules/ExternalProject.cmake:3716 (_ep_add_download_command)
  CMakeLists.txt:23 (ExternalProject_Add)

It appears that git is not installed system-wide on the CI runners and CMake cannot find it. I have added all platforms to ci.baseline.txt.

CI fails on all platforms with
-- The CXX compiler identification is MSVC 19.31.31106.2
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files/Microsoft Visual Studio/2022/Enterprise/VC/Tools/MSVC/14.31.31103/bin/Hostx64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Downloading CPM.cmake to D:/buildtrees/bext-ut/x64-windows-rel/cmake/CPM_0.31.1.cmake
-- CPM: adding package<EMAIL_ADDRESS>(v1.6.0)
CMake Error at D:/downloads/tools/cmake-3.22.2-windows/cmake-3.22.2-windows-i386/share/cmake-3.22/Modules/ExternalProject.cmake:2666 (message):
  error: could not find git for clone of packageproject.cmake-populate
Call Stack (most recent call first):
  D:/downloads/tools/cmake-3.22.2-windows/cmake-3.22.2-windows-i386/share/cmake-3.22/Modules/ExternalProject.cmake:3716 (_ep_add_download_command)
  CMakeLists.txt:23 (ExternalProject_Add)

It appears that git is not installed system-wide on the CI runners and CMake cannot find it. It builds successfully on my local machine. I have added all platforms to ci.baseline.txt.

We can try to patch out the use of git

-- Downloading CPM.cmake to D:/buildtrees/bext-ut/x64-windows-rel/cmake/CPM_0.31.1.cmake
-- CPM: adding package<EMAIL_ADDRESS>(v1.6.0)

It is not just git but an entire package manager. This must be avoided.

It is not just git but an entire package manager. This must be avoided.

Thanks for the hint, it turned out to be easy to disable it.

Thanks!


Add an additional data property within ctx.callbackQuery to improve bot designs

Lets say you have an inline keyboard along with its corresponding callbackQuery registering and associated callback function as seen below.
...
const inlineKeyboard = [
    [
        {
            text: "Add 1",
            callback_data: "cb_add_1",
        },
        {
            text: "Add 2",
            callback_data: "cb_add_2",
        },
        ...
    ],
];

...
await ctx.reply("Header", {
    parse_mode: "Markdown",
    reply_markup: {
        inline_keyboard: inlineKeyboard,
    },
});
...

bot.callbackQuery("cb_add_1", cb_add_1);
bot.callbackQuery("cb_add_2", cb_add_2);
...

Is it possible to add an additional data property such as "additional_data" within ctx.callbackQuery so that we can reduce the amount of bot.callbackQuery(...) calls we need to make. The ctx.callbackQuery.additional_data property would then be handled within the callback function given to a single bot.callbackQuery(...) call.
Example:
...
const inlineKeyboard = [
    [
        {
            text: "Add 1",
            callback_data: "cb_add",
            additional_data: "1",
        },
        {
            text: "Add 2",
            callback_data: "cb_add",
            additional_data: "2",
        },
        ...
    ],
];

...
await ctx.reply("Header", {
    parse_mode: "Markdown",
    reply_markup: {
        inline_keyboard: inlineKeyboard,
    },
});
...

bot.callbackQuery("cb_add", cb_add);  // The cb_add function would handle the ctx.callbackQuery.additional_data property
...

This is limited by what Telegram supports. The Bot API doesn't permit an extra field in a button, so there's no way for grammY to support it.
Instead, you can use cb_add:1 and cb_add:2 as button data and then rely on
bot.callbackQuery(/cb_add:(\d+)/, cb_add)

to handle all callback queries with a regular expression. The result of the regex will be provided in ctx.match.
You can also pass an array of possible values to bot.callbackQuery in order to reduce the number of calls to it.


Most of the Twig.filters-functions crashes if you pass an undefined value to them.

I discovered this by accident (misspelled a var-name).  :)
But if you pass a undefined value to the "escape"/"e" filter twig.js crashes, because the  function assumes the value is a string and just calls .replace(...) on the value.
From what I can see most of the filters will have this problem.
I can easily fix most of the filter functions so they handle an undefined value, but I'm confused about how best to handle those that don't return a string, like "reverse", "sort", "key", "join" and "merge".

I'm closing this since it is so old. If there are still outstanding issues with filters not validating their arguments correctly, please open a new issue.


Markdown mathmode

Math environments in latex code blocks in a markdown file are correctly detected.
However, markdown math regions denoted through $ $, $$ $$, $$$ $$$, are not detected correctly; the math mode imaps are not triggered in them.

i'd happily accept a PR to fix this, but since I'm not using neovim as my daily latex driver anymore, I'm not inclined to work on this myself.


Object/DyamicObject/DynamicArray get/set callbacks should support std::exception

Object changes made in 01e05078dcc14de74e02e9b1b6a8462a363d2cc2

Completed in bd173983c72bba0f412b288113665cc65730e0ec


Add ability to make LLM extraction requests through the go package

It looks like the extractorOptions params are commented out. Wondering if there's a reason for it

Wondering the same thing.


No pypi packages for 2.0.1

Hi,
When will azure-iot-device V2.0.1 be available on Pypi?
Check Pypi

v2.01 has been released with connected property

@LouanDuToitS3, @robbinvandamme, thank you for your contribution to our open-sourced project! Please help us improve by filling out this 2-minute customer satisfaction survey


websocket connection error

Trying this:
    ws_host = "wss://trompa-mtg.upf.edu/ce/graphql" // this what returned by the trompace.config
    async with websockets.connect(ws_host, subprotocols=['graphql-ws']) as websocket:
        print("Do something")

I get the following error:
  File "/home/aggelos/envs/tpl/lib/python3.7/site-packages/websockets/client.py", line 517, in __aenter__     return await self   File "/home/aggelos/envs/tpl/lib/python3.7/site-packages/websockets/client.py", line 547, in __await_impl__     extra_headers=protocol.extra_headers,   File "/home/aggelos/envs/tpl/lib/python3.7/site-packages/websockets/client.py", line 296, in handshake     raise InvalidStatusCode(status_code) websockets.exceptions.InvalidStatusCode: server rejected WebSocket connection: HTTP 400 python-BaseException
Probably is related to secure connection

I'll take a look into this, but first of all try with ws://, because this connection isn't secure. See if that works

No it doesn't. I get a invalid URI message.
File "/home/aggelos/envs/tpl/lib/python3.7/site-packages/websockets/client.py", line 517, in aenter
return await self
File "/home/aggelos/envs/tpl/lib/python3.7/site-packages/websockets/client.py", line 557, in await_impl
self.handle_redirect(exc.uri)
File "/home/aggelos/envs/tpl/lib/python3.7/site-packages/websockets/client.py", line 483, in handle_redirect
new_wsuri = parse_uri(uri)
File "/home/aggelos/envs/tpl/lib/python3.7/site-packages/websockets/uri.py", line 66, in parse_uri
raise InvalidURI(uri) from exc
websockets.exceptions.InvalidURI: https://trompa-mtg.upf.edu/ce/graphql isn't a valid URI
python-BaseException
Process finished with exit code 1
Is it possible to have a non-secure wensocket connection over a secure server??? In any case the config.py https://github.com/trompamusic/trompa-ce-client/blob/2fcad51a59d0337f2d824ee24a64acf42c0666fb/trompace/config.py#L58 should be updated in the set_server() function to return the correct URI.

Was invalid configuration in trompa-mtg nginx configuration


Displaying score in Snake Game

Adding 'score' variable and display it in the Snake Game

could you add a description to the PR?

There is another detail that I'd consider adding, when the game ends, I'd display the score.

At last, the score text should not affect the game, I think that the text would overlap with the snake/apples.

do you have some new screenshots for us ?

do you have some new screenshots for us ?

Score in-game:

Score when the game ends

can we align a bit more to the right the Score in the game, maybe aligned with the box game


Add MonadError instance for TemplatesT

Thanks. I simplified the instance and added changelog, will make a release in a while.

https://hackage.haskell.org/package/kontrakcja-templates-0.12


Passing response headers

We are serving files from the backend with URLs like "http://example.com/images/some.pdf,1234" and a header Content-Disposition: filename="some.pdf", so if a user wants to save the file it has a "correct" filename.
But the imageproxy is not passing through these headers and I also did not find an option to do so.
Would this be a new feature, is there an existing option to do so or is it not adviced in general to passResponseHeaders.
CC: @handcode

There are a handful of headers we copy from the upstream response...
https://github.com/willnorris/imageproxy/blob/da55ecefd3a9893ae1eced3e1846e98494990628/imageproxy.go#L239
https://github.com/willnorris/imageproxy/blob/da55ecefd3a9893ae1eced3e1846e98494990628/imageproxy.go#L260
I'd be fine accepting a patch to add Content-Disposition to that list.
Though I'm kinda curious... you're using imageproxy for PDF files?  Why?  Just for the https enforcement and caching, I suppose?

We could of course now "simply" include Content-Disposition in line 239, but what do you think about making the list of headers that are copied from the response configurable (--originHeaders option)?
This would then also allow someone to remove unwanted headers such as ETags.
But since I'm not a Go hacker, I can't easily make a patch for it.
And to answer your question about PDFs:
We use a combination of nginx/imageproxy for image processing and media-file chaching (the nginx itself also has a cache to reduce the load on the imageproxy).
The reason to route something like PDFs through the imageproxy (and not directly from the nginx to the origin) is the limitation of the backends. (signed-requests, allowHosts)


Add MediaExtension for BRAW support

It looks like on macOS 14 you'll finally be able to add your own codecs to macOS! Woohoo!
See: https://developer.apple.com/documentation/mediaextension
We'll look at adding an official BRAW MediaExtension plugin ASAP.

Unfortunately the MediaExtension API/SDK has disappeared from the Apple Universe, so I'm going to close this issue for now.


Update Basel data for 2022

Data was updated: https://data.bs.ch/explore/dataset/100096/table/?disjunctive.art&sort=termin


maoOS 构建client报错

版本号
~ 1.7.1
什么问题
~ 运行npm run build-client构建报错，貌似可以构建出文件，但是会报错停止
如何复现此问题
~
✖ 1 problem (1 error, 0 warnings)
报错信息
[error] ./client/containers/Project/Interface/InterfaceCol/InterfaceColContent.js

/Users/shifeng/Documents/work/yapi/vendors/client/containers/Project/Interface/InterfaceCol/InterfaceColContent.js
  170:9  error  componentWillMount is deprecated since React 16.999.0, use UNSAFE_componentWillMount instead, see https://reactjs.org/docs/react-component.html#unsafe_componentwillmount. Use https://github.com/reactjs/react-codemod#rename-unsafe-lifecycles to automatically update your components                       react/no-deprecated
  468:9  error  componentWillReceiveProps is deprecated since React 16.999.0, use UNSAFE_componentWillReceiveProps instead, see https://reactjs.org/docs/react-component.html#unsafe_componentwillreceiveprops. Use https://github.com/reactjs/react-codemod#rename-unsafe-lifecycles to automatically update your components  react/no-deprecated

✖ 2 problems (2 errors, 0 warnings)

[error] ./client/components/ModalPostman/index.js

/Users/shifeng/Documents/work/yapi/vendors/client/components/ModalPostman/index.js
  64:3  error  componentWillMount is deprecated since React 16.999.0, use UNSAFE_componentWillMount instead, see https://reactjs.org/docs/react-component.html#unsafe_componentwillmount. Use https://github.com/reactjs/react-codemod#rename-unsafe-lifecycles to automatically update your components  react/no-deprecated

✖ 1 problem (1 error, 0 warnings)

[error] ./client/containers/Project/Interface/InterfaceList/Run/AddColModal.js

/Users/shifeng/Documents/work/yapi/vendors/client/containers/Project/Interface/InterfaceList/Run/AddColModal.js
  44:3  error  componentWillMount is deprecated since React 16.999.0, use UNSAFE_componentWillMount instead, see https://reactjs.org/docs/react-component.html#unsafe_componentwillmount. Use https://github.com/reactjs/react-codemod#rename-unsafe-lifecycles to automatically update your components                       react/no-deprecated
  49:3  error  componentWillReceiveProps is deprecated since React 16.999.0, use UNSAFE_componentWillReceiveProps instead, see https://reactjs.org/docs/react-component.html#unsafe_componentwillreceiveprops. Use https://github.com/reactjs/react-codemod#rename-unsafe-lifecycles to automatically update your components  react/no-deprecated

✖ 2 problems (2 errors, 0 warnings)

[error] ./client/components/ModalPostman/VariablesSelect.js

/Users/shifeng/Documents/work/yapi/vendors/client/components/ModalPostman/VariablesSelect.js
  79:9  error  componentWillReceiveProps is deprecated since React 16.999.0, use UNSAFE_componentWillReceiveProps instead, see https://reactjs.org/docs/react-component.html#unsafe_componentwillreceiveprops. Use https://github.com/reactjs/react-codemod#rename-unsafe-lifecycles to automatically update your components  react/no-deprecated

✖ 1 problem (1 error, 0 warnings)

npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR<EMAIL_ADDRESS>build-client: `NODE_ENV=production ykit pack -m`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>build-client script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR!      /Users/shifeng/.npm/_logs/2019-07-17T08_10_50_615Z-debug.log

什么浏览器
~  Chrome浏览器
什么系统（Linux, Windows, macOS）
macOS 10.14

已解决，是eslint错误导致程序退出，在eslintrc中加入 "react/no-deprecated": ["off"] 消除错误

舒服了

搞了半天   特么源码有问题  就问问还维护么   去哪儿技术团队是不是解散了


[Wallets] Explore "funny" verification UI in wallets (Valora)

Housekeeping

[ ] Connect with @aleksey-nov

Context
Assumptions

"Decentralised phone number verifications are slow", because (among other reasons):

a. Generating on-chain randomness takes time
b. Sending SMS message through the Attestation Service takes time

"Users expect fast phone number verifications", because

a. attention spans are short
b. phone number verifications are "usually" fast (when requesting OTP from Amazon, Google, etc for 2FA)
What can we do?
We can:

improve the verification speed (i.e. effort at protocol level), or
increase user patience and understanding (i.e. effort at wallet level)

Idea
Design wallet UIs that entertain users and visualise what "magic" is happening in the background.
e.g.

"While user waits for text messages, Valora displays message saying "Cooking up some magic in the background"
"generating randomness on chain so you don't really on centralised services"
"connecting you to decentralised validators"
"validators are sending you a message"

or gamify process

Google internet error dinosaur game

Hypotheses:

Users might be more patient if they are educated/entertained about the process

Closing this ticket in favour of this Github Discussions topic: https://github.com/celo-org/celo-monorepo/discussions/9306


Add documentation to gh pages

Host docs on gh pages.

Docs are now hosted on https://github.com/evanthegrayt/planter/tree/gh-pages


join examples?

any examples on how to perform a join?
here is how sqlx did it: https://snippets.aktagon.com/snippets/757-how-to-join-two-tables-with-jmoiron-sqlx

actually looks to work similarly, (a comma got me again in the db tag), tho leaving this here for others since docs don't cover

Hi and thank you for opening this issue.
Indeed, I didn't cover the join case in the docs explicitly, because the join isn't the responsibility of the scany library.
I provided explanations of how to reuse structs with scany (this is what you would want to do in case of a join) in that docs section<EMAIL_ADDRESS>Basically this section explains all mechanisms that one would one to use in case of a join.
But I think you are right, and having a high-level example of a join that showcases real-world usage would be great.
I will think of a way of adding it to the docs or as a separate article. Thanks!

Waiting for examples about how to create backref and etc

@arkady-bagdasarov Could you please explain in more detail what kind of examples you need?

@georgysavva for example, i how to implement has many relations? example from gorm

@georgysavva for example, i how to implement has many relations? example from gorm

@arkady-bagdasarov scany isn't an ORM. It doesn't work as gorm. You can't implement many to many or one to many relations with scany, because scany knows nothing about your relations and schema. The only way how scany can help you is by reusing your structs when you write your one to many/many to many queries with a JOIN.
Please check that docs section for details on how to reuse your structs with scany<EMAIL_ADDRESS>

Thanks!

@georgysavva  Hi!
Question related to Joins examples.
SELECT o.id, o.name,  FROM table AS o

Will scan work in this case (using AS)?

Thanks for the answer, regarding Joins. Faced today an issue with JOIN query
scany: rows contain a duplicate column 'id'

SELECT
                o.id, vi.id
 FROM table1 AS o LEFT JOIN table2 AS vi ON o.vi_id = vi.id WHERE o.id = $1

I feel that this is regular case to select ids with the same field name from different tables

Hi! The problem here is that the database translates "o.id" and "vi.id" to the same column name "id" and you need to explicitly alias them to make it work with scany:
SELECT
o.id AS "o.id", vi.id AS "vi.id"
FROM table1 AS o LEFT JOIN table2 AS vi ON o.vi_id = vi.id WHERE o.id = $1

so in this case how will structures look like?
type Table1 struct {
    ID string `db:"id"
}
type Table2 struct {
    ID string `db:"id"
}

type Table1 struct {
    ID string `db:"o.id"
}
type Table2 struct {
    ID string `db:"vi.id"
}

Depends on how you combine those to structs into a single one that you pass to scany. You can do it like that:
type Table1 struct {
    ID string `db:"id"
}
type Table2 struct {
    ID string `db:"id"`
}
type Result struct {
    O *Table1 `db:"o"`
    VI *Table2 `db:"vi"`
}

Oh, thanks, this is a really nice example. Didn't expect it will work this way

Oh, thanks, this is a really nice example. Didn't expect it will work this way

@Metalscreame Check out this doc section for more details<EMAIL_ADDRESS>

I'm struggling to make this work if you don't mind helping me. I have a SignUpLink (sign_up_links table) and Account (accounts table)
type SignUpLink struct {
	ID        *uuid.UUID
	AccountID *uuid.UUID
	Link      *string
	CreatedAt *time.Time
}

type Account struct {
	ID        *uuid.UUID
	Name      *string
	CreatedAt *time.Time
}

type SignUpLinkAccount struct {
	Account    *Account    `db:"a"`
	SignUpLink *SignUpLink `db:"sl"`
}

I want to do something like this (and handle the issue with duplicate column as mentioned above, but I'm not sure how to go about it)
SELECT sl.*, a.*
FROM public.signup_links sl
INNER JOIN public.accounts a ON sl.account_id = a.id
WHERE sl.id = $1;

rows, _ := s.db.Query(ctx, stmt, id)

	if err := pgxscan.Scan(signUpLinkAccount, rows); err != nil {
		return nil, newStoreError(err)
	}


New vscode icon is horrible

VSCode Version: 1.17.2
OS Version: Win 10

Steps to Reproduce:

Install fresh VSCode
Suffer

Reproduces without extensions: Yes
Solution: Just use the old one. If the similarity to the visual studio icon is an issue, google "vscode icon" for tons of great alternatives as a source for inspiration.
P.S. Sorry guys, I know this issue is kinda silly but I couldn't resist: new icon looks terrible. Especially the color is bad.

Just in case I'm talking about this one:

Duplicate of #35783


Support for missing ActivityPub vocabulary (sharedInbox, etc.)

In trying to declare a sharedInbox endpoint on an ActivityPub Actor object, I tried:
const actor = _.person()
  ⋮
  .endpoints(_.sharedInbox().url('https://ar.al/inbox'))

I get the error _TypeError: .sharedInbox is not a function.
.sharedInbox is undefined.
I can see that it is present in the activitystreams-context package in the context.json file:
    "sharedInbox": {
      "@id": "as:sharedInbox",
      "@type": "@id"
    }

But it:

Doesn’t have a static method defined in activitystreams.js
Doesn’t have a getter defined in _models/object.js
Is not added to the graph in reasoner.js
Doesn’t have a test defined in test.js

As I’m new to the library, before I go ahead and add it in those places:
(a) Is this the right way to do it (was it just not added or am I missing something?)
(b) I see that, e.g., Image is added to the graph twice. Is this the way to do it?
graph.add({
  subject: as.Image,
  predicate: rdfs.subClassOf,
  object: as.Document
});
// ⋮
graph.add({
  subject: as.image,
  predicate: rdf.type,
  object: owl.ObjectProperty
});

(c) Is there a reason for all this duplication? Would it be a worthwhile use of time to try and refactor the library so that elements are defined once if possible to keep things DRY? (As I see it, to add sharedInbox, along with the other optional properties defined in ActivityPub §4.1 – streams, preferredUsername, proxyUrl, oauthAuthorizationEndpoint, oauthTokenEndpoint, provideClientKey, signClientKey) would require making ~four entries for each, so ~24 definitions.
(d) Given that ActivityPub is extensible, does this mean that the parser will break and need to be updated to support every possible extension statically?
Thanks in advance :)
CC @evanp

Aww, crud. Yes, it'd be nice if stuff wasn't defined in different places. For now, I'm happy to take a PR with all these changes made.


How to conformance test minimal compliant spec (pull only)

Hi,
is there a way to to verify a system that fulfills the minimal requirements of the spec (All registries conforming to this specification MUST support, at a minimum, all APIs in the Pull category.) with the provided conformance test suite?
Even when running with all other OCI_TEST_... set to 0, setup and teardown still push layers, manifests, tags.
For context, I'm working on a separate read-only frontend part of "compatible" registry.

Did you use the options steps listed at: https://github.com/opencontainers/distribution-spec/blob/main/conformance%2FREADME.md#pull

OMG, I didn't scroll down enough. My bad


Q: Change nanodbc::result iteration model and rename bool end()

Mateusz already did some work in #155, and introduced the result_iterator. His todo list now contains implementation of .begin() and .end() of the nanodbc::result.
I am writing this just to support his activity that is a bit farther in his branch, to attract the attention :)
Changing the iteration model will make the processing more natural to all C++ developers. Being a container, the nanodbc::result seems to be natural candidate for that C++ kind of iteration. Also, there will be no doubt what values should .begin() and namely the new .end() have when called after the execute -- as discussed for example in #78.
However, the older .end() returns bool, and it conflicts with classical end iterator method/value. The older .end() could be renamed to .end_of_data() or so. The name could be that long to make a psychological push to use the classical iteration or--even better--using the range-based loop whenever possible.

@pepr Thanks for supporting this idea.
The current bool end() could be renamed to bool at_end() - as also corresponding to internal flag - or if we want an ugly name, then I'd go for a self-descriptive end_of_result since data is ambiguous (result data, row data, column value data...?)

Regarding rationale and intention of the current bool end(), it is important to have a look at https://github.com/lexicalunit/nanodbc/issues/78#issuecomment-161364477 (whole discussion too) and #88.

The requested feature has been Implemented in #155


Single Trip calendar throws date errors sometimes. Debug

When user go from alltrips and click on a trip. Then in the single trip clicks on the calendar. Sometimes the singletrip calendar throws an date error.

was using startTime instead of start


Is nextRTC use a single thread to handle connections ?

I'm trying to understand the code and I feel that there is a single thread that handle all incoming connections but I'm not sure of it.
I've read carefully the documentation and it's not mentioned anywhere that the server is single threaded or using multiple threads to handle connections.
It would be useful to make the documentation more clear on this point.
Thanks,

I'll prepare a section in documentation about threading model in NextRTC.
NextRTC is based on java standard websocket implementation, so there should
be one thread per endpoint instance. Endpoint instance is ussualy created
when new connection is opened but it's not a rule. I suspect that each
server has configuration where you can set max number of executors.
Outgoing messages are synchronized on session (server client connection),
but message sender has own pool to process outgoing traffic.
You have to keep that in mind during implementation own signals and event
handlers.
05.02.2018 4:35 PM "LorenzoMcMaster"<EMAIL_ADDRESS>napisał(a):
I'm trying to understand the cngode and I feel that there is a single
thread that handle all incoming connections but I'm not sure of it.
I've read carefully the documentation and it's not mentioned anywhere that
the server is single threaded or using multiple threads to handle
connections.
It would be useful to make the documentation more clear on this point.
Thanks,
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/mslosarz/nextrtc-signaling-server/issues/30, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ADysRYGVt38nro-c9CHrTuA9ypGX2hp0ks5tRx_OgaJpZM4R5oO6
.

No longer valid in version RC5 you can decide how you're going to handle connections. :)


Format changes/cleanup of the TSV file

Have just chatted through the TSV format with @jeff-k , and am logging here my proposed changes to the format, which Jeff is happy with
Current format
31	0	30	A	453	1	454	0	0	453	11	A:453;T:1	primer_calls_ignored=0/0;total_primer_bases=453/1;unfiltered_depth=454;total=453/1;amplicon_overlap=1;amplicon_totals=453/1;amplicon_names=nCoV-2019_1_pool1 32	1	31	C	469	1	470	0	0	469	11	C:469;T:1	primer_calls_ignored=0/0;total_primer_bases=469/1;unfiltered_depth=470;total=469/1;amplicon_overlap=1;amplicon_totals=469/1;amplicon_names=nCoV-2019_1_pool1 33	2	32	C	508	1	509	0	0	508	11	C:508;T:1	primer_calls_ignored=0/0;total_primer_bases=508/1;unfiltered_depth=509;total=508/1;amplicon_overlap=1;amplicon_totals=508/1;amplicon_names=nCoV-2019_1_pool1 34	3	33	A	530	7	537	0	0	530	71	A:530;T:6;C:1	primer_calls_ignored=0/0;total_primer_bases=530/7;unfiltered_depth=537;total=530/7;amplicon_overlap=1;amplicon_totals=530/7;amplicon_names=nCoV-2019_1_pool1 35	4	34	A	532	11	543	0	0	532	11	1	A:532;G:11	primer_calls_ignored=0/0;total_primer_bases=532/11;unfiltered_depth=543;total=532/11;amplicon_overlap=1;amplicon_totals=532/11;amplicon_names=nCoV-2019_1_pool1 36	5	35	C	547	7	554	0	0	536	71	-:4;C:547;T:2;G:1	primer_calls_ignored=0/0;total_primer_bases=536/7;unfiltered_depth=554;total=547/7;amplicon_overlap=1;amplicon_totals=547/7;amplicon_names=nCoV-2019_1_pool1
Proposals:

[x] Add header line
[x] Make one column each for A,C,G,T,-
[x] turn semi-colon separated things into individual columns
[x] Use primer_calls_ignored, unfiltered_depth etc as column headers, so cells contain just the values
[x] Add a column with the reference bases
[x] Remove the redundant column (we have both 0-based and 1-based coords in the reference as two columns (the 30 and 31 in that first row)

So proposed format is:
I'm going to give two columns. Left-hand one is the proposed header text for a column in the tsv, and the right-hand one is the explanation of what it is

Header text
Meaning

Pos.ref
coordinate in the ref genome

Base.ref
base in the ref genome (may be -)

Pos.cons
corresponding (in MSA) base of consensus

Base.cons
base in the consensus (may be - )

A
Total reads with an A at this position (raw count, no exclusion of primer bases etc)

C
Total reads with an C at this position (raw count, no exclusion of primer bases etc)

G
Total reads with an G at this position (raw count, no exclusion of primer bases etc)

T
Total reads with an T at this position (raw count, no exclusion of primer bases etc)

-
Total reads with an gap at this position (raw count, no exclusion of primer bases etc)

RawDepth
total raw count of reads at this position (no exclusion of primer bases etc)

Clean.Tot.cons
total reads which agree with consensus, after applying viridian rules (eg dont count reads where this posn is a primer,if there is an overlapping amplicon); "clean" read support

Clean.Tot.noncons
total reads which disagree with the consensus, after applying viridian rules, disagreement with consensus from "clean" reads

InPrimer
boolean for whether this position is in a primer that is seen in the data (if a primer scheme has multiple alternate primers at one end of an amplicon, those which are not seen in this read-set are ignored)

SchemeAmpCount
Number of amplicons overlapping this position in the amplicon scheme (property only of scheme, not the reads/data)

P.ignored.cons
total reads had a primer here, which agree with consensus, but which were ignored by viridian applying rules in primers

P.ignored.noncons
total reads had a primer here, which DISagree with consensus, but which were ignored by viridian applying rules in primers

P.notignored.cons
total reads had a primer here, which agree with the consensus, and were NOT ignored by viridian, applying rules in primers

P.cons
total reads which agree with consensus, in a primer (no filtering, raw count)

P.noncons
total reads which disagree with consensus, in a primer (no filtering, raw count)

AmpOv
Number of amplicons which overlap this position, after excluding dropped amplicons

This looks super helpful for casually interrogating outputs 🙏
On Wed, 10 Aug 2022 at 13:18, Zamin Iqbal @.***> wrote:

Have just chatted through the TSV format with @jeff-k
https://github.com/jeff-k , and am logging here my proposed changes to
the format, which Jeff is happy with
Current format
31 0 30 A 453 1 454 0 0 453 11 A:453;T:1
primer_calls_ignored=0/0;total_primer_bases=453/1;unfiltered_depth=454;total=453/1;amplicon_overlap=1;amplicon_totals=453/1;amplicon_names=nCoV-2019_1_pool1
32 1 31 C 469 1 470 0 0 469 11 C:469;T:1
primer_calls_ignored=0/0;total_primer_bases=469/1;unfiltered_depth=470;total=469/1;amplicon_overlap=1;amplicon_totals=469/1;amplicon_names=nCoV-2019_1_pool1
33 2 32 C 508 1 509 0 0 508 11 C:508;T:1
primer_calls_ignored=0/0;total_primer_bases=508/1;unfiltered_depth=509;total=508/1;amplicon_overlap=1;amplicon_totals=508/1;amplicon_names=nCoV-2019_1_pool1
34 3 33 A 530 7 537 0 0 530 71 A:530;T:6;C:1
primer_calls_ignored=0/0;total_primer_bases=530/7;unfiltered_depth=537;total=530/7;amplicon_overlap=1;amplicon_totals=530/7;amplicon_names=nCoV-2019_1_pool1
35 4 34 A 532 11 543 0 0 532 11 1 A:532;G:11
primer_calls_ignored=0/0;total_primer_bases=532/11;unfiltered_depth=543;total=532/11;amplicon_overlap=1;amplicon_totals=532/11;amplicon_names=nCoV-2019_1_pool1
36 5 35 C 547 7 554 0 0 536 71 -:4;C:547;T:2;G:1
primer_calls_ignored=0/0;total_primer_bases=536/7;unfiltered_depth=554;total=547/7;amplicon_overlap=1;amplicon_totals=547/7;amplicon_names=nCoV-2019_1_pool1
Proposals:

Add header line
Make one column each for A,C,G,T,-
turn semi-colon separated things into individual columns
Use primer_calls_ignored, unfiltered_depth etc as column headers, so
cells contain just the values
Add a column with the reference bases
Remove the redundant column (we have both 0-based and 1-based coords
in the reference as two columns (the 30 and 31 in that first row)

So proposed format is:
I'm going to give two columns. Left-hand one is the proposed header text
for a column in the tsv, and the right-hand one is the explanation of what
it is
Header text Meaning
Pos.ref coordinate in the ref genome
Base.ref base in the ref genome (may be -)
Pos.cons corresponding (in MSA) base of consensus
Base.cons base in the consensus (may be - )
A Total reads with an A at this position (raw count, no exclusion of
primer bases etc)
C Total reads with an C at this position (raw count, no exclusion of
primer bases etc)
G Total reads with an G at this position (raw count, no exclusion of
primer bases etc)
T Total reads with an T at this position (raw count, no exclusion of
primer bases etc)

Total reads with an gap at this position (raw count, no exclusion of
primer bases etc)
Depth total raw count of reads at this position (no exclusion of primer
bases etc)
Tot.cons total reads which agree with consensus, after applying viridian
rules (eg dont count reads where this posn is a primer,if there is an
overlapping amplicon)
Tot.noncons total reads which disagree with the consensus, after applying
viridian rules
P.ignored.cons total reads had a primer here, which agree with consensus,
but which were ignored by viridian applying rules in primers
P.ignored.noncons total reads had a primer here, which DISagree with
consensus, but which were ignored by viridian applying rules in primers
P.notignored.cons total reads had a primer here, which agree with the
consensus, and were NOT ignored by viridian, applying rules in primers
P.cons total reads which agree with consensus, in a primer (no filtering,
raw count)
P.noncons total reads which disagree with consensus, in a primer (no
filtering, raw count)
AmpOv Number of amplicons which overlap this position, after excluding
dropped amplicons

—
Reply to this email directly, view it on GitHub
https://github.com/iqbal-lab-org/viridian_workflow/issues/92, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAHWAAGNTM5LW4VH7VDEW3LVYOM2JANCNFSM56EJ4OPA
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

There's a flexible engine for dumping out per position stats now.  If I've lined up the columns correctly, it's got values 967, 971, 971, .... But should be a bool, and present for every ref position, like the SchemeAmpCount column.

Sorry, I'd preserved the previous definition which was how many of the bases are in primers. I think the bool version can be inferred by what we have, right?

well, the spec above is
boolean for whether this position is in a primer that is seen in the data (if a primer scheme has multiple alternate primers at one end of an amplicon, those which are not seen in this read-set are ignored)

I don't know how the current count compares to this planned bool:
"boolean for whether this position is in a primer that is seen in the data (if a primer scheme has multiple alternate primers at one end of an amplicon, those which are not seen in this read-set are ignored)"
Specifically, can the count be >0 but bool be False? eg does primer only being seen in a single read result in that primer being included in the inferred primer scheme?
Also, can the count be 0 but bool be True? eg when reads suggest an amplicon is present, but none of the primers are found in the reads for that amplicon, we use all primers for that amplicon

Just instead of a bool, we have a count. So 0 instead of False and the count instead of True.
The reason the counts are not available for ref positions that aren't represented by consensus calls is because "boolean for whether this position is in a primer that is seen in the data"

So you mean 0==False, and >=1 means True

I think noone wants the bool to be one for primer positions that are never seen in the data, so maybe this works

Have you changed the semantics of any of the other columns or are the other definitions literally true?

I wanted the bool to show the positions that are in primers, in the inferred primer scheme the we use for assembly. This should be present for every position in the reference genome.

that is what this is doing. Jeff is just saying that it wont mention primers that are never seen in any reads

Yes that's right. Counting bases that are in primers is critical for my tests and was an original column. In the slack thread Martin requested an additional bool column that's true when the count is >0 and Zam combined these two columns in this issue.
I didn't read the details when I implemented this because the only difference between Zam's columns and what we already had is the change from int to bool in this one column. Nothing else has changed.
Anyway, there's apparently an off-by-one error in the reference position column.

I now understand what Martin is saying i think, which is not what I thought a couple of comments back. I'll update on Monday


Demo: save generated images

Changes:

In the Gradio demo, generated images were not being saved. Now every generated image will be automatically saved to the directory ./outputs as a .png file. The filename will be a timestamp with the format: <year>_<month>_<day>-<hour>_<minute>_<second>.png
Edited some spelling and grammar in the Gradio demo description to help with clarity.

Tested these changes on a Linux machine with A6000 GPU.

Thank you for your contribution. However, I'm not sure if users want to save all the images.

That's fair. To be honest, when I first tried the Gradio demo, I did not see any way to download the generated image in the interface, which is why I added this. I see now that it is possible to download an image by clicking in the upper right corner of the generated image in the UI. Many image generation apps do save all generated images by default, but this is a preference, rather than a necessary feature.
I have commented out the automatic image saving in this PR, just in case you want to incorporate my spelling/grammar edits.
Thanks so much for your work on this incredible project. OmniGen is truly a paradigm shift in image generation.

Thank you rubendax, I'd been searching specifically for how to enable this ability and found it here.
I appreciate the ability to save to a folder of choice, downloading each time to the browser's default download folder was becoming cluttered. Although the PNG files are much larger, having this convenience is worth the option for me.
Also, since I didn't know how to use GitHub correctly to use the 'branch' I manually swapped app.py for this pull request to my installation. My guess is that's not the way to do it!
As a hobbyist, OmniGen is already a much-appreciated part of my toolkit; this is only my 2nd day of using it.

@vootox You should clone my branch called demo-save_outputs. This has the auto-saving functionality enabled and is otherwise up-to-date with all the improvements from upstream VectorSpaceLab:main (at least as of today 2024-11-05).
git clone -b demo-save_outputs https://github.com/rubendax/OmniGen

@staoxiao I have added a checkbox to the UI to allow the user to toggle on or off saving all generated images as .png files to ./outputs. In this branch, rubendax:main, this functionality is toggled off (unchecked) by default.

@vootox You should clone my branch called demo-save_outputs. This has the auto-saving functionality enabled and is otherwise up-to-date with all the improvements from upstream VectorSpaceLab:main (at least as of today 2024-11-05).
git clone -b demo-save_outputs https://github.com/rubendax/OmniGen

Thank you @rubendax, I have this installed and running, I like this a lot. Now, I'm wondering if there could be a script for batch processing images... just kidding, but it would be nice!