Building for Windows

Hello,
I'm trying to bulid Zyn-fusion like a win-VST on Ubuntu 16.04 and 14.04 but I get this error:
`zlib-1.2.7.tar.gz                                            100%[=============================================================================================================================================>] 547,22K   593KB/s    za 0,9s
2018-01-16 09:51:41 (593 KB/s) – „zlib-1.2.7.tar.gz“ uloženo [560351/560351]
tar: zlib-1.2.7: Funkce read selhala: je adresářem
tar: Páska na začátku, končím
tar: Error is not recoverable: exiting now
Building static library libz.a version 1.2.7 with x86_64-w64-mingw32-gcc.
Checking for off64_t... Yes.
Checking for fseeko... Yes.
Checking for strerror... Yes.
Checking for unistd.h... Yes.
Checking for stdarg.h... Yes.
Checking whether to use vs[n]printf() or s[n]printf()... using vs[n]printf().
Checking for vsnprintf() in stdio.h... Yes.
Checking for return value of vsnprintf()... Yes.
Checking for attribute(visibility) support... No.
./ztest30865: 1: ./ztest30865: MZ����@���: not found
./ztest30865: 2: ./ztest30865: Syntax error: word unexpected (expecting ")")
Looking for a four-byte integer type...

As far as roadmap is concerned, CMAKE + MSYS2? That way we can do away with the VM for Windows builds. It looks like it would require quite a bit of work though.

@trayes The build system/buildable systems depend upon the needs of users and contributors. At the moment (and historically) there aren't many people developing zyn on windows, so the mingw approach is being used to cross compile. Once the issues are sorted within this thread I'm going to want to add CI builds via travis-ci to ensure that the mingw build becomes easier to setup/use. IIRC travis-CI only has ubuntu and MacOS build machines.
If you're interested on doing development on a windows machine targeting a windows machine, then I would say that getting a msys build could be helpful, though it would certainly take a fair amount of time.
If you have a recent (windows 10 IIRC) windows install then one workaround before setting up anything native could be to use the linux subsystem for windows. I've been using that at my day job and have had decent success using it for my own work (though I haven't applied it to anything like zyn).

That's a good point, and would be a great use case for LSW, which I haven't tried yet.
I'll try to get cracking on it.

Tried building again with doxygen, fltk, and cxxtest installed and no luck.
How do I work with zyn in WSL? I rubied build-linux.sh and successfully installed. However I could find related documents on how alsa/jack works with WSL, or if it's possible to get any kind of graphic representation in WSL...

or if it's possible to get any kind of graphic representation in WSL

You'll need an X server to get a graphical representation.
I use 'XLaunch' and it has worked well enough for my uses on windows.

how alsa/jack works with WSL

No idea. I haven't tried to output any audio from a WSL program. My guess is that ALSA works, but not particularly well.

How do I work with zyn in WSL?

I would recommend trying to get the windows version built via the build-windows.rb script from within a WSL shell. If that's possible, then you get a native windows binary without the need for a Linux VM.
You should be able to run the Linux binary produced by the build-linux.rb, however I would expect that to not be the desired final result.

I got to 91% building in WSL after taking adviced from: https://www.kvraudio.com/forum/viewtopic.php?p=7030831#p7030831
Right now the error returns:
[ 91%] Linking CXX executable zynaddsubfx.exe
collect2: error: ld returned 1 exit status
src/CMakeFiles/zynaddsubfx.dir/build.make:107: recipe for target 'src/zynaddsubfx.exe' failed
make[2]: *** [src/zynaddsubfx.exe] Error 1
CMakeFiles/Makefile2:1533: recipe for target 'src/CMakeFiles/zynaddsubfx.dir/all' failed
make[1]: *** [src/CMakeFiles/zynaddsubfx.dir/all] Error 2
Makefile:138: recipe for target 'all' failed
make: *** [all] Error 2
ERROR: 'make' failed
There's too many smaller error prior to this and I couldn't find anything related from a glance.
I'll try building on ubuntu later.

collect2: error: ld returned 1 exit status

A linker error indicates that some component at a previous stage failed to build. I'd recommend looking for an error earlier in the build process.

Anyone had have succes building Zyn-fusion for windows? I really want to know how to build it
:) :3

If you want, you can try the native windows build scripts for msys2's mingw64.exe

But, what difference is with the .cmake based one

How can I build it with the msys2 scripts?, sorry for my inexperience, I'm new

No Problem, everybody has to start from somewhere :)
First download msys2 from here: https://www.msys2.org/

For some reason I had to launch the build script like this for it to work:
MSYSTEM="" sh build-mingw64.sh
Without erasing the $MSYSTEM value it would fail with this error:
No wscript file in directory C:/Users/person/Downloads/zyn-fusion-build/mruby-zest-build/deps/pugl
Even though the wscript file is there, I think there is some problem with parsing the path. I have python2.7 in my path if that matters.

Thanks for the help, I will give it a try

Thank you guys, It worked amazingly good. Thanks MatschbirneLive for your step by step manual. I only have one cuestion... can it be updated?
I can't thank you enough for your help. I'm pretty happy to have a powerful synth in my budget project.
:) :3

What do you mean by updated ?
If you mean the synth, the build script downloads the latest source code of zynaddsubfx from git, so to update you just simply need to rebuild zynfusion again.

This way (msys2) it works like a charm ;-) Thank you very much

Thank you, you are the best :)

No Problem, everybody has to start from somewhere :)
First download msys2 from here: https://www.msys2.org/
Then open the msys2 shell and type
pacman -Syu
If told to restart the shell and then type
pacman -Su
Now you just need git, to install just type:
pacman -S git
Now you can close the msys2 shell and now you have to open the mingw64.exe (Should be in the same directory as the shell)
Now You need to type
git clone https://github.com/zynaddsubfx/zyn-fusion-build.git
cd zyn*
./build-mingw64.sh
Now the script should download all required packages and then build ZynFusion
Hope it helps

Hello,
I tried these steps and it failed. Because of a file called backtrace
I then followed the steps from here:
https://github.com/zynaddsubfx/zyn-fusion-build/issues/33
That got me a bit further and then it failed because I didnt have python2 yet.
I installed python2 and that got me a bit further again.
But now I have a bunch of errors starting with:
D:/Applications/zyn-fusion-build/mruby-zest-build/mruby/src/backtrace.c: In function 'print_backtrace_saved':
D:/Applications/zyn-fusion-build/mruby-zest-build/mruby/src/backtrace.c:122:23: error: 'mrb_state' {aka 'struct mrb_state'} has no member named 'backtrace'
eventually resulting in:
rake aborted!
Command failed with status (1): [gcc -I/d/Applications/zyn-fusion-build/mru...]
/d/Applications/zyn-fusion-build/mruby-zest-build/mruby/lib/mruby/build/command.rb:31:in _run' /d/Applications/zyn-fusion-build/mruby-zest-build/mruby/lib/mruby/build/command.rb:36:in rescue in _run'
/d/Applications/zyn-fusion-build/mruby-zest-build/mruby/lib/mruby/build/command.rb:32:in _run' /d/Applications/zyn-fusion-build/mruby-zest-build/mruby/lib/mruby/build/command.rb:89:in run'
/d/Applications/zyn-fusion-build/mruby-zest-build/mruby/lib/mruby/build/command.rb:113:in `block (2 levels) in define_rules'
Caused by:
Command failed with status (1): [gcc -I/d/Applications/zyn-fusion-build/mru...]
/d/Applications/zyn-fusion-build/mruby-zest-build/mruby/lib/mruby/build/command.rb:33:in _run' /d/Applications/zyn-fusion-build/mruby-zest-build/mruby/lib/mruby/build/command.rb:89:in run'
/d/Applications/zyn-fusion-build/mruby-zest-build/mruby/lib/mruby/build/command.rb:113:in `block (2 levels) in define_rules'
Tasks: TOP => default => all => /d/Applications/zyn-fusion-build/mruby-zest-build/mruby/build/host/lib/libmruby.flags.mak => /d/Applications/zyn-fusion-build/mruby-zest-build/mruby/build/host/lib/libmruby.a => /d/Applications/zyn-fusion-build/mruby-zest-build/mruby/build/host/src/backtrace.o
(See full trace by running task with --trace)
make: *** [Makefile:42: windows] Error 1
I've tried removing the print function because who needs printing in a finished product ;) but it didnt help.
 I would love to use this as a 64-bit vst in Ableton Live ;)
Kind regards,

MSYSTEM="" sh build-mingw64.sh

Dude, I love you (no homo).

MSYSTEM="" sh build-mingw64.sh

Dude, I love you (no homo).

@msebi,
I don't remember how I figured that out or why it was necessary, but I'm glad it helped someone else also.


[BEAM-13430] Remove propdeps and replace with compileOnly

Still generate provided in the output POM.
Remove some unnecessary "provided" dependencies from examples.
add a dependency on runners:core-java from runners:java-fn-execution which was previously implicit
Remove usage of JacksonFactory which is deprecated, replace with GsonFactory
Fix dependencies of sdks:java:extensions:sql and other packages with issues revealed by this change

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Choose reviewer(s) and mention them in a comment (R: @username).
[ ] Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
[ ] Update CHANGES.md with noteworthy changes.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

retest this please

retest this please

retest this please

retest this please

Change to SQL test for BEAM-13506 LGTM

Python precommit failure looks like a flake

What was the reasoning behind going with compileOnly instead of creating a new provided configuration?
Note there were a couple of compileOnly dependencies before this change that now will be marked as provided such as all the compileOnlyAnnotations within BeamModulePlugin.groovy.

What was the reasoning behind going with compileOnly instead of creating a new provided configuration?
Note there were a couple of compileOnly dependencies before this change that now will be marked as provided such as all the compileOnlyAnnotations within BeamModulePlugin.groovy.

@lukecwik - should this be a jira tracked as part of https://issues.apache.org/jira/browse/BEAM-13430 ?


Usability improvements to snapshot functionality

The snapshot functionality is solid - love it so far! Here's a couple of suggestions to make it even better:

Have "ddev restore-snapshot" automatically restore the most recent snapshot when no snapshot_name is provided.
Add a "ddev snapshot --list" flag to show a list of all available snapshots for a given project.

I could also be convinced to move "ddev restore-snapshot" to "ddev snapshot --restore" - I think having a small set of commands is one of the really nice things about DDEV.
-mike

Thanks!

Closing in favor of #1163, comment moved over to that one.

I hope you'll try out v1.17.0-alpha3, which has this feature,  and give your feedback!

I hope you'll try out v1.17.0-alpha3, which has this feature,  and give your feedback!


Regex Test

@Zoey2936 Is everything still working? Changed the domain regex in the frontend

no, the ci needs to be green

@DavidCraftDev ping

@Zoey2936 Pong

@Zoey2936 I think I have solved the problem, but before you merge, can you please have a look on the regex I implemented to check if the domain names are correct? /^(?:*.)?(?:[^().]+|([^)]))*[^.]$/

Bad, upstream merge


Add Chain saving functionality to Database

Closes #203

small clean in peer
Adds a String method to protocol Magic

Codecov Report

Merging #206 into dev will decrease coverage by 0.04%.
The diff coverage is 56.47%.

@@            Coverage Diff            @@
##             dev     #206      +/-   ##
=========================================
- Coverage   52.7%   52.66%   -0.05%
=========================================
  Files         75       75
  Lines       2533     2611      +78
=========================================
+ Hits        1335     1375      +40
- Misses      1078     1100      +22
- Partials     120      136      +16

Impacted Files
Coverage Δ

pkg/database/table.go
0% <0%> (ø)
:arrow_up:

pkg/peer/peer.go
26.5% <33.33%> (-7.85%)
:arrow_down:

pkg/database/leveldb.go
53.84% <40%> (-25.11%)
:arrow_down:

pkg/chain/chaindb.go
63.63% <63.63%> (ø)

pkg/crypto/publickey/publickey.go
47.36% <0%> (-2.64%)
:arrow_down:

pkg/connmgr/connmgr.go

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e12255d...ffd6ec0. Read the comment docs.

Award decentralisedkev 300 points


Http::assertSent on get request causes Undefined Index exception

Laravel Version: 7.6.2
PHP Version: 7.4.4

Description:
When doing an Http::assertSent I am getting Undefined index: Content-Type when testing a Http::get request that wouldn't have a Content-Type.
ErrorException : Undefined index: Content-Type
   .../Illuminate/Http/Client/Request.php:72
   .../Illuminate/Http/Client/Request.php:182
   .../Illuminate/Http/Client/Request.php:136
   .../Illuminate/Http/Client/Request.php:237
  // ... line on my test

Steps To Reproduce:
Here is an example that is similar to my actual code:
public function get(): array
{
    return \Http::acceptJson()
        ->get('https://google.com', ['apikey' => 'aaa-bbb-ccc'])
        ->json();
}

Then in my test I have something like the below:
public function testGet()
{
    \Http::fake(['*' => \Http::response(['response' =>'Test return data'], 200)]);

    // .. call get()

    \Http::assertSent(function (Request $request) {
        $this->assertStringStartsWith('https://google.com', $request->url());
        $this->assertSame('aaa-bbb-ccc', $request['apikey']); // Undefined index: Content-Type
        return true;
    });
}

Cause
Looking at the line causing the error it seems it just needs to check if the key it is searching for exists in the headers array before trying to access it:
Illuminate/Http/Client/Request.php:
public function isForm()
{
    return $this->hasHeader('Content-Type', 'application/x-www-form-urlencoded');
}

...

public function hasHeader($key, $value = null)
{
    ...
    // Needs an Arr::has or array_key_exists here? This ⬇ is causing the error.
    return empty(array_diff($value, $this->headers()[$key]));
}

Thanks. Feel free to PR.

@driesvints I made a start on this and have added $request['foo'] check to the below test:
    public function testGetWithArrayQueryParam()
    {
        $this->factory->fake();

        $this->factory->get('http://foo.com/get', ['foo' => 'bar']);

        $this->factory->assertSent(function (Request $request) {
            return $request->url() === 'http://foo.com/get?foo=bar'
                && $request['foo'] === 'bar';
        });
    }

But it appears that on get requests you cannot access the query params this way. Is this intended behaviour or just  a side effect of get using the guzzle query option?

No idea atm I'm afraid :/

@driesvints Welp 🤷 I have assumed its a side effect and added support please see PR.


Why reuse default to True for loop_function() in seq2seq.rnn_decoder ?

My question concerns the rnn_decoder() in seq2seq.
I am batchnormalizing activation maps in the loop_function(). To this end, I call the tf.contrib.layers.batch_norm() function. Like many other optionalities, it calls variables using tf.get_variable().
Ideally, you'd set reuse=None during the first call of loop_function(). That way, tf.get_variable() functions can construct variables. During second and subsequent calls of loop_function() you set reuse=True.
However, in rnn_decoder() reuse is set to True also during the first call. This doesn't allow the tf.get_variable() functions to construct variables.
My question is: why is this so? Am I doing something wrong? Or what is the logic here?
More specifically, my loop_function is:
    def loop_function(output, i):
      ##some code##
      some_variable = tf.contrib.layers.batch_norm(some_variable,is_training= self.is_training)
      ##some code##
      return some_variable

I made a work-around, by defining my own rnn_decoder. I copy-paste the Tensorflow implementation and set reuse=None during the first call
def rnn_decoder_rob(decoder_inputs, initial_state, cell, loop_function=None,
                scope=None):
  REUSE=None
  with variable_scope.variable_scope(scope or "rnn_decoder"):
    state = initial_state
    outputs = []
    prev = None
    for i, inp in enumerate(decoder_inputs):
      if loop_function is not None and prev is not None:
        with variable_scope.variable_scope("loop_function", reuse=REUSE):
          inp = loop_function(prev, i)
      if i > 0:
        variable_scope.get_variable_scope().reuse_variables()
        REUSE = True
      output, state = cell(inp, state)
      outputs.append(output)
      if loop_function is not None:
        prev = output
  return outputs, state

@ebrevdo can you comment on this?  Thanks!

@tatatodd is that still current?

Closing due to inactivity. Feel free to re-open if you would like us to look again.


The latest stable 1.0.0 release is erroring out Unexpected token " in JSON at position 0

Environment
npx @cardano-graphql/cli system-info

{ os:
{ arch: 'x64',
cpus:
{ count: 8,
model: 'Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz' },
release: '4.14.181-142.260.amzn2.x86_64',
totalMem:<PHONE_NUMBER>8,
type: 'Linux' },
version: '1.0.0' }
Platform
docker image in k8s
Runtime

[ ] Node.js
[v] Docker

Runtime version:
19.06.x
Steps to reproduce the bug
The moment I start cardano-graphql I get
SyntaxError: Unexpected token " in JSON at position 0
    at JSON.parse (<anonymous>)
    at createStrictSyntaxError (/application/node_modules/body-parser/lib/types/json.js:158:10)
    at parse (/application/node_modules/body-parser/lib/types/json.js:83:15)
    at /application/node_modules/body-parser/lib/read.js:121:18
    at invokeCallback (/application/node_modules/raw-body/index.js:224:16)
    at done (/application/node_modules/raw-body/index.js:213:7)
    at IncomingMessage.onEnd (/application/node_modules/raw-body/index.js:273:7)
    at IncomingMessage.emit (events.js:194:15)
    at endReadableNT (_stream_readable.js:1125:12)
    at process._tickCallback (internal/process/next_tick.js:63:19)

And it's in loop.
What is the expected behavior?
No logs, as nothing is connecting to the endpoint

I've just added issue templates to better assist with support @ksemaev . Would you mind patching this issue using the markdown from https://github.com/input-output-hk/cardano-graphql/issues/new?template=bug.md ?

Done, please take a look. Although it doesn't cause any problems AFAIK

I can't reproduce this, however the error is indicating a malformed client request (body-parser extracts the query from the HTTP request). Do you have a client immediately calling the API? Maybe try master to see if there's any difference, and reopen if you can provide a way for me to reproduce this in isolation.


Trying to use it with Odoo but no results.

Very exciting library! I'm trying to use it with Odoo but port 5555 outputs connection refused.
Maybe I'm missing something? Or if you have any advice, would be appreciated.

I have no idea what "Odoo" is, but general advice is to check your firewall settings and/or try a different port.

Just fixed it, it was environmental issue. Thanks a lot for responding. Though, when I click continue execution the next time I pass the breakpoint it throws:
  File "/usr/lib/python2.7/bdb.py", line 49, in trace_dispatch
    return self.dispatch_line(frame)
  File "/usr/lib/python2.7/bdb.py", line 68, in dispatch_line
    if self.quitting: raise BdbQuit
BdbQuit

Thanks for the library!

Please provide as much details as possible to reproduce your problem. Also, there's a known bug when BdbQuit is thrown when set_trace() is put on the very last line of a .py file.

My app (Odoo) is in a docker container. Host OS is Windows. Not sure how can you reproduce easily. :(
And no, in my case the set_trace() is not at the end of .py file.

Unfortunately, without being able to reproduce the issue I cannot help you much.


Job configuration containing CodeBuild lost after Jenkins reboot

Issue seen on Jenkins 2.32.1 with plugin code commit '0046d84' (mavenID 	com.amazonaws:aws-codebuild:1.1):
Hello,
I'm experiencing that after a Jenkins reboot the Jenkins job containing the CodeBuild Plugin as Buildstep is losing its buildstep configuration completly.
Thanks

I may add, the job configuration contains two build steps, a shell step first, followed by a CodeBuild step.

The configuration for the plugin is stored in /jobs//config.xml under the  item. Can you provide more information about the Jenkins server on which this issue occurred? It may be moving/deleting the config.xml.

Might be an issue with Jenkins ver. 2.32.1, I can't reproduce this error on Jenkins ver. 2.32.2. In my view the issue can be closed then.


[BUG]UE 5.4.4,  UStruct may mismatch cause Guid changed

In UE 5.4.4,  UStruct may mismatch cause Guid changed
Test CSharp code

[UStruct]
public struct FMyStruct
{
    [UProperty(PropertyFlags.BlueprintReadWrite, Category = "MyProperty")]
    public int IntValue;

    [UProperty(PropertyFlags.BlueprintReadWrite, Category = "MyProperty")]
    public int IntValue2;
}

[UClass]
public class AMyCharacter : ACharacter
{
    [UProperty(PropertyFlags.BlueprintReadWrite, Category = "MyProperty")]
    public FMyStruct MyStruct { get; set; }
}

Create BP_MyPlayer inheriting AMyCharacter, and then compile C#

#229


Update tesla-motors in sources-dist-stable.json

Need to update as Tesla changed authentication and older version will fail.

RE-CHECK!

Automated adapter checker
ioBroker.tesla-motors

[ ] :heavy_exclamation_mark: [E150] No common.connectionType found in io-package.json
[ ] :heavy_exclamation_mark: [E152] No common.dataSource found in io-package.json

Add comment "RE-CHECK!" to start check anew


Move BrokerSelector and implementations to pinot-common

BrokerSelector is useful in other components when an appropriate broker has to be selected. For example,
the controller has to choose a broker that can process all tables in a V2 query.
Move BrokerSelector to pinot-common so that other components can have access to the interface and implementations.
Also add CommonTenantBrokerSelector which will error out if a broker that can process all the tables in a query cannot
be found.
Remove similar code in the controller that chose the right tenant and replace it with a broker selector.
Tags:
multi-stage
cleanup

Codecov Report
Attention: Patch coverage is 0% with 111 lines in your changes missing coverage. Please review.

Project coverage is 0.00%. Comparing base (59551e4) to head (4969acb).
Report is 599 commits behind head on master.

Files
Patch %
Lines

...ava/org/apache/pinot/common/broker/BrokerInfo.java
0.00%
24 Missing :warning:

...apache/pinot/common/broker/ExternalViewReader.java
0.00%
21 Missing :warning:

...che/pinot/common/broker/DynamicBrokerSelector.java
0.00%
19 Missing :warning:

...inot/common/broker/CommonTenantBrokerSelector.java
0.00%
16 Missing :warning:

...main/java/org/apache/pinot/client/BrokerCache.java
0.00%
13 Missing :warning:

...t/controller/api/resources/PinotQueryResource.java
0.00%
11 Missing :warning:

...apache/pinot/controller/BaseControllerStarter.java
0.00%
3 Missing :warning:

...pache/pinot/common/broker/BrokerSelectorUtils.java
0.00%
2 Missing :warning:

.../main/java/org/apache/pinot/client/BrokerData.java
0.00%
1 Missing :warning:

...org/apache/pinot/common/broker/BrokerSelector.java
0.00%
1 Missing :warning:

Additional details and impacted files
@@              Coverage Diff              @@
##             master   #13361       +/-   ##
=============================================
- Coverage     61.75%    0.00%   -61.76%
=============================================
  Files          2436     2475       +39
  Lines        133233   136280     +3047
  Branches      20636    21163      +527
=============================================
- Hits          82274        0    -82274
- Misses        44911   136280    +91369
+ Partials       6048        0     -6048

Flag
Coverage Δ

custom-integration1
?

integration
0.00% <0.00%> (-0.01%)
:arrow_down:

integration1
?

integration2
0.00% <0.00%> (ø)

java-11
?

java-21
0.00% <0.00%> (-61.63%)
:arrow_down:

skip-bytebuffers-false
0.00% <0.00%> (-61.75%)
:arrow_down:

skip-bytebuffers-true
0.00% <0.00%> (-27.73%)
:arrow_down:

temurin
0.00% <0.00%> (-61.76%)
:arrow_down:

unittests
?

unittests1
?

unittests2
?

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

@vrajat it looks like some unit tests in DynamicBrokerSelectorTest and ExternalViewReaderTest are failing due to this change.

I think DynamicBrokerSelector had a bug. It was returning a list of ZK servers in DynamicBrokerSelector.getBrokers(). The test also used the list of ZK servers as the expected results. I checked other implementations and they do return broker information and not ZK server information. So I have fixed it here: https://github.com/apache/pinot/pull/13361/files#diff-9aba27bda6f4d7695e6032c5354a3d249fb4d10f7c23c8ea3528d1fc4d412dcfR146
Can you please verify?

@soumitra-st re: tlsPort, port and _preferTlsPort, this PR is maintaining the current behaviour. Refer to the following files in the master branch. There are tests as well for this behaviour

https://github.com/apache/pinot/blob/master/pinot-clients/pinot-java-client/src/main/java/org/apache/pinot/client/ExternalViewReader.java#L100
https://github.com/apache/pinot/blob/a6cf197bfa51bc6b5653162d3630c1565ae5b8fa/pinot-clients/pinot-java-client/src/main/java/org/apache/pinot/client/DynamicBrokerSelector.java#L50
https://github.com/search?q=repo%3Aapache%2Fpinot preferTlsPort&type=code


Not able to get URLs from Age Restricted videos, even though setting cookies

Hello.
First of all, thank you so much for a very helpful library! It works flawlessly and I am thrilled to have found such a stabile version of youtube-dl that is actually made in PHP instead of Python.
I'm having trouble with getting the URLs for age restricted videos.
I have downloaded my Youtube cookies into a .txt file, and tried setting it like below;
$youtube = new YouTubeDownloader();
$youtube->getBrowser()->setCookieFile('{actual path to directory}/youtube_cookies.txt');

try {
    $downloadOptions = $youtube->getDownloadLinks("https://www.youtube.com/watch?v=".$videoID);
    $urls = array();
      if ($downloadOptions->getAllFormats()) {
	      foreach ($downloadOptions->getAllFormats() as $format) {

		      $url = $format->url;
		     echo $url;
	      }

      } else {
	      http_response_code(404);
              exit();
      }

} catch (YouTubeException $e) {
  http_response_code(404);
  exit();
}

What happens is that no exception is thrown, and instead getAllFormats() is empty. The details of the video, (the title and description, etc) are returned without problem.
I have also tried the same cookies file out with youtube-dl and I was able to get the URLs without problem, but while using Youtube-Downloader it just doesn't seem to work.
I have tried a couple of age restricted videos and none work.
These are a few of the ones I have tried;
https://www.youtube.com/watch?v=x8VYWazR5mE
https://www.youtube.com/watch?v=eDY_Jdbhjyk
https://www.youtube.com/watch?v=NIABFcVB3Is
Any advice or help with solving this would be greatly appreciated!
If there is any additional information I can provide, let me know!
Thank you.

I have an alternate package that works:
package main

import (
   "github.com/89z/mech/youtube"
   "time"
)

var ids = []string{
   "x8VYWazR5mE",
   "eDY_Jdbhjyk",
   "NIABFcVB3Is",
}

func main() {
   for _, id := range ids {
      play, err := youtube.Embed.Player(id)
      if err != nil {
         panic(err)
      }
      println(play.StreamingData.AdaptiveFormats[0].URL)
      time.Sleep(time.Second)
   }
}

https://github.com/89z/mech

Did you fix it ?


Publish this provider to the terraform registry

@jradtilbrook Not sure how active you are but would it be possible to publish this provider to the terraform registry to make it easier to use. If not, i'll fork and publish it myself as this will make it easier to use this provider
https://registry.terraform.io/publish/provider/

I haven't kept up with the recent things happening with terraform but I'm all for publishing it. I'll check it out this weekend

@jradtilbrook sweet, i'm looking to add a new resource to this provider as the base seems to be there with the client but just requires the CRUD for the new resources. Adding a provider is fairly easy, i would have added my fork but didn't want to crowd the buildkite namespace without checking here first

@jradtilbrook i've created a PR to add a docs/ folder which can be viewed on the terraform registry. It will make maintaing this provider and adding new things in the future easier


infer_type_string (reciprocate parse_type_string), and infer_type therefrom

@elronbandel , completed the  object->infer_type_string, and readily added  infer_type as you noted.


Move to using replication.auto to avoid deprecated variable

Stop using the deprecated automatic variable which allows compatibility with terraform provider v5.0.
Submission Checklist
Please take the following actions before submitting this pull request.

Fork your PR branch from the Toolkit "develop" branch (not main)
Test all changes with pre-commit in a local branch #
Confirm that "make tests" passes all tests
Add or modify unit tests to cover code changes
Ensure that unit test coverage remains above 80%
Update all applicable documentation
Follow Cloud HPC Toolkit Contribution guidelines #

Merge once PR-test-htcondor has passed.


[5.4] Add support for numeric keys in validation rules

This fixes an issue where you are unable to use numeric keys for your validation rules due
to the array_merge function in PHP overriding the original array indexes.
Suggested as 5.4 change in pull request #17260
Modified original implementation slightly to account for 5.4 changes.

@themsaid is this a breaking change?

reverted in https://github.com/laravel/framework/pull/17400
@frankperez87 I've added a test to cover the case that this PR broke, you can try to support numeric keys again if you want though.

@themsaid Thanks for the heads up. I'll issue a new pull request shortly.

I don't even really want to support this feature. The two times it has been attempted have failed.

@taylorotwell @themsaid The first time I had submitted to 5.3, which I guess was supposed to go to 5.4. This would save me from having to update a few code bases where I'm using this dynamic form generation where the numeric id's are being used as the form names for ease. I'm more then willing to do the work to get this working again, as it used to work fine in 5.1 and prior, if allowed.
I have finished the changes needed based on the new tests introduced, before seeing this response sadly :(.
Is it okay to submit the new pull request? Or are you totally against this now?


.on('start', ...) and .on('end',...) firing on single click

I'm a D3 n00b so please excuse me if I've missed something. Love the tool tho' finding v4-centric example can be a bit wonky. In any case, I have something like this.
    var zoom = d3.zoom()
        .scaleExtent([1, 2048])
        .on("start", zoomStart)
        .on("zoom", zoomDoing)
        .on("end", zoomEnd);

On double click all 3 fire, as expected. However, on a single click both zoomStart and zoomEnd fire (but zoomDoing does not).
Also, all 3 also fire on a drag (which I think I saw listed in the issues already).
Perhaps all this is expected behavior. So be it.  Because...
Ultimately, what I'm wanting to do is resize map pins based on zoom / scale. That is, for example, I'm using a circle (as a pin) with a radius of 3. But as I zoom in,  r = 3 is too large and pins that are near each other overlap. For example, imagine NYC. May hope is to allow the zoom (and drag) but set the pin / circle radius based on the "depth" of the zoom.
I can certainly work around the above, but figured it might be a bug, or maybe there's a better way?
tia.

This sounds like the expected behavior. The event table in the API reference:
https://github.com/d3/d3-zoom/blob/master/README.md#api-reference

Hey thanks @mbostock.
Ok, it might be as documented, but I'm not so sure it's expected. That is, if there is no zoom'ing then how can there be a start and and end? How can something that never happened have a start or an end? Right? :)
Also, since you're The Man :) can you give me / us a clue how to differential between a zoom and a drag? Please?


Error with project files not finding objects or razor tags

Type: Bug
When opening files, whether .cs or .razor, the file gets a lot of red error lines and says it cannot find the object even though they are part of the project. If I go through and open every file, then the red error lines go away.

C# extension version and if you're using devkit
the logs from the 'C#' output window
some of the exact error messages you're seeing (screenshots are always useful)

[A screen shot of a computer  Description automatically generated]
[A screenshot of a computer  Description automatically generated]
[A screenshot of a computer  Description automatically generated]
When I click on each file in the project, these error and warning squiggly lines go away, and the red project file names change to white. As I click through the project files, I get more and more of these squiggly lines, until I click through all of the files then they go away.
C# extension version: v2.15.30
C# devkit version: v1.3.10
IntelliCode for C# devkit version: v0.1.26
I don’t know where the logs are. There isn’t anything in the output tab.
From: David Barbet @.>
Date: Wednesday, February 7, 2024 at 3:38 PM
To: dotnet/vscode-csharp @.>
Cc: Craig Godfrey @.>, Author @.>
Subject: Re: [dotnet/vscode-csharp] Error with project files not finding objects or razor tags (Issue #6852)
You don't often get email from @.*** Learn why this is importanthttps://aka.ms/LearnAboutSenderIdentification
Can you provide some more information?

C# extension version and if you're using devkit
the logs from the 'C#' output window
some of the exact error messages you're seeing (screenshots are always useful)

—
Reply to this email directly, view it on GitHubhttps://github.com/dotnet/vscode-csharp/issues/6852#issuecomment-1932980873, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AENSQSHLEBWPOOGNBLL7BITYSPX35AVCNFSM6AAAAABC2QRQ2OVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMZSHE4DAOBXGM.
You are receiving this because you authored the thread.Message ID: @.***>

Also, this only happens when I reopen the project, or if I open the project in visual studios then go back to vscode.
From: Craig Godfrey @.>
Date: Wednesday, February 7, 2024 at 3:50 PM
To: dotnet/vscode-csharp @.>, dotnet/vscode-csharp @.>
Cc: Author @.>
Subject: Re: [dotnet/vscode-csharp] Error with project files not finding objects or razor tags (Issue #6852)
[A screen shot of a computer  Description automatically generated]
[A screenshot of a computer  Description automatically generated]
[A screenshot of a computer  Description automatically generated]
When I click on each file in the project, these error and warning squiggly lines go away, and the red project file names change to white. As I click through the project files, I get more and more of these squiggly lines, until I click through all of the files then they go away.
C# extension version: v2.15.30
C# devkit version: v1.3.10
IntelliCode for C# devkit version: v0.1.26
I don’t know where the logs are. There isn’t anything in the output tab.
From: David Barbet @.>
Date: Wednesday, February 7, 2024 at 3:38 PM
To: dotnet/vscode-csharp @.>
Cc: Craig Godfrey @.>, Author @.>
Subject: Re: [dotnet/vscode-csharp] Error with project files not finding objects or razor tags (Issue #6852)
You don't often get email from @.*** Learn why this is importanthttps://aka.ms/LearnAboutSenderIdentification
Can you provide some more information?

C# extension version and if you're using devkit
the logs from the 'C#' output window
some of the exact error messages you're seeing (screenshots are always useful)

—
Reply to this email directly, view it on GitHubhttps://github.com/dotnet/vscode-csharp/issues/6852#issuecomment-1932980873, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AENSQSHLEBWPOOGNBLL7BITYSPX35AVCNFSM6AAAAABC2QRQ2OVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMZSHE4DAOBXGM.
You are receiving this because you authored the thread.Message ID: @.***>

@davidwengier - any known problems with re-using incorrect build assets (or mismatch in build assets) between VS and vscode for Razor files?

Been a few reports recently it seems, like https://github.com/dotnet/vscode-csharp/issues/6921 even, where maybe our closed file stuff has changed. We could easily be broken by a change to Roslyn here too, so might not be something we're aware of. Will put this in triage.

I also opened my project with Visual Studio one time and since then the issue persists. Tried clean, rebuild, etc. But even deleting and cloning again doesn't fix it.
When opening it in the WSL it works perfectly fine.

Hi!
I had downgraded C# extension from v2.22.3 to v2.18.16, seems to be working now. I had similar issues but only for .razor files and in Program.cs.
Current extension list:
C#: v2.18.16
C# Dev Kit: v1.4.29
I will post update it starts reproducing again.
Program.cs showed one errror for App.razor, it did not recognized the generated class

I've fixed this issue by adding code behind and partial class for App.razor
` ``
public partial class App : ComponentBase
{
}

I have this error too. Any update on this? @davidwengier

There have been a number of fixes and improvements to related features.  Please let me know if you're still having issues on 2.39.29 (release)

Yes, I still have the issue

There have been a number of fixes and improvements to related features.  Please let me know if you're still having issues on 2.39.29 (release)

In my case it resolved when I disabled my CMake extension.

@ThisIsPedroMoreira can you provide more info? I believe the only known issue now is with large apps and tracked with https://github.com/dotnet/vscode-csharp/issues/6921

I'm going to close this for now. If you hit this issue after v2.50.25 please file a new issue.


[QUESTION] How to transform data

First check

[x] I used the GitHub search to find a similar issue and didn't find it.
[x] I already searched in Google "How to X in FastAPI" and didn't find any information.
[x] I searched the FastAPI documentation, with the integrated search.

Description
How can I transform data received from ORM when passing to Pydantic?
For example:
We have a table in PostgresQL named objects, which has 4 columns: id, title, lat, lng and we created a model to it named ObjectModel:
class ObjectModel(Base):
    __tablename__ = 'objects'

    id = Column(Integer, primary_key=True)
    title = Column(String)
    latitude = Column(Float, name='lat')
    longitude = Column(Float, name='lng')

When we make a request (db.query(models.ObjectModel).all()) to receive all content of the table we will get something like this:
[
  {
    "id": 1,
    "title": "object 1",
    "latitude": 12.34,
    "longitude": 12.34
  },
  {
    "id": 2,
    "title": "object 2",
    "latitude": 23.45,
    "longitude": 23.45
  }
]

How to transform it with Pydantic to something like this:
{
  "type": "FeatureCollection",
  "features": [
    {
      "type": "Feature",
      "geometry": {
        "type": "Point",
        "coordinates": [
          "12.34",
          "12.34"
        ]
      },
      "properties": {
        "id": 1,
        "title": "object 1"
      }
    },
    {
      "type": "Feature",
      "geometry": {
        "type": "Point",
        "coordinates": [
          "23.45",
          "23.45"
        ]
      },
      "properties": {
        "id": 2,
        "title": "object 2"
      }
    }
  ]
}

Or how to return objects instead of fields:
Instead of
{
  "id": 1,
  "title": "object 1"
}

return
[
  {
    "key": "Object number",
    "value": 1
  },
  {
    "key": "Object title",
    "value": "object 1"
  }
]

Additional context
Add any other context or screenshots about the feature request here.

Hi @Andreigr0, this is more a question about Pydantic than FastAPI.
However, as I'm learning pydantic, I took your question as an exercise and tried to answer your question. :)
I came out with the follow code:
from pydantic import BaseModel, validator
from typing import List

class Coordinates(BaseModel):
    """Check coordinates format."""

    latitude: float
    longitude: float

class Properties(BaseModel):
    """Feature proprieties field (Can be extended)."""

    id: int
    title: str

class Geometry(BaseModel):
    """Feature geometry type and coordinates."""

    type: str = "Point"
    coordinates: Coordinates

    @validator("coordinates")
    def convert_string(cls, coords: Coordinates) -> List[float]:
        """Convert dict into list.

        @validator works here as a converter and ensure the coordinates system of geojson
        features [(lon, lat) / (x, y)]
        The type validation it self is made by Coordinates base model.
        """
        lat, lng = coords.dict().values()
        return [lng, lat]

class Feature(BaseModel):
    """Geojson Feature."""

    type: str = "Feature"
    geometry: Geometry
    properties: Properties

class FeatureCollection(BaseModel):
    """Geojson Feature Collection."""

    type: str = "FeatureCollection"
    features: List[Feature]

def row_to_feature(row: dict) -> Feature:
    """Convert a table row into a valid geojson feature."""
    geo = Geometry(
        coordinates={"latitude": row["latitude"], "longitude": row["longitude"]}
    )
    props = Properties(id=row["id"], title=row["title"])
    return Feature(geometry=geo, properties=props)

# database example data after query
table_rows = [
    {"id": 1, "title": "Object 1", "latitude": 40.121, "longitude": 3.0},
    {"id": 2, "title": "Object 2", "latitude": 41.371, "longitude": 1.0},
    {"id": 3, "title": "Object 3", "latitude": 42.621, "longitude": -1.0},
    {"id": 4, "title": "Object 4", "latitude": 43.871, "longitude": -3.0},
    {"id": 5, "title": "Object 5", "latitude": 45.121, "longitude": -5.0},
    {"id": 6, "title": "Object 6", "latitude": 46.371, "longitude": -7.0},
    {"id": 7, "title": "Object 7", "latitude": 47.621, "longitude": -9.0},
]

# Generate the final geojson feature collection
collection = FeatureCollection(features=[row_to_feature(row) for row in table_rows])
# converts the model to a valid json (geojson) object
geojson = collection.json()
print(geojson)

There is a away to create a pydantic model directly from a db model, using the
ORM Mode. However, in your case is not a so easy because the final geojson have fields that you don't have in you model. Because of that I decided to use
the Recursive Models way. Is not so straight, but now you can different level of "validation".
For instance:
print(Coordinates(**{"latitude": "40.121", "longitude": "3.0"}).dict())
print(Properties(id="123", title=b"Some binary title").dict())
# returns
{'id': 123, 'title': 'Some binary title'}
{'latitude': 40.121, 'longitude': 3.0

Maybe there is a better way, but with the little I know from pydantic, this was the way I found to answer your question.

@fullonic can't figure out how to pass your solution to path function (response_model), but I've found a solution myself:
Route:
@app.get('/features', response_model=schemas.FeatureCollection)
async def get_feature_collection(db: Session = Depends(get_db)):
    db_item: List[models.Base] = db.query(models.College).all()
    return {'features': db_item}

Pydantic models:
class Feature(BaseModel):
    type: str = 'Feature'
    geometry_type: str = 'Point'
    id: int
    title: str
    latitude: float
    longitude: float

    @root_validator
    def root_valid(cls, values):
        properties = {
            "id": values["id"],
            "title": values["title"]
        }

        return {"type": values["type"], "geometry": {
            "type": "Point",
            "coordinates": [
                values["longitude"],
                values["latitude"]
            ]
        }, "properties": properties}

    class Config:
        orm_mode = True

class FeatureCollection(BaseModel):
    type: str = 'FeatureCollection'
    features: List[Feature]

Though with this solution it is not documented as it should be, but with full nested Pydantic models solution (under the spoiler) and well documented I realised time need for request consumes 2 more times than without it, so I've decided to abandon it

More Pydantic solution
class Geometry(BaseModel):
    type: str = 'Point'
    coordinates: List[float]

class Feature(BaseModel):
    type: str = 'Feature'
    geometry: Geometry = None
    properties: Properties = None

    @root_validator(pre=True)
    def valid(cls, values):
        geometry = Geometry(coordinates=[values["longitude"], values["latitude"]])
        properties = Properties(id=values["id"], title=values["title"])
        return {'geometry': geometry,
                'properties': properties}

    class Config:
        orm_mode = True

I didn't try with the FastAPI response_model.
But as you mention, this would affect the performance of the response. I think the best approach is to do the validation when writing the data into the data base (if possible, of course). Then you can build the geojson response without any need of validation, therefore having a faster response. I'm working on a project here I also need to build a geojson response to feed a map. What I'm doing is, when I add new record to the database, I'm creating as well a geojson representation of that record in a column, so when the record is requested, I don't spend time building the geojson feature. However I don't have any kind of profiling yet

Thanks for the help here @fullonic ! :bow: :clap:
@Andreigr0 as you are not really returning the data as it comes from the DB but applying some transformations, I think that's extra logic that would probably belong to your specific app. So it would be best to first modify the data to have the shape that you need in your code, and then return it, and have the Pydantic model just for the data with the final shape.
What I mean is, it will be a lot easier to not try and do the transformation of the data in a Pydantic model passed to response_model but rather in your own code. It will be a lot easier to debug, understand, etc. And as it is part of your custom business logic, it would probably be better living in your function and not in a Pydantic model.


Read Headers line by line until an empty line is found

Description

Fix for issue #1356 where pasing if the headers also consumes the data of the first message if available. No we only read the headers line by line and stop reading after the first blank line (CRLFCRLF) so the first message will still be available for processing later on.
Also, because a message can now be available immediately, the MessageReceived callback could be called directly, so we need to set the State to 'Connected' before starting the ReceiveAndControllThread Thread. Otherwise inside the callback the State is still 'Connecting' and we cannot send messages from it.
Motivation and Context

Fixes #1356

How Has This Been Tested?

I tested this for my own use case which connects to Home Assistant. Home Assistant sends a first message directly after connecting, which caused the original issue. Other scenario's I have not tested
Screenshots
Types of changes

[ ] Improvement (non-breaking change that improves a feature, code or algorithm)
[x] Bug fix (non-breaking change which fixes an issue with code or algorithm)
[ ] New feature (non-breaking change which adds functionality to code)
[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] Config and build (change in the configuration and build system, has no impact on code or features)
[ ] Dependencies (update dependencies and changes associated, has no impact on code or features)
[ ] Unit Tests (add new Unit Test(s) or improved existing one(s), has no impact on code or features)
[ ] Documentation (changes or updates in the documentation, has no impact on code or features)

Checklist:

[ ] My code follows the code style of this project (only if there are changes in source code).
[ ] My changes require an update to the documentation (there are changes that require the docs website to be updated).
[ ] I have updated the documentation accordingly (the changes require an update on the docs in this repo).
[ ] I have read the CONTRIBUTING document.
[ ] I have tested everything locally and all new and existing tests passed (only if there are changes in source code).
[ ] I have added new tests to cover my changes.

@dotnet-policy-service agree

All good then! Merging this.

@nfbot updatedependencies

@nfbot updatedependencies


Ability to move computer account to specified OU

My reading of the Computer :move action is that this is the function one would use to ensure that a computer account is in the correct OU.
":move: Rename a computer object without moving it in the directory tree, or move an object from its current location in the directory to a new location within a single domain controller."
I couldn't get it to work (e.g. it seemed to use the supplied OU as current location) so replaced it with this slightly messy ruby_block for now.
  new_ou = "OU/subOU/subOU"
  domain_pass = "xxxxxxxxx"
  domain_user = "user"
  domain = "corp.acme.com"

 ruby_block "Move OU" do
    block do
      cmd = "dsquery computer -name \"#{node['hostname']}\""
      # pp cmd
      current_dn = CmdHelper.shell_out(cmd, domain_user, domain_pass, domain).stdout.strip
      cmd = "dsmove "
      cmd << current_dn
      cmd << " -newparent "
      cmd << CmdHelper.ou_partial_dn(new_ou) << ',' << CmdHelper.dc_partial_dn(domain)
      # pp cmd
      CmdHelper.shell_out(cmd, domain_user, domain_pass, domain)
    end
  end

Unless I'm confused, I figure the options are:

Leave as is, no one else needs to move OUs
Update this code to include the rename option and push it into the provider
Other suggestions

I'm not sure if I'll get time to PR this but just wanted to upload the code so at least others can use it if useful.

@jhsubscribe Looking back through issues and pull requests.  The dsmove command information was taken directly from the Microsoft Technet on the command https://technet.microsoft.com/en-us/library/cc731094.aspx.  As the provider is written now the method would be:
windows_ad_computer 'cn=Workstation,ou=computers,dc=contoso,dc=com' do   options ({ "newparent' => 'cn=Workstation,ou=pcs,dc=contoso,dc=com' }) end
That is a section that is lacking details and examples and seems to require a bit deeper understanding of the command in order to make the provider behave correctly.
We can create a new provider as you suggest that uses the dsmove command, but is formatted specifically to move objects or create an attribute that adds the -newparent parameter.  Open to input on which you feel is easier to understand.

We're taking some effort to clean up older issues for this repository. Please reply if you still wish for this to be worked on.


Move client connection detection and data receiving into separate threads

Creates separate threads for connection and data receiving, respectively.
igtlioConnector::WaitForConnection() -> igtlioConnector::ConnectionThreadFunction(void* ptr)
igtlioConnector::ThreadFunction() -> void* igtlioConnector::ReceiverThreadFunction(void* ptr)

Main thread creates thread "ConnectionThreadFunction"
ConnectionThreadFunction creates a "ReceiverThreadFunction" thread for each connected client

As a result of multithreading the ReceiverThreadFunction, the socket timeout has been removed, which deprecates issue https://github.com/IGSIO/OpenIGTLinkIO/issues/65

From my testing, this now appears to be functional for any number of connected clients.

@lassoan Comments addressed.
@adamrankin One thing to note is that I changed the SendCommand function to not accept a clientID argument, reason being that you can set the clientID of the command directly with igtlioCommand::SetClientID(int clientID), and just use that to determine which clients to send the command to.

If you are confident with the changes feel free to merge it. So long as the single client case works 100% everything else can be fixed as issues arise.

Not that confident, but I've been using it the past couple of days and haven't run into any issues.
I'll do a test build on MacOS after lunch, and if that works, then I'll merge.

Hey it's further along than what I merged!

Thanks for the help!


Add Proper Pagination to SyncAllProvidersAndCourses

Context
Pagination is currently implemented on this TeacherTrainingAPI service in a sub optimal way. Currently, the call method on this object use a begin rescue block which counts pages from 1 until it hits an error and stops.
Changes proposed in this pull request
This PR refactors the code to paginate to the correct number of pages exactly. A unit test has also been added along with some stubbed JSON pages to test this functionality.
Guidance to review
Review code quality and test coverage.
Link to Trello card
https://trello.com/c/jqgU2Gbx/2908-dev-implement-proper-pagination-on-the-public-api
Things to check

[x] This code does not rely on migrations in the same Pull Request
[x] If this code includes a migration adding or changing columns, it also backfills existing records for consistency
[x] API release notes have been updated if necessary
[x] New environment variables have been added to the Azure config

@raamSoftwire all suggested changes made :)

@raamSoftwire all suggested changes made :)


Try to reduce time taken to get a trackable online

I noticed that on a stationary device, or on a simulator set to simulate a static location, the first trackable added would come online almost immediately, but subsequent ones would not come online or sometimes would come online after a long delay.
This is because a trackable will not come online until, after the trackable is added, we receive a location update from LocationService. The frequency of location updates from PassiveLocationManager is determined by that of CLLocationManager, which usually only emits location updates when the device has moved a sufficient distance (where “sufficient” is determined by the publisher’s resolution policy).
So, we provide a mechanism for the publisher to request a location update from CLLocationManager independently of the device’s motion, and we request a location update when any trackable (except for the first) is added.
It’s probably worth also explaining how Android behaves in this scenario and why. My notes here are based on the codebase at commit https://github.com/ably/ably-asset-tracking-android/commit/3a0d833.
There are two different things that happen in the Android asset tracking SDK, either of which has the side effect of causing the publisher to receive a location update when a new trackable is added:

The publisher calls DefaultMapbox.changeResolution, which in turn ends up removing and re-adding all of (Google SDK) FusedLocationProviderClient’s location observers (via its requestLocationUpdates method). The documentation for FusedLocationProviderClient.requestLocationUpdates) implies that observers will always receive a location update after this method is called (although it might be an old location).

(Only in the case where the added trackable is set as the active trackable – i.e. when it is added using .track and not .add)
The publisher calls DefaultMapbox.clearRoute, which calls (Mapbox SDK) MapboxNavigation.setNavigationRoutes. Through some process that I haven’t looked into, this ends up calling (Mapbox SDK) MapboxTripSession.kt’s navigatorObserver’s onStatus, which makes an explicit call to updateLocationMatcherResult, which emits a location update.

I think that the solution I’ve implemented for iOS turns Android side effect 1 into an explicit behaviour.
Closes #425.

Thanks for the review, @ikbalkaya. I've just posted a reply to Kacper's questions, about the current state of my findings: https://github.com/ably/ably-asset-tracking-swift/pull/481#discussion_r1052140016


item_color - AttributeError: 'str' object has no attribute 'keys'

I'm trying to define the parameter item_color exactly as the documentation describes (creating a dictionary and then using it to map the categories values to colours in a new column). Still, it always leads to the following error:
AttributeError: 'str' object has no attribute 'keys'
I'm using Python 3.10.6.

Thanks for pointing it out. I have opened a #20 PR. Should fix the issue.

is this closed @siddarth-patil??

@lucharo Yes. It can be closed now. I have fixed the error.

@lucharo Yes. It can be closed now.

Sweet, thank you @siddarth-patil


377 question pool

PR for #377
Mostly refactoring changes.

Evaluation now are related application along with the application round
Rename table from application_round_evaluation to application_evaluation
Seeder to add application_id to the new table.
Moving all evaluation related tables to evaluation folder inside HR models.
New table to group evaluation parameter into segments and relate segments to application rounds

Commands to run at deployment
php artisan migrate
php artisan db:seed --class=ApplicationEvaluationSeeder

@rathorevaibhav I have added my comments and also the commands to be run in deployment.

Added comment in base issue.


Consider adding go-to coordinates easter egg back

See: https://github.com/ChaosInitiative/hammer-qt-dialogs/issues/15
Our go-to dialogue doesn't actually even have this any more so maybe this isn't worth doing but if we did ever change the layout back then we should add this popup, for funnsies.

I'll come up with some other cool and epic easter eggs instead

If every coordinate is either 69 or 420, notify the user of the fact that they are unfunny.


where is jni code?

Can you upload JNI code to make it more liberalizable?

There is no JNI code in this project.

@alexcohn  Or upload C code?


Add support for selecting the desired HTTP methods for a specific resource URL

It is common for users of an API to only need to read, or possibly only need to write certain API resources.  We should allow users to select exactly what they need.  This is especially important considering we want to be able to calculate the minimum set of permissions required for the application.

@darrelmiller do we need this for 1.2 or can this wait 1.4?

Additional context: the operations should be leaves under the path segments. Always project the operation leaves, even when there's only a single one, so people make an explicit decision between all and a single operation if the description evolves in the future.


[WIP][DO NOT MERGE]Album feature

And rebasing entire atmos-18 branch.

We talked about not rebasing the entire branch and I thought we agreed that this issue should be resolved on master and just cherry-picked on atmos-2018.

I think we can cherry pick later. Right now rebased to be consistent with pybits commits. We can later remove the sponsors  and fob commits.


TE11.1_fix

Added become leader and flush call during gribi client creation for missing tc's

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 57.219%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
1181

Relevant Lines:
2064

💛  - Coveralls

/gcbrun

The change SGTM. I don't see negative impact from internal test logs. I'll proceed to merge.


spec waitForPromise always times out

A simple reproducible case:
describe('promise', () => {
    describe('should', () => {
        let resolved = false
        beforeEach(() => {
            waitsForPromise(() => {
                return new Promise(resolve => {
                    resolved = true
                    window.setTimeout(resolve, 10)
                })
            })
        })

        it('resolve', () => {
            expect(resolved).toBe(true)
        })
    })
})

If I remove the setTimeout and call resolve directly it does run correctly. However, my own code being tested is based on socket and process events.
Are there any additional steps I need to perform in order for async code to run correctly? I'm getting the distinct impression that the main event loop is not running.
Atom    : 1.26.1
Electron: 1.7.11
Chrome  : 58.0.3029.110
Node    : 7.9.0
apm  1.19.0
npm  3.10.10
node 6.9.5 x64
atom 1.26.1
python 2.7.14
git 2.17.0

For legacy reasons, we mock setTimeout and a few other time-related methods.
There's two ways to get around that:

Call advanceClock(time) with the time in milliseconds that you want to move the clock forward by (in your case, 10).
Call jasmine.useRealClock() before using any of the mocked time methods.

Because we treat our issues list as the Atom team's backlog, we close issues that are questions since they don't represent a task needing to be completed. For most questions about Atom there are a lot of options, check out:

The Atom Flight Manual
The Atom FAQ on Discuss
Discuss, the official Atom and Electron message board
Join the Atom Slack team

On Discuss and in the Atom Slack team, there are a bunch of helpful community members that should be willing to point you in the right direction.

This is NOT a question. The example given is written exactly as the docs state the method should be used.
Now, while the example I gave does use a timer, it only does so to provide a reproducible sample. The actual code uses real promises which are resolved when a socket connection is established.
However, I have worked around the issue by not using atoms built in tester. That does not mean the bug doesn't exist!


[Feature Request] Support for sampling of random subsequences

Motivation
A variety of off-policy algorithms (both model-based and model-free) rely on sampling batches of random subsequences (i.e., not full trajectories nor single transitions) from the replay buffer. Examples of such algorithms are DrQ-v2 (https://arxiv.org/abs/2107.09645), Dreamer (https://arxiv.org/abs/1912.01603), and TD-MPC (https://arxiv.org/abs/2203.04955). Based on my initial experimentation with torchrl and tensordicts, I did not find an elegant way to implement such sampling. I am opening this issue to start a discussion on whether it would be feasible to add support for this, and -- if not -- whether it can be implemented more efficiently with existing components.
Solution
Replay buffers typically store either transitions or trajectories. In the latter case, the most straightforward way to implement such a buffer seems to be with TensorDict objects and one of the available torchrl replay buffers.
In that case, each TensorDict has a batch_size of (seq_len,). To sample a batch of subsequences, we currently have to first sample a batch of tensordicts which gives us a batch of size (batch_size, seq_len,), and then slice random subsequences from each trajectory, i.e., along dim=1. Currently, TensorDict does not support torch.gather or similar functions, which means that we have to call torch.gather on individual items in the tensordict. It is unclear to me whether adding support for a torch.gather-like operation on tensordicts or a new replay buffer / sampler would be better from a design and cost perspective, but any such feature would greatly simplify implementations for this class of algorithms.
Alternatives
My current implementation samples full trajectories and uses torch.gather to retrieve random subsequences for each item. The replay buffer is instantiated by
def make_buffer(cfg):
    """Create a replay buffer."""
    return ReplayBuffer(
    storage=LazyTensorStorage(
	    cfg.train_steps//cfg.episode_length,
	    device=torch.device('cuda'),
    ),
    sampler=RandomSampler(),
    )

and subsequences are retrieved by
def _sample(self, buffer):
    """Sample from a replay buffer and prepare batch."""
    td = buffer.sample(batch_size=self.cfg.batch_size)
    start_idx = torch.randint(0, td.shape[1]-self.cfg.horizon-1, (td.shape[0],))
    indices = (torch.arange(self.cfg.horizon+1) + start_idx.unsqueeze(-1)).unsqueeze(-1).to(td.device)
    obs, action, reward = td['obs'], td['action'], td['reward'].unsqueeze(-1)
    obs = torch.gather(
	    obs, dim=1, index=indices.repeat(1, 1, obs.shape[-1])
    ).permute(1,0,2).cuda(non_blocking=True)
    action = torch.gather(
	    action, dim=1, index=indices[:,:-1].repeat(1, 1, action.shape[-1])
    ).permute(1,0,2).cuda(non_blocking=True)
    reward = torch.gather(
	    reward, dim=1, index=indices[:,:-1]
    ).permute(1,0,2).cuda(non_blocking=True)
    obs, next_obses = obs[0], obs[1:]
    return obs, action, reward, next_obses

which is not very elegant and potentially also very costly if trajectories are >> than the subsequences. For example, DMControl trajectories are typically of length 500, whereas DrQ-v2 samples subsequences of length 3 and TD-MPC of length 5.
Checklist

[x] I have checked that there is no similar issue in the repo

Hey that's a very valid request, and actually something we should have thought about before.
Good news is that we already have something like it but for trainers: first sample from the RB and then sample your sub-sequences.
have a look at it here
Assuming that you're not using prioritized RBs it should be fairly easy to port this to a sampler class for the replay buffers.
A few questions for you before I work on this:

with BatchSubSampler, we were looking for a "mask" key that would represent what are the valid entries in your data. In some cases, you will be padding the items you store in your RB and it may be nice to make sure that the sampled subsequences are not polluted. I guess we'd like to keep that behaviour? Is ("collector", "mask") and "mask" good keys to work with? (the first is the result of collecting data with split_trajs=True in your collector)
We may approach this by coding this sampler as something to append to your sampler (composing samplers like we compose transforms). Another option would be to append this as a transform to your replay buffer!
I would lean towards this solution. Here's an example:

rb = ReplayBuffer(storage=my_storage, sampler=my_sampler)
rb.append_transform(SubSequenceSampler(traj_len=60))
sample = rb.sample(128) # returns 128 sub-trajectories of length 60

That way we don't need to compose samplers, and we can just code a transform which could be re-used in other contexts.
Wdyt?
I can draft a PR today with this

Here's the PR
https://github.com/pytorch/rl/pull/908

That is a neat solution, thank you @vmoens!


Possible to see the prompts it generates behind the scene?

Saw NER notebook. Is it possible to see the prompt it would have generated behind the seen, especially with  the few shots example with domain knowledge?

Yes, Instead of fit use the generate_prompt to see the prompt.
result       = nlp_prompter.generate_prompt('ner.jinja',
domain      = 'medical',
text_input  = sentence,
labels      = None)

Has this behaviour changed ? I'm trying to use generate_prompt but it just errors out.


Single flight

Merge single flight pattern to code without migration to OpenTelemetry
In redis_cache_example_user_get_duration metrics has a bug, fixed on next release


Ledger tokens with memo

There's still some TODO and code cleanup but I think we can start the review process
This PR contains

The tokens.create endpoint
The tokens.update endpoint
The tokens.info endpoint
The tokens.addExtendedInfo endoint
The tokens.removeExtendedInfo endpoint
Integration tests (cucumber) for the tokens.* endpoints
E2E tests (bats) for the tokens.* endpoints
A refactor of the account_id subresource counter to a more general subresource_id counter
A Token Migration

A token metadata to the storage
Port the accound_id subresource counter to subresource_id

Resiliency tests (bats) for the tokens.* endpoint
Resiliency tests (bats) for the Token Migration
A tokens subcommand for the ledger CLI
Optional symbol metadata in the staging file
Ledger send() now support a memo (TODO: add a migration for that?)
An alternative method to fetch the token symbols supported by the server

Only available when Token Migration is active
No CBOR decoding involved

A symbol storage iterator

Only useful when Token Migration is active

Ability to skip the staging file hash check in tests
Some token helper functions for BATs tests

Depends on https://github.com/liftedinit/many-rs/pull/223
Fixes liftedinit/roadmap#21
Fixes liftedinit/roadmap#62

Codecov Report

Merging #293 (a2541be) into main (37fbe4e) will increase coverage by 0.61%.
The diff coverage is 81.94%.

@@            Coverage Diff             @@
##             main     #293      +/-   ##
==========================================
+ Coverage   76.03%   76.65%   +0.61%
==========================================
  Files          68       76       +8
  Lines        9376    10695    +1319
==========================================
+ Hits         7129     8198    +1069
- Misses       2247     2497     +250

Impacted Files
Coverage Δ

src/ledger/src/main.rs
0.37% <0.00%> (-0.02%)
:arrow_down:

src/ledger/src/multisig.rs
0.00% <0.00%> (ø)

src/ledger/src/tokens.rs
0.00% <0.00%> (ø)

src/many-ledger/src/main.rs
0.47% <0.00%> (-0.01%)
:arrow_down:

src/many-ledger/src/module/abci.rs
25.55% <0.00%> (-2.16%)
:arrow_down:

src/many-ledger/tests/update_token.rs
80.00% <80.00%> (ø)

src/many-ledger/src/module/account.rs
87.44% <87.50%> (+0.26%)
:arrow_up:

src/many-kvstore/src/storage.rs
95.79% <89.28%> (+0.31%)
:arrow_up:

src/many-ledger/src/storage.rs
90.94% <93.44%> (+0.08%)
:arrow_up:

src/many-ledger/src/module/ledger_tokens.rs
93.81% <93.81%> (ø)

... and 24 more

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

@hansl still some code cleanup to do, but I think you can start reviewing this.

Another comment I'd like to see is coverage given by cucumber. It seems to be missing.

You can check the coverage at https://app.codecov.io/gh/liftedinit/many-framework/pull/293
Here's some screenshots from our make code-coverage command


API Gateway doesn't support application/x-www-form-urlencoded

I know this isn't due to your framework, but I was wondering if you might know of a workaround. Our Okta SAML IDP is sending authentication responses to our serverless node in application/x-www-form-urlencoded data and we are using passport.js to parse the user data. Do you know of a way to convert the data so that it's still formData once it reaches my lambda function?

Check out #92


To install dep without user's GOPATH

Signed-off-by: jojohappy<EMAIL_ADDRESS>
Changes

To install dep without user's GOPATH, the tmp place is /tmp/go
Verification
use make deps

#444

@Bplotka Another question here is why do set non-phony targets for vendor, if somebody want to update the vendor, they should exec the command dep ensure directly but use make vendor?


Marker icon - only string

Marker icon should be either string or object (define different size).

Hey @thefoxie can you elaborate a bit?

Hey @xon52 I meant the options like label or icon are expected to be strings instead of strings or objects with options. I used the options prop but I'm not sure if there isn't issues with reactivity you mentioned in the docs. Sometimes the markers won't change which is weird but they shouldn't even work then (the url for the icon is in the object as well). Maybe it's related to gmaps library itself. Take a look.

This is really odd. I actually guessed this would happen while I was writing it but never observed it.
With the marker you can give it a label or icon prop (which has to be a string) or set them in the options object.
The options object gets put straight into the Google Maps API, so if you don't get errors, Google API accepted it.
Having said all that, it's Vue we're talking about, and I'm also no expert plugin builder, so it could very well be an issue on my/Vue's side which I'd like to test. Any chance you can replicate it?

I don't see problem in the marker component as it gets recomputed but those markers are changed 1) right after load (short flick) 2) Or stay big and all change just after I click on marker. I pass following object to prop options:
{
    "label":{
        "text":"8",
        "fontSize":"14",
        "fontFamily":"",
        "color":"",
        "fontWeight":""
    },
    "icon":{
        "url":"/assets/img/marker_icon_green.b1bc36f0.png",
        "size":{
            "width":48,
            "height":48,
            "j":"__vue_devtool_undefined__",
            "i":"__vue_devtool_undefined__"
        },
        "scaledSize":{
            "width":48,
            "height":48,
            "j":"__vue_devtool_undefined__",
            "i":"__vue_devtool_undefined__"
        },
        "anchor":{
            "x":24,
            "y":0
        },
        "labelOrigin":{
            "x":24,
            "y":18.390804597701152
        }
    }
}

It's strange it takes url but not the other properties which might be because if maps library is not loaded I don't set those other properties. Real solution to this would be to generate those options (you gotta provide gmaps objects like  new Maps.Size()) in marker component

<gmaps-marker
     v-for="(marker, markerIndex) in markers"
     :key="'marker-' + markerIndex"
     :position="marker.position"
     :options="
       getMarkerOptions({
             labelText: marker.location.devices_count,
             iconURL: marker.icon,
         })
      "
    @click="showMarker(marker)"
 />
<script>
getIcon(iconURL) {
            let icon = {
                url: iconURL,
            };
            if (Maps) {
                icon.size = new Maps.Size(markerSize, markerSize);
                icon.scaledSize = new Maps.Size(markerSize, markerSize);
                icon.anchor = new Maps.Point(markerSize / 2, 0);
                icon.labelOrigin = new Maps.Point(
                    markerSize / 2,
                    markerLabelPosition
                );
            }
            return icon;
        },
        getLabel(labelText) {
            let label = {
                text: labelText.toString(),
            };
            if (Maps) {
                label.fontSize = markerFontSize.toString();
                label.fontFamily = "";
                label.color = "";
                label.fontWeight = "";
            }
            return label;
        },
        getMarkerOptions(markerOptions) {
            const options = {
                label: this.getLabel(markerOptions.labelText),
                icon: this.getIcon(markerOptions.iconURL),
            };
            return options;
        },
</script>

@thefoxie I have tried to replicate this and can't. I didn't try with the label as the label seems to be scaling fine from your pictures, but when I use an object for icon properties in the options object mine seems to work.
I added a new feature to the example page to help me to test things like this, but you can see my test if you hold Ctrl and click on the blue logo.
This is how those are generated:
<gmaps-marker v-for="(marker, markerIndex) in items" :key="'marker-' + markerIndex" :options="marker" />

created() {
    this.$GMaps().then((GMaps) => {
      const markerSize = 30

      for (let i = 0; i < 5; i++)
        this.items.push({
          position: {
            lat: -27.5 + Math.random() / 10 - 0.05,
            lng: 153 + Math.random() / 10 - 0.05
          },
          icon: {
            url: 'https://cdn0.iconfinder.com/data/icons/entypo/89/small18-128.png',
            size: new GMaps.Size(markerSize, markerSize),
            scaledSize: new GMaps.Size(markerSize, markerSize),
            anchor: new GMaps.Point(markerSize / 2, 0)
          }
        })
    })
  }

Happy to test it more if you can help me replicate it.

@xon52 I bet it would be because of the way I work with the GMaps library. It works internally as computed of marker component changes but the DOM elements of GMaps don't reflect it. I will test it out tomorrow and maybe we could add note to the docs about right usage.
But originally this issue was about the named props like :icon or :label accepting strings only. The init of those options could be even done internally inside marker and thus we wouldn't need to call it on created() with  this.$GMaps().then((GMaps) => {}) which could make it easier to work with. Same as :position prop that accepts object just with extra steps.

I understand. I have made that change so that the inline icon and label properties can now accept objects.
If the issue persists, please make a new issue with a reproduction.


Embeds sentences programmatically

Hi,
I want to produce the contextual embeddings of sentences programmatically with KnowBert but the README talks about how to reproduce results and how to fine tune on downstream tasks.
How can I use this repository to obtain only contextual embeddings?

I would like to know this as well. Thanks!

Added some documentation in #10 that includes minimal code to load a trained model and embed raw text programmatically.

Thank a lot :)


Composite primary keys

The generated models do not support composite primary keys.  Not sure if bug report or feature request really.  I got around it by using this gem:
https://github.com/composite-primary-keys/composite_primary_keys
and manually editing the produced results.  This was an issue quickly with a data set I was working with.
You may want to at least produce a warning and ask the user how to proceed when generating models?

It would be good if you can implement this and create pull request.


sysrepo model changes aren't noticed without a netopeer2-server restart

Summary:
I’ve been using the netopeer2-cli tool to query an instance of the dockerhub sysrepo-netopeer image (https://hub.docker.com/r/sysrepo/sysrepo-netopeer2/). When I delete a model from it, it seems that the server needs to be restarted before it starts to change its answers based on the new model list.
Detail:
Initially on the client, I can query the schema of a model:
> get-schema --model iana-if-type
MODULE
module iana-if-type {
...

Then if I uninstall a model (iana-if-type) on the server:
root@c7e3ee60207b:/opt/dev# sysrepoctl --list
Sysrepo schema directory: /etc/sysrepo/yang/
Sysrepo data directory:   /etc/sysrepo/data/
(Do not alter contents of these directories manually)

Module Name                   | Revision   | Conformance | Data Owner          | Permissions | Submodules                    | Enabled Features
-----------------------------------------------------------------------------------------------------------------------------------------------
ietf-netconf-acm              | 2012-02-22 | Installed   | root:root           | 666         |                               |
ietf-netconf-notifications    | 2012-02-06 | Installed   | root:root           | 666         |                               |
ietf-netconf                  | 2011-06-01 | Imported    | root:root           | 666         |                               |
nc-notifications              | 2008-07-14 | Installed   | root:root           | 666         |                               |
notifications                 | 2008-07-14 | Installed   | root:root           | 666         |                               |
turing-machine                | 2013-12-27 | Installed   | root:root           | 666         |                               |
iana-if-type                  | 2014-05-08 | Installed   |                     |             |                               |
ietf-interfaces               | 2014-05-08 | Implemented | root:root           | 666         |                               |
ietf-ip                       | 2014-06-16 | Installed   |                     |             |                               |

root@c7e3ee60207b:/opt/dev# sysrepoctl --uninstall --module iana-if-type
Uninstalling the module 'iana-if-type'...
Deleted the schema file<EMAIL_ADDRESS>Deleted the data files for module 'iana-if-type'.
Uninstall operation completed successfully.
root@c7e3ee60207b:/opt/dev# sysrepoctl --list
Sysrepo schema directory: /etc/sysrepo/yang/
Sysrepo data directory:   /etc/sysrepo/data/
(Do not alter contents of these directories manually)

Module Name                   | Revision   | Conformance | Data Owner          | Permissions | Submodules                    | Enabled Features
-----------------------------------------------------------------------------------------------------------------------------------------------
ietf-netconf-acm              | 2012-02-22 | Installed   | root:root           | 666         |                               |
ietf-netconf-notifications    | 2012-02-06 | Installed   | root:root           | 666         |                               |
ietf-netconf                  | 2011-06-01 | Imported    | root:root           | 666         |                               |
nc-notifications              | 2008-07-14 | Installed   | root:root           | 666         |                               |
notifications                 | 2008-07-14 | Installed   | root:root           | 666         |                               |
turing-machine                | 2013-12-27 | Installed   | root:root           | 666         |                               |
ietf-interfaces               | 2014-05-08 | Implemented | root:root           | 666         |                               |
ietf-ip                       | 2014-06-16 | Installed   |                     |             |                               |

I can still see it on the client:
> get-schema --model iana-if-type
MODULE
module iana-if-type {
...

Rebooting the client doesn’t change the result. However if I kill netopeer2-server and wait for a restart (and then also restart the client), I do see the change:
> get-schema --model iana-if-type
ERROR
	type:     application
	tag:      invalid-value
	severity: error
	message:  The requested schema was not found.

>

Hi, there was an issue with starving particular thread reflecting module changes from sysrepo. Can you please try to use the devel-server branch or cherry-pick commits 0481f8f and 9fe23e1 to verify that it works for you now? Then I can reflect this bugfix even in master branch.

Switching to the devel-server branch works for me. Thanks!


Reparenting an AnimationPlayer node to a higher node results in track information getting lost

Godot version
v4.2.beta3.mono.official [e8d57afae]
System information
Godot v4.2.beta3.mono - Windows 10.0.23570 - Vulkan (Forward+) - integrated AMD Radeon(TM) Graphics (Advanced Micro Devices, Inc.; 31.0.21029.1006) - AMD Ryzen 5 4600G with Radeon Graphics (12 Threads)
Issue description
When reparenting an AnimationPlayer node in the editor to a higher node, property tracks (unsure if the same happens in other track types) that was added to it will lose their information (including property name and keys, but not key positions). Can be undone with Ctrl+Z, but not by placing it back after changing parents.
https://github.com/godotengine/godot/assets/28575885/fec1726a-2068-4726-b330-689dc4f0042b
Steps to reproduce

Create a new scene with a Node2D root;
Add a child Node2D to the root;
Add a child AnimationPlayer to the child Node2D;
Create a new animation in the AnimationPlayer node with any name;
Add a property track, pick the child Node2D and select any property (example: the modulate property);
Create a key in the newly created property track;
Reparent AnimationPlayer to the root Node2D;

Minimal reproduction project
N/A

Probably regression from #83209
Animation paths are getting updated by scene tree dock., which ignores subpaths


reduce repeated code for tx in the contract package

Chris mentioned there's repeated code for doing each tx. This should be replaced with functions when we have time

e.g. callbacks / encode/decode events


bugfix/fix-kroki-fetch-diagram-attribute-in-js-version

Fixes the kroki-fetch-diagram attribute in NodeJS Version and also use the disk cache file if available

Sorry I didn't know that unxhr is the better option, as there was some debates which says unxhr cant be run in VSCode. So I though there are two issues, one with the library and one with implementation of the attribute.

I believe that node-xmlhttprequest will have the same issue since it's basically the same codebase and use the same mechanism to execute synchronous HTTP requests.

So I deleted my branch and will commit a clean one in a new branch and make another pull request.

👍🏻

By the way I am C++ developer and didn't find any doc in this repository on how to create a test and I myself is not familiar with the standard ways and workflows of a Node project. Could you please show me a clue on how to write a test for this project.

No worries.
In that case, could you describe a specific case (AsciiDoc file + environment) that currently doesn't work?

Thank you
This is should be a fix for:
https://github.com/asciidoctor/asciidoctor-vscode/issues/683
and probably:
https://github.com/asciidoctor/asciidoctor-kroki/issues/397
the sample document would be:
= Title
:safe: unsafe
:data-uri:
:allow-uri-read:
:kroki-fetch-diagram:

== Subtitle

[plantuml]
----
@startuml
Bob -> Alice : Request
Alice -> Bob : Response
@enduml
----

The only setting applied is enabling kroki.
if I replace :kroki-fetch-diagram: with :kroki-fetch-diagram!: the diagram is showing

I don't think it will fix this issue.
The reason kroki-fetch-diagram option does not work in VS code is because unxhr (and node-xmlhttprequest) are executing a Node script synchronously using:
spawn(process.argv[0], ["-e", execString])

https://github.com/driverdan/node-XMLHttpRequest/blob/97966e4ca1c9f2cc5574d8775cbdacebfec75455/lib/XMLHttpRequest.js#L507
require('child_process').execSync(`"${process.execPath}" ...`)

https://github.com/ggrossetie/unxhr/blob/4fbc39978b99180c24cd31ac802c2b4ae5dca3c6/lib/XMLHttpRequest.js#L520C60-L524
It works relatively well in a Node environment, since process.execPath (or process.argv[0]) will resolve to the path to the node binary.
Unfortunately, when running in VS code, Node is somehow embedded and cannot be used to run scripts. So process.execPath will resolve to the binary code which cannot execute Node scripts.
See https://github.com/ggrossetie/unxhr/issues/98
I haven't try Enrik Berkhan solution but I believe it's the only way to fix this issue.

Thank you for your detailed description.
But after the changes in this merge request I was able to retrieve diagram files and have them embedded in the final HTML output and the export is single standalone HTML file again. Currently I am using these modifications on my local VSCode AsciiDoc plugin.
Before these changes, I needed to export using the Ruby version of AsciiDoctor to have an standalone HTML file.

I can confirm that unxhr seems to be working on the latest version of VS code. I can retrieve the image from Kroki and the file is also written on the disk:

The issue is that the file is written at the root of the repository but we try to load the image from the "output directory".
If I do something like (don't do that!):

Then it's working!
Anyway, what we should do is embed images as data URI if data-uri is set.
As a next step, we should try to find a solution to make it work in preview environment (such as VS code) when data-uri is not set.

Since we cannot use data-uri in VS code, I went for kroki-data-uri:

Thank you
As a matter of fact I always set the imagesoutdir to the temp folder of the OS and also set an attribute in the setting of the VSCode plugin for preview and check it using AsciiDoc syntax:
ifdef::vscode-preview[]
:kroki-fetch-diagram!:
endif::[]
ifndef::vscode-preview[]
:kroki-fetch-diagram:
endif::[]

So I get best performance in preview.

I think if I don't set the imagesoutdir the Ruby version also saves the file in the root directory of the document.

Since we cannot use data-uri in VS code, I went for kroki-data-uri:

Oh, I didn't know that data-uri is not working in VSCode. Because I did use it before to embed external images inside final HTML export and if I didn't use it, the result HTML just keep a URL to its location

Oh, I didn't know that data-uri is not working in VSCode. Because I did use it before to embed external images inside final HTML export and if I didn't use it, the result HTML just keep a URL to its location

Reading local files from VS code is not supported by Asciidoctor.js since we need to use a FileSystemProvider https://code.visualstudio.com/api/references/vscode-api#FileSystemProvider

Thank you so much.
I read your final merged code to learn more about the internal functionality of the system and also how to write the required tests.
I have only two questions.

Could you please describe a little more why couldn't we just use data-uri and needed a new attribute for it (kroki-data-uri)?
Is the new code uses any file caching?

Because I saw that no file is ever created and so every time all the diagrams should be fetched again, which really takes quite some times on big documents that normally contains no changes in most of diagrams.
I tested the behavior with the Ruby version and it also creates fetched images in the imagesoutdir, so maybe it is more consistent if JS version also behaves similarly and provides the file caching mechanisms.
I also test it with my original pull request, exported an HTML from the test diagram (which creates a file in the imagesoutdir) and then stopped the Kroki container. Then I export another HTML with no server, and the file is read from the disk and embedded in the final HML. Also if I delete/remove the cached image on the disk and retry to export, the HTML just contains a text of the PlantUML diagram and no image and no link to the server is included (because of kroki-fetch-diagram attribute).
Thanks

If it is of any help, I also updated my code to remove extra parameter in save function and also return the full file path of the file in case of no data-uri attribute, And also included the test from the merged commit (with one change: kroki-data-uri -> data-uri)

Could you please describe a little more why couldn't we just use data-uri and needed a new attribute for it (kroki-data-uri)?

We support both but in VS code data-uri is forcibly disabled since reading local files from VS code is not supported by Asciidoctor.js and will throw an error.
Example:
= Title
:data-uri:

image:path/to/foo.png[]

The above document used to throw an error in VS code so we've decided to forcibly disable data-uri.

Is the new code uses any file caching?

No, we are reading the content and converting it to base64 in memory.

I tested the behavior with the Ruby version and it also creates fetched images in the imagesoutdir

When using data-uri? That's a bit unexpected. In my opinion if you want to embed images in your HTML document, we should not write them in the imagesoutdir. We could however write them in a "cache directory".
This is somehow related to https://github.com/asciidoctor/asciidoctor-kroki/issues/90 and something we could implement independently of data-uri (or kroki-data-uri).

Yes its is correct that data-uri is not supported directly in VSCode and the developer tool shows that only the image file address is there, but when we export an HTML with this attribute enalbed, it puts a base64 encoding of the image in the resulting file.

When using data-uri? That's a bit unexpected. In my opinion if you want to embed images in your HTML document, we should not write them in the imagesoutdir. We could however write them in a "cache directory".

I also think that is correct, but what about when data-uri is combined with kroki-fetch-diagram? Shouldn't it first fetch (and save the diagram to the disk) and then encode it in the resulting document?
Thank you

When using data-uri?

Yes, in my exprience:
This is the sample file (test.ad):
= Title
:data-uri:
:diagram-server-type: kroki_io
:diagram-server-url: https://kroki.io

[plantuml]
----
@startuml
Alice -> Bob: Authentication Request
Alice <-- Bob: Another authentication Response
@enduml
----

and the command to run is:
asciidoctor -r asciidoctor-diagram test.ad

This creates a picture file: (diag-plantuml-md5-98ade79bc4f3c15dbb15304e0097be58.png) and a HTML file with the image data base64 encoded in it.

I also tested the asciidoctor-kroki directly (instead of asciidoctor-diagram) and the result is:

If I only use data-uri it only puts a link of the diagram to the server in the resulting HTML.
If I use both data-uri and kroki-fetch-diagram it first downloads the file to the imagesoutdir (or beside the document, if not set) and then base64 encode it in the resulting HTML.

This is the sample file (test.ad):
= Title
:data-uri:
:kroki-fetch-diagram:

[plantuml]
----
@startuml
Alice -> Bob: Authentication Request
Alice <-- Bob: Another authentication Response
@enduml
----

and the command to run:
asciidoctor -r asciidoctor-kroki test.ad

It makes more sense to write the disk on disk when kroki-fetch-diagram is defined. So it seems that Asciidoctor Diagram automatically fetches and writes files on disk?

It makes more sense to write files on disk when kroki-fetch-diagram is defined. So it seems that Asciidoctor Diagram automatically fetches and writes files on disk?

Yes it looks like that "Asciidoctor Diagram" does that with only data-uri enabled and "Asciidoctor Kroki" (Ruby version) does that when both data-uri and kroki-fetch-diagram is set.

Sorry I lost track of this pull request. Can we close it or do we still need to do something?

Hello, Sorry I thought you might be busy, so I didn't bother anymore
I created a local branch for myself, which embed the diagram images if both kroki-fetch-diagram and data-uri attributes are set it fetches the diagram and save it on the disk which is used as a cache and also base64-encode it in the final output
But I see that after the changes made here, it has conflicts with current codes


Updates class-validator and class-transformer to latest versions

This PR updates class-validator and class-transformer to their latest versions, allowing projects that depend on class-transformer-validator, to update to the latest versions of the peer dependencies.
Changes:

The latest class-validator no longer lists @types/validator as a runtime dependency, that's why it's been added to the list of dependencies here.
The 0.5.0 version of class-transformer replaces the usage of the word Class in methods to Instance. Example: plainToClass is renamed to plainToInstance.

Just chiming in, those functions while deprecated were removed accidentally, 0.5.1 re-added them. However, it's still a good idea to use the new name.

Thanks for the update @NoNameProvided, upgrading to 0.5.1 does indeed work with the existing version of class-transformer-validator


fix(transformer/class-properties): transform private property accesses in static prop initializers

Transform private property accesses in static prop initializers. e.g.:
Input:
class C {
  static #x = 123;
  static y = this.#x;
}

Transformed:
class C {}
var _x = { _: 123 };
babelHelpers.defineProperty(C, "y", babelHelpers.assertClassBrand(C, C, _x)._);

this.#x has been transformed to babelHelpers.assertClassBrand(C, C, _x)._.

[!WARNING]
This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

#7483  👈 (View in Graphite)
#7482
#7481
#7480
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Performance increase is a bit suspicious. It's doing more work rather than less. But I can't find any error.

Mystery solved. Calling the visitor methods (e.g. transform_private_field_expression) from a 2nd place was causing them not to be inlined, which made enter_expression get inlined when it wasn't before. I've inlined enter_expression in #7485, and now this PR has no perf impact. All good!

I think I know this transform better than anyone else at present, so merging without review.

Merge activity

Nov 26, 6:35 AM EST: The merge label '0-merge' was detected. This PR will be added to the Graphite merge queue once it meets the requirements.
Nov 26, 6:35 AM EST: A user added this pull request to the Graphite merge queue.


[ListItem] Missing Hover With leftCheckbox

Images & references
http://www.material-ui.com/#/components/list
Problem description
ListItems which have a checkbox using the rightCheckbox Prop do not have the hover effect like the rest of the ListItems.
Versions

Material-UI: 16.1
React:
Browser:

That was fixed on the v1-beta branch.


Cannot import osdp library from python3.9

Describe the bug
After successfully building and installing the python library, error was encountered when trying to import the osdp library.
Expected behavior
The osdp library should be imported without error.
Observed behavior
When importing the osdp library via import osdp, below error is encountered:
➜  build git:(15) ✗ python3.9
Python 3.9.5 (default, May  4 2021, 03:36:27)
[Clang 12.0.0 (clang-1<IP_ADDRESS>)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import osdp
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: dlopen(/Users/dexinqi/Library/Python/3.9/lib/python/site-packages/osdp-1.5.0-py3.9-macosx-11-x86_64.egg/osdp.cpython-39-darwin.so, 2): Symbol not found: _hash32_djb2
  Referenced from: /Users/dexinqi/Library/Python/3.9/lib/python/site-packages/osdp-1.5.0-py3.9-macosx-11-x86_64.egg/osdp.cpython-39-darwin.so
  Expected in: flat namespace
 in /Users/dexinqi/Library/Python/3.9/lib/python/site-packages/osdp-1.5.0-py3.9-macosx-11-x86_64.egg/osdp.cpython-39-darwin.so

Additional context
I'm on macOS 11.4
This issues is encountered in version 771eae5, the current master.
This issues does not exist in release v1.4.0

@qdx, thanks for reporting the issue. the c-utils submodule when through some cleanup which has caused this issue. I've fixed it now.

Also added PyOSDP builds and import check to CI builds. This should not happen again, sorry about the hassle.


Issue with k8s.io/docs/tutorials/hello-minikube/

"Create a Deployment"

kubectl create deployment hello-minikube --image=k8s.gcr.io/echoserver:1.4

Results:
"error:  no matches for extension/, Kind=Deployment"

Do you have information of the version of minukube and kubectl you’re using?

/priority awaiting-more-evidence