[Performance] transfer_engine_bench get poor throughput with RDMA protocol

First of all, thank you for open-sourcing TransferEngine. However, I encountered some performance issues during my testing:
Performance Discrepancies
Issue 1: Single NIC Performance

Test Setup: Performance test between two nodes using a single NIC.
Observed Throughput:

transfer_engine_bench achieved only 6.56 GB/s.

Expected Throughput:

ib_send_bw (RDMA perftest tool): 11.47 GB/s.

Commands Used:

target:
./transfer_engine_bench --mode=target --metadata_server=<IP_ADDRESS>:2379 --local_server_name=<IP_ADDRESS>:12346 --device_name=mlx5_0 --operation=read

initiator:
./transfer_engine_bench --metadata_server=<IP_ADDRESS>:2379 --segment_id=<IP_ADDRESS>:12346 --local_server_name=<IP_ADDRESS>:12340 --device_name=mlx5_0 --operation=read

ib_send_bw server:
ib_send_bw -d mlx5_0 -F --run_infinitely -p 12300 -x 7

ib_send_bw client:
ib_send_bw -d mlx5_0 -F --run_infinitely <IP_ADDRESS> -p 12300 -x 7

Issue 2: Multi NIC Performance

Test Setup: Performance test between two nodes using two NIC by two parallel processes.
Observed Throughput:

transfer_engine_bench achieved only 1.57GB/s.

Expected Throughput:

ib_send_bw (RDMA perftest tool): 11.47 GB/s.

Commands Used:

target:
nohup ./transfer_engine_bench --mode=target --metadata_server=<IP_ADDRESS>:2379 --local_server_name=<IP_ADDRESS>:12346 --device_name=mlx5_0 --operation=read &
nohup ./transfer_engine_bench --mode=target --metadata_server=<IP_ADDRESS>:2379 --local_server_name=<IP_ADDRESS>:12347 --device_name=mlx5_1 --operation=read &

initiator:
nohup ./transfer_engine_bench --metadata_server=<IP_ADDRESS>:2379 --segment_id=<IP_ADDRESS>:12346 --local_server_name=<IP_ADDRESS>:12340 --device_name=mlx5_0 --operation=read &
nohup ./transfer_engine_bench --metadata_server=<IP_ADDRESS>:2379 --segment_id=<IP_ADDRESS>:12347 --local_server_name=<IP_ADDRESS>:12341 --device_name=mlx5_1 --operation=read &

ib_send_bw server:
ib_send_bw -d mlx5_0 -F --run_infinitely -p 12300 -x 7
ib_send_bw -d mlx5_1 -F --run_infinitely -p 12301 -x 7

ib_send_bw client:
ib_send_bw -d mlx5_0 -F --run_infinitely <IP_ADDRESS> -p 12300 -x 7
ib_send_bw -d mlx5_1 -F --run_infinitely <IP_ADDRESS> -p 12301 -x 7

Questions

How can I deal with the poor performance?
Is it reasonable to use multi processes to test multi NIC?

How can I deal with the poor performance?

By default, transfer_engine_bench repeatedly performs reads/writes with --threads=4 threads. For each iteration, it submits --batch_size=128 work requests, each of them transfers --block_size=4096 bytes data. You can try to tune these parameters to see performance improvement (increase --block_size and --threads=4, reduce --batch_size=128, ...)

Is it reasonable to use multi processes to test multi NIC?

Your execution produces poor performance because both instances are binded with the same processor. You can set --device_name=mlx5_0,mlx5_1 (do not add space and comma) to test multi NIC using one target/initiator process.

By default, transfer_engine_bench repeatedly performs reads/writes with --threads=4 threads. For each iteration, it submits --batch_size=128 work requests, each of them transfers --block_size=4096 bytes data. You can try to tune these parameters to see performance improvement (increase --block_size and --threads=4, reduce --batch_size=128, ...)

Yes, I have experimented parameters like these, but the performance remains at the same level. Do you have any best practices to share?

Your execution produces poor performance because both instances are binded with the same processor. You can set --device_name=mlx5_0,mlx5_1 (do not add space and comma) to test multi NIC using one target/initiator process.

The transfer_engine_bench achieved better throughput which changed from 1.57GB/s to 2.37GB/s, but it is still not satisfactory. Besides, your vllm integration use one target/initiator process per rank, might have the same issue?

We have tested the latest code in our environment.

For case 1, the transfer engine bench reports 10.85 GB/s~11.77 GB/s at default configuration, while ib_write_bw is 13.01 GB/s.
For case 2, the aggregated BW is greater than the theorical BW of a single RNIC.

Does your machine share with other users, especially they consume CPU resources?

The performance issue can be resolved by adding IBV_ACCESS_RELAXED_ORDERING to the memory region access flags. However, I'm curious as to why this flag was not set initially.

The performance issue can be resolved by adding IBV_ACCESS_RELAXED_ORDERING to the memory region access flags. However, I'm curious as to why this flag was not set initially.

Thanks for your report. I am preparing a PR to enable this flag if user requested.

I notice that transfer_engine_bench use block_size as TransferRequest's length, dose it means PagedAttention's block_size?
In practice, is every block must be registered once? If that's the case, it could lead to a significant number of registered memory regions. Are there any potential performance implications associated with this? Additionally, when is a block unregistered? Is it after the request is processed or only once the inference server is destroyed?

A block can be transferred for multiple times, if the previous one is completed.

This means every PagedAttention's block registered once? As I mentioned earlier, this could lead to a significant number of registered memory regions? I'm considering how the TransferEngine integrates with the PagedAttention's block.

If these PageAttention's blocks are stored contiguously, they can be registered as one memory region. You may consider change the allocation of PageAttention's blocks.

Thank you, the issue can now be closed.


Why do we build tdjson.dll, why could't just release in this repo?

It said for another program language interface, we should use tdjson.dll, why didn't this repo provide tdjson.dll built version?

Because binary files are not portable. You must build them yourself for target OS and architecture.

Because binary files are not portable. You must build them yourself for target OS and architecture.

I mean, can we just build for all possible common OS such as Windows, Linux, and MacOS.

No, it is not possible. Binary files aren't compatible even between different versions of the same OS.

so, if I make a tool for everyone to use, other people on windows may not even able to use it? is there any way to resolve this issue?

@wrharper-AASP
On Windows you can build the executable for a fixed processor architecture and a chosen minimum supported Windows version. Then the executable will work for that and all newer Windows version, if they support running binaries for the chosen architecture and you ship all the dependent shared libraries along with the executable.


Compilation Issue on Raspberry Pi

First of all I wanted to express my appreciation for your excellent library.
I've been trying to use it for a project involving a Raspberry Pi, but unfortunately, I encountered a compilation issue.
Error Message
> dub build
    Starting Performing "debug" build using /usr/bin/ldc2 for arm, arm_hardfloat.
    Building handy-httpd 8.3.0: building configuration [library]
'-sse4.2' is not a recognized feature for this target (ignoring feature)
SSE: false
source/handy_httpd/components/multivalue_map.d(251,27): Error: cannot implicitly convert expression `idx` of type `long` to `uint`
source/handy_httpd/components/multivalue_map.d(137,46): Error: cannot implicitly convert expression `idx` of type `long` to `uint`
source/handy_httpd/components/multipart.d(251,19): Error: cannot implicitly convert expression `index` of type `long` to `uint`
source/handy_httpd/components/multipart.d(251,28): Error: cannot implicitly convert expression `index + cast(long)subset.length` of type `long` to `uint`
source/handy_httpd/components/multipart.d(98,31): Error: template instance `handy_httpd.components.multipart.indexOf!(immutable(char))` error instantiating
source/handy_httpd/components/multipart.d(102,35): Error: cannot implicitly convert expression `boundaryIdx + 9L` of type `long` to `uint`
source/handy_httpd/components/multipart.d(134,25): Error: cannot implicitly convert expression `nextIdx` of type `long` to `uint`
source/handy_httpd/components/multipart.d(138,39): Error: cannot implicitly convert expression `nextIdx` of type `long` to `uint`
source/handy_httpd/components/multipart.d(138,50): Error: cannot implicitly convert expression `nextIdx + cast(long)boundary.length + 4L` of type `long` to `uint`
source/handy_httpd/components/multipart.d(146,49): Error: cannot implicitly convert expression `elementStartIdx` of type `const(ulong)` to `uint`
source/handy_httpd/components/multipart.d(146,68): Error: cannot implicitly convert expression `elementEndIdx` of type `const(ulong)` to `uint`
source/handy_httpd/components/multipart.d(167,31): Error: cannot implicitly convert expression `bodyIdx` of type `ulong` to `uint`
source/handy_httpd/components/multipart.d(189,37): Error: cannot implicitly convert expression `nextHeaderIdx` of type `ulong` to `uint`
source/handy_httpd/components/multipart.d(193,37): Error: cannot implicitly convert expression `nextHeaderIdx` of type `ulong` to `uint`
source/handy_httpd/components/multipart.d(193,54): Error: cannot implicitly convert expression `headerEndIdx` of type `const(long)` to `uint`
source/handy_httpd/components/multipart.d(196,64): Error: cannot implicitly convert expression `nextHeaderIdx` of type `ulong` to `uint`
source/handy_httpd/components/multipart.d(196,81): Error: cannot implicitly convert expression `nextHeaderIdx + 2LU` of type `ulong` to `uint`
source/handy_httpd/components/multipart.d(206,54): Error: cannot implicitly convert expression `headerValueSeparatorIdx` of type `const(long)` to `uint`
Error /usr/bin/ldc2 failed with exit code 1.

Environment
Raspberry Pi Model: [Specify the model]
OS: Linux 5.10.60-1-ARCH #1 SMP Sat Aug 21 14:56:38 UTC 2021 armv7l GNU/Linux
Compiler: LDC - the LLVM D compiler (1.36.0):
Library Version: 8.3.0
Steps to Reproduce
> git clone https://github.com/andrewlalis/handy-httpd.git
> cd handy-httpdj
> dub build

thank you

Thank you very much.
Now compiler complains:
source/handy_httpd/components/websocket/frame.d(244,32): Error: cannot implicitly convert expression `payloadLength` of type `ulong` to `uint`
source/handy_httpd/components/websocket/frame.d(175,33): Error: template instance `handy_httpd.components.websocket.frame.readPayload!(SocketInputStream*)` error instantiating
source/handy_httpd/components/websocket/manager.d(191,42):        instantiated from here: `receiveWebSocketFrame!(SocketInputStream)`

see my PR.
Thank you again

Additional error messages should be fixed in v8.3.2. Please reopen if there are more issues with ARM, as, like I said, I can't really test it myself, nor does github actions offer arm runners.


Module parse failed: Unexpected token by library containing .svg file

Command
build, serve
Is this a regression?

[ ] Yes, this behavior used to work in the previous version

The previous version in which this bug was not present was
No response
Description
Hello,
I'm currently facing an issue where one of the libraries we use in the project contains a .svg file which currently prevents angular from starting the local server or building
The error we see is:
You may need an appropriate loader to handle this file type, currently no loaders are configured to process this file. See https://webpack.js.org/concepts#loaders
> <?xml version="1.0" encoding="UTF-8"?><svg id="Ebene_3" xmlns="http://www.w3.org/

I havent been able to add a custom webpack config yet - any pointers would be appreciated.
Currently the angular config is using "builder": "@angular-devkit/build-angular:browser",
Is there a built-in way to ignore e.g. svg from being processed? Or a way to add a file handler?
Minimal Reproduction

install @needle-tools/engine or any npm package containing a .svg file
run ng serve in the project (see env below)

make sure to import the library in one of your files
Repo: https://github.com/luixxma/angular-needle
Exception or Error
You may need an appropriate loader to handle this file type, currently no loaders are configured to process this file. See https://webpack.js.org/concepts#loaders
> <?xml version="1.0" encoding="UTF-8"?><svg id="Ebene_3" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" viewBox="0 0 160 187.74"><defs><style>.cls-1{fill:url(#Unbenannter_Verlauf_122);}.cls-2{fill:#f3e600;}.cls-3{fill:url(#Unbenannter_Verlauf_113);}.cls-4{fill:url(#gradient-6);}.cls-5{fill:url(#Unbenannter_Verlauf_110);}.cls-6{fill:url(#Unbenannter_Verlauf_101);}.cls-7{fill:#9c3;}.cls-8{fill:#ffe1

Your Environment
Angular CLI: 16.0.1
Node: 16.16.0
Package Manager: npm 8.19.2
OS: win32 x64

Angular: 16.0.1
... animations, cdk, cli, common, compiler, compiler-cli, core
... forms, material, platform-browser, platform-browser-dynamic
... platform-server, router

Package                         Version
---------------------------------------------------------
@angular-devkit/architect       0.1600.1
@angular-devkit/build-angular   16.0.1
@angular-devkit/core            16.0.1
@angular-devkit/schematics      16.0.1
@nguniversal/builders           16.0.2
@schematics/angular             16.0.1
rxjs                            7.8.1
typescript                      4.9.5

The mentioned library is importing the SVG like a ECMA module. This is Webpack specific behaviour which is not supported and not intended to work.
import _logo from "./logo.svg"
export const logoSVG = _logo;

Closing as the failure is correct here and by design.

Hello @alan-agius4 thanks for the fast reply. Do you have any information why it works in webpack 5 when running in a next.js context?
Do you have information about how to modify the webpack.config for angular-cli projects?

Fixed by embedding the svg string instead of importing it

Do you have any information why it may work in webpack 5 when running in a next.js context?

Probably next.js allows using Webpack specific feature.

How do you recommend asset files inside packages should be loaded/handled?

This is an issue with the library I suggest to file an issue with them. Inlining the SVG is a good solution.

Do you have information about how to modify the webpack.config for angular-cli projects?

You can use 3rd party builders like ngx-build-plus. Note that using custom builders or custom webpack configuration are not supported by the Angular team. The fact that currently the CLI uses Webpack by default is an implementation detail and it will change in the future.

Yes next.js exposes the webpack config and you can modify it as you'd wish.
How would you recommend to transform code when building an Angular app or adding loader plugin? Does Angular provide an API for that which is decoupled from the bundler being used?
From reading and searching I've came across this reply a few times and I wonder why you dont expose it and if you ever switch the bundler in a major release then people would have to adjust their projects IF they chose to make "lower-level" adjustments? Perhaps you could also add a warning when someone does this?


New version: LeNgocKhoa.Laragon version 6.0.0

Pull request has been created with WinGet Automation :rocket:
Microsoft Reviewers: codeflow:open?pullrequest=https://github.com/microsoft/winget-pkgs/pull/115090&drop=dogfoodAlpha

@wingetbot run

@wingetbot run


Develop stream 20220613

Implemented the block_sort_merge
Merge branch '424-typo-in-device_adjacent_difference-large-indices-test'
Add target for benchmark_config_tuning
Merge branch 'fix-rocthrust-backwards-compatibility'
Add compatibility method to offset_lookback_scan_prefix_op
Add definition for wrapped_reduce_config::architecture_config::params
Dynamic config dispatch in device_reduce
Add helper of stream ordered memory copies, use it in segmented radix sort
Merge branch 'fix-coloring-in-formatting-script'
Improved histogram benchmarks
Improve device_merge_sort.hpp sort performance with an average of 8%
Reimplemented device_reduce_by_key using lookback scan
Merge branch 'fix-format-double-error'
Merge branch 'Redundand-HIP_CHECK-definitions-removal'
Merge branch 'hotfix_custom_floats_in_segmented_radix_sort'
Fix radix_merge_sort partial specialization to cover all cases

The latest commit temporarily disables config dispatch when building for windows. It can be reverted once the windows runtime has an implementation for hipGetStreamDeviceId


Add the contents of /sys/fs/pstore to mayday tarballs

Current situation
See https://github.com/flatcar-linux/mayday/issues/1 for context.
All documentation about acquiring core dumps from Flatcar / CoreOS refer to /sys/fs/pstore and ERST logs. You can't get these logs in mayday, and when diagnosing a kernel panic they are required.
Impact
Diagnosing kernel panics is very challenging on large fleets.
Ideal future situation
These files are included automatically if they exist.

If in the future people have systemd-pstore.service enabled to automatically clean pstore, we should also cover /var/lib/systemd/pstore/ where the dumps are moved from pstore.

PR is here: https://github.com/flatcar-linux/mayday/pull/7

Merged and scheduled for inclusion in the next Alpha and Edge release, then the following Beta and Stable releases.


Docs.MS CI Updates for Microsoft.Azure.CognitiveServices.Knowledge.QnAMaker

CI Update for release of Microsoft.Azure.CognitiveServices.Knowledge.QnAMaker

Docs Build status updates of commit 009db5c:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

bundlepackages/azure-dotnet-preview.csv
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel


No file for react native?

From @tishad770 on April 9, 2017 6:36
Copied from original issue: joeblau/gitignore.io#336

React native is a combination of a bunch of templates — technically react native is this:
https://www.gitignore.io/api/osx%2Cbuck%2Cnode%2Clinux%2Cxcode%2Cgradle%2Candroid
I think I'll create a task to compose templates.

I added stacks and also added a React Native stack - https://www.gitignore.io/api/reactnative


compilation problem

code
+package sandbox

+alias org.eolang.io.stdout

[args...] > app
  stdout "SOME" > @

mvn clean coompile
result: SOME
change "SOME" in code on "SOMETHING"
code
+package sandbox

+alias org.eolang.io.stdout

[args...] > app
  stdout "SOMETHING" > @

mvn compile
result still: SOME

@IngeniariusSoftware It have been introduce by https://github.com/cqfn/eo/pull/453. mvn clean compile works as expected?

@andreoss yes, mvn clean compile works correctly, but I expect that after modifying eo file and run command mvn compile I got new result, but in reality I got the same

@IngeniariusSoftware it's fixed, try version 0.20.0

Job gh:cqfn/eo#473 is not assigned, can't get performer


[Fix] Broken method: onMockedResponseSent

Fix: https://github.com/valendres/playwright-msw/issues/16

Updated msw version and deps.
Fix the method.

Hey @Azurewarth0920, my apologies for this, but the CI pipeline was broken when you published your PR (not caused by you). I've fixed this now, but it unfortunately caused a merge conflict with your PR. Instead of asking you resolve this merge conflict, I decided to create a new PR with your changes so that we could get this merged ASAP.
Please see https://github.com/valendres/playwright-msw/pull/19 for the PR 🙂


Issue #7169

Added some important features.

@mfbehrens99 Oh no, this is embarrassing for me. I didn't see this PR before I worked on #7169 myself in 2c9aa35fe4bf11a99e8733862d4e792b5471a89e 😭. Sorry about that!
In the future, if you put the issue hash (#7169) in the comment instead of the title then a link to the PR will appear in the issue thread. This can help avert miscommunications like this. I also should have read all my notifications 😂


Dogfood Everything: Review documentation of all tutorials you wrote to clean up before v1

Migrated from GitLab: https://gitlab.com/meltano/meltano/-/issues/1020
Originally created by @dmor on 2019-09-20 14:19:37

[x] Review home page content (@bencodezen)
[x] Review Introduction (@bencodezen)
[x] Review Local Installation (@bencodezen)
[x] Review Cloud Installation (@bencodezen)
[x] Review Getting Started (@bencodezen)
[ ] Review Advanced Tutorials

[x] Carbon
[ ] Salesforce (BLOCKED - no valid credentials)
[x] GitLab
[x] Custom Extractor
[ ] Custom Transformations (BLOCKED - no valid credentials)
[x] Tap-Postgres with Meltano
[x] Loading CSV Files into Database
[x] Jupyter

[x] Review Command Line Tools (@bencodezen)

View 38 previous comments from the original issue on GitLab


sipjs.com is unavailable

Bad certificates and 404 (The page you're looking for could not be found.)

@khalitovsv You can still access to the doc here -> https://gitlab.com/onsip/sipjs.com for the moment

We are investigating this. We suspect it's something with gitlab's new release, but we did not change anything. As @Hakim-Bou said, you can visit the gitlab repo for the documentation in the meantime.
Thanks,
James

This is now resolved. Thanks again for letting us know!


Add topographic universal transformer (TUT)

The Universal Transformer (UT) architecture adds recurrence to the classic transformer by way of depth-wise weight sharing. More recently, the Sparse Universal Transformer (SUT) combines the UT with sparse mixture of experts, resulting in depth-wise weight sharing, and horizontal weight differentiation, much like our idea for the recurrent columnformer.

Here we add the Topographic Universal Transformer (TUT), which combines the UT with topographic mixture of experts (#5). The network consists of a sheet of transformer modules ("columns"), applied to the input recursively as in UT. The MLP at each position in the sheet is assigned weights from a pool of experts by way of a softmax weighted combination using static learned coefficients. The self-attention is as of now unchanged wrt to the UT, meaning in particular the weights are shared across depth and width.

In the TUT, we think of the sheet of columns as being spatially embedded. Right now, the spatial embedding is a stack of layers in 3D space, to stay close to the classic ViT. But whereas in the ViT, each layer is computed sequentially with purely feedforward communication, in the TUT, all layers are computed at every time step with arbitrary communication both within layers and between layers. But as in ViT, we still feed input into the "first" layer, and read output from the "last" layer.
This poses a question regarding what do to about skip connections between time steps, to encourage information flow through the network. In the ViT, these skip connections between layers that bypass attention are quite important. In the TUT, a naive implementation of skip connections following ViT and UT results in within-layer self-recurrent communication. This does not encourage feedforward information flow. Alternatively, we can introduce direct feedforward skip connections between the pseudo-layers of the TUT.
Here we see a comparison of the attention matrices in both cases. The diagonal blocks correspond to within-layer communication, below diagonal blocks are forward communication, and above diagonal blocks are feedback communication. You can see in the self-recurrent skip case, self-attention does not really learn to use layers 2-5. Whereas in the feedforward case, there is more overall communication.


fixed #35: incorrect hmac signature encoding

@smaxwellstewart thanks for the PR but I don't get why we should return the output of hash_hmac in a raw format. Could you please explain? thanks

@cirpo theres an open issue with a more detailed explanation
@smaxwellstewart the changes seem to be ok, but could you fix the tests that are now broken?

https://github.com/namshi/jose/issues/35

I can add fixes for the tests for sure.

Seems like it's all passing apart from HHVM. Anybody have any ideas why that test is faling? Does it matter?

@smaxwellstewart for HHVM it doesn't matter, thanks :)

all good! Im gonna publish a new major, since as soon as you roll the library out old clients using hmac will not work (as verification will not pass for tokens generated pre-fix)

tagged 5.0.0


Added myself

Oh my mad, thats my habit using auto refactoring feature on vs code by press ctrl + alt + l,......
i am sorry


Limit of 499 entries for .include

Does JSForce limit the results coming from a relationship with .include()? I have a custom relationship in my Account table, but my inner query does not return more than 499 records:
  conn
    .sobject('Account')
    .select('the,fields,I,need')
    .include('Account_Contacts__r')
      .select('the,other,fields,I,need')
      .end()
    .execute({ autoFetch: true, maxFetch: 1000000 })
    .on('record', (res) => {
      // process the records
      // ...
    });

I have also tried adding a limit to it, with no success:
  conn
    .sobject('Account')
    .select('the,fields,I,need')
    .include('Account_Contacts__r')
      .select('the,other,fields,I,need')
      .limit(1000000)
      .end()
    .execute({ autoFetch: true, maxFetch: 1000000 })
    .on('record', (res) => {
      // process the records
      // ...
    });

Do you have solution for fix or workaround it?

I am fairly certain that SFDC paginates the data for subqueries, so it may not return all records in the initial query.  Providing a limit will not impact how many records are returned with each set of query results.
The only thing I can think to do would be to use the queryLocator (returned as part of the subquery) to fetch the next set of records and continue until they are done.  Unfortunately since this is a subquery, you will have to do it for each of parent records that have remaining child records that were not returned.
You might want to re-think your query strategy if you have large numbers of child records (e.x. query child records and include parent fields and in javascript reduce the data back into the parent child structure)


Biased Histogram Implementation

This a biased histogram implementation based on Fast Computation of Approximate Biased Histograms on Sliding Windows over Data Streams.
Attn: @anthonydahanne @mathieucarbou @cschanck

Can one of the admins verify this patch?

@chrisdennis : I just saw this PR has not been reviewed and merged yet... I see I have the rights to merge. Would you like me to review it next week ?
BTW, what is the use case ? Is it to be able to have percentiles in M&M ?

Yes, this adds support for windowed percentiles.  It had sat on my machine for quite some time, and I was a little concerned that it might bitrot, so I created a PR.  Whether you review/merge or not is I guess a question of how high a priority percentiles are to M&M.

CC @anthonydahanne

I think now would be a good time to review and merge this

@chrisdennis : can you please rebase ? I'll merge after. Thanks!

@chrisdennis could you please rebase (once again) and then provide javadoc to the constructors (BarSplittingBiasedHistogram) ? thank you !


Fixes scroll offset when allowTransform="false" and dragged element is in position fixed div

Hi,
This code has fixed for me the wrong offset on the dragged element when scrolling. I met this problem while I had disabled allowTransform and while the dragged element was inside a position fixed div with fixed height and width.
To use it :
<div class="col-lg-6 draggable-media" ng-drag="true" ng-drag-data="media" data-allow-transform="false" data-scroll-offset="true">
    Dragged HTML
</div>

It should fix at least :

#142 : my (or my team) previous issue
#157 : Maybe. But he does not seem to use allowTransform="false" and my fixes has only be tested with this configuration (https://github.com/fatlinesofcode/ngDraggable/compare/master...bein-sports:feat-scroll-offset?expand=1#diff-045be7ab7709e1d9df374e76bac2659cR48)
#158 : Same as #157

Regards

+1


Add redshift-data: cancel_statement, describe_statement, execute_statement, get_statement_result

Add following actions for redshift-data:

cancel_statement
describe_statement
execute_statement
get_statement_result

Codecov Report

Merging #4832 (989b5ca) into master (7194456) will increase coverage by 0.01%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master    #4832      +/-   ##
==========================================
+ Coverage   95.16%   95.17%   +0.01%
==========================================
  Files         594      599       +5
  Lines       64241    64399     +158
==========================================
+ Hits        61132    61290     +158
  Misses       3109     3109

Flag
Coverage Δ

servertests
36.07% <99.36%> (+0.15%)
:arrow_up:

unittests
95.10% <100.00%> (+0.01%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

moto/backend_index.py
100.00% <ø> (ø)

moto/__init__.py
98.46% <100.00%> (+0.01%)
:arrow_up:

moto/redshiftdata/__init__.py
100.00% <100.00%> (ø)

moto/redshiftdata/exceptions.py
100.00% <100.00%> (ø)

moto/redshiftdata/models.py
100.00% <100.00%> (ø)

moto/redshiftdata/responses.py
100.00% <100.00%> (ø)

moto/redshiftdata/urls.py
100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7194456...989b5ca. Read the comment docs.


Update jquery.image-gallery.js

add alt attribute to gallery image element

1. Why is this change necessary?
the alt attribute is missing in the gallery img element
2. What does this change do, exactly?
add alt attribute in the gallery img element
3. Describe each step to reproduce the issue or behaviour.

go to product detail page
click on product image
check in gallery image element in developer console

4. Please link to the relevant issues (if any).
5. Which documentation changes (if any) need to be made because of this PR?
6. Checklist

[ ] I have written tests and verified that they fail without my change
[ ] I have squashed any insignificant commits
[ ] This change has comments for package types, values, functions, and non-obvious lines of code
[ ] I have read the contribution requirements and fulfil them.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Hello,
thank you for creating this pull request.
I have opened an issue on our Issue Tracker for you. See the issue link: https://issues.shopware.com/issues/SW-25525
Please use this issue to track the state of your pull request.

Thanks for your Contribution! :blue_heart:


update logo position

I have added the resized logo using Aidan's suggestion. Let me know what you think of the size/position of the logo.

If you have approved this PR I am happy to merge it.

Will wait for the other PR to merge and then rebase from develop to avoid conflicts


Fix shadow map rendering on AMD cards

Fixes shadow map rendering at wrong location (half distance from light to occluder) on AMD cards.
Fixes #12

I confirmed this is still an issue with latest AMD drivers and Defold on Windows.  The fix resolves it on web and Windows.  Also confirmed the fix still does not affect rendering on Nvidia hardware.
Tested on RX 6800, RTX 3060, and GTX 1070


Update circe-generic, ... to 0.11.0

Updates

io.circe:circe-generic
io.circe:circe-generic-extras
io.circe:circe-literal
io.circe:circe-parser

from 0.10.1 to 0.11.0.
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention @scala-steward in the comments below.
Have a nice day!

Looks like perhaps java8 time was just folded into circe???
[warn] 	::::::::::::::::::::::::::::::::::::::::::::::
[warn] 	::          UNRESOLVED DEPENDENCIES         ::
[warn] 	::::::::::::::::::::::::::::::::::::::::::::::
[warn] 	:: io.circe#circe-java8_2.12;0.11.0: not found

confirmed, circe-java8 for version 0.11.0 has not been released to maven central

Ah that's awesome that steward just fixed this on its own.

Nice, can merge this now


Zoning / PUD application status nearby

We'd like to include notifications of nearby PUD / Zoning statuses in a map + list for each PresCat building.
We will probably want to talk to the Office of Zoning about getting access to this data systematically. They have an online system and posted PDFs. For now, this is the information available:
PDF list of filings by Ward
Interactive Zoning Information System
Board of Zoning calendar of upcoming hearings
Zoning Commision calendar of upcoming hearings
Zoning map (contains current pending PUD areas in red)

I'm still getting errors when I try to add to the database, but here's a CSV of (I think all of) the zoning data on http://maps.dcoz.dc.gov/zr16/. https://www.dropbox.com/s/5ntjt1pxgwy58ul/pud.csv?dl=0
The code that I used to generate it is here: https://gist.github.com/salomoneb/4eaec0e1f578418eb682151bb2e6816f
I scraped the data by querying an endpoint that I found on the map page: https://maps2.dcgis.dc.gov/dcgis/rest/services/DCOZ/Zone_Mapservice/MapServer/25/query?f=json&where=Square =' + SQUARE NUMBER + '&returnGeometry=false&spatialRel=esriSpatialRelIntersects&outFields=*
The script constructs URLs using each property square number in the dc_tax table and requests data from each one, outputting the results in the CSV.
This is the first real Python script I've written and wildly impractical for ongoing purposes, but it does give us a large baseline snapshot. We'll also probably need to clean the data because many of the values seem to be in inconsistent formats.
If you have a contact at the Office of Zoning, getting direct access to the database is probably the way to go.

One thing to point out now that I'm looking at this on my work computer with a functional copy of Excel: check out the "case_type_relief" column, which has at least six PUD-related categories.

Awesome! Good stuff - congrats on your first 'real' Python script 👍  It's not wildly impractical either - just a few optimizations as well as some things to make it robust if we need to call the endpoint regularly (and a sleep function to keep their servers happy). Other things to look into are whether they also provide a 'latest' endpoint we could tap for just updates/changes, or a bulk endpoint of some sort.
There is a GIS Steering Committee coming up on Thursday at 10am, I'll go to that and see if I can make any contacts in the Zoning office. They have been doing lots of modern GIS stuff, including recently released tools including the maps.dcoz.dc.gov page you cited, so there's likely someone there that would be interested in talking to us about putting their data to use.
What exactly is each row of this? The case_type_relief column looks like it means nearly every row has some sort of zoning issue associated with it, so does this only return results for square numbers that have associated zoning changes/

Noting per our discussion last Tuesday that we'll want to refactor the code to include properties that are on the zoning map but don't have any case data associated with them. We talked about putting these in a separate CSV. I'll do this, upload the code to scripts/analysis, and add the modified data to the database (if anything changes).

Sorry for the delay. This is done. Before I submit a PR, can you give me access to the secrets.json file needed to set up AWS so that I can upload the scraped data?

@salomoneb I wanted to check, do you have any documentation on that API endpoint? We should contact the office of zoning to verify we know how up-to-date that endpoint is; I can do this if you let me know what you have/haven't sent to them already.

Unfortunately, I don't have any documentation. I found it by looking at the script sources on the zoning map page. http://maps.dcoz.dc.gov/zr16/
Looking at the dateFiled column of the scraped results, the most recent, legitimate-looking date is Monday, December 12, 2016. There are also some dates that are listed as being filed in 2020, but I don't know why that is.
The map itself is labeled on the map portal page as a 2016 map, so it's possible that it doesn't include 2017 data. I called the zoning office, but wasn't able to reach the map person. I'm going to try again in a bit.

Notes from our conversation with Matthew Holden at Office of Zoning:
-API endpoint @salomoneb mentions above is best place to get up to date data from zoning. Data is not completely accurate, with older data more likely to be incorrect or missing - Matt guessed about 70% accurate. We should link to the actual filings themselves when we cite specific zoning info so that people can get real data out of the filing documents when needed.

Building permit data from opendata.dc.gov is best source; it is updated automatically on a weekly basis from the relevant database at DCRA (although those are a combo of manually/systematically updated so there will be some lag there).
-No programatic source of the number of inclusionary zoning units that are attached to the PUD updates from the map API endpoint. Have to dig into the filings themselves. Key thing to look for is the 'benefits' requirements attached to zoning relief (also includes things like parks, amenities, etc.)

Other notes:

Raw building permit data is compiled manually by DCRA and automatically pulled into opendata.dc.gov weekly.
In the building permit data, look at the permit_subtype_name field for more info about project.
People only go through Office of Zoning after they’ve applied for a building permit through DCRA and need a special exception or exemption.
Using public amenities to get subsidy (?)
Some zoning information is incorrect. Likely applies more to cases that are over 5 years old.
Inclusionary zoning - requires certain percentage of residential unit to be set aside for affordable housing. (see https://dhcd.dc.gov/service/inclusionary-zoning-affordable-housing-program). Varies based on zones. New zoning boundaries defined here: http://dcgis.maps.arcgis.com/apps/MapJournal/index.html?appid=ae5830535a384a7f9ae40527f607b32e. 2016 boundaries seem to have replaced ones defined in 1958, which (I think) are what we scraped from the zoning endpoint in the existingZoning field. We’d need to update the field this field by mapping old boundaries to new ones.
Zoning changes occur based on map amendments.

@salomoneb I'm not sure if it helps, but I did some quick and dirty analysis of the pud.csv file you scraped to check and see whether the PUD case files represented in that csv might include useful details on the Affordable Housing inventory. It's a different topic than what you're covering in this Issue but there may still be some useful context in the notes I drafted.  Bottom line is we will probably need to scrape more data from the endpoint mentioned above if we want to figure out how much detail those PUD filings may have on AH inventory. #90

Note the PUD scraper that @salomoneb wrote should be adapated to follow the BaseApiConn approach in the /python/sources folder to line up with our current code. We'll want to refactor his scraper into this new format once #198 is done (to provide us a with a source of SSL's) or possibly utilizing the same data as #140 (which would be another source of all SSL's - but note that for this scraper we also want PUD's for non-residential stuff as commercial development nearby to an existing affordable housing unit is important)

Putting on hold until either 198 or 140 is done.


Improve Listen() implementation in SqsNotificationListener

See what we can do to improve this logic based on the information in David Fowlers async guide

Specific patterns:

The ContinueWith on line 104 is against the guidance.
the  using (var linkedCts on line 216 appears to be correct, as the CancellationTokenSource is disposed after use.
Launching threads happens via the IMessageProcessingStrategy on line 259. This might or might not be good.

I'm currently having a play about some more with IAsyncEnumerable (blog post almost finished), which I'm also experimenting with putting in your spike, and exposing it optionally on the side. I'm not suggesting going down that route for JustSaying, but could become part of a nice composable toolkit that plays nicely with JustSaying.

The concerns listed here have been addressed by these :tada: :
https://github.com/justeat/JustSaying/pull/440/commits/77dba4bbc91c63cf4d424b6da3b455357f6f74fe
https://github.com/justeat/JustSaying/pull/403
https://github.com/justeat/JustSaying/pull/453
https://github.com/justeat/JustSaying/pull/440/commits/d2ed621e564c0dcfbfb65681f1aeb7efb23df066


cleanup istft wrapper

remove istft & tests from audio/functional

Can you also remove this from the docs?


Rule: skipped tests in shared schemes

I think understand the idea here, but this isn't an area I'm terrible familiar with. From @danielpunkass

To clarify: this covers the scenario where you are testing a subset of tests by using the scheme editor's Test -> Info tab:

The problem is it's too easy to accidentally check this in. Now your shared scheme has permanently disabled tests that won't run on CI, on other devs' Macs, etc.

The unchecked boxes correlate with a SkippedTests node in the .xcscheme file:

Scanning for the presence of that node in any shared schemes would be sufficient for this rule.

Thank you so much for the explanation. This makes sense, and I think this'll be great!

Addressed with 90e461ca798f7600134646d7501a46d4bb5a10a4


Fix pyBQM.energies

Not sure what's up with CircleCI right now. Getting fatal: couldn't find remote ref refs/pull/1046/head. Seems like an issue on their end. I will give it an hour to resolve itself before digging deeper.


[BUG] VDBs culled after a certain distance

Environment
Operating System: (Windows 10 Pro)
Version / Commit SHA: (commit version eb5de021bfed434322c33f2e51ec3aa187867385)
Describe the bug
When a vdb has been added to unreal 5, after a certain range it will be culled off.
I had a look at what controls this but there doesn't seem to be a way to control this. Is this functionality possible or need to be added?
To Reproduce
Steps to reproduce the behaviour:

Open UE5 and add vdb into any scene
Move camera away from vdb
You will see the vdb fade after a certain distance

Expected behaviour
I expected that it would be visible at further distances, especially as I was working on cloud landscape looks and after certain ranges its no longer visible

Hi @philTunni, we don't maintain any plugins for Unreal. You'll be better off filing an issue with whatever plugin you're using to import vdb files or Unreal directly if they have native support for this.

I did contact Thibault (who made the plugin) and from their investigation they pointed me here:
"I've been able to reproduce the issue and it is not Unreal related. It is an issue with NanoVDB indeed. Sadly, there isn't much I can do about it. I recommend posting an issue on their github"

I see - unfortunately as we don't have any knowladge of how VDB has been integrated into this plugin we can't assist without more info. However it sounds like the creator of the plugin has identified the issue in NanoVDB which is causing the problem? If they were able to get in touch and describe the issue that could help - we can look to address anything on the VDB side but anything specific to UE or UE glue code would have to be fixed by the plugin maintainer.

Closing, but please feel free to create a new issue if the creator is able to provide more information.


Using feross/simple-peer could be great!

Hi there. Maybe it could be good idea to make a twist using your Simple Peer library for a more clean implementation. Thanks Feross for pushing the limits of WebRTC. By the way good news in the Apple world: https://blog.peer5.com/apple-announces-support-for-webrtc-in-safari-11/

lxjs-chat does use simple-peer! I accidentally left the old peer.js file in there, so I just deleted it to avoid confusion.


Test against kerberized Kafka environments

See https://github.com/spring-projects/spring-integration-kafka/issues/123#issuecomment-232620365

As of today, we have support to pass kerberos settings both at the producer and consumer level. They are passed as k/v pairs chained under .configuration property.
For example, at the producer level, you'd pass settings like: spring.cloud.stream.kafka.bindings.<channelName>.producer.configuration.sasl.kerberos.service.name=VALUE.

The list of all consumer properties in Kafka 0.9 is here: http://kafka.apache.org/090/documentation.html#newconsumerconfigs
(they can all be prefixed with spring.cloud.stream.kafka.bindings..consumer.configuration)
The list of all producer properties in Kafka 0.9 is here: http://kafka.apache.org/090/documentation.html#producerconfigs
(they can all be prefixed with spring.cloud.stream.kafka.bindings..producer.configuration)
Here is an example on configuring clients and brokers for kerberos: http://kafka.apache.org/documentation.html#security_sasl_kerberos


Feature/request more feedback

Checklist

[X] I run yarn run webpack:build:main: the project builds without errors.
[X] I run yarn lint: the project builds without code style warnings.
[X] I tested the changes and all related features on the test server https://artemistest.ase.in.tum.de.

Motivation and Context
It allows user to request more feedback for text exercises.
Description
Student can only open up to 3 complaints, but sometimes student wants only to get more feedback on his score, that is where this feature is useful.
Steps for Testing

Log in to ArTEMiS
Navigate to any course with submitted Text Exercise
Open Text Exercise and click Complain
Enter the reason for Complain and click 'Request More Feedback' (1st screenshot)
Navigate to the same exercise in Tutor Dashboard
Click 'Open Assesment' and you can see More Feedback request on the bottom (2nd screenshot)

Screenshots

@maxr96 Can we please move the implementation to a feature branch in this repo? I think you should have access now, right?
I think we talked about changing some parts of the GUI to separate complaints and requests for more feedback more directly. Do you have any progress since then? Would be great to release this in 3.2.0 in the upcoming week to the students because they are really demanding this feature!
Please merge the newest changes from develop and double check that everything works as intended.

Can you please also make sure to write tests for the new functionality (this PR still has the old template)? If you have any questions about it, let me know.
This will be tackled in the new PR from feature branch in this repo


Change the outdated emphasis block into the current theme

The "!!! important" usage within the documentation is used within the Forge epoch, however NeoForge turned to use a novel format for these highlighted paragraphs. Hence make up for the missed one.

Why not do it for all the pages which have the issue?

Why not do it for all the pages which have the issue?

Only found this one by accident when reading the documentation. If needed I can check all pages to discover similar issues.

Please do and thanks. There's only a few more instances based on a quick search.

Please do and thanks. There's only a few more instances based on a quick search.

OK, I'll work on then.

All documentation files have been checked and fixed up targeting the same issue.


Remove code that restricts checkbox to single owner

Pull Request Template
Description
Remove code that limited the debt fraction checkboxes to single owner, as this code will be present in all financial models after https://github.com/NREL/ssc/pull/1208 is merged
Fixes # https://github.com/NREL/SAM/issues/1708
To test:

All financial models with DSCR options should have this checkbox
The checkbox should work where it is present. See https://github.com/NREL/ssc/pull/1208 for rough instructions. Financial models that are reasonably profitable by default might need input adjustments to test the lower bound.

Type of change

[X] New feature (non-breaking change which adds functionality)

Checklist:
If you have added a new compute module in a SSC pull request related to this one, be sure to check the Process Requirements.

[X] My code follows the style guidelines of this project
[X] I have performed a self-review of my own code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[X] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[X] New and existing unit tests pass locally with my changes
[ ] Any dependent changes have been merged and published in downstream modules
[ ] I have checked my code and corrected any misspellings

Great suggestion! Added in the latest push:


Firefox issue with  canvas

If one of the slides is a <video> canvas, video won't play in Firefox. Works fine in Chrome.

@skynet can you please create an example here http://jsfiddle.net/L0hntr35/ :smile: - also just to confirm have you put the <video> inside a container like a <div> ?

Without slick.js: http://jsfiddle.net/cbg1p3x6/15/
With slick.js: http://jsfiddle.net/cbg1p3x6/16/

http://jsfiddle.net/cbg1p3x6/21/
That's working for me in FF and Chrome now... for your information and anyone else: This is 100% not something to do with Slick.
You set the source to video/mp4 but Firefox only sort-of supports it. I could get it to run in another tab using the direct link to video and activating a Quicktime plugin on my Mac, but it still wouldn't work in the JSFiddle. -- https://developer.mozilla.org/en-US/docs/Web/HTML/Supported_media_formats
Anyway, the solution is to supply an ogv source for Firefox. I would also suggest you look in to supplying a webm source for IE.
Please close this issue :smile:
Regards, Simon.

Did actually test with .ogv before. And actually not working for me in Chrome now. I'll come up with more examples when I get a chance. Thanks.

@skynet : bottom line is that HTML5 apis are new and temperamental in each browser. It has absolutely zero to do with Slick. Video is especially finnicky. -- I can prove it as your video doesn't even work in Firefox without the carousel:

I tested my JSFiddle and it works in both Chome, Safari and Firefox. (Safari wont autoplay).
If you have more trouble using the native HTML5 <video> API then I suggest you try something like videojs or jwplayer. Both have saved me many times in the past :smile:
Simon

Simon - if you look at the code it is actyally using videojs
Thanks for your support!

Ah gotcha :)
Just checked on my mobile Chrome (android) (on the bus,  sorry) with the
video-js classes removed and it works fine :) - maybe you should pursue any
further assistance with noting its a library conflict (well, not for me)
and not a <video> problem  :grin:
Best,  Simon
On 19 Jan 2015 14:12, "Ionel Roiban"<EMAIL_ADDRESS>wrote:

Simon - if you look at the code it is actyally using videojs
http://www.videojs.com/
Thanks for your support!
—
Reply to this email directly or view it on GitHub
https://github.com/kenwheeler/slick/issues/932#issuecomment-70450298.


Dcml annotation parser

Import for DCML corpora tsv files
DONE

functional import with the possibility to include only notes, or notes, measures and harmony.
Dependency in pandas
Some updates on score classes such as Cadences, Roman numerals and Phrases.
Updated the estimate symbolic duration function in music.py.

TODOs (For new PRs)

Remove pandas dependency:
This will be planned for later using numpy.genfromtxt. It can be done but the process is much more complicated because it transforms  many strings to byte strings and it will take some major restructuring. For now the code raises a warning that pandas is required. For the future, we replace with numpy.genfromtxt and replace the warning. A relevant Issue will be opened.
Move globals from partitura.utils.music to separate script called partitura.utils.globals to avoid circular import errors. (This will be done in a following PR after this one is merged to not break other components).
Include infer root note, bass note and pcset from Roman Numeral and Chord Symbol strings. (This will be done together or after the globals PR)
Notebook and github discussion with documentation about RomanNumerals and Intervals and such

Codecov Report
Attention: 28 lines in your changes are missing coverage. Please review.

Comparison is base (8ee8ba9) 85.19% compared to head (62b297e) 84.46%.
Report is 2 commits behind head on develop.

Files
Patch %
Lines

partitura/utils/music.py
45.83%
13 Missing :warning:

partitura/score.py
92.07%
8 Missing :warning:

partitura/io/importdcml.py
95.52%
6 Missing :warning:

tests/test_dcml_import.py
92.30%
1 Missing :warning:

Additional details and impacted files
@@             Coverage Diff             @@
##           develop     #345      +/-   ##
===========================================
- Coverage    85.19%   84.46%   -0.73%
===========================================
  Files           81       83       +2
  Lines        13925    14190     +265
===========================================
+ Hits         11863    11986     +123
- Misses        2062     2204     +142

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

The PR is good, I'm requesting some cosmetic changes for better code readability.


Get RT and QT when they are in objects of type TweetWithVisibilityResults

resolve the same problem than this issue https://github.com/vladkens/twscrape/issues/53 but for tweets with limited replies that must be in retweetedTweet and quotedTweet keys. Must also resolve this issue: https://github.com/vladkens/twscrape/issues/55
I've also added a correction to RT rawContent reconstruction when truncated. (https://github.com/vladkens/twscrape/issues/74)
In that case top level mentionned users seems to be containing the username so it seems more correct to always use the username found in retweetedTweet.user.username (that also seems to be the new handle).

@vladkens a little review ? :)


[New API] : E-learning Portal

Brief Description 🤓
E-learning Platform
TechStacks
Typescript, Node Js, Express Js, Prisma, PostgresSQL , Node Mailer, Cloudinary API
API End Points
1. (2FA) Authentication Routes
/api/v0.1/auth/login                                      POST               //for login
/api/v0.1/auth/register                                   POST               //for register
/api/v0.1/auth/forgot-password                            GET                //for forgot password
/api/v0.1/auth/reset-password                             POST               //for reset password

2. Users Routes
/api/v0.1/users                                           GET                //for all users
/api/v0.1/users/:id                                       GET                //for a single user
/api/v0.1/users/:id                                       DELETE             //for delete a user
/api/v0.1/users/:id                                       PUT                //for update a user
/api/v0.1/users/:userId/enrollments                       GET                //for get all enrolled courses of a user

3. Course Routes
/api/v0.1/course                                          POST               //for create a course admin only
/api/v0.1/course?category=Python&level=Advance            GET                //for get all courses and you can add queries but its optional
/api/v0.1/course?category=JS&level=Beginner&page=1&pageLimit=3               //pagination with all the queries but its all optional
/api/v0.1/course/:id                                      PUT                //for update a course admin only
/api/v0.1/course/:id                                      DELETE             //for delete a course admin only
/api/v0.1/course/:id                                      GET                //for get a single course
/api/v0.1//course/trending                                GET                //for popular courses based on user enrollment
/api/v0.1/course/:courseId/enroll                         POST               //for enroll in a course
/api/v0.1/course/:courseId/leave                          DELETE             //for leave from a course
/api/v0.1/course/:courseId/reviews                        GET                //for getting all reviews of a course

4. Review Routes
/api/v0.1/review                                         POST               //for create a review
/api/v0.1/review/:id                                     DELETE             //for delete a review
/api/v0.1/review/:id                                     PUT                //for update a review

5. Profile Routes
/api/v0.1/users/:userId/profile                          POST               //for set profile and profile pic of a user
/api/v0.1/users/:userId/profile                          PUT                //for update profile and profile pic of a user

How will you develop it?🤩
I would develop it using the powers of Typescript and Express Js and my knowledge of APIs
Deployment
Yes
Select the open source program you are contributing to
Other
Code of Conduct

[X] I follow CONTRIBUTING GUIDELINES of this project.

Greetings @Puskar-Roy🎀 ! , We are excited to have you dive into another issue with us!🚀
Your involvement in our project has been invaluable, and we're confident that your skills and insights will help us conquer this challenge😇.  Use command /gdsc to self assign the issue.  This issue represents an exciting opportunity for us to improve and enhance our project, and we are thrilled to have you on board.
We request you to follow CONTRIBUTING GUIDELINES.
Looking for your PR soon!
In case of any issues, you can contact us on Discord.❣

/gdsc

Hey @Puskar-Roy ! I wanted to inform you that we have closed issue.Thank you for your understanding, and we look forward to your continued engagement with our repository🤗.
In case of any issues, you can contact us on Discord.
Thank you! ❣
Happy Coding! ✨
Will See you soon❣


Docker client panics when deploying compose file

When I deploy a file which contains a service connected to an external network, and I try
to provide an alias for the container on said connection, the docker client panics with a
segmentation fault.

Initialize a swarm
Deploy a compose file with network connection as described
Get a SIGSEGV!

Describe the results you received:
The following output:
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x10 pc=0x66e588]

goroutine 1 [running]:
panic(0xab4480, 0xc420012080)
        /usr/local/go/src/runtime/panic.go:500 +0x1a1
github.com/docker/docker/cli/command/stack.convertServiceNetworks(0xc4201a8420, 0x7ffe3ee2a82a, 0xd, 0xc4203ec118, 0x6, 0x0, 0x0, 0x0)
        /usr/src/docker/.gopath/src/github.com/docker/docker/cli/command/stack/deploy.go:255 +0x168
github.com/docker/docker/cli/command/stack.convertService(0x7ffe3ee2a82a, 0xd, 0xc4203ec118, 0x6, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
        /usr/src/docker/.gopath/src/github.com/docker/docker/cli/command/stack/deploy.go:519 +0x42e
github.com/docker/docker/cli/command/stack.convertServices(0x7ffe3ee2a82a, 0xd, 0xc42031b4d0, 0x7ffe3ee2a82a, 0xd, 0xc4201a8720)
        /usr/src/docker/.gopath/src/github.com/docker/docker/cli/command/stack/deploy.go:446 +0x15b
github.com/docker/docker/cli/command/stack.deployCompose(0xc4201d3440, 0x0, 0x0, 0x7ffe3ee2a817, 0x12, 0x7ffe3ee2a82a, 0xd, 0x0, 0x4, 0xc4203ec018)
        /usr/src/docker/.gopath/src/github.com/docker/docker/cli/command/stack/deploy.go:108 +0x572
github.com/docker/docker/cli/command/stack.runDeploy(0xc4201d3440, 0x0, 0x0, 0x7ffe3ee2a817, 0x12, 0x7ffe3ee2a82a, 0xd, 0x0, 0x0, 0x0)
        /usr/src/docker/.gopath/src/github.com/docker/docker/cli/command/stack/deploy.go:69 +0xe5
github.com/docker/docker/cli/command/stack.newDeployCommand.func1(0xc4203c98c0, 0xc4203c2e40, 0x1, 0x3, 0x0, 0x0)
        /usr/src/docker/.gopath/src/github.com/docker/docker/cli/command/stack/deploy.go:48 +0x93
github.com/docker/docker/vendor/github.com/spf13/cobra.(*Command).execute(0xc4203c98c0, 0xc42000c0c0, 0x3, 0x3, 0xc4203c98c0, 0xc42000c0c0)
        /usr/src/docker/.gopath/src/github.com/docker/docker/vendor/github.com/spf13/cobra/command.go:646 +0x26d
github.com/docker/docker/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0xc4200918c0, 0xc420091d40, 0xc420348f00, 0xc42002c440)
        /usr/src/docker/.gopath/src/github.com/docker/docker/vendor/github.com/spf13/cobra/command.go:742 +0x377
github.com/docker/docker/vendor/github.com/spf13/cobra.(*Command).Execute(0xc4200918c0, 0xc4200918c0, 0x10e7ce0)
        /usr/src/docker/.gopath/src/github.com/docker/docker/vendor/github.com/spf13/cobra/command.go:695 +0x2b
main.main()
        /usr/src/docker/.gopath/src/github.com/docker/docker/cmd/docker/docker.go:93 +0xcb

Describe the results you expected:
A successfully deployed compose file
Additional information you deem important (e.g. issue happens only occasionally):
Happens consistently
Output of docker version:
Client:
 Version:      1.13.0-rc1
 API version:  1.25
 Go version:   go1.7.3
 Git commit:   75fd88b
 Built:        Fri Nov 11 19:47:07 2016
 OS/Arch:      linux/amd64

Server:
 Version:             1.13.0-rc1
 API version:         1.25
 Minimum API version: 1.12
 Go version:          go1.7.3
 Git commit:          75fd88b
 Built:               Fri Nov 11 19:47:07 2016
 OS/Arch:             linux/amd64
 Experimental:        true

Output of docker info:
Containers: 129
 Running: 4
 Paused: 0
 Stopped: 125
Images: 296
Server Version: 1.13.0-rc1
Storage Driver: aufs
 Root Dir: /var/lib/docker/aufs
 Backing Filesystem: extfs
 Dirs: 912
 Dirperm1 Supported: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host macvlan null overlay
Swarm: active
 NodeID: 8agsrahzgub7m48dr66508iuv
 Is Manager: true
 ClusterID: 9l3fu4qrystn2d4h5dgly9uhc
 Managers: 1
 Nodes: 1
 Orchestration:
  Task History Retention Limit: 5
 Raft:
  Snapshot Interval: 10000
  Number of Old Snapshots to Retain: 0
  Heartbeat Tick: 1
  Election Tick: 3
 Dispatcher:
  Heartbeat Period: 5 seconds
 CA Configuration:
  Expiry Duration: 3 months
 Node Address: <IP_ADDRESS>
 Manager Addresses:
  <IP_ADDRESS>:2377
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 8517738ba4b82aff5662c97ca4627e7e4d03b531
runc version: ac031b5bf1cc92239461125f4c1ffb760522bbf2
init version: N/A (expected: v0.13.0)
Security Options:
 apparmor
 seccomp
  Profile: default
Kernel Version: 4.4.0-45-generic
Operating System: Ubuntu 16.04 LTS
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 15.56 GiB
Name: docker-x86-1
ID: NKCQ:BOL4:6JUP:BUHD:LU46:KLAQ:FDKZ:VP7I:U5FN:VYDI:OS2J:SP3K
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
WARNING: No swap limit support
Labels:
 provider=generic
Experimental: true
Insecure Registries:
 <IP_ADDRESS>/8
Live Restore Enabled: false

Additional environment details (AWS, VirtualBox, physical, etc.):
Physical machine running Ubuntu 16.04 LTS

After more extensive testing it appears to happen any time that you define a service's networks section with a dictionary rather than a list.

@techninja1008 could you provide the docker-compose.yml file you used ? :angel:

Ah nevermind, managed to reproduce, working on a fix.


A little Typo

https://github.com/networkservicemesh/networkservicemesh/blob/3ece13683fd36f2db60c4671a62251e7a4b86486/examples/cmd/vppagent-nsc/nsc.go#L65

:) can you propose a PR, it's worth it although small.

I made a patch set for it, and raised a PR.

FIxed in #1052


[feature] Add a way to edit entries

Probably in two scenarios: after submitting the entry (maybe an edit button next to the "saved" message) and from the data view page.
The data entries themselves do not have enough information to recreate the form that was used to make them. Instead, there could be a simple page with each (key, value) pair, where you could click to edit them. There should probably also be a way to add new fields.

In progress. Added an option to edit after entries are submitted.
Working on adding "edit" and "delete" buttons to entries in the data display. The current approach (bb46487) might not be ideal because the functions disappear when the row html is turned back into a string from a jQuery object. Might make more sense to add a field to each entry with the buttons that can just be written directly. (That might still run into the same problems. I'm not thinking through this rn.) Might just want to write out the buttons as strings with normal "onclick" attributes.
Will eventually need to add a delete function too.

Added a function to delete entries from the data table as of (a2e9c96). I think the one thing I want to do before merging this is to add an "undo" (or just "delete") button to the "Entry saved" toasts.

Actually probably not going to add an "undo" button. I was imagining it in a situation where the entry window was still open, but it would not be cool to accidentally click it after having left the entry.


Fix credential locker example

Hi,
this PR addresses the issue #3045. It implements the proposed solution by catching the exception thrown by vault.FindAllByResource and returns null.

@SvenLauterbach : Thanks for your contribution! The author(s) have been notified to review your proposed change.

Actually, it would be better to update the behavior of the FindAllByResource method, because not finding a requested resource it not something unexpected, but I didn't know where I could address this and I also think, this might be harder to fix than the example.

The first one is easy, but the second suggestion is hard, since I don't know, which exception is thrown if there is no resource found. I event tried ex.GetType().FullName which yield System.Exception, so this seems to be the most specific exception.
The best I could do to narrow the exception down to this specific exception is:
catch(Exception ex) when (ex.Message.EndsWith("Cannot find credential in Vault"))
{
    return null;
}

Would this be ok?

@SvenLauterbach : Thanks for your contribution! The author(s) have been notified to review your proposed change.

@SvenLauterbach : Thanks for your contribution! The author(s) have been notified to review your proposed change.


Support for changing user info in Firebase Auth plugin

The Firebase Auth plugin is missing support for changing user information.

FYI @collinjackson

Ya, where is user.updateProfile?

@LiveRock https://github.com/flutter/plugins/pull/222/files#diff-2ed885c78d229177455b276c34e93577R234

Has this been resolved ?

@ueabu ensure you have the latest version of the plugin.
See https://github.com/flutter/plugins/blob/master/packages/firebase_auth/test/firebase_auth_test.dart#L774

Why it isn't possible to update a phone number?

@lschmierer
This issue has been moved to https://github.com/FirebaseExtended/flutterfire/issues/1001. Any further collaboration will be done there.


Support for footnote is incomplete in Jupinx

It seems footnotes are failed in Jupinx.
For example dyn_stack:

code:
Define the government's one-period loss function [#f1]_

.. rubric:: Footnotes

.. [#f1] The problem assumes that there are no cross products between states and controls in the return function.  A simple transformation  converts a problem whose return function has cross products into an equivalent problem that has no cross products. For example, see :cite:`HansenSargent2008` (chapter 4, pp. 72-73).

.. [#f2] The government would make different choices were it to choose sequentially, that is,  were it to select its time :math:`t` action at time :math:`t`. See the lecture on :doc:`history dependent policies <hist_dep_policies>`

.. [#f8] :cite:`HansenSargent2008` (chapter 16), uses this model as a laboratory to illustrate an equilibrium concept featuring robustness in which at least one of the agents has doubts about the stochastic specification of the demand shock process.

.. [#f9] They used this method to compute a rational expectations competitive equilibrium.  Their key step was to eliminate price and output by substituting from the inverse demand curve and the production function into the firm's first-order conditions to get a difference equation in capital.

.. [#f10] These calculations were performed by functions located in `dyn_stack/oligopoly.py <https://github.com/QuantEcon/QuantEcon.lectures.code/blob/master/dyn_stack/oligopoly.py>`_.

.. [#f12] For another application of the techniques in this lecture and how they related to the method recommended by :cite:`KydlandPrescott1980`, please see :doc:`this lecture <hist_dep_policies>` .

This PR should fix this issue: https://github.com/QuantEcon/sphinxcontrib-jupyter/pull/99

[ ] jupinx should be updated to include clickable footnotes to return to the actual footnote location in the text.

@DrDrij still not sure how to support rendering of the footnotes (within the text). The  markup is passing through but the markdown (which is embedded in the html) is clearly not rendering via nbconvert. Is there a way to fix this in html -- otherwise I may need to add html links in the sup. Could you let me know what you would need (just a <aref>) and I will update jupinx.

@mmcky No way to reliably do it with the HTML that is generated.
So a footnote reference such as [2](#f2)
Should be rendered as [2]

this should be fixed by: https://github.com/QuantEcon/sphinxcontrib-jupyter/pull/119

@fkazemian would you be able to check the footnotes fix? It should now be live on compare.

Hi @mmcky sure

in the lecture they are fine:

but we can't have access to the related text by footnotes:

(amss)

@jstac do you want click back capability from a footnote to the referenced location in the text?
@fkazemian can you let me know what the html is for the click backs?

@mmcky Here's my suggestion:
Current inline footnote link:

That would let us reduce the beginning-of-period indebtedness for some other history [2]

Current footnote:

[2] From the first-order conditions for the Ramsey problem, there exists another realization s̃ t with the same history up until the previous period, i.e., s̃ t−1=st−1, but where the multiplier on constraint (11) takes a positive value, so γt(s̃ t)>0.

Proposed inline footnote link:

That would let us reduce the beginning-of-period indebtedness for some other history [2]

Proposed footnote:

[2] From the first-order conditions for the Ramsey problem, there exists another realization s̃ t with the same history up until the previous period, i.e., s̃ t−1=st−1, but where the multiplier on constraint (11) takes a positive value, so γt(s̃ t)>0.

Hi
It's ```

[1] ........
```
I add` [1]` inside `` tags:
![screenshot from 2018-10-04 11-36-42](https://user-images.githubusercontent.com/20714542/46448412-0a01a000-c7ca-11e8-8f46-aa55dd4d49c7.png)
I think If we delete [1] and add[1]inside  tags it should be fine.

thanks @DrDrij and @fkazemian. This issue should now by fixed by: https://github.com/QuantEcon/sphinxcontrib-jupyter/pull/126. Rerunning compiler now and will upload to compare-lectures shortly.

@fkazemian can you please check this now. Thanks.

Hi @mmcky , sure

@mmcky , Thanks it just works fine, in compare with the live version,  index in footnotes are bold and you use tag  for them, It which is not an issue, I change the label to fixed.

thanks @fkazemian this is great. I will close the issue.
@jstac are you happy with the bold footnote numbers here?


811366A lists many old deprecated courses as prerequisites

811366A Projektitoiminta lists many old deprecated courses as prerequisites

Course itself was deprecated, fixed.


Specifying width or height sets it on all nodes

We need to set width and height on our SVG icons in order to avoid bad behavior by Internet Explorer. See https://thatemil.com/blog/2015/03/15/sizing-svg-background-images-in-internet-explorer/
However, when I set width and height in the SVG URL querystring, it applies width and height to all nodes, including RECT, which mangles the image. I really just want to apply width and height to the SVG node.
Would it be an acceptable change to svg-transform-loader to only apply width/height attributes to the SVG node, and not apply it to any other node?

Please read the docs

Thanks. Sorry. I missed the detail about providing a selector


windows icon on desktop looks pixely

there is a problem displaying 96x96 icons on the desktop. find a way to fix it.

resolved in https://github.com/arzynik/wowstat/releases/tag/3.0.2


Update duplication threshold

Description
Updates CodeClimate duplication threshold values.
Type of change

[x] Bug fix (non-breaking change which fixes an issue)

Change Status

[x] Complete, tested, ready to review and merge

How Has This Been Tested?

[x] Config params added by Bernie so it should work fine.

Checklist

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[x] My code has been reviewed by at least one peer
[x] I have commented my code, particularly in hard-to-understand areas
[x] I have made corresponding changes to the documentation
[x] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[x] New and existing unit tests pass locally with my changes
[x] There are no merge conflicts

 CodeClimate Duplication Threshold Update


chore: fix unencrypted logs mismatch for AVM with a +4

Benchmark results
No base data found for comparison.

Detailed results
All benchmarks are run on txs on the Benchmarking contract on the repository. Each tx consists of a batch call  to create_note and increment_balance, which guarantees that each tx has a private call, a nested private call, a public call, and a nested public call, as well as an emitted private note, an unencrypted log, and public storage read and write.
This benchmark source data is available in JSON format on S3 here.
Proof generation
Each column represents the number of threads used in proof generation.

Metric
1 threads
4 threads
16 threads
32 threads
64 threads

proof_construction_time_sha256
5,707
1,552
712
778
782

L2 block published to L1
Each column represents the number of txs on an L2 block published to L1.

Metric
8 txs
32 txs
64 txs

l1_rollup_calldata_size_in_bytes
772
772
772

l1_rollup_calldata_gas
6,856
6,856
6,856

l1_rollup_execution_gas
587,396
587,396
587,396

l2_block_processing_time_in_ms
1,365
5,145
10,081

l2_block_building_time_in_ms
32,584
129,165
256,934

l2_block_rollup_simulation_time_in_ms
32,400
128,521
255,686

l2_block_public_tx_process_time_in_ms
20,339
85,361
168,437

L2 chain processing
Each column represents the number of blocks on the L2 chain where each block has 16 txs.

Metric
5 blocks
10 blocks

node_history_sync_time_in_ms
15,126
28,368

node_database_size_in_bytes
21,323,856
37,965,904

pxe_database_size_in_bytes
29,868
59,425

Circuits stats
Stats on running time and I/O sizes collected for every kernel circuit run across all benchmarks.

Circuit
protocol_circuit_simulation_time_in_ms
protocol_circuit_witness_generation_time_in_ms
protocol_circuit_proving_time_in_ms
protocol_circuit_input_size_in_bytes
protocol_circuit_output_size_in_bytes
protocol_circuit_proof_size_in_bytes
protocol_circuit_num_public_inputs
protocol_circuit_size_in_gates

private-kernel-init
158
3,650
22,118
19,985
61,999
86,720
2,643
1,048,576

private-kernel-inner
609
4,830
40,668
89,053
61,999
86,720
2,643
2,097,152

private-kernel-reset-small
576
2,397
23,307
117,961
61,999
86,720
2,643
1,048,576

private-kernel-tail
532
4,031
37,854
86,849
79,454
10,688
267
2,097,152

base-parity
6.60
951
3,583
128
64.0
2,208
2.00
131,072

root-parity
49.7
67.5
40,899
27,064
64.0
2,720
18.0
2,097,152

base-rollup
783
2,301
43,599
111,158
957
3,136
31.0
2,097,152

root-rollup
94.8
50.1
8,322
11,518
821
3,456
41.0
524,288

public-kernel-app-logic
245
134
815
96,978
85,095
116,448
3,572
4,096

public-kernel-tail
871
660
967
388,207
7,755
10,176
251
512

public-kernel-setup
227
167
919
138,309
85,095
116,448
3,572
4,096

public-kernel-teardown
231
211
1,070
143,320
85,095
116,448
3,572
4,096

merge-rollup
6.50
32.9
2,243
2,760
957
3,136
31.0
65,536

private-kernel-tail-to-public
N/A
9,464
72,934
N/A
N/A
116,960
3,588
4,194,304

Stats on running time collected for app circuits

Function
app_circuit_proof_size_in_bytes
app_circuit_proving_time_in_ms
app_circuit_size_in_gates
app_circuit_num_public_inputs

SchnorrAccount:entrypoint
16,128
47,891
2,097,152
437

Test:emit_nullifier
16,128
2,711
65,536
437

FPC:fee_entrypoint_public
16,128
16,579
524,288
437

FPC:fee_entrypoint_private
16,128
8,930
524,288
437

Token:unshield
16,128
50,659
2,097,152
437

SchnorrAccount:spend_private_authwit
16,128
2,706
131,072
437

Token:transfer
16,128
35,097
2,097,152
437

Tree insertion stats
The duration to insert a fixed batch of leaves into each tree type.

Metric
1 leaves
16 leaves
64 leaves
128 leaves
512 leaves
1024 leaves
2048 leaves
4096 leaves
32 leaves

batch_insert_into_append_only_tree_16_depth_ms
11.2
18.4
N/A
N/A
N/A
N/A
N/A
N/A
N/A

batch_insert_into_append_only_tree_16_depth_hash_count
16.7
31.8
N/A
N/A
N/A
N/A
N/A
N/A
N/A

batch_insert_into_append_only_tree_16_depth_hash_ms
0.655
0.564
N/A
N/A
N/A
N/A
N/A
N/A
N/A

batch_insert_into_append_only_tree_32_depth_ms
N/A
N/A
52.0
81.7
266
505
1,003
1,950
N/A

batch_insert_into_append_only_tree_32_depth_hash_count
N/A
N/A
95.9
159
543
1,055
2,079
4,127
N/A

batch_insert_into_append_only_tree_32_depth_hash_ms
N/A
N/A
0.532
0.504
0.481
0.472
0.475
0.467
N/A

batch_insert_into_indexed_tree_20_depth_ms
N/A
N/A
62.5
120
377
740
1,495
2,924
N/A

batch_insert_into_indexed_tree_20_depth_hash_count
N/A
N/A
106
208
692
1,363
2,707
5,395
N/A

batch_insert_into_indexed_tree_20_depth_hash_ms
N/A
N/A
0.544
0.537
0.512
0.509
0.517
0.509
N/A

batch_insert_into_indexed_tree_40_depth_ms
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
67.6

batch_insert_into_indexed_tree_40_depth_hash_count
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
108

batch_insert_into_indexed_tree_40_depth_hash_ms
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
0.592

Miscellaneous
Transaction sizes based on how many contract classes are registered in the tx.

Metric
0 registered classes
1 registered classes

tx_size_in_bytes
84,741
673,219

Transaction size based on fee payment method
| Metric |  |
| - |  |
Transaction processing duration by data writes.

Metric
0 new note hashes
1 new note hashes
2 new note hashes

tx_pxe_processing_time_ms
29,649
4,316
100,799

Metric
0 public data writes
1 public data writes
2 public data writes
4 public data writes
8 public data writes

tx_sequencer_processing_time_ms
1,403
2,729
2,090
3,259
1,990


[4.3] Bug 1820242: nodeca daemon should tolerate all taints

Backport of #457.

/assign @ricardomaraschini

Same concern as https://github.com/openshift/cluster-image-registry-operator/pull/473#issuecomment-608778915.

@crawford per discussion in https://github.com/openshift/cluster-image-registry-operator/pull/473#issuecomment-610884982 - this is how the CAs get onto nodes in 4.x today. Any change in this behavior needs to be addressed as a new feature.

does the operator itself report an error if the pod is unable to start?
trying to figure out how this intercepts with taint based evictions.
see: https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/#taint-based-evictions

@derekwaynecarr the operator doesn't monitor this daemonset. If you have a better idea how to keep this daemonset on nodes for kibana, elasticsearch, etc. that have NoExecute taints, feel free to open a new BZ against 4.5.

/bugzilla refresh

/retest

/cherrypick release-4.2


Batch K-Means optimization

Description
Please include a summary of the change. For large or complex changes please include enough information to introduce your change and explain motivation for it.
Changes proposed in this pull request:

/intelci: run

Don't think that i can make a good review here. But the comment i have - description of this change is empty

It'd be good if you provide information about performance gains made by these optimizations

/intelci: run

/intelci: run

/intelci: run

/intelci: run


Aligned Human Genome couldn't convert to Adam

I did this steps-

Index : Reference human genome using bow tie ( A. fna)
Align : A. fna and A.fastq-- Done successfully--Got the output in A.fastq

When I am trying to convert A.fastq(Aligned) to adam , its not working .
Any suggestion will be helpful.
Thanks

Index : I used Bow tie for index reference genome
Alignment: data (.fastq ) with the indexed reference genome
Output : .fastq file (Aligned with index)

But whenever  I m trying to convert  .fastq file (Aligned with index) to adam using the command:
./adam-submit transformAlignments hdfs://ip-10-48-3-5.ips.local:8020/user/rokshan.jahan/data/SRR1517848.fastq hdfs://ip-10-48-3-5.ips.local:8020/user/rokshan.jahan/SRR1517848.adam
It didnt give the correct parquet files inside .adam

You shouldn't be getting a fastq file out of Bowtie, you should be getting a SAM file, so that part of the workflow looks suspicious. What error are you getting when you run transformAlignments?

I got the .adam . There was parquet files. But Data was not correct

I used BWA for indexing and alignment
Did this steps:

Indexed the reference genome
Align the .fastq with the  Indexed the reference genome and got the output as .sam
Convert the .sam to .adam
Output was correct
But my concern is : I want to skip the .sam conversion,
**   I want to align the .fastq with the  Indexed  reference genome and got the output as .adam

Is there any way I can do that?


doc(README): update

I have improved README with the help of grammarly and DeepL.

Nice! Thank you!


Force HPP-FCL bindings dependency

Currently,  Pinocchio Python bindings can be built:

Without FCL
With FCL, but without FCL bindings
With FCL, including the bindings

I believe the second option is unreasonable and only adds complexity. If Python bindings are built with HPP-FCL, then HPP-FCL bindings should be enforced. Ideally, we should add something like this to the CMakeLists.txt:
IF(BUILD_PYTHON_INTERFACE AND BUILD_WITH_COLLISION_SUPPORT)
  ADD_REQUIRED_DEPENDENCY("hpp-fcl-python")
ENDIF(BUILD_PYTHON_INTERFACE AND BUILD_WITH_COLLISION_SUPPORT)

so we need a way to detect hpp-fcl Python bindings via cmake.
What do you think?

We cannot check FCL bindings with pkg-config. I mean this is not a standard way of doing it. In addition, we are not sure yet to get the latest FCL version on all the platforms.
@nim65s Is hpp-fcl latest release available on robotpkg?
To do it correctly, I would suggest using directly the packaging of CMake, which allows exporting additional targets, such as the Python bindings.

In addition, we are not sure yet to get the latest FCL version on all the platforms.

For this, it will be enough to do a new HPP-FCL release and then increase the minimum version required by Pinocchio. This way everybody using HPP-FCL will be forced to upgrade (I do not see why anybody should want to keep old versions).

To do it correctly, I would suggest using directly the packaging of CMake, which allows exporting additional targets, such as the Python bindings.

Yes, I think this may be the correct way.

Changing the minimal version number is indeed in the pipe. As I mentioned previously,I’m waiting the update from robot from robotpkg first. I will close the issue and I will take care of all it soon.

On robotpkg, hpp-fcl is on v1.2.1. The current latest is 1.2.2. I'll update it.

@jcarpent  1.2.2 is available

@nim65s Thanks a lot!


[BUG] setState() is not get correct value

Description
I found an issue in components/user/UserLink.js line number 29. Here we update state based on previous state, but current implementation of setState() is not correct.
Because,
Whenever we update state and that depend on the previous state, for example, as we do it here components/user/UserLink.js line number 29. We should not do it like this but we should use an alternative form of this state updating function.

Why we shouldn't do it like this?
In many cases it will work fine, but according to React docs, React schedules state updates, it doesn't perform them instantly. And therefore, theoretically, if we schedule a lot of state updates at a same time, we could be depending on outdated or incorrect state snapshot if we use this approach.
Screenshots
No response
Additional information
Would love fix this if this a valid issue.
Thanks

Good spot, but I think that code is no longer required, as the clicks are not longer displayed on the profile page and if I remember correctly this was added just to reduce the extra call to the API to get the latest click statistics for that link.
So basically this code can now be removed.

hey @eddiejaoude, here it might not concern

Good spot, but I think that code is no longer required, as the clicks are not longer displayed on the profile page and if I remember correctly this was added just to reduce the extra call to the API to get the latest click statistics for that link.
So basically this code can now be removed.
I have assigned it to you @badrivlog

thank you for giving me a chance to contribute


no hacks work

update you hacks

YOOOOOOOOO RXZYX FIX YO CODE

Bruh.

Learn how to realize that the dev isn't hard working because the cheats aren't working, before you criticize other people.

yes it do

Learn how to realize that the dev isn't hard working because the cheats aren't working, before you criticize other people.

Are u ed? its patched


Comments button

@leanne1 @jon-ec things done here:

Remove firebase config from git
Remove inline comments, only the comments button left which in theory will focus the Blabbr tab but right not the API doesn't have anything for it
Reshuffle comments to have add comment at the bottom, after talking to Marcos (ie, read the comments before you submit anything).

👍


Deserializing array of arbitrary values

I am interested in deserializing arbitrary values contained in JSON array. The contents and data types inside the array may change anytime.
The objective is to deserialize data so it may be further processed and stored to disk using more effective serialization. The performance is key factor here, which brought me to investigate Jil.
Using Json.NET this can be achieved quite easily.
        string json = @"[1, true, 1.2345, null, ""test""]";
        var values = JsonConvert.DeserializeObject<object[]>(json);

Values array would then contain the values as .NET (boxed) primitive types (long, bool, double, object, string). The array would then be really easy to use for further processing and efficient storage.
However the Json.NET deserialization has quite large performance overhead.
Deserializing the same data with Jil seems to produce dynamic JsonObjects. I am having hard time figuring out what to do with the dynamic objects to get to the "raw" data. Is there feasible to way to achieve what I am trying to do using Jil?

It works, but it's quite a bit of work and quite error prone, especially if you don't know anything about the types you are deserializing (e.g. your null value above, decimal vs. double, Guids etc.).
The steps are:

obtain typeconverter:  var converter = TypeDescriptor.GetConverter(dynamic);
Find out if the dynamic is a complex object by converting it to a Dictionary<string,object>
Find out if the dynamic is an array by converting it to an array etc.
Find out if the dynamic is a string, value types, primitve types by converting it to string etc.

And repeat that until your done, TypeConverters have a method CanConvertTo which will try the conversion without throwing an exception. This is not designed for performance e.g. CanConvertTo then doing ConvertTo basically amounts to doing the conversion twice. Personally I wouldn't try this if you don't know the types you're deserializing before hand, simply because JSON is not as typesafe as C#.

Thanks Tornhoof. I did try your suggestion and indeed I was able to get it working quite similarly as Json.NET. However the conversions via the TypeConverter absolutely ruined the performance.

Yes, as I wrote, it does the conversion twice, if you know the types, you can skip the CanConvertTo step and it's pretty much twice as fast, but even then it's only marginally faster than JSON.NET.


Fix backslash line endings for grpc_python/Dockerfile.

Without the correct line endings docker had errors interpreting '&&' as a command.

I have signed the CLA and am a Google employee - see previous pull request with cla:yes : https://github.com/Polymer/platform-dev/pull/50

Change content looks good; principle demands that I wait for results from continuous integration... which probably isn't even affected by the content of this change. :-P

The travis tests won't cover building docker files :-)


Increase DEFAULT_CHUNK_SIZE to reduce transfer overhead

Fixes #86 by setting the DEFAULT_CHUNK_SIZE to 15MiB.

Hey just wanted to check in -- are there any steps I need to take to move this forward?

I did approve, but my approval doesn't count IIUC.


Any license for this repo since I want to reuse your code

Thank you for your interest in this repo.
I added license.

Thank u so much, and close this issue


make cacheSearchResultsForSeconds example consistent with ts-types

The example in the README.md is inconsistent with the provided Typescript types.
Change Summary
moved
cacheSearchResultsForSeconds: 2 * 60, // Cache search results from server. Defaults to 2 minutes. Set to 0 to disable caching
outside of the server property.
PR Checklist

[x] I have read and signed the Contributor License Agreement.

Thank you for catching that @valerius21!

@valerius21 It turns out that the types were actually wrong. cacheSearchResultsForSeconds actually needs to be inside the server key. I've pushed out a fix in v2.4.0-0.


[Developer Experience]: Add Cypress Test for 'Contact Us' Image Component

Developer Experience Issue
Please describe your Developer Experience Issue.
A clear and concise description of what the problem is. Ex. I have an issue when [...]
Add Cypress test for 'Contact Us' Image Component.
Describe the solution you'd like
A clear and concise description of what you want to happen. Add any considered drawbacks.
Add a Cypress test that would:

Test that the image renders properly on Desktop
Test that a background image renders properly on mobile

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
n/a
Teachability, Documentation, Adoption, Migration Strategy
If you can, explain how users will be able to use this and possibly write out a version in the docs.
Maybe a screenshot or design?
n/a

I'll work on this

@chadstewart can I work on this?

@chadstewart can I work on this?

Sure @ciradu2204, feel free!

Great, thanks

Resolved in #141


Cleanup Container loading code

There is a bunch of logic in Container.load() that can be easily implemented as a layer on top of core layer, not as part of core layer.
This change removes such concepts as

IContainerLoadMode.opsBeforeReturn ===  "sequenceNumber"
IContainerLoadMode.pauseAfterLoad
LoaderHeader.sequenceNumber

Improvements:

There is no more overlap with pending state options - it was possible (before) to ask to load from pending state, and up to sequence number - that would not work correctly.
It was possible before to ask for non-paused, but loaded up to some sequence number Container - it would happily return container loaded from snapshot that has higher sequence number than the one where we needed to stop. This combo is no longer possible.
Our E2E tests were covering some combination that do not matter and that should not have been existed.
No need for a bunch of checks in Loader layer around non-valid combos or types.
There is reference to https://dev.azure.com/fluidframework/internal/_workitems/edit/5055 in the code. That issue should be resolved with this PR, as we set op handler before we process any ops (including cached ops) with this PR.

What is not fixed / done:

This function should move somewhere else. Not sure where.
This logic will hang if container gets disposed while processing ops - there are no checks for that.
Logic around checking container.deltaManager.inbound.length looks fishy - I have not touched it though.
Eventually, we should remove ability to reach out into the guts of Container (DeltaManager, pausing queues). We will need to figure out how to implement scenarios that needs to be implemented and ban the rest.

We discussed this implementation option back when the feature was originally built. My main concern then was basically the same as it is now, and is basically your fourth bullet point you mention above: "do we want to add more dependencies on 'reaching into the guts of Container'?"
It's nice to externalize functionality that not everyone uses (esp. for the bundle size win!), but I think this change comes with some added risk of hidden fragility, that we have more cases outside of the Container/Loader classes to find and consider when changing these internals' behaviors.
I'm OK with moving to this standalone function approach, but I wouldn't call this a layer separation. It's not really a separate layer on top of the core layer if it must reach down into the internals of the layer beneath. So re: where the function goes: I'd keep this function in container-loader until it can be separated from container-loader guts.

Here is my train of thought:

As much as possible, try not to create new modalities. Can you use Container.close() instead of pausing? I.e. why container needs to be in some different mode of operation, given that it's in somewhat broken state (read on)?

if the answer is yes, then there is absolutely no non-public APIs used here.

If you need pause() state / API, add it properly as public API. This will force you deal with spec complexity, to very clearly articulate how this functionally works in presence of other modalities / functionality.

if we have pause(), should there be unpause()?
Can user call something like waitContainerToCatchUp()? Container.connect()? Container.disconnect() in this mode? What is the behavior? Do we document all the limitations? Do we have tests to cover that spec complexity?
Why does it have to have delta connection at the end of this process, and what it would mean to lose connection and get stuck in "CatchingUp" state (we can't move out of it as we do not process ops)? What do we do with telemetry noise as eventually we will see errors complain about not being able to move to "Connected" state after some time.

Can we reuse some of the existing modalities, like close container on reconnect (summarizer mode).

What is the intersection of that feature with loading from pending state? Using other connection / catchup options?
Is forking / branching a better direction here if closed container does not cut it? Sooner or later, we will have to build it. Tony believes it's not that hard to build, and he even has FHL-level prototype demonstrating summarizer running off runtime implemented branching. It by itself does not solve any of the problems, but it creates an environment where it's Ok for local agent to control "global" properties like stop inbounding changes/ops.
FYI - Data virtualization will be broken if container is always paused.

You would need to (eventually) build progress reporting and cancelation into this flow.

This is generic issue with FF, we are limping along, but this flow creates unique challenges.
Container load never retries network errors. If we fail (including offline), we return back error to user, letting end user retry (if they feel like it). We do so because user does not have anything to work with - Container object is only returned once load is over, so there is nothing to observe progress / cancel it if we were using retry loop there.
All other flows (including socket connection, ops fetching) use infinite retry loops and provide some way to observe / cancel those (Container.disconnect(), Conainer.close()). See test coverage in recent PRs #20627, #20547 for future Loop flows in this area - real apps that run at scale do care about user experience in those cases that happen very infrequently, but impact user impression a lot.
This flow is somewhere in between of what we have. It's load, but it needs ops, and thus it could run into infinite retry loop if user is offline, while not providing any way for caller to observe its status / cancel request.
At a minimum, we should add AbortSignal as an argument into this flow and use it.

Unless we are planning to invest a lot of resources here to make it better, I'd rather prefer this API to be marked @internal and/or @experimental. Having it on the public surface is a bit frightening.

Can you use Container.close() instead of pausing?

Unclear but I doubt it - I don't think closed state is sufficiently well-defined to know whether it's a good option.  Its meaning differs through the layers and has lots of side effects.

if the answer is yes, then there is absolutely no non-public APIs used here.

I'd also say listening to "op", access of the container.deltaManager in general, and direct inspection of the deltaManager's various sequence number properties are all things that probably shouldn't be public.
I think the rest of your comment is just general problems with the normal load flow, which I agree needs work.


update html_root_url for env_logger

Closes: #141
@dtolnay there is a documentation line in env/Cargo.toml here : https://github.com/morrme/log/blob/issue141/env/Cargo.toml#L7
Should that be updated as well?

Yes. I reopened #139 to track that.


Fix missing OpenImageIO error message if film output buffer write fails

To retrieve the actual error message for OpenImageIO when the buffer write fails, you have to use the geterror method of the buffer object. Using the global geterror function is for special cases only and returns an empty string when used in this case.

Thanks, merged.