Paginator Next Request URL help

Howdy,
Not sure if I'm confused or not but I'm trying to use the Microsoft Graph API to get a list of users.
When I make a request I get a @odata.nextLink field in the JSON response.
This URL is similar to the original request endpoint but has a $skiptoken or a $skip query parameter added.
I'm looking at custom Pagination but I can't work out how to modify the $request to use the URL in the @odata.nextLink field.
I assume this should be done in the applyPagination function but just not sure how best to change the URL.
protected function applyPagination(Request $request): Request
{
    if ($this->currentResponse instanceof Response) {
        // Something like this?
        $request->setEndpoint($this->currentResponse->json('@odata.nextLink'));
    }

    return $request;
}

Microsoft say:

Important
You should include the entire URL in the @odata.nextLink property in your request for the next page of results. Depending on the API that the query is being performed against, the @odata.nextLink URL value will contain either a $skiptoken or a $skip query parameter. The URL also contains all the other query parameters present in the original request. Do not try to extract the $skiptoken or $skip value and use it in a different request.

... so I'm hesitant to go against that and extract the parameter and append it using the CursorPaginator.
Any help would be appreciated.
Thanks
Ben

FYI - I have since tested extracting the $skiptoken parameter using the Spatie URL package and that worked.
However I'd still like to know if it's possible to change the full URL in the $request?
protected function getNextCursor(Response $response): int|string
{
    $url = Url::fromString($response->json('@odata.nextLink'));
    return $url->getQueryParameter('$skiptoken'); // <-- might need to check both $skip and $skiptoken
}

//...

protected function applyPagination(Request $request): Request
{
    if ($this->currentResponse instanceof Response) {
        $request->query()->add('$skiptoken', $this->getNextCursor($this->currentResponse)); // <-- might need to use $skip instead $skiptoken in some cases
    }

    if (isset($this->perPageLimit)) {
        $request->query()->add('$top', $this->perPageLimit);
    }

    return $request;
}

Hey @avenjamin firstly I want to apologise for being slow to reply to your message.
I'm going to see if there's a way I can introduce the ability to overwrite the URL on a per-request basis for situations like these. However I think you are on the right track with modifying the query parameters.
Do you have an example of how the @odata.nextLink value?

Hey @Sammyjo20 - no worries :)
That would be great if you could - I'm sure this won't be the only time someone will want this feature.
The original URL is:
https://graph.microsoft.com/v1.0/users?$select=id,mail,accountEnabled,displayName,department
And the @odata.nextLink URL is:
https://graph.microsoft.com/v1.0/users?$select=id%2cmail%2caccountEnabled%2cdisplayName%2cdepartment&$skiptoken=RFNwdAIAAQA....snip....GG5AAAAA
As you can see the URL is practically the same just with some URL escaping and the addition of the $skiptoken.
I'm only hesitant due to what Microsoft say in the docs which I'm reading as don't modify the URL basically and that in some instances they use $skip instead.
I suppose Microsoft could potentially set the $skiptoken to a different server hostname for load balancing. e.g. graph-na-01.microsoft.com but I haven't seen them do this yet:

Important
You should include the entire URL in the @odata.nextLink property in your request for the next page of results. Depending on the API that the query is being performed against, the @odata.nextLink URL value will contain either a $skiptoken or a $skip query parameter. The URL also contains all the other query parameters present in the original request. Do not try to extract the $skiptoken or $skip value and use it in a different request.

Hey @avenjamin
Thanks for the information - I've had a look into building a new url() API for Saloon's requests/connectors but I think it's going to be too difficult to retrofit it into v3 for everyone. I would recommend that you add your own setEndpoint() method on your request, which internally sets a property. When the endpoint is a full URL it'll be preferred over the base URL defined on the connector.
class YourRequest extends Request
{
     protected string $endpoint = '/default';

     protected function resolveEndpoint()
     {
         return $this->endpoint;
     }

     public function setEndpoint(string $endpoint): void
     {
         $this->endpoint = $endpoint;
     }
}

// ...

protected function applyPagination(Request $request): Request
{
    if ($this->currentResponse instanceof Response) {
        $request->setEndpoint($this->currentResponse->json('@odata.nextLink'));
    }

    // ...
}

Thanks @Sammyjo20 - maybe in v4 :)
I did as you suggested and it's working well so far.
I also just used the same technique for another API which has the page number as part of the URL (not a query parameter).
We can close this for now.
Cheers
Ben


migration from spring-cloud-netflix core to spring clound-netflix Zuul -- Caused by: org.springframework.beans.factory.BeanDefinitionStoreException: Failed to process import candidates for configuration class [com.XXXX.XXXXX.XXXXXX]; nested exception is java.io.FileNotFoundException: class path resource [org/springframework/cloud/commons/httpclient/HttpClientConfiguration.class] cannot be opened because it does not exist

Describe the bug
to resolve security vulnerability migration from spring-cloud-netflix core to spring clound-netflix Zuul then i got error like
Caused by: org.springframework.beans.factory.BeanDefinitionStoreException: Failed to process import candidates for configuration class [com.XXXX.XXXXX.XXXXXX]; nested exception is java.io.FileNotFoundException: class path resource [org/springframework/cloud/commons/httpclient/HttpClientConfiguration.class] cannot be opened because it does not exist
Sample
If possible, please provide a test case or sample application that reproduces
the problem. This makes it much easier for us to diagnose the problem and to verify that
we have fixed it.

Can you provide a complete, minimal, verifiable sample that reproduces the problem? It should be available as a GitHub (or similar) project or attached to this issue as a zip file.

If you would like us to look at this issue, please provide the requested information. If the information is not provided within the next 7 days this issue will be closed.

Closing due to lack of requested feedback. If you would like us to look at this issue, please provide the requested information and we will re-open the issue.


Don't Merge! PR real time test for webhook case

Docs Build status updates of commit 817a06e:
:clock10: Pending: waiting for processors (26 builds ahead of you)
:warning: Docs Build is busy, currently there are 26 builds ahead of this one, for more information you can view the Build queue graph on the Docs Portal.

Docs Build status updates of commit 817a06e:
:clock10: Full build: the average full build time is 5 min(s) 55 sec(s), this is based on the last 770 build(s) for this repository.

Docs Build status updates of commit 817a06e:
:warning: Validation status: warnings

File
Status
Preview URL
Details

docs-ref-conceptual/testing.md
:warning:Warning
View (azure-cli-latest) View (azure-cli-2017-03-09-profile) View (azure-cli-2018-03-01-hybrid)
Details

docs-ref-conceptual/testing.md

[Warning] Missing attribute: title. Add a title string to show in search engine results.
[Warning] Missing attribute: title. Add a title string to show in search engine results.
[Warning] Missing attribute: title. Add a title string to show in search engine results.

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Merging multiple different NotificationPolicies

We have the following use case:

Initially created NotificationPolicy inside of our cluster
New team needs new Lines to be added to this NotificationPolicy.policy
We do not want to manage the NotificationPolicy centrally

Our desired workflow would include a new CRD ( maybe NotficationPolicyPatch ?? ) that allows us to specify only a desired addition to the NotificationPolicy.policy field.
The provider would then need to made aware of this, and dynamically recreate the NotificationPolicy.policy field from the initial state and any additions made in NotficationPolicyPatch resources during reconciliation
Would such a feature be desired on this provider?

You can now specify notification policies for each alert rule. See the notificationSettings of the rule group resource: https://marketplace.upbound.io/providers/grafana/provider-grafana/v0.13.2/resources/alerting.grafana.crossplane.io/RuleGroup/v1alpha1#doc:spec-forProvider-rule-notificationSettings
See https://grafana.com/docs/grafana/latest/alerting/alerting-rules/create-grafana-managed-rule/#configure-notifications-simplified for more info


[Integration Tests] [BlockStoreSignaled] Additional tests

These tests primarily relate to detecting the presence or absence of messages getting exchanged between nodes.

@fassadlr please review or dismiss


UnexpectedResponseInHistoricRateResponse Error

When attempting to run the following command:
bos accounting chain-fees --month 8 --year 2020 --csv

The command hangs for several minutes, then returns:
UnexpectedResponseInHistoricRateResponse

try --rate-provider coincap

Bingo, that fixed it. Thanks for the quick response!


Implement Euclidean division in the MatchingEngine

At the present time, the matching engine is done only with integer prices.
However, having fractional prices or fixed fractions (like 5 decimals) would
be nice.
But the tokens are not divisible, so purely using fractions is not working and
so we need to use the Euclidean division to be able to handle more general
prices. This causes a number of complications:

Suppose you want to buy token1 using token0 and we have 9 token0 on hand.
Then at a price of 2.5 you may fill 2 token1 for a price of 5 token0. And that's it at a price of 2.5
But then the price of 3 you can get another token1 and that may be something that you want.

There are other issues with serialization and the ordering but that is secondary.

Hello is this issue open? I'd love to contribute to it. As I don't see any good-first-issues 🙃, if there is any other good-first-issue I would be happy to work on it!

Hello is this issue open? I'd love to contribute to it. As I don't see any good-first-issues 🙃, if there is any other good-first-issue I would be happy to work on it as well!

Hey @ashu26jha, Thanks for reaching us! We're still discussing a few details on this particular issue. Let me work with the team on generating some more good-first-issues !


[ALLUXIO-1835] Change the client to support multiple workers on a local node

A client can only connect to one worker on a machine, using the hostname as the identifier. To support multiple workers on a node, the client needs to identify a worker via the address (hostname+port).

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/9010/
Test PASSed.

Merged build finished. Test PASSed.

@yupeng9 Thanks for the patch, I left a few minor comments.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/9043/
Test PASSed.

@yupeng9 Looks good overall, one comment on the javadoc and another from the first pass about thread safety.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/9052/
Test PASSed.

I just left a few minor comments. Thanks!

@calvinjia PTAL. Thanks.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/9130/Build result: ABORTED[...truncated 1208 lines...]Running alluxio.client.lineage.LineageFileSystemTestTests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.641 secRunning alluxio.client.lineage.LineageMasterClientTestTests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.557 secRunning alluxio.client.block.BufferedBlockOutStreamTestTests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.003 secRunning alluxio.client.block.BufferedBlockInStreamTestTests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.004 secRunning alluxio.client.block.BlockWorkerClientHeartbeatExecutorTestTests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.35 secRunning alluxio.client.block.BlockMasterClientTestTests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.576 secRunning alluxio.client.block.AlluxioBlockStoreTestTests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.394 secRunning alluxio.client.block.BlockStoreContextTestBuild timed out (after 60 minutes). Marking the build as aborted.Results :Tests run: 135, Failures: 0, Errors: 0, Skipped: 0[ERROR] There was a timeout or other error in the forkBuild was abortedchannel stopped
Test FAILed.

Merged build finished. Test FAILed.

Jenkins, test this please

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/9134/Failed Tests: 0
Test FAILed.

Merged build finished. Test FAILed.

Jenkins, test this please

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/9143/Failed Tests: 1org.alluxio:alluxio-core-client: 1alluxio.client.block.BlockStoreContextTest.acquireWorkerLimitTest
Test FAILed.

@yupeng9 I left a few minor comments. Is the unit test failure flaky?

@calvinjia PTAL

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/9178/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/9194/
Test PASSed.

LGTM


Refactor ldap find member pipe

Replaced by #1580, which will deprecate the LdapFindMemberPipe in favor of a LdapIsMemberOfPipe.


RDPWrap issue 10.0.17763.437 even after update

Hello,
I have Windows 10 64bit pc. I have applied fix #720. According to the RDP Wrapper Configuration everything is good.   However, when I run RDPCheck.exe, it spits out en error:
"The number of connections to this computer is limited and all connections are in use right now. Try connecting later or contact your system administrator."
If I try to connect to it from another PC, it acts like the Wrapper is not even there, and attempts to sign off the second use.

I have restarted the computer. I have reinstalled the RDPwrapper. Same issue. Does anyone have any ideas?

There are six other tickets already open for 10.0.17763.437. There is no fully working solution at this time.

https://github.com/stascorp/rdpwrap/issues/744
Tested and working on many machines with 10.0.17763.437

Thanks, j1nseng!
That worked beautifully after I restarted Remote Desktop Services.


FileWriter in tf.summary solved

Issue: #37113

Lemme Solve this and create a  PR again.

Closing this PR and creating a new one.

Link to New PR : https://github.com/tensorflow/tensorflow/pull/37223


Add more checking in CheckForkWarningConditions

Regarding to my situation, the chain is needed to be invalid by invalidblock but that chain has higher difficulties than the correct chain.
After invalidate block and staying on correct chain, this warning is in debug.log:
CheckForkWarningConditions: Warning: Found invalid chain at least ~6 blocks longer than our best chain.
So, this modification would possibly make this warning not shown in my case but do correctly checking regarding to the warning sentence.
BTW, I do understand that the bitcoin core has same validation (https://github.com/bitcoin/bitcoin/blob/master/src/validation.cpp#L1177) so I'm not sure if we need to modify it this way or not.

The modification seems kind of reasonable but from blockchain point of view "the correct chain" is actually the one that gets invalidated manually because at the time of fork it had overwhelmingly higher amount of work. Also I'm not sure we should silently continue here - it's still huge amount of work in parallel chain which shouldn't be ignored imo, you just need to provide the right message. So this should probably go to https://github.com/dashpay/dash/pull/1234/files#diff-7ec3c68a81efff79b6ca22ac1f1eabbaR1860 smth like
if(pindexBestInvalid->nHeight > chainActive.Height() + 6)
    LogPrintf("%s: Warning: Found invalid chain at least ~6 blocks longer than our best chain.\nChain state database corruption likely.\n", __func__);
else
    LogPrintf("%s: Warning: Found invalid chain which has higher work (at least ~6 blocks worth of work) than our best chain.\nChain state database corruption likely.\n", __func__);

(not sure if that sounds right in English, feel free to fix it :) )


Generate HTML report after each run

The old pre-1.6.0 paradigm - where tuih was basically called automatically after each run - ensured that the HTML report sitting in the ptt_files/ folder was from the most recent test run. This is no longer the case, as we expect the user to run tuih manually. Should we accept that as expected new behavior? Or auto-gen the report (without auto launching it) with each test run?
While we're on the topic, what about time/date-stamping each report?

1.11.0 calls html_gen without launching


add restore doc and fix restore bug

What type of PR is this?
/kind bug
/kind documentation
What this PR does / why we need it:
The restoration which based on a schedule type backup leads to incorrect behavior in the program. Therefore, the entire PR has fixed this issue and completed the documentation for How to use Restore
Which issue(s) this PR fixes:
Fixes #
Special notes for your reviewer:
Does this PR introduce a user-facing change?:

add restore doc and fix restore bug

/label tide/merge-method-squash


C# 8 Readiness

We need to evaluate and update Razor for C# 8. This will align with 3.0
https://github.com/dotnet/csharplang/milestone/8
https://channel9.msdn.com/Events/dotnetConf/2018/S103

Thanks @rynowak.

Additional note: https://github.com/dotnet/csharplang/issues/1630#issuecomment-423263733

Additional note: Need to handle the nonnulltypes feature when things are turned on at the compiler level. In generated code we set bits to null in several locations without being explicit on the types.
Reacting to this could be as simple as adding pragma warnings or changing how certain assignments generate.


added memory usage buckets, removed unused labels

Codecov Report

Merging #6 into master will not change coverage.
The diff coverage is 100.00%.

@@             Coverage Diff             @@
##              master        #6   +/-   ##
===========================================
  Coverage     100.00%   100.00%
  Complexity        56        56
===========================================
  Files              6         6
  Lines            233       223   -10
===========================================
- Hits             233       223   -10

Impacted Files
Coverage Δ
Complexity Δ

src/Metrics/Metrics.php
100.00% <100.00%> (ø)
18.00 <2.00> (+2.00)

src/Metrics/Storage.php
100.00% <100.00%> (ø)
17.00 <0.00> (-2.00)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0a39e3d...7cac929. Read the comment docs.


time.ParseDuration() does not allow days (auto-vacuum-int)

In the docs it describes rqlited -auto-vacuum-int=14d, however the flag that handles that does not actually cater for days (invalid value "14d" for flag -auto-vacuum-int: parse error). As far as I'm aware, the only valid units of time measure supported are s, m, and h.
So 14 days would be -auto-vacuum-int=336h :+1:

You are absolutely right -- thank you.
https://pkg.go.dev/time#ParseDuration


Import/Export: Create template component

I had to change the last commit, so these changes were cherry picked onto the import export branch.


Use with fifo?

I would like to use fastp with input coming over a fifo.  Since you're a single pass tool, it appears this should work.  But no matter how I try to set things up, I get
[2024-07-15T17:46:51-0500] [ERROR] [fastp_report.sh]
fastp_report.sh stopped at line 170.
REASON=Input read 1 file sample.L.fifo.fastq is empty or does not exist.

Should this in fact work?

what's your command?

fastp -i "$Input1Path" -I "$Input2Path" -w 6 -Q -A --overrepresentation_analysis >&2
The two Input?Path are fifos

leftFifo="sample.L.fifo.fastq"
rightFifo="sample.R.fifo.fastq"
mkfifo "$leftFifo"
mkfifo "$rightFifo"
zcat sample_1.fastq.gz > "$leftFifo" &
zcat sample_3.fastq.gz > "$rightFifo" &
Input1Path="$leftFifo"
Input2Path="$rightFifo"

In case that's what you were asking


Fix url encoding for web, Linux and Windows in share plugin

Description
This fixes the issue mentiond in https://github.com/fluttercommunity/plus_plugins/issues/235
Are there any existing unit tests which I can add a test to?
Related Issues
Fixes https://github.com/fluttercommunity/plus_plugins/issues/235
Fixes probably also https://github.com/fluttercommunity/plus_plugins/issues/226
Checklist
Before you create this PR confirm that it meets all requirements listed below by checking the relevant checkboxes ([x]).
This will ensure a smooth and quick review process. Updating the pubspec.yaml and changelogs is not required.

[ ] I read the Contributor Guide and followed the process outlined there for submitting PRs.
[ ] My PR includes unit or integration tests for all changed/updated/fixed behaviors (See Contributor Guide).
[ ] All existing and new tests are passing.
[ ] I updated/added relevant documentation (doc comments with ///).
[ ] The analyzer (flutter analyze) does not report any problems on my PR.
[ ] I read and followed the Flutter Style Guide.
[x] I am willing to follow-up on review comments in a timely manner.

Breaking Change
Does your PR require plugin users to manually update their apps to accommodate your change?

[ ] Yes, this is a breaking change (please indicate a breaking change in CHANGELOG.md and increment major revision).
[x] No, this is not a breaking change.

awesome! thanks a lot.

Are there any existing unit tests which I can add a test to?

have you tried to run the e2e "integration" tests present in the main project (share_plus) on these two platforms?

have you tried to run the e2e "integration" tests present in the main project (share_plus) on these two platforms?

Only Web seems to have integration tests.
My idea would be to add a unit test project for each platform project.
Then I can stub the MethodChannel of the launch package and make sure that each call is correctly encoded.
I also just noticed that Windows has the same problem, so I fixed that too.

Only Web seems to have integration tests.

The integration tests are on share_plus on purpose and execute from the example folder, they are platform agnostic. Please read the CONTRIBUTING.md document is explained there. When you run the test, you select which platform you want it to run.
(another different thing is if they work or not but that's how this project is structured)

after fixing the merge issues I saw that the tests aren't passing, I will leave that to you @ueman :) once it is fix I will release the packages and merge

thanks for the fix! I don't know why CI is canceling the job all the time, will investigate :/

packages has been published


Typescript doesn't allow using Table as namespace

Getting the following error:
Cannot use namespace 'Table' as a type.
With this code:
feelingItems!: Table<FeelingItem, number>;
Facing the same issue when copy-pasting the demo code on the repository readme

Caused due to conflicts with global.d.ts


cannot download and run test data

Hi, I was using the singularity to download and run the command line below:
nextflow run nf-core/chipseq -profile test, singularity
and it throw the error message below, any idea? I also attached the whole log file with this post. Thank you.
Liang
------ERROR MESSAGE---------
nextflow.log
Oct-12 05:25:32.752 [Task monitor] DEBUG n.processor.TaskPollingMonitor - !! executor local > tasks to be completed: 6 -- submitted tasks are shown below
~> TaskHandler[id: 57; name: PHANTOMPEAKQUALTOOLS (SPT5_T0_R1); status: RUNNING; exit: -; error: -; workDir: /lab-share/RC-Data-Science-e2/Public/Liang/p14_Wenxiang/work/69/85501f4e3a77ef2ad96073e2026b49]
~> TaskHandler[id: 54; name: PHANTOMPEAKQUALTOOLS (SPT5_T15_R1); status: RUNNING; exit: -; error: -; workDir: /lab-share/RC-Data-Science-e2/Public/Liang/p14_Wenxiang/work/7e/692b92aeee92c5be6c6b8aaf1e817d]
~> TaskHandler[id: 61; name: PHANTOMPEAKQUALTOOLS (SPT5_T0_R2); status: RUNNING; exit: -; error: -; workDir: /lab-share/RC-Data-Science-e2/Public/Liang/p14_Wenxiang/work/7f/9c0ef16a0425153ee9c47017253a18]
~> TaskHandler[id: 64; name: PHANTOMPEAKQUALTOOLS (SPT5_T15_R2); status: RUNNING; exit: -; error: -; workDir: /lab-share/RC-Data-Science-e2/Public/Liang/p14_Wenxiang/work/44/8e9903dc8a91ac5f13c4bfa9227983]
~> TaskHandler[id: 72; name: PHANTOMPEAKQUALTOOLS (SPT5_INPUT_R2); status: RUNNING; exit: -; error: -; workDir: /lab-share/RC-Data-Science-e2/Public/Liang/p14_Wenxiang/work/59/2bb1d23c4e2498562224083c6f768e]
~> TaskHandler[id: 80; name: PHANTOMPEAKQUALTOOLS (SPT5_INPUT_R1); status: RUNNING; exit: -; error: -; workDir: /lab-share/RC-Data-Science-e2/Public/Liang/p14_Wenxiang/work/03/cd81ad85c2b4b753f0e99884039d56]
Oct-12 05:26:27.337 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 57; name: PHANTOMPEAKQUALTOOLS (SPT5_T0_R1); status: COMPLETED; exit: -; error: nextflow.exception.ProcessException: Process exceeded running time limit (8h); workDir: /lab-share/RC-Data-Science-e2/Public/Liang/p14_Wenxiang/work/69/85501f4e3a77ef2ad96073e2026b49]
Oct-12 05:26:27.409 [Task monitor] ERROR nextflow.processor.TaskProcessor - Error executing process > 'PHANTOMPEAKQUALTOOLS (SPT5_T0_R1)'
Caused by:
Process exceeded running time limit (8h)
Command executed:
RUN_SPP=which run_spp.R
Rscript -e "library(caTools); source("$RUN_SPP")" -c="SPT5_T0_R1.mLb.clN.sorted.bam" -savp="SPT5_T0_R1.spp.pdf" -savd="SPT5_T0_R1.spp.Rdata" -out="SPT5_T0_R1.spp.out" -p=2
cp spp_correlation_header.txt SPT5_T0_R1_spp_correlation_mqc.tsv
Rscript -e "load('SPT5_T0_R1.spp.Rdata'); write.table(crosscorr$cross.correlation, file="SPT5_T0_R1_spp_correlation_mqc.tsv", sep=",", quote=FALSE, row.names=FALSE, col.names=FALSE,append=TRUE)"
awk -v OFS='	' '{print "SPT5_T0_R1", $9}' SPT5_T0_R1.spp.out | cat spp_nsc_header.txt - > SPT5_T0_R1_spp_nsc_mqc.tsv
awk -v OFS='	' '{print "SPT5_T0_R1", $10}' SPT5_T0_R1.spp.out | cat spp_rsc_header.txt - > SPT5_T0_R1_spp_rsc_mqc.tsv
Command exit status:
Command output:
################
ChIP data: SPT5_T0_R1.mLb.clN.sorted.bam
Control data: NA
strandshift(min): -500
strandshift(step): 5
strandshift(max) 1500
user-defined peak shift NA
exclusion(min): 10
exclusion(max): NaN
num parallel nodes: 2
FDR threshold: 0.01
NumPeaks Threshold: NA
Output Directory: .
narrowPeak output file name: NA
regionPeak output file name: NA
Rdata filename: SPT5_T0_R1.spp.Rdata
plot pdf filename: SPT5_T0_R1.spp.pdf
result filename: SPT5_T0_R1.spp.out
Overwrite files?: FALSE

any ideas why

HI @sunliang3361 ! Apologies for the late response. It seems like the process that is failing has exceeded the default resource requirements, in this case the time restriction:
Caused by:
Process exceeded running time limit (8h)

No idea why it should be taking that long on your compute set-up because the whole pipeline test should finish in 30-40 mins. Can you try by adding --skip_spp to the command-line and seeing if the pipeline completes?

Thank you for your response. If I use --skip_spp, the pipeline can be completed. By using this parameter, how will this affect the final result? I guess it's just a QC step and will not affect so much, right?

when I tried to run my own samples. I got the following error message. Is this the reference genome issue? Thanks.

Oct-15 09:24:08.663 [main] DEBUG nextflow.cli.Launcher - $> nextflow run nf-core/chipseq -profile singularity --input samples.csv --genome GRCm38 --max_memory 40.GB --max_cpus 8 --outdir results --skip_diff_analysis --skip_spp -r 1.2.1
Oct-15 09:24:08.866 [main] INFO  nextflow.cli.CmdRun - N E X T F L O W  ~  version 20.01.0
Oct-15 09:24:09.571 [main] DEBUG nextflow.scm.AssetManager - Git config: /home/ch213537/.nextflow/assets/nf-core/chipseq/.git/config; branch: master; remote: origin; url: https://github.com/nf-core/chipseq.git
Oct-15 09:24:09.581 [main] DEBUG nextflow.scm.AssetManager - Git config: /home/ch213537/.nextflow/assets/nf-core/chipseq/.git/config; branch: master; remote: origin; url: https://github.com/nf-core/chipseq.git
Oct-15 09:24:21.191 [main] DEBUG nextflow.scm.AssetManager - Git config: /home/ch213537/.nextflow/assets/nf-core/chipseq/.git/config; branch: master; remote: origin; url: https://github.com/nf-core/chipseq.git
Oct-15 09:24:21.192 [main] INFO  nextflow.cli.CmdRun - Launching nf-core/chipseq [marvelous_poincare] - revision: 0f487ed76d [1.2.1]
Oct-15 09:24:21.729 [main] DEBUG nextflow.config.ConfigBuilder - Found config base: /home/ch213537/.nextflow/assets/nf-core/chipseq/nextflow.config
Oct-15 09:24:21.731 [main] DEBUG nextflow.config.ConfigBuilder - Parsing config file: /home/ch213537/.nextflow/assets/nf-core/chipseq/nextflow.config
Oct-15 09:24:21.745 [main] DEBUG nextflow.config.ConfigBuilder - Applying config profile: singularity
Oct-15 09:24:22.483 [main] DEBUG nextflow.config.ConfigBuilder - Available config profiles: [cfc_dev, denbi_qbic, bi, genotoul, bigpurple, uppmax, docker, gis, utd_ganymede, conda, singularity, icr_davros, munin, prince, czbiohub_aws, hebbe, cfc, uzh, ccga_med, debug, test, genouest, cbe, ebc, ccga_dx, crick, google, kraken, phoenix, shh, awsbatch, pasteur, uct_hpc, test_full, binac]
Oct-15 09:24:22.554 [main] DEBUG nextflow.Session - Session uuid: 2b400376-b58e-474c-8494-1fd368869076
Oct-15 09:24:22.554 [main] DEBUG nextflow.Session - Run name: marvelous_poincare
Oct-15 09:24:22.555 [main] DEBUG nextflow.Session - Executor pool size: 40
Oct-15 09:24:22.579 [main] DEBUG nextflow.cli.CmdRun -
Version: 20.01.0 build 5264
Created: 12-02-2020 10:14 UTC (05:14 EDT)
System: Linux 3.10.0-1062.12.1.el7.x86_64
Runtime: Groovy 2.5.8 on Java HotSpot(TM) 64-Bit Server VM 1.8.0_144-b01
Encoding: UTF-8 (UTF-8)
Process<EMAIL_ADDRESS>[<IP_ADDRESS>]
CPUs: 40 - Mem: 187.6 GB (168.3 GB) - Swap: 0 (0)
Oct-15 09:24:22.692 [main] DEBUG nextflow.Session - Work-dir: /lab-share/RC-Data-Science-e2/Public/Liang/p14_Wenxiang/p14/work [nfs]
Oct-15 09:24:22.724 [main] DEBUG nextflow.Session - Observer factory: TowerFactory
Oct-15 09:24:22.725 [main] DEBUG nextflow.Session - Observer factory: DefaultObserverFactory
Oct-15 09:24:22.985 [main] DEBUG nextflow.Session - Session start invoked
Oct-15 09:24:22.989 [main] DEBUG nextflow.trace.TraceFileObserver - Flow starting -- trace file: /lab-share/RC-Data-Science-e2/Public/Liang/p14_Wenxiang/p14/results/pipeline_info/execution_trace.txt
Oct-15 09:24:23.747 [main] DEBUG nextflow.script.ScriptRunner - > Launching execution
Oct-15 09:24:23.757 [main] DEBUG nextflow.Session - Workflow process names [dsl1]: FASTQC, output_documentation, CONSENSUS_PEAKS, MULTIQC, SORT_BAM, CONSENSUS_PEAKS_DESEQ2, PHANTOMPEAKQUALTOOLS, BWA_MEM, BIGWIG, CONSENSUS_PEAKS_COUNTS, BWA_INDEX, get_software_versions, CHECK_DESIGN, MERGED_BAM_REMOVE_ORPHAN, MAKE_GENE_BED, CONSENSUS_PEAKS_ANNOTATE, TRIMGALORE, PLOTPROFILE, MACS2_QC, PICARD_METRICS, PLOTFINGERPRINT, MERGED_BAM_FILTER, MACS2, MACS2_ANNOTATE, MERGED_BAM, IGV, PRESEQ, MAKE_GENOME_FILTER
Oct-15 09:24:24.025 [main] DEBUG nextflow.file.FileHelper - Creating a file system instance for provider: S3FileSystemProvider
Oct-15 09:24:24.036 [main] DEBUG nextflow.Global - Using AWS temporary session credentials defined in default section in file: /home/ch213537/.aws/credentials
Oct-15 09:24:24.037 [main] DEBUG nextflow.file.FileHelper - Using AWS temporary session token for S3FS.
Oct-15 09:24:24.053 [main] DEBUG nextflow.file.FileHelper - AWS S3 config details: {session_key=FwoGZX.., secret_key=8VOQ6V.., region=us-east-1, access_key=ASIA5F..}
Oct-15 09:24:24.722 [main] WARN  c.a.a.p.i.BasicProfileConfigLoader - Your profile name includes a 'profile ' prefix. This is considered part of the profile name in the Java SDK, so you will need to include this prefix in your profile name when you reference this profile from your Java code.
Oct-15 09:24:25.432 [main] DEBUG nextflow.Session - Session aborted -- Cause: The authorization header is malformed; the region 'us-east-1' is wrong; expecting 'eu-west-1' (Service: Amazon S3; Status Code: 400; Error Code: AuthorizationHeaderMalformed; Request ID: 8JCR5ZAH0R3GCS9M; S3 Extended Request ID: wwgKyeqItk9uQpWH9XTherNK2Yo+aY3FpwTFPuWBQgbbSxayWnmUbL1f0VAw8k+ajoxvaw6DT1E=)
Oct-15 09:24:25.448 [main] ERROR nextflow.cli.Launcher - @unknown
com.amazonaws.services.s3.model.AmazonS3Exception: The authorization header is malformed; the region 'us-east-1' is wrong; expecting 'eu-west-1' (Service: Amazon S3; Status Code: 400; Error Code: AuthorizationHeaderMalformed; Request ID: 8JCR5ZAH0R3GCS9M; S3 Extended Request ID: wwgKyeqItk9uQpWH9XTherNK2Yo+aY3FpwTFPuWBQgbbSxayWnmUbL1f0VAw8k+ajoxvaw6DT1E=)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1712)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1367)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1113)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:770)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:744)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:726)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:686)
at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:668)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:532)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:512)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4914)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4860)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4854)
at com.amazonaws.services.s3.AmazonS3Client.listObjects(AmazonS3Client.java:880)
at com.upplication.s3fs.AmazonS3Client.listObjects(AmazonS3Client.java:104)
at com.upplication.s3fs.util.S3ObjectSummaryLookup.lookup(S3ObjectSummaryLookup.java:116)
at com.upplication.s3fs.S3FileSystemProvider.getAccessControl(S3FileSystemProvider.java:894)
at com.upplication.s3fs.S3FileSystemProvider.checkAccess(S3FileSystemProvider.java:588)
at java.nio.file.Files.exists(Files.java:2385)
at sun.reflect.GeneratedMethodAccessor20.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:101)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:323)
at org.codehaus.groovy.runtime.callsite.StaticMetaMethodSite.invoke(StaticMetaMethodSite.java:44)
at org.codehaus.groovy.runtime.callsite.StaticMetaMethodSite.call(StaticMetaMethodSite.java:89)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
at org.codehaus.groovy.runtime.callsite.StaticMetaMethodSite.call(StaticMetaMethodSite.java:94)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:127)
at nextflow.extension.FilesEx.exists(FilesEx.groovy:453)
at nextflow.file.FileHelper.checkIfExists(FileHelper.groovy:971)
at nextflow.file.FileHelper$checkIfExists$1.call(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
at nextflow.file.FileHelper$checkIfExists$1.call(Unknown Source)
at nextflow.Nextflow.file(Nextflow.groovy:158)
at nextflow.Nextflow$file.callStatic(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallStatic(CallSiteArray.java:55)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callStatic(AbstractCallSite.java:196)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callStatic(AbstractCallSite.java:216)
at Script_cc06130a.runScript(Script_cc06130a:171)
at nextflow.script.BaseScript.runDsl1(BaseScript.groovy:153)
at nextflow.script.BaseScript.run(BaseScript.groovy:189)
at nextflow.script.ScriptParser.runScript(ScriptParser.groovy:225)
at nextflow.script.ScriptRunner.run(ScriptRunner.groovy:218)
at nextflow.script.ScriptRunner.execute(ScriptRunner.groovy:126)
at nextflow.cli.CmdRun.run(CmdRun.groovy:273)
at nextflow.cli.Launcher.run(Launcher.groovy:460)
at nextflow.cli.Launcher.main(Launcher.groovy:642)

Hi @sunliang3361  It shouldn't be much of a loss if you --skip_spp. It just provides some additional QC and I included it because its something that the ENCODE do routinely, although I almost never look at the results for this. I would be tempted to try the command without --skip_spp to see if the pipeline finishes. It could just have been an intermittent issue unless you have seen it happen reproducibly.
The issue you are seeing above is related to this file in your home directory (can you rename this file temporarily and try again):
Oct-15 09:24:24.036 [main] DEBUG nextflow.Global - Using AWS temporary session credentials defined in default section in file: /home/ch213537/.aws/credentials

I am assuming you have/are using AWS for other stuff and the region specified there is conflicting with the region required to download the reference genome data as indicated by this message:
Oct-15 09:24:25.432 [main] DEBUG nextflow.Session - Session aborted -- Cause: The authorization header is malformed; the region 'us-east-1' is wrong; expecting 'eu-west-1' (Service: Amazon S3; Status Code: 400; Error Code: AuthorizationHeaderMalformed; Request ID: 8JCR5ZAH0R3GCS9M; S3 Extended Request ID:

Unfortunately, this is something that has to be fixed on the Nextflow side so the easiest solution is to temporarily rename that file, run the pipeline to download all of the genome data, save it all in the results/genome/ directory using the --save_reference parameter, and then the next time you run the pipeline you can adjust the parameters to use your local copy so you dont have to download it again i.e.
nextflow run nf-core/chipseq \
    --input samples.csv \
    --genome GRCm38 \
    --fasta <path/to/downloaded/fasta> \
    --gtf </path/to/downloaded/gtf> \
    --bwa_index </path/to/downloaded/bwa/index> \
    --max_memory 40.GB \
    --max_cpus 8 \
    --outdir results \
    --email <your_email_address> \
    --skip_diff_analysis
    -profile singularity
    -r 1.2.1

Thank you. if I don't rename the aws file and download genome fasta, gtf and bwa_index to local folder, I can directly use --fasta, --gtf, and --bwa_index, and no need to use --genome (I think you have this parameter in the above reply), right?

Not exactly, its better to use --genome too because there are some settings that come from using that parameter that would normally have to be specified manually.e.g. these two.
If you want to download everything separately first then thats fine but if you don't want to use --genome then you would have to specify at least the following:
nextflow run nf-core/chipseq \
    --input samples.csv \
    --fasta <path/to/downloaded/fasta> \
    --gtf </path/to/downloaded/gtf> \
    --bwa_index </path/to/downloaded/bwa/index> \
    --macs_gsize 1.87e9 \
    --blacklist </path/to/downloaded/blacklist/file> \
    --max_memory 40.GB \
    --max_cpus 8 \
    --outdir results \
    --email <your_email_address> \
    --skip_diff_analysis
    -profile singularity
    -r 1.2.1

Thank you so much. The pipeline seems working now.

No worries. Glad it's working 🙂


Handle duplicated antibody

[ ] Design indications
[x] Backend check
[ ] Frontend implementation

Catalog number + vendor id

@Anto8819 the backend is ready: need to call /antibody/user and the user is retrieved from the cookie


Feature request: broadcast only to one peer (Private transaction broadcasting)

There is some research that if a transaction is broadcasted to many peers simultaneously, it is easier to detect the source of the transaction.
https://docs.wasabiwallet.io/why-wasabi/NetworkLevelPrivacy.html#private-transaction-broadcasting-1
https://github.com/bitcoin/bips/blob/master/bip-0156.mediawiki
I propose to broadcast to a random peer and disconnect the peer (but keep the connection open with other peers). If Tor is available, use only onion peers. Then wait for some time, if the transaction is sent back by any other peer. If this happens, consider the transaction successfully broadcasted. Otherwise, if after some time it is still not sent back, send it to another random peer etc. Disconnect the peer after sending to it immediately, but don't disconnect other peers for some time. Wait for some random time for each peer after successfully receiving the transaction back, then close the connection. If it stops right after receiving the transaction, this can be used as heuristics to detect the link between the connection and the transaction.

To address this, we could probably have multiple broadcast strategies:
a) when using Tor, keep the current strategy (broadcast to many peers at once for the sake of speed)
b) when using clearnet, use the strategy you outline above
I think that if we're coming through a fresh Tor circuit for every broadcast, it doesn't really matter if we are being surveilled since there is nothing of value to learn. At worst, the adversary sees that the transaction came through Tor.
The current broadcast strategy is fast and generates a high confidence level that a transaction indeed went out. The time spent broadcasting is minimal. The problem with broadcasting to a single peer is that it can fail at various stages and takes more time, including the time spent watching for the inv message from other peers advertising our own transaction to us.
Thoughts?

I propose to add a flag for the strategy:
--strategy=[private|fast]

Then people would choose what they prefer. I would keep default "private" though just to increase to anonymity set of people sending transactions in a private manner.
I would like to use the private strategy even through Tor, because an adversary can happen to sit on the network channel between me and Tor and detect transaction broadcast network pattern. I think it is possible to correlate Tor traffic produced by sending to many peers at the same time with new transaction appearing publicly (in adversary's mempool). Given the fact that different transactions often have different sizes (and it affects the amount of Tor traffic) and not many transactions are broadcasted at the same time in the network, I think it is possible to detect the exact transaction, exploiting the the knowledge that majority sends to all the peers simultaneously.

Indeed, it seems best to offer the choice of both strategy and network type. That way we end up with a matrix of options where the user can mix and match the tradeoffs they're comfortable with.
I'll put this on the roadmap for the next version.

--strategy=[private|fast]

Looks like using the Tor network is both private and fast because you can submit to a few peers simultaneously. While using clearnet is less private (but more private compared to sending to centralized block explorers) and slow because you want for a tx roundtrip before sending to another peer. Right?

@starius I believe this PR does what you proposed: https://github.com/alfred-hodler/pushtx/pull/5
Transaction propagation is much faster than I expected so we don't need to send to multiple peers at all.

@vasild

--strategy=[private|fast]

Looks like using the Tor network is both private and fast because you can submit to a few peers simultaneously. While using clearnet is less private (but more private compared to sending to centralized block explorers) and slow because you wait for a tx roundtrip before sending to another peer. Right?

Even with onion peers it is better for privacy to send to one peer and wait. If you send to many onion peers, your ISP will see a spike of Tor traffic on your connection and can correlate it with appearance of a transaction in mempool.

Resolved in #5 .


Any Bilibili videos more than 10mins are downloaded as multiple files

Hi,
I am using the latest version of youtube-dl.
However, i noticed that any videos that are more than 10mins long on Bilibili is downloaded as multiple filed.
Tried the following options:
--buffer-size 1G
--http-chunk-size 1G
Still no luck.
Could you please have a look and let me know how to download long Bilibili videos to a single video file.
Command i used:
youtube-dl -f 1 -o "%(title)s-%(id)s.%(ext)s" --write-description https://www.bilibili.com/video/av10860038

You can't download as a single file cause it's not served as single file.

Hi,
it looks to me that the origin website is serving the video as one file. As in the UI download website http://www.urlgot.com/en_US/, when the url is parsed, it shows it as one single file, like the following.

And the file can be downloaded as a single file.

Could you please have another look of the issue?

These are not original files. All linked to d*.urlgot.com.

Why couldn't youtube-dl auotmaticaly combine the parts using ffmpeg if present? Or at least have a flag for it.


no plugins listed after installation

git vogue plugins
git-vogue looks for plugins in:
    /home/robertc/.cabal/libexec/git-vogue

git-vogue knows about the following plugins:
robertc@lifeless-z140:~/personal/haskell/scratchpad$ ls /home/robertc/.cabal/libexec/git-vogue
ls: cannot access /home/robertc/.cabal/libexec/git-vogue: No such file or directory

I assume you are using cabal 1.18? In that version is a bug, where copyHook is not called. https://github.com/haskell/cabal/issues/1805.

I was; is it possible to make git-vogue error if built by cabal 1.18?

@rbtcollins You can probably do the following to work around the bug in Cabal:
mkdir -p /home/robertc/.cabal/libexec/git-vogue
mv /home/robertc/.cabal/bin/git-vogue-* /home/robertc/.cabal/libexec/git-vogue/

Alternatively, you can defined the GIT_VOGUE_PATH environmental variable by adding something like this to your shell config:
export GIT_VOGUE_PATH="$HOME/.cabal/bin"

Actually, you probably shouldn't use GIT_VOGUE_PATH. I think it'll try to use all the executable files as plugins.

@rbtcollins Still need to address your suggestion and see if we can bail out/print some kind of help if the installation fails. Perhaps we could also detect if the libexec dir does not exist, and print a suggestion.


Changing tags class style on focus

I am trying to have the border change on this element when the user is focused on the input box typing in tags.
<div class="tags" ng-class="{focused: hasFocus}">

According to my developer tools, when the input box is focused, it should be div.tags.focused. I've done the following but it still does not change the style. Any tips that I am not trying or is something wrong here?
div.tags.focused, tags-input .tags.focused  {
    border:1px solid red;
}

same here

the problem comes from a -webkit-appearance: textfield; on the .tags class
here's a fix -- implement locally
.tags {
    -moz-appearance: initial;
    -webkit-appearance: initial;
}

-webkit-appearance is being deprecated anyway

@zelias500 -webkit-appearance doesn't work for me. Any other work around is really appreaciated

    .host {
      outline: 0;
    }

This issue was selected to be spring-cleaned by a script. More information can be found here.
If anyone thinks this was a mistake and this issue should be reopened, please leave a message below explaining why. Before doing so, please consider reading the CONTRIBUTING file.


[WIP] Add support for install/uninstall container images

Based on the atomic command, install and uninstall will
pull an image and then install or uninistall a container image
based on the INSTALL and UNINSTALL labels.
Signed-off-by: Daniel J Walsh<EMAIL_ADDRESS>

Ill take this and run with it.

Replaced by https://github.com/containers/libpod/pull/1463


Implement UIView addSubview + makeConstraints extension

Extension for UIView and NSView that adds new addSubview function which takes (ConstraintMaker) -> Void closure as an argument, calls default addSubview function and applies closure to the added view.
As a result, code that used to look like this:
addSubview(myView)
myView.makeConstraints {
...
}

Will look like this:
addSubview(myView) {
...
}

Based on my experience of using SnapKit, adding subview then applying constraints to it is one of the most common patterns in screen-building code. So potentially this little extension might reduce the number of code lines and slightly improve code readability.

I appreciate your input here -- however as part of the direction of SnapKit is to minimise any sort of extension of System API's as much as possible (hence nearly everything is namespaced under .snp.) I'm going to close this for now.
These extensions are easily available via your own extensions in your apps though and I recommend to keep doing that.


No "latest" docker tag present on docker hub

Hi,
I'm not sure if this is intentional, but the opensuse/portus docker image on docker hub does not have a latest tag at present. Hence, I can't easily pull the latest image. There is a head tag - are people supposed to use that instead?
Thank you!

The example files at https://github.com/SUSE/Portus/tree/master/examples/compose use the head tag, so I guess that's the one to use.

I would suggest adding a latest to point to the latest stable release and head is probably the current bleeding edge release ?

We didn't have a latest tag because that word has a different meaning to different people: latest as in latest stable release, or latest as in master ?
After some time, I think it would be a good thing to point to the latest stable release. Furthermore, I'd like to add another tag: development, or something like that. That's because head was broken for quite some time and people could simply use the Dockerfile from the root of the project just fine. So, development would simply be a build from the root of the project on the master branch.
What do you think ?

Generally, if I do a docker pull XYZ:latest I expect to get something that will work. If there are unstable releases on docker hub, they probably shouldn't be tagged as latest
Having a dev tag certainly makes sense for people to know this isn't a stable release.

latest for stable, dev or head to master branch (I'm tend to prefer head but I'm not sure if people would understand that this might be an unstable version).
Which terms do people usually use in the docker world?

@vitoravelino note that dev and head would be the master branch but from two different sources:

dev for what we've got in Github.
head for what we produce on OBS when fetching the master branch from Github.

Generally, if I do a docker pull XYZ:latest I expect to get something that will work. If there are unstable > releases on docker hub, they probably shouldn't be tagged as latest
Having a dev tag certainly makes sense for people to know this isn't a stable release.

Let's do that 👍

Current state:

[x] Add a latest tag.
[x] Remove old tags (2.0 and 2.1).
[ ] Add a development tag.
[ ] Documentation: improve the README, the gh-pages branch, etc. It should be crystal clear where to fetch each version, how to deploy it, etc.

The development tag is kind of tricky because you cannot specify multiple source repositories for automated builds on Docker Hub. I'll probably add a SUSE/portus image which only includes the development tag (while pointing to the right production-ready images).

I think we can leave the development tag for another time (#1544)


WebAuthn: signup

Codecov Report

Merging #942 (7b5f864) into main (d814c7b) will decrease coverage by 0.00%.
The diff coverage is 96.61%.

:exclamation: Current head 7b5f864 differs from pull request most recent head 20f3eb9. Consider uploading reports for the commit 20f3eb9 to get more accurate results

@@            Coverage Diff             @@
##             main     #942      +/-   ##
==========================================
- Coverage   96.52%   96.52%   -0.01%
==========================================
  Files          39       39
  Lines        6212     6268      +56
  Branches      384      393       +9
==========================================
+ Hits         5996     6050      +54
- Misses        216      218       +2

Impacted Files
Coverage Δ

packages/core/src/machines/index.ts
98.51% <96.61%> (-0.15%)
:arrow_down:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


Sort Universities by most applications received.

Describe the task
Currently, on HR universities page, we have all the universities listed down on the basis of dates of addition. We would like to have a default listing in such an order, that maximum applications received corresponding to a university should be listed first.

Expected Time
1.Sorting Universities by most applications received. .2-4 hours

@P4NK4J @GauravGusain98 can you share why we need the sorting in that? I mean what is the use case here?
I think if we sort by title it will be easy for a user to search for it and there are better chances that universities with different names will come together.


[Bug]: Can't add a service user after deleting a service user using the same name.

Preflight Checklist

[X] I could not find a solution in the documentation, the existing issues or discussions
[X] I have joined the ZITADEL chat

Environment
ZITADEL Cloud
Describe the bug

To reproduce
Go to service users in a project.
Delete a service user.
Add a new service user with the same name.
When you try to create the user, it says that the user exists.
Screenshots
No response
Expected behavior
No response
Version
No response
Operating System
No response
Relevant Configuration
No response
Additional Context
No response

Hei @dakshitha
I just tried to reproduce this, but I couldn't for me it works perfectly fine.
Did you still see the user in the overview? After deleting?

No. The user wasn't showing. But I get the message that the user already exists.

Hi @dakshitha and @hifabienne, I've tried to reproduce this issue to try to fix this issue.
If I start a new Zitadel instance with the default org (Zitadel) and try to create a service user, delete it, and add it once again in the same org, there's no problem, the service user is created as expected.
If I create a service user with username test in the default org (Zitadel) and try to create the same service user with username test then I get the "User already exists".
So I guess we  may have two alternatives:

@hifabienne is this the expected behavior? we can only have one service user with that username throughout all Zitadel orgs or
@dakshitha could it be that the nextjs username did exist in another org and show the error as Zitadel may require that the username is unique throughout all orgs?

@hifabienne let me know if you want me to change current behavior as a reproducible bug

Hi @dakshitha and @hifabienne, I've tried to reproduce this issue to try to fix this issue.
If I start a new Zitadel instance with the default org (Zitadel) and try to create a service user, delete it, and add it once again in the same org, there's no problem, the service user is created as expected.
If I create a service user with username test in the default org (Zitadel) and try to create the same service user with username test then I get the "User already exists".
Do you mean create another service user in a different org with the  same username as the one in the default org?
So I guess we may have two alternatives:

@hifabienne is this the expected behavior? we can only have one service user with that username throughout all Zitadel orgs or
@dakshitha could it be that the nextjs username did exist in another org and show the error as Zitadel may require that the username is unique throughout all orgs?
Yes, this could have been the case.

@hifabienne let me know if you want me to change current behavior as a reproducible bug

Hei @doncicuto
Yes that is intended. Usernames have to be unique across all organizations.
This is for human as well as for service users.

I will close this issue for now, as I can't reproduce.
Feel free to reopen when it appears again


Fix wardenjs ESM in Node.js

You can test that it works by doing this:

create a Node.js project (e.g. create a folder and run npm init)
run npm install @wardenprotocol/wardenjs<EMAIL_ADDRESS>react-dom
create index.mjs with the following content:import * as Foo from "@wardenprotocol/wardenjs";
console.log(Foo);

run node index.mjs

I cleaned up import paths by removing dist/ and fixed all occurences in SpaceWard as well.
closes #274

Looks good, it's working for provided example, thanks!


Wait for logout to be sent

Session termination can be initiated by either the gateway or the client by sending a Logout (35=5).
Upon receiving the Logout request, the contra party will respond with a Logout message signifying a
Logout reply. Upon receiving the Logout reply, the receiving party will terminate the connection

How is this related to this PR?


Fix ioctl constants for musl target envs.

Heya!
I ran across this issue today while trying to build a portable static binary using the x86_64-unknown-linux-musl target. Took a bit of digging to make sure I understood what was going on, and while I may still be off the mark, I believe this is a fix to my issue.
Thanks!!

According to musl's source, the ioctl function signature takes an int as the request argument (i.e. an i32) which is reflected in this crate's ioctl binding. It looks like when the ioctl constants were added that glibc's default of a c_ulong type was used for the musl values as well, rather than a c_int type. This change updates these constants to a c_int so that they match the expected function call type.
Here is a minimal reproduction of the issue. Given this Rust program:
extern crate libc;

use libc::{ioctl, winsize, STDOUT_FILENO, TIOCGWINSZ};

fn main() {
    let mut wsize = winsize {
        ws_row: 0,
        ws_col: 0,
        ws_xpixel: 0,
        ws_ypixel: 0,
    };
    unsafe {
        ioctl(STDOUT_FILENO, TIOCGWINSZ, &mut wsize);
    }
    println!("Sizes: {{ rows: {}, cols: {}, xpixel: {}, ypixel: {} }}",
             wsize.ws_row,
             wsize.ws_col,
             wsize.ws_xpixel,
             wsize.ws_ypixel);
}

When run against the x86_64-unknwon-linux-gnu and
x86_64-unknown-linux-musl targets, we see the difference in behavior:
> cargo clean

> cargo run --target=x86_64-unknown-linux-gnu
   Compiling libc v0.2.11
   Compiling libc-musl-ioctl v0.1.0 (file:///src/libc-musl-ioctl)
     Running `target/x86_64-unknown-linux-gnu/debug/libc-musl-ioctl`
Sizes: { rows: 28, cols: 211, xpixel: 0, ypixel: 0 }

> cargo clean

> cargo run --target=x86_64-unknown-linux-musl
   Compiling libc v0.2.11
   Compiling libc-musl-ioctl v0.1.0 (file:///src/libc-musl-ioctl)
src/main.rs:13:30: 13:40 error: mismatched types:
 expected `i32`,
    found `u64` [E0308]
src/main.rs:13         ioctl(STDOUT_FILENO, TIOCGWINSZ, &mut wsize);
                                            ^~~~~~~~~~
src/main.rs:13:30: 13:40 help: run `rustc --explain E0308` to see a detailed explanation
error: aborting due to previous error
Could not compile `libc-musl-ioctl`.

To learn more, run the command again with --verbose.

Working against this fix:
> cargo clean

> cargo run --target=x86_64-unknown-linux-gnu
    Updating git repository `https://github.com/fnichol/rust-lang-libc.git`
   Compiling libc v0.2.11 (https://github.com/fnichol/rust-lang-libc.git?branch=fix-musl-ioctl-constants#3285f387)
   Compiling libc-musl-ioctl v0.1.0 (file:///src/libc-musl-ioctl)
     Running `target/x86_64-unknown-linux-gnu/debug/libc-musl-ioctl`
Sizes: { rows: 28, cols: 211, xpixel: 0, ypixel: 0 }

> cargo clean

> cargo run --target=x86_64-unknown-linux-musl
   Compiling libc v0.2.11 (https://github.com/fnichol/rust-lang-libc.git?branch=fix-musl-ioctl-constants#3285f387)
   Compiling libc-musl-ioctl v0.1.0 (file:///src/libc-musl-ioctl)
     Running `target/x86_64-unknown-linux-musl/debug/libc-musl-ioctl`
Sizes: { rows: 28, cols: 211, xpixel: 0, ypixel: 0 }

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @alexcrichton (or someone else) soon.
If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.
Please see the contribution instructions for more information.

@bors: r+ 78d9be216e0806d624a4717ce0a3e716b13197e3
Thanks!

:hourglass: Testing commit 78d9be2 with merge a252236...

:boom: Test timed out

@bors: retry

:hourglass: Testing commit 78d9be2 with merge 54fc5c4...

:boom: Test timed out

@alexcrichton Thanks for the quick turnaround! Is there anything I can do to help the bot, or is it just caught in timeout loop?

@bors: retry
Nah unfortunately the appveyor build is sometimes really slow ...

:hourglass: Testing commit 78d9be2 with merge fb5008c...

:sunny: Test successful - status-appveyor, travis

Great!


Realsense D435 Camera

librealsense
2.36.0 RELEASE

OS
Windows

Please provide a description of the problem
Hi,
Backend in rs2_create_device(info_list:000002319C04C6B0, index:0):
MFCreateDeviceSource(_device_attrs, &_source) returned: HResult 0x800706d9: "There are no more endpoints available from the endpoint mapper."
I get this error message when I run the viewer. Device manager recognises and installs the drivers for the camera D435, but it is not accessible either with the viewer or with RecFusion App. I'm running Version	10.0.19042 Build 19042
of windows. Thanks,
John

Hi @johnkflynn   Could you first check please if access to the camera is set to On in the privacy settings of Windows 10, please?  Instructions for doing so are in the link below.
https://github.com/IntelRealSense/librealsense/issues/3381#issuecomment-503424789

On Tue, 25 Aug 2020 at 12:01, MartyG-RealSense<EMAIL_ADDRESS>wrote:

Hi @johnkflynn https://github.com/johnkflynn Could you first check
please if access to the camera is set to On in the privacy settings of
Windows 10, please? Instructions for doing so are in the link below.
#3381 (comment)
https://github.com/IntelRealSense/librealsense/issues/3381#issuecomment-503424789
Hi, yes access to camera granted. I checked that a few times. I have
uninstalled and reinstalled. When RealsenseViewer is starting it says
camera found, but then hangs. On a rare occasion it gets into the programme
it then says please connect a camera. All permissions for going through
firewalls etc, granted. This started about 20 August.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/IntelRealSense/librealsense/issues/7185#issuecomment-679956008,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AFUFEERNMOSIWLMHD57MCN3SCOKXFANCNFSM4QKQKFWQ
.
--

John K Flynn

Hi, thanks for all your help. I think I may have just solved it! When I
first installed it, it told me it needed an upgrade to the latest firmware.
At the time this was <IP_ADDRESS>. I did this and problems started. I have
just now reverted to Firmware <IP_ADDRESS>, the realsense viewer now works!
So does RecFusion! Again once again for the help.
John K Flynn
On Tue, 25 Aug 2020 at 17:32, MartyG-RealSense<EMAIL_ADDRESS>wrote:

The details in the case linked to below seem to be a close match to your
situation:
#3879 https://github.com/IntelRealSense/librealsense/issues/3879
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/IntelRealSense/librealsense/issues/7185#issuecomment-680134544,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AFUFEEQKP5KQNXAHGUUQT4LSCPRR7ANCNFSM4QKQKFWQ
.

O! Dear, Switched on the computer. Downloaded 3dot software. It found and
opened the camera. Then after a short while said it can't find the camera.
Intel Realsenseviewer  discovers the camera but then bombs out. Checked
obvious things the RS dfu service is running, drivers loaded just nothing
can relate to it. Sorry to bother you again.
John K Flynn
On Tue, 25 Aug 2020 at 18:37, MartyG-RealSense<EMAIL_ADDRESS>wrote:

Yes, problems can sometimes be corrected if you perform a firmware
downgrade. You can then re-upgrade to the most recent firmware if you wish.
Bear in mind that the camera sets a maximum limit of 20 firmware downgrades
though before you can only upgrade to a newer firmware than the current one
after that.
Glad to hear that you found a solution!
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/IntelRealSense/librealsense/issues/7185#issuecomment-680170489,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AFUFEEXEPMVHGWQC2RKBA4DSCPZFFANCNFSM4QKQKFWQ
.

It's no trouble at all.
It is a positive sign that the camera can at least be detected.  It sounds like a disconnection issue.  Are you using the official short USB cable supplied with the camera or a longer USB cable of your own choice, please?

Yes. It's the official one.I suspect it is a connection issue as well. I
have to go and do shopping. I'll be back later.
John K Flynn
On Wed, 26 Aug 2020 at 11:55, MartyG-RealSense<EMAIL_ADDRESS>wrote:

It's no trouble at all.
It is a positive sign that the camera can at least be detected. It sounds
like a disconnection issue. Are you using the official short USB cable
supplied with the camera or a longer USB cable of your own choice, please?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/IntelRealSense/librealsense/issues/7185#issuecomment-680808066,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AFUFEESBPZJ5BHZDHNHDAODSCTSZNANCNFSM4QKQKFWQ
.

A little bit more information I forgot to mention that I am running this on a Dell xps 13 7390 laptop. I suspect that is the problem. I connected the camera to the Desktop PC - no problem. It fired up right away. When I look at the setup on the laptop, the hub is connected by a thinner cable than the camera. While it can recognise the camera, I suspect it doesn't have enough power to drive the camera. I am awaiting a thicker c to c cable from Amazon. I will let you know how I get on.

@johnkflynn  Thanks very much for the update - I look forward to your report on tests with the new cable.   Good luck!

Hi MartyG, I think I may have exhausted every option I have looked for. I bought the Dell docking station thinking a 100W would be great and would solve the problem! It helped to the extent RealsenseViewer recognised the D435 camera but then said the Metadata needs to be enabled on this commuter. I don't know how to do this. I tried Dot3D it found the camera but wouldn't or couldn't connect. I tried RecFusion who on starting showed a brief clip of what the camera was seeing, then nothing. I have removed and reinstalled a number of time. I have tried different cables. Would you know the names of the drivers I should have installed?. On my computer they are discovered as Intel Realsense Depth camera 435 with RGB Modual Depth. Driver =  Microsoft dated 21/06/2006, USB Video device. The second entry with the RGB Module RGB: there are 4 drivers ksthunk.sys: SecureUSBVideo.dll: usbvideo.sys: and WdmCompanionFiter.sys. On my main computer it looks like the drivers are from Intel. The Camera works with RecFusion on this computer!

On Windows, you can enable camera metadata in the RealSense Viewer program.  If you launch the Viewer with the camera plugged in, you may get a pop-up message asking if you wish to enable metadata.  If you do not get the pop-up, unplugging the camera and plugging it back into a different USB port can trigger the message.
If you can get an initial image before disconnection occurs, it sounds as though the drivers are present and okay.  Here is an image of my own Device Manager with a D435i camera, with the drivers under the Cameras category.

Hi MartyG, When I run Realsense viewer I keep getting error messages. I
attach a screenshot. You have the same entries in the Camera section of
device manager. If you press the properties button what are the details of
the driver? Is it microsoft or realsens?
[image: image.png]
On Sat, 29 Aug 2020 at 12:11, MartyG-RealSense<EMAIL_ADDRESS>wrote:

On Windows, you can enable camera metadata in the RealSense Viewer
program. If you launch the Viewer with the camera plugged in, you may get a
pop-up message asking if you wish to enable metadata. If you do not get the
pop-up, unplugging the camera and plugging it back into a different USB
port can trigger the message.
If you can get an initial image before disconnection occurs, it sounds as
though the drivers are present and okay. Here is an image of my own Device
Manager with a D435i camera, with the drivers under the Cameras
category.
[image: image]
https://user-images.githubusercontent.com/41145062/91635225-9df83880-e9ee-11ea-9b94-ea5ab74f7dbd.png
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/IntelRealSense/librealsense/issues/7185#issuecomment-683275589,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AFUFEERAA3TKF3HKZQCGNX3SDDO57ANCNFSM4QKQKFWQ
.

Your drivers look fine to me.  The properties of my drivers are below:

Your drivers are from Microsoft mine are from Intel. See Screenshot
[image: image.png]
On Sun, 30 Aug 2020 at 12:11, MartyG-RealSense<EMAIL_ADDRESS>wrote:

Your drivers look fine to me. The properties of my drivers are below:
[image: image]
https://user-images.githubusercontent.com/41145062/91657543-b3866480-eab9-11ea-9714-ba340e0557cb.png
[image: image]
https://user-images.githubusercontent.com/41145062/91657555-c436da80-eab9-11ea-861d-1eb0727f5e37.png
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/IntelRealSense/librealsense/issues/7185#issuecomment-683407318,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AFUFEESHTKIVOZ3AWFLG5OTSDIXVZANCNFSM4QKQKFWQ
.

I have the same date on my drivers as yours.  You have had the RealSense Viewer working before, so there likely is not a problem with the drivers that are installed.

OK. On my first screenshot there was an error shown in the viewer. Any idea
what is causing it?
John K Flynn
On Sun, 30 Aug 2020 at 12:47, MartyG-RealSense<EMAIL_ADDRESS>wrote:

I have the same date on my drivers as yours. You have had the RealSense
Viewer working before, so there likely is not a problem with the drivers
that are installed.
[image: image]
https://user-images.githubusercontent.com/41145062/91658174-d1a29380-eabe-11ea-8f6f-efd7de80db02.png
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/IntelRealSense/librealsense/issues/7185#issuecomment-683410764,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AFUFEEWMJNJE7XCI5JIQ5J3SDI355ANCNFSM4QKQKFWQ
.

None of the images in the replies that you are posting by email are appearing on the GitHub website unfortunately so I cannot see them.  When I said that I could "see" your drivers, I was referring to the text details of them posted earlier.  Apologies for the confusion.
https://github.com/IntelRealSense/librealsense/issues/7185#issuecomment-683272164
The MFCreateDeviceSource errors that you are experiencing are Windows errors and not RealSense errors.  So this suggests that the problem is with the Windows version on the affected computer and not from the RealSense SDK software.
The link below has some information that you can use as a starting point to investigate the problem.  There may not be anything that the RealSense team can do to correct a Windows error on your system that is not caused by RealSense.
https://thegeekpage.com/fix-there-are-no-more-endpoints-available-from-the-endpoint-mapper-error-in-windows-10/

Hi Marty,
I am still having problems. I have been looking at svchost.exe as the culprit, but no suggestions seems to work. One result is realsense viewer v2.38.1 stays open a little longer and asks the question do you want to turn on Advanced mode. When I say yes the viewer shut down. If I tried to switch on the stereo module I get this error message:
:
UNKNOWN in rs2_get_option(options:128C6C60, option:Backlight Compensation):
Device must be powered to query video_proc!
I have searched high and low to try and solve this. The D435 works perfectly on my Desktop, but on my Dell XPS 13 Lap top No Go. Any thoughts would be welcomed! Is there a command prompt I could run to switch on Advanced Mode, or Change the filters?
John

As your problems apparently began at SDK version 2.36.0, it may be worth investigating whether the problem is related to a conflict with an OpenVINO function that was introduced into the Viewer in 2.36.0.
This can be tested simply by downloading the 32.5 MB standalone version of the Viewer - Intel.RealSense.Viewer.exe - from the SDK's Assets list, as this version does not have the OpenVINO function.  Conflicts with the OpenVINO function that occur with the Viewer packaged with the SDK installation therefore do not occur in the standalone version.
You can download the standalone version for SDK 2.38.1 from here:
https://github.com/IntelRealSense/librealsense/releases/tag/v2.38.1

Once downloaded, it should be launched from the location that it was downloaded to and not launched using the desktop icon (which points to the original Viewer version that was installed with the SDK).  If you wish, you can create a desktop shortcut for the standalone Viewer executable.

This is the viewer I am using, and downloaded from where you said. When I select a preset or try to switch on the stereo mode this is the error message I get:
Backend in rs2_get_option(options:110F1D10, option:Exposure):
hr returned: HResult 0xc00d36b6: "This object needs to be initialized before the requested operation can be carried out."
If I try to close the error box I need to press the close button about 15 - 18 times. I can then move around and change the viewing aspect of the desktop, but if I chose the RGB module it closes down the whole process.

One way to eliminate whether the camera is responsible for the problems would be to download a sample bag file from the link below.  Once it is downloaded, unplug the camera, launch the Viewer and try drag and dropping the bag file into the center panel of the Viewer to load the file into the Viewer for playback.
If you are able to change the options in the side panel without problems then this will suggest that something related to the camera being connected to the computer is causing problems.
https://github.com/IntelRealSense/librealsense/blob/master/doc/sample-data.md

I download the stairs bag file and started viewer. It load ok asking to connect a camera. I draged the Stairs. bag and got the following error.

I just tried it again and now it runs the bag file!

Okay, great!  After the bag is loaded in, are you able to use the side-panel options available without an error occurring?

Yes I can access all modules post etc and no errors!

Okay, thanks!  That would suggest that the problem is related to when the camera is connected.  Since the camera functions fine on your desktop computer, it is unlikely that the camera hardware, the camera firmware driver or the USB cable are at fault.  That would make it more likely that the problem is occurring on the USB port.
Do you have a different port on the laptop that you could try the camera in if you have not done so already (either USB 3 or USB 2 will be fine, since a D435 will work with both).

I have tried every  usb port 4 type C on the Laptop and also on the Dell Power connector. I'm beginning to wonder if this Dell xps 13 7390 is a bit too new? The resolution on the screen is 3840x2160. I have tried to lower all the resolutions but it made no difference.

There was a recent case where the camera worked fine in a full size USB port but not in a USB-C port (if you mean you are plugging the camera into a USB-C micro size port on the laptop).  In that case, the RealSense user found that the full size USB ports had an Intel USB controller with xHCI support and the USB-C port that didn't work had a different USB controller without xHCI support.
https://github.com/IntelRealSense/librealsense/issues/6899

I think you are getting closer. This what I found in Device Manager. There two ports that hasxHCI support I have download the camera into each. No joy. When I went to the events viewer. I found they were not starting!  usbxhci.inf. Device PCI\VEN_8086&DEV_15D4&SUBSYS_09621028&REV_02\3936C0722DB3020000 had a problem starting.
Driver Name: usbxhci.inf
Class Guid: {36fc9e60-c465-11cf-8056-444553540000}
Service: USBXHCI
Lower Filters:
Upper Filters:
Problem: 0x15
Problem Status: 0x0

Microsoft has a hidden hardware Troubleshooter option in Windows 10 that can scan for hardware problems and try to automatically correct them.
https://support.intelrealsense.com/hc/en-us/community/posts/360049169174/comments/360012666273

Well sorry to say it did not solve the problem. Uninstalling and reinstalling solved the problems of the drivers not starting. Both USBXHCI.inf and USBXHC.inf started. I ran the recommend realsense viewer 2.38.1 and got  this
. I also ran Dota3d. It couldn't find the camera. Recfusion when it firsts loads it shows a snapshot of what the camera is looking at, but then refuses to show anything else. When you ask it to research it discovers the camera but the screen remains blank. The hidden troubleshooter didn't find a problem.

I considered your latest information carefully.  If the D435 is not working properly with Recfusion or 3DOT, this makes me think that the problem may not be with the RealSense SDK software but with the camera drivers (the firmware driver or the RGB / depth drivers).
Assuming that the firmware driver is okay, since it is stored inside the camera and it works on the desktop computer: that would leave the Depth and RGB drivers.  These are stored on the individual computer, so there is the potential there fore differences between the laptop and the desktop.
Could you open the Device Manager please, go to the Camera category.  Right-click on each of the two drivers and select Uninstall device to remove the driver.  When both of the drivers are uninstalled, unplug the camera, wait a couple of seconds and re-insert the camera.  This should reinstall the two drivers over your internet connection.
This will demonstrate whether your laptop had problems with its installed drivers.

I have done this a number of times now. I have just done once more. It first load Microsoft drivers 2006 version. Realsense viewer just hangs. After another few tries the Realsens Depth and RGB load up.
This allows Realsense viewer to recognise the camera, but won't go further because of an error. e.g. last message. If you press the ok button about 17 times it allows me to expand the sterio mode but not allow me to save. If I press the RGB it bombs out with this error
|librealsense|2.38.1 RELEASE|
|OS|Windows|
Intel RealSense Viewer / Depth Quality Tool has crashed with the following error message:
Program terminated due to an unrecoverable SEH exception:
Access Violation!

I have checked, as much as I can all the drivers loadedn Device Manager. I have a number of different dates on them. I have only had this laptop since July. I used Laplink to copy material from my old one. I wonder if I copied something that is screwing the drivers up? The installation date 27/07/2020. Some of the drivers are from about 20 August to 03/09/2020. I can't see this happening but when you are reaching the limits you try and grasp onto any straw that happens to flow past.

As you have tried most other courses of action, you may as well try upgrading the camera firmware to the latest version (<IP_ADDRESS>), since this is the recommended firmware version for SDK 2.38.1.
If that does not make a difference then I will escalate your case to my manager.

I’ve already tried this. Do you think that it is the latest processor or
other hardware on the dell XP’s 7390? Clutching at straws now.
On Mon, 7 Sep 2020 at 19:36, MartyG-RealSense<EMAIL_ADDRESS>wrote:

As you have tried most other courses of action, you may as well try
upgrading the camera firmware to the latest version (<IP_ADDRESS>), since
this is the recommended firmware version for SDK 2.38.1.
If that does not make a difference then I will escalate your case to my
manager.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/IntelRealSense/librealsense/issues/7185#issuecomment-688468337,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AFUFEEQONYXQEPIXXUSVD7LSEURZHANCNFSM4QKQKFWQ
.
--

John K Flynn

The 400 Series cameras can work with any Intel processor.  There is the issues related to the Viewer and Depth Quality Tool on some computers in https://github.com/IntelRealSense/librealsense/issues/7080#issuecomment-671807292 but that would not account for the improper detection of the camera in software such as Recfusion .
Does your laptop otherwise work okay other than the camera problems and does not have problems with non-camera software?

This laptop is only about two months old. This is the first time I have
tried to connect a 3d camera. I have found an old r300 or r200 and I'm
going to try that to see what happens. I'll let you know.
On Mon, 7 Sep 2020 at 20:06, MartyG-RealSense<EMAIL_ADDRESS>wrote:

The 400 Series cameras can work with any Intel processor. There is the
issues related to the Viewer and Depth Quality Tool on some computers in #7080
(comment)
https://github.com/IntelRealSense/librealsense/issues/7080#issuecomment-671807292
but that would not account for the improper detection of the camera in
software such as Recfusion .
Does your laptop otherwise work okay other than the camera problems and
does not have problems with non-camera software?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/IntelRealSense/librealsense/issues/7185#issuecomment-688478120,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AFUFEEWOF5WJICXQPAFF7CLSEUVM7ANCNFSM4QKQKFWQ
.

The newness of the computer should not make a difference.  Where problms occur where it works on one computer but not another, it can be related to the USB ports on the computer that it does not work with.
Using an old R200 camera requires more setup than the modern cameras.  You have to install its drivers using a special Windows program called the Depth Camera Manager (DCM) and then install an obsolete SDK from 2016 to use with it, as the current SDK is not compatible with R200.

A way to eliminate once and for all whether the laptop's USB ports are the problem may be to attach the camera to a mains electricity powered USB 3 hub.  The hub ports would have their own USB controllers separate from the laptop's, and be supplied with a stable electricity source from the mains socket, helping to avoid glitches related to instability of supplied power to the computer's built-in ports.

At the beginning we thought it was a power problem. I have a number of powered hubs and they didn't work. I bought the official Dell Dock WD19 180W and that didn't solve the problem. At the moment I am going through the whole computer getting rid of anything that hints of camera, realsense opengl etc. In device manager In the view tag I ticked show hidden items. Under cameras there were 8 mentions of D435 depth and RGB I removed them all. I am just about to insert the camera and see what happens.

If you only have one camera attached then there should only be two D435 drivers listed under Camera - RGB and Depth.

You've probably heard a great shout of glee!! Its working! Both Dot3D and RecFusion. These are the steps I did. I uninstalled OpenNI 2.2 SDK prime sense.
I uninstalled openGL.
I uninstalled Realsense sdk v2.38.1.
In device manager I used the view tag to show hidden things.
In the camera section 8 hidden entries appeared 4 for d435 Depth camera and 4 for d435 RGB I uninstalled them all.
I didn't use realsense viewer to open the camera. I used dot3d.
I will try realsense viewer2.38.1 later to see if it now works. I just wanted the pleasure of seeing the camera work! MartyG A great Big Thank You!!!

Awesome news!   I wish you the best of luck with oyur further tests   :)

Just to bring everything up-to-date. I have tried the camera with 3 or 4 different scanning programmes, all ok. So I then turned to RealsenseViewer 2.38.1. The programme started without any error, but wouldn't allow me to turn on either the stereo module or the RGB module stopping with this error message: Invalid Value in rs2_open_multiple(sensor:103592C0, profiles:0EF75AD0, count:1):
Failed to resolve the request:
Format: Y8, width: 1280, height: 800
Into:
Formats:
Y8I

Apologies for the delay in responding further.  I did tests with D415 and D435.  1280x800 was available in USB 3 mode but only as high as 1280x720 in USB2 mode.  So I suspect that you may have got the "Failed to resolve the request" message because your camera was in USB2 mode at the time.
This error typically occurs when the SDK is asked to do something that it cannot carry out, perhaps due to a mistake in the stream definition that the SDK cannot understand or because it is a mode that this particular camera doesn't support at the time the request is made (e.g due to being in the more limited USB2 mode at the time)..

How would you switch it between between usb2 and 3? I only have usb 3 and 3.1. One of the things that I remembered was that this xps laptop 7390 comes with a fantastic scree.n When I checked just now it is running at 3840X2160. Perhaps this is what causes the problem. I was reading awhile ago, when you start an app the writing is so small on the screen that it is impossible to read. I read that this was because old Graphic code was compiled into the programme that didn't take advantage of the dynamic change in up-to-date resolutions. I did start to lower the resolutions but I am afraid I didn't continue down that path because I was side-tracked. I may give it another go.

You do not need a USB2 port in order for the camera to be mis-detected as USB2.  Often, it can happen when the camera is in a USB3 port.  I believe it is related to how the USB controller is currently interpreting the state of the port.
An example of mis-detection that can occur is when the camera's USB connector is inserted slowly instead of quickly and firmly, or if it jiggles slightly back out of the port (screw-lock USB cables can help to avoid this by securing the camera firmly in the port).
I haven't heard of librealsense being used on a screen with 3840X2160 resolution and there appears to be no information about anyone else having done so, unfortunately.  So I have no advice that I can offer about that.

Hi @johnkflynn   Do you require further assistance with this case, please?   Thanks!

Hi MartyG,
Thanks for all your help.Everything seems to be working fine. I keep away
from the relsense viewer. Sometimes it works and other times it disconnects.
John
John K Flynn
On Fri, 18 Sep 2020 at 19:46, MartyG-RealSense<EMAIL_ADDRESS>wrote:

Hi @johnkflynn https://github.com/johnkflynn Do you require further
assistance with this case, please? Thanks!
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/IntelRealSense/librealsense/issues/7185#issuecomment-695028448,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AFUFEEUG4RWQ3AQ4DIWI5TDSGOTJ5ANCNFSM4QKQKFWQ
.

Okay, so is it okay to close this case please if you are satisfied that you do not need further help?   Thanks for the update!

Case closed due to no further comments received.

This issue went through a lot, but I had the exact same error message as the original post and this was the only relevant Google result. As I've lost 2 weeks of my life finding the issue, I figured I would share the solution.
https://stackoverflow.com/questions/68638694/there-are-no-more-endpoints-available-from-the-endpoint-mapper-when-creating
TL;DR: Make sure the "Windows Camera Frame Server" service is running.

Thanks so much @Freefly18 for sharing the solution that worked for you with the RealSense community!  I'm sorry to hear that you had such a difficult experience.


Update rail only if controls are visible.

'timeupdate' eventListener calls the updateCurrent and updateDuration methods even if the controls are hidden. This is unnecessary as tons of CPU cycles are wasted which results in bad battery life. So make the eventListener check controlsAreVisible.

@vivekannan Thanks for pointing this. We are preparing a PR to solve this issue

Actually, its seem that the timeupdate event is assured to be called at least once every 250ms in Chrome and other browsers. In Firefox its even worse (or better), for every frame!!! I am working on an app that uses this player as a base where battery life is important. So I re-wrote the code so that the callback itself is removed after the controls are hidden and reapplied when they are visible. Makes a reallllly big difference in battery life when you are watching hours of videos.

@vivekannan I see. Check the PR linked to this issue and let me know if that works for you

In my app, I have changed it into something like this,
function showControls() {
    //show controls code
   this.media.addEventListener('timeupdate', this.timeUpdateCallback);
}

function hideControls() {
    //hide controls code
   this.media.removeEventListener('timeupdate', this.timeUpdateCallback);
}

function timeUpdateCallback() {
   // notice the this.player reference as 'this' refers to the media DOM element here as this function
   // is called with the media element as context
   this.player.updateDuration();
   this.player.updateCurrent();
}

Thanks for sharing your answer. Did you by any chance tested the PR linked to this issue? Just to verify that it solves your issue as well from the core

Yeah, this should work. This was my initial solution. Shall I close the issue?

@vivekannan You can close this one once it's merged into the master branch. I'll let you know when this happens


Observable footer

ported from https://github.com/d3/d3/commit/1582f4afb151f9705ebd43b7839456c841af3319
changes:

copyright: Observable, Inc.
releases URL
license URL

I think it's fine with the link in the header, but we can revisit


Related rows menu item on row pages

Unlike the semantic search button this won't need the OpenAI API key to be set, because you don't need to calculate a new embedding to run this.

Prototype of the SQL for this feature against a table with a compound primary key:
select
  "news".*,
  embeddings_cosine(
    "_embeddings_news"."emb_text_embedding_3_small_512",
    (select
      emb_text_embedding_3_small_512
    from
      _embeddings_news
    where
      id = 102
      and date = '2018-01-28')
  ) as _similarity
from
  "news"
  join "_embeddings_news" on news.id = _embeddings_news.id
  and news.date = _embeddings_news.date
where
  "_embeddings_news"."emb_text_embedding_3_small_512" is not null
order by
  _similarity desc


.NET Core-Support

What would you like to be added:
Hi Syft-Team,
are there any plans to support generating SBOMs for .NET Core projects?
Why is this needed:
.NET Core is a widely used ecosystem.
Additional context:

Hi @ckotzbauer, I'd love to see this! I don't have much experience with .NET Core personally. It'd be good for us to capture the details of what we'd need to scan here in the issue.

@luhring
I have a couple of questions in mind for that:

Should the cataloguing happen only in compiled state or from source too? (When source is relevant too, this would be better to handle with #373)
What information is needed to generate a spec-compliant SBOM? Dependencies, Versions, Licenses, ...

When the project is compiled, I think the most important thing would be the .deps.json file which is generated from dotnet and placed at the compilation-target dir.
This is a very basic example from the microsoft docs (e.g. Package licenses are not covered there):
{
  "runtimeTarget": {
    "name": ".NETCoreApp,Version=v3.0",
    "signature": ""
  },
  "compilationOptions": {},
  "targets": {
    ".NETCoreApp,Version=v3.0": {
      "store.manifest/1.0.0": {
        "dependencies": {
          "StartupDiagnostics": "1.0.0"
        },
        "runtime": {
          "store.manifest.dll": {}
        }
      },
      "StartupDiagnostics/1.0.0": {
        "runtime": {
          "lib/netcoreapp3.0/StartupDiagnostics.dll": {
            "assemblyVersion": "<IP_ADDRESS>",
            "fileVersion": "<IP_ADDRESS>"
          }
        }
      }
    }
  },
  "libraries": {
    "store.manifest/1.0.0": {
      "type": "project",
      "serviceable": false,
      "sha512": ""
    },
    "StartupDiagnostics/1.0.0": {
      "type": "package",
      "serviceable": true,
      "sha512": "sha512-xrhzuNSyM5/f4ZswhooJ9dmIYLP64wMnqUJSyTKVDKDVj5T+qtzypl8JmM/aFJLLpYrf0FYpVWvGujd7/FfMEw==",
      "path": "startupdiagnostics/1.0.0",
      "hashPath": "startupdiagnostics.1.0.0.nupkg.sha512"
    }
  }
}

@luhring  Opened a PR #951 with initial support


SQL INSERT INTO SELECT is not supported in Optimizer

I have found that INSERT INTO SELECT is not supported in our optimizers.  I modified the soon-to-be-dumped simple optimizer to check the overall status of  this in my repository, and it worked OK.  This means that, if we add a support in our new optimizer, the function will just work.
This is a typical test that should work when it is supported: INSERT a,b,c into table1 SELECT a,b,c from table1 where a=10;.

Verified by #762