Refresh Docker Examples

The docker examples use out of date images that have subsequently been replaced by an official versions.
https://www.elastic.co/guide/en/elasticsearch/reference/current/docker.html
https://www.elastic.co/guide/en/kibana/current/_configuring_kibana_on_docker.html
https://www.elastic.co/guide/en/logstash/current/docker.html
https://www.elastic.co/guide/en/beats/filebeat/current/running-on-docker.html
The proposal is to use docker compose to produce an Elastic Stack example that ingests an example log file using two archiectures:

Filebeat->Elasticsearch<-Kibana
Filebeat->Logstash->Elasticsearch<-Kibana

We can provide more complex examples over time.
@rudijs given your initial contribution would you be interested in assisting?
@dliappis @ESamir  do we have examples for the above already?
@asawariS

@gingerwizard Hi Dale, yep I'm interested to help out. Starting the end of next week I'll have some time to follow up on this. I'll clone and create a branch to get started.

Hi @rudijs we have actually started an effort here.  This PR will deliver a compose file that starts up a full stack with beats,LS,ES and kibana containers + supporting apache, nginix and mysql.
The latter will act as datasources. The file also installs also beats and x-pack thus giving a full stack experience. I would appreciate a review if you have time when i make the PR.

https://github.com/elastic/examples/pull/165


Packaged exe getting hang while opening.

I had open linked project in 4.6.1 and package it in windows x64.
When I launch .exe it is hanged.

Still having issues?
What version are you using now? If you are using the latest version, you could try my fork.
I am not going to let this amazing plugin die - even if it is not founded by me :D


Create codeql.yml

This enables the CodeQL scanner in this repo.

Codecov Report
Base: 100.00% // Head: 100.00% // No change to project coverage :thumbsup:

Coverage data is based on head (757d535) compared to base (e784e45).
Patch has no changes to coverable lines.

Additional details and impacted files
@@            Coverage Diff            @@
##              main        #8   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files            3         3
  Lines           52        52
=========================================
  Hits            52        52

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


Scope the custom-elements-manifest.mjs to a directory and ignore itself.

https://github.com/Shopify/ruby-lsp/issues/2918
Sometimes I'm working on repos without the analyzer, and creating the custom-elements-manifest.mjs can be quite disruptive.
could we do something like:
.cem_lsp
|-- .gitignore
|-- custom-elements-manifest.config.mjs

and in the .gitignore have the following: *
happy to make a PR if you're open to it.

Hi Konnor, long time now ~see~ uhm ~talk~ ehh write!
I like the idea and am open to supporting it. I've not had much time to put into this project as of late but am planning on picking up the slack come next year.
I took a look at https://github.com/Matsuuu/custom-elements-language-server/blob/main/lib/custom-elements-languageserver-core/src/cem/analyzer.ts#L161-L198 , and from what I've gathered from the ancient hieroglyphs I've written ages ago, this should already be supported..?
Can you try out using a cem config like your proposal and report back if it works?

Hi there! It has been a while!
and im not talking about my own manifest, for some reason the language server is generating this file for me, even when I have custom-elements-manifest.js in my repo. Which as i look at it now, may not be detected by the LSP 🤔

I think due to how vscode works, I'm forced to use mjs files for configuration at least in the current version. There was some really iffy stuff that stopped me from utilizing all js file configs but I could look into it now with fresh eyes. That might be the reason it's not noticing your file.

Looking at the code, I can't even remember writing this workaround but noticed it: https://github.com/Matsuuu/custom-elements-language-server/blob/main/lib/custom-elements-languageserver-core/src/cem/analyzer.ts#L135-L152
It generates the mjs version for the plugin to parse, and should delete it after that, but seems like there's something stopping the plugin from deleting it, therefore it not being deleted.
As for remedies to your trouble right now, I'd recommend trying to port your .js config to mjs for now, if possible. That should resolve some of the issues for now.
I am currently in the middle of a huge crunch period at our company so time for this project is extremely limited, but I'll try and find some time to work on this in the coming month or two.


Initial 0.31.0 commit.

Checklist

[x] Used a personal fork of the feedstock to propose changes
[x] Bumped the build number (if the version is unchanged)
[x] Reset the build number to 0 (if the version changed)
[x] Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
[x] Ensured the license file is being packaged.

@conda-forge-admin, please rerender

Hi @leofang , @BastianZim , the new Kokkos package is built with CUDA as of PR 18. For now, I've pinned the Kokkos version, but I was wondering what would be the best way to deal with that considering that PennyLane does not depend on CUDA.

I see that this is being discussed in https://github.com/conda-forge/kokkos-feedstock/issues/23. I also just noticed this. (And, again, conda & mamba exhibit different behaviors...)

Yes, thanks for weighing in and providing a solution. Once this is resolve, I'll revise the recipe and relax the kokkos pinning.

When the changes introduced in #24 are effective in Kokkos v4, I'll relax the Kokkos version.

@conda-forge-admin, please rerender

Ok, so OSX-ARM builds are failing. ENABLE_OPENMP=OFF but there is an error
2023-06-28T18:56:09.3561180Z CMake Error at /Users/runner/miniforge3/conda-bld/pennylane-suite_1687978107802/_build_env/share/cmake-3.26/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
2023-06-28T18:56:09.3564370Z   Could NOT find OpenMP_CXX (missing: OpenMP_CXX_FLAGS OpenMP_CXX_LIB_NAMES)
2023-06-28T18:56:09.3565150Z Call Stack (most recent call first):
2023-06-28T18:56:09.3565900Z   /Users/runner/miniforge3/conda-bld/pennylane-suite_1687978107802/_build_env/share/cmake-3.26/Modules/FindPackageHandleStandardArgs.cmake:600 (_FPHSA_FAILURE_MESSAGE)
2023-06-28T18:56:09.3566660Z   /Users/runner/miniforge3/conda-bld/pennylane-suite_1687978107802/_build_env/share/cmake-3.26/Modules/FindOpenMP.cmake:577 (find_package_handle_standard_args)
2023-06-28T18:56:09.3567290Z   /Users/runner/miniforge3/conda-bld/pennylane-suite_1687978107802/_build_env/share/cmake-3.26/Modules/CMakeFindDependencyMacro.cmake:76 (find_package)
2023-06-28T18:56:09.3568320Z   /Users/runner/miniforge3/conda-bld/pennylane-suite_1687978107802/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pla/lib/cmake/Kokkos/KokkosConfig.cmake:39 (FIND_DEPENDENCY)
2023-06-28T18:56:09.3568890Z   cmake/process_options.cmake:19 (find_package)
2023-06-28T18:56:09.3569210Z   cmake/process_options.cmake:233 (FindKokkos)
2023-06-28T18:56:09.3569910Z   CMakeLists.txt:65 (include)

The error didn't happen when pulling Kokkos 3.7.02. Any idea?

I'm going to merge this now and come back for Kokkos v4 in another PR.


Implement battery status "charge state"

Implement battery status "charge state" on mavlink battery status message
Describe problem solved by this pull request
the field charge_state is implemented on the uorb side of the code (named battery_state) but not being sent to the ground.
Describe your solution
Pass the data from the uorb to mavlink side
Test data / coverage
simulator
Additional context
Also added the missing enum values of UNHEALTHY & CHARGING to the uorb enum, but there is no implementation for them for now

We can likely implement the charge state on the PX4 side by monitoring when the power module current measurement goes negative.

We can likely implement the charge state on the PX4 side by monitoring when the power module current measurement goes negative.

you right, but that for other pr. I think

@mhkabir

Seems harmless,

Its harmless but let ground control to know the state of the battery, without needing to read the parameters. I am not sure how QGC decide to paint the battery icon to red, but this field should be the index for that.

... but I'm not sure about the semantics of the "charging" state being a "warning". Maybe better to rename the uorb message field to "state"?

That can be done, but it will require changes on other places that use the battery_status_s::BATTERY_WARNING_xxx. suce the land detector, the commander and so on.
I can do that. if you think its better...

Let's rename the field then.

Please carefully check for any logic that relies on the numeric value of the warning (now state) increasing with severity.

@mhkabir with your and @dagar reviews I think its better to remove the UNHEALTHY and CHARGE states from the uorb, to revert the 'state' back to be 'warning'. so the number of the var is increasing when the battery health degrading.
there are more points other than the two that you ask me to look at that uses the warning as 'degrading'.
anyway, for now UNHEALTHY and CHARGE can't be set by the publisher. and when it will it can be put on other variable.
So basically on the last two commits I pushed, I reverted the warning to state.
If you think that is ok, I can squash all that mess to one commit...
Do you agree?

@dagar @mhkabir , any reason not to merge this?


Cloudflare Workers guide needs PDP clarifications

Emily did a walk-through of the guide, and found several areas where we can improve our docs. For details, refer to this Notion doc.

Duplicate of https://github.com/prisma/docs/issues/3557


Feature: Fix comment edit/delete permissions

Purpose:
Fixes https://openscience.atlassian.net/browse/OSF-5681
If a project's comment level changes from public to private, non-contributors can still edit and delete any comments that they made while the comment level was public.
Changes:

The can_edit field on the CommentSerializer now checks if the user can comment on the project (not just if they are the author).
The CommentDetailPermissions.has_object_permission also checks if the user can comment on the project.

Pass complete, no bugs found :star2:
Putting this in RTM when the build passes.


Automator: update common-files@master in istio/ztunnel@master

Generated by Automator - 2024-10-15T19:15:44+00:00

@istio-testing: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

test_ztunnel
aea1d7b68a2a0b3435175460eaf2d591f807c671
link
true
/test test

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

/retest

@istio-testing: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

test_ztunnel
aea1d7b68a2a0b3435175460eaf2d591f807c671
link
true
/test test

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@istio-testing: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

test_ztunnel
340f0d742952dde958b341c22b75c9ac197f7612
link
true
/test test

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

/retest


Add dot env for setup local dev environment

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 100.0%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
857

Relevant Lines:
857

💛  - Coveralls


Multi Project repo - error in Test Current, Next Minor, and Next Major

When building one of the projects, which has the dependency to another, it shows this error
Error: The remote server returned an error: (403) Forbidden.
This comes after stating that the base app wasn't built and its trying to download from artifacts. However, the base app was built in the previous build step.
It's the same repo you already have access to from yesterday.
Am I doing something wrong again?

The "download from different repo" is the one base app that is defined in a different repo. However, the other dependency is in the same repo and is built before this app. This is the full error:
Checking project dependencies
Checking appDependencyProbingPaths
Dependency to projects '*' in https://github.com/nav-x-llc/library@main, version latest, release status release
Warning: No performance test apps found in bcptTestFolders in .AL-Go\settings.json
Project: xxx-Manufacturing
Project dependencies: xxx-Customizations
Locating all Apps artifacts from probing paths
xxx-Customizations not build, downloading from artifacts
Analyzing artifacts
https://api.github.com/repos/nav-x-llc/xxx/actions/artifacts?per_page=100&page=1
Error: The remote server returned an error: (403) Forbidden.

Forbidden

The token that I have defined is a token that has full access to anything. The main thing is that in the "Test Current, Next Major..." it should be building the apps, but it says "not build, downloading from artifacts". Why would it not build it or rather it did build it, but why would it think that it didn't build it?

This is a bug when using UseProjectDependencies.
A design flaw actually - will find a way around this.

The problem was, that in PR check builds, test current, test next minor and test next major - the workflow doesn't publish any artifacts - and shouldn't
The fix is to create apps and testapps artifacts with a different name pattern and have a time-to-live of 1 day.


Simple example

Hi Tobias,
Thanks for posting your Event System, I saw you give an overview of this system on episode 13 of The Hot Path Show and I think you gave a pretty good high level overview of how it works. Would it be possible to have a simple example of how to use this? I have been trying to setup a simple test unsuccessfully. Maybe just an example that queues up some numbers or something on a button press and debugs the number output?

Hello! Sorry, I did not see this. I will put up an example tomorrow!


Be fully compatible with PHP 5.6's built-in cookie components

Introduce the solution accepted by @ocram in #11.
Tests pass now: https://travis-ci.org/LeSuisse/PHP-Cookie/builds/184847081

Thank you so much!
Merged in https://github.com/delight-im/PHP-Cookie/commit/22f2c19750a6ad3dbf69a8ef3ea0e454a8e064fa


React v15.0

Please provide support for React 15.0.

Tested with React 15.0.1 and the component works fine, just needs to update the dependencies.


Foxx manifest does not accept script names with 3 "-" in the name

This is really funny, but @pluma asked for an investigative ticket about this.
In my manifest.json:
  "scripts": {
    "setup": "scripts/setup.js",
    "teardown": "scripts/teardown.js",
    "clean-by-user": "scripts/clean-by-user.js"
  },

this triggers:
2015-10-19T14:58:56 [3088] ERROR Error in foxx route '{ "match" : "/foxxes/scripts/:name", "methods" : [ "post" ], "constraint" : { "name" : "/[^/]+/" } }': 'Arguments to path.join must be strings - have undefined', Stacktrace:
2015-10-19T14:58:56 [3088] ERROR TypeError: Arguments to path.join must be strings - have undefined
2015-10-19T14:58:56 [3088] ERROR   at /usr/share/arangodb/js/common/bootstrap/module-fs.js:318:15
2015-10-19T14:58:56 [3088] ERROR   at Array.filter (native)
2015-10-19T14:58:56 [3088] ERROR   at Object.exports.join (/usr/share/arangodb/js/common/bootstrap/module-fs.js:316:28)
2015-10-19T14:58:56 [3088] ERROR   at ArangoApp.loadAppScript (/usr/share/arangodb/js/server/modules/org/arangodb/foxx/arangoApp.js:423:17)
2015-10-19T14:58:56 [3088] ERROR   at executeAppScript (/usr/share/arangodb/js/server/modules/org/arangodb/foxx/manager.js:510:18)
2015-10-19T14:58:56 [3088] ERROR   at Object.runScript (/usr/share/arangodb/js/server/modules/org/arangodb/foxx/manager.js:814:10)
2015-10-19T14:58:56 [3088] ERROR   at /usr/share/arangodb/js/apps/system/_admin/aardvark/APP/foxxes.js:289:28

If I change the name to clean-user then it works fine.

Likely already existed in 2.6. I'll look into this when I'm on site tomorrow.

This is weird. The problem seems to occur in fs.join(this._root, this._path, filename) (as the error indicates, one of the arguments is undefined). I'm not sure how this can vary based on the number of dashes in the script name. I'll have to investigate further.

I'm pretty sure this bug will no longer be reproducible in 2.8 as the implementation of the module loader has changed but I'll leave this open for now.

Unassigning this as I'm knee-deep in Foxx and can't reproduce the problem in 2.8. Someone should try to reproduce this in 2.7 and see if we can patch it.

This also seems to no longer happen in 3.0. Closing this as it's unlikely we'll backport a bugfix to 2.7 at this point.


Networking: Do not scan the network interfaces in the namespace in hyperstart.go

We scan the network namespace twice in the case of CNM. Once once the pre-hook returns and the second time in hyperstart.go.
This is done to handle the CNI case where the spec does not return the MTU of the network interface.
https://github.com/containernetworking/cni/blob/master/SPEC.md#result
We do not scan the network ns in the case of CNI as we rely on the result field.
We should scan the network ns even in the case of CNI as the namespace may change later due to the changes we may need to make to connect the network to the VM.

https://github.com/containernetworking/cni/issues/513

@mcastelino several comments about this issue. I agree I would prefer not to scan several times the network namespace, it could save us some time. But right now, because we are relying on the Result type from CNI package, even in case of CNM implementation, hyperstart needs to read again because the results does not carry the MTU info.
I have seen the comment about the issue you've opened on CNI repo, and I wonder if we should not move to our own NetworkResult structure so that we can have all the field we want without having to justificate everything and battle for a new field. @sameo what do you think about us defining our own Result structure ? Also, I think it could help us to simplify the translation into Hyperstart fields. And we could follow that by moving the network scan to the network interface level (I am suggesting that we don't rely on specific return from CNI but scan whatever network plugin we're using).

@sboeuf yes we should define our own results structure that embeds the CNI struct. We need more information than what CNI provides. Also I would to scan upfront and populate this struct vs scanning later. We should however retain what the CNI plugin returns as it may contain information we may not be able to obtain via scanning easily.
Once we have support for things like SRIOV and vhost-user this scanning in hyper start will not work anyway. I had to hack up the macvtap implementation also due to this issue.

@mcastelino @sboeuf Just to add here, for the SRIOV case,  I am fetching the MTU info the first time the network namespace is scanned. Physical endpoints are simply ignored during the second scan, as they are not even available in the namespace after being assigned to the VM.

@mcastelino @sboeuf It makes perfect sense to define our own networking result structure and include the CNI Result data in it.

@sboeuf with anonymous struct fields we will not be bound to the CNI results struct. If we do go away from the CNI result struct the dependent code will not change as long as the field names are retained.
@amshinde do you want to make this change or should I?

@mcastelino I am actually working on this change.

@amshinde I recollect that in your VFIO PR also you have some code in the rescan. So this PR may change/simplify your code also.

@mcastelino https://github.com/containers/virtcontainers/pull/470

@amshinde @sboeuf   Fixed via #470.


Handle null values passed to an include statement

Previously a null value was interpreted as 'not present in this context' and the parent context was queried. If the parent had a non-null value then the wrong value was used in the included template.
This PR provides a test case and the fix.

As it appears this repo is dying, I have decied to create and maintain a new fork of Pebble at JavaEden/Pebble, and I would love to get this PR merged into the new repo. Please take this PR and open a new one to the dev branch of JavaEden/pebble.
The future of Pebble has been shaky for a while now, but I refuse to let Pebble die. It is a great project, and I hope to give it same love and care that the original maintainers have as they built Pebble. If you have any questions about the future of Pebble, please reach out on Gitter. Let's keep this community alive, I hope to see you over at the new location!
New Pebble Home

Original announcement of new ownership: https://github.com/PebbleTemplates/pebble/issues/311#issuecomment-379496941
https://github.com/JavaEden/pebble
https://bintray.com/javaeden/Eden/Pebble
https://gitter.im/JavaEden/pebble


Inconsistencies in upload/processing

Describe the bug
~65% of files uploaded fail to upload/be processed
To Reproduce
Deploy solution
Upload 100 pdfs
Note that on average, only ~65 of them process successfully
Expected behavior
I would've expected no more than a couple to fail per 100.
Please complete the following information about the solution:

[] Version: [e.g. v1.0.0]
[] Region: [e.g. us-east-1]
[y] Was the solution modified from the version published on this repository?
I downsized the elasticsearch container is all.
[n] If the answer to the previous question was yes, are the changes available on GitHub?
[y] Have you checked your service quotas for the sevices this solution uses?
[y] Were there any errors in the CloudWatch Logs?
I found errors in the lambda dashboard but couldn't hunt them down in cloudwatch successfully.

Screenshots
If applicable, add screenshots to help explain your problem (please DO NOT include sensitive information).
Additional context
I thought it was odd that it failed to process so much, but that it must be a bug of some sort when many failed to even upload in the first place. Additionally, all documents that will pass processing do so rather quickly (~1-2 mins), then all the failures seem to wait for a max timeout (5-10mins). This puzzled me as it seems like there's a max timeout with no retry logic maybe?

Hi!  Could you share more about how the 100 pdfs were uploaded?  Did you upload them manually through the UI or did you dumped the pdfs into the bulk processing s3 bucket?

Hi!  Could you share more about how the 100 pdfs were uploaded?  Did you upload them manually through the UI or did you dumped the pdfs into the bulk processing s3 bucket?

Also, what size and how many pages were the pdfs used?

Also, what size and how many pages were the pdfs used?

yup sorry. these were uploaded in a batch via UI. Most fo the pdfs were ~100kb size, some as large as 2mb, most were ~2pages long, some closer to 20-30 pages.

yup sorry. these were uploaded in a batch via UI. Most fo the pdfs were ~100kb size, some as large as 2mb, most were ~2pages long, some closer to 20-30 pages.

I tested my own deployment today and I'm not able to reproduce the issue.  I used the UI for upload and drag/dropped 100 pdfs of sizes similar to yours.  I noticed only a few that wouldn't upload due to my weak internet connectivity.  For those cases you see a red "X" on the upload page.  Is this what you're seeing?  This might be caused by the upload time limit for each.  Those pdfs that were successfully uploaded however were processed to completion for me: their status becomes "Ready" in the documents listing.  Otherwise could there be some concurrency with other workloads running in your account?  like other lambda functions.

I tested my own deployment today and I'm not able to reproduce the issue.  I used the UI for upload and drag/dropped 100 pdfs of sizes similar to yours.  I noticed only a few that wouldn't upload due to my weak internet connectivity.  For those cases you see a red "X" on the upload page.  Is this what you're seeing?  This might be caused by the upload time limit for each.  Those pdfs that were successfully uploaded however were processed to completion for me: their status becomes "Ready" in the documents listing.  Otherwise could there be some concurrency with other workloads running in your account?  like other lambda functions.

Nothing else running in my account atm but the solution. Any reason why the downiszed elasticsearch could cause this? I can't see why. It seems strange because the majority of the failures were after uploading, i.e. not an internet connectivity issue.

Nothing else running in my account atm but the solution. Any reason why the downiszed elasticsearch could cause this? I can't see why. It seems strange because the majority of the failures were after uploading, i.e. not an internet connectivity issue.

Thanks for looking into this i'll dig into my logs but it's good to have the sanity check.

Thanks for looking into this i'll dig into my logs but it's good to have the sanity check.

I'm able to reproduce something that may look like your issue when I reduce the size of elasticsearch.   I see that many pdfs in my batch of 100 don't fully upload successfully into the system.  When I look into DynamoDB (the DUSStack-dusstackdocumentstable...) I see only about 60% appearing there fully ingested.  Those there have all succeeded in their processing, i see no failures.  So it appears that the document ingestion is stressed when elasticsearch is sized too small.  You could try with the regular size elasticsearch, see if that makes a diff.

I'm able to reproduce something that may look like your issue when I reduce the size of elasticsearch.   I see that many pdfs in my batch of 100 don't fully upload successfully into the system.  When I look into DynamoDB (the DUSStack-dusstackdocumentstable...) I see only about 60% appearing there fully ingested.  Those there have all succeeded in their processing, i see no failures.  So it appears that the document ingestion is stressed when elasticsearch is sized too small.  You could try with the regular size elasticsearch, see if that makes a diff.

Thank you! I'll give it a go.
On Fri, Jan 29, 2021, 12:11 PM Pierre<EMAIL_ADDRESS>wrote:

I'm able to reproduce something that may look like your issue when I
reduce the size of elasticsearch. I see that many pdfs in my batch of 100
don't fully upload successfully into the system. When I look into DynamoDB
(the DUSStack-dusstackdocumentstable...) I see only about 60% appearing
there fully ingested. Those there have all succeeded in their processing, i
see no failures. So it appears that the document ingestion is stressed when
elasticsearch is sized too small. You could try with the regular size
elasticsearch, see if that makes a diff.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/awslabs/document-understanding-solution/issues/151#issuecomment-769993948,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ALDJQRU2K26CH44HDWDVQL3S4MB47ANCNFSM4WSUTCHA
.

Thank you! I'll give it a go.
On Fri, Jan 29, 2021, 12:11 PM Pierre<EMAIL_ADDRESS>wrote:

I'm able to reproduce something that may look like your issue when I
reduce the size of elasticsearch. I see that many pdfs in my batch of 100
don't fully upload successfully into the system. When I look into DynamoDB
(the DUSStack-dusstackdocumentstable...) I see only about 60% appearing
there fully ingested. Those there have all succeeded in their processing, i
see no failures. So it appears that the document ingestion is stressed when
elasticsearch is sized too small. You could try with the regular size
elasticsearch, see if that makes a diff.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/awslabs/document-understanding-solution/issues/151#issuecomment-769993948,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ALDJQRU2K26CH44HDWDVQL3S4MB47ANCNFSM4WSUTCHA
.

Also, on the files not uploading, I'm not sure it's actually a connectivity issue. I checked the console and noticed 1 CORS error per failure:
select:1 Access to XMLHttpRequest at 'https://povxsia3kb.execute-api.us-east-1.amazonaws.com/prod/document' from origin 'https://dfw4r8wch7jpu.cloudfront.net' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.
Looks like CORS headers might not be getting set properly for every request to store in S3? like a missing await on the request to get CORS header maybe?

Also, on the files not uploading, I'm not sure it's actually a connectivity issue. I checked the console and noticed 1 CORS error per failure:
select:1 Access to XMLHttpRequest at 'https://povxsia3kb.execute-api.us-east-1.amazonaws.com/prod/document' from origin 'https://dfw4r8wch7jpu.cloudfront.net' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.
Looks like CORS headers might not be getting set properly for every request to store in S3? like a missing await on the request to get CORS header maybe?

UPDATE:
Resizing the elasticsearch didn't seem to help. In fact, I redeployed the entire solution and it didn't to behave any differently.
I was getting the following error in DUSStack-dusstackjobresultprocessor... lambda:
[ERROR] AttributeError: 'list' object has no attribute 'add' Traceback (most recent call last):   File "/var/task/lambda_function.py", line 192, in lambda_handler     return processRequest(request)   File "/var/task/lambda_function.py", line 156, in processRequest     comprehendAndMedicalEntities[key].add(val)
I deleted the entire blocks of code dedicated to comprehend medical & Kendra, as I'm not interested in either of those, and have now uploaded 2 batches of ~30 pdfs with no failures.

Thank you so much for this new info! I think we have two issues.  The resizing is definitely making my deployment struggling with the ingestion of a large batch of pdfs.  But secondly, I think some of your pdfs have something in them that makes our code throw the error you are reporting.  Would it be possible to share links to your pdfs that cause the issue, so we can use them in test?  Hopefully those have copyrights that allow this?

After looking at the code, I can see the need to replace that list add() with an append() instead.  Hopefully this will solve the issue,  expect a commit in the next days.  Still, that would be awesome if you could provide some links to your pdfs that seem to trigger that code execution there.

I'm afraid I can't provide the PDFs as they have PII in them, but I will note that no single PDF would fail every time. Always around a third, but never necessarily the same ones, so I don't think it's related to their specific contents.

Hi @spencerrichardhenry There seems to be a bug in the way some items were being processed for documents.
Based on your logs, we have pushed a fix : https://github.com/awslabs/document-understanding-solution/pull/154
Please update your solution and let us know if you still see the issues

looks good


Add pseudo field for windowrulev2

Describe your PR, what does it fix/add?
As the title says, this adds a "pseudo:" field for windowrulev2.
Is it ready for merging, or does it need work?
It's ready.

hm. I don't like this. Pseudo is... technically layout-specific. Maybe we should add some method to define layout-specific windowrules? like layout:...?

Indeed, pseudo: is technically layout-specific, but the window still holds the pseudo attribute on every layout.
That being said, I added a layout: field and made so updateDynamicRules() is called on all windows every time handleWindowRuleV2() is ran.

hm, I am unsure how to deal with this... nicely. Considering pseudo is exposed to everything, I'd say let's just keep it the old way for now I guess.. I'll do something in the future with this

Stupid question/suggestion. Can't you just use window tagging for some of this? Layouts change behavior/do magic stuff based on existence of some window tag(s). Users can tag windows however they want (rules, manually etc) and stuff Just Works[tm].
Just document which tags do what for the layouts.
I dunno, just seems like the tag system could lead to some pretty useful/powerful stuff layout wise if the layouts wanted to take advantage. I guess the one downside is tags can't have values, hmmm.
You'd still have the 'layout specific and possibly colliding' thing either way tho. Although some tags might make sense to share between layouts (all master/stack based layouts could use a 'master' tag on windows to place them into the master stack, etc)


[Improve][Zeta] Disable hdfs filesystem cache of checkpoint

…cache function is disabled by default.

Purpose of this pull request

Does this PR introduce any user-facing change?

When using hadoop-aws-3.1.4.jar and aws-java-sdk-bundle-1.11.271.jarto connect hdfs or s3 file systems, the default mode is cache. In multithreaded scenarios, FileSysyem objects are often closed, resulting in the closure of the connection pool. If the objects are taken from the cache, some unknown exceptions will be caused
How was this patch tested?

no
Check list

[ ] If any new Jar binary package adding in your PR, please add License Notice according
New License Guide
[ ] If necessary, please update the documentation to describe the new feature. https://github.com/apache/seatunnel/tree/dev/docs
[ ] If you are contributing the connector code, please check that the following files are updated:

Update change log that in connector document. For more details you can refer to connector-v2
Update plugin-mapping.properties and add new connector information in it
Update the pom file of seatunnel-dist

[ ] Update the release-note.

I don't have an oss environment, so I can't run oss unit tests:

Could you add a test case to cover this bug?

Could you add a test case to cover this bug?

Currently, there is no bug for connection closure in hdfs cache, so it is a precaution in advance.


The configuration for Llama-7b on 4 RTX4090

Hello, I want to  run train_ppo_llama_ray.sh on 4 RTX4090, should I modify the actor_num_gpus_per_node/critic_num_gpus_per_node in train_ppo_llama_ray.sh ? As the default script is for 8 gpus, what else should I pay attention to or should be modified?

1,1,1,1 with adam offload

@hijkzzz Thank you for replying! But I met this problem, do you  know how to solve it ?
Status message: Job entrypoint command failed with exit code 1, last available logs (truncated to 20,000 chars):
    values, debugger_breakpoint = worker.get_objects(object_refs, timeout=timeout)
  File "/usr/local/bin/python3.10/dist-packages/ray/_private/worker.py", line 866, in get_objects
    raise value
ray.exceptions.RayActorError: The actor died unexpectedly before finishing this task.
        class_name: ActorModelRayActor
        actor_id: 53688e714f4881c3b3028ed402000000
        pid: 3752
        namespace: f4c18cbd-bbfb-4d8b-acf3-3aa591111fe9
        ip: <IP_ADDRESS>
The actor is dead because its worker process has died. Worker exit type: SYSTEM_ERROR Worker exit detail: Worker unexpectedly exits with a connection error code 2. End of file. There are some potential root causes. (1) The process is killed by SIGKILL by OOM killer due to high memory usage. (2) ray stop --force is called. (3) The worker is crashed unexpectedly due to SIGSEGV or other unexpected errors.

Do you have a more detailed logs?

i success on the following configuration:
`
set -x
export PATH=$HOME/.local/bin/:$PATH
ray job submit --address="http://<IP_ADDRESS>:8265"
--runtime-env-json='{"working_dir": "/openrlhf", "pip": "/openrlhf/requirements.txt"}'
-- python3 examples/train_ppo_ray.py
--ref_num_nodes 1
--ref_num_gpus_per_node 1
--reward_num_nodes 1
--reward_num_gpus_per_node 1
--critic_num_nodes 1
--critic_num_gpus_per_node 1
--actor_num_nodes 1
--actor_num_gpus_per_node 1
--pretrain /root/.cache/huggingface/hub/llama-2-7b-chat-hf
--reward_pretrain /root/.cache/huggingface/hub/models--OpenLLMAI--Llama-2-7b-rm-anthropic_hh-lmsys-oasst-webgpt/snapshots/a982afeed00fac9767d53aecde5b88947b1be194
--save_path /openrlhf/examples/test_scripts/ckpt/7b_llama
--micro_train_batch_size 2
--train_batch_size 128
--micro_rollout_batch_size 4
--rollout_batch_size 1024
--max_epochs 1
--prompt_max_len 1024
--generate_max_len 1024
--zero_stage 2
--bf16
--actor_learning_rate 5e-7
--critic_learning_rate 9e-6
--init_kl_coef 0.01
--prompt_data Open-Orca/OpenOrca,Dahoas/full-hh-rlhf,tasksource/oasst1_pairwise_rlhf_reward
--prompt_data_probs 0.4,0.5,0.1
--max_samples 80000
--normalize_reward
--actor_init_on_gpu
--adam_offload
--flash_attn
--gradient_checkpointing
--lora_rank 4
`

@LinkyLiu Ray actor has died unexpectedly, please check ray log in /tmp/ray/session_latest/logs/: raylet.out, raylet.err, job-xxx.log. There should be more information about why the actor die.


Adding support for MySQL and PG migrations

Related swagger change: Azure/azure-rest-api-specs#10494
Adding support for MySQL and PG migrations into the SDK.

@m-nash This is ready to be checked in and released.

@m-nash This is ready to be checked in and released.

/azp run net - mgmt - ci

/azp run net - mgmt - ci

I approve this breaking change.


RAM Load/Write doesn't work

Loading from the genMDM RAM seems not to work.
Stock instruments load instead of saved.

Even with the genMDM Max for Live control strip, recalled instruments seem to sound broken.
Maybe this isn't worth implementing.


Fix CI failures

Signed-off-by: Ce Gao<EMAIL_ADDRESS>

It seems that we have no item in GCS
https://storage.googleapis.com/kubernetes-jenkins/pr-logs/pull/kubeflow_hp-tuning/27/kubeflow-hp-tuning-presubmit

@jlewi
I meet with one problem when I am trying to fix CI failures. I'd appreciate it if you could have a look.
https://console.cloud.google.com/kubernetes/pod/us-east1-d/kubeflow-testing/kubeflow-test-infra/kubeflow-hp-tuning-presubmit-e2e-27-4e8c7ed-19-af0f-2710394068?project=kubeflow-ci&tab=logs&duration=PT1H&container_summary_list_tablesize=10&log_entry_list_tablesize=50
I think it may be caused by some features I don't know in GKE. I tried to hack it https://github.com/kubeflow/hp-tuning/pull/27/commits/aacd31e0bd0b97282c7e924aea9278fc30bcd5ec but failed.

/assign @YujiOshima
Thanks to jose5918@ , we solved the issue in GKE and CI should work now.

🎉🎉🎉🎉🎉🎉🎉🎉
Thank you @gaocegege @jose5918 !!!
/lgtm


Integer Division

                // .divi => div(.divi, base, instructions), TODO: this

https://github.com/PatrickTorgerson/ego/blob/a6c0cefefc0927e3f2ac036c27f3a91c099ca578/src/vm.zig#L47

I'd rather not have implicit truncating integer division. One option is having integer division result in a floating point value when it cannot be guaranteed to be an integer.
var i = int: 41
i = i / 2 // error: expected int found float (integer division results in float)

const forty = 40;
i = forty / 2 // ok: result can be guaranteed to be int at compile-time

This will require explicit casts
i = int(i/2) // but what is cast behavior? trunc? round?

// also this is wierd
i *= 2 // ok
i /= 2 // error?


Can not set user variable without enabling multipleStatements

We are using version 2.17.1 in our production. However I noticed that I am unable to set variable unless I enable multipleStatements - which has nothing to do with user defined variables. It is necessary to mention that we are using createPool method.
The following code does not work,
const query1 = `SET @var1 = 123`;
const query2 = `SELECT @var1`;

db.query(query1, (err1, result1) => {
  db.query(query2, (err2, result2) => {
    console.log(result2); //@var1 is NULLL
  });
});

However after enabling multipleStatements in createPool configuration, and rewriting the query as following gives the desired result,
const query = `SET @var1 = 123; SELECT @var1';

db.query(query, (err, result) => {
  console.log(result[1]); //@var1 is 123 in this case
});

I am not sure if this is a bug or expected behavior.
Perhaps in the first case I am using two different pool instances thus the variable is session specific only to the first query and not the second, as mention in the MySQL docs ?

Is your db variable a pool object? If so, then db.query

This is a shortcut for the pool.getConnection() -> connection.query() -> connection.release() code flow. Using pool.getConnection() is useful to share connection state for subsequent queries. This is because two calls to pool.query() may use two different connections and run in parallel.


Pre-Train models on CIFAR-100

Pre-train all models on CIFAR100 so that it can be given as a transfer learning option.
Train on torch==1.5.1 and after training, upload models to S3

Trained the following model on CIFAR100:

Resnet18: 61.34%
Resnet34: 62.16%
MobileNet v2: 52.22%


"Could not find DRM device" when trying to run Qt apps with EGLFS on Pi4

Description
I have been trying to fix this for days now without success. I wanto run my application using eglfs, but I am getting the message Could not find DRM device!
 So I am thinking that there must be something wrong with my meta-raspberrypi setup. I would think that the qtbase-%.bbappend would take care of providing the right configuration for the Pi4.
PACKAGECONFIG_GL:rpi = "${@bb.utils.contains('DISTRO_FEATURES', 'x11 opengl', 'gl', \
                        bb.utils.contains('DISTRO_FEATURES',     'opengl', 'eglfs gles2', \
                                                                       '', d), d)}"
PACKAGECONFIG_GL:append:rpi = "${@bb.utils.contains('MACHINE_FEATURES', 'vc4graphics', ' kms', '', d)}"
PACKAGECONFIG_GL:append:rpi = " gbm"
PACKAGECONFIG_FONTS:rpi = "fontconfig"
PACKAGECONFIG:append:rpi = " libinput examples tslib xkbcommon"
PACKAGECONFIG:remove:rpi = "tests"

OE_QTBASE_EGLFS_DEVICE_INTEGRATION:rpi = "${@bb.utils.contains('MACHINE_FEATURES', 'vc4graphics', '', 'eglfs_brcm', d)}"

do_configure:prepend:rpi() {
    # Add the appropriate EGLFS_DEVICE_INTEGRATION
    if [ "${@d.getVar('OE_QTBASE_EGLFS_DEVICE_INTEGRATION')}" != "" ]; then
        echo "EGLFS_DEVICE_INTEGRATION = ${OE_QTBASE_EGLFS_DEVICE_INTEGRATION}" >> ${S}/mkspecs/oe-device-extra.pri
    fi
}
RDEPENDS:${PN}:append:rpi = "${@bb.utils.contains('MACHINE_FEATURES', 'vc4graphics', '', ' userland', d)}"
DEPENDS:append:rpi = "${@bb.utils.contains('MACHINE_FEATURES', 'vc4graphics', '', ' userland', d)}"

I have been playing around with a lot of values and have tried adding all of the above myself before finding this file, but am always coming to the same results.

I have it working well with Yoe Distro on rpi4-64

@kraj I am using kas. Should the tool matter? In the docs also Kas is recommended

@kraj I am using kas. Should the tool matter? In the docs also Kas is recommended

it matters, if you see in meta-yoe
there are bunch of changes to make qt5 work with eglfs.

Looks to me like only linuxfb was added if you compare it to the .bbappend from meta-raspberrypi
# This does not work when wayland and x11 both are enabled in DISTRO_FEATURES and x11 image
# is built but works ok for eglfs and wayland images
QT_CONFIG_FLAGS += "${@bb.utils.contains('DISTRO_FEATURES', 'wayland', '-qpa wayland', \
                                         bb.utils.contains('DISTRO_FEATURES', 'x11', '', \
                                                          '-qpa eglfs', d), d)}"

PACKAGECONFIG:append = " libinput eglfs examples linuxfb tslib"

PACKAGECONFIG_GL = "${@bb.utils.contains('DISTRO_FEATURES', 'x11 opengl', 'gl', \
                    bb.utils.contains('DISTRO_FEATURES',     'opengl', 'eglfs gles2', \
                                     '', d), d)}"

And shouldn't these changes be in meta-raspberry Pi then, if they fix it?

@kraj I tried using the qtbase_%.bbappend from meta-yoe, but it did not change anything unfortunately

Problem was that I had dtoverlay=vc4-kms-v3d and not dtoverlay=vc4-fkms-v3d set in the ``config.txt```...


Add support of react-v0.14-rc1

Add support of react-v0.14-rc1

Hi!
Just published v0.7.0 with react-v0.14-rc1 support
Giulio

Super, Thank You!!!!!

Let me know if everything's ok :+1:

All works fine, everything's ok :+1:
Thank you once again!

Great. Thanks for your feedback. Happy hacking


Update deploy script to use variable FLASHSYSTEM_DRIVER_FULL_IMAGE_NAME in file manager_config_patch.yaml

Please help to review, thanks.

@iamniting I have updated the commit-msg, thanks for your input.


ValueError: num_samples should be a positive integer value, but got num_samples=0

In the middle of training as your instruction, I had some errors below.
File "/opt/anaconda3/envs/venv/lib/python3.7/site-packages/torch/utils/data/sampler.py", line 104, in init
"value, but got num_samples={}".format(self.num_samples))
ValueError: num_samples should be a positive integer value, but got num_samples=0

same error as above. did u find a solution @xiahaifeng1995 @camongman

that's is the endswith problem, if your datsets is not png


Unable to open pdf from my domain-demandtechreports.com

Link to PDF file (or attach file here):
Configuration:

Web browser and its version:Chrome '59.0.3071.104'
Operating system and its version: windows 8
PDF.js version:Version 2.0
Is an extension:

Steps to reproduce the problem:

web/viewer.html?file=http://www.Demandtechreports.com/IBMResources/fulfilling-the-promise-of-omnichannel-retailingS.pdf

What is the expected behavior? (add screenshot)
Should display pdf stored in folder(folder is stored on domain-demandtechreports.com)
What went wrong? (add screenshot)
-Error
1.Unexpected Server Response
PDF.js v1.8.188 (build: ad1023ff)
Message: Unexpected server response (0) while retrieving PDF "http://www.demandtechreports.com/IBMResources/fulfilling-the-promise-of-omnichannel-retailingS.pdf"

Link to a viewer (if hosted on a site other than mozilla.github.io/pdf.js or as Firefox/Chrome extension):

See https://github.com/mozilla/pdf.js/wiki/Frequently-Asked-Questions#faq-xhr . Closing, since based on provided information, the library works as expected.


Ignore CFDs that were moved to closed while processing all open CFDs

fixes #1847
Due to loading CFD ids and processing them one by one ew can encounter situations where the task that moves CFDs to the closed_cfds table moves a CFD while another actor still processes this CFD as part of the open CFD id list.
We introduce a db::Error that models all errors, so we can distinguish the OpenCfdNotFound case.
When loading all open CFDs we are ignoring cases where a CFD cannot be found in cfds (the table representing open CFDs) and print a trace log.
We don't handle load_open_cfd error cases of OpenCfdNotFound separately because it is not expected to call this function and fail (because the CFD should already be final if it was moved to closed_cfds).

bors r+


Fine tuning using "finetune_lora.sh" file

Hi, thanks again for the amazing work here! When I try to fine tune the model with our sample data, I was able to initialize some parts of the training but I got the following issue related to "cpu "in the picture. I ran this on the google colab with one video and a formatted json file to try to make it run first. Please be noted that I actually downloaded these models locally on google colab because otherwise I would not be able to run. You can assume that there is no directory issue here.

---------------------------- finetune_lora.sh code: -----------------------------
finetune_lora.sh code:
fintune_lora.sh
!/bin/bash
Environment Variables
ARG_WORLD_SIZE=${1:-1}
ARG_NPROC_PER_NODE=${2:-1}
ARG_MASTER_ADDR="<IP_ADDRESS>"
ARG_MASTER_PORT=16666
ARG_RANK=0
Multiple conditions
if [ ! -n "$WORLD_SIZE" ] || [ ! -n "$NPROC_PER_NODE" ]; then
WORLD_SIZE=$ARG_WORLD_SIZE
NPROC_PER_NODE=$ARG_NPROC_PER_NODE
fi
if [ ! -n "$MASTER_ADDR" ] || [ ! -n "$MASTER_PORT" ] || [ ! -n "$RANK" ]; then
MASTER_ADDR=$ARG_MASTER_ADDR
MASTER_PORT=$ARG_MASTER_PORT
RANK=$ARG_RANK
fi
echo "WORLD_SIZE: $WORLD_SIZE"
echo "NPROC_PER_NODE: $NPROC_PER_NODE"
Training Arguments
GLOBAL_BATCH_SIZE=128
LOCAL_BATCH_SIZE=4
GRADIENT_ACCUMULATION_STEPS=$[$GLOBAL_BATCH_SIZE/($WORLD_SIZE*$NPROC_PER_NODE*$LOCAL_BATCH_SIZE)]
Log Arguments
export TRANSFORMERS_OFFLINE=0 # 0 = offline
export WANDB_PROJECT=videollama2_vllava
RUN_NAME=videollama2_vllava_lora
DATA_DIR="/content/drive/My Drive/ColabNotebooks/Video_llm2/videollava_sft"
OUTP_DIR="/content/drive/My Drive/ColabNotebooks/Video_llm2/fine_tuning_result"
export CUDA_LAUNCH_BLOCKING=1
export CUDA_VISIBLE_DEVICES=0
export HF_HUB_OFFLINE=0
torchrun --nnodes=1 --nproc_per_node=1
--master_addr=$MASTER_ADDR --master_port=$MASTER_PORT
--node_rank=$RANK /content/VideoLLaMA2/scripts/custom/videollama2/train_flash_attn.py
--lora_enable True
--lora_r 128
--lora_alpha 256
--mm_projector_lr 2e-5
--bits 4
--deepspeed "/content/VideoLLaMA2/scripts/zero3.json"
--version mistral
--vision_tower /content/VideoLLaMA2/scripts/custom/openai/clip-vit-large-patch14-336/
--mm_projector_type stc_connector
--model_name_or_path /content/VideoLLaMA2/Mistral-7B-Instruct-v0.2/
--data_path "${DATA_DIR}/videochatgpt_llavaimage_tune.json"
--data_folder "${DATA_DIR}/video"
--pretrain_mm_mlp_adapter "/root/.cache/huggingface/hub/models--DAMO-NLP-SG--VideoLLaMA2-7B-Base/snapshots/main/models--DAMO-NLP-SG--VideoLLaMA2-7B-Base/snapshots/610da3cc29bc29e16e44cb3ba340af09da4994eb/mm_projector.bin"
--mm_vision_select_layer -2
--mm_use_im_start_end False
--mm_use_im_patch_token False
--image_aspect_ratio pad
--num_frames 8
--bf16 True
--tf32 True
--fp16 False
--output_dir "${OUTP_DIR}/finetune_${RUN_NAME}"
--num_train_epochs 1
--per_device_train_batch_size $LOCAL_BATCH_SIZE
--per_device_eval_batch_size 4
--gradient_accumulation_steps $GRADIENT_ACCUMULATION_STEPS
--evaluation_strategy "no"
--save_strategy "steps"
--save_steps 500
--save_total_limit 99
--learning_rate 2e-5
--weight_decay 0.
--warmup_ratio 0.03
--lr_scheduler_type "cosine"
--logging_steps 1
--model_max_length 2048
--gradient_checkpointing True
--dataloader_num_workers 4
--report_to tensorboard
--run_name $RUN_NAME
------------------------------------ full outputs -----------------------------------
I can also cpoy and paste the full outputs which contain the error msg if needed because it is quite long. Here I can show you where I failed and couple lines before that.

---------------------------------- update -------------------------------
Just for your information, I also changed this so that I do not have to train from scratch, but it still gives me the same error from above.

Formatting inputs...Skip in lazy mode
Parameter Offload: Total persistent parameters: 1143040 in 387 params
0% 0/1 [00:00<?, ?it/s]Traceback (most recent call last):
File "/content/VideoLLaMA2/scripts/custom/videollama2/train_flash_attn.py", line 12, in
train(attn_implementation="flash_attention_2")
File "/content/VideoLLaMA2/scripts/custom/./videollama2/train.py", line 1035, in train
trainer.train()
File "/usr/local/lib/python3.10/dist-packages/transformers/trainer.py", line 1885, in train
return inner_training_loop(
File "/usr/local/lib/python3.10/dist-packages/transformers/trainer.py", line 2216, in _inner_training_loop
tr_loss_step = self.training_step(model, inputs)
File "/usr/local/lib/python3.10/dist-packages/transformers/trainer.py", line 3238, in training_step
loss = self.compute_loss(model, inputs)
File "/usr/local/lib/python3.10/dist-packages/transformers/trainer.py", line 3264, in compute_loss
outputs = model(**inputs)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/deepspeed/utils/nvtx.py", line 15, in wrapped_fn
ret_val = func(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/deepspeed/runtime/engine.py", line 1842, in forward
loss = self.module(*inputs, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1561, in _call_impl
result = forward_call(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/peft/peft_model.py", line 922, in forward
return self.base_model(
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1561, in _call_impl
result = forward_call(*args, **kwargs)
File "/content/VideoLLaMA2/scripts/custom/./videollama2/model/language_model/videollama2_mistral.py", line 90, in forward
return super().forward(
File "/usr/local/lib/python3.10/dist-packages/transformers/models/mistral/modeling_mistral.py", line 1139, in forward
outputs = self.model(
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1561, in _call_impl
result = forward_call(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/transformers/models/mistral/modeling_mistral.py", line 1014, in forward
layer_outputs = self._gradient_checkpointing_func(
File "/usr/local/lib/python3.10/dist-packages/torch/_compile.py", line 24, in inner
return torch._dynamo.disable(fn, recursive)(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/torch/_dynamo/eval_frame.py", line 489, in _fn
return fn(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/torch/_dynamo/external_utils.py", line 17, in inner
return fn(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/torch/utils/checkpoint.py", line 482, in checkpoint
return CheckpointFunction.apply(function, preserve, *args)
File "/usr/local/lib/python3.10/dist-packages/torch/autograd/function.py", line 553, in apply
return super().apply(*args, **kwargs)  # type: ignore[misc]
File "/usr/local/lib/python3.10/dist-packages/torch/utils/checkpoint.py", line 261, in forward
outputs = run_function(*args)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1561, in _call_impl
result = forward_call(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/transformers/models/mistral/modeling_mistral.py", line 738, in forward
hidden_states, self_attn_weights, present_key_value = self.self_attn(
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1561, in _call_impl
result = forward_call(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/transformers/models/mistral/modeling_mistral.py", line 366, in forward
query_states, key_states = apply_rotary_pos_emb(query_states, key_states, cos, sin, position_ids)
File "/usr/local/lib/python3.10/dist-packages/transformers/models/mistral/modeling_mistral.py", line 159, in apply_rotary_pos_emb
cos = cos[position_ids].unsqueeze(unsqueeze_dim)
RuntimeError: indices should be either on cpu or on the same device as the indexed tensor (cpu)
0% 0/1 [00:04<?, ?it/s]
[2024-07-22 19:45:32,893] torch.distributed.elastic.multiprocessing.api: [ERROR] failed (exitcode: 1) local_rank: 0 (pid: 21944) of binary: /usr/bin/python3
Traceback (most recent call last):

Hello,I'm a phD student from ZJU, I also use videollama2 to do my own research,we create a WeChat group to discuss some issues of videollama2 and help each other,could you join us? Please contact me: WeChat number == LiangMeng19357260600, phone number == +86<PHONE_NUMBER>0,e-mail ==<EMAIL_ADDRESS>


Revert file to a specific commit in history.

Sometimes one would like to see the history for the specific file and revert to a certain commit. Is there any plan to do so?
Seems that it was related to #316, #241 and maybe #328 .

In first Brackets-Git it's awsome. So, you already finished the #316 and #241 ones, could you try this? Of course I don't the knowlegde enough about it, but since that you have finished the button of history, by the way its really great, putting a button at right just to revert the offline version seems to me, so close and easy of todo. Count the commits until that position of button revert and sent the command git revert HEAD~var_count .
`


Script doesn't work with Ubuntu 20.04

Hi,
Thank you for working on this ― brilliant idea!
Unfortunately, the script doesn't work with Ubuntu 20.04.
$ sudo ./nv-gpu-powerlimit-setup.sh --gpus 0 --powerlimit 280
[sudo] password for gianluca:
Checking OS version ...
[OK] Ubuntu 20.04.4 LTS
Checking for NVIDIA GPU and Driver version
[OK] Found NVIDIA GPU
Driver Version = 510.47.03
Creating /usr/local/etc/nv-powerlimit.conf
[OK] /usr/local/etc/nv-power-limit.conf created
setting permission chmod 644 on /usr/local/etc/nv-power-limit.conf
Creating /usr/local/sbin/nv-power-limit.sh
#!/usr/bin/env bash

# Set power limits on all NVIDIA GPUs
# powerlimit is sorced from /usr/local/etc/nv-powerlimit.conf

# Make sure nvidia-smi exists
command -v nvidia-smi &> /dev/null || { echo >&2 "nvidia-smi not found ... exiting."; exit 1; }

CONFIG_FILE=/usr/local/etc/nv-power-limit.conf
source ${CONFIG_FILE}
POWER_LIMIT=${POWERLIMIT}
MAX_POWER_LIMIT=$(nvidia-smi -q -d POWER | grep 'Max Power Limit' | tr -s ' ' | cut -d ' ' -f 6)

# ToDo:
# Max power limit check is blocking for multi-GPU
# will fix with arrays when I add support for setting independent power limits
#if [[ ${POWER_LIMIT%.*}+0 -lt ${MAX_POWER_LIMIT%.*}+0 ]]; then
    /usr/bin/nvidia-smi --persistence-mode=1
    /usr/bin/nvidia-smi  --power-limit=${POWER_LIMIT}
#else
#    echo 'FAIL! POWER_LIMIT set above MAX_POWER_LIMIT ... '
#    exit 1
#fi

exit 0

[OK] /usr/local/sbin/nv-power-limit.sh created
setting permissions: chmod 744 on /usr/local/sbin/nv-power-limit.sh
Creating /usr/local/etc/systemd/system/nv-power-limit.service
[Unit]
Description=NVIDIA GPU Set Power Limit
After=syslog.target systemd-modules-load.service
ConditionPathExists=/usr/bin/nvidia-smi

[Service]
User=root
Environment="PATH=/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin"
ExecStart=/usr/local/sbin/nv-power-limit.sh

[Install]
WantedBy=multi-user.target

[OK] /usr/local/etc/systemd/system/nv-power-limit.service created
setting permissions: chmod 644 on /usr/local/etc/systemd/system/nv-power-limit.service
linking /usr/local/etc/systemd/system/nv-power-limit.service to /etc/systemd/system/nv-power-limit.service
Finished setting NVIDIA GPU power-limit service

After running this command, the max power for the RTX 3090 (device 0) is still 350W.
Please let me know if I can provide you with any more useful info.

Thanks for reporting that!
I believe that is from a recent bug in systemd on 20.04 (not sure which version), something to do with
systemctl daemon-reload  ???
the subsequent  systemctl start nv-power-limit.service is not running the script
Workarounds:
1)Do a system restart. The service is enabled so it should run at boot and run the script that's in /usr/local/sbin, which will use the value that is set in /usr/local/etc/nv-power-limit.conf

You could also manually run
sudo /usr/local/sbin/nv-power-limit.sh

Check the POWERLIMIT value in /usr/local/etc/nv-power-limit.conf  You can edit that at (as root) any time and rerun the script or reboot.
I would try the reboot first and then do
nvidia-smi -q -d POWER
to confirm the powerlimits
I'll add a line to run the script manually during setup as a workaround --Don

It seems the power limit is reset somewhere. Changing the "After" to "nvidia-persistenced.service" makes the power limit of the GPU with a monitor connected change successfully, but the other GPUs' do not. My workaround is to repeat the power limit setting several times.
Here is my script:
#!/usr/bin/env bash

# Set power limit on all NVIDIA GPUs

# Make sure nvidia-smi exists

command -v nvidia-smi &> /dev/null || { echo >&2 "nvidia-smi not found ... exiting."; exit 1; }

REPEAT=10
NUM_GPUS=$(nvidia-smi --list-gpus | wc -l)
POWER_LIMIT=300

echo "Detect $NUM_GPUS GPUs"

cnt=0
while [[ $cnt -lt $REPEAT ]]; do
    for ((i=0; i < $NUM_GPUS; i++)) do
        MAX_POWER_LIMIT=$(nvidia-smi -i $i -q -d POWER | grep 'Max Power Limit' | tail -n 1 | tr -s ' ' | cut -d ' ' -f 6)
        if [[ "${POWER_LIMIT%.*}" -lt "${MAX_POWER_LIMIT%.*}" ]]; then
	    CURRENT_POWER_LIMIT=$(nvidia-smi -i $i -q -d POWER | grep 'Power Limit' | head -n 1 | tr -s ' ' | cut -d ' ' -f 5)
	    if [[ "${POWER_LIMIT%.*}" -ne "${CURRENT_POWER_LIMIT%.*}" ]]; then
	        echo "Setting power limit of GPU $i"
		echo "Current power limit of GPU $i is $CURRENT_POWER_LIMIT"
                /usr/bin/nvidia-smi -i $i --power-limit=${POWER_LIMIT}
                /usr/bin/nvidia-smi -i $i --persistence-mode=1
	    fi
        else
            echo "FAIL! POWER_LIMIT:$POWER_LIMIT set above MAX_POWER_LIMIT:$MAX_POWER_LIMIT"
            exit 1
        fi
    done
    sleep 1
    cnt=$((cnt+1))
done

exit 0

Thanks for adding that useful modification!

closing because of new version release


Fixes broken build due to spotless and error-prone

Error-prone needs a fix, looking into it.

@thesmallstar I think if you fix the current 1 problem, the compileIntegrationTest may yield no violation.

@percyashu yes :) Let's see what Travis says now.

LGTM


Workplan Page - days left warning

the days left area of the accordion, when it has a warning, is not very noticeable in its dark yellow.
we need to make it pop more to show that a warning is on display

As a user
I want see instantly what phases are out of time
so that I can add extension milestone to extend the phase timeline
Acceptance Criteria
Scenario 1:
GIVEN My Workplans is open
WHEN I click on any Work which timeline is beyond legislative timeline
THEN I see days left in Error/Dark rgba (163.30,34,1) and a exclamation symbol indicator
Screenshot

@maria-kuzmenko colour code for new warning colour pls!

@djnunez-aot , link to the design - https://www.figma.com/file/XcYynwH7xrO2P30uqewaLS/For-Development?type=design&node-id=41-142&mode=dev
What should be done:

Exclamation indicator next to the status should be recoloured into
background: rgba(252, 237, 238, 1)
exclamation mark: rgba(163, 30, 34, 1)

This component we are reusing and change a triangle to it


[FEATURE] Option to retry a failed task

Hi
Is your feature request related to a problem? Please describe.
I'm always frustrated when I need to manually execute a script to retry a failed task. I know there is a possibility to setup a failover and retry strategy, but sometimes it is more suitable to retry the task immediately on demand.
Describe the solution you'd like
It would be great if there is a button in the UI for the failed task to retry it. It would execute the task with the same parameters.
Describe alternatives you've considered
There is a django-celery-results app alternative which I can add a custom action to retry the specific tasks, but celery-insights is much better tool. The only missing thing is the "retry button".
Additional context
Snippet I currently use:
def retry_task(task_id):
    meta = celery_app.backend.get_task_meta(task_id)
    task = celery_app.tasks[meta['name']]
    task.apply_async(args=meta['args'], kwargs=meta['kwargs'])

Thank you in advance!

I'm intentionally kept Celery Insights as read-only until now. Added remote control operations such as Publish task, Revoke task, Restart worker, etc. really require some Authorization layer, and itself needing some Authentication layer.
This is rather hard to do well in a stateless container, with a simple enough setup.
Any suggestions, feature ideas or pull requests and greatly appreciated 🌟
Let's open a discussion about this topic to continue the conversation


OptionParser should allow flag with optional value

In #4499 it has come up that OptionParser does not allow for a flag to be used either with a value (--flag=value) or without a value (--flag). In the referenced PR it happend that --color a.cr was misinterpreted as flag --color with value a.cr, whereas it is meant to be a value-less flag --color and an unnamed argument a.cr.
Currently, the format --flag [value] allows a flag value to be missing, but this only works if it is the last argument or the next argument is also a flag. It would also match an unnamed argument. There should be a way to prevent that.
Now, obviously, this can't work with an empty space as separater between flag an value as this would always be ambiguous, but it would work if it was enforced to be an equals sign: --foo arg would be an value-less flag, --flag=value would be a flag with value value.

this only works if it is the last argument or the next argument is also a flag.

It depends options handler order. For example:
require "option_parser"

def parse(args)
  p [:parse, args]
  OptionParser.parse(args) do |opts|
    opts.on("-a [val]", "") do |val|
      p [:a, val]
    end

    opts.on("-b [val]", "") do |val|
      p [:b, val]
    end

    opts.unknown_args do |args|
      p [:unknown_args, args]
    end
  end
  puts
end

parse(%w(-a -b b))
# [:parse, ["-a", "-b", "b"]
# [:a, "-b"]
# [:unknown_args, ["b"]]

parse(%w(-b -a a))
# [:parse, ["-b", "-a", "a"]
# [:a, "a"]
# [:unknown_args, []]
# NOTE: why "-b" is not called?

Wow, this looks like there are some serious issues with optional parameters.

Have you already decided? I hope I don't push you too much to stress you.

So are you usually trying to similarly discourage new contributors? Or should I take it as a special discriminatory honor? Or you just don't care?

@urde-graven If you have a patch that fixes some issue in OptionParser, please open a PR. If it behaves like Ruby's OptionParser then it will likely be accepted.

@asterite Only patch already mentioned in https://github.com/crystal-lang/crystal/issues/5338#issuecomment-452672895. I have not worked on it further nor opened PR due to the discussion here. If It's ok, I'll open it.

@urde-graven If it works like Ruby's OptionParser, then yes, please open a PR. Thank you!

@straight-shoota I'm sorry that it took me so long, but could you explain the issue with some example code and command line invocation?

@asterite

The first problem ("separate argument matched when optional value is defined together with flag") is explained also in https://github.com/crystal-lang/crystal/pull/4499#issuecomment-348510730. Example:

require "option_parser"

def parse(args)
  p [:parse, args]
  OptionParser.parse(args) do |opts|
    opts.on("-c[always|never|auto]", "--color=[always|never|auto]", "") do |val|
      p [:color, val]
    end
    opts.unknown_args do |args|
      p [:unknown_args, args]
    end
  end
  puts
end

parse(%w(--color a.cr))
# old OptionParser:
# [:parse, ["--color", "a.cr"]]
# [:color, "a.cr"]
# [:unknown_args, []]
# with patch:
# [:parse, ["--color", "a.cr"]]
# [:color, ""]
# [:unknown_args, ["a.cr"]]

parse(%w(-c a.cr))
# old OptionParser:
# [:parse, ["-c", "a.cr"]]
# [:color, "a.cr"]
# [:unknown_args, []]
# with patch:
# [:parse, ["-c", "a.cr"]]
# [:color, ""]
# [:unknown_args, ["a.cr"]]

Example for "flag matched as optional value" is in https://github.com/crystal-lang/crystal/issues/5338#issuecomment-348524535 (and the patch also fixes problem with missing call of -b, which I overlooked before). Output of the provided example with patched OptionParser:

[:parse, ["-a", "-b", "b"]]
[:a, ""]
[:b, "b"]
[:unknown_args, []]

[:parse, ["-b", "-a", "a"]]
[:a, "a"]
[:b, ""]
[:unknown_args, []]

Example for the third problem ("shifted argument matched as value because arguments were removed by previous matches") follows:

require "option_parser"

def parse(args)
  p [:parse, args]
  OptionParser.parse(args) do |opts|
    opts.on("-a [val]", "") do |val|
      p [:a, val]
    end
    opts.on("-b [val]", "") do |val|
      p [:b, val]
    end
    opts.unknown_args do |args|
      p [:unknown_args, args]
    end
  end
  puts
end

parse(%w(-b -a a c)
# old OptionParser:
# [:parse, ["-b", "-a", "a", "c"]]
# [:a, "a"]
# [:b, "c"]
# [:unknown_args, []]
# with patch:
# [:parse, ["-b", "-a", "a", "c"]]
# [:a, "a"]
# [:b, ""]
# [:unknown_args, ["c"]]

"OptionParser should take short-flag's missing information about option from long flag" - from spec:

args = ["-f", "12", "-g", "13", "-h", "14"]
captured_f = captured_g = captured_z = nil
parser = OptionParser.parse(args) do |opts|
  opts.on("-f", "--flag X", "some flag") { |flag_value| captured_f = flag_value }
  opts.on("-g", "--gflag=Y", "another flag") { |flag_value| captured_g = flag_value }
  opts.on("-h", "--hopt=[Z]", "optional flag") { |flag_value| captured_z = flag_value }
end
captured_f.should eq "12"
captured_g.should eq "13"
captured_z.should eq ""
args.should eq ["14"]
parser.to_s.should contain "   -f, --flag X"

@urde-graven Thank you!
I think all the existing problems in OptionParser come from the fact that I don't (didn't?) understand how option parsing work in general. I thought --flag value was valid syntax. But it seems these two are only valid:

-xvalue: when -x is defined (single letter) then the value comes right after the first letter
--some=value: when --some is defined, the value must be given with right after the name followed by = and the value

With that we can easily allow optional values:

for -x[value]: nothing comes after x
for --some=[value]: nothing comes after some

Is what I'm saying correct?
Is there any program out there where you can say --flag value?

There is no standard concept for option parsing. There are different styles employed by different programs.

Is there any program out there where you can say --flag value?

Yes, of course. Loads of them. I'd say this is actually more common than --flag=value. And there is nothing wrong with that notation per se. It can only be ambiguous when the value is optional.

Cool, then. Feel free to continue forward with this, then (I won't have any opinion at all because I don't know about this subject).

https://www.gnu.org/software/libc/manual/html_node/Argument-Syntax.html

There is one related issue; an empty argument is indistinguishable from a missing argument:
require "option_parser"

def parse_args(args)
  OptionParser.parse(args) do |parser|
    parser.on("-c [x]", "--color [x]", "optional arg") { |x| p x }
  end
end

# argument missing from first set of args only
parse_args %w(-c)     # => ""
parse_args ["-c", ""] # => ""

# argument missing from first set of args only
parse_args %w(--color)     # => ""
parse_args %w(--color=)    # => ""
parse_args ["--color", ""] # => ""

That means if a shell script does something like crystal --color=$(... 2>/dev/null), it might accidentally enable colored output because --color= would become equivalent to --color. The case for short options is avoidable because #11537 implies the '' in -c '' is not parsed as the argument to the -c.
Argp for example distinguishes the first two cases:
#include <stdio.h>
#include <argp.h>

static struct argp_option options[] = {
    {.name = "a", .key = 'a', .arg = "x", .flags = OPTION_ARG_OPTIONAL},
    {0},
};

static error_t parse_opt (int key, char *arg, struct argp_state *state) {
    switch (key) {
    case 'a':
        printf("\"%s\"\n", arg);
        break;
    default:
        return ARGP_ERR_UNKNOWN;
    }
    return 0;
}

static struct argp argp = {.options = options, .parser = parse_opt};

int main(int argc, char **argv) {
    char *args1[] = {"./a.out", "--a"};                 /* ./a.out --a */
    argp_parse(&argp, 2, args1, ARGP_NO_ERRS, 0, NULL); /* "(null)"    */

    char *args2[] = {"./a.out", "--a="};                /* ./a.out --a= */
    argp_parse(&argp, 2, args2, ARGP_NO_ERRS, 0, NULL); /* ""           */

    /* Argp requires `=` for optional arguments (#11537) */
    char *args3[] = {"./a.out", "--a", ""};             /* ./a.out --a '' */
    argp_parse(&argp, 3, args3, ARGP_NO_ERRS, 0, NULL); /* "(null)"       */

    return 0;
}

In order to differentiate no value and empty value, the argument value could represent that with nil and "". But that would require a change to the method signature, because the proc currently only yields String. And it probably doesn't make sense to have this type when a value is required because user code would need to handle the nil case which can never happen.
This is a result of runtime string parsing to determine the optionality of a value.
A possible workaround for that would be to change the method signature, for example by introducing a new parameter for determining optionality which would allow to change the block type to yield String?. This would be backwards compatible because the new block type only applies to the new signature. Method calls using the old signature without option argument won't be affected.
def on(short_flag : String, long_flag : String, description : String, *, optional_value : Bool, &block : String? ->)

An alternative would be an overload that accepts a proc instance instead of capturing a block.
def on(short_flag : String, long_flag : String, description : String, proc : String? ->)

This would be a bit more tedious to use though, because you need to create an explicit Proc instance:
parser.on("-c", "--color", "", -> (value : String?) do
end)


Error - 12300 Invalid Content-Type

Hi, i have implemented audio call in my iOS App with backend server of php. I'm getting the access token from the my server and than uses all the delegates , as per following the steps of twilio swift project. Now when i make a call, it successfully registered my push credentials first,  than on click of call button it first show ringing in my console and than call fails. When i check my voice dashboard on twilio i get this error,
Twilio is unable to process the Content-Type of the provided URL. Please see the Twilio Markup XML Documentation for more information on valid Content-Types.
You must return a Content-Type for all requests. Requests without a Content-Type will appear in the Debugger as a 502 Bad Gateway error., These are the parameters that comes in the API,
`

ApplicationSid
"APaecd77856d0049b8b80baa3c41f193a2"

ApiVersion
"2010-04-01"

Called
""

Caller
"client:alice"

CallStatus
"ringing"

to
"+923244544783"

From
"client:alice"

To
""

CallSid
"CA98977f223a883f16b3585165066adde1"

Direction
"inbound"

AccountSid
"ACbc55b9853285730abd137444750e53c6"

I'm making client to client call through phone number. In identity i pass my phone number and in to i pass . other client phone number. I'm confused that why it is failing the call. Showing this error in my console, [VERBOSE TVOCall] handlePJSIPInviteStateDisconnected. Error: Error Domain=com.twilio.voice.error Code=603 "Connection error (603)." UserInfo={NSLocalizedDescription=Connection error (603).,

Thanks for your quick response, we have checked that our TwiML have a  verb and some text in it. whenever we hit the Url from postman it gives us a call and that verb works and speaks the the text but the error comes only when we run the app and try to call. Can we provide you our URL that we are using for call and giving in our TwiML app? So that you can test it for us as we are unable to understand the problem.

Hi @Hamza123Imran
I checked the Call SID again and from the error message I think you might want to check the URL provided in the TwiML application. It seems that you put the same link twice thus when the Twilio Voice service is trying fetch the TwiML, it tried to hit an URL that is in incorrect format.
-bobie

Hi @Hamza123Imran
Directly to your last question, which i think should be the main goal for your app integrating the Voice iOS SDK: if you take a look at makeCall.php you will see to make a client-to-client call you can just use the $dial->client($to); line. The callee client needs to be registered with that identity to be able to receive the call notification (which I believe you already did) and accept the call-invite, then the audio should be flowing between the caller and callee once the call is connected.

we have implemented makeCall.php on our server side, the code is in this link http://softdepth.com/projects/raven/customer/audiocall with to,from,token params. When we hit this url on postman we receive call with the same message that is written in TwiML xml file.
Now when we hit the this method in ios app
let parameteres =  ["To" : otherUserNumber,
"Caller" : myPhoneNumber,
"From" : myPhoneNumber,
"from" : myPhoneNumber!,
"to" : otherUserNumber
]
print(uuid)
    print(parameteres)
    call = TwilioVoice.call(accessToken!, params:parameteres as? [String : String] , uuid:uuid, delegate: self)
    print(call!)
    self.callKitCompletionCallback = completionHandler

it query on Twilio server starts ringing and after couple of seconds it show error and the error comes in our Twilio Debugger console.
Can you please help in setting up the flow of call from ios to ios app if we are missing something? and guide us in details how Twilio call works and how we have to make our TwiML to make a simple call with Twilio VOIP? may be we are unable to understand the proper flow. We are stuck here and our work is pending. Kindly help us in understanding the documentation for us, its a bit confusing.

Hi @Hamza123Imran
Sorry for the delay.
I checked the Account SID and for some reason I still saw couple of errors (2018/9/18) indicating that the URL specified in the Voice incoming request doesn't seem to be correct.
Also if you believe the response from your makeCall.php endpoint is correct, the response should look like this:
<?xml version="1.0" encoding="UTF-8"?>
<Response>
    <Dial callerId="14081234567">
        <Client>bobie</Client>
    </Dial>
</Response>

Let me know if this helps.
-bobie

Hi i have check my url for makecall on postman it is showing me same response as you said above,
https://prnt.sc/kyddm7
But when this url is hit in voice dashboard we get content type error, why it is not showing same response in voice dashboard?
We Follow up following steps

Audio token generated from PHP API here is screenshot how we are generating audio token
http://prntscr.com/kydp1s
After getting token from PHP we are calling voice delegate in IOS please check screen shot
http://prntscr.com/kydpz6
Twillio App hit makeCall API which mention here http://prntscr.com/kydrq6
And makeCall API returning correct response http://prntscr.com/kydsf2
here is the code inside makeCall API function http://prntscr.com/kydt58
Now where the actual mistake we are doing can you identify that?

Hi i have check my url for makecall on postman it is showing me same response as you said above,
We Follow up following steps

Audio token generated from PHP API here is screenshot how we are generating audio token http://prntscr.com/kydp1s
After getting token from PHP we are calling voice delegate in IOS please check screen shot http://prntscr.com/kydpz6
Twillio App hit makeCall API which mention here http://prntscr.com/kydrq6
And makeCall API returning correct response http://prntscr.com/kydsf2
here is the code inside makeCall API function http://prntscr.com/kydt58
Now can you identify that where we are making mistake? Because response on postman is right but when hit on voice dashboard it shows content type error?

Closing this issue for now. @Hamza123Imran please feel free to re-open or create a new issue in the future.
Thanks,
-bobie


Access the user’s route data for a workout

Is your feature request related to a problem? Please describe.
I'd like to access the user's route data from a workout.
Describe the solution you'd like
Possibly add a new method called "getWorkoutRoute"
Additional context
Hey guys, I think this feature would be useful to have. The data could be used for displaying cool visuals and for data analytics.
This article is a good reference.
Although I'm not familiar with Obj-C, I attempted to implement this into my project.
What I have so far:

New Permission added "WorkoutRoute"
Exported a method getWorkoutRoute (id is required in the input. Id can be found from a workout. [[sample UUID] UUIDString])
Added a method to Methods_Workout called "workout_getRoutes" that calls "fetchWorkoutSample" followed by "fetchWorkoutRoute"
Added a method to Queries called "fetchWorkoutSample" that retrieves HKWorkout  for the UUID provided
Added a method to Queries called "fetchWorkoutRoute"  that uses the HKWorkout from the "fetchWorkoutSample" method to retrieve the workout route data. This method executes two queries in reference to this article.

Edit: After struggling for over a day, I was able to successfully get all the location data for a route :)
Issues/Questions:

I'm not sure if a workout can have multiple routes associated with it. I assumed one. If a workout can have multiple routes, I'm not sure how to perform an async batch query for each route to fetch the location data and then send that data through the callback function.

I will add the configuration to my fork.

Hey How did you pull the UUID string from the anchored workout? I am not sure how to access that array of data

In the getAnchoredWorkouts query, you can access the UUID string from the Id field. It's not in the documentation, but you should see it in an anchored workout response. I hope this helps.

Ok if you're able to hop in discord sometime and brainstorm with me how to do it, I have swift code made I just need help integrating it into react native.

Yeah, I can most definitely try to help you out.

Any updates on this? I'd love to be able to access this data.

@devth I'm considering opening a PR in the near future, but feel free to take a look at my fork.

Thanks @nguyening20, I'll check out your fork! And please do open a PR if you can.

@nguyening20 amazing stuff, I need to use workoutRoute too, will use your fork for now but would be great if you could raise a PR.
@macelai are you still maintaining this repo?

issue

Just request HKWorkoutRoutes and you should get it.

hey @walterholohan, we are open to new PRs

@walterholohan This method only allows you to retrieve the workout route. It looks like you are trying to ask for write permissions. Moving AppleHealthKit.Constants.Permissions.WorkoutRoute to the read array should solve the issue here.

@macelai I have raised a PR https://github.com/agencyenterprise/react-native-health/pull/222/files that will allow users to ask for write permissions for WorkoutRoute

@nguyening20 apologies but it seems that since I forked off your repo and then tried to create a PR against this repo it has included all your changes in the PR too. Can you take a review of the changes - https://github.com/agencyenterprise/react-native-health/pull/222

@walterholohan I don't see a method to handle saving any workout route data to Apple HealthKit so I don't necessarily understand why you would need to ask for write permissions in this case.

@nguyening20 we have built an apple watch app for our RN app (https://www.runna.com/) that tracks a user while running. And as part of this, we write to the workoutRoute in HealthKit. This was the only library I could find that would allow us to ask the user to accept write permissions for health kit via the mobile app.

@walterholohan Please consider sharing or adding how you configured writing/saving workout routes to Apple HealthKit. I know @mgfuller333 and I would like this functionality included with this library. Thanks.

@nguyening20 will do. At the moment we are saving routes to HealthKit via the apple watch app, once we have this fully working I can share the swift code with you.

@nguyening20 I will close my PR as its just a duplicate of yours


Type: add helper function for anonymous records

Fixes #146

In the associated test, for the typeof(u); line, clang warns warning: declaration does not declare anything [-Wmissing-declarations]. However it doesn't do that if you replace typeof(u) with union U. aro does not issue the warning either way. Not sure what the most correct/consistent behavior would be.

In the associated test, for the typeof(u); line, clang warns warning: declaration does not declare anything [-Wmissing-declarations]. However it doesn't do that if you replace typeof(u) with union U. aro does not issue the warning either way. Not sure what the most correct/consistent behavior would be.

GCC and clang both seem to only error on typeof so we should too. It was simple to implement so I added it along with some comments to explain why the anonymous record check works.


Add display names to reference implementations

Note that this PR also contains commits from #166, which should be merged first.
Closes #141

Codecov Report

Merging #167 (fc993b4) into main (419f5bd) will increase coverage by 0.01%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main     #167      +/-   ##
==========================================
+ Coverage   96.04%   96.06%   +0.01%
==========================================
  Files          34       34
  Lines        2046     2056      +10
==========================================
+ Hits         1965     1975      +10
  Misses         81       81

Impacted Files
Coverage Δ

pybryt/reference.py
98.05% <100.00%> (+0.13%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 419f5bd...fc993b4. Read the comment docs.


IdP - per client token TTL

Allow clients to set their own TTL for access tokens.

Done, pushed into the mainline.


[db-ip] Handle Geocoder::QuotaExceeded gracefully

Geocoder::QuotaExceeded errors were not being generated correctly, and the response was coming back empty but with no indication.

Looks good to me - could you add another test case and fixture for it, similar to: https://github.com/alexreisner/geocoder/blob/master/test/fixtures/db_ip_com_invalid_key
Thanks!

Thanks for having a look at this @TrangPham! I've added the fixture file, but not quite sure how it gets picked up by the tests. Got a pointer?

@cv Thanks for your quick response :)
Here's an example: https://github.com/alexreisner/geocoder/blob/6589b9b2cd6d1cdda06bd1f598fe9ad070d8e5a9/test/unit/lookups/geocodio_test.rb#L63-L67

Looks like there's an unrelated CI failure with jruby 1.9, but other than that all tests are passing now! :)

Thanks! Looks like the error is unrelated :)


Don't forward _ping requests to the primary

Please sign your commits following these rules:
https://github.com/docker/docker/blob/master/CONTRIBUTING.md#sign-your-work
The easiest way to do this is to amend the last commit:
$ git clone -b "improve_ping_again" git@github.com:wsong/swarm.git somewhere
$ cd somewhere
$ git commit --amend -s --no-edit
$ git push -f

Amending updates the existing PR. You DO NOT need to open a new one.

In https://github.com/docker/swarm/pull/2492, we noticed that the /_ping endpoint forwards to the primary now, which creates an HTTP connection hijacking which causes subsequent /info requests to get redirected straight to the primary. This PR ensures that /_ping requests are still entirely handled by the local node while still checking the primary's status on replicas.

we noticed that the /_ping endpoint forwards to the primary now, which creates an HTTP connection hijacking which causes subsequent /info requests to get redirected straight to the primary

Requests should be independent of each other. If this is the case we have a bug. Can you provide the reproduce sequence for this?

There's more discussion in https://github.com/docker/swarm/pull/2526. We never fully figured out a solid repro for this issue, but both me and @michelvocks saw this issue.

@dongluochen @wsong You are right. This is a bug. I spent more time on investigating the issue and the root of it is this commit: https://github.com/docker/engine-api/commit/5dd6452d4dbe3e772728545e9531e3a27dd1e7a9
The client is reusing the same connection. That means it first sends the _ping request and then uses the same connection to send the /info request. The hijacking method in swarm looks like this:
cp := func(dst io.Writer, src io.Reader, chDone chan struct{}) {
		io.Copy(dst, src)
		if conn, ok := dst.(interface {
			CloseWrite() error
		}); ok {
			conn.CloseWrite()
		}
		close(chDone)
	}
	inDone := make(chan struct{})
	outDone := make(chan struct{})
	go cp(d, nc, inDone)
	go cp(nc, d, outDone)

The go-routine reads the whole stream and then forwards it to the primary docker swarm manager and because we are using the same connection for the _ping request as well as the /info request it will forward both requests. I'm now not quite sure what's the right way to go when looking at the design. Should this be fixed in the docker client or at docker swarm?

Rebased on top of #2492

@wsong was there further discussion about this PR? Have you faced the issue lately?

This isn't so much an issue as it is a feature request; because Swarm replicas can occasionally lose their connection to the primary, we want to be able to hit an endpoint telling us whether or not there is currently an elected primary, from the perspective of this replica. This is useful when, for instance, you add a new Swarm manager and it takes a while to connect to the KV store; we can hit _ping in a retry loop to determine when the new manager is ready.

This is no longer relevant now that we block in api/replica.go until we have an elected primary cluster manager.


sp-core build fails when depending on master branch

We are depending on sp-core master branch and there is currently a build failure caused by `tiny-bip39:
error[E0184]: the trait `Copy` may not be implemented for this type; the type has a destructor
   --> /home/tobin/.cargo/registry/src/github.com-1ecc6299db9ec823/tiny-bip39-0.8.0/src/language.rs:116:24
    |
116 | #[derive(Debug, Clone, Copy, PartialEq, Zeroize)]
    |                        ^^^^ Copy not allowed on types with destructors
    |
    = note: this error originates in the derive macro `Copy` (in Nightly builds, run with -Z macro-backtrace for more info)

Looking at the tiny-bip39 codebase it looks like the zeroize derive was removed so an upgrade to 0.8.1 should fix this.
Please Note; I didn't do a PR because I can't clone this repo. FTR here is the error when I tried to do so.
Cloning into '/home/tobin/build/github.com/paritytech/substrate'...
remote: Enumerating objects: 629848, done.
remote: Counting objects: 100% (22402/22402), done.
remote: Compressing objects: 100% (8351/8351), done.
remote: Total 629848 (delta 14493), reused 20904 (delta 13557), pack-reused 607446
Receiving objects: 100% (629848/629848), 372.77 MiB | 5.16 MiB/s, done.
Resolving deltas: 100% (411630/411630), done.
fatal: multiple updates for ref 'refs/remotes/origin/pr/7744' not allowed

the reason you can't clone is probably due to your gitconfig https://stackoverflow.com/questions/54922704/git-cloning-error-fatal-multiple-updates-for-ref-not-allowed
Which version of rust are you using ? we are using 0.8.0 and CI is successful.

@tobin-crypto any update?


[Svelte Flow] On WebKitGTK CSS animations blurry the whole app

Describe the Bug
In WebKitGTK when a css animation is playing (atleast on the node) the whole website gets blurry.
Your Example Website or App
Not mine, but the bug also happens here, and my app has similar styles. https://svelteflow.dev/examples/styling/turbo-flow
Steps to Reproduce the Bug or Issue
.
Expected behavior
As a user I expect the website to not be blurry when a css animation is playing.
Screenshots or Videos
https://github.com/xyflow/xyflow/assets/77930083/31c06704-7d7b-4897-a871-254ea9c5cbde
Platform

OS: [Linux]
Browser: [Epiphany, webkit2gtk-4.1, webkitgtk-6.0]
Version: [Epiphany 45.3]

Additional context
I've also tested a simple css transition and during the transition, the whole app gets blurry.

I believe this has something to do with Webkit2GTK and subpixel rendering. There are a couple of hacks around how to prevent this but afaik its very system dependent. I currently don't have a linux machine I could easily test this on - any help would be appreciated to see if any of the hacks work :)

Unfortunately none of the "hacks"  fixed the issue.


Boilerplate Update & re-generate test harness

boilerplate update
make boilerplate-update

test harness re-generate - uses new osde2e image
 make e2e-harness-generate

SDCICD-1415

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 45.00%. Comparing base (c82d00f) to head (7c2e6d6).

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #154   +/-   ##
=======================================
  Coverage   45.00%   45.00%
=======================================
  Files          12       12
  Lines         680      680
=======================================
  Hits          306      306
  Misses        360      360
  Partials       14       14


Can't log on to new installation with admin/admin

Hi,
I have just set up a new installation but am unable to log in using admin/admin. I seem to get the same error as described in https://github.com/kiswa/TaskBoard/issues/71. The api directory is writeable, sqlite is installed and I have run sudo a2enmod rewrite expires; sudo service apache2 restart. I'm running on Ubuntu 14.04.
Can you help?
Many thanks

Same issue. I uploaded all files onto two different standard web hosts. In one case, there is no error message, but I cannot log in with admin/admin. In the second case, there is a message, reading: API Error. Ensure api directory is writable.
This happens even after setting 0777 for the api directory and all of its subdirectories and files. The PHP versions are 5.3.29 and 5.6.3.

@Torsten-K I'm not sure what to say, other than I need to dig into making the API layer more cross-version compatible. I can't remember right now if 5.3.29 will even work with TaskBoard, and I do remember someone noticing issues with 5.6.x (unfortunately, upstream).
There have been others mentioning that some hosts block the Authentication header, which may be contributing to your problem.
Take a look through some of the other resolved issues to see if anything they've tried helps you out.


ftp.get() hangs for 60 seconds in node v10, but not in node v8

Specifically, when I run the test case below (fetching a single file from a local FTP server), it takes ~53msecs on node v8, but 60000msecs on node 10.
Exact console output listed below as well.
const Ftp = require('ftp');

const host = '<IP_ADDRESS>';
const user = 'anonymous';
const fileName = 'axis.dg';

let client = new Ftp();

client.on('ready', () => {
  console.time('get()');
  client.get(fileName, (err, stream) => {
    stream.on('end', () => {
      console.timeEnd('get()');
      client.destroy();
    });
  });
});

client.connect({host, user});

The exact console output, if it helps:
$ nvm use 8
Now using node v8.11.4 (npm v5.6.0)

$ node test.js
get(): 53.476ms

$ nvm use 10
Now using node v10.11.0 (npm v6.4.1)

$ node test.js
get(): 60081.924ms

After update node version from 8.9.2 to 10.2.0, I also encounter this problem.
But don't know why?

@kingeaglewang: Note that this module appears to have been abandoned (3+ years since a new version published, 1+ years since a contributer has commented on an issue).
I've switched to the basic-ftp module, and have been working with the author on a new release (see the feature/v3 branch).

I also encounter this problem after upgrade node version to 10.16.5,
Is there a bug triggered on 10.x node version ?
any suggestion will be appreciated!

I did some investigation. Node.JS v10 changed to use the way for socket clean up at EOF.
Node.JS v8: Find onread() in https://github.com/nodejs/node/blob/v8.16.2/lib/net.js.
Node.JS v10 Find onStreamRead() in https://github.com/nodejs/node/blob/v10.17.0/lib/internal/stream_base_commons.js.
I'm not sure about the actual difference yet.
But I noticed in connection.js, this node-ftp library sends MODE S when all data is received, switching backing to stream mode. It might be required by compress mode, switching to MODE Z already, and switching back to MODE S.
After commenting out the line of self.send('MODE S'..., keeping _emit('end') and _emit('close'), I don't have 2-min hang on Node.JS v10 any more.
function ondone() {
        console.log(`${Date.now()} done=${done} lastreply=${lastreply}`);
      if (done && lastreply) {
        self._send('MODE S', function() {
          source._emit('end');
          source._emit('close');
        }, true);
      }
    }

@mscdex Time to archive this repo / hand off ownership to someone else?  (... conspicuously not volunteering, however. 😝 )