drop_last_batch=True for IterableDataset map function is ignored with multiprocessing DataLoader

Describe the bug
Scenario:

Interleaving two iterable datasets of unequal lengths (all_exhausted), followed by a batch mapping with batch size 2 to effectively merge the two datasets and get a sample from each dataset in a single batch, with drop_last_batch=True to skip the last batch in case it doesn't have two samples.

What works:

Using DataLoader with num_workers=0

What does not work:

Using DataLoader with num_workers=1, errors in the last batch.

Basically, drop_last_batch=True is ignored when using multiple dataloading workers.
Please take a look at the minimal repro script below.
Steps to reproduce the bug
from datasets import Dataset, interleave_datasets
from torch.utils.data import DataLoader

def merge_samples(batch):
    assert len(batch['a']) == 2, "Batch size must be 2"
    batch['c'] = [batch['a'][0]]
    batch['d'] = [batch['a'][1]]
    return batch

def gen1():
    for ii in range(1, 8385):
        yield {"a": ii}

def gen2():
    for ii in range(1, 5302):
        yield {"a": ii}

if __name__ == '__main__':

    dataset1 = Dataset.from_generator(gen1).to_iterable_dataset(num_shards=1024)
    dataset2 = Dataset.from_generator(gen2).to_iterable_dataset(num_shards=1024)

    interleaved = interleave_datasets([dataset1, dataset2], stopping_strategy="all_exhausted")
    mapped = interleaved.map(merge_samples, batched=True, batch_size=2, remove_columns=interleaved.column_names,
                             drop_last_batch=True)

    # Works
    loader = DataLoader(mapped, batch_size=32, num_workers=0)
    i = 0
    for b in loader:
        print(i, b['c'].shape, b['d'].shape)
        i += 1

    print("DataLoader with num_workers=0 works")

    # Doesn't work
    loader = DataLoader(mapped, batch_size=32, num_workers=1)
    i = 0
    for b in loader:
        print(i, b['c'].shape, b['d'].shape)
        i += 1

Expected behavior
drop_last_batch=True should have same behaviour for num_workers=0 and num_workers>=1
Environment info

datasets version: 2.16.1
Platform: macOS-10.16-x86_64-i386-64bit
Python version: 3.10.12
huggingface_hub version: 0.20.2
PyArrow version: 12.0.1
Pandas version: 2.0.3
fsspec version: 2023.6.0

I have also tested on Linux and got the same behavior.

My current workaround this issue is to return None in the second element and then filter out samples which have None in  them.
def merge_samples(batch):
    if len(batch['a']) == 1:
        batch['c'] = [batch['a'][0]]
        batch['d'] = [None]
    else:
        batch['c'] = [batch['a'][0]]
        batch['d'] = [batch['a'][1]]
    return batch

def filter_fn(x):
    return x['d'] is not None

# other code...
mapped = mapped.filter(filter_fn)


Split arc sector files

Hi! Thank you for helping out with Embedded Graphics development! Please:

[ ] Check that you've added passing tests and documentation
[ ] Add a simulator example(s) where applicable
[ ] Add a CHANGELOG.md entry in the Unreleased section under the appropriate heading (Added, Fixed, etc) and appropriate crate (embedded-graphics, simulator, tinytga, tinybmp) if your changes affect the public API
[ ] Run rustfmt on the project
[ ] Run ./build.sh (Linux/macOS only) and make sure it passes. If you use Windows, check that CI passes once you've opened the PR.

PR description
Closes #368.

Comments addressed as far as I know :)

Thanks!

And also good catch with the unrelated changes.

Yeah, not sure why I didn't catch them as part of #361 but hey, at least we caught them before public release.


Typed functions do not type check correctly when bound with arguments

Tested versions

Reproducible in 4.4.dev2, 4.3.stable, 4.2.2.stable, 4.1.4.stable, and 4.0.4.stable

System information
Godot v4.3.stable (77dcf97d8) - NixOS #1-NixOS SMP PREEMPT_DYNAMIC Wed Sep 18 17:24:10 UTC 2024 - X11 - GLES3 (Compatibility)
Issue description
Consider the following code:
extends Node

func a(data: Array):
	print(data)

func b(data: Array[int]):
	print(data)

func _ready():
	# Calling it works fine
	a([1, 2])
	b([1, 2])

	# Binding and calling the function with the untyped array works
	a.bind([1, 2]).call()

	# Binding and calling the function with the typed array doesn't.
	b.bind([1, 2]).call()

Executing it produces this error in the editor:

Invalid type in function 'Control(Test.gd)::b (Callable)'. Cannot convert argument 1 from Callable to Array.
When the code should not result in any errors.

Similarly, If you use the following code instead, you can get an even funnier message:
func c(function: Callable):
	function.call()

func _ready():
	c.bind(4).call()

Invalid type in function 'Control(Test.gd)::c (Callable)'. Cannot convert argument 1 from Callable to Callable.
These may be different issues - the second bit of code should throw an error, but the error should really be:
Invalid type in function 'Control(Test.gd)::c (Callable)'. Cannot convert argument 1 from int to Callable.

Steps to reproduce

Create a function which takes a typed array as an argument
Bind an array with the right type to the function and run .call() on it.

Minimal reproduction project (MRP)
test.zip

Tested with https://godotengine.github.io/godot-commit-artifacts/ under https://github.com/godotengine/godot/commit/ae45d19ad69d4a23146ab04c85adca00188a0d94 through wine, the issue still seems to exist there. The Linux build failed to compile through nix.

Thank you for reporting, consolidating in:

https://github.com/godotengine/godot/issues/92215

See there for more details, if you think something was missed about this and it's not the same issue, please comment here and it can be reopened

@AThousandShips There is the minor detail that the error message always seems to report the type as "Callable" despite what the object is. Which causes weird messages such as "cannot convert Callable to Callable". I'll edit my message down to just that issue instead, unless it can also be consolidated into the other issue as well.
Also, I'm blind. I do search for these issues beforehand with a bunch of keywords but I guess I don't search hard enough...

That is arguably part of the same issue, but if it isn't I'd say to open a new issue to be clear what it is

See also:

https://github.com/godotengine/godot/issues/97433

I think I'll leave it be for now, as on further overview you already explicitly mention this in that post.


Fail setting HuggingFaceTokenizer

I'm trying to run my application, but I'm getting this message error:
detailMessage -> Model "sentence-transformers/all-mpnet-base-v2" on the Hub doesn't have a tokenizer
I have tried to find this tokenizer model into https://huggingface.co/models, but I don't found any model starting with "sentence-transformers".
I have tried to start the application in debug mode and change the value of "identifier" (HuggingFaceTokenizer.class:80) to test with another model (obrizum/all-mpnet-base-v2) and it has returned a value, only if we are using version 0.22.1.
UPDATE
https://github.com/huggingface/tokenizers/issues/1242

The issue has been resolved by huggingface. 🚀

I did switch to the v1 on the 0.8.2-SNAPSHOT but I will go back to v2 now that it is fixed.


Multiple run instrument trace files do not render properly

If you do multiple runs of a trace, it seems like speed scope only renders the last run?   Or kind of merges them together?  It's kind of hard to tell.
Here is an example trace on safari.

The instruments trace import only visualizes the last run
Each run only visualizes the main thread

It would be great to fix both of those things, and they may need different solutions

When I was trying to make a good example file for this ticket, I created an instruments file that doesn't render at all.  Barely any spans are running since the safari app is fairly idle.  There are also 2 runs.   I was also using instruments v10.  When I create a single run v10 instruments file although, it does render properly with speedscope.
https://uber.box.com/s/hfboka3vmnvxfn8da30r4fnnjwk9m7kh

Thanks for the test file! I haven't tested with Instruments 10 at all -- when was that released?

It was released at WWDC as part of the xcode 10 beta.  You will need an apple developer account to download it , but I think the free account can do it.   Figma has iOS apps on the app store, so your apple account with figma should definitely be able to download the dmg for you.
https://developer.apple.com/xcode/  <- tap the blue download button in the top right
Here is an example one run instruments v10 looking at itunes: https://uber.box.com/s/vk4h1nisc8okb4b2a4qzamns5x0nk19l


Fix home link clicking on mouse hover

When the user hovers over the "Home" link, it's treated as a mouse click.  This is because of the dropdown class that causes a click in JS.

Thank you!


System.PullRequest.PullRequestNumber does not exist

During PR build the variable System.PullRequest.PullRequestNumber does not exist, correct name for this variable is
System.PullRequest.PullRequestId

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 11d3469c-83c2-c2f6-41d0-08f82c330bbc
Version Independent ID: a373c48d-6395-3002-d2d8-bbc1fea2e48e
Content: Predefined variables - Azure Pipelines
Content Source: docs/pipelines/build/variables.md
Product: devops
Technology: devops-cicd-build
GitHub Login: @juliakm
Microsoft Alias: jukullam

System.PullRequest.PullRequestNumber is populated only for github, that's what I missed there


fix-anchors: implement testing

what

Fix #105
Implement #94

why

we need to support expanding anchors/aliases in YAML
we should be unit-testing for regressions, as found in #93

references
#93

Your development orb has been published. It will expire in 30 days.
You can preview what this will look like on the CircleCI Orb Registry at the following link:
https://circleci.com/developer/orbs/orb/bjd2385/dynamic-continuation?version=dev:0f0cdcb8f0f1ca53c160b889531423b66dee60ef

Your development orb has been published. It will expire in 30 days.
You can preview what this will look like on the CircleCI Orb Registry at the following link:
https://circleci.com/developer/orbs/orb/bjd2385/dynamic-continuation?version=dev:407ebab844e0bce59ebbc33429d7d4b3a15054b5

Your development orb has been published. It will expire in 30 days.
You can preview what this will look like on the CircleCI Orb Registry at the following link:
https://circleci.com/developer/orbs/orb/bjd2385/dynamic-continuation?version=dev:0b03388156b79ac8b27996b4fcafa12f2db6c2c9

Your development orb has been published. It will expire in 30 days.
You can preview what this will look like on the CircleCI Orb Registry at the following link:
https://circleci.com/developer/orbs/orb/bjd2385/dynamic-continuation?version=dev:64574fda16fdfa5e1e72df4c30714c410ff7b1cd

Your development orb has been published. It will expire in 30 days.
You can preview what this will look like on the CircleCI Orb Registry at the following link:
https://circleci.com/developer/orbs/orb/bjd2385/dynamic-continuation?version=dev:d3b7ec50a18549f223febb1d98815ce80b056e95

Your development orb has been published. It will expire in 30 days.
You can preview what this will look like on the CircleCI Orb Registry at the following link:
https://circleci.com/developer/orbs/orb/bjd2385/dynamic-continuation?version=dev:f56515176d6c21757187a967fe06d6cb7e51c4f3

Your development orb has been published. It will expire in 30 days.
You can preview what this will look like on the CircleCI Orb Registry at the following link:
https://circleci.com/developer/orbs/orb/bjd2385/dynamic-continuation?version=dev:a9213a07d3e1f4a74624b085ae875d323471ed1c

Your development orb has been published. It will expire in 30 days.
You can preview what this will look like on the CircleCI Orb Registry at the following link:
https://circleci.com/developer/orbs/orb/bjd2385/dynamic-continuation?version=dev:0396728e9404f483d56ae2f46f7d69025c095c2d

Your development orb has been published. It will expire in 30 days.
You can preview what this will look like on the CircleCI Orb Registry at the following link:
https://circleci.com/developer/orbs/orb/bjd2385/dynamic-continuation?version=dev:b0d756ca76e5d6e7bebbca0db14cc00361fcd688


DHCP Option ID 234 is wrong

[Enter feedback here]
DHCP Option ID is 235 not 234.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: df445cda-3b0a-2ad7-e3cc-23e159013530
Version Independent ID: a6bb5f97-d350-f7b7-77f7-b0093322289d
Content: Delivery Optimization reference - Windows Deployment
Content Source: windows/deployment/update/waas-delivery-optimization-reference.md
Product: w10
Technology: windows
GitHub Login: @jaimeo
Microsoft Alias: jaimeo

Did you know you can make simple edits like this yourself? Just select the "Edit" button (right next to the "Feedback" button) and make the fix. We'll welcome the contribution.


Screenshot out of date

Code of Conduct

[X] I have read and agree to the GitHub Docs project's Code of Conduct

What article on docs.github.com is affected?
https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-branches-in-your-repository/renaming-a-branch#renaming-a-branch
What part(s) of the article would you like to see updated?
The screenshots and description of the "Branches" button is out of date with the current UI. Now the "Branches" button is inside the dropdown when you click on a branch.
Additional information
No response

Yeah looks fine now, not sure what I was seeing before, my bad

@matthewkennedy5 No worries! Thanks for your interest in helping us improve the docs!  👍


Rename Distribution.InitialBalances.(governance → treasury)

Blocked on https://github.com/dfinity/ic/pull/1755

Will you make the same change in snsdemo?


Add commenting describing application version to saved spreadsheets

Fixes #89.

@griffie Add a comment to cell B1 that describes the application version used when saving. Not sure where to, or if I even should, add a comment for exported GISAID and IRIDA files. Neither has empty header cells.

Hmm, I think having the DH version is important. The way you have it now works. For GISAID I don't think they need to know this info so if the top line gets tripped out it's ok. But I think it would be good to have for IRIDA. Would it be possible to add it as a header after the last column (ie after the "author" field)?

Yes, it's possible. Should we make it consistent and add it after the author field for the regular format as well?

Yeah, sounds good! :)


Ignore exceptions

I would like to use wwhrd in our CI pipeline and I would like to have it silent (with -q), that is without any output, if there are no license violations to take care of. This is, if an exception is granted by adding it to .wwhrd.yml and the flag -q is given, I would like to have no output.
Would you accept a PR to add this functionality? Should this be implemented as an additional flag to maintain the backwards compatibility?

I think we should probably just raise the default log level to Error for the -q flag, this would make a run with exceptions silent.


Updates coming Soon

Hello,
For those how haven't joined the community (https://groups.google.com/g/ansible-lockdown) and watching the ansible-lockdown repository for rhel7-stig. We are trying to inform as many of you as possible of upcoming changes and improvements we are hoping to release very shortly.
Please see the community notification below.
Hello,
Thank you to everyone in the Ansible-Lockdown community who has contributed to RHEL7 STIG/CIS. Our team at MindPoint Group has been working with the entirety of the Ansible-Lockdown project, and we have some significant updates for both RHEL 7 STIG and CIS. With these updates, some larger changes have been made. I have these changes/updates outlined below.
Testing:
CI/CD - We have implemented some automated testing pipelines to test pull requests into the devel and main branches. With the current workflow, the community will PR into the devel branch (never the main branch) for review by the administrators. When your PR is created, the first check will remain the DCO check. The second check is a functional testing pipeline that will automatically perform a functional test of the branch the PR is initiated from. Once both tests pass, someone from the Administrator Team will review the changes and merge them into the devel branch. From there, an additional review is completed before the devel branch is merged into the main branch. Only the Admin Team will perform PRs/merges into the main branch. There is also an automated pipeline for PRs from devel to main. Please do not edit the .github/workflows files since those are used as part of the pipeline.
Compliance Checking – MindPoint Group has been working to create our own compliance audit scan tool. The tool uses a goss framework executable to run custom checks that we have created. The goal is to provide a more thorough check for control compliance and decrease the number of false positives/negatives. For example, it will check the configuration file related to the control and as well as checking if that configuration is active. With a smarter scan, we can hopefully identify attempts to trick scanners as well (for example stacking a parameter in a config file where the first instance is enabled and second disabled. Most audit tools search for the first instance but the application might look for the last instance of the parameter, thus making the scanning tool think it's enabled). In testing, we have found that our audit scan runs significantly faster than other audit tools, reducing audit times. Our audit tool and profiles will have their own repositories in the Ansible-Lockdown org, but within the remediation role there will be an integrated way to incorporate the audit. Keep an eye out for the audit tools as they are released. We plan on developing a goss audit profile for each current remediation role. Going forward, we plan to release a remediation role and goss audit tool profile simultaneously.
Role Updates:
RHEL 7 STIG/CIS – We have re-written much of the RHEL 7 STIG and CIS roles to increase clarity and readability and address some functionality items. We performed these updates while creating our goss testing framework for each of these roles. We plan on pushing our update to the devel and main branches. We will move the current devel and master branches to a devel_stable_ and master_stable_ branch in the respective repositories. Accordingly, community members who rely on the current version can still use that version going forward; this process will not remove what is currently there. The latest versions of the roles have also been updated to comply with the latest benchmarks.
Role Architecture – All roles will change with regard to the structure in the tasks folder. Taking CIS as an example, there will be a folder per section and yaml files for each sub-section. For example control 1.2.1 in CIS will be located in RHEL7-CIS/tasks/section_1/cis_1.2.x.yml. The cis_1.2.x.yml file will contain all controls related to section 1.2.x. This will hopefully make updates to roles a bit easier with less risk. This matches the architecture of our audit tool, creating consistency across remediation and audit platforms. The end goal is to repeat this architecture (the best we can) on STIG roles, but we are starting with CIS.
Existing PRs and Issues – With all of these changes comes the task of cleaning up existing PR’s and issues. Our plan is to close all of the existing PR’s and issues because of the re-work. Our team is growing and should be able to stay on top of the new issues and PRs as they come in.
Again, I would like to thank the community for your involvement in this project. The input and work from the community has contributed significantly to the success of this project. Please keep an eye out for these changes, which will be rolling out in the coming weeks.
-George

Hello,
We have merged our changes into the devel branch and closed all open issues/PRs. Please use the latest version in the devel branch and let us know if there are any issues that need to be addressed. We are using the next week or so as a test/dev cycle before doing a full on 1.0.0 release in the main branch.
-George

Releases are now live and tagged


Adding coveralls.io

Changes Unknown when pulling c8f7a3e118141747a0e8016eb36aaaf886a521d4 on nacho.coveralls into ** on master**.

Changes Unknown when pulling 698cd098e948145d6fcba197d7300ce48a965247 on nacho.coveralls into ** on master**.


How to run test

Hi,
By following your doc I don't get how to run a unitTest with the angular-cli config. Can you help me for that please?
Thanks.

Hi. First build your app with ng build then run karma start.

Running tests will be simpler in the future by just running ng test and you can track the progress in issue #70
Until that time you can follow the instructions in the above comment and find more information in the README.md

I always have this error when I run
karma start

04 02 2016 21:37:01.677:WARN [watcher]: Pattern
"/Users/steph/Project/node_modules/angular2/bundles/testing.js" does not match any file.
04 02 2016 21:37:01.706:WARN [watcher]: Pattern "/Users/steph/Project/dist/**/*.css" does not match any file.
04 02 2016 21:37:01.737:WARN [karma]: No captured browser, open http://localhost:9876/
04 02 2016 21:37:01.747:INFO [karma]: Karma v0.13.19 server started at http://localhost:9876/
04 02 2016 21:37:01.753:INFO [launcher]: Starting browser Chrome
04 02 2016 21:37:03.434:INFO [Chrome 48.0.2564 (Mac OS X 10.11.2)]: Connected on socket /#g6JpnEvQqb0ummP6AAAA with id 80857190
04 02 2016 21:37:03.780:WARN [web-server]: 404: /rxjs/Subject
04 02 2016 21:37:03.782:WARN [web-server]: 404: /rxjs/observable/fromPromise
04 02 2016 21:37:03.782:WARN [web-server]: 404: /rxjs/operator/toPromise
04 02 2016 21:37:03.783:WARN [web-server]: 404: /rxjs/Observable
Chrome 48.0.2564 (Mac OS X 10.11.2) ERROR
Error: XHR error (404 Not Found) loading http://localhost:9876/rxjs/Subject
at error (/Users/steph/Project/node_modules/systemjs/dist/system.src.js:1028:16)
at XMLHttpRequest.xhr.onreadystatechange (/Users/steph/Project/node_modules/systemjs/dist/system.src.js:1036:13)

Are you using the master?

I don't really know, I installed angular-cli via npm

You are not using master then. Please see the readme how to use it.
On Friday, 5 February 2016, stephanec1<EMAIL_ADDRESS>wrote:

I don't really know, I installed angular-cli via npm
—
Reply to this email directly or view it on GitHub
https://github.com/angular/angular-cli/issues/178#issuecomment-180182776
.

Ok thank you for your help jkuri :+1:

@stephanec1 the latest version on npm is a bit out of date in regards to master, sorry you had trouble!


buffers scroll too far down

Check for existing issues

[X] Completed

Describe the bug / provide steps to reproduce it
when scrolling down in a buffer it scrolls until the last line is at the top. that seems rather useless to me. I often wanna quickly go to the bottom of the file, but with thed after flicking down I always have to move back up a bit to actually see anything
Environment
Zed: v0.90.2 (stable)
OS: macOS 13.2.1
Memory: 16 GiB
Architecture: aarch64
If applicable, add mockups / screenshots to help explain present your vision of the feature
No response
If applicable, attach your ~/Library/Logs/Zed/Zed.log file to this issue.
If you only need the most recent lines, you can run the zed: open log command palette action to see the last 1000.
No response

There are some - including me - that, when writing code, prefer to have the cursor close to the top of their monitor. This is why most editors allow to scroll past the end of the file. However, it might make sense to put it behind a preference (e.g. in Sublime Text the preference is called scroll_past_end).

ok, yah, then at least such a preference would be really nice :)

I believe this is covered by #1053. I've love the option too. Please give it a 👍!

Appreciate the feedback @maddanio - I think we will close this one in favor of #1053, like @wzxu mentioned, to keep the 👍 in one place.


poseWorldLandmarks missing from holistic js api

Please make sure that this is a solution issue.
System information (Please provide as much relevant information as possible)

OS Platform and Distribution (e.g., Linux Ubuntu 16.04, Android 11, iOS 14.4): html javascript
MediaPipe version: 0.5.1635989137 as per https://www.jsdelivr.com/package/npm/@mediapipe/holistic
Solution: Holistic
Programming Language and version: JavaScript, chrome

Describe the expected behavior:
holistic.onResults should also provide poseWorldLandmarks in the same way as pose.onResults
Actually I wish holistic.setOption should provide option to let us enable and enable/disable each feature. And worldPosition should be boolean option to change every landmarks into world position. So we could have hand and face mesh in world position too
Standalone code you may have used to try to get what you need :

  const pose = new Pose({
    locateFile: (file) => {
      if(file.startsWith("pose"))
        return `https://cdn.jsdelivr.net/npm/@mediapipe/pose/${file}`;
      return `https://cdn.jsdelivr.net/npm/@mediapipe/holistic/${file}`; // by unknown reason. Sometimes holistic file would trying to be loaded in here
    }
  });

  pose.setOptions({
    modelComplexity: 1,
    smoothLandmarks: true,
    enableSegmentation: false,
    smoothSegmentation: false,
    refineFaceLandmarks: true,
    minDetectionConfidence: 0.2,
    minTrackingConfidence: 0.2
  });

  pose.onResults((result) => {
     result.poseWorldLandmarks; // exist
  });

  const holistic = new Holistic({
    locateFile: (file) => {
      if(file.startsWith("pose"))
        return `https://cdn.jsdelivr.net/npm/@mediapipe/pose/${file}`;
      return `https://cdn.jsdelivr.net/npm/@mediapipe/holistic/${file}`;
    }
  });

  holistic.setOptions({
    modelComplexity: 1,
    smoothLandmarks: true,
    enableSegmentation: false,
    smoothSegmentation: false,
    refineFaceLandmarks: true,
    minDetectionConfidence: 0.2,
    minTrackingConfidence: 0.2
  });

  holistic.onResults((results) => {
     result.poseWorldLandmarks; // missing
  });

Other info / Complete Logs :
undefined

Hi @Thaina ,can you provide steps to reproduce this issue.

@sureshdagooglecom Every holistic sample in js has this issue. poseWorldLandmarks just missing

@sureshdagooglecom What response do you need?

As you can see just debug the holistic result object.

There was no poseWorldLandmarks and also no mention about world coordinate for left and right hand
As per document here there should be output for world landmark https://google.github.io/mediapipe/solutions/holistic.html#pose_world_landmarks

And also there was not even a mention about left/right hand in world coordinate
What happen to you?

@sureshdagooglecom What are you waiting for?

@Thaina The cdn version does contain the world landmark, but it is result.ea instead of 'result.poseWorldLandmarks'. However, poseWorldLandmarks is indeed missing in the npm package export. @sureshdagooglecom Would you mind taking a look at the missing poseWorldLandmarks in @mediapipe/holistic? I believe this is a bug because 'result.poseWorldLandmarks' is available in @mediapipe/pose and 'result.poseWorldLandmarks' is not in any version of holistic in the npm package.

@senzen888 Thank you very much for your clarification
Indeed poseWorldLandmarks exist in @mediapipe/pose and I need to use that instead of holistic.
But I don't use npm version directly (I don't have knowledge about new js import system at all).
However I can still confirm that it not exist in npm version too. Because I use JSDoc typescript for js autocomplete in vscode which installed from npm. And the typescript version also not present poseWorldLandmarks in holistic.onResults
And so, what would it will be the fixed of this problem? Did you will have us use ea property instead? I wish that would be properly mentioned in documents too then
Thank you very much again

Hey everyone! I also noticed that there is only the ea option in @mediapipe/pose. I am also wondering about this line in the documentation: z: Should be discarded as currently the model is not fully trained to predict depth, but this is something on the roadmap.. When I use poseWorldLandmarks I actually get some decent results for the z axis but now I'm not sure what to use?

@henning101 In my opinion, there was difference model between normal landmarks and world landmarks and currently now poseLandmarks cannot give you decent z
So if you really need to map xyz value to 3D application you should use poseWorldLandmarks
While poseLandmarks made for other usage like gesture and segmentation. Especially relative to screen

@Thaina Hi Thaina! Thank you for your reply! I worked on it a bit more and I realized that I need more stability for the z coordinate in general. Do you maybe know some good approach to derive a z approximation from the 2D pose e.g. via keypoint angles.

@Thaina Yes, the way around this would be to simply use ea for now if you are using the CDN version. I unfortunately cannot improve the docs because I am not part of google and figured this out via logging. @sureshdagooglecom would you mind exposing ea in the type definition. I did this locally and resolved the issue for me.

Hi @Thaina ,
Hope https://github.com/google/mediapipe/issues/3155#issuecomment-1079803517 helps to resolve this issue and please reopen this issue if still issues persists.

The issue is not solved. That was only short term workaround. And I would suggest please properly fix the API

@sureshdagooglecom I can't reopen this issue by my own

Is this already fixed?

Hello @Thaina,
We are upgrading the MediaPipe Legacy Solutions to new MediaPipe solutions However, the libraries, documentation, and source code for all the MediaPipe Legacy Solutions will continue to be available in our GitHub repository and through library distribution services, such as Maven and NPM.
You can continue to use those legacy solutions in your applications if you choose. Though, we would request you to check new MediaPipe solutions which can help you more easily build and customize ML solutions for your applications. These new solutions will provide a superset of capabilities available in the legacy solutions.

Hello @Thaina, We are upgrading the MediaPipe Legacy Solutions to new MediaPipe solutions However, the libraries, documentation, and source code for all the MediaPipe Legacy Solutions will continue to be available in our GitHub repository and through library distribution services, such as Maven and NPM.
You can continue to use those legacy solutions in your applications if you choose. Though, we would request you to check new MediaPipe solutions which can help you more easily build and customize ML solutions for your applications. These new solutions will provide a superset of capabilities available in the legacy solutions.

Would the holistics will be made into the list of solutions ?

@Thaina,
This is already in our roadmap(Check here). We will let you know once becomes available from our end. For now we can not provide any timeline about the release date.  Thank you!

Thank you very much
Still I would like to ask, will the upgrade version support multiple holistic landmarks and stream of depth image too? Depth camera become more common and so support it would make holistic more accurate

@Thaina,
This needs to be checked with new solutions available. For now, We do not have any such documentation where the changes done compare to old legacy with upgraded one are available. Our new solution has been improved and performance increased. Further, We will let you know if any relevant document will be available from our end. However, We would like you to explore upgraded solution and revert back if the poseWorldLandmarks are missing from it.  Thank you!

So the upgrade are not done yet? Are there any progress?

@Thaina,
We will working towards publish the solution as soon as possible, Which still is in our roadmap And will let you know once becomes available from our end.

To be honest I have to ping this to keep the bot out from marking this as stale

Thanks for the replay back, Will keep the issue until the solution will be available from our end And will let you know.

@kuaashish Thank you very much. I see there is poseWorldLandmarks API as expect
However, seem like there is no faceWorldLandmarks equivalence for faceLandmarks. Did it impossible to provided or it just being left out by some reason?

Hi @Thaina,
Currently, there are no faceWorldlandmarks available for the faceLandmarker. However, we currently lack information on this matter. In the meantime, could you please submit a feature request using the provided template with the necessary fields? We will share this with our team, and if there is enough demand, we may consider adding this feature in the near future. We would appreciate your confirmation to mark this issue as resolved and close the status, As our new Holistic Task API now returns the poseWorldlandmarks.
Thank you!!


Handle missing maps in state page tabs

Eight notable maps are missing for the states with only two congressional districts.

ID, MT, NH, RI, and WV

The nonexistent radar diagrams for missing maps is a bit of big hammer ugle hack, but it works and is rare.


Cannot get the widget to read data from excel file, and with original widget date don't show up!

HI,
i created a new job file that extracts information from an excel file and send it to the widget.
i tried that job file with "List" widget and "birthday" and data shows correctly, but when i try it with the TIMELINE widget, data never shows up, i just get an empty grey rectangle with the title on the top, that's it.
can someone help me with it please?
thanks
this is my code:
require 'date'
require 'roo'
require 'active_support'
require 'active_support/core_ext'
timeline_FILENAME = 'timeline_data.xlsx'
SCHEDULER.every '3s', :first_in => '1s' do
timeline = []
timeline_report = Roo::Excelx.new(timeline_FILENAME)
(2..timeline_report.last_row).each do |row|
name = timeline_report.cell(row, 1)
name.sub!(/(\S+),\s*(\S+).*/, '\2 \1')
date = timeline_report.cell(row, 2)
date = Date.new(Date.today.year, date.month, date.day)
if date < Date.today then
  date += 1.year
end

timeline.push({name: name, date: date})

end
get next 10 timeline
timeline = timeline.sort{|a, b| a[:date] <=> b[:date]}[0..9]
assemble data to send to client
timeline_data = []
timeline.each do |timeline|
timeline_data.push({
'label' => timeline[:name],
'value' => timeline[:date].strftime("%-m/%-d")
})
end
send_event('data', { items: timeline_data,
class: 'icon icon-background icon-calendar'})
end

Have a look at the jobs/timeline.rb file in the repo... you need to send your data in the same format as is being sent in that job.  Basically its an array of json objects, where each json object is an event.
events << {
          name: event["name"],
          date: event["date"],
          background: event["background"],
          opacity: 0.5
        }

Thanks a lot, i'll give it a shot! :)


[BUG] Image scaling is abnormal

The runing:
self.picam2 = picamera2.Picamera2()
self.picam2.configure(self.picam2.preview_configuration({"size": (1536, 896)}))
self.picam2.start_preview(Preview.NULL)
self.picam2.start()
....
frame = self._webcam.picam2.capture_array()
if frame is None:
return
frame = cv2.cvtColor(frame, cv2.COLOR_RGB2BGR)
cv2.imshow("Camera", frame)
Both are multiples of 64 (1536, 896)
Then the final image gets only the reduced part.. not all full screen

Hi, thanks for the question. Could you say what kind of Pi and what kind of camera you are using?
If you were able to supply a standalone short script that I could run directly, that would be very helpful. Please also include the code within code tags (triple backquotes) so that the formatting does not get destroyed. Thank you!

Thanks for posting the updated script. It runs fine for me but I'm not quite sure what you mean by "abnormally scaled". Are you referring to the fact that the image is cropped and that you do not see the full field of view of the camera?
The v2 camera supports several different readout modes at different resolutions, and if we don't tell it then it will "guess" which one to use. You can find out the available modes by typing libcamera-hello --list-cameras. In this case, it's choosing a 1920x1080 mode because it is larger than the requested size and it has the correct aspect ratio. However, because it needs to use the full sensor resolution it means that it has to crop all the edges off.
In your case, you could replace the self.picam2.configure(self.picam2.preview_configuration({"size": (1536, 896)})) line by
        config = self.picam2.preview_configuration({"size": (1536, 896)}, raw={"size": (1640, 1232)})
        self.picam2.configure(config)

Here, specifying a "raw" stream forces the camera into the 2x2 full field of view binning mode (I took these numbers from the output of libcamera-hello --list-cameras). Of course, it still has to crop off the top and bottom because of the aspect ratio, but you should see the full width.
Am I answering the right question? Thanks!

...
Are you referring to the fact that the image is cropped and that you do not see the full field of view of the camera? Yes!!
        config = self.picam2.preview_configuration({"size": (1536, 896)}, raw={"size": (1640, 1232)})
        self.picam2.configure(config)

I have set it but it still doesn't show full screen, libcamera-hello --list-cameras <- There is display 1920 x 1080. Assuming that 1920 x 1080 is set, only the 200 x 100 part can be seen. Is it possible that the crop is abnormal? or Is there a way to reset the screen?

Hi again, I'm still not really exactly understanding what you've tried and what result you have got. Can you just confirm:

What image size do you want?
What preview configuration did you ask for?
What values does frame.shape report?
If you capture an image (e.g. self.picam2.capture_file("test.jpg")) does that have the resolution and field of view that you expect?

Thanks!

Hey folks, maybe I can chime in on this. Seeing the same issue on my end. Perhaps I'm misunderstanding the documentation.
Here's the gist of the code I'm using:
def captures():
    resolutions = [(640, 480), (3200, 2400), (3280, 2464)]
    for res in resolutions:
        width, height = res[0], res[1]
        camera = Picamera2()
        camera.configure(camera.create_still_configuration(main={"size": res}))
        camera.capture_file(f"picture_{width}x{height}.jpg")

I expected this to output the full image as captured by the sensor, but scaled to the given resolution.
Instead, it actually takes a photo of a crop around the middle of the image sensor!
Here are the photos:
(640, 480)

(3200, 2400)

(3280, 2464)

And here are the crop areas, outlined in red:
(640, 480) inside of (3280, 2464)

(3200, 2400) inside of (3280, 2464)

What is the recommended approach to taking a full image with the camera (no cropping!) and simply getting back a low-resolution version. I know I can simply use cv2.resize() but I think there's a faster, camera-hardware-based version of accomplishing this, no?

Hi, the problem is that when you specify only the output resolution, the camera doesn't know what sensor mode you really want. You might want the full field of view, or you might want a reduced field of view if that gives you a faster framerate (for example).
For the v2 camera I would recommend either:
        camera.configure(camera.create_still_configuration(main={"size": res}, raw={"size": (1640, 1232)}))

or
        camera.configure(camera.create_still_configuration(main={"size": res}, raw={"size": (3280, 2464)}))

The former will give you a faster framerate but lower quality (because the sensor is only outputting 1640x1232). The latter will give you better quality (the sensor is outputting its full resolution) but a slower framerate.
Running libcamera-hello --list-cameras will list the available raw sensor resolutions.
The next release of Picamera2 will also support listing all the sensor modes, and will even tell you what framerate and field of view they all support.

Hi, is there still anything we can help you with here? If there is please let us know, otherwise I'll consider closing this issue shortly.

Hi again, I think I'll close this issue for the time being. If anything else comes up, or there's anything else for us to look at, please feel free to open a new issue. Thanks!


Make NSwag.MSBuild usable from .netstandard

There's currently nothing within that package that would prevent us from using it within a .net standard library (e.g. from within the client API C# project) except the way how it is packaged.
Can you please create a package that supports .netstandard2.0 ? Thanks alot :)

A visual studio restart solved the restore problem. Closed.


Manage log levels

Hello, nice package, thanks for it :)
I have a small question regarding log level management in AWS, and what would be the best approach to manage those.
I work in a 3-stages project ("dev", "staging" and "production") with AWS Lambda.
I would like the "dev" stage to be the most verbose, and increase the level for the other ones.
(e.g. perhaps "info" for "staging" and "warn" for "production", to be defined)
The deployment of node.js handlers is done via serverless framework where I can use environment parameters.
I was thinking to set-up a custom parameter called for instance log-level that would become the minimal level you wish to log (default to "debug")...
But how to reflect that with this package, should I write my own instance of LambdaLog programatically disabling some levels based on it:
import { LambdaLog } from 'lambda-log';

const levels = {
  debug: 0,
  info: 1,
  warn: 2,
  error: 3
};

const logLevel = process.env.LOG_LEVEL in levels
  ? levels[process.env.LOG_LEVEL]
  : levels.error;

export default new LambdaLog(
  {
    debug: Boolean(process.env.DEBUG)
  },
  {
    debug: () => logLevel <= levels.debug ? 'debug' : false,
    info: () => logLevel <= levels.info ? 'info' : false,
    warn: () => logLevel <= levels.warn ? 'warn' : false
  }
);

Or is there another way to do so?

@flosch-hb That would be the correct way to handle it, yes. Since this package is not built using the same ideas of other logging packages, such as winston, there is currently no option to only log to a certain level. I do like the idea of adding it as a feature sometime in the near future, it will, of course, require some major underlying changes similar to your example in which I'm not sure is fully beneficial as a whole.
Long story short, what you wrote is the way I'd handle the same thing in one of my applications if I needed the same. I'll continue to look into a type of feature that could handle something like this for a future version.

I am completely fine with that approach, I tested it and it works, that is all I need :)

Having the ability to easily set minimum log level for a given environment seems key. Otherwise this logger looks nice.


Can I provide a docker installation method?

Quick and easy installation with docker

Quick and easy installation with docker

Sure! Glad to your contribution.

Already support run/debug project in docker container


Feature/rename event

This fixes [#OSF-4342] and [#OSF-4441]
Adds a class to the event module so that rename events are handled correctly.

Please open a new PR targeted at the feature/aggregate-emails branch.


added restartsec and startlimitinterval configurations

Codecov Report

Merging #152 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #152   +/-   ##
=======================================
  Coverage   92.43%   92.43%
=======================================
  Files           4        4
  Lines         542      542
=======================================
  Hits          501      501
  Misses         41       41

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 68d1687...0faf68a. Read the comment docs.

By default, when you configure Restart=always as we did, systemd gives up restarting your service if it fails to start more than 5 times within a 10 seconds interval. Forever.
There are two [Unit] configuration options responsible for this:
StartLimitBurst=5
StartLimitInterval=10
The simple fix that always works is to set StartLimitInterval=0. This way, systemd will attempt to restart your service forever.
It’s a good idea to set RestartSec to at least 1 second though, to avoid putting too much stress on your server when things start going wrong.
Source

By default, when you configure Restart=always as we did, systemd gives up restarting your service if it fails to start more than 5 times within a 10 seconds interval. Forever.
There are two [Unit] configuration options responsible for this:
StartLimitBurst=5
StartLimitInterval=10
The simple fix that always works is to set StartLimitInterval=0. This way, systemd will attempt to restart your service forever.
It’s a good idea to set RestartSec to at least 1 second though, to avoid putting too much stress on your server when things start going wrong.
Source

The article shows different from what  you are proposing.
StartLimitBurst=5
StartLimitIntervalSec=10
vs

StartLimitBurst=5
StartLimitInterval=10

@oguzhaninan Did you close this because of the questions or because you decided it was no longer relevant?

@pryorda You ask simple questions and I don't have time. Sorry.

👍


Implement password recovery

Proposed Changes

Add support for password recovery.

solves #307
Type of Changes
What type of change this Pull Request brings to Falko?

[ ] Bugfix
[x] New Feature

Checklist

[x] This Pull Request has a significant name.
[ ] The build is okay (tests, code climate).
[x] This Pull Request mentions a related issue.

Screenshots
Reset link:

Forgot password page:

Reset password form:

Email sent:

@wallacybraz Could you solve the conflicts so we can merge?


turn on line numbers and column by default

I love this vimrc! I found it very useful to also include the column number in the status bar, as it helps adhering to coding standards such as the 80 or 120 character limit per line.
Also, while it is duplicated information, I think having number set also makes working with files easier.
These changes are to the basic vimrc. I submit these changes in the hopes that it helps out future users of the basic version, as I was/am.

:+1:

Please add this to your own config:
https://github.com/amix/vimrc#how-to-include-your-own-stuff
You can see line numbers at bottom right. I think adding them on every line creates a lot of distraction.


"Check Dependencies" and "Exit Status: 65" in Xcode 8 problem

This issue is for people searching hard in fastlane/issue.
First you should read Official Codesigning Guide
Second, the confusing error is because of xcpretty bug https://github.com/supermarin/xcpretty/issues/235
then, quote as I first write in #6219

You guys should check gym log normaly in your ~/library/logs/gym/xxx.log
For our project, it's the codesign problem.
And we solve that by manual setting in Xcode 8 build setting with developer team and the provisioning profile specifier.
like
DEBUG:
"AwesomeApp Dev"
RELEASE:
"AwesomeApp AppStore"
these are the name in your developer portal profiles
and you can use Fastlane to change the profile use something like
def get_profile_name(uuid)
  profiles_path = File.expand_path("~") + "/Library/MobileDevice/Provisioning Profiles/*.mobileprovision"
  profile_paths = Dir[profiles_path]

  profile_paths.each do |profile_path|
    profile = Plist.parse_xml(`security cms -D -i '#{profile_path}'`)
    if profile['UUID'] == uuid
      return profile['Name']
    end
  end
end

lane :yourlane do
  cert
  sh '"sigh manage -p 'AwesomeApp' --force"' # I have to say the pattern mapping logic in the source file sigh/local_manage some kinds not easy to use
  sigh
  profile_name = get_profile_name lane_context[SharedValues::SIGH_UDID]
  gym(xcargs: "PROVISIONING_PROFILE_SPECIFIER='#{profile_name}'")
end

That's what we get everything works again without the recommended match.
good luck!

It may not good for multi target project

just write a action:
module Fastlane
  module Actions
    module SharedValues
    end

    class GetProfileNameAction < Action
      def self.run(params)
        uuid = "#{params[:uuid]}"
        UI.message "Request Profile name of UUID: #{uuid}"

        profiles_path = File.expand_path("~") + "/Library/MobileDevice/Provisioning Profiles/*.mobileprovision"
        profile_paths = Dir[profiles_path]

        profile_paths.each do |profile_path|
          profile = Plist.parse_xml(`security cms -D -i '#{profile_path}'`)
          if profile['UUID'] == uuid
            return profile['Name']
          end
        end

      end

      #####################################################
      # @!group Documentation
      #####################################################

      def self.description
        "Pass in Profile UUID and get its name"
      end

      def self.available_options
        [
          FastlaneCore::ConfigItem.new(key: :uuid,
                                       env_name: "FL_GET_PROFILE_NAME_UUID", # The name of the environment variable
                                       description: "UUID for GetProfileNameAction", # a short description of this parameter
                                       verify_block: proc do |value|
                                          UI.user_error!("No UUID for GetProfileNameAction given, pass using `uuid: 'UUID'`") unless (value and not value.empty?)
                                       end)
        ]
      end

      def self.return_value
        "Requested profile name if exist"
      end

      def self.authors
        ["https://github.com/puttin"]
      end

      def self.is_supported?(platform)
        true
      end
    end
  end
end

then you can put in your fastlane/actions/ folder
get_profile_name(uuid: "xxx")


Generational  ZGC not supported

The ZGC log pattern seems to have changed since jdk21, do you have a plan to support it?
Known changes for CYCLE_START:
before:
[20.134s][info ][gc          ] GC(1) Garbage Collection (System.gc()) 22M(1%)->24M(1%)

after:
[251.570s][info][gc          ] GC(5) Major Collection (Proactive) 684M(23%)->346M(12%) 0.473s

Yes, Pull Requests are also welcome here though (it's an ever moving target!)

Hi, I have work on genZ underway. I'm willing to push the work in my local branch if there is interest

Hi, I have work on genZ underway. I'm willing to push the work in my local branch if there is interest

@kcpeppe  I am sure interested.  But sorry for my naive question, why is it a local branch not the official repository?

I also  have a preliminary implementation regarding genz, and can meet my own needs ,don't know if it should be submited.

You should both push branches as part of a PR request and let's have a look at them!

Yes, please push and let's see if we can integrate them.

Hi everyone, may I ask when the next version is planned to be released? We are also looking forward for the generational ZGC support and I see that the latest version available in maven central is from a year ago.


i18n(ko-KR): update getting-started.mdx

What kind of changes does this PR include?

Translated content

Description
update getting-started.mdx

Related Issue / Implementation PR
N/A

LGTM!


🤗 [REQUEST] - zephyr-7b-beta

Model introduction
https://huggingface.co/HuggingFaceH4/zephyr-7b-beta
Model URL
https://huggingface.co/HuggingFaceH4/zephyr-7b-beta
Data type
bfloat16
Additional instructions (Optional)
No response
Author
No
Security

[X] I confirm that the model is safe to run which does not contain any malicious code or content.

Integrity

[X] I confirm that the model comes from unique and original work and does not contain any plagiarism.

cc: @ganler

(e2) kamil@uCorei7:~/ml/evalplus$ python evalplus/evaluate.py --dataset humaneval --samples ./worksdir2/humaneval/mistral-7b-zephyr-7b-beta_temp_0.0-sanitized/ Load from ground-truth from /home/kamil/.cache/evalplus/2670b22d020cab45ede5e8285730b6ee.pkl Reading samples... 164it [00:00, 580.47it/s] 100%|████████████████████████████████████████████████████████████████████████████████████████████████████| 164/164 [00:01<00:00, 131.68it/s] Base {'pass@1': 0.27439024390243905} Base + Extra {'pass@1': 0.2073170731707317}
that's from me

I'm able to get a slightly better results using a prompt (since zephyr is a chat model from my understanding): https://github.com/evalplus/evalplus/commit/d553815ecbdff731a8754cf6e206976ae78293ad

Updated to the leaderboard. Thanks!

Sorry somehow I messed up the code where the results are actually coming from a non-chat version (https://github.com/evalplus/evalplus/blob/d553815ecbdff731a8754cf6e206976ae78293ad/codegen/model.py#L280).
However, if I use the chatting mode, it seems the performance is not very promising since the generated format seems to be uncontrollable.


Change edit_uri default branch to v0.x.x in mkdocs.yml

Looks like I missed something, will have a look later.

Did you use the command to update the golden tests or did you update them manually?

I did it manually.

BTW, if this is too distracting for you, I can finish the PR myself, thanks for going straight to a PR instead of an issue!

If you can quickly fix it would be highly appreciated :)

Force-merging, as I only re-generated the golden tests (and applied my own suggestion :smile: )


Upgrade bootstrap version to 4.3.1

With bootstrap version 3.3.7, the following vulnerabilities have been reported by RetireJS during the scan performed on Identity Server,
XSS in data-template, data-content and data-title properties of tooltip/popover
XSS in data-target property of scrollspy
XSS in collapse data-parent attribute
XSS in data-container property of tooltip
The above vulnerabilities reported at the following scripts have been eliminated by upgrading to the latest bootstrap version which is 4.3.1
<IS_HOME>/repository/deployment/server/jaggeryapps/dashboard/js/bootstrap.min.js

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


browser: cleanup

This still needs a lot of work.
I still don't know exactly what this function is meant to be doing,  I'll continue cleaning it up and review against the DH documentation.
WIP PR do not merge.

I need to lint my code damn

@dcousens ping on this?

@calvinmetcalf still going, just had Australia day weekend away.

@dcousens sorry missed your last comment, I added 2 more commits to this pull, one cleaned up the indentations, spaceing and the function names we use in dh.js and the second fixes an api bug where I wasn't returning this to allow chaining and I also gave returnValue a better name

@calvinmetcalf we still have a lot to do, but I think this is a good first pass.

ok I can publish


[sophus] build failure

Host Environment

OS: MacOS Big Sur
Compiler: Clang 12

To Reproduce
./vcpkg install sophus
Failure logs
-- Building x64-osx-dbg
CMake Error at scripts/cmake/vcpkg_execute_build_process.cmake:145 (message):
Command failed: /Users/manu/Bin/vcpkg/downloads/tools/cmake-3.19.2-osx/cmake-3.19.2-macos-universal/CMake.app/Contents/bin/cmake --build . --config Debug --target install -- -v -j17
Working Directory: /Users/manu/Bin/vcpkg/buildtrees/sophus/x64-osx-dbg
See logs for more information:
/Users/manu/Bin/vcpkg/buildtrees/sophus/install-x64-osx-dbg-out.log
From install-x64-osx-dbg-out.log, the error appears to be a parameter that are unused
install-x64-osx-dbg-out.log:
[1/17] /Library/Developer/CommandLineTools/usr/bin/c++  -I/Users/manu/Bin/vcpkg/installed/x64-osx/include/eigen3 -I/Users/manu/Bin/vcpkg/buildtrees/sophus/src/v1.0.0-8c41b1d2f0.clean -Wall -Werror -Wextra -Wno-deprecated-register -std=c++14 -stdlib=libc++ -Wno-deprecated-register -O0 -g -arch x86_64 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk -std=gnu++14 -MD -MT test/core/CMakeFiles/test_common.dir/test_common.cpp.o -MF test/core/CMakeFiles/test_common.dir/test_common.cpp.o.d -o test/core/CMakeFiles/test_common.dir/test_common.cpp.o -c /Users/manu/Bin/vcpkg/buildtrees/sophus/src/v1.0.0-8c41b1d2f0.clean/test/core/test_common.cpp
[2/17] : && /Library/Developer/CommandLineTools/usr/bin/c++ -Wall -Werror -Wextra -Wno-deprecated-register -std=c++14 -stdlib=libc++ -Wno-deprecated-register -O0 -g -arch x86_64 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk -Wl,-search_paths_first -Wl,-headerpad_max_install_names  test/core/CMakeFiles/test_common.dir/test_common.cpp.o -o test/core/test_common   && :
[3/17] /Library/Developer/CommandLineTools/usr/bin/c++ -DGFLAGS_IS_A_DLL=0 -I/Users/manu/Bin/vcpkg/installed/x64-osx/include -I/usr/local/include/eigen3 -I/Users/manu/Bin/vcpkg/installed/x64-osx/include/eigen3 -I/Users/manu/Bin/vcpkg/buildtrees/sophus/src/v1.0.0-8c41b1d2f0.clean -isystem /usr/local/include -Wall -Werror -Wextra -Wno-deprecated-register -std=c++14 -stdlib=libc++ -Wno-deprecated-register -O0 -g -arch x86_64 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk -std=gnu++14 -MD -MT test/ceres/CMakeFiles/test_ceres_se3.dir/test_ceres_se3.cpp.o -MF test/ceres/CMakeFiles/test_ceres_se3.dir/test_ceres_se3.cpp.o.d -o test/ceres/CMakeFiles/test_ceres_se3.dir/test_ceres_se3.cpp.o -c /Users/manu/Bin/vcpkg/buildtrees/sophus/src/v1.0.0-8c41b1d2f0.clean/test/ceres/test_ceres_se3.cpp
FAILED: test/ceres/CMakeFiles/test_ceres_se3.dir/test_ceres_se3.cpp.o
/Library/Developer/CommandLineTools/usr/bin/c++ -DGFLAGS_IS_A_DLL=0 -I/Users/manu/Bin/vcpkg/installed/x64-osx/include -I/usr/local/include/eigen3 -I/Users/manu/Bin/vcpkg/installed/x64-osx/include/eigen3 -I/Users/manu/Bin/vcpkg/buildtrees/sophus/src/v1.0.0-8c41b1d2f0.clean -isystem /usr/local/include -Wall -Werror -Wextra -Wno-deprecated-register -std=c++14 -stdlib=libc++ -Wno-deprecated-register -O0 -g -arch x86_64 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk -std=gnu++14 -MD -MT test/ceres/CMakeFiles/test_ceres_se3.dir/test_ceres_se3.cpp.o -MF test/ceres/CMakeFiles/test_ceres_se3.dir/test_ceres_se3.cpp.o.d -o test/ceres/CMakeFiles/test_ceres_se3.dir/test_ceres_se3.cpp.o -c /Users/manu/Bin/vcpkg/buildtrees/sophus/src/v1.0.0-8c41b1d2f0.clean/test/ceres/test_ceres_se3.cpp
In file included from /Users/manu/Bin/vcpkg/buildtrees/sophus/src/v1.0.0-8c41b1d2f0.clean/test/ceres/test_ceres_se3.cpp:1:
In file included from /Users/manu/Bin/vcpkg/installed/x64-osx/include/ceres/ceres.h:37:
In file included from /Users/manu/Bin/vcpkg/installed/x64-osx/include/ceres/autodiff_cost_function.h:130:
/Users/manu/Bin/vcpkg/installed/x64-osx/include/ceres/internal/autodiff.h:201:30: error: unused parameter 'src' [-Werror,-Wunused-parameter]
static void Apply(const T* src, JetT* dst) {}
^
/Users/manu/Bin/vcpkg/installed/x64-osx/include/ceres/internal/autodiff.h:201:41: error: unused parameter 'dst' [-Werror,-Wunused-parameter]
static void Apply(const T* src, JetT* dst) {}
^
2 errors generated.
Additional context
the compilation works in windows 10

it will be awesome if you can upgrade to the version 1.1.0 as well !

it will be awesome if you can upgrade to the version 1.1.0 as well !

ok fixed! the issue was that I have ceres 1.14 that was installed in /usr/local. Upgrading to ceres with brew solved the problem.
I will close this one and open another ticket for upgrading to 1.1.0

ok fixed! the issue was that I have ceres 1.14 that was installed in /usr/local. Upgrading to ceres with brew solved the problem.
I will close this one and open another ticket for upgrading to 1.1.0


⚠ Bofrost.es has degraded performance

In d00671f, Bofrost.es (https://www.bofrost.es) experienced degraded performance:

HTTP code: 200
Response time: 1502 ms

Resolved: Bofrost.es performance has improved in 1c17c96 after 23 minutes.


CausalELM.jl Registration

@JuliaRegistrator register()

Registration pull request created: JuliaRegistries/General/77545
After the above pull request is merged, it is recommended that a tag is created on this repository for the registered package version.
This will be done automatically if the Julia TagBot GitHub Action is installed, or can be done manually through the github interface, or via:
git tag -a v0.1.0 -m "<description of version>" 7c95ec730c65ff2fd725f97ed186a991c5df7f0e
git push origin v0.1.0

@JuliaRegistrator register

Error while trying to register: "Tag with name v0.1.0 already exists and points to a different commit"


Improve search listing

It looks like we have more meetup groups now than the search listing shows, i.e. cannot find the new Elm Brazil group:
https://meetdown.app/search/brazil
https://meetdown.app/group/a3bb9a/Elm-Meetup-Brazil
So either we need to show more results, or actually implement search :D

There isn't a limit on the number of search results. The reason that group isn't appearing in the group search is because the owner has marked it as a private.
That said, I'll leave this issue open since improving the group search is still something that should eventually be done.


(Bug)Minted with whitelist: incorrect initialization of min/max

Test URL : localhost with latest commit of branch 2.0 : https://github.com/poanetwork/token-wizard/commit/962130704e30e1a8b49bd97bc385f8f231b5639f
Steps:

Create MintedCapped crowdsale , see parameters below.
Open https://ropsten.etherscan.io/address/0x4feec2b6944e510eef031d96330cb70f9051c440#readContract . and execute  getWhitelistStatus  for whitelisted address

Expected result:

max_spend_remaining   uint256 :<PHONE_NUMBER>00000000000

Actual result:

max_spend_remaining   uint256 :<PHONE_NUMBER>000000

As a  whitelisted investor buy minCap
As a whitelisted investor buy maxCap
Result: disabled to buy maxCap, warning is dispaled that hardCap already reched
But whitelisted investor able to buy maxCap in first transaction

Exec ID: 0xbee52e230da6d4654e0312ef85079076faf999490e95cf75d7009f6ae6d8dd29
Ropsten
Crowdsale parameters:
{
  "name": "scenarioE2eMintedWhitelist",
  "ticker": "test",
  "decimals": 18,
  "reservedTokens":[
    {
      "address" :"0xecDFec71AbCD5a612eAf9169B19E907eAA2717d9",
      "dimension": "percentage",
      "value": 250
    },
    {
      "address" :  "0x56B2e3C3cFf7f3921Dc2e0F8B8e20d1eEc29216b",
      "dimension": "tokens",
      "value":100000000
    }
  ],
  "walletAddress":"0x56B2e3C3cFf7f3921Dc2e0F8B8e20d1eEc29216b",
  "gasprice": 1,
  "mincap": 1,
  "tiers":[
    {
      "name":"###1",
      "allowModify": true,
      "isWhitelisted":true,
      "startDate": "360000",
      "startTime": "360000",
      "endDate":"23/02/2050",
      "endTime":"12:34",
      "rate":20000,
      "supply": 250,
      "whitelist":[
        {
          "address": "0xF16AB2EA0a7F7B28C267cbA3Ed211Ea5c6e27411",
          "min": 50,
          "max": 150
        },
        {
          "address": "0x9E963042D581D262AdE4E31541360e7FDAeE70C6",
          "min": 50,
          "max": 150
        }
      ]
    },
    {
      "name":"###2",
      "allowModify": true,
      "isWhitelisted":true,
      "startDate": "",
      "startTime": "",
      "endDate":"",
      "endTime":"",
      "rate":10000,
      "supply": 200,
      "whitelist":[
        {
          "address": "0xdDdC96314b7f16cB243Cb07e1dE081CA367095E7",
          "min": 25,
          "max": 250
        }]}]}

If you are reporting a problem with Token Wizard, please include the following information:
Which network did you use? (Mainnet, Kovan, Rinkeby, etc.)
Ropsten
If you were able to create it, what is the URL of your crowdsale?
http://localhost:3000/invest?exec-id=0xbee52e230da6d4654e0312ef85079076faf999490e95cf75d7009f6ae6d8dd29&networkID=3
Do you have screenshots showing the problem?

Do you see errors in the dev console? If yes, please include a screenshot
No

@dennis00010011b
minimum_contribution is in tokens (decimals are accounted)
but max_spend_remaining is in wei in Auth_os

Works as expected


Merge 'Fix templateId Property Duplicating Issue' to 6.0.x

[Auto generated]
This PR merges the PR #4507.
Link : https://github.com/wso2/product-is/actions/runs/4615437099

This is a test PR


New image is not recognized after changing the .src property and waiting for onload

I encountered the following when working on a project that replaces pictures dynamically using javascript. The attached html and js code demonstrate the problem, but require two pictures named pic/pic03.jpg and pic/pic04.jpg .
It appears that exif-js only recognizes a changed picture if the image element is created from scratch. Replacing the .src property is not enough. The issue can be tested by changing the if(0) to a if(1) in line 19 of exif_bug.js. Click on the picture to change the picture. With 1 as argument the EXIF timestamp (DateTimeOriginal) of the picture stays the one of the original picture, but with 0 the changed image is found by exif.js and the new timestamp is shown correctly.
exif_bug.html.txt
exif_bug.js.txt

This is because exif-js attaches an exifdata property to the Image element on first execution. When getData is called with the same Image element, it checks if the exifdata property is present and reuses that instead of reprocessing the image, even if the image source has changed. See here.
One way to work around this is to clear the exifdata property before calling getData:
delete img.exifdata
It would probably be a good idea to add a parameter to getData that prevents the caching behavior.

This workaround seems to be working for me....thanks!


Added back type: 'local' to TechDocs config schema

Hey, I just made a Pull Request!
Added back type: 'local' to TechDocs config schema for publisher.
Tested this change using the app-config.yaml in this repo as it has the config that would fail:
https://github.com/backstage/backstage/blob/eada5c4f2279e062027c7a6e6fabcfdf9398b35d/app-config.yaml#L151-L158'
Using this command: yarn backstage-cli config:check --lax --strict --config app-config.yaml
Closes #26014
:heavy_check_mark: Checklist

[x] A changeset describing the change and affected packages. (more info)
[ ] Added or updated documentation
[ ] Tests for new functionality and regression tests for bug fixes
[ ] Screenshots attached (for UI changes)
[x] All your commits have a Signed-off-by line in the message. (more info)

Ah my bad, I didn't notice that the key itself was removed - appreciate the fix @awanlin!

No worries @johnphilip283 👍


add hade/stce/pbmte bits to menvcfg

Add hade/stce/pbmte bits to menvcfg
Add menvcfgh with hade/stce/pbmte bits.

#137 needs to be part of this PR

I closed the other PR #137 and moved the contents to this PR. Also as you might notice the AR changed the extension name yesterday from Svadu to Svhad.

AR changed it back to Svadu.


"Sending search request"

Hi,
My smart home actually started with Yeelight lights, and I was disappointed that HomeKit wasn't supported out-of-the-box. Enter HOOBS and and your plugin :)
I do have a question though, I am getting a "Sending search request" message in the log every 15s. I've tried looking through the readme for a FAQ, but couldn't find any.
Is it normal that this appears every 15s?
Thanks!

Same, wondering if this is normal.. i dont remember seeing this before.
@vieira hope you can shed some yeelight.

@joshua1000 @gilshahar7 Hello, this should only happen until all known lights are found. If you have any of them powered down it continues to search until the bulb gets power and is reconnected.
From my testing, without this "active" search, it would take some time for a bulb to be available (broadcast itself) if it was turned off (either by accident or intentionally) and then turned on.

Strange. I've moved the two color E27s behind the TV (not as nice as LED strips but works, will upgrade at some point) and the strip is behind my desk shining up the wall (if that makes sense).  I'm not sure what it's searching for... Any ideas what to look for?

6/23/2020, 11:56:17 PM [Yeelight] Received advertisement from 5533d 7.6/23/2020, 11:56:20 PM [Yeelight] Received advertisement from 5533d 7.6/23/2020, 11:56:23 PM [Yeelight] Received advertisement from 5533d 7.6/24/2020, 12:01:26 AM [Yeelight] Received advertisement from 9c2525. 6/24/2020, 12:01:32 AM [Yeelight] Received advertisement from 9c2525. 6/24/2020, 12:01:38 AM [Yeelight] Received advertisement from 9c2525. 6/24/2020, 12:40:20 AM [Yeelight] Received advertisement from 9c2621. 6/24/2020, 12:40:26 AM [Yeelight] Received advertisement from 9c2621. 6/24/2020, 12:40:32 AM [Yeelight] Received advertisement from 9c2621. 6/24/2020, 12:56:18 AM [Yeelight] Received advertisement from 5533d7. 6/24/2020, 12:56:21 AM [Yeelight] Received advertisement from 5533d7. 6/24/2020, 12:56:24 AM [Yeelight] Received advertisement from 5533d7. 6/24/2020, 1:56:18 AM [Yeelight] Received advertisement from 5533d7. 6/24/2020, 1:56:21 AM [Yeelight] Received advertisement from 5533d7. 6/24/2020, 1:56:24 AM [Yeelight] Received advertisement from 5533d7. 6/24/2020, 2:01:29 AM [Yeelight] Received advertisement from 9c2525. 6/24/2020, 2:01:35 AM [Yeelight] Received advertisement from 9c2525. 6/24/2020, 2:01:41 AM [Yeelight] Received advertisement from 9c2525. 6/24/2020, 2:40:24 AM [Yeelight] Received advertisement from 9c2621. 6/24/2020, 2:40:30 AM [Yeelight] Received advertisement from 9c2621. 6/24/2020, 2:40:36 AM [Yeelight] Received advertisement from 9c2621. 6/24/2020, 2:56:19 AM [Yeelight] Received advertisement from 5533d7. 6/24/2020, 2:56:22 AM [Yeelight] Received advertisement from 5533d7. 6/24/2020, 2:56:25 AM [Yeelight] Received advertisement from 5533d7. 6/24/2020, 3:56:20 AM [Yeelight] Received advertisement from 5533d7. 6/24/2020, 3:56:23 AM [Yeelight] Received advertisement from 5533d7. 6/24/2020, 3:56:26 AM [Yeelight] Received advertisement from 5533d7. 6/24/2020, 4:01:29 AM [Yeelight] Received advertisement from 9c2525. 6/24/2020, 4:01:35 AM [Yeelight] Received advertisement from 9c2525. 6/24/2020, 4:01:41 AM [Yeelight] Received advertisement from 9c2525. 6/24/2020, 4:40:24 AM [Yeelight] Received advertisement from 9c2621. 6/24/2020, 4:40:30 AM [Yeelight] Received advertisement from 9c2621. 6/24/2020, 4:40:36 AM [Yeelight] Received advertisement from 9c2621. 6/24/2020, 4:56:20 AM [Yeelight] Received advertisement from 5533d7. 6/24/2020, 4:56:23 AM [Yeelight] Received advertisement from 5533d7. 6/24/2020, 4:56:26 AM [Yeelight] Received advertisement from 5533d7. 6/24/2020, 5:56:20 AM [Yeelight] Received advertisement from 5533d7. 6/24/2020, 5:56:23 AM [Yeelight] Received advertisement from 5533d7. 6/24/2020, 5:56:26 AM [Yeelight] Received advertisement from 5533d7. 6/24/2020, 6:01:29 AM [Yeelight] Received advertisement from 9c2525. 6/24/2020, 6:01:35 AM [Yeelight] Received advertisement from 9c2525. 6/24/2020, 6:01:41 AM [Yeelight] Received advertisement from 9c2525. 6/24/2020, 6:40:24 AM [Yeelight] Received advertisement from 9c2621. 6/24/2020, 6:40:30 AM [Yeelight] Received advertisement from 9c2621. 6/24/2020, 6:40:36 AM [Yeelight] Received advertisement from 9c2621. 6/24/2020, 6:56:20 AM [Yeelight] Received advertisement from 5533d7. 6/24/2020, 6:56:23 AM [Yeelight] Received advertisement from 5533d7. 6/24/2020, 6:56:26 AM [Yeelight] Received advertisement from 5533d7. 6/24/2020, 7:56:21 AM [Yeelight] Received advertisement from 5533d7. 6/24/2020, 7:56:24 AM [Yeelight] Received advertisement from 5533d7. 6/24/2020, 7:56:27 AM [Yeelight] Received advertisement from 5533d7. 6/24/2020, 8:01:28 AM [Yeelight] Received advertisement from 9c2525. 6/24/2020, 8:01:34 AM [Yeelight] Received advertisement from 9c2525. 6/24/2020, 8:01:40 AM [Yeelight] Received advertisement from 9c2525.
Right now the issue appears to be solved, HOOBS is not looking for the lights anymore. Instead, the light appear to be announcing themselves... See enclosed log from last night. I really don't know what to make of this. Maybe I should just get rid of these and replace them with newer, HomeKit compatible models or abandon Xiaomi altogether...

Hello @joshua1000, that's normal, the bulbs periodically broadcast their status. What's your concern?

Hi, thanks for answering.
It just seems strange to have the log spammed with light announcing themselves...
What has started to happen recently is that the homebridge service within hoobs randomly stops. I haven't figured out why yet.
I have three plugins, yours, LGTV WebOS and Daikin BRP WiFi controller plugins and have started running with only the LGTV plugin, which appears to have helped but I need more time to be sure. This would mean that it's either the lights or the A/C unit (or their respective plugins) that are causing outages. I though the constant announcements could be flooding the service (just an uneducated guess).
I've also noticed a much faster response from HOOBS, having uninstalled the Yeelight and LGTV plugins.
So, as this may or may not have anything to do with your plugin, I will keep testing and get back to you if I find anything...
Thanks for now!

@vieira I have been having similar issues in HOOBS recently, and the only plugin I use is yours. My HOOBS crashes after a few hours, possibly due to the broadcasting of devices.
The search feature stops if all known devices are found and connected; otherwise, you will send a broadcast message every 15 seconds.
The thing is, I have a few lights between an intelligent switch that cuts the power, and for those lights, I only use your plugin for the adaptive lighting feature.
Would you be open to a pull request where I make the polling time a configurable value so that I can set it to every 5 minutes, for example, and optionally turn off the logging for the search feature?
I fully understand that devices can take a while to start working, but that is acceptable for my use case.

I did some research today, and I noticed my HOOBS crashes with the read-only error when it reaches 1MB in file size.
I could work around it by removing the log file after starting HOOBS. It doesn't recreate it and stop logging anything, but it doesn't crash anymore.

@renelux you should probably use logrotate or something equivalent to rotate those old logs before they reach the threshold.
I can change some the things the plugin logs to debug level to reduce the noise a bit but without rotating the logs it will eventually reach 1mb and crash.

@renelux you should probably use logrotate or something equivalent to rotate those old logs before they reach the threshold.
I can change some the things the plugin logs to debug level to reduce the noise a bit but without rotating the logs it will eventually reach 1mb and crash.

isn't it possible to add boolean option for debugging mode or something like this ? (default disabled)
will help maintain cleaner logs ..


Conflict of file names on Windows due to its case-insensitivity

If module declares two symbols equal in letters, but different in case, one of .html files generated by ddox will replace another.
Example:
struct Case;
void case();

Will generate the only Case.html (or case.html)

There is --file-name-style=lowerUnderscored (or lowerCase, upperCase or upperUnderscore) to work around that. On NTFS file systems it is in theory possible to use a case sensitive mode, but the --file-name-style solution is the only one that works for all file systems in general, such as FAT (it's really a file system issue and not an OS issue).

See also #91 - there is a warning message on Windows and OS X now.


@jimp/cli

What's Changing and Why
Makes jimp into a CLI where you can apply any image manipulation functions and even load custom plugins and types
todo:

[x] load images for blit, composite, mask
[x] set up bin
[x] create

closes #606
What else might be affected
nothing
Tasks

[x] Add tests
[x] Update Documentation
[x] Update jimp.d.ts
[x] Add SemVer Label

@edi9999 what do you think about this? anything I should add or change?


Add intent helpers for stopping services.

Anko has helper functions for starting services but doesn't have helper functions for stopping them. I added the necessary functions to handle this use case.

Thanks for the contribution!


Revert "Roll engine to b64c88766dbd83bf4bab1cd1cd1757139b6cdb74."

Reverts flutter/flutter#8124
Reason: performance regressions, see https://github.com/flutter/flutter/issues/8188.

Thanks.

After talking we decided to abandon the roll back.


C++ SDK: Define new Abstracted Data Models & L2/L1 interfaces

The following issue is to track two changes to the implementations:

New Data model Interface:
The existing solution has developers coding to an implementation of a uProtocol message (CloudEvent) that works for device-2-cloud and vice versa but doesn't scale for embedded platforms. The following changes will create an interface for the datamodel (called UMessage) that will allow greater flexibility to developers to support multiple forms of representing the data & metadata.

uBus & uTransport APIs:
After implementing the bus and transports more than twice, we have seen the need to define an interface that is common to all  SDKs and implementations (Android, Linux, Cloud, mobile, etc...).

Replaced with the 1.5 issue


Not showing product video on the product page on the shop.

Terms

[X] Before you openning this issue, i have checked if the issue has already been reported.

Bagisto Version(s) affected
master
Issue Description
If the video is added to the product, and trying to play view it is showing blank on the product page.
Preconditions

Admin should be logged in.
Admin should have at least one created product with added video.

Steps to reproduce

Go to 'Shop'
Click on 'that product'
Hover to video
See the error

Expected Result
Should be played video.
Actual Result
Showing blank page instead of playing video. Here
Screenshots
No response
Additional context
No response

Hi @Amitk-webkul
This issue is already opened and fixed previously
Can you please check once??

Now, this issue is fixed. so I am closing this issue.
And I have attached a screenshot Here, for future reference.


The problems from my 1+ year experience with CLS

I'm writing it as a note to myself and maybe other guys who plan/use CLS (continuation-local-storage).
I met several practical problems with it. But my another issue hangs about half-year+ without an answer (not even sure if it's actual any more, after AL updates), so I'm not asking anything from devs. Anyway I'm not using CLS for anything serious any more.
Just noting the things I met for a year+ of trying to make it work (with occasional success), so that people know the difficulties ahead, cause some of them are rather hidden.
The patching problem:

There are many modules which do not support CLS.  CLS just doesn't work with them or works with serous bugs (ye I know why).
Most modules can be monkeypatched to support CLS. But that's only possible if the needed functionality is exported, otherwise I need to fork/really patch it.
In practice the monkeypatch tends to break on new major versions of the modules. Then repatching takes time to dive deep into the updated module once again.

The private versions patching problem:

Even if I manage to patch something, like bluebird v3, there's a chance that a 3rd-party module in my app uses it's own version of bluebird, say v2, deep inside it's node_modules hierarchy.
So I must watch not only over modules I need, but also track any module in the node_modules tree of my app, to see if there's a private version of a CLS-unfriendly module, and patch that. Every module/submodule/subsubmodule/... must support CLS or be patched for it.
That greatly extends the area of maintenance I need to do to keep everything CLS'able. Also gives so much space for bugs in case if something updates and I miss it.

The dangerous bugs problem:

The bugs introduced by a missed CLS-unfriendly updates of a module or it's submodule may be subtle yet really destructive and hard to fix back. Imagine a payment going to a wrong account. Or a regular user getting an admin view of page with all the secret information.
That prevents relying on CLS even if though mostly works.

I admire the idea behind the module/async-listener. Async chains in Node.JS are like green threads. A context for an async chain is so cool, it's a must. It's like thread local variables, widely used by so many systems.
Maybe there's something that can be done in the node core to make it work everywhere, make every module CLS'able without critical perf loss?
It would be great to get more attention to the module and its functionality from the minds of the community.

All of the above exactly mirrors my experiences from using CLS for about a year. We still use it, as I have not found any other way to have "thread local storage" in Node. I admire CLS for coming up with a solution that mostly works.
Coming to Node with a background in Java, Python and Ruby, the lack of async chains was the biggest surprise. I still wonder how people properly track transactions, authentication and other contextual data through a large system.

As of Node 6, Trevor Norris's AsyncWrap API is a documented, stable part of the core Node API. It would be excellent to port CLS to take advantage of AsyncWrap when it's available, as I believe that it addresses a number of the points raised above (and Trevor is a much better steward of AsyncWrap than I've been able to be for async-listener, which is after all only my backport of Trevor's previous sketch at this kind of API). It would also be excellent to see CLS extended to directly incorporate all of the known shims for packages like Bluebird that require a little extra help to work with CLS.
If anyone felt like taking on that work, I would be happy to add them to the repo and list of owners on npm, because it's quite clear to me that my other commitments don't afford me the time to take on that work, which is pretty time-consuming and finicky.

AsyncWrap is indeed a good alternative to async-listener, but it doesn't provide enough functionality to cover the continuation-local-storage use cases.
OP mentioned 'the patching problem'. I call it the 'user space queuing problem' (more details here). It stems from the fact that any user space code (JS or native) could queue callbacks can resume them in a context different. The status quo here has been to discover such code, and get them monkey patch them to ensure they continue working. This is a never ending battle.
IMO a longer term solution would be to

Get the concept of 'async context' into the language and/or host environment.
Have the module authors (e.g. bluebird) uphold the contract with the language, by ensuring their modules propagate the async context.

Ideally we want this to work with no monkey patching at all.
Authors of bluebird have indicated that they have little interest in propagating context for continuation-local-storage; they do propagate the current domain, however, since domains are part of core. This is an understandable position, and bluebird is not the only module with this disposition.
In my opinion, to get a solution that is better than 'mostly works', we need:

AsyncWrap in node.js.
Zones in TC39. This brings in the concept of async context into the language.

I think cls should remains as a basic component to deal with async contexts/zones.
Should we propose a new repo to implement APIs compatible to Zone in TC39?

I don't think CLS would ever change its API to match zones. Likely a new polyfill would implement the zones API and people would just start using that.
It's possible CLS might start using a zones polyfill internally though, at some point. Currently it uses an async-listener polyfill, and there's some effort happening to upgrade to an async_wrap polyfill.

What's the point of using CLS API if Zone from TC39 has already provided a
similar set of APIs?
On Tue, Aug 23, 2016 at 6:21 AM, Stephen Belanger<EMAIL_ADDRESS>wrote:

I don't think CLS would ever change its API to match zones. Likely a new
polyfill would implement the zones API and people would just start using
that.
It's possible CLS might start using a zones polyfill internally though, at
some point. Currently it uses an async-listener polyfill, and there's some
effort happening to upgrade to an async_wrap polyfill.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/othiym23/node-continuation-local-storage/issues/59#issuecomment-241569490,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAU6FT80f9AUCg8_YlQTb9DFRp_qLvsiks5qiiDigaJpZM4HZ9ff
.

Zones are lower level. CLS is purely just meant to be a data structure to store data scoped to the execution context, and likely it'd remain as just that.
If someone wants deeper control of how the async contexts are linked, they can reach for whatever Zones API polyfill surfaces.

As of 2018, with node 8.0 async hook features, wonder if CLS implementation still the best way for implementing async context in node.js, if yes, do you guys have any better suggestion? Thank you very much and appreciate your help!

@ckitt I think the plan is for CLS to be rewritten to utilise Async Hooks.
cls-hooked module claims to do exactly this, but I don't know how stable it is at present.
Also, Async Hooks has been going through some changes to tackle problems that became evident at its first release, particularly as it relates to Promises. Again, I can't tell you where this work is up to - I'm out of date - and whether Async Hooks is really ready for prime time at this point.

The overall tracking issue for any changes to async-hooks is at https://github.com/nodejs/diagnostics/issues/124.
For what it is worth we use cls-hooked on top of async-hooks in production without any real issues. It generally just works. I am pretty sure there is nothing else out there that is any better, and to be honest I also think it is good enough in the vast majority of cases.

Thanks for the prompt answer! @holm @overlookmotel
I am asking because we hit some strange problem when using cls under node 8.x environment (not sure if this is the root cause), the context is not removed properly after the execution ended, which causes cls to return the other execution context from other async execution (this is very serious in our use cases as we use cls for storing permission information of user). After studying the source code from cls, we suspect that this related to bugs in async-listener, thus we are asking if there's any plan for cls to be integrated with Async Hook.

I would definitely recommend using cls-hooked. It has much less quirks than this original module, since it requires much less monkey patching of other libraries. Before we had to monkey-patch many libraries to get the context updated at all times.
With cls-hooked using async-hooks you get pretty much everything working out of the box, you only really need to monkey-patch promise libraries.

I'm the author of the original issue.
Is cls-hooked totally free of the described problems, and it provides "async-local variables" just like CLS?

Yep. It’s the same module, they just pulled out all the monkey-patching and async-listener parts and replaced them with async-hooks. The user-facing surface of the module is the same. The only difference being that this module only works on older versions of Node.js while cls-hooked only works on newer versions.

@qard that's really super great news, updating the readme would be cool if you can.

probably a lot of issues could be closed here too, with the same link and caveat re needing a newer node.

@holm and others who use cls-hooked in production.
After a close look, I really see no difference between this module and cls-hooked. The internals are different: hook instead of monkey-patching, the problems are the same.
Mongoose, bluebird - exactly same failures.
Or please explain :)

@iliakan I don't know about Mongoose, but I can explain the failure with bluebird.
Bluebird internally does it's own queueing which by default doesn't work with async_hooks (or indeed async-listener, which is why it didn't work with this old version of CLS). Bluebird needs to be patched to support async_hooks. There's an issue open on bluebird repo for this, but I don't think it's been done yet.
You can patch bluebird using cls-bluebird module to play nice with CLS. That module has been tested with old CLS, but not cls-hooked, so I can't guarantee it'll work but in principle it should (I wrote the current version).
The other alternative is to use native JS Promises - those support async_hooks.
Likely Mongoose gets CLS contexts mixed up for same reasons as bluebird - it does some internal queuing. Since async_hooks is part of Node core, Mongoose should support it - I'd suggest opening an issue for that if there isn't already. This will be affecting not just CLS, but other uses of async_hooks e.g. tracing.
Hope that helps somewhat.

Yes, that’s exactly the same issue that I was talking about in the beginning of the thread.

Thanks for the prompt answer! @holm @overlookmotel
I am asking because we hit some strange problem when using cls under node 8.x environment (not sure if this is the root cause), the context is not removed properly after the execution ended, which causes cls to return the other execution context from other async execution (this is very serious in our use cases as we use cls for storing permission information of user). After studying the source code from cls, we suspect that this related to bugs in async-listener, thus we are asking if there's any plan for cls to be integrated with Async Hook.

I still see this issue in cls hooked ("^4.2.2"). Generally the context of previous api call appears. It is like it holding some reference to the old context and it is there till it is not garbage collected (Speculating -as it suddenly changes to some new value). @ckitt were you able to figure out why it is really happening?

For mongoose, see https://github.com/mongoosejs/mongoose-async-hooks


Fix missing last frame of MPEG-TS content

Issue
With TS contents, any codec, the last access unit is never passed on to the input sample queue. The reason is that the end of the sample is currently triggered by the payload_unit_start_indicator (pusi) of the first TS packet of the following access unit, which doesn't exist for the very last access unit. As a result the playback of any TS content always misses the last frame.
For frame accurate applications this seems a non negligible issue.
This problem was already discussed here https://github.com/google/ExoPlayer/issues/7909 and that ticket is still open.
Proposed solution
In our understanding, for a file based case, it is reasonable to assume that the end of stream is a valid end of the last access unit, provided all the accumulated TS packets of the access unit have correct TS header parameters such as transport_error_indicator (=0) and continuity_counter.
This pull request comes from a fix we implemented in our LCEVC-Exo fork. The idea is that on end of stream:

TsExtractor sends an empty data payload with flagged pusi to PesReader;
PesReader recognises the end of input case from the empty payload and calls a modified packetFinished() with boolean endOfInput parameter to the ES reader;
The ES reader calls a new SampleReader method called end() that flushes the accumulated access unit;

Thanks for the contribution! It looks like something went wrong with your PR setup as it includes thousands of commits (mostly by our team). Could you try to rebase/clean this up?

Sorry, had wrongly left the base branch set to release :( should be fixed now.

Please sign the CLA and we can take a look.

Ready for review, sorry for the delay.

I found that lots of tests in the extractor.ts package fail with these changes.
I was able to fix many of them by updating the golden test files by temporarily setting DumpFileAsserts.DUMP_FILE_ACTION to WRITE_TO_LOCAL (and this is expected, since we expect to see more samples at the end of playback), but there are still a lot of failures (not due to golden test file mismatch). Please could you try running these tests and investigate the failures? They are mostly of this form:
Index 0 out of bounds for length 0
java.lang.ArrayIndexOutOfBoundsException: Index 0 out of bounds for length 0
	at androidx.media3.common.util.ParsableByteArray.readUnsignedByte(ParsableByteArray.java:260)
	at androidx.media3.extractor.ts.SectionReader.consume(SectionReader.java:71)
	at androidx.media3.extractor.ts.TsExtractor.read(TsExtractor.java:331)
	at androidx.media3.test.utils.ExtractorAsserts.consumeTestData(ExtractorAsserts.java:529)
	at androidx.media3.test.utils.ExtractorAsserts.consumeTestData(ExtractorAsserts.java:510)
	at androidx.media3.test.utils.ExtractorAsserts.assertOutput(ExtractorAsserts.java:452)
	at androidx.media3.test.utils.ExtractorAsserts.assertBehavior(ExtractorAsserts.java:401)
	at androidx.media3.extractor.ts.TsExtractorTest.sampleWithAit(TsExtractorTest.java:114)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)

Also: I tried to push my commit updating the golden test files, but was unable to. Is it possible that you need to allow permission to me to push commits to this branch? https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/allowing-changes-to-a-pull-request-branch-created-from-a-fork
We will likely need to push more changes during internal review as well, so please check that access has been granted. Thanks!

@icbaker you should have access now. Let me check the tests.

I received an invite to collaborate on https://github.com/v-novaltd/androidx-media which I've accepted, but I'm still not able to push commits to this PR. I did some experimentation with https://github.com/androidx/media/pull/443 and I see the same problem (can't push changes from androidx back to my icbaker personal fork) if I untick the 'allow edits by maintainers' on the PR itself.

I can't see this tickbox on this PR, I think only you can as the creator of the PR. Can you please make sure it's ticked? Thanks!
GitHub docs on this: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/allowing-changes-to-a-pull-request-branch-created-from-a-fork

Please also take a look at the couple of comments I added in the code itself.

Ah, it's possible you can't tick this box this because v-novaltd is a GitHub organisation rather than an individual user: https://github.com/orgs/community/discussions/5634
And the recommended workaround in https://github.com/orgs/community/discussions/5634#discussioncomment-2155236 is exactly what you've done.
Let me have a play around and see if I can work with the permission you've granted me. Sorry for being a bit unfamiliar here, we've recently changed the way we handle PRs to a process that requires pushing our own commits to the PR before merging it (to avoid 'evil merges') - you can see the full discussion about that here: https://github.com/google/ExoPlayer/commit/e8117496e0e8bb0761a4d3f387786be9a23c3002#commitcomment-107715150

I'm afraid I don't see your comments yet.

Sorry, I didn't realise they were stuck in 'pending' state - they should be published now I think.

Thanks for the fixes!
I'm going to send this for internal review now. You may see some more commits being added as I make changes in response to review feedback. Please refrain from pushing any more substantive changes as it will complicate the internal review - thanks!


Feature Request: Add static option to run jamf policy after update has applied

Add the option to run a jamf policy after the update has applied.
Such as or example
--postupdatereboot-runjamfpolicy
Adds into the configuration
<key>RunJamfPolicyAfterUpdateReboot</key>
<true/>
Which runs any policy with the Custom Trigger of "SupermanRunAfterUpdateReboot" after the machine has rebooted. Assuming there are policies set with that custom trigger.
Customers would only need to put policies in with that trigger.

This would be a good addition... but more likely to add in v4.0 timeframe.

I've decided to re-organize the workflow so all non-restart updates are going to happen after the restart as a default behavior... so you got luck and this one is moving to v3.0.