Terragrunt cannot assume role (using Atlantis)

TL;DR;
IAM role assume works perfectly fine from a shell inside docker. It works also using an aws cli command inside Atlantis. But it throws an error when I want to use Terragrunt "way" of assuming - --terragrunt-iam-role & TERRAGRUNT_IAM_ROLE env.
Details:
In my current project, I had to incorporate Terragrunt into Atlantis (I've done this before many times using K8s and working on multi-account architecture) using ECS on AWS. To be short in this introduction - I already have ECS (Fargate) hosting my docker, but unfortunately seems that I'm struggling with issues that Terragrunt cannot assume my IAM roles.
I start my Fargate task with the task execution role and task role attached. Then by the built-in feature in Terragrunt for role-assume, I would like to switch my role and context into another AWS account.
IAM role for ECS task (let's called: task_role or source role) has attached permissions to assume next role (let's called it automation_role_account_B or just target role):
arn:aws:iam::12345:role/task_role
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "sts:AssumeRole"
            ],
            "Resource": [
                "arn:aws:iam::123456:role/automation_role_account_B"
            ]
        }
    ]
}

Where, a "target role" of course has adjusted trust relationship policy to allow being assumed be previous one:
arn:aws:iam::123456:role/automation_role_account_B
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::12345:role/task-role"
      },
      "Action": "sts:AssumeRole"
    }
  ]

From AWS perspective everything looks fine....
Now fun starts as I build my docker with Atlantis + AWS CLI + Terragrunt etc.
Inside Atlantis I have the opportunity to execute custom commands/workflows where I use Terragrunt:
workflows:
  terragrunt-test:
    plan:
      steps:
      - env:
          name: TERRAGRUNT_IAM_ROLE
          value: arn:aws:iam::123456:role/automation_role_account_B
      - env:
          name: TF_CLI_ARGS
          value: "-no-color"
      - env:
          name: TERRAGRUNT_TFPATH
          command: 'echo "/home/atlantis/.atlantis/bin/terraform$ATLANTIS_TERRAFORM_VERSION"'
      - run: aws sts get-caller-identity
      - run: terragrunt plan -no-color

I'm using in my use case variable TERRAGRUNT_IAM_ROLE to provide an IAM role I would like to assume, but the same (unsuccessful) results I have got when I used a dedicated terragrung flag: terragrunt plan --terragrunt-iam-role ...
The error that I have got clear claims about 403 error (access denied):
[terragrunt] 2021/11/17 23:45:02 Terraform version: 0.14.9
[terragrunt] 2021/11/17 23:45:02 Reading Terragrunt config file at /home/atlantis/.atlantis/repos/test/we-devops/atlantis-example/3/default/terragrunt/my-test/eu-west-1/apps/test-app/terragrunt.hcl
[terragrunt] 2021/11/17 23:45:02 Running command: bash /home/atlantis/.atlantis/repos/test/we-devops/atlantis-example/3/default/terragrunt/../util/get_user_env.sh my-test/eu-west-1/apps/test-app
my-test[terragrunt] 2021/11/17 23:45:02 run_cmd output: [account-test]
[terragrunt] 2021/11/17 23:45:02 Assuming IAM role arn:aws:iam::123456:role/automation_role_account_B
[terragrunt] 2021/11/17 23:45:03 AccessDenied: User: arn:aws:sts::12345:assumed-role/task-role/decf22cd78b34e1ab9472d19e06d1a2f is not authorized to perform: sts:AssumeRole on resource: arn:aws:iam::123456:role/automation_role_account_B
	status code: 403, request id: 613250cc-58e1-4fd8-a76f-64230b004050
[terragrunt] 2021/11/17 23:45:03 Unable to determine underlying exit code, so Terragrunt will exit with error code 1

{
    "UserId": "AROARLQNH54TED56DS6AX:decf22cd78b34e1ab9472d19e06d1a2f",
    "Account": "12345",
    "Arn": "arn:aws:sts::12345:assumed-role/task-role/decf22cd78b34e1ab9472d19e06d1a2f"
}

But.... when I assume a role any other way, it works!.
I can for instance add one more line to Atlantis like this:
- run: aws sts assume-role --role-arn arn:aws:iam::123456:role/automation_role_account_B --role-session-name cs
and this will work.
I connected to the Fargate container using EXEC functionality (https://aws.amazon.com/blogs/containers/new-using-amazon-ecs-exec-access-your-containers-fargate-ec2/) and get access to an interactive shell inside my container, where again I was able just successfully assume my role:
aws sts get-caller-identity

aws sts assume-role --role-arn arn:aws:iam::123456:role/automation_role_account_B --role-session-name cs

Completly don't get it :/
Versions of applications inside my docker:
PYTHON_VERSION=3.8.7-r1
AWS_CLI_VERSION=2.0.30
TERRAGRUNT_VERSION=v0.27.0 (but I have testes as well on the latest 0.35.10, and exactly the same error)
TERRAFORM: 0.14.9

I observe the exact same behaviour when running terragrunt in a gitlab runner on an ec2 instance.
The ec2 instance role has permission to assume a role in another account but terragrunt fails with the same type of 403 while attempting to assume the cross account role in the AWS terraform provider.
In the same gitlab job running on the instance the AWS cli can assume the role using aws sts assume-role.
Because I am not using iam_role or --terrangrunt-iam-role but instead assuming the role with the terraform provider it seems like the issue may also exist in terraform. Or possibly the AWS Go SDK since both terraform and terragrunt use that.

Does the role have restrictions on any of the following?

Session duration. Terragrunt defaults to 3600 if the session duration isn't provided (https://terragrunt.gruntwork.io/docs/reference/cli-options/#terragrunt-iam-assume-role-duration)
Session name. Terragrunt will not configure a role session name if it isn't provided (https://terragrunt.gruntwork.io/docs/reference/cli-options/#terragrunt-iam-assume-role-session-name)

I'm seeing this too, i believe its this block here https://github.com/gruntwork-io/terragrunt/blob/300e1e2a9dca27f524ff4bca9e6b665d8e4fa538/aws_helper/config.go#L128 the env vars attempt is made here: https://github.com/gruntwork-io/terragrunt/blob/300e1e2a9dca27f524ff4bca9e6b665d8e4fa538/aws_helper/config.go#L138
We are trying to chain assume roles but it breaks because the environment vars from the first assume role need to happen then a 2nd assume role.

I encountered this issue when working with older infrastructure (deployed ~12 months ago on tf 0.13 + terragrunt 0.25) and while I can't guarantee this is all strictly necessary, what finally worked for me was:

reverting both terraform and terragrunt to the particular versions initially deployed with
setting version  = "3.47.0" in my aws provider block (deprecated but it works in .13)
terragrunt init
rm .terraform/plugins/registry.terraform.io/-/aws
cp .terraform/plugins/registry.terraform.io/hashicorp/aws .terraform/plugins/registry.terraform.io/-/aws

Those last couple of steps were because I could not figure out how to force init to download an older plugin for -/aws which is what tf .13 actually kept trying to use. I don't believe it's relevant for later versions of terraform.
Anyway. After all that, I was able to plan, apply, etc without the constant 403s. Unfortunately my scenario is specific enough and my solution broad/inelegant enough that it may be of no value in either identifying the source of this problem or helping anyone else work around their own issues.

@jvanceACX im not sure its the same issue, the block i highlighted is from terragrunt which seems to at least be where my issue is from. It doesnt even make it down to terraform before assuming the wrong role.

Hello. I just wanted to leave a note that we stumbled upon what seems to be the same issue as described here.
We can't run terragrunt from GH actions when using a special github IAM role that is assumed via OIDC.
IAM role chaining happens and terraform/terragrunt can't assume the correct role. When I removed the AWS credentials and token environment variables and added static credential files - everything started to work.
To me it looks like it might be a terraform issue, rather than a terragrunt issue. Because the errors where present even when testing with terraform (in the debug environment).

@juozasget did you ever manage to resolve this for OIDC runs? I'm having the same issue I think...

@jhrr Hey! Late reply - but no, we just continued to use static credentials to workaround the issue. By now we are using IAM roles via OIDC + Identity Center for human users with dynamic credentials. So it would be nice to get this working with dynamic credentials also :)


ImportError: cannot import name 'linux' from 'pyffmpeg.static.bin.linux'

Hi there! I just want to try out your package, but it didn't work for me.
OS: Ubuntu 20.04
Python: 3.10.3
Logs:
    ff = FFmpeg()
  File "/home/wsontopski/.cache/pypoetry/virtualenvs/maddox-E1xGxQ_g-py3.10/lib/python3.10/site-packages/pyffmpeg/__init__.py", line 52, in __init__
    self._ffmpeg_file = Paths().load_ffmpeg_bin()
  File "/home/wsontopski/.cache/pypoetry/virtualenvs/maddox-E1xGxQ_g-py3.10/lib/python3.10/site-packages/pyffmpeg/misc.py", line 58, in load_ffmpeg_bin
    from .static.bin.linux import linux
ImportError: cannot import name 'linux' from 'pyffmpeg.static.bin.linux'

@LostInDarkMath Hi, the tests have passed, I hope it is working by now

Thanks for your fix!


allow custom format

Please follow the guide below

You will be asked some questions and requested to provide some information, please read them carefully and answer honestly
Put an x into all the boxes [ ] relevant to your issue (like that [x])
Use Preview tab to see how your issue will actually look like

WARNING
All invalid issues will be rejected!!

Before going further

If your problem is a bug with youtube-dl or a request for new site support please report it here

Make sure you are using the latest youtube-dl-gui version (Click the Settings icon and then About to view the current version)

Make sure you are using the latest youtube-dl version (Click the Settings icon and then Update to update to the latest youtube-dl version)

Make sure you searched the bugtracker for similar issues including closed ones

Make sure to read the FAQs file

[x ] I think my problem is NOT with youtube-dl
[ x] I've verified and i assure that I'm running youtube-dl-gui 0.4
[ x] I assure that i am using the latest version of youtube-dl
[X ] Searched bugtracker
[ X] I've read the FAQs file

What is the purpose of your issue?

[ ] Bug report
[ X] Feature request (request for a new functionality)
[ ] Question
[ ] Other

Please remove any sections between (---) if they are not related to your issue

Feature request (request for a new functionality)
Please make sure that the requested feature is NOT already in the TODO list

[ X] I've verified and i assure that my requested feature is NOT in the TODO list

maybe related to "Mix formats option" ?
What operating system do you use ?
windows10

We need possibility to use format  such as  -f "bestvideo[height<=?480]+bestaudio[ext=m4a]/best[height<=480]"
instead of just
because sometimes there is no high resolution video but split video/audio instead,
That would solve the MP3 problem too where it sometimes download 2GB  4K  video to strip 60MB of audio content, when m4a file is available, to avoid that the Mp3 format should use  :
-f "bestaudio[ext=m4a]/best[height<=480]"
to limit  to best audio with m4a format (so can be converted to mp3) and if not available, limit the max video size to be downloaded (only care here is the audio,  no need to download HUGE 2160p video one !)
maybe it is already supported , but I can't find any INI file or customization of the command line

This project is abandoned. There is a fork here, but I don't if it supports the feature you want.
My own fork  enables you to add any command-line options you like: click Edit > General download options, then add something to the Extra command line options box, for example -f "bestaudio[ext=m4a]/best[height<=480]"

Thank you so much for your prompt reply !


Refactor sharpness

Part of #2279.
Changes:

Made sharpness_image private (_sharpness_image now), as sharpness already handles single images and _equalize_image is private as well.
Made sharpness compatible with grayscale images (previously only worked with RGB).
Added the name argument and applied a tf.name_scope.
Decorated with @tf.function. Now runs ~60% faster. (colab)
Removed the clip_by_value invocation since the sharpening kernel is already normalized.
Refactored the tests. sharpness and equalize now both handle all input shapes.

Also removed a faulty test for equalize. That test was for the now unsupported channels_first and was passing because it was just renaming the name_scope to channels_first :D.

@abhichou4
You are owner of some files modified in this pull request.
Would you kindly review the changes whenever you have the time to?
Thank you very much.

@aaronmondal can you make the colab link public to "anyone who knows the link can access it"? Other than this, the PR LGTM.

Link: https://colab.research.google.com/drive/1uZzz4G-Bfme1-9jaBsMPYTzykA25GsTu?usp=sharing
Sorry I always forget that the default link is not public :O


[patch] EHealth-1730: improve test for termination declaration

Coverage remained the same at 90.853% when pulling e27c93e44aa4ceabf0c4bb189cf280ef531adcda on 1730-test into 9866a725de2c334f0de4d5b44bf561da518e2e11 on master.


Validate date properly

What version of Ajv are you using? Does the issue happen if you use the latest version?
I am using AJV version 6.9.1. I can also see this issue in version 6.0.0
JSON Schema
{
    "properties": {
        "birth-date": {
            "type": "string",
            "format": "date"
        }
    }
}

Sample data
{
    "birth-date": "2019-13-39"
}

Your code
var Ajv = require('ajv');
var ajv = new Ajv({
    allErrors: true
})
var validate = ajv.compile(schema);
var valid = validate(data);
if (!valid) console.log(validate.errors);

Validation result, data AFTER validation, error messages
No Error. Which is wrong because "2019-13-42" is not a valid date. Month can't be 13 and day can't be 39.

What results did you expect?
{
    "keyword": "format",
    "dataPath": "['birth-date']",
    "schemaPath": "#/properties/birth-date/format",
    "params": { "format": "date" },
    "message": "should match format date"
}

Duplicate of
https://github.com/epoberezkin/ajv/issues/221


TypeORMLegacyAdapter: Client does not support authentication protocol requested by server; consider upgrading MySQL client

:wave:
The TypeORM adapter doesn't currently work with MySQL 8, it throws with the following error: "Client does not support authentication protocol requested by server; consider upgrading MySQL client".
TypeORM uses the mysql package when it's available, and unfortunately this doesn't support the default authentication method of MySQL 8. The alternative package mysql2 works with MySQL 8, and TypeORM can use that package as well, but when both packages are installed it defaults to mysql because of this switch. The TypeORM adapter currently installs mysql as a dependency which is why things are breaking even if the user has manually installed mysql2.
https://github.com/typeorm/typeorm/issues/2093#issuecomment-706692887 for reference.

Strange, we run our typeorm mysql tests with the mysql:8 container and they've been passing. Are you using any other different settings for the DB or special authentication settings?

Nope, just the default MySQL 8 auth mode. The problem should be mysql: https://github.com/mysqljs/mysql/issues/1507, https://github.com/mysqljs/mysql/pull/2233, and many other issues on that repo.


Search measure error fixes

fix error when duty_expression is not present for measure_component.
fixed explicit end_date in measure search.

 [DIT] Search function not throwing accurate results.

 [DIT] Search: end date filters for before/after should also include undefined (3)

 [DIT] Search is returning with zero results for an existing commodity code which has got measure associated with it (5)

 [DIT] Change measures search to be able to search by multiple measure IDs - comma separated (2)


Seemingly accident update to the April 28th release

RVC-Beta.7z on the hugging face page was accidentally updated by mistake yesterday morning central time. (In the timezone where most of the developers live, I'm thinking this was ~9 or 10pm roughly I believe?) In addition to this, the updated20230428.zip file was silently updated and replaced. This makes me think it wasn't intentional, as it still reads april 28th in the filename, and no new release tag was properly created. The RVC-beta.7z that was updated also fails to properly launch go-web.bat, and the updated20230428.zip has similar problems (complaints about a missing english translation on the newest commit as of the accident?)
To remedy this for the people using Google Colab, I made my own fork that has a backup download of the updated20230428.zip:
https://github.com/kalomaze/Retrieval-based-Voice-Conversion-WebUI/releases/tag/20230428updated
If this wasn't a mistake, or I am misunderstanding the situation, please let me know. I also have a backup of the RVC-beta.7z as it was before on this google drive:
https://drive.google.com/file/d/1JQ0i16m7WqN76rqpp2THgNMcLDuJFuny/view?usp=sharing

I have fixed it.


HDFS-17499. removeQueuedBlock in PendingDataNodeMessages has memory leak

Description of PR
Introduced by HDFS-17477. The PendingDataNodeMessages#removeQueuedBlock() will create an empty list stored in queueByBlockId for every incremental block report processed by the BlockManager.
How was this patch tested?
Tested with TestLargeBlockReport#testBlockReportSucceedsWithLargerLengthLimit()
For code changes:

[X] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
[ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
[ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
[ ] If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

@ayushtkn @goiri Please take a look.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 49s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

-1 :x:
test4tests
0m  0s

The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
50m 20s

trunk passed

+1 :green_heart:
compile
1m 22s

trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1

+1 :green_heart:
compile
1m 14s

trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

+1 :green_heart:
checkstyle
1m 14s

trunk passed

+1 :green_heart:
mvnsite
1m 22s

trunk passed

+1 :green_heart:
javadoc
1m 10s

trunk passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1

+1 :green_heart:
javadoc
1m 42s

trunk passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

+1 :green_heart:
spotbugs
3m 18s

trunk passed

+1 :green_heart:
shadedclient
42m 12s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
1m 13s

the patch passed

+1 :green_heart:
compile
1m 16s

the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1

+1 :green_heart:
javac
1m 16s

the patch passed

+1 :green_heart:
compile
1m  6s

the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

+1 :green_heart:
javac
1m  6s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
1m  2s

the patch passed

+1 :green_heart:
mvnsite
1m 13s

the patch passed

+1 :green_heart:
javadoc
0m 57s

the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1

+1 :green_heart:
javadoc
1m 32s

the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

+1 :green_heart:
spotbugs
3m 20s

the patch passed

+1 :green_heart:
shadedclient
43m 52s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
269m 43s

hadoop-hdfs in the patch passed.

+1 :green_heart:
asflicense
0m 42s

The patch does not generate ASF License warnings.

432m  2s

Subsystem
Report/Notes

Docker
ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6772/1/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/6772

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets

uname
Linux 3232c21f6f76 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 72e8113586364b18a8b536aa4062f2129b324236

Default Java
Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6772/1/testReport/

Max. process+thread count
2487 (vs. ulimit of 5500)

modules
C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6772/1/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

the commit misses Jira id:
https://github.com/apache/hadoop/commit/027b4c32590b7156442d7642240c4c830ee4f1f3


Increase cache size

Codecov Report
Base: 100.00% // Head: 100.00% // No change to project coverage :thumbsup:

Coverage data is based on head (58959a2) compared to base (b1125d1).
 Consider uploading reports for the commit d86118a to get more accurate results

Additional details and impacted files
@@            Coverage Diff            @@
##              main      #207   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files            2         2
  Lines           39        39
=========================================
  Hits            39        39

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


Add typescript support

related to #456 .
add basic ts support and rewrite utils to ts as module with minimal deps

seams file middleware.ts (js) not used in main code at all, or i missed something?

Hi there,
I appreciate the help, but I'm unwilling to switch the implementation language. You're free to use any language you want for language bindings and custom protocol implementations.


Missing file in insert_editor_js breaks page editor

Issue Summary
If a non-existent file is referenced from an insert_editor_js hook then django won't render a large part of the page editor template at all. There's no indication of what caused this thanks to django suppressing exceptions from template includes.
Steps to Reproduce

reference a non-existent file in an insert_editor_js hook:

# wagtail_hooks.py
from django.contrib.staticfiles.templatetags.staticfiles import static
from django.utils.safestring import mark_safe
from wagtail.wagtailcore import hooks

@hooks.register('insert_editor_js')
def editor_js():
    return mark_safe('<script src="%s"></script>' % static('js/does-not-exist.js'))

set DEBUG=False
run collectstatic
try to edit a page: there will be lots of javascript errors on the console and e.g. image choosers won't work

More details
When developing locally with DEBUG=True then django.contrib.staticfiles.templatetags.staticfiles.static won't raise an exception for missing files, instead it will return the path even if the file doesn't exist. This causes no other issues at all, it will be just a 404 later on.
However with DEBUG=False and using the manifest storage django.contrib.staticfiles.templatetags.staticfiles.static does raise an exception via {% hook_output 'insert_editor_js' %}: https://github.com/wagtail/wagtail/blob/master/wagtail/admin/templates/wagtailadmin/pages/_editor_js.html#L28
Which is currently ignored by django if DEBUG=False but at least this behaviour will change in django 2.1: https://docs.djangoproject.com/en/1.11/ref/templates/builtins/#include
Moreover, since hook_output is in a template include this exception will turn the whole _editor_js.html template into an empty string, not just the hook_output tag, omitting 10 more javascript files that are required for the page editor to work.
Maybe this doesn't affect many people and it really isn't Wagtail's fault but it isn't obvious at all why suddenly the page editor is broken when there are no exceptions raised with either DEBUG=True/False. I'm not sure if there's a good fix for this, or even if it's worth addressing it until django 2.1 but at least it might come in handy for others having the same issue.
Technical details

Django version: 1.11.5
Wagtail version: 1.13.1

@balazs-endresz thanks for this issue and putting in some awesome detail.
Not sure on the solution here but wanted to add a note about what the Django docs state:

Deprecated since version 1.11:
Silencing exceptions raised while rendering the {% include %} template tag is deprecated. In Django 2.1, the exception will be raised.

Without some kind of exception to catch, I am not sure how we could check if the static files exist when the files are built up.

We could add a new include which contains only {% hook_output 'insert_editor_js' %}, so if there's a missing file then at least the rest of _editor_js.html wouldn't disappear. I can add a PR if you think it's worth addressing it.

@balazs-endresz
That sounds like an awesome solution, that way if the include fails it will only fail for content from the hooks.
Would love a PR if you can, it will need to preserve any ordering that already happens.
Can you also revise the  implementation of the other similar hooks.

insert_editor_css
insert_editor_js
insert_global_admin_css
Insert_global_admin_js

Any other thoughts @gasman ?

I've added tests but also realised that an exception will still break all the other e.g. insert_editor_js hooks if only one of them raises an exception because they are all evaluated at the same place: https://github.com/wagtail/wagtail/blob/db6d36845f3f2c5d7009a22421c2efab9968aa24/wagtail/admin/templatetags/wagtailadmin_tags.py#L171
So, I'm not convinced a partial fix like this is all that helpful.

@balazs-endresz I think that will still be helpful though, it will break all hooks but at least that is easier to diagnose and won't break all css or js.
Also, with Django 2.0 - it will be easy to see where the error is coming from when errors are thrown, it will report that the hooks template is broken.

I think there's a strong case for saying: if it's going to break, better for it to break in a major conspicuous way, rather than causing subtle breakage on a few plugins that might sneak unnoticed onto production.

Yep, let's leave this to be fixed at the Django end.

I don't mind leaving it as it is either but it's going to sneak into production anyway thanks to the serve() function not failing locally when DEBUG = True and the includes will be fixed only in Django 2.1 not 2.0. Not changing this at least means that it's easier to notice that something has gone wrong.


Documentation search

Small head's up on the documentation:
Searching for "placeLimit.." and clicking to on the placeLimitOrder method shows me a 404 (whereas it is still available in Gitlab pages)

Yep, still working out the new docs here :)

The Github CDN is just updating the content of the docs. Should be all sorted out by itself in a few hours at latest. The content is in the gh-pages branch, but not yet deployed fully on their CDN.
In the meantime you could generate the docs locally with yarn doc

Ah ok sure, my bad!

Seems to be a jekyll issue, that gh pages doesn't automatically work with underscores in filenames.
Should be fixed with https://github.com/nash-io/api-client-typescript/pull/11

works now


Binding hiredis client to a VRF device

I'm trying to port the hiredis client onto a box running Linux 4.14 kernel that supports multiple VRFs. The Redis server may be reachable through any of the configured VRFs. I was wondering if there is a possibility of adding a new option to redis(Async)ConnectWithOption()  calls to bind the TCP socket to a specific VRF device (using SO_BINDTODEVICE). Note that the source IP does not work for me since the IP address can be reused across VRFs on the same box.
Thanks
Ref: https://www.kernel.org/doc/Documentation/networking/vrf.txt

Sorry about the delay, I'm going through old issues.
It's certainly possible to add another flag.  I suppose in this case it would have to be a flag accompanied with associated data (for the device name).  Another option is to allow you to configure a callback for setting socket options in whichever way you want.

Either option sounds fine. Thanks for taking a look into this.
Cheers    On Saturday, July 11, 2020, 01:47:19 AM GMT+5:30, Michael Grunder<EMAIL_ADDRESS>wrote:
Sorry about the delay, I'm going through old issues.
It's certainly possible to add another flag. I suppose in this case it would have to be a flag accompanied with associated data (for the device name). Another option is to allow you to configure a callback for setting socket options in whichever way you want.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

I've been going through all our old issues in preparation for the upcoming release and had time to look at this.
Since we define _XOPEN_SOURCE the only feasible way to add the functionality is to allow users to set a custom callback that we invoke after a successful call to socket.
A small example on how to use it to set SO_BINDTODEVICE


Microsoft Graph returning null account even after passing a valid account ID

Core Library
MSAL Node (@azure/msal-node)
Core Library Version
4.46.1
Wrapper Library
Not Applicable
Wrapper Library Version
NA
Public or Confidential Client?
Confidential
Description
I am encountering a weird issue with Microsoft Graph on an integration that was built a few years back.
This issue started happening a few months back. After I sync a Microsoft Account and provide email and calendar read/write access, everything works fine for some time. I am able to retrieve emails and calendar events. However, after some time, I notice that when a call is made to GetAccountAsync with a valid AccountID, null is returned. This is causing AcquireTokenSilent to fail with the following error:
Error Code: user_null
Error Message: No account or login hint was passed to the AcquireTokenSilent call.
I have also noticed that this happens under the following scenarios:

When the WebJob (console app) is run every 15 minutes, I encounter this issue
To narrow down the root cause, I have deleted the WebJob to see if the issue occurs on the web app. It looks like the issue starts to occur after an hour or so even without the web job running.

I have upgraded to the latest version of MSAL and implemented <IP_ADDRESS> version of Microsoft.Identity.Client. I am using .NET Framework 4.8 and this is a .NET MVC 5 app.
Error Message
Error Code: user_null
Error Message: No account or login hint was passed to the AcquireTokenSilent call.
I have also noticed that this happens under the following scenarios:
Msal Logs
No response
MSAL Configuration
NA

Relevant Code Snippets
public async Task<string> GetAccessTokenAsync()
{
    string accessToken;
    UserExternalApp.Scope = string.IsNullOrWhiteSpace(UserExternalApp.Scope) ? "" : UserExternalApp.Scope;

    // Load the app config from web.config
    var microsoftScopes = UserExternalApp.Scope.Replace(' ', ',').SplitAndTrim(new char[] { ',' }).ToList();
    var accountID = UserExternalApp.ExternalUserAccountID;

    var app = ConfidentialClientApplicationBuilder.Create(ClientID)
        .WithRedirectUri(DefaultRedirectUrl) // https:\//mywebsite.com
        .WithClientSecret(Secret)
        .Build();

    app.AddDistributedTokenCache(services =>
    {
        services.AddDistributedSqlServerCache(options =>
        {
            options.ConnectionString = System.Configuration.ConfigurationManager.ConnectionStrings["Connection"].ConnectionString;
            options.SchemaName = "dbo";
            options.TableName = "TokenCache";
            options.DefaultSlidingExpiration = TimeSpan.FromMinutes(90);
        });
    });

    try
    {
        var account = await app.GetAccountAsync(accountID);
        var query = app.AcquireTokenSilent(microsoftScopes, account); // This is where the error is thrown
        var acquireTokenSilent = await query.ExecuteAsync();

        accessToken = acquireTokenSilent.AccessToken;
    }
    catch
    {
        // This is the error thrown:
        // Exception Type: MsalUiRequiredException
        // Error code: user_null
        // Exception Details: No account or login hint was passed to the AcquireTokenSilent call.
        throw;
    }
    return accessToken;
}

Reproduction Steps

Log in to Web app and sync Microsoft account
Authentication is performed and tokens are saved in the database
Check Outlook email from the web app. This step is successful
Come back after an hour or so and try to check Outlook email through the web app
Error is encountered.

Expected Behavior
Email should be retrieved successfully after a few hours
Identity Provider
Azure AD / MSA
Browsers Affected (Select all that apply)
None (Server)
Regression
No response
Source
External (Customer)

Sorry, I think I posted this under the wrong repo.


singularity: 4.2.1 -> 4.2.2

Things done
Diff: https://github.com/sylabs/singularity/compare/v4.2.1...v4.2.2
Changelog: https://github.com/sylabs/singularity/releases/tag/v4.2.2
cc @ShamrockLee

Built on platform(s)

[x] x86_64-linux
[x] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[x] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
25.05 Release Notes (or backporting 24.11 and 25.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

The updated singularity executable fails to run the image it builds, but the one on release-24.11 does run the new image.
nixpkgs_factor-lang on  build-python-structured-attrs [$] took 1m54s
❯ nix run github:NixOS/nixpkgs/refs/pull/366871/merge#singularity -- exec $(nix eval github:NixOS/nixpkgs/refs/pull/366871/merge#singularity.tests.image-hello-cowsay.outPath) hello | nix run github:NixOS/nixpkgs/refs/pull/366871/merge#singularity -- exec $(nix eval github:NixOS/nixpkgs/refs/pull/366871/merge#singularity.tests.image-hello-cowsay.outPath) cowsay
error (ignored): error: SQLite database '/home/shamrock/.cache/nix/eval-cache-v5/c0fae9bbfb61f1149c57e9549d0683e7adeafbf6619efe24b5595636f6153c32.sqlite' is busy
error (ignored): error: SQLite database '/home/shamrock/.cache/nix/eval-cache-v5/c0fae9bbfb61f1149c57e9549d0683e7adeafbf6619efe24b5595636f6153c32.sqlite' is busy
FATAL:   While checking image: could not open image /home/shamrock/Projects/NixOS/nixpkgs_factor-lang/"/nix/store/zigigs3gzxq04fgickkr3m0y8qlh4j8p-singularity-image-hello-cowsay.sif": failed to retrieve path for /home/shamrock/Projects/NixOS/nixpkgs_factor-lang/"/nix/store/zigigs3gzxq04fgickkr3m0y8qlh4j8p-singularity-image-hello-cowsay.sif": lstat /home/shamrock/Projects/NixOS/nixpkgs_factor-lang/": no such file or directory
FATAL:   While checking image: could not open image /home/shamrock/Projects/NixOS/nixpkgs_factor-lang/"/nix/store/zigigs3gzxq04fgickkr3m0y8qlh4j8p-singularity-image-hello-cowsay.sif": failed to retrieve path for /home/shamrock/Projects/NixOS/nixpkgs_factor-lang/"/nix/store/zigigs3gzxq04fgickkr3m0y8qlh4j8p-singularity-image-hello-cowsay.sif": lstat /home/shamrock/Projects/NixOS/nixpkgs_factor-lang/": no such file or directory

nixpkgs_factor-lang on  build-python-structured-attrs [$]
❯ nix run github:NixOS/nixpkgs/release-24.11#singularity exec /nix/store/zigigs3gzxq04fgickkr3m0y8qlh4j8p-singularity-image-hello-cowsay.sif hello
INFO:    Mounting image with FUSE.
WARNING: passwd file doesn't exist in container, not updating
WARNING: group file doesn't exist in container, not updating
Hello, world!

The updated singularity executable fails to run the image it builds, but the one on release-24.11 does run the new image.

Weird... Why do the tests succeed then ?

Because the tests only include image building (tests.image-hello-cowsay), not image running.

Update:
Sorry for the noise. It's not the issue of Singularity, but with nix eval. SingularityCE runs the image without problems.


[iOS] Apply bottom navigation bar design

Currently, the bottom navigation bar of the iOS app does not follow the design defined in Figma.
Please update it to follow the design.
Design

@shimastripe
Thank you for opening a Pull Request!
I'll assign you 🙏🏼


Fixed #34545 -- Corrected the number of months in installation FAQ.

Fixed : 34545

@Akash-Kumar-Sen Thanks :+1: Welcome aboard :boat:

Thank you @felixxm.


Toggle widgets to set visibility of bonds

References #29

Integrated bond controls (size, color, and visibility) into the UI, and are updated in real time. Created PR.
[t:140]


ENH: Use Fletcher to better support strings, structs, and arrays

I recently learned about Fletcher, which provides a pandas extension dtype that is backed by pyarrow.
Since it's backed by arrow, it avoids having to use the object dtype for strings, structs, and arrays. But also, since it's only 0.1, this should probably be opt-in.
Similar to https://github.com/pydata/pandas-gbq/issues/242, since both these proposals use extension dtypes.

The project was archived earlier this year and is no longer supported (https://github.com/xhochy/fletcher/tree/master) — closing.

Indeed. ArrowDtype is now built-in to Pandas. https://github.com/googleapis/python-bigquery-pandas/issues/621 tracks a way to integrate with ArrowDtype in a way consistent with other pandas I/O methods.


Seems you don't have the required permissions on C:\Users\hp\AppData\Local\Pub\Cache\global_packages\fvm On Windows FVM requires to run as an administrator or turn on developer mode: https://bit.ly/3vxRr2M[BUG]

@Rohit-ChauhanRC can I have the output of fvm doctor?
also run the fvm use command again with "--verbose" flag after it to get all the output


Fix Dockerfile

Installing libspatialindex-dev started failing because of 404, so I added an apt-get update first
The SRTM download failed, because the base image doesn't have curl by default

With these changes, I've successfully run the tool on montlake for all 3 data sources. Analyzing them now.

I spoke too soon -- srtm also requires GDAL:
curl -s -o spool/N47/N47W123.hgt.gz.temp https://s3.amazonaws.com/elevation-tiles-prod/skadi/N47/N47W123.hgt.gz && mv s
pool/N47/N47W123.hgt.gz.temp spool/N47/N47W123.hgt.gz
gunzip spool/N47/N47W123.hgt.gz 2>/dev/null || touch spool/N47/N47W123.hgt
gdal_translate -q -co TILED=YES -co COMPRESS=DEFLATE -co ZLEVEL=9 -co PREDICTOR=2 spool/N47/N47W123.hgt cache/N47/N47W123.tif 2>/dev/null || touch cache/N47/N47W123.tif
rm spool/N47/N47W123.hgt
make: Leaving directory '/root/.cache/elevation/SRTM1'
make: Entering directory '/root/.cache/elevation/SRTM1'
gdalbuildvrt -q -overwrite SRTM1.vrt
make: gdalbuildvrt: Command not found
Makefile:14: recipe for target 'SRTM1.vrt' failed

I tossed gdal-bin in the Docker's apt-get, but that messes up the PROJ version, just like on my local system. I'll find a workaround to get the GDAL binaries onto the image not using apt, likely using the compile instructions you sent earlier.

Just a progress update: I added the gdal install instructions to the end of the Dockerfile:
RUN wget http://download.osgeo.org/gdal/3.2.1/gdal-3.2.1.tar.gz && \
tar -xvzf gdal-3.2.1.tar.gz && \
cd gdal-3.2.1 && \
./configure --with-spatialite && \
make -j8 && \
make install

And it almost worked, failing during linking:
ar: `u' modifier ignored since `D' is the default (see `U')
/bin/sed: can't read /usr/lib/x86_64-linux-gnu/libsqlite3.la: No such file or directory
libtool:   error: '/usr/lib/x86_64-linux-gnu/libsqlite3.la' is not a valid libtool archive
make[1]: *** [libgdal.la] Error 1

I'll keep iterating

In the script that I yanked that GDAL install code from, the following are installed earlier with a comment marking them as PROJ prerequisites:
sudo apt-get install -y sqlite3 libsqlite3-dev libtiff-dev libcurl4-openssl-dev pkg-config
My suspicion is that one or both of the sqlite packages there are needed by GDAL and not actually needed by PROJ, so not already installed in the base Docker image you started from.

Thanks for the hints -- I got the full image to build now! I'll try running it and forcing SRTM tonight.

Is this now running reliably enough that it's worth merging?

Is this now running reliably enough that it's worth merging?

I'll do a fresh build and import data with it now to be certain

Nope. :(  It builds fine, but at runtime: gdalbuildvrt: error while loading shared libraries: libgdal.so.28: cannot open shared object file: No such file or directory

That error message looked familiar, and sure enough https://stackoverflow.com/a/32807686/2121470 is in my browser history.  I haven't actually run into this with Docker/linux, but I think I had to do something similar on my Mac to get an older version of GDAL working.  So if libgdal.so.28 exists somewhere in that Docker image, it may be fixable with either a symlink or a PATH edit.
I wish I understood what the .number at the end signified, given that it never seems to line up with the GDAL version....

Correction: if a file with the exact name exists this should be fixable with a PATH edit.  If libgdal.so or one with a different .number exists, try the symlink.

Well that was silly. I was originally using a base image with proj, but there's also one with gdal installed: https://hub.docker.com/r/osgeo/gdal
After switching to it, I got the SRTM import working end-to-end! The results appear to be breaking the contour visualization, but that's another issue I'm looking into next.


[Laravel 5.4] Missing role_id from user create/edit query

Laravel Version: 5.4.12
Voyager Version: 0.11.0
PHP Version: 5.6.25

Description:
role_id not stored when add/edit the user.
Laravel debugbar shows below query :-
insert into users (name, email, password, avatar, updated_at, created_at) values ('Test user one'<EMAIL_ADDRESS>'***', 'users/default.png', '2017-02-19 13:25:20', '2017-02-19 13:25:20')
Apparently role_id is missing from query. which folder do I need to modify?

What is the user model called?

Sorry, I'm very new to laravel and voyager :( So not sure how to find it but the debugbar shows below. Does it help?
uri : POST admin/users
middleware : web, admin.user
as : voyager.users.store
controller : \TCG\Voyager\Http\Controllers\VoyagerBreadController@store
namespace : App\Http\Controllers
prefix : /admin
where
file : \vendor\tcg\voyager\src\Http\Controllers\VoyagerBreadController.php:202-220

Below query only returns name, email, password, avatar only. so I edit the database data_rows columns add as 1, edit as 1 (default setting was all 0 apart from required) so it seems to be working.
which columns should be 1? brows, read and delete should be 1 as well?
select * from data_rows where data_rows.data_type_id = '3' and data_rows.data_type_id is not null and add = '1'

I think just need to change the file of \vendor\tcg\voyager\publishable\database\seeds\DataRowsTableSeeder.php
But not sure which one should be 1 ??
$dataRow = $this->dataRow($userDataType, 'role_id');
if (!$dataRow->exists) {
$dataRow->fill([
'type'         => 'text',
'display_name' => 'role_id',
'required'     => 1,
'browse'       => 0,
'read'         => 0,
'edit'         => 1,
'add'          => 1,
'delete'       => 0,
'details'      => '',
])->save();
}
This is my first time github comment so not sure how to display coding :( sorry!

I think edit and add should be one.
Does that fix it, then I will make a pull request to fix this.

You just need add permission to Edit the 'role_id' column
go to:
http:// {YOUR DOMAIN} /admin/database/bread/users/edit
and edit checkboxes in below in screenshot

@ArmSALArmy  Cheers, yip it works as well :)

Reference: #799


[BUG] PHP7.3 installation issues.

Describe the bug
New git clone  installation. Rename and update the .env file to use php73, mariadb106
which Branch / PHP Version are you using ?
main
Steps to reproduce
Steps to reproduce the behavior:

Git Clone
rename sample.env to .env
Use php73 as the PHP version, update to use mariadb106 as database
docker compose up -d

Expected behaviour
Successful installation of docker with PHP 73
Screenshots
If applicable, add screenshots to help explain your problem.
 They have been ignored, or old ones used instead.
------
failed to solve: process "/bin/sh -c apt-get -y update --fix-missing &&     apt-get upgrade -y &&     apt-get --no-install-recommends install -y apt-utils &&     rm -rf /var/lib/apt/lists/*" did not complete successfully: exit code: 100

That’s because Debian stretch has been abandoned. Easiest fix is to upgrade to buster. Don’t forget to update the “Install useful tools and install important libaries” section too – see the php74/Dockerfile for inspiration.

Thanks for the answer @bramus
...where was this 6 hours ago!? Haha. I just spent 6 hrs trying to debug this. But at least I learned a lot about docker and sources.list files and such.
I solved it by updating the sources.list file to the archived sources. I didn't want to change the OS version in case any of the subsequent setup/config relied on "stretch".
So, in the php73 Dockerfile in ./bin/php73 I added these lines after FROM php:7.3-apache-stretch :
# Remove the old sources.list file.
RUN rm /etc/apt/sources.list

# Replace with this list.
RUN echo 'deb http://archive.debian.org/debian/ stretch main contrib non-free\ndeb http://archive.debian.org/debian/ stretch-proposed-updates main contrib non-free\ndeb http://archive.debian.org/debian-security stretch/updates main contrib non-free' >> /etc/apt/sources.list

I found these sources from https://www.geekersdigest.com/apt-etc-apt-sources-list-for-every-debian-release/
docker compose works for me now 😊

Thanks both for the answers here @bramus  @sebsebseb123 - I will try both of them and let you know the results.
Before posting this issue, I did start to merge some PHP74/Dockerfile conf in to PHP73/Dockerfile after reading stretch had been abandoned but came across installation errors when running docker compose up -d.
I believe this issue also occurs on PHP72 and possible earlier versions. Would it be a case of using fix but with FROM php:7.2-apache-buster
@bramus @sebsebseb123 with this issue being a core issue within the repo, it might worth creating a pull-request for this fix.

Hi Guys,
thanks for bringing this up. And sorry for the delay. I just added a fix with #248. Please reopen the issue if there is still an issue.


Pretraining Hubert on Music

Hi!
currently, i'm trying to pretrain a hubert base model on music data. The dataset consists of about 1000 Songs of different genres. I followed the instructions on https://github.com/facebookresearch/fairseq/blob/main/examples/hubert/README.md as closely as possible and took the hubert_base_librispeech default as config, except some minor changes concerning the distributed computing stuff.
I'm aware, that the parameters of the default config might not be appropriate for music data. Nonetheless I hoped I could reach a baseline that I could improve further. Sadly I got the following behaviour:

The loss went down from over 10 to about 7 but jumped up to a higher value again, where it stayed. I can tell that it stays there because there was another longer run, that I've deleted already. Does anyone have an Idea where this problem might come from? I included the screenshot of the loss scale, because it looked suspicious
Thanks in advance
Edit: I noticed too late that this is a fairseq related question. I'm closing this issue to reopen it on their site. Sorry for this

Edit: I noticed too late that this is a fairseq related question. I'm closing this issue to reopen it on their site. Sorry for this


Add update notify to master

Add update notify to master
This issue introduced by #13514

@dbw9580 could you help take a look?

alluxio-bot, merge this please


ci: use up to date docker images of core and node

Overview

Resolves #33.
Checklist

[x] New and updated code has appropriate documentation
[x] New and updated code has new and/or updated testing
[x] Required CI checks are passing
[x] Visual proof for any user facing features like CLI or documentation updates
[x] Linked issues closed with keywords

Codecov Report

Merging #34 (e074f0e) into main (4868ef6) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main      #34   +/-   ##
=======================================
  Coverage   54.02%   54.02%
=======================================
  Files           2        2
  Lines          87       87
=======================================
  Hits           47       47
  Misses         34       34
  Partials        6        6

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


UNEXPECTED RESET WHEN BLE SCANNING AND HIGH DENSITY OF DEVICES, HELP... (IDFGH-10241)

Dear Sirs
we have founded a problem that is critical in our product.
Suddlenly we have very often (20 or 30 secons) unexpected resets form the BT library. It occurs when we place our product in a high density bluetooth devices.
We use BT Dual Mode as we need both BT Classic and BLE.
Using at this moment ESP IDf v4.4.4-439-gc25d92074e
We sart and stop BT Classic  Inquirys (3 secs scan) Stop, Wait some seconds an Start again, for looking near devices.
We also launch BLE Scan for close devices.
As said when we place the product in a hign density bluetooh devices we have started to see BT crashes very very often.
There are some logs of the crashes (not always the same message).
E (29892) task_wdt: Task watchdog got triggered. The following tasks did not reset the watchdog in time:
E (29892) task_wdt:  - IDLE (CPU 0)
E (29892) task_wdt: Tasks currently running:
E (29892) task_wdt: CPU 0: BTC_TASK
E (29892) task_wdt: CPU 1: IDLE
E (29892) task_wdt: Aborting.
[0;31mE (24922) BT_OSI: list_append osi_calloc failed.
E (24923) BT_HCI: host_recv_pkt_cb couldn't aquire memory for inbound data buffer.
E (26816) task_wdt: Task watchdog got triggered. The following tasks did not reset the watchdog in time:
E (26816) task_wdt:  - IDLE (CPU 0)
E (26816) task_wdt: Tasks currently running:
E (26816) task_wdt: CPU 0: BTC_TASK
E (26816) task_wdt: CPU 1: IDLE
E (26816) task_wdt: Aborting.
I Attach the files with the crashses.
Its really urgent to find what is happenuing as this is a Stopper for our product release, and we have some thousands working.
01 Error.txt
02 Error.txt
03 Error.txt
01BluError.txt
04 Capture.txt

@andurbal
Apparently the problem is that the chip is running out of memory.

Why does the memory run out, and how much memory is left when the code is initialized? Is there a task with a higher priority than the Bluetooth task working?
BLE duplicate scan can be turned on to check duplicate broadcast packet reporting, which reduces system memory consumption
If the above questions have not solved your problem, please provide the sdkconfig file

Thanks for you suggestions. We have tried all of them.
Finally we have found the problem and it was on our side, due that for some
reason somebody is broadcasting an invalid PDU not following Bluetooth
guidelines  and our PDU parser crashed in a loop not returning to the GAP
event handler.
I cannot share here the invalid PDU but if you are interested I can send a
PM.
Thanks
El vie, 26 may 2023 5:45, weijian_Espressif @.***>
escribió:

@andurbal https://github.com/andurbal
Apparently the problem is that the chip is running out of memory.

Why does the memory run out, and how much memory is left when the
code is initialized? Is there a task with a higher priority than the
Bluetooth task working?
BLE duplicate scan can be turned on to check duplicate broadcast
packet reporting, which reduces system memory consumption
If the above questions have not solved your problem, please provide
the sdkconfig file

—
Reply to this email directly, view it on GitHub
https://github.com/espressif/esp-idf/issues/11504#issuecomment-1563772285,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AS2R42IR4I6JDJRAQFFL2PDXIARONANCNFSM6AAAAAAYOYLLRI
.
You are receiving this because you were mentioned.Message ID:
@.***>

Thanks for sharing the updates. Feel free to reopen.


Be more asynchronous in OpenCL code generator

Currently, Futhark will generate clFinish() calls after most OpenCL operations.  This is useful for debugging, but the real reason is that I do not fully understand the OpenCL dependency chain model.  It would be nice if we could exploit a little more latency hiding (and sometimes we launch several totally independent kernels that could be run concurrently).  Some initial tests showed that this gets us a speedup of more than 10% on LocVolCalib.
I wonder if we can simply use the OpenCL API or if we will need a simple runtime system for managing the dependencies.  I'm fine with either.  Futhark currently employs no intelligence at runtime, so we have lots of room to grow.

I have removed all unnecessary clFinish() calls.  Combined with my recent optimisations to reduction, the performance improvements are so significant that I have a hard time trusting the timings anymore.


Allow selection of Heap or Direct buffers for CommandHandler.buffer

Feature Request
Is your feature request related to a problem? Please describe
On my application during load spikes the command CommandHandler.stack grows to 250ish command and the buffer grows up to 1.5Gb.  (when redis data values being returned are only Kbs in size)
heap buffers being visible in heap dumps, aid in diagnosis of memory issues
Describe the solution you'd like
If you used alloc.buffer(...) instead of alloc.directBuffer(...) it would allow the user to choose between heap or direct buffers with  -Dio.netty.noPreferDirect=true

That's in place now. Care to give Lettuce 5.3.0.BUILD-SNAPSHOT a try? The aggregation buffer in CommandHandler isn't bounded. It could make sense to add an upper bound so that the fill ratio is hit earlier. On the other side, a bound can easily break applications that work with large values.


Make ibctesting.TestCoins be multiple coins

See discussion: https://github.com/cosmos/ibc-go/pull/6517#discussion_r1629638022
Todo after transfer tests have been refactored to not cause too many conflicts.

For Admin Use

[ ] Not duplicate issue
[ ] Appropriate labels applied
[ ] Appropriate contributors tagged/assigned

Can you assign me this issue sir?

Thanks, @NisTun!


Question: How often do I have to change the access token?

Love the plugin. Using it to get specific user's photos and it works great. I was wondering how often I have to switch out the access token though? I used the tool on your site to grab the access token and just pasted it in my js. Does that access token ever have to update?
Thanks for any help.

Short answer
Never (for now)
@stevenschobert nowadays it keeps working of the same way or the access tokens have expiration?


Deprecate config policy provider

What would you like to be added?
Now that we have support for rego policy provider, we should deprecate config policy provider as it is very limited and not the encouraged way to manage policy in ratify moving forward. Policy language providers will be the future supported behavior including CEL support in the future.
I propose the following timeline:

1.2: we announce plans to deprecate the config policy provider. In this release, we will switch default helm chart, config.json, and samples to use the rego policy provider.
1.3 we remove all documentation for it. Add deprecate warning to headers of all files related to it
1.4 we remove implementation completely, remove e2e tests. Remove extra JSON policy config paths we have in executor.go

Anything else you would like to add?
No response
Are you willing to submit PRs to contribute to this feature?

[ ] Yes, I am willing to implement it.

We should hide it/make it invisible/remove docs during 1.x releases and implement removal at the 2.0 release

Create a comprehensive list of impacted resources/touchpoints we'll need to modify as part of the proposal.

We have to continue supporting this until 2.0. Create 2.0 milestone and deprecation announcement.


Weighted Traversals

Hay guys <3 this is my corona open source output! :P
Scope & Purpose
PR adds a mode field to traversal options (deprecating old bfs: true) now allowing following value: dfs (default), bfs and weighted. In weighted mode there is now a weightAttribute and defaultWeight attribute with same semantics as for shortest path traversals. Paths are enumerate by increasing length. Furthermore returned path has a new attribute weights that contains the weights at each individual step.
For example:
for x, v, p in 0..10 outbound "places/York" graph "kShortestPathsGraph" options {mode: "weighted", weightAttribute: "travelTime", uniqueVertices: "path"}
    FILTER p.edges[*].travelTime ALL < 3
    let totalTime = LAST(p.weights)
    FILTER totalTime < 6
    SORT totalTime DESC
    return {path: p.vertices[*]._key, weight: LAST(p.weights), weights: p.edges[*].travelTime}

returns
[
  {
    "path": [
      "York",
      "London",
      "Birmingham",
      "Carlisle"
    ],
    "weight": 5.3,
    "weights": [
      1.8,
      2.5,
      1
    ]
  },
  {
    "path": [
      "York",
      "London",
      "Birmingham"
    ],
    "weight": 4.3,
    "weights": [
      1.8,
      2.5
    ]
  },
  {
    "path": [
      "York",
      "London",
      "Brussels"
    ],
    "weight": 4.3,
    "weights": [
      1.8,
      2.5
    ]
  },
  {
    "path": [
      "York",
      "London"
    ],
    "weight": 1.8,
    "weights": [
      1.8
    ]
  },
  {
    "path": [
      "York"
    ],
    "weight": 0,
    "weights": []
  }
]

I'd like to ask for your feed back on this. Of course I will add tests later. It should work with all other options like uniqueVertices and uniqueEdges. Important case uniqueVertices: "global" <3

[x] Strictly new functionality (i.e. a new feature / new option, no need for porting)
[x] The behavior in this PR can be (and was) manually tested (support / qa / customers can test it)
[x] The behavior change can be verified via automatic tests

Testing & Verification
This PR adds tests that were used to verify all changes:

[ ] Added new integration tests (i.e. in shell_server / shell_server_aql)

Additionally:

[x] I ensured this code runs with ASan / TSan or other static verification tools

Documentation
Will add if you like the change.

[ ] Added a Changelog Entry (referencing the corresponding public or internal issue number)
[ ] Added a new section in the Manual

I fixed pruning so instead of crashing server you get with
for v, e, p in 0..10 outbound "places/York" graph "kShortestPathsGraph"
    prune v._key == @target
    options {mode: "weighted", weightAttribute: "travelTime", uniqueVertices: "path"}
    filter v._key == @target
    return {path: p.vertices[*]._key, weight: LAST(p.weights), weights: p.edges[*].travelTime}

following result

path
weight
weights

["York","Edinburgh","Leuchars","StAndrews"]
7.2
[4,3,0.2]

["York","Carlisle","Glasgow","Edinburgh","Leuchars","StAndrews"]
8.7
[3.5,1,1,3,0.2]

["York","London","Birmingham","Carlisle","Glasgow","Edinburgh","Leuchars","StAndrews"]
10.5
[1.8,2.5,1,1,1,3,0.2]

HI @corona3000
this is awesome!!
Thank you very much for the contribution, this is very appreciated from our side.
From my first rough review this looks very good.
I will do in-depth review tomorrow morning.
Unfortunately for legal reasons we require you to sign the Contributor Agreement
before we can accept your pull requests.
Also we need to include this feature into the SmartGraphs of ArangoDB (enterprise-mode) which is unfortunately closed source before we can release it.
I am happy to keep you updated on the progress there.

@mchacki Happy to hear that you like. I am looking forward on your review. Can you put this code just in open source edition? I guess you need different solution for smart graph anyways? Your open source version has license Apache 2.0 which is fine for me.


⚠ Empornium has degraded performance

In 02e0139, Empornium ($EMP) experienced degraded performance:

HTTP code: 200
Response time: 2512 ms

Resolved: Empornium performance has improved in 57bd6ca.


Extension host terminated unexpectedly. Code:<PHONE_NUMBER>

VSCode Version: 버전 1.20.1
커밋 f88bbf9137d24d36d968ea6b2911786bfe103002
날짜 2018-02-13T15:34:36.336Z
셸 1.7.9
렌더러 58.0.3029.110
노드 7.9.0
아키텍처 x64
OS Version: windows 10

Steps to Reproduce:

run vscode
/C:/Program Files/Microsoft VS Code/resources/app/out/vs/workbench/workbench.main.js:4172 Extension host terminated unexpectedly. Code:<PHONE_NUMBER>  Signal:  null

Does this issue occur when all extensions are disabled?: No I think php and git has problem

Just to confirm, does it reproduce when you run
code --disable-extensions

?

I ran Code with the --disable-extensions switch and I still get the error message. I do not have any folders or files loaded. Just an empty editor on the Welcome page.

So here's an interesting wrinkle: I just rebooted my computer and now I'm not seeing the error. I just installed Code and did not reboot before launching it. Could this be something as silly as pathing information not being in-place from the install? Should VS Code require a system reboot before launching the first time?

@TonganJedi Good to know it works for you.
@qkrtmddml Let me know the answer to my question.

Hi. I have a similar issue. If I install any kind of extension(including official ones), I get this error. I don't get any errors if I run --disable-extensions.

@e1337us3r Try to figure out which extension causes it, by disabling extensions one by one.
@qkrtmddml Let me know the answer to my question.

@joaomoreno I did what you said. Literally, any extension causes this problem. Only way to not get this error is to not install any extensions.

@e1337us3r If you fully reinstall Code, does it get fixed?


[FIX] Create nice buttons to scroll available sheets

Hello! Thank you for this great project.
It will be great to create buttons like in example for scrolling sheets in project.

Thank you for your time.

Yes, It's possible lets connect with hangout!!

How can I contact you?

Yes, It's possible lets connect with hangout!!

How can I contact you?

you can used QQ. or author Email: https://github.com/myliang


Why my results are all NAN？

I run the validate_alexnet_on_imagenet.ipynb with tensorflow 1.90,but the probs of my results are all NAN?
Could you help me?

Is everything set up correctly? (checkpoint path, image path, etc.)

Image path is right. Is  the  checkpoint file "bvlc_alexnet.npy"? If is,I had set it to the same path with current dir.

No, checkpoint file is only created after you had trained the model, it will be in the form of 'checkpoint_epoch.ckpt'. The file bvlc_alexnet.npy is the pre-trained weights for AlexNet. Can you post some code/error so its easier to debug

If I load model from the file bvlc_alexnet.npy, do I need to set the checkpoint file path? My code is same with the "validate_alexnet_on_imagenet". There is no error. The results of "probs" are all NAN.
My friend run it with tensorflow-1.60-gpu and he succeed, while I use tensorflow-1.90-cpu. Does the version of thsorflow causes this problem,such as gpu or cpu?

Version of TF does not matter. I use TF 1.09 and it works fine. Checkpoint file path is only set before training so that it saves the model. Since you are performing validation, you need to give the path to the latest checkpoint your model has trained until (until the last epoch). AlexNet then loads the latest checkpoint and validates based on the given image. Check that you are not setting load_pretrained_weights = False
There could be some error that I could think of:

The checkpoint path is not correct. Check ./checkpoint_final_epoch.ckpt
The weights are not being read properly. Try to print the weights. It will be in the form of conv1/weights0 etc. See in AlexNet.py to print the layer and weights.
The path to your test images are wrong. Check /path/test/images label
Lastly, in extreme circumstances, your model has not learned anything. Check TensorBoard weights during training.

If none of these work, take a look at your friends code and see what's wrong.

Thanks for your help very much.I have removed current tensorflow and install it in ubuntu 16.04 with tensorflow-1.10.The results are not NAN but they are very low.Anyway,I can continue to finetune Alexnet on my dataset.Thanks you again.

Sorry that I didn't answer for so long, but I became father recently and I did not find much time to turn on the computer.
Thanks @Prem95 for your detailed help and answering. It is very much appreciated. Anyway I'm unsure if it solves the problem of @HITWeihua. To my understanding, he is trying to use the notebook to validate the pretrained ImageNet weights against some images of ImageNet. In this case, there is no need to have any 't use TensorFlow for some time now and didn't try with the most recent version, because I switched to PyTorch, but I'll have a look into it. As long as TF didn't change anything dramatically it should work or only needs minor fixes.

@kratzert Congrats on being a dad! Also, thank you for making this repo.

@kratzert ,Congratulations! And thanks for your suggestions very much.To be honest,I didn't have much expectation for someone to help me.Anyway,thank you! @Prem95 and @kratzert.

Thanks to both of you. I'm glad both of you found their way to this code and think it's actually useful. I'll close this issue because it seems the question is solved. If not feel free to reopen.


WoWDL.net doesn't exist anymore!

Your launcher tries to download the client from WoWDL.net, which is, however, down and won't come back online, as this is most likely the result of a DMCA strike by Blizzard.

you found any way to bypas this issue? Im stuck on dowlonading 0!!

you found any way to bypas this issue? Im stuck on dowlonading 0!!

Unfortunately not, as you either provide the files yourself / which I wasnt able to source on the internet yet / or the script tries to take care of it via WoWDL, which is dead.

you found any way to bypas this issue? Im stuck on dowlonading 0!!

Unfortunately not, as you either provide the files yourself / which I wasnt able to source on the internet yet / or the script tries to take care of it via WoWDL, which is dead.

you can go with sanctuary wow launcher to get client, its a fork of this project and link is in their dc. whats what i did at least

you found any way to bypas this issue? Im stuck on dowlonading 0!!

Unfortunately not, as you either provide the files yourself / which I wasnt able to source on the internet yet / or the script tries to take care of it via WoWDL, which is dead.

you can go with sanctuary wow launcher to get client, its a fork of this project and link is in their dc. thats what i did at least

Unfortunately they don't have an MacOS compatible version.


Standardize metrics and events

This issue follows the Spike to identify inconsistencies in metrics and events https://github.com/criblio/appscope/issues/718.
We determined that the following rules should be put in place:

sourcetype must match the source i.e.  source=http.req, sourcetype=http
sourcetype + operation = source
Body fields should be consistent for all events
If a field is present in events and metrics the name should match
If there is a field in data that exists also in the body, we should remove the appropriate duplicate.
All metrics should have a unit field (for events that field is less important, since we aren't aggregating them)
Field names should use underscores instead of dot notation
All durations should be reported in milliseconds
Timestamps are in epoch seconds dot fractional seconds

We should make changes to field names as per this spreadsheet (internal only): https://docs.google.com/spreadsheets/d/1vtz5UiNmoSjhTZh1mYj1D253WL1Jk98M/edit?usp=sharing&ouid=102281914230889011606&rtpof=true&sd=true
Action items:

Make the relevant changes to adhere to the above ruleset
Make the name changes per the above spreadsheet
We should create constants in our code for metric and event names ; like we do for field names
We should add the above rules as a comment in the top of report.c
We should look through examples to ensure that values/units are correct
net.protocol appears twice in the net.conn.close event

I'm documenting here also (out of scope of this issue) that we discussed eventually creating unit tests to assert that the rules above are being followed, which everyone agreed upon.

another inconsistency:
for http 2.0 the value for http_status_code is a string; "200". for http 1.1 it is a number; 200.

The inconsistency founded while working on another task:

net_peer_port, net_host_port should be IMHO type number now it is string
in remote_protocol, fs_op_stat, fs_write, fs_read, net_conn_close, net_conn_open, net_conn_duration, net_dns_duration, net_dns_req, net_dns_resp, fs_duration, net_error, net_other, net_port, net_rx, net_tx, net_tcp, net_udp ,fs_error, fs_op_close, fs_op_open, fs_op_seek events we have a duplicate field of pid in body and data
minor thing is the inconsistency in order of fields in http-req and http-resp

http-req
            "http_method",
            "http_target",
            "http_flavor",            <<< 3rd field
            "http_scheme",
            "http_host",
            "http_user_agent",
            "net_transport",
            "net_peer_ip",
            "net_peer_port",
            "net_host_ip",
            "net_host_port"

http-resp
            "http_method",
            "http_target",
            "http_scheme",
            "http_flavor",             <<< 4th field
            "http_status_code",
            "http_status_text",
            "http_server_duration",
            "http_host",
            "http_user_agent",
            "net_transport",
            "net_peer_ip",
            "net_peer_port",
            "net_host_ip",
            "net_host_port",
            "http_response_content_length"

[ ] net_peer_port, net_host_port should be type integer
[ ] duplicate field of pid in body and data
[ ] event and metric name changes
[ ] http_status_code should be type integer
[ ] order of fields in http-req and http-resp
[ ] consistency in body fields among all events
[ ] remove raw metric events from schema definitions
[ ] export example events

One more inconsistency: regarding event which sourcetype is console, for this type of event we miss id and channel in event envelope.
{"type":"evt","body":{"sourcetype":"console","_time":1643371120.345928,"source":"stdout","host":"94b8c0b41668","proc":"ls","cmd":"ls","pid":299,"data":{"message":"bin   dev  home  lib32\tlibx32\tmnt  proc  run\t srv  tmp  var\nboot  etc  lib\t lib64\tmedia\topt  root  sbin  sys  usr\n"}}}

vs fs.open event where these fields are present
{"type":"evt","id":"94b8c0b41668-curl-curl wttr.in","_channel":"20575309296296","body":{"sourcetype":"fs","_time":1643374812.4796691,"source":"fs.open","host":"94b8c0b41668","proc":"curl","cmd":"curl wttr.in","pid":649,"data":{"file_name":"/etc/hosts","proc_uid":0,"proc_gid":0,"proc_cgroup":"0::/system.slice/containerd.service","file_perms":644,"file_owner":0,"file_group":0,"op":"fopen"}}}


sort data

hi
i am new in elastic search
whithin this package how can i sort an d pagination?
$ads = advertise::searchByQuery(array('match' => array('status' => '1')),null, null, 18, 0, ['created_at' => 'desc']);
i use this function but nothing happend
hepl me plz

hi
I have this problem too. And I tested all kinds of methods and sorting didn't work.
`$ads = advertise::searchByQuery([
'bool' =>[
'must'=>[
'term'=>[
'status'=>1,
]
]
],
'sort'=>[
'category_id'=>['order'=>'desc'],
]
                ]);`

or this       $query['bool']['filter']= ['term' => ['city' => $request->city]];             $query['bool']['sort']=['created_at'=>['order'=>'desc']];

hi
I have this problem too. And I tested all kinds of methods and sorting didn't work.
`$ads = advertise::searchByQuery([
'bool' =>[
'must'=>[
'term'=>[
'status'=>1,
]
]
],
'sort'=>[
'category_id'=>['order'=>'desc'],
]
            ]);`

or this
            $query['bool']['filter']= ['term' => ['city' => $request->city]];             $query['bool']['sort']=['created_at'=>['order'=>'desc']];             $ads = advertise::searchByQuery($query);

hi
I have this problem too. And I tested all kinds of methods and sorting didn't work.
`$ads = advertise::searchByQuery([
'bool' =>[
'must'=>[
'term'=>[
'status'=>1,
]
]
],
'sort'=>[
'category_id'=>['order'=>'desc'],
]
        ]);`

or this
            $query['bool']['must']= ['term' => ['status' =>1]];             $query['bool']['sort']=['created_at'=>['order'=>'desc']];             $ads = advertise::searchByQuery($query);

Bump! I cannot find the solution either


NYC numbers

the total for New York City isn't matching the sum of the 5 counties for 3/25. is this because the city/county sources are different?

Yup

Left: https://coronavirus.health.ny.gov/county-county-breakdown-positive-cases
Right: https://www1.nyc.gov/assets/doh/downloads/pdf/imm/covid-19-daily-data-summary.pdf
Work for this was done by @aed3 in #281.
@aed3, should we discard New York City in favor of a sum of the boroughs from the PDF?

I could. Not sure if I should though. The weird thing is the numbers from ny.gov site were updated more recently but have a smaller total, so I don't know which on is more accurate.

Yeah @aed3 I don't know which number to take either... There are probably some methodology differences...

Transferred to https://github.com/covidatlas/li/issues/389

Transferred to https://github.com/covidatlas/li/issues/390

Transferred to https://github.com/covidatlas/li/issues/391

Transferred to https://github.com/covidatlas/li/issues/392

Transferred to https://github.com/covidatlas/li/issues/393

Transferred to https://github.com/covidatlas/li/issues/394

Transferred to https://github.com/covidatlas/li/issues/395

Transferred to https://github.com/covidatlas/li/issues/396

Transferred to https://github.com/covidatlas/li/issues/397


[question] a few questions / feature requests

Hi,
I have a few questions:

I would like to return errors from the key store since I currently have to drop the errors in order to satisfy the interface. Would you accept such an PR if I change the interface?

Are message keys deleted at some point? I see that there is a DeletePk function however, it's only used to delete skipped keys as far as I can see. Since we have to safe all messages encrypted - we can just persist the messages keys and will be able to decrypt all messages from those keys at any point in time (or am I wrong)?

cheers,
Florian
P.s. thanks for this great repo

Hi, @florianlenz.

Yes, I think it's totally ok. What's more, the actual key store doesn't return errors just for the sake of simplicity.
Yes, correct. You can store the message keys and that's all required to decrypt a corresponding message.


Autodux doesn't work in CodeSandBox

I wanted to recreate the component of Eric's article in codesandbox: https://codesandbox.io/s/m9v130182j
But autodux can't be compiled by CodeSandbox and throws:
TypeError
Cannot read property 'package' of undefined

Here is the code:
import React, { Fragment, useReducer } from "react";
import ReactDOM from "react-dom";
import autodux from "autodux";

import "./styles.css";

const {
  reducer,
  actions: { click }
} = autodux({
  slice: "counter",
  initial: 0,
  actions: {
    click: count => count + 1
  }
});

const Counter = ({ clicks, onClick }) => (
  <Fragment>
    Clicks: {clicks}
    <button className="click-button" onClick={onClick}>
      Click
    </button>
  </Fragment>
);

function App() {
  const [clicks, dispatch] = useReducer(reducer, reducer());
  return (

      <Counter clicks={clicks} onClick={() => dispatch(click())} />;

  );
}

const rootElement = document.getElementById("root");
ReactDOM.render(<App />, rootElement);

I'm experiencing this issue too, not sure what the problem is.


Fix HPWL mismatch

Reproduce using:
pytest -s -v -- tests/pdk/finfet_pdk/test_annotation.py::test_dcl
The test exits at the first instance of mismatch between HPWL_ILP and HPWL_extend

I will take a look, thanks.

In this PR,

can we drop HPWL_extend and instead use HPWL_ILP exclusively?
make sure HPWL_extend matches the checker in Python

@854768750 There is also the scenario where ILP isn't called. For a single instance hierarchy with no PlaceOnGrid

@soneryaldiz @854768750 I'm not able to reproduce the failing test in CI. Its:
tests/pdk/finfet_pdk/test_circuits.py::test_comparator_analog

@srini229 , I also could not reproduce locally. I revised test to tolerate a single error.

Internal tests are passing.


New docs: Coding boxes not always need line numbers

Some sample code would be just one-liners and terminal commands that don’t need line numbers.

resolved with https://github.com/blockstack/docs.blockstack/commit/66b1a5a9cdea9827394baeaf2ce13130ddf3aa58


Consider using high 16 bits of token type as token modifiers

Consider doing it in a way where having < 32 modifiers does not require the usage of the tokentype's high bits.

We closed this issue because we don't plan to address it in the foreseeable future. If you disagree and feel that this issue is crucial: we are happy to listen and to reconsider.
If you wonder what we are up to, please see our roadmap and issue reporting guidelines.
Thanks for your understanding, and happy coding!


Subclassed model with ConvLSTM2D layer can't be saved as a SavedModel in TF2.2

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): custom code, extended an example from TF guides
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 18.04/Mac OS 10.15
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device:
TensorFlow installed from (source or binary): binary
TensorFlow version (use command below): 2.1 and 2.2
Python version: 3.6
Bazel version (if compiling from source):
GCC/Compiler version (if compiling from source):
CUDA/cuDNN version: 10.1
GPU model and memory: 1080Ti 11Gb

Describe the current behavior
As a header states the model build with Subclassing API with ConvLSTM2D layer inside can't be saved as a SavedModel. Given that keras (.h5) model format doesn't support saving subclassed models I am left with no option to save the model architecture to file.
The issues appears in TF2.2 while there seems to be no bug in earlier version 2.1
Describe the expected behavior
The code is to work without issues in both TF2.2 and TF2.1
Standalone code to reproduce the issue
Provide a reproducible test case that is the bare minimum necessary to generate
the problem. If possible, please share a link to Colab/Jupyter/any notebook.
https://colab.research.google.com/drive/1zfhnbz_dHfPloT9mzk0ei5F4aFaZGgnt?usp=sharing
Other info / logs Include any logs or source code that would be helpful to
diagnose the problem. If including tracebacks, please include the full
traceback. Large logs and files should be attached.
import numpy as np
import tensorflow as tf
print(tf.__version__)
from tensorflow import keras
from tensorflow.keras.layers import ConvLSTM2D, Bidirectional, LSTM

class CustomModel(keras.Model):
  def __init__(self, hidden_units):
    super(CustomModel, self).__init__()
    self.lstm = Bidirectional(ConvLSTM2D(filters=16, kernel_size=(1, 1), return_sequences=True, return_state=True))
    self.dense_layers = [keras.layers.Dense(u) for u in hidden_units]

  def call(self, inputs, training=None, mask=None):
    x = inputs
    x, _, _, _, _ = self.lstm(x)
    for layer in self.dense_layers:
      x = layer(x)
    return x

model = CustomModel([16, 16, 10])
# Build the model by calling it
input_arr = tf.random.uniform((1, 10, 10, 10, 5))
outputs=model.predict(input_arr)
model.save('my_model')

# Delete the custom-defined model class to ensure that the loader does not have
# access to it.
del CustomModel

loaded = keras.models.load_model('my_model')

Similar issue discussed on stackoverflow
https://stackoverflow.com/questions/61362953/keras-convlstm2d-valueerror-when-saving-model

Saving works in TF 2.3.0-rc0, thanks!


we could not able to save as vector pdf on server side.

Hi,
we were looking for save canvas on server side as vector pdf, here we not get exact solution.
May i know, is there is a possibility to save as vector pdf on server side...???

I am not sure, but maybe you can try svgcanvas and wkhtmltopdf?

Thanks for your replay...
we got solution with https://github.com/zenozeng/p5.js-svg/
But We changed your p5-svg.js file
p5.prototype.saveSVG = function() {
// don't use slice on arguments because it prevents optimizations
var args = arguments;
args = [args[0], args[1], args[2]];
    var svg;

    if (args[0] instanceof p5.Graphics) {
        svg = args[0]._graphics.svg;
        args.shift();
    }

    if (typeof args[0] == "object") {
        svg = args[0];
        args.shift();
    }

    var filename = args[0];
    var ext = args[1];

    var p = this;
    this._makeSVGFrame({
        svg: svg,
        filename: filename,
        extension: ext,
        callback: function(err, frame) {

$.ajax({
type: "POST",
url: "script.php",
data: {
imgBase64: frame.imageData
}
});
$.ajax({
type: "POST",
url: "script.php",
data: {
imgBase64: frame.imageData  // Passing dataurl to ajax page
}
});
//p.downloadFile(frame.imageData, frame.filename, frame.ext); // We hide default download
}
});
};
// Server Side Script
$img = $POST['imgBase64'];
$image_name = time().''.'P5.svg';
$file   = file($img);
$result = file_put_contents($image_name, $file);

:+1:

BTW, you can directly override p5Instance.downloadFile on the server side, then you can call saveSVG() directly.


Re-write HQQ Fused Gemm Transpose

@msaroufim
Re-write transposed fused gemm

Previous transpose implementation required weights to be unpacked / repacked.
This was particularly problematic for fine-tuning, since it introduced additional overhead either during the forward or backwards pass as noted by @mobicham in CUDA-MODE discord discussions.
Given that the intended use-case for the fused kernel is for fine-tuning, removing this overhead is essential.
The new version now enables the user to pack the weights once: the only change to run the transposed kernel is to pass transposed=True, no repacking needed for the weights or reshaping needed for scales, zeros.

Checks

[x] All tests pass for the new transpose implementation
[x] Benchmarks show no significant performance difference between transposed and non-transposed kernels

Changes

[x]  Rewrite transpose kernel implementation
[x] Add docstring explaining the revised design.
[x] Updated tests and benchmarks to check correctness and performance of transpose kernel.
[x] Updated /prototype/hqq/README.md with Usage section.

Great job @jeromeku , in a full training setup, the kernel is 1.11x - 1.26x faster than dequantize() -> bfp16 torch.matmul (vanilla HQQ ref). This is in-line with the first benchmark I did on the layer level:
QLoRA fine-tuning (4-bit, group-size=None) | Llama2-7B | SPDA attention | GPU: A6000 ADA

batch_size=1, ctx_size=1024
HQQ ref (pad to ctx_size)                 : 2.06 it/sec | 18.9 GB
HQQ ref (pad to ctx_size) + torch.compile : 2.33 it/sec | 18.6 GB -> 1.13x faster
Triton mm                                 : 2.60 it/sec | 18.8 GB -> 1.26x faster

batch_size=2, ctx_size=1024
HQQ ref (pad to ctx_size)                 : 1.17 it/sec | 32.2 GB
HQQ ref (pad to ctx_size) + torch.compile : 1.32 it/sec | 31.0 GB -> 1.13x faster
Triton mm                                 : 1.33 it/sec | 31.8 GB -> 1.14x faster

batch_size=1, ctx_size=2048
HQQ ref (pad to ctx_size)                 : 1.07 it/sec | 33.3 GB
HQQ ref (pad to ctx_size) + torch.compile : 1.22 it/sec | 32.0 GB -> 1.14x faster
Triton mm                                 : 1.19 it/sec | 32.9 GB -> 1.11x faster

This is a setup when all the prompts are longer or equal to ctx_size, so it depends on the datasets. Normally with a better sampling, you'd group prompts of similar size within the same batch and you only pad to the longest sequence, which could be 1.5x faster with HQQ ref for some samples. It is possible to mimic something like this with Triton mm by warming-up for a couple of pre-defined sizes [batch_size, 256], [batch_size, 512], etc... and pad to the closest one.

Is it possible to make it compatible with torch.compile()? Currently it crashes when I try to compile the model. HQQ ref with torch.compile(model) is faster and uses a bit less memory. I would expect the same behavior with Triton mm.

* [x]  Benchmarks show no significant performance difference between transposed and non-transposed kernels

The older benchmark numbers are not consistent with the previous benchmark, I think it has to do with the shapes:
https://gist.github.com/mobicham/48e3ad537da6748e39d8a4ce27bfd612#file-triton_mm_benchmark-py-L154-L191


Manual database changes not reflecting in Uptime Kuma while app is running

⚠ Please verify that this bug has NOT been raised before.

[X] I checked and didn't find similar issue

🛡 Security Policy

[X] I agree to have read this project Security Policy

Description
Hello-
I am attempting to change some values in the database directly with DB Browser for SQLite. For example, flipping the active value of a monitor from 1 to 0.  When I do this while Uptime Kuma is not running, when I start up Uptime Kuma it picks up my changes just fine.
But when Uptime Kuma is already running and I attempt to make this change to the database, those changes are not reflected in Uptime Kuma. And then when I do something in Uptime Kuma that requires an update to the database (say changing the name of a monitor), the changes I made through DB Browser are overwritten like they never happened.
I get the same behavior when I use Python to interact with the database. I am also getting some other weird behavior that seems inconsistent regarding changes not persisting when they are done across different clients (UK, DB Browser, and Python), but I think they are all related to some underlying issue. This is just a hunch, and once this issue is resolved, if those other problems persist I can make new issue(s).
👟 Reproduction steps

Spin down Uptime Kuma.
Modify a monitor's active value in the database directly using something like DB Browser.
Spin up Uptime Kuma, and see it picks up the change. Yay.
Then attempt to change it again using DB Browser. Uptime Kuma does not pick up the change. Not yay.

👀 Expected behavior
The expected behavior is that making changes to the database should be accurately reflected in Uptime Kuma, especially when Uptime Kuma is running. And when changes are made in Uptime Kuma, they should not overwrite the changes I made to the database directly.
This expected behavior does work like this when I interact with the database with my python API and DB Browser (with Uptime Kuma spun down). Meaning if I update the database with DB Browser, my Python API (which is using the same database) does accurately see the changes. And when I make changes to the database through my Python API, DB Browser sees the changes correctly.
I would like this behavior to also work with Uptime Kuma, that way I can manually update the database without having to spin down Uptime Kuma.
😓 Actual Behavior
Instead, UK does not see the changes I made to the database, and will overwrite my manual changes when it makes its own update to the database.
🐻 Uptime-Kuma Version
1.17.0
💻 Operating System and Arch
MacOS 11.6.6
🌐 Browser
Chrome Version 102.0.5005.115
🐋 Docker Version
Docker version 20.10.12, build e91ed5707e
🟩 NodeJS Version
No response
📝 Relevant log output
N/A.

I change it to feature-request, as it is an expected behavior and I think it is quite normal for relational database without live data.

I see. I disagree that it is normal for any database to behave in this way, but it is your project, so you definitely have final say on application behavior.
I have never encountered any other application that behaves this way, which is why I feel it's a bug. This behavior is pretty critical for the RESTful API I am developing to function correctly, so I hope that you consider changing this to reflect the expected behavior above.
Thanks!

Given the current architecture of Uptime Kuma I would classify this is a wontfix (read: not in scope) issue.
The reason behind this is that SQLite would require going to the C-API-level (I think should not be opening this pandoras box)  and MariaDB would require even worse hacks.
=> Polling would be the only option
=> given how minor the feature is, this is not worth the very significant engineering effort required.
closing as not planned due to the rationale explained above (if a good argument is made why this is nessesary/.. we can reopen)
Instead of expecting that we look if somebody changed the database while we were not looking:

stop your instance before
make a backup
tamper with it
restart the instance

Alternatively (this is the actual path that I think this should be done with):

Use the unofffical https://pypi.org/project/uptime-kuma-api/
wait for #118 to be finally ready ^^


Theme Repo does not pick up updates

I realized I am running a very old version of the theme (due to the many discord css updates) but on theme repo the 'source' is from 2021: https://github.com/LuckFire/amoled-cord/blob/d201810b52483c8f108f1c4ea355f6f5c34cb054/src/support/betterdiscord/AMOLED-Cord.theme.css (when clicking on the source button)

I don't know if this is a themerepo issue (doubt it but could be) or some miss configuration :)

Dunno what client mod this is, but not an issue with the repo 👍

pretty sure this is goosemods store but goosemod is gone :/ so idk

I assumed that but I had the same thought so I wasn't sure lol

No, this is from DevilBro's BD plugin. https://github.com/mwittrien/BetterDiscordAddons/tree/master/Plugins/ThemeRepo/
Since the plugin is listed here: https://betterdiscord.app/themes?filter=amoled-cord it is also listed in the repo.
It can be argued if this is a issue with a repo or the plugin, but since all other themes have version control (which reloads the theme and updates the css) and 'update' and work fine with it, I think this is a repo issue 🙂

Ohh if it's a DB plugin, it's most likely getting it from the website, so yes it is probably a repo issue because the site isn't picking up the new theme.css file. Once I rework AMOLED-cord tho it'll most likely be fixed, however I'll reopen this issue and leave it opened til that's done 👍

Once I rework AMOLED-cord tho it'll most likely be fixed

ETA for the rework?

Whenever a "good" client mod works, such as Replugged. I could use BetterDiscord, but I'd have to figure out a way to do stuff in the structure that I was before which I don't think I can easily do.

i did not mean to close that LMAO

This should be fixed now.