Server: Replace use of Celluloid::Future for async threads

Fixes #2126 to log any errors raised by the async thread blocks
Using Celluloid::Future to execute async blocks in threads without actually checking the future value is broken, in that it silently ignores any resulting errors. This results in bugs like #2584 which are very hard to notice.
Fix the server to just use a simple AsyncHelper#async_thread do ... end which at least logs the errors, making them slightly more likely to be noticed during development. It does not actually handle the errors, though.

Celluloid::Future uses a thread pool, how much this change will impact resource usage?

Celluloid::Future uses a thread pool, how much this change will impact resource usage?

From reading the celluloid 0.17 code, I suspect that Celluloid::Future.new { ... } doesn't actually use a thread pool, it looks like each call spawns a new thread without any limit on the concurrency. Have not confirmed that, though, and the documentation / discussion on issues is unclear on the subject.

https://github.com/celluloid/celluloid/blob/v0.17.3/lib/celluloid/future.rb#L38

The Celluloid::Future.new implementation is very tricky to read, but I think the return super unless block means that the block never actually gets passed to #initialize, so that code using Celluloid.internal_pool.get is not actually used? Instead, it uses the Internals::ThreadHandle.new path, which is a new thread, not a pool.
As I said, would need to test to actually figure it out... :confused:

Does not look like a threadpool to me, every Celluloid::Future.new gets a new thread:
irb(main):028:0> require 'pp'; threads = {}; (1..100).map { |i| Celluloid::Future.new { [ i, Thread.current.to_s] } }.map{|f| f.value }.each { |i, thread| (threads[thread] ||= []) << i }; pp threads
{"#<Celluloid::Thread:0x00562cea91ff60>"=>[1],
 "#<Celluloid::Thread:0x00562cea91f880>"=>[2],
 "#<Celluloid::Thread:0x00562cea91e9a8>"=>[3],
 "#<Celluloid::Thread:0x00562cea91e390>"=>[4],
 "#<Celluloid::Thread:0x00562cea91de18>"=>[5],
 "#<Celluloid::Thread:0x00562cea91d878>"=>[6],
 "#<Celluloid::Thread:0x00562cea91d2d8>"=>[7],
 "#<Celluloid::Thread:0x00562cea91ccc0>"=>[8],
 "#<Celluloid::Thread:0x00562cea91c770>"=>[9],
 "#<Celluloid::Thread:0x00562cea91c1a8>"=>[10],
 "#<Celluloid::Thread:0x00562cea91e930>"=>[11],
 "#<Celluloid::Thread:0x00562cea91cd38>"=>[12],
 "#<Celluloid::Thread:0x00562cea917d38>"=>[13],
 "#<Celluloid::Thread:0x00562cea9178b0>"=>[14],
 "#<Celluloid::Thread:0x00562cea917400>"=>[15],
 "#<Celluloid::Thread:0x00562cea916f78>"=>[16],
 "#<Celluloid::Thread:0x00562cea916ac8>"=>[17],
 "#<Celluloid::Thread:0x00562cea916640>"=>[18],
 "#<Celluloid::Thread:0x00562cea9161b8>"=>[19],
 "#<Celluloid::Thread:0x00562cea915d30>"=>[20],
 "#<Celluloid::Thread:0x00562cea915880>"=>[21],
 "#<Celluloid::Thread:0x00562cea9153f8>"=>[22],
 "#<Celluloid::Thread:0x00562cea914f48>"=>[23],
 "#<Celluloid::Thread:0x00562cea914a70>"=>[24],
 "#<Celluloid::Thread:0x00562cea914570>"=>[25],
 "#<Celluloid::Thread:0x00562cea9140c0>"=>[26],
 "#<Celluloid::Thread:0x00562cea90fac0>"=>[27],
 "#<Celluloid::Thread:0x00562cea90f4d0>"=>[28],
 "#<Celluloid::Thread:0x00562cea90ee40>"=>[29],
 "#<Celluloid::Thread:0x00562cea90e8a0>"=>[30],
 "#<Celluloid::Thread:0x00562cea90dfb8>"=>[31],
 "#<Celluloid::Thread:0x00562cea90c960>"=>[32],
 "#<Celluloid::Thread:0x00562cea90f930>"=>[33],
 "#<Celluloid::Thread:0x00562cea90ddb0>"=>[34],
 "#<Celluloid::Thread:0x00562cea90d810>"=>[35],
 "#<Celluloid::Thread:0x00562cea90d2c0>"=>[36],
 "#<Celluloid::Thread:0x00562cea90cc30>"=>[37],
 "#<Celluloid::Thread:0x00562cea90c280>"=>[38],
 "#<Celluloid::Thread:0x00562cea907e10>"=>[39],
 "#<Celluloid::Thread:0x00562cea907960>"=>[40],
 "#<Celluloid::Thread:0x00562cea9074d8>"=>[41],
 "#<Celluloid::Thread:0x00562cea907050>"=>[42],
 "#<Celluloid::Thread:0x00562cea906bc8>"=>[43],
 "#<Celluloid::Thread:0x00562cea906740>"=>[44],
 "#<Celluloid::Thread:0x00562cea9062b8>"=>[45],
 "#<Celluloid::Thread:0x00562cea905e30>"=>[46],
 "#<Celluloid::Thread:0x00562cea9059a8>"=>[47],
 "#<Celluloid::Thread:0x00562cea905520>"=>[48],
 "#<Celluloid::Thread:0x00562cea905098>"=>[49],
 "#<Celluloid::Thread:0x00562cea904be8>"=>[50],
 "#<Celluloid::Thread:0x00562cea904738>"=>[51],
 "#<Celluloid::Thread:0x00562cea9042b0>"=>[52],
 "#<Celluloid::Thread:0x00562cea8fbd40>"=>[53],
 "#<Celluloid::Thread:0x00562cea8fb818>"=>[54],
 "#<Celluloid::Thread:0x00562cea8fb340>"=>[55],
 "#<Celluloid::Thread:0x00562cea8fae90>"=>[56],
 "#<Celluloid::Thread:0x00562cea8fa9e0>"=>[57],
 "#<Celluloid::Thread:0x00562cea8fa530>"=>[58],
 "#<Celluloid::Thread:0x00562cea8f9c48>"=>[59],
 "#<Celluloid::Thread:0x00562cea8f9540>"=>[60],
 "#<Celluloid::Thread:0x00562cea8f9090>"=>[61],
 "#<Celluloid::Thread:0x00562cea8f8c08>"=>[62],
 "#<Celluloid::Thread:0x00562cea8f8780>"=>[63],
 "#<Celluloid::Thread:0x00562cea8f82f8>"=>[64],
 "#<Celluloid::Thread:0x00562cea8dfc58>"=>[65],
 "#<Celluloid::Thread:0x00562cea8df640>"=>[66],
 "#<Celluloid::Thread:0x00562cea8df1b8>"=>[67],
 "#<Celluloid::Thread:0x00562cea8dea60>"=>[68],
 "#<Celluloid::Thread:0x00562cea8ddb88>"=>[69],
 "#<Celluloid::Thread:0x00562cea8dd700>"=>[70],
 "#<Celluloid::Thread:0x00562cea8dd278>"=>[71],
 "#<Celluloid::Thread:0x00562cea8dcdf0>"=>[72],
 "#<Celluloid::Thread:0x00562cea8dc968>"=>[73],
 "#<Celluloid::Thread:0x00562cea8dc4e0>"=>[74],
 "#<Celluloid::Thread:0x00562cea8dc058>"=>[75],
 "#<Celluloid::Thread:0x00562cea8d3b60>"=>[76],
 "#<Celluloid::Thread:0x00562cea8d36b0>"=>[77],
 "#<Celluloid::Thread:0x00562cea8d3228>"=>[78],
 "#<Celluloid::Thread:0x00562cea8d2d28>"=>[79],
 "#<Celluloid::Thread:0x00562cea8d28a0>"=>[80],
 "#<Celluloid::Thread:0x00562cea8d2418>"=>[81],
 "#<Celluloid::Thread:0x00562cea8d1f40>"=>[82],
 "#<Celluloid::Thread:0x00562cea8d1a90>"=>[83],
 "#<Celluloid::Thread:0x00562cea8d15b8>"=>[84],
 "#<Celluloid::Thread:0x00562cea8d1130>"=>[85],
 "#<Celluloid::Thread:0x00562cea8d07d0>"=>[86],
 "#<Celluloid::Thread:0x00562cea8cbe10>"=>[87],
 "#<Celluloid::Thread:0x00562cea8cb988>"=>[88],
 "#<Celluloid::Thread:0x00562cea8cb500>"=>[89],
 "#<Celluloid::Thread:0x00562cea8cb050>"=>[90],
 "#<Celluloid::Thread:0x00562cea8cab28>"=>[91],
 "#<Celluloid::Thread:0x00562cea8ca600>"=>[92],
 "#<Celluloid::Thread:0x00562cea8ca128>"=>[93],
 "#<Celluloid::Thread:0x00562cea8c9ca0>"=>[94],
 "#<Celluloid::Thread:0x00562cea8c97f0>"=>[95],
 "#<Celluloid::Thread:0x00562cea8c9368>"=>[96],
 "#<Celluloid::Thread:0x00562cea8c8eb8>"=>[97],
 "#<Celluloid::Thread:0x00562cea8c8a30>"=>[98],
 "#<Celluloid::Thread:0x00562ceabe7338>"=>[99],
 "#<Celluloid::Thread:0x00562ceabe6b18>"=>[100]}

PTAL @jakolehm? This is ugly, but AFAIK it's equivalent and better than the blind Celluloid::Future.new, and I can't think of a better approach without reworking how async mutations run entirely.
Fixing the MongoPubsub error handling is a separate issue, this shouldn't make it any worse...

Yeah, it seems that celluloid 0.17 behaves a bit differently with plain futures.

Just to be clear I don't think this is the ideal solution, but at least now we might notice these errors. Once we start seeing those errors, we might have a better idea of what we should be doing to handle them.


Replace mkdirp with own implementation

Same idea like in https://github.com/zeit/next.js/issues/6941
This PR replaces mkdirp node module by a simpler/lighter implementation. It reduces install size of 40kB+ according to packagephobia.
Node v10.12.0 introduces a recursive option to fs.mkdir and we can use it when we drop node 8 support.

Could you do a benchmark similar to the bench directory?

@timneutkens

Could you do a benchmark similar to the bench directory?

I added a benchmark in bench/recursive-copy.

@lucleray could you share the results?

@timneutkens
~ cd bench
~ yarn run bench:recursive-copy
test recursive-copy npm module
{ sum: 1067.209105, nb: 10, avg:<PHONE_NUMBER>9999999 }
test recursive-copy custom implementation
{ sum: 582.435537, nb: 10, avg: 58.24355369999999 }

So if we can't make it faster let's maybe ncc the mkdirp module 👍

I added a piece of code to detect node version and, when available use mkdir(dir, {recursive:true})
mkdirp ends up being faster on node >= v10.12 :
luc ~/w/n/n/bench ‣ nvm use lts/latest
luc ~/w/n/n/bench ‣ yarn run bench:mkdirp
yarn run v1.15.2
warning package.json: No license field
$ node mkdirp/run.js
test mkdirp npm module
{ sum:<PHONE_NUMBER>9999998, nb: 100, avg: 1.7770820999999999 }
test mkdirp custom implementation
{ sum:<PHONE_NUMBER>0000006, nb: 100, avg: 1.4206693600000007 }
test mkdirp sync npm module
{ sum: 162.313997, nb: 100, avg: 1.62313997 }
test mkdirp sync custom implementation
{ sum: 115.01636, nb: 100, avg: 1.1501636 }
✨  Done in 1.27s.

@lucleray since this makes the install size bigger let's postpone this until January of next year when Node 8 is EOL (and we can only use the built-in version). 🙏 Thank you for all your efforts!

@Timer I forgot to remove mkdirp from packages/next/package.json hence the bigger size. I'll do it now.


Vickel: Watch tradingview-magical-keyboard-shortcuts.com

Vickel requests the watch of the watch_keyword tradingview-magical-keyboard-shortcuts\.com. See the MS search here and the Stack Exchange search in text, in URLs, and in code.

tradingview-magical-keyboard-shortcuts\.com has been seen in 1 true positive, 0 false positives, and 0 NAAs.

Approved by Nick in Charcoal HQ


Support for short options with numeric values, e.g, -p80

Seems like a reasonable feature to support which wouldn't step on other parsing goals.
Open question
What do you do with --p80?
Refs: https://github.com/pkgjs/parseargs/pull/59

With one dash or two? With one, I’d expect it’s short for -p 80

what should the behavior with two dashes be--p80?

A long option named “p80”, only.

As per previous comment, I think --p80 is a usage of a long option named p80.

As for the short option -p80, long reply. My opinion. 😄
I do not think treating numbers as special for combining with short options is the correct high level approach. It is tempting and it is more familiar, but it is opinionated and breaks some other cases.
In particular, digits are valid short flags. If p is a flag then expanding -p1 to -p -1 is self consistent.
One affordance that supporting (any) value directly following a short option allows is supplying negative numbers or other values starting with a dash. So if v expects a value, you can enter -v-123 or -v--warnings. I think of this as the same affordance we have with long options and using =, --value=-123 or --value=--warnings.
I would prefer to have no support for trailing values than treat numbers specially in this regard. Then we still have a simple self-consistent story. We support grouping short flags. We do/don't support trailing values. No special cases.

I completely agree that if i do -pa or -p1, the only two options for both should be equivalent to either -p -a/-p -1 unconditionally, or, -p a/-p 1 iff a and 1 are not configured as options and also -p is configured as an option with a value.

(Small nit, I disagree with iff a and 1 are not configured as options but save that discussion for if we actually he'd towards support for trailing values!)

What i mean is, with zero config, I’d expect them to all be considered as options, not values.

Support for trailing values for short options has landed, so current behaviour is:
const { parseArgs } = require('@pkgjs/parseargs')
const options = { port: { short: 'p', type: 'string' }};
console.log(parseArgs({ options, strict: false }));

% node p80.js -p80
{ values: { port: '80' }, positionals: [] }
% node p80.js -a80
{ values: { '0': true, '8': true, a: true }, positionals: [] }


Compile GPU windows

Hello,
I am sorry if this is not place to write. I just wanted to ask you if you know if there is possibility to run solution with GPU on windows platform  TENSORFLOW_LITE_GPU. I tried with your release but with no success..
Thank you very much,
Josip

I have tried to build TensorflowLite library with GPU delegates for Windows, but it failed.
I think it's better to ask at the official repo, https://github.com/tensorflow/tensorflow .

Thank you very much.
I also tried but whatever I looked I could not find desktop support.
Thank you once more


ImportError: Failed to import plugin saleor.plugins.webhook.plugin.WebhookPlugin: No module named 'pytest'

What I'm trying to achieve
Upgrading my Saleor core to 3.6 via a forked (and unchanged) GitHub repo I have onto Heroku
Steps to reproduce the problem

I go to my Heroku Deploy dashboard for the app
I enter 3.6 in the Deploy a GitHub Branch and hit Deploy

At the end I get an issue installing pytest:
Traceback (most recent call last):
  File "/app/saleor/plugins/apps.py", line 24, in load_and_check_plugin
    plugin = import_string(plugin_path)
  File "/app/.heroku/python/lib/python3.9/site-packages/django/utils/module_loading.py", line 17, in import_string
    module = import_module(module_path)
  File "/app/.heroku/python/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 790, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/app/saleor/plugins/webhook/plugin.py", line 43, in <module>
    from .shipping import get_excluded_shipping_data, parse_list_shipping_methods_response
  File "/app/saleor/plugins/webhook/shipping.py", line 18, in <module>
    from .tasks import trigger_webhook_sync
  File "/app/saleor/plugins/webhook/tasks.py", line 29, in <module>
    from ...webhook import observability
  File "/app/saleor/webhook/observability/__init__.py", line 3, in <module>
    from .payloads import dump_payload
  File "/app/saleor/webhook/observability/payloads.py", line 16, in <module>
    from .obfuscation import (
  File "/app/saleor/webhook/observability/obfuscation.py", line 23, in <module>
    from ...graphql.api import schema
  File "/app/saleor/graphql/api.py", line 9, in <module>
    from .checkout.schema import CheckoutMutations, CheckoutQueries
  File "/app/saleor/graphql/checkout/schema.py", line 13, in <module>
    from ..payment.mutations import CheckoutPaymentCreate
  File "/app/saleor/graphql/payment/mutations.py", line 38, in <module>
    from ..checkout.mutations.utils import get_checkout
  File "/app/saleor/graphql/checkout/mutations/__init__.py", line 3, in <module>
    from .checkout_complete import CheckoutComplete
  File "/app/saleor/graphql/checkout/mutations/checkout_complete.py", line 28, in <module>
    from ...order.types import Order
  File "/app/saleor/graphql/order/types.py", line 30, in <module>
    from ...order import OrderStatus, calculations, models
  File "/app/saleor/order/calculations.py", line 11, in <module>
    from ..order.tests.test_calculations import TaxData
  File "/app/saleor/order/tests/test_calculations.py", line 5, in <module>
    import pytest
ModuleNotFoundError: No module named 'pytest'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/app/manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/app/.heroku/python/lib/python3.9/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_line
    utility.execute()
  File "/app/.heroku/python/lib/python3.9/site-packages/django/core/management/__init__.py", line 395, in execute
    django.setup()
  File "/app/.heroku/python/lib/python3.9/site-packages/django/__init__.py", line 24, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/app/.heroku/python/lib/python3.9/site-packages/django/apps/registry.py", line 122, in populate
    app_config.ready()
  File "/app/saleor/plugins/apps.py", line 20, in ready
    self.load_and_check_plugin(plugin_path)
  File "/app/saleor/plugins/apps.py", line 26, in load_and_check_plugin
    raise (ImportError(f"Failed to import plugin {plugin_path}: {e}"))
ImportError: Failed to import plugin saleor.plugins.webhook.plugin.WebhookPlugin: No module named 'pytest'

What I expected to happen
Version is deployed OK
System information
Saleor version:
3.6
Operating system:

[ ] MacOS

Hey @amityweb,
Thanks for raising this. It looks like there is an incorrect import added by mistake. If you need a temporary solution, you could install requirments_dev.txt instead of requirments.txt. I will prepare a fix in the meantime.

 If there’s info online about the requirements file I’ll take a look but that’s new to me hope I can find how to do it! Thanks.


Incorrect error about meaningless access property combination

Importing the SystemRDL snippet below returns this error:

.\mytestmap.rdl:6:21: error: Field 'EnablePs' access property combination is meaningless: sw=w; hw=na;
StrobeBit_t EnablePs[0:0];

addrmap TopAddrMap {
    reg {
        field StrobeBit_t {sw=w; hw=na; singlepulse; reset=1'b0;};
        field StatusBit_t {sw=r; hw=r;};

        StrobeBit_t EnablePs[0:0];
        EnablePs->desc = "Write a 1 to this bit to enable the power supply";

        StrobeBit_t DisablePs[1:1];
        DisablePs->desc = "Write a 1 to this bit to disable the power supply";

        StatusBit_t PsEnabled[0:0];
        PsEnabled->desc = "This bit returns a 1 if the power supply is enabled";

        // Control the state of the PsEnabled bit
        PsEnabled->hwset = EnablePs;
        PsEnabled->hwclr = DisablePs;
    } PowerSupplyControl;

};

The EnablePs and DisablePs fields work together to set or clear the PsEnabled bit (which is read by HW). They are write-only by SW and do not need to be read by HW, so I think the property combination should be considered valid (at least for the case where the field in question controls another field that is read by either SW or HW).

Fixed in 1.25.4


Update expressroute-locations.md

Global Connect missing with peering location Oslo

@FloKlaffenbach : Thanks for your contribution! The author(s) have been notified to review your proposed change.

#reassign:charwen

@charwen Would you review/resolve this pull request? Thanks!

@FloKlaffenbach You might choose to close this PR and open a new PR in the private repo. You can sign off directly in the private repo. However, if your proposed update requires review by the content team, you can leave this PR open.
To edit an article in the private repo from within the documentation, after you select Edit on an article page,  add “-pr” to the repo name in the URL.

To post in the private repo (azure-docs-pr), you might need to join the MicrosoftDocs organization.

Related links in the internal Microsoft Docs contributor guide:
Public vs. private repos | Move a commit to a different repo (public -> private) | GitHub for content reviews
Quick reference for contributing to the docs | Make changes to a Docs article | PR review quality criteria

The issue that's addressed in the PR appears to be addressed in master. We'll close the PR. Thanks.


Unable to Open From system problems tab of line, will generate to read double addres From File

VS Code Version:
OS Version:

Steps to Reproduce:

Does this issue occur when all extensions are disabled?: Yes/No

These diagnostics are generated by the extension and so the links associated to them. It seems the extension is creating invalid or not existing links. So please file the issue against the extension.


Add low-level helper routines for data exchange

This PR tries to remove some duplicated code by introducing low-level helper functions for receiving and sending data.
BTW: I am wondering about the except struct.error as err: In which circumstances can the struct.unpack raise an exception? I think only if it gets not the correct number of bytes. This, in turn, can only happen, if the _socket.recv does not return the correct number of bytes (which is, by documentation, possible). Maybe this should be handled in the new reading routines?

BTW: I am wondering about the except struct.error as err: In which circumstances can the struct.unpack raise an exception? I think only if it gets not the correct number of bytes. This, in turn, can only happen, if the _socket.recv does not return the correct number of bytes (which is, by documentation, possible). Maybe this should be handled in the new reading routines?

Are you suggesting to add this in this pull request, or will you prepare this later on? Should we create an issue for this?

Deleting more lines that are re-added. I like those refactorings. Makes the code better understandable :-).


Incompatibility with ratatui v0.26.0

I want to update to ratatui to v0.26.0, but it includes at least one breaking change, which makes the structs emitted by ansi-to-tui incompatible with the new version of ratatui.
Maybe the new style field in Line? breaking changes
Please consider updating the ratatui dependency.

After testing this crate against 0.26.0, I haven't found any error with the current implementation. There's just one test which is broken but I don't think it's related in any way.

Can you try doing a  cargo tree -i ratatui and paste the output here ?

Hi, I worked with two crates and mixed something up of where which ratatui version was used. I am sorry. Everything works fine. Thank you for your readiness and wilingness to help.

Yeah it's a semver hazard like mentioned in the issue #32 From the next release onward I'll make a new release for every ratatui update.


[DEV-3406] additional export data (#1825)

DEV-3406 additional export data

DEV-3406 order folders alphanumerically

DEV-3406 refactor

Release Checklist
Pre-Release

[ ] Check migrations

No database related infos (sqldb-xxx)
Impact on GS (new/removed columns)

[ ] Check for linter errors (in PR)
[ ] Test basic user operations (on DFX services)

Login/logout
Buy/sell payment request
KYC page

Post-Release

Test basic user operations
Monitor application insights log

@lapatric ship it 🎉


Move method code refactoring for locateOriginalText method

locateOriginalText method can be moved to CompareTemplateOutputHandler class from LicenseCompareHelper class.

Resolved with PR #297
There may be a similar issue and opportunity in the SPDX Java Library code.


Some TOC and minor updates.

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

casoper-playground/TOC.md
:white_check_mark:Succeeded

casoper-playground/additional-tools.md
:white_check_mark:Succeeded

casoper-playground/data-packages.md
:white_check_mark:Succeeded

casoper-playground/management-packages.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Add google signin (frontend)

I want to add the google signin frontend

please assign me these tasks

Go ahead @AK261102

yes sir

@sid0000007 google sign in ,you want full stack or only frontend ?

@ AK261102: Sign-In comes in front-end.

@AK261102: Sign-In comes in front-end.

okk sir


bug: UI element overlap in account display

Campus Ambassador overlaps the user name

@tanmaychandekar98 please look into this. Will require wrapping the name.

Closing as dormant and there have been significant changes to the page to render this bug report as useful now.


feat: add dataloader instrumentation

Which problem is this PR solving?
The dataloader package is fairly popular in instances where batching of several operations into one is beneficial (for example batching multiple DB reads into one query against the DB). The package has about 3,3M weekly downloads and is quite a crucial part of the GraphQL ecosystem on the backend (it is the main approach to solving the n+1 problem).
Currently dataloaders are not traceable, although their way of work nicely falls under the example use case for linking spans.
Short description of the changes

Added new instrumentations package - @opentelemetry/instrumentation-dataloader

👋
Added myself to the owners file. I'm not 100% sure about the approach I have taken with wrapping the default export constructor itself. I couldn't think of any other way to get access to the batch load function, it seems to work, but not quite sure whether there is a better approach to achieve that (one that would also properly un-patch itself for example).

Should I also add this to the auto-instrumentations-node package as a dependency?

Should I also add this to the auto-instrumentations-node package as a dependency?

Your readme.me holds If total installation size is not constrained, it is recommended to use the @opentelemetry/auto-instrumentations-node... - so yes.
I think it's also needed to add it in release-please-config.json otherwise release automation will ignore it.

Any updates on this? I think I have covered everything, but please let me know if there is something I'm missing

Congrats, @henrinormak, on your first contribution!
We'll merge the release PR shortly.


Question about render quality and multi-scale

Thanks for your great work.
I noticed that the torch-ngp did not implement multi-scale, and I guess this may be the reason for the image blur. So I would like to ask if you have any plans to achieve multi-scale?
Thanks.

The real reason is instant-ngp use all train/val/test set to train, which is set to default in GUI mode now. You can try it now!

@ashawkey Thanks for your awesome work! I have tried official instant-ngp with only training data, still achieving high performance, >34 PSNR with 1 min on LEGO. I wonder if any clue has been figured out for the quality issue.

@JaminFong You are right, the performance in PSNR is still worse compared to instant-ngp (as I mentioned in readme's update logs). Honestly, I haven't figured out why for now, but I'm still trying.
If you have any idea, welcome to report in issue/pull request.

@ashawkey Thanks for the reply! I really admire your implementation work. I am also delving into ngp and happy for further discussion if any idea got.


Fix server side rendering

Fix ReactInterval for server-side by moving all operation to lifecycle methods
Eliminate state
Simplify shouldComponentUpdate and remove extra dependency
Improve example to be able to change callback and also dynamically add and destroy ReactInterval

Fixes #8

Here we go @ericfong

Great thanks


Adds github link to lexical playground

May be could have added it inside the react app 😛  But I thought adding it to the HTML is faster and renders way before the playground even mounts
After:

LGTM, although I kinda liked clean index.html :)

Maybe we can move this to being in the App.js rather than adding to index.html?

bump


[nemo-calendar] Remove deprecated use of Incidence::setHasGeo(). Fixes JB#60292

No-op if bool parameter is true and otherwise there area lready calls to setGeoLatitude/Longitude.
@dcaliste

Yes, no need to setHasGeo(), by construction with the previous setGeoLat/Lon, the src.hasGeo and dest.hasGeo must be equal.
Good to me.

Ideally, one should do something like:
 dest->startUpdates();
 *dest.staticCast<KCalendarCore::IncidenceBase>() = *src.staticCast<KCalendarCore::IncidenceBase>();
 dest->endUpdates();

and maybe reset some properties that we may not want to copy (if any).

Ideally, one should do something like

Yea, could be. Not sure immediately why there is such an explicit property handling, but can be considered then separately some day.


Updated support page

@katarzyna-koltun-mx Updated Support Page

@mengdix I've reviewed the update and it looks good. Jean also approved it. I have 1 question - why is "Certified" and "Partner" put in brackets in lines 121 and 122?

@mengdix I've reviewed the update and it looks good. Jean also approved it. I have 1 question - why is "Certified" and "Partner" put in brackets in lines 121 and 122?

@katarzyna-koltun-mx Parts of this component are supported directly by product, others require investigation with Partners first. Principally Partners are always first line of support.

@mengdix So should "(Certified) Mendix (Partners)" be understood as "Mendix Support, Mendix Partners or Certified Mendix Partners"? I'd like to be explicit and the current wording is a bit ambiguous.

@katarzyna-koltun-mx In the context of who to reach out to regarding issues: Certified Mendix Partners

@mengdix Got it, in that case I've removed the brackets and will publish directly.


Warning system does not consider preprocessing.

PreProcessing has undefined functions listed as basically nonexistent, although this is recognized by the processor and therefore discarded, warning still takes note of this.

probably intentional, but having to wrap your entire event around a define instead of taking action based on a specific function is rather UNPLEASENT

The preprocessor does what it says. It just preprocesses to remove the unused branches of ifdefs. So there's no way for this to be possible. And I'm not entirely sure what you're trying to say since you haven't provided a specific warning that this is flagging.


checkov: 2.5.4 -> 2.5.6

Diff: https://github.com/bridgecrewio/checkov/compare/refs/tags/2.5.4...2.5.6
Changelog: https://github.com/bridgecrewio/checkov/releases/tag/2.5.6
Description of changes

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.11 Release Notes (or backporting 23.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 259326 run on x86_64-linux 1

2 packages built:

checkov
checkov.dist


Classiq

Update the new things

Hi @kamali1331 - if you could be more specific we could address this issue. Thank you.


Update AppsService.CreateInstallationToken with API changes

GitHub Developer API announcement:
https://developer.github.com/changes/2018-08-16-renaming-and-deprecation-of-github-app-installation-access-token-route/
This would be a great PR for any new contributor to this repo or a new Go developer.
All contributions are greatly appreciated!
Feel free to volunteer for any issue, and we can send you an invite to contribute to the repo (which you then accept after you enable two-factor authentication) and the issue can be assigned to you so that others don't attempt to duplicate the work.
Please check out our CONTRIBUTING.md guide to get started.
Thank you!

I'd like to work on this.

@gmlewis I'd like to work on this.

@gmlewis I'd like to work on this.

Since the issue was left open for more than a month, I decided to submit a PR.


[Video Stitcher]: Tests have been consistently failing for a couple of days

See logs here.
It seems like resources take longer to be available now?

There's an email from Karl on this, I'll loop you in.


Update sys-dm-os-wait-stats-transact-sql.md

Adding description for the new wait BUFFERPOOL_SCAN, from new feature Buffer Pool Parallel Scan

Learn Build status updates of commit a2e3691:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/relational-databases/system-dynamic-management-views/sys-dm-os-wait-stats-transact-sql.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the learn.microsoft.com contributor guidesPost your question in the Learn support channel

Learn Build status updates of commit 10de465:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/relational-databases/system-dynamic-management-views/sys-dm-os-wait-stats-transact-sql.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the learn.microsoft.com contributor guidesPost your question in the Learn support channel

@westMSFT,
Can you review the proposed changes?
IMPORTANT: When the changes are ready for publication, add a #sign-off comment to signal that the PR is ready for the review team to merge.
#label:"aq-pr-triaged"
@MicrosoftDocs/public-repo-pr-review-team

@jborsecnik I have already reviewed the proposed changes, and am awaiting response to my suggestions.

Learn Build status updates of commit 301c7cd:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/relational-databases/system-dynamic-management-views/sys-dm-os-wait-stats-transact-sql.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the learn.microsoft.com contributor guidesPost your question in the Learn support channel

Learn Build status updates of commit 5cc81de:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/relational-databases/system-dynamic-management-views/sys-dm-os-wait-stats-transact-sql.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the learn.microsoft.com contributor guidesPost your question in the Learn support channel

#sign-off

This PR will be replaced by a new one because I also did some article refreshment.


upgrade liteide to 27.2.1 and download link

1.upgrade liteide to 27.2.1
2.upgrade ...macosx.zip to ...macosx-qt5.zip

Thank you for the contribution. It was merged directly as 9c3e175e054f56d5aa2cfd93e6b0e213af319002 to keep commit history cleaner. Your contribution is still credited to you.

Thank you for the contribution. It was merged directly as 679c1a14f8cc9e03bbb8370ea92483421941b60d to keep commit history cleaner. Your contribution is still credited to you.


Ios bugfix resetrootview

when keyboard pick up and change the rootview frame ,freeze rootview frame. when keyboard drop down , unfreeze rootview frame

Current coverage is 88.44% (diff: 100%)

Merging #1700 into ios-feature-20161208 will not change coverage

@@           ios-feature-20161208      #1700   diff @@
======================================================
  Files                        41         41
  Lines                      2233       2233
  Methods                     354        354
  Messages                      0          0
  Branches                    500        500
======================================================
  Hits                       1975       1975
  Misses                      258        258
  Partials                      0          0

Powered by Codecov. Last update 8f55a44...6efdaba


Queue using array

Queue program using array
What type of PR is this? (check all applicable)

[x] 🚀 Added Name
[x] ✨ Feature
[x] 🌟 stared the repo
[x] 🐛 Grammatical Error
[x] 📝 Documentation Update
[x] 🚩 Other

Description
Add Link of GitHub Profile

hey @keshavsingh4522,
Pls assign this to me, want to work on the same


fix: realm creation with an idp and custom auth flow

Use case :

I want to create a new realm with an IDP and a custom First Broker login auth flow.

Now the importer fail with the following error :
Cannot create realm 'realmWithIdentityProviders': {"error":"unknown_error"}
...
HTTP 500 Internal Server Error
...

On Keycloak Side, we've got the following log
org.keycloak.models.ModelException: No available authentication flow with alias: my-custom-flow-alias

The test case ImportIdentityProvidersIT.shouldCreateOtherIdentityProviderWithCustomFirstLoginFlow reproduce this case.
This is solved by removing IDP from initial realm creation and let them create after realm (and auth flows) creation

Nice catch! Can you put something inside the Changelog to cover this?

Too fast ;-) It-is possible to release this change ?

The new test doesn't cover the bug except the test runs alone.
I adjust the test and create a new release.


Model load failed

When I run example.py, I get “_pickle.UnpicklingError: A load persistent id instruction was encountered, but no persistent_load function was specified.” Can you provide the version of pytorch?

The torch version we use is 1.12.1. After checking, there is no problem with the code. And we will check the integrity of the uploaded file, which will take some time


Please support deno-postgres

It would be nice to provide support for deno-postgres.

Hi @silvioprog I suspect that support is possible and am open to it!
It seems like deno-postgres supports template tags for .queryObject and .queryArray. I'm not familiar with the package, are there any others?

The thing that makes me hesitate here is that .queryObject`  and .queryArray`  seem extremely generic things to match for syntax highlighting.
The last thing I want is to start supporting an unbounded number of template tags. Even the recent addition of sqlFoo`  and dbFoo`  have me a bit nervous, if I'm honest.
🤔


Supports external GPU?

Hi! I was wondering whether you knew if the kinect body tracking sdk would work with an external gpu. I have a laptop (macbook) with windows via bootcamp - so can use the sensor sdk but not the body tracking sdk - and was worried about spending the money on an external gpu without knowing whether or not it would work.

This question might be related to issue #882
Currently the body tracking SDK will use the GPU device 0 to run the algorithm. I would suspect it will always use your laptop GPU for the current version. But we are planning to ship a version that allows the customer to choose which GPU to use, then I think it will allow you to use an external GPU.


KazooClient bug? connection time-out, but not retry another zk server.

My question is that:
when client connect to yahoo.com:80, there is a timeout exception was throwed,  But it didn't retry connect to <IP_ADDRESS>:2181( this is my real zk server).  Anybody can help me?  thanks.
source code:
import logging
logging.basicConfig(level=logging.DEBUG)
from kazoo.client import KazooClient
from kazoo.retry import KazooRetry
_retry = KazooRetry(max_tries=1000, delay=0.5, backoff=2)
_zk = KazooClient(hosts="yahoo.com:80,<IP_ADDRESS>:2181", logger=logging, read_only=True, timeout=30, connection_retry=_retry)
_zk.start()
_zk.stop()
log:
DEBUG:root:ZK loop started
DEBUG:root:Skipping state change
INFO:root:Connecting to yahoo.com:80
DEBUG:root:    Using session_id: None session_passwd:<PHONE_NUMBER>0000000000000000000000
INFO:root:Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd='\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=True)
WARNING:root:Connection dropped: socket connection broken
DEBUG:root:Connection stopped
Traceback (most recent call last):
File "/tmp/1.py", line 7, in
_zk.start()
File "/home/s/apps/wdshopping/python/lib/python2.7/site-packages/kazoo-1.2.1-py2.7.egg/kazoo/client.py", line 476, in start
raise self.handler.timeout_exception("Connection time-out")
kazoo.handlers.threading.TimeoutError: Connection time-out

i'm facing this as well. did you ever figure it out?

Try setting the kazoo.client.KazooClient.start timeout higher than the kazoo.client.KazooClient timeout.
For your example:
_zk.start(timeout=40)

^^^ Doing the above can help, but might mask an issue with timeouts once the connection is established. Kazoo seems to use the client session timeout (the one in kazoo.client.KazooClient) for each attempted TCP connection before negotiating a session, but later on, it divides this number by the number of hosts in the ensemble to decide on its internal "connect timeout" (e.g. how long to wait before a write request or a reconnect times out.)
So, if you set a low timeout in kazoo.client.KazooClient.__init__, you might run into issues where reconnecting or submitting requests times out much faster than you expected.
I'm opening a separate issue to check whether this is intended behavior.


Design of the sampling algorithm for texts

I am referring to the implementation of text sampling method for neighborhood data. My understanding is that it randomly removes a subset of words from the original text.
Is this a limitation or special design? It seems to me that sometimes a text is classified not only for the presence of certain words, but also for the absence of others. For example, a message might classified as non-insulting not because of the words in it, but the words NOT in it.
So sampling texts by removing only would result in a narrower space to search in, which could result in a different class distributions even in the vicinity, e.g., all sampled data have positive labels as an extreme case. Appreciate it if you can elaborate on the sampling part.

This was a conscious choice. While there are cases where the absence of terms explains more than the presence, I think that text explanations become a bit confusing when adding 'absence' explanations.
In the insult example you gave, the way to interpret a 'null' explanation (no words) is 'there is no evidence of insult in this message, and in the absence of such evidence the model predicts non-insult'. Would it be better to explain it as 'none of the following words are in it: [list of 100 insulting words']? Maybe, but I think it becomes too confusing.

@marcotcr thanks for reply. Yes the difficulty arises when I am thinking about generalizing the model to other data. For example, when using complicated features (e.g., RNN rather than TFIDF) for texts, I am not sure if the sampling-by-removing method would still work, because removing a word might actually result in a complicated disjoint regions in the feature space, and thus the "neighborhood" assumption may not hold any more for the interpretable linear approximation. At least the signs of coefficients would not be directly interpretable as in the TFIDF cases.
Do you think it would be useful to make sampling a separate class, just like the Explanation class, so that sampling can be customized in different ways for different types of data/feature combination? Thanks!

@dolaameng I've been thinking along similar lines (i.e., how the sampling procedure affects RNNs). I think it may be reasonable as long as the removed terms are replaced with UNK tokens rather than completely removed. Did you have another sampling procedure in mind?

I do have the option of replacing the removed term s with UNKs (just set bow=False), and I've used it to explain LSTMs with (I think) good results.
I think I may separate the data types (text, tabular, image) further in a future refactor, to make it easier to plug in different sampling strategies. Do you have other sampling ideas though? I couldn't think of anything other than:

removing words
replacing words with UNK
replacing words with other words (via language model, random words, whatever)

@andrewyates @marcotcr I agree it is challenging to keep the sampling part flexible and interpretable at the same time.  Considering that the computation of sampling needs to be fast to be useful, many complicated methods (e.g.,language modelling) may already be off the table.
Actually I am not even sure if the "sampling" and "feature-selection" steps should be decoupled, since both of them depend on the data/tasks, and thus highly correlated to each other.
No I don't really have any better ideas. But in the context of neural networks, it might be interesting to look at the gradient w.r.t inputs to say which parts are contributing to the final predictions. Again it is not something new - I think deep dreams  by Google was related to this idea. But I am not sure if anyone has done similar things to texts data. The drawback is that its result and interpretation will be specific to your model.
I am happy to see any new explorations.

Yeah, we're trying to always treat the models as black boxes (so no looking at gradients for now). Anyway, I'm actively working on this, hopefully I'll make some progress soon : ).
For now, I'll close the issue. Thanks for the insightful discussion.


Feature: Refactor Dockerfile to be usable for running youtube-to-anchorfm with a docker run command and add some additional fixes and refactors

This PR refactors Dockerfile a bit and adds logging facility in order to additionally log into a file
In particular, this PR include these commits:

Refactor Dockerfile: install npm dependencies on image build time and only run youtube-to-anchorfm when container is starting
Refactor Dockerfile: remove workdir since github actions runs the docker image with different workdir
Implement logging to file and create a png screenshot file in the directory where logs are kept
The logs are configured to be kept in ./logs. The screenshot is configured to be saved in ./logs/screenshot.png. This change will be useful in the scenario where youtube-to-anchorfm is run with the docker image. IT should be possible to mount a volume in order to see the logs and the screenshot. Note that the log format is the same in the console, i.e the users won't notice a difference when using youtube-to-anchorfm with github action.
Fix problem of program hanging infinitely when selecting schedule date that is a lot of months difference from current date
When scheduling a date that is a lot of months away or before current date, it requires clicking on the calendar for changing month a lot of times. Sometimes this hands the program forever. The implemented solution adds a 500ms sleep on every click on calendar that changes month
Add a connection timeout to youtube-dl calls so it will fail if there is no internet connection instead of waiting indefinitely
youtube-dl calls for getting video info or downloading might hand indefinitely if there is no internet connection. This commit fixes that by adding a connection timeout
Use absolute path specification for target when mounting package.json and package-lock.json in Dockerfile
This helps to make Dockerfile compatible with podman build

To me is looking good; could you please print the build log here? Then I'll approve

I'd it's ok for everyone I'm going to merge it tomorrow

I updated this PR to include a case where spotify auth accept page is not shown at all and instead the new episode wizard is shown immediately after logging in.
Here is a log of one running of the application:
[2024-09-16T21:10:25.203] [INFO] default - Getting JSON video info for video id a3ICNMQW7Ok
[2024-09-16T21:10:28.265] [INFO] default - title: Wildlife Windows 7 Sample Video
[2024-09-16T21:10:28.265] [INFO] default - description: All clips is property of Microsoft Windows.
[2024-09-16T21:10:28.265] [INFO] default - Upload date: {"year":"2012","month":"Oct","monthAsNumber":"10","monthAsFullWord":"October","day":"26"}
[2024-09-16T21:10:28.265] [INFO] default - Downloading thumbnail for video id a3ICNMQW7Ok
[2024-09-16T21:10:28.267] [INFO] default - Downloading audio for video id a3ICNMQW7Ok
[2024-09-16T21:10:32.570] [INFO] default - Downloaded audio for video id a3ICNMQW7Ok
[2024-09-16T21:10:34.504] [INFO] default - Downloaded thumbnail for video id a3ICNMQW7Ok
[2024-09-16T21:10:34.505] [INFO] default - Posting episode to anchorfm
[2024-09-16T21:10:34.505] [INFO] default - Launching puppeteer
[2024-09-16T21:10:38.936] [INFO] default - Setting language to English
[2024-09-16T21:10:38.982] [INFO] default - Trying to log in and open episode wizard
[2024-09-16T21:10:38.982] [INFO] default - -- Accessing new Spotify login page for podcasts
[2024-09-16T21:10:38.993] [INFO] default - -- Logging in
[2024-09-16T21:10:40.869] [INFO] default - -- Trying to accepting spotify auth
[2024-09-16T21:10:41.870] [INFO] default - -- Waiting for episode wizard to open
[2024-09-16T21:10:45.849] [INFO] default - -- Episode wizard is opened
[2024-09-16T21:10:45.850] [INFO] default - -- No need to accept spotify auth
[2024-09-16T21:10:45.850] [INFO] default - Uploading audio file
[2024-09-16T21:10:45.850] [INFO] default - -- Uploading audio file
[2024-09-16T21:10:45.860] [INFO] default - -- Waiting for upload to finish
[2024-09-16T21:10:52.909] [INFO] default - -- Audio file is uploaded
[2024-09-16T21:10:52.909] [INFO] default - Filling required podcast details
[2024-09-16T21:10:52.910] [INFO] default - -- Adding title
[2024-09-16T21:10:55.358] [INFO] default - -- Adding description
[2024-09-16T21:10:55.578] [INFO] default - -- Schedule publishing for date:  26 October, 2012
[2024-09-16T21:10:55.578] [INFO] default - -- Setting publish date
[2024-09-16T21:12:11.270] [INFO] default - -- Selecting content type(explicit or no explicit)
[2024-09-16T21:12:11.279] [INFO] default - -- Selection content sponsorship (sponsored or not sponsored)
[2024-09-16T21:12:11.286] [INFO] default - Filling optional podcast details
[2024-09-16T21:12:11.286] [INFO] default - -- Clicking Additional Details
[2024-09-16T21:12:11.313] [INFO] default - -- Uploading episode art
[2024-09-16T21:12:11.342] [INFO] default - -- Saving uploaded episode art
[2024-09-16T21:12:11.362] [INFO] default - -- Waiting for uploaded episode art to be saved
[2024-09-16T21:12:18.470] [INFO] default - Skipping Interact step
[2024-09-16T21:12:18.471] [INFO] default - -- Going to Interact step so we can skip it
[2024-09-16T21:12:18.500] [INFO] default - -- Waiting before clicking next again to skip Interact step
[2024-09-16T21:12:19.500] [INFO] default - -- Going to final step by skipping Interact step
[2024-09-16T21:12:19.521] [INFO] default - Save draft or publish
[2024-09-16T21:12:19.521] [INFO] default - -- Saving draft
[2024-09-16T21:12:36.364] [INFO] default - Yay
[2024-09-16T21:12:36.452] [INFO] default - Finished successfully.

Check if the changes are okay and approve the PR :D
Here are the commit messages of the last two commits I made:
Implement optional spotify login auth accept to support a case where the spotify auth accept page is not shown
Fix some prettier reports in anchorfm-pupeteer/index.js


Adding Backlog Grooming and Daily Scrum Calls text

@jenharris A couple more big chunks of content when you have some time to review...

Added sprint planning text as well. Looks like it'll all be in the same pull request @jenharris .

Adding sprint demo text also. @jenharris it's ready for your review whenever.

Thanks @jmheaster - I will try to review today


Manager service

Create a service with all the ticketing managers inside
With dynamic (__call) methods as getTicketManager
And the get method with a class as parameter
All the manager will be in an array with this syntax
Ticket::class => TicketManager

No real interest at the moment.


nixos/pam: add libpwquality

Motivation for this change
An attempt at adding libpwquality to the pam config
reference: https://deer-run.com/users/hal/linux_passwords_pam.html
/etc/security/pwquality.conf docs: https://github.com/libpwquality/libpwquality/blob/master/doc/man/pwquality.conf.5.pod
depends on #161652

Should the config be under pam? or no? It certainly didn't feel like an option to go under pam.services.<name>.enablePWQuality but maybe I'm wrong
Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
22.05 Release Notes (or backporting 21.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[ ] Fits CONTRIBUTING.md.

I need to get some feedback on the libpwquality package changes and pull them into their own PR. Might need to go in staging since I'm guessing it needs a lot of rebuilds

new libpwquality with pam (from https://github.com/NixOS/nixpkgs/pull/161652) now merged so rebased. I'll undraft this now
I'm thinking maybe the pwquality config should be moved from security.pam.pwquality just to security.pwquality ?
Or maybe we keep it specific if we can force the pam module to just use the config generated for it and not read /etc/security/pwquality.conf ?
open to thoughts here

Sorry it took a while. Can you please fix the conflicts and move the release notes to 23.05 ?

Thanks for taking a look. Yeah I'll rebase it & ping you

rebased to the new pam setup (which is very nice).
modified the settings to try follow rfc 42

Is this the library that systemd would like to have for password quality?

Looks like passwdqc is another option and seems popular with gentoo and altlinux but poettering seems to prefer pwquality (or at least that distros standardise)
https://github.com/systemd/systemd/issues/15055
https://github.com/systemd/systemd/pull/20425/files

I'm not sure if the pwquality stuff needs to actually be moved into pamOpts it can be applied service by service :thinking:
https://github.com/NixOS/nixpkgs/blob/nixos-unstable/nixos/modules/security/pam.nix#L97
https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/programs/shadow.nix#L190-L216
cc @Majiir for guidance


Issue with document-level security expression and date range

I am having an issue while trying to set a document-level security expression in my opendistro-for-elasticsearch instance. This is the expression I'm trying to apply:
{
  "bool": {
    "must": [
      {
        "terms": {
          "name": [
            "John"
          ]
        }
      },
      {
        "range": {
          "birthdate": {
            "gte": "now-30y"
          }
        }
      }
    ]
  }
}

I am getting the following error message when I try to search the index with a user linked to the Role:

Request failed to execute. Call: Status code 400 from: POST /users/_search?typed_keys=true. ServerError: Type: search_phase_execution_exception Reason: "all shards failed" CausedBy: "Type: parse_exception Reason: "could not read the current timestamp" CausedBy: "Type: illegal_argument_exception Reason: "'now' is not allowed in DLS queries"""

In the elasticsearch forum, I was pointed to these limitations, but I'm not sure if opendistro uses xpack under the hood or if it implements the security roles in a different way.
Do you have any idea on how to create a document-level security role with a relative date to now?
Thanks.

Playing around with the repository I found out that the "culprit" is this line:
https://github.com/opendistro-for-elasticsearch/security/blob/68e7e08e2f4306f974c2c0b17c8e50fa5a67c4d4/src/main/java/com/amazon/opendistroforelasticsearch/security/configuration/OpenDistroSecurityFlsDlsIndexSearcherWrapper.java#L65
and the configuration value that it tries to get. If I add that config value to elasticsearch.yml and set it to true, everything works fine and the document-level security expression is applied correctly.
Now I'm wondering: probably there's a reason why that line is there, does anybody know?
Thanks.

It is a security risk, as now is not fixed, so admin needs to accept that DLS definition depends on the time query is executed.


Support for Redis v6+ ACL auth

Affected Puppet, Ruby, OS and module versions/distributions

Puppet: 7.2+
Ruby:
Distribution: RedHat 9
Module version: 11.0.0

How to reproduce (e.g Puppet code you use)
Can't find parameter with name masteruser when searching the repo.
What are you seeing
Zero matches
What behaviour did you expect instead
As of Redis 6 when ACL was introduced and sentinel got support for the following parameters:
Redis:
masterauth
masteruser
Sentinel
sentinel auth-user
sentinel auth-pass
So I expect to be able to specify this for sentinel and redis replicas and so on.

I've opened two PR's: #528 and #529, the only difference is that the last one does not treat the parameters as sensitive types.

What are the next steps and what is the expected timeline for?

Review
Merge
Release

I am not a Redis expert, but I had a quick look. You can sometimes get a quick review by posting on Slack.


adding support for suggested_comment_sort

This adds support for suggested_comment_sort in the set_settings() method.

Pushed commit that adds a default parameter for suggested_comment_sort.

Looks great! Thanks @caleywoods for opening, and thanks @13steinj for the guidance.

Probably not the last time I'll violate PEP 0008

You can do pip install flake8 and then use flake8 while in the praw directory to look for any pep issues. Make sure to do that for each commit and you'll avoid most trouble with Travis.

Thanks @caleywoods for the PR, @13steinj for the comments, and @voussoir for merging. Great work all around!

@bboe - What's the ETA for that to be available? I did a pip search praw about an hour ago and was still showing me up to date on 3.3.0

@caleywoods I just might make a release for this version. However, if you want to get the latest before I do, you can run:
pip install --upgrade https://github.com/praw-dev/praw/archive/master.zip

Making a release requires updating the changelog and a few other things so I just want to make sure everything is in that could be in for a release.

Done - That cleared up the warning at the start of the log too (as expected). Thanks!


Animation Digital Network

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

[X] I understand that I will be blocked if I intentionally remove or skip any mandatory* field

Checklist

[X] I'm reporting that yt-dlp is broken on a supported site
[X] I've verified that I have updated yt-dlp to nightly or master (update instructions)
[X] I've checked that all provided URLs are playable in a browser with the same IP and same login details
[X] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[X] I've searched known issues and the bugtracker for similar issues including closed ones. DO NOT post duplicates
[X] I've read the guidelines for opening an issue
[X] I've read about sharing account credentials and I'm willing to share it if required

Region
Germany
Provide a description that is worded well enough to be understood
I have tried to download a video (https://animationdigitalnetwork.com/de/video/1233-loner-life-in-another-world) where it is in Japanese and have German subtitle. The Subtitles would be fine downloaded and mark in the metadata as German. But the audio is shown as ENGLISH, what is completely wrong.
I have tried some different commands like:
Japanese: %(meta_language)s" --embed-metadata -f [language=jap]
japanese: %(meta_language)s" --embed-metadata -f [language=jap]
Japanese: %(meta_language)s" --embed-metadata -f [language=jpn]
or
Japanese: %(meta_language)s" --embed-metadata -f [language=jp]
but nothing will write the audio metadata correct to japanese ,
Provide verbose output that clearly demonstrates the problem

[X] Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
[ ] If using API, add 'verbose': True to YoutubeDL params instead
[X] Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output
[debug] Command-line config: ['-vU', '-u', 'PRIVATE', '-p', 'PRIVATE', 'japanese:', '%(meta_language)s --embed-metadata -f [language=jpn] --sub-langs de --write-subs --sub-format ssa -v -a C:\\Users\\Stefan\\Desktop\\adn.txt -o Z:\\Anime', 'Serien\\Loner', 'Life', 'in', 'Another', 'World\\Loner', 'Life', 'in', 'Another', 'WorldS%(season_number)02dE%(episode_number)02d.%(ext)s']
[debug] Encodings: locale cp1252, fs utf-8, pref cp1252, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version<EMAIL_ADDRESS>from yt-dlp/yt-dlp [4bd265539] (win_exe)
[debug] Python 3.10.11 (CPython AMD64 64bit) - Windows-10-10.0.22631-SP0 (OpenSSL 1.1.1t  7 Feb 2023)
[debug] exe versions: ffmpeg 7.1-full_build-www.gyan.dev (setts), ffprobe 7.1-full_build-www.gyan.dev
[debug] Optional libraries: Cryptodome-3.21.0, brotli-1.1.0, certifi-2024.08.30, curl_cffi-0.5.10, mutagen-1.47.0, requests-2.32.3, sqlite3-3.40.1, urllib3-2.2.3, websockets-14.1
[debug] Proxy map: {}
[debug] Request Handlers: urllib, requests, websockets, curl_cffi
[debug] Loaded 1837 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version<EMAIL_ADDRESS>from yt-dlp/yt-dlp
yt-dlp is up to date<EMAIL_ADDRESS>from yt-dlp/yt-dlp)
[generic] Extracting URL: japanese:
[generic] japanese:: Downloading webpage
ERROR: Unable to handle request: Unsupported url scheme: "japanese" (requests, urllib, websockets, curl_cffi)
Traceback (most recent call last):
  File "yt_dlp\YoutubeDL.py", line 1624, in wrapper
  File "yt_dlp\YoutubeDL.py", line 1759, in __extract_info
  File "yt_dlp\extractor\common.py", line 742, in extract
  File "yt_dlp\extractor\generic.py", line 2393, in _real_extract
  File "yt_dlp\extractor\common.py", line 898, in _request_webpage
  File "yt_dlp\YoutubeDL.py", line 4162, in urlopen
  File "yt_dlp\networking\common.py", line 130, in send
yt_dlp.networking.exceptions.NoSupportingHandlers: Unable to handle request: Unsupported url scheme: "japanese" (requests, urllib, websockets, curl_cffi)

[generic] Extracting URL: %(meta_language)s --embed-metadata -f [language=jpn] --sub-langs de --write-subs --sub-format ssa -v -a C:\Users\Stefan\Desktop\adn.txt -o Z:\Anime
ERROR: [generic] '%(meta_language)s --embed-metadata -f [language=jpn] --sub-langs de --write-subs --sub-format ssa -v -a C:\\Users\\Stefan\\Desktop\\adn.txt -o Z:\\Anime' is not a valid URL. Set --default-search "ytsearch" (or run  yt-dlp "ytsearch:%(meta_language)s --embed-metadata -f [language=jpn] --sub-langs de --write-subs --sub-format ssa -v -a C:\Users\Stefan\Desktop\adn.txt -o Z:\Anime" ) to search YouTube
  File "yt_dlp\extractor\common.py", line 742, in extract
  File "yt_dlp\extractor\generic.py", line 2361, in _real_extract

[generic] Extracting URL: Serien\Loner
ERROR: [generic] 'Serien\\Loner' is not a valid URL. Set --default-search "ytsearch" (or run  yt-dlp "ytsearch:Serien\Loner" ) to search YouTube
  File "yt_dlp\extractor\common.py", line 742, in extract
  File "yt_dlp\extractor\generic.py", line 2361, in _real_extract

[generic] Extracting URL: Life
ERROR: [generic] 'Life' is not a valid URL. Set --default-search "ytsearch" (or run  yt-dlp "ytsearch:Life" ) to search YouTube
  File "yt_dlp\extractor\common.py", line 742, in extract
  File "yt_dlp\extractor\generic.py", line 2361, in _real_extract

[generic] Extracting URL: in
ERROR: [generic] 'in' is not a valid URL. Set --default-search "ytsearch" (or run  yt-dlp "ytsearch:in" ) to search YouTube
  File "yt_dlp\extractor\common.py", line 742, in extract
  File "yt_dlp\extractor\generic.py", line 2361, in _real_extract

[generic] Extracting URL: Another
ERROR: [generic] 'Another' is not a valid URL. Set --default-search "ytsearch" (or run  yt-dlp "ytsearch:Another" ) to search YouTube
  File "yt_dlp\extractor\common.py", line 742, in extract
  File "yt_dlp\extractor\generic.py", line 2361, in _real_extract

[generic] Extracting URL: World\Loner
ERROR: [generic] 'World\\Loner' is not a valid URL. Set --default-search "ytsearch" (or run  yt-dlp "ytsearch:World\Loner" ) to search YouTube
  File "yt_dlp\extractor\common.py", line 742, in extract
  File "yt_dlp\extractor\generic.py", line 2361, in _real_extract

[generic] Extracting URL: Life
ERROR: [generic] 'Life' is not a valid URL. Set --default-search "ytsearch" (or run  yt-dlp "ytsearch:Life" ) to search YouTube
  File "yt_dlp\extractor\common.py", line 742, in extract
  File "yt_dlp\extractor\generic.py", line 2361, in _real_extract

[generic] Extracting URL: in
ERROR: [generic] 'in' is not a valid URL. Set --default-search "ytsearch" (or run  yt-dlp "ytsearch:in" ) to search YouTube
  File "yt_dlp\extractor\common.py", line 742, in extract
  File "yt_dlp\extractor\generic.py", line 2361, in _real_extract

[generic] Extracting URL: Another
ERROR: [generic] 'Another' is not a valid URL. Set --default-search "ytsearch" (or run  yt-dlp "ytsearch:Another" ) to search YouTube
  File "yt_dlp\extractor\common.py", line 742, in extract
  File "yt_dlp\extractor\generic.py", line 2361, in _real_extract

[generic] Extracting URL: WorldS%(season_number)02dE%(episode_number)02d.%(ext)s
ERROR: [generic] 'WorldS%(season_number)02dE%(episode_number)02d.%(ext)s' is not a valid URL. Set --default-search "ytsearch" (or run  yt-dlp "ytsearch:WorldS%(season_number)02dE%(episode_number)02d.%(ext)s" ) to search YouTube
  File "yt_dlp\extractor\common.py", line 742, in extract
  File "yt_dlp\extractor\generic.py", line 2361, in _real_extract

does it work if you use yt-dlp -s jp  URL of video

It looks like there's a quoting issue with your arguments

Nope it works well with german as language.

does it work if you use yt-dlp -s jp URL of video

No, it makes this error: ERROR: [generic] 'jp' is not a valid URL.

Yeah because your quoting is wrong


Incorrect tool-tip and definition path for import statements in js file

VSCode Version: Code 1.11.2 (6eaebe3b9c70406d67c97779468c324a7a95db0e, 2017-04-13T08:03:11.395Z)
OS Version: Windows_NT ia32 10.0.14393
Extensions:

Extension
Author
Version

xml
DotJoshJohnson
1.7.0

EditorConfig
EditorConfig
0.9.3

java
redhat
0.1.0

html-preview-vscode
tht13
0.1.1

cordova-tools
vsmobile
1.2.7

Steps to Reproduce:

Clone https://github.com/reactjs/react-redux (edit: and run npm install)
Open the new repo folder with vscode.
Edit the file src/components/connectAdvanced.js.
Hover your mouse pointer over the string 'invariant' on the line that says import invariant from 'invariant'.
Press F12 or otherwise activate the Go to definition command.

In step 4, I expected to see a tool-tip displaying the path to my local node_modules directory inside of my repo. Instead, I was shown a path to the global node_modules/@types directory on my system.
Likewise in step 5, when I pressed F12 vscode opened a index.d.ts instead of opening the local node_modules/invariant/index.js file.
Knowing the types that vscode is using to display Intellisense is not really useful to me because they almost never match the version of the code that I'm actually working with. I would strongly prefer to at least be able navigate to the actual implementation easily instead of to the .d.ts definition file.
Thank you.

Thanks for the suggestion.
This is currently a limitation of how the TypeScript server that powers both our JS and TS intellisense works. We use d.ts files for module intellisense in order to provide rich suggestions without having to parse or understand the JavaScript implementation. This is done both for performance reasons and to leverage TS language features when describing JavaScript APIs
Ideally the d.ts files should be transparent in this whole process however. I know the TS team has discussed improving this and here are a few related issues on the TypeScript side:
https://github.com/Microsoft/TypeScript/issues/14479
https://github.com/Microsoft/TypeScript/issues/6209
Closing this as upstream since there is not much we can do on the VSCode side to fix this. I'll make sure the TS team is aware of the issue
Thanks


⚠ Processors has degraded performance

In 0066b7f, Processors ($STATUS_URL) experienced degraded performance:

HTTP code: 200
Response time: 45 ms

Resolved: Processors performance has improved in 90547fe.


Necessary additions for role subscriptions

Pull Request Etiquette

[x] I have checked the PRs for upcoming features/bug fixes.
[x] I have read the contributing guidelines.

Changes

[x] Internal code
[ ] Library interface (affecting end-user code)
[ ] Documentation
[ ] Other: _____

Closes Issue: NaN
Description
This adds a lot of new constants, introduced for role subscriptions.
References:

[x] discord/discord-api-docs#5724
[ ] discord/discord-api-docs#5541
It's unclear if this permission is actually relevant/released
[x] discord/discord-api-docs#5766
[x] discord/discord-api-docs#5771
[ ] discord/discord-api-docs#5828
We don't currently handle these flags at all

This needs to await PR merges on docs repo.

I'll review in more depth then, but from my initial scan this seemed fine


Adding network statistics to container metrics

Network statistics for containers is read from sysfs. Now we can get data such as no.of.bytes, packets, error and dropped packets both recieved and transferred from the container. The code is picked from docker's libcontainer.
I did also check the performance impact and found that the implementation takes very little time to collect the network statistics.

I've made a couple of inline comments about returning zero values along with errors and testing the network stats.
On the libcontainer front, please could you say which sections of code were copied from libcontainer? Also, IANAL but you don't seem to have followed section 4.c of the Apache v2 license in respect of the libcontainer code. I'm thinking specifically of copyright and attribution notice.

HI @glyn,
Thanks for the review, i would address the review comments.
Actually the code is not from libcontainer, but docker daemon's execdriver. the function i picked is readSysfsNetworkStats. Here is the link https://github.com/docker/docker/blob/master/daemon/execdriver/driver_linux.go#L66
Its a simple technique of reading from sysfs. we could write our own function to do the job. But definitely the idea came from comparative study of container metrics. On the other hand i didn't want to import the complete package of execdriver, just for some utility function.
Now i am clueless on how to go about it with the licensing issue. so request to take a call on this matter. please feel free to reject this PR, if it violates some license.
Thanks
@shashidharatd

You could avoid the IP issue by rewriting the function from scratch.

ok, i will write the utility function from scratch.

Thanks! If you can develop it with its own test, that would be great.

Hi @glyn,
Sorry for the delayed response. Writing the tests really took some time.
Now I have rewritten the code and added the code for testing metrics for network device.
I have used the ping utility to test number of packets sent and received. However getting accurate number of packets is difficult because there are ARP messages sent and received on the interface. I have set static ARP entries and disabled IPv6, but still some IPv6 ND RA transmissions are received
while testing. We cannot control the incoming packets, So i have used a delay of 3 seconds after the link is created and then tested the number of packets sent and received using ping utility.
Still had not tried to test error packets and drop packets statistics. Wanted to know if the testing method was OK or is there a better way to test? please suggest.
Thanks
@shashidharatd

That sounds like a reasonable way of doing integration testing. We need to avoid tests which will fail randomly, so it may be worth sending a 10 or so pings and then ensuring the number of packets is within a certain range.
You might consider adding a unit test where you inject a fake for accessing the network stats file. Then you can check the rest of the logic is ok. However this may not be worth it, depending on the complexity of that logic.

I would like to keep it generic and include only RxBytes and TxBytes and remove other stats which would be useful to implement in other backends.
As suggested i would test whether the number of bytes transferred and received are within certain range to account for other packets entering and exiting the interface.

Sounds good. (Presumably you meant "... remove other stats which would be difficult to implement in other backends".)

yes :) Sorry for the mess up in the sentence.

No problem - just checking to be on the safe side.

Thanks for your time and comments @julz will try to address the comments

Hi @julz,
I have reworked the code for your comments. I hope all the changes are according to your expectation. please check the fixes.
Thank you
@shashidharatd

Merged.
Thanks @shashidharatd!
@craigfurman and @julz

Thank you @julz  & @craigfurman


chore: Update hugr

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 90.88%. Comparing base (25fed3e) to head (d348a30).

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #392   +/-   ##
=======================================
  Coverage   90.88%   90.88%
=======================================
  Files          48       48
  Lines        5340     5340
=======================================
  Hits         4853     4853
  Misses        487      487

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Critical Vulnerabilities in aws inspector

Hi @abhirpat,
Last week in AWS inspector critical vulnerabilities has been identified in multiple lambdas. We ran the v6.0.1 version of cloudformation. Could you please help in this.

CVE-2024-7042 - @langchain/community, langchain

CVE-2024-7774 - langchain

Hi @anjugds ,
thanks for reporting this with detailed information. We will look into this and revert back.

Hi @abhirpat
We ran the v6.1.3 version of cloudformation and this is a critical vulnerability in our account. Could you please help us with this?

hi @fhoueto-amz , We are currently awaiting a resolution, as this issue is impacting a client project and is preventing further production deployments. Your prompt assistance is needed, as this involves a critical vulnerability with prompt injection and SQL injection.

@mfarnga v6.1.5 CVE Patch is out, see.
Closing this ticket.


Update build-docs.yml and added French and Arabic

Thank you for submitting a pull request!
Description
fixed if statement of deploying in CI because it does not deploy when crontab launch CI every day.
CIが毎日起動した際に、デプロイの部分が実行されなかったので、IF文の条件式からpushと指定していた部分を削除しました。
Type of Pull Request

[ ] Adding a feature
[ ] Fixing a bug
[ ] Maintaining documents
[x] Others (fixed bug of CI)

Verify the followings

[x] Code is up-to-date with the main branch
[x] No build errors after npm run build
[x] No lint errors after npm run lint
[x] No errors on using charites help globally
[x] Make sure all the exsiting features working well
[x] Have you updated documentation?

Refer to CONTRIBUTING.MD for more details.

Now translations for French and Arabic started in Transifex, I have added both languages into documentation through PR #86
@hfu @ubukawa @naogify Could you merge this PR into main branch?

Thank you! I confirmed French and Arabic have been adde. If other colleagues are also okay, I will merge them!


Add pyflakes version requirements

The version of pyflakes restricted by flake8 is too old.
And The old version causes an error when testing python3.5
related features.
The following error shows up when make test
Traceback (most recent call last):
  File "/Users/carlcarl/.pyenv/versions/3.5.0/lib/python3.5/multiprocessing/process.py", line 254, in _bootstrap
    self.run()
  File "/Users/carlcarl/.pyenv/versions/3.5.0/lib/python3.5/multiprocessing/process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "/Users/carlcarl/.pyenv/versions/aiohttp-3.5.0/lib/python3.5/site-packages/flake8/reporter.py", line 74, in process_main
    self._process_main()
  File "/Users/carlcarl/.pyenv/versions/aiohttp-3.5.0/lib/python3.5/site-packages/flake8/reporter.py", line 50, in _process_main
    self.input_file(filename)
  File "/Users/carlcarl/.pyenv/versions/aiohttp-3.5.0/lib/python3.5/site-packages/flake8/engine.py", line 95, in input_file
    return fchecker.check_all(expected=expected, line_offset=line_offset)
  File "/Users/carlcarl/.pyenv/versions/aiohttp-3.5.0/lib/python3.5/site-packages/pep8.py", line 1412, in check_all
    self.check_ast()
  File "/Users/carlcarl/.pyenv/versions/aiohttp-3.5.0/lib/python3.5/site-packages/pep8.py", line 1358, in check_ast
    checker = cls(tree, self.filename)
  File "/Users/carlcarl/.pyenv/versions/aiohttp-3.5.0/lib/python3.5/site-packages/flake8/_pyflakes.py", line 43, in __init__
    withDoctest=self.withDoctest)
  File "/Users/carlcarl/.pyenv/versions/aiohttp-3.5.0/lib/python3.5/site-packages/pyflakes/checker.py", line 294, in __init__
    self.handleChildren(tree)
  File "/Users/carlcarl/.pyenv/versions/aiohttp-3.5.0/lib/python3.5/site-packages/pyflakes/checker.py", line 547, in handleChildren
    self.handleNode(node, tree)
  File "/Users/carlcarl/.pyenv/versions/aiohttp-3.5.0/lib/python3.5/site-packages/pyflakes/checker.py", line 588, in handleNode
    handler = self.getNodeHandler(node.__class__)
  File "/Users/carlcarl/.pyenv/versions/aiohttp-3.5.0/lib/python3.5/site-packages/pyflakes/checker.py", line 458, in getNodeHandler
    self._nodeHandlers[node_class] = handler = getattr(self, nodeType)
AttributeError: 'FlakesChecker' object has no attribute 'ASYNCFUNCTIONDEF'

flake8 has dependency on pyflakes >= 0.8.1, < 0.10... not sure that this will work as expected. I guess better to ask flake8 developers to release of new version with update pyflakes and pep8

According to their discussions:
https://gitlab.com/pycqa/flake8/issues/68
https://gitlab.com/pycqa/flake8/issues/72
Upgrade to the newer version of pep8 may bring more problems.
The best I think is to specify the version of flake8 and pyflakes(maybe 1.0.0).
Currently, the version of flake8 is not specified. This may also cause problems if the newer version is released in the future.


Fix oldestPosts behavior

This requires additional testing.

This should fix #75. But let's deploy and test before the conclusion.

Will try again later.


Revert "Add status condition to MDR CSV sample"

This reverts commit 08149beedb65ab094cea7b2f546408791f0f88a0.
The sample is used in the UI to create MDR CR, which is not supposed to have status field by default.
see https://issues.redhat.com/browse/ECOPROJECT-1585

Even if the sample is provided by Console UI when creating MDR CR manually, it is not actually used for the creation, so it doesn't affect MDR logic.


[Proposal] Refactor generate_sae_table.py and pretrained_sae_loaders.py to share code

Proposal
Currently, the converters in pretrained_sae_loaders.py cannot be used in generate_sae_table.py because they return both a config file and the full SAE. As a result, we need to remember to manually re-create a lot of the functionality of the converters in generate_sae_table.py every time we add a new converter, and it's easy to forget / have these code paths be subtly different.
We should refactor the converter interface so there's a way to return only the config so that generate_sae_table.py can use the same converter code path as in pretrained_sae_loaders, and we don't need to manually duplicate code between these two files.
Checklist

[x] I have checked that there is no similar issue in the repo (required)

@chanind I was looking at this issue and made some changes which I think might be better than changes to the PretrainedSaeLoader interface. Basically, they make it so that each loader function calls a get function, which takes a repo_id: str and folder_name: str, and then generate_sae_table() just calls these get functions to get the configs. The changes were easy to make since it followed what people had started to do, and also I think the changes to PretrainedSaeLoader would mean two of the three returns args are Optional and are often ignored (_), which seems like a code smell.
Please let me know what me know what you think! I can open a PR either way.

Sorry for the slow reply! This looks like it's on the right track. I was hoping as well that we could have a function that can load the config given the model_info and will internally figure out the correct conversion function to use to do that, e.g. get_sae_config(model_info), so that function could be called both in the pretrained_sae_loaders and in generate_sae_table so we don't need to remember to add another if-statement into generate_sae_table everytime we add a new SAE converter; it would be great if that just happens for free. Otherwise, the changes in your diff look like they're moving in that direction, or could at least easily become what I'm talking about.


fix: Check if the user has another requests on the same day with any status

Related Issues

https://github.com/codescalers/CSHR/issues/378

When I create a request and have an approved request on the same day I get this.

This requested by https://github.com/codescalers/CSHR/issues/378#issuecomment-2309610511

When I create a request and have an approved request on the same day I get this.

@Mahmoud-Emad The message here is misleading as I'm having an approved request, not a pending one. can we rephrase it to The user has another request with the same dates. Please review the previous request before submitting a new one.

@zaelgohary Please check


Distinguish between admin and applicant profile creators

Description
Please explain the changes you made here.
Release notes
The title of the pull request will be used as the default release notes description. If more detail is needed to communicate to partners the scope of the PR's changes, use this release notes section.
Checklist
General
Read the full guidelines for PRs here

[ ] Added the correct label: < feature | bug | dependencies | infrastructure | ignore-for-release | database >
[ ] Created unit and/or browser tests which fail without the change (if possible)
[ ] Performed manual testing (Chrome and Firefox if it includes front-end changes)
[ ] Extended the README / documentation, if necessary

User visible changes

[ ] Followed steps to internationalize new strings
[ ] Wrote browser tests using the validateAccessibility method
[ ] Tested on mobile view. See mobile device mode
[ ] Manually tested at 200% size
[ ] Manually evaluated tab order

New Features

[ ] Add new FeatureFlag env vars to server/conf/helper/feature-flags.conf
[ ] Conditioned new functionality on a FeatureFlag
[ ] Wrote browser tests with the feature flag off and on, etc.

Instructions for manual testing
If instructions are needed for manual testing by reviewers, include them here.
Issue(s) this completes
Fixes #<issue_number>; Fixes #<issue_number>...

Current dependencies on/for this PR:

main

PR #5832

PR #5851   👈

PR #5730

PR #5731

PR #5816

This comment was auto-generated by Graphite.

Merge activity

Nov 15, 8:21 AM: Graphite rebased this pull request as part of a merge.


Introduce Graph Library

There needs to be an easy place to host and run graphs that we've built so far.
Make it easy for folks find useful graphs they might actually want to use.

Dupe of #233


fix(awsutil): properly determine partition for all supported regions

Overview
Refactors code to allow for all regions to be supported for the AWS_DEFAULT_REGION environment variable by doing a lookup to determine it's proper partition. It also happens to simplify the code as well.
Resolves #142

:tada: This PR is included in version 3.0.0-beta.43 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:

:tada: This PR is included in version 3.0.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


[Feature Request] Delete organisation from UI

Problem description
I can't delete an organisation from the ui either from the organisation itself or root.
Desired solution
An organisation delete button.

This should be implemented in the backend before we can do this in the UI. Depends on https://github.com/vantage6/vantage6/issues/241


micsthepick: Blacklist high\Wtech\Wcbd

micsthepick requests the blacklist of the keyword high\W*tech\W*cbd. See the Metasmoke search here and the Stack Exchange search here.

high[^A-Za-z0-9]*tech[^A-Za-z0-9]*cbd has been seen in 0 true positives, 0 false positives, and 0 NAAs.

Coverage remained the same at 65.558% when pulling 1240cd1bf3ebac0283b9e6552ac7a27b3779236d on auto-blacklist-1522485787.3495157 into 4735a139eca4bd28a632143467659eb8b67a7592 on master.

!!/approve


cmd/go: critical path scheduling

Currently go tool schedules work in topological order, this leads to suboptimal
parallelization. It's possible that towards end, very few work items are available
leading to idle processors.

Critical path scheduling gives much better results in this respect:
http://en.wikipedia.org/wiki/Critical_path_method

Here is a prototype CL:
https://golang.org/cl/13235046/diff/3001/src/cmd/go/build.go
I've observed up to 2x 'go test -a std' speedup on a 32-way machine.

We can also assign meaningful weights to work nodes: e.g. test > link > compile.
It would also be super useful to somehow assign weights to test nodes proportional to
test run time (e.g. runtime tests take significantly more than than most packages). This
will have significant effect even on low-core machines (as runtime tests will start
first).

@dvyukov are you planning to migrate that CL to gerrit and mail? Does it speed up make.bash as well?

Is anybody planning to review it to +2 state?

I promise to review it thoroughly (although I'll be in and touch until late March). And rsc asked for it to be mailed. If you don't want to bother, do you mind if I take it over?
The existing CL looks possibly incomplete. assignPriorities contains (lines 648-649):
if a.priority < a1.priority+1 {
}

Take over it.
That if should be:
if a.priority < a1.priority+a.weight {
a.priority = a1.priority+a.weight
}
Also weight must be more sensible, that is, represent real cost of action. Something along the lines of:
link > compile
run test > link
Fake instant actions should have weight << everything else.
The goal here is to avoid serial tail of actions that run one after another delaying whole thing.
Ideally 'run test' weights represent actual execution time, so that runtime test starts first rather then starts late and delays all.bash. This may be possible in 'go test' continuos mode that monitors filesystem changes. But that's of course not a part of this change.

And compiling anything involving cgo should be even heavier weight than
compiling any other packages.
If we reorder tests, do we need to preserve the order of showing the
results?
I think we tried hard to show a sorted result for "go test -short std".

And compiling anything involving cgo should be even heavier weight than
compiling any other packages.

good idea

If we reorder tests, do we need to preserve the order of showing the
results?

absolutely
but that is already ensured by chaining printing actions with dependencies, otherwise they would be printed out of order even today with p>1
so you don't need to do anything special here

I still plan to take over this CL. I am just waiting for the compiler performance to stabilize a bit, so that I can get more realistic measurements of the relative speeds of different kinds of actions.

This is pretty high priority for our all-compile builder. I increased it from 2 cores to 16 cores and we got almost no additional speed out of the builder.

I'll put this back in my queue (next step is attaching some more realistic weights.), but I'm unlikely to make much progress this week.
In the meantime, if you pull in https://go-review.googlesource.com/#/c/8918 as-is, does it help? It didn't show much improvement for me locally.

@bradfitz any numbers on that CL on a big machine?
@dvyukov @minux I spent a bit of time looking at tool exec times for 6g, 6l, and asm. asm is very fast, linking is very slow, but 6g has a huge amount of variability--two orders of magnitude. The variability goes down a tiny bit if you look at the number of files to compile, but not much. The variability is down to one order of magnitude if you look at the number of bytes to be compiled, but I don't think that we want to stat every file before getting started. Any other ideas for things to look at? As is, I think the best we can do for weights is very rough estimates, like fake=1, asm=10, gc=100, link=1000, cgo=1000. Any other ideas?

We already do stat every file before getting started (to compute staleness). So maybe that information can be saved somewhere?

Number of files look like a good first approximation.
Does this thing help with 8-16 cores?

Probably also open files to read build tags.

Sorry, I haven't had any free time to test this.

No prob. I'm in the middle of running some tests now, actually, and it looks better than before. Expect a minimal CL and more comments here later today.

My earlier optimism was the result of a bug.
I have 8 logical (4 real) CPUs on my laptop. It spent the afternoon running three build tests in a loop: go build -a std, go test -short std, all.bash. I had everything else shut down, measured wall time, and ran the results through a t-test. At 50 iterations (20 for all.bash, which is SLOW), the CL showed no statistically significant improvement, and the means were within 1%.
I then hacked in the best available predictor: Store previous action times on disk. Even with almost perfect weights, I got pretty much the same results.
Furthermore, this re-ordering hurts the experience of running all.bash, as there are long stalls between test outputs, which then come in spurts.
Maybe this helps with lots more cores than I have--Brad, I guess I would like your help confirming that after all. As it is, though, I don't feel comfortable mailing the CL, since I have no way to demonstrate its value.

What has changed? Dmitry's initial post reported a up
to 2x improvement on go test -a std. Is it because the
compiler is now written in Go and are slower?

Dmitry's initial post said 2x speed-up on a 32 core machine. Maybe my machine (which is probably typical) doesn't have enough cores to benefit.

Where is the CL? I can test on a 16-way machine.

CL 8918. It is a straight port of Dmitry's original CL.

My initial observations:
time taken for go test -a std is not a good performance
measures for this change because there are some packages
(e.g. math/big, runtime) that take a very long time to test,
so the serial portion of the runtime is significantly skewed
by those packages.
Comparing the time of go install -a std cmd on the 16-way
machine showed no significant impact on run time (~28s,
CPU usage ~700%).
Comparing the time of go install -a std cmd golang.org/x/tools/cmd/godoc
on the same machine still doesn't show any much impact
(~34s, CPU usage ~600%)
The critical path scheduling will only show benefit where
are many floating tasks with positive slacks, perhaps the
dependency graph has changed a lot after the Go rewrite
of the toolchain? Or perhaps there are unneeded dependency
edges in the graph?

I've done some testing using time go build -a -p 4 std.
Before:
real	0m11.527s
real	0m11.478s
real	0m11.353s
real	0m11.544s
real	0m11.586s

After:
real	0m10.443s
real	0m10.496s
real	0m10.487s
real	0m10.348s
real	0m10.451s

There is clear 10% win.
I've also modified cmd/go to print number of executing actions every 500ms to confirm the win.
Before:
EXECUTING 1
EXECUTING 1
EXECUTING 1
EXECUTING 4
EXECUTING 4
EXECUTING 4
EXECUTING 4
EXECUTING 4
EXECUTING 4
EXECUTING 4
EXECUTING 4
EXECUTING 4
EXECUTING 4
EXECUTING 4
EXECUTING 4
EXECUTING 4
EXECUTING 2
EXECUTING 1
EXECUTING 1
EXECUTING 1
EXECUTING 1
EXECUTING 2

After:
EXECUTING 1
EXECUTING 1
EXECUTING 1
EXECUTING 4
EXECUTING 4
EXECUTING 4
EXECUTING 4
EXECUTING 4
EXECUTING 4
EXECUTING 4
EXECUTING 4
EXECUTING 4
EXECUTING 4
EXECUTING 4
EXECUTING 4
EXECUTING 4
EXECUTING 4
EXECUTING 1
EXECUTING 1
EXECUTING 4

There is clear difference towards end of build -- old code leads to lower parallelism because previous actions were scheduled badly.
Also, the win highly depends on work graph structure. It is possible to come up with dependency structure which will be handled arbitrary inefficiently by the old code. So some users can see higher speedup than we see on std lib. Moreover, currently scheduling depends on package names (or how does it order them now?), which is just weird. You can increase/reduce build speed by renaming packages.
Also, better scheduling will have more impact when we make actions finer-grained. In particular split cgo compilation and maybe asm compilation.
So I am sure that this is a positive change and we should do it. The additional complexity is minimal and very localized.

There is clear 10% win.

I don't know why I can't replicate that speed-up. Maybe you should take the CL back over, with my apologies. It doesn't make sense for me to continue to work on it when I cannot observe its impact (and thus can't tell whether my changes hurt or help).

better scheduling will have more impact when we make actions finer-grained.

Agreed. It is very coarse right now. And having thought about it more, simply saving the execution time of past identical actions might not be a bad long-term strategy for weight estimation.

In particular split cgo compilation and maybe asm compilation.

asm compilation is very fast. At least in the stdlib, there's very little cgo, so I don't think this will impact all.bash much, but it could make a big difference for heavy cgo users.

@josharian have you tried running exactly "time go build -a -p 4 std"?
What does the following debug output shows on your machine?
diff --git a/src/cmd/go/build.go b/src/cmd/go/build.go
index fda126b..ceca4c5 100644
--- a/src/cmd/go/build.go
+++ b/src/cmd/go/build.go
@@ -18,6 +18,7 @@ import (
        "os"
        "os/exec"
        "path"
+       "sync/atomic"
        "path/filepath"
        "regexp"
        "runtime"
@@ -1139,6 +1140,12 @@ func (b *builder) do(root *action) {
        if buildN {
                par = 1
        }
+       var executing uint32
+       go func() {
+               for range time.NewTicker(500*time.Millisecond).C {
+                       fmt.Printf("EXECUTING %v\n", executing)
+               }
+       }()
        for i := 0; i < par; i++ {
                wg.Add(1)
                go func() {
@@ -1154,7 +1161,9 @@ func (b *builder) do(root *action) {
                                        b.exec.Lock()
                                        a := b.ready.pop()
                                        b.exec.Unlock()
+                                       atomic.AddUint32(&executing, 1)
                                        handle(a)
+                                       atomic.AddUint32(&executing, ^uint32(0))
                                case <-interrupted:
                                        setExitStatus(1)
                                        return

Too late for Go 1.5.

I'm spending what time I have on SSA. Hopefully someone else wants to pick this back up.

My WIP CL is 8918.

I fooled around with josharian's CL during a slow evening, I got nowhere but I'll report my findings, just to add another datapoint. This is on a 4-core intel machine.
First of all I applied  dvyukov patch (the one that prints the number of currently executing jobs every 500ms), and run go build -a -p 4 std .
Before:
EXECUTING 1
EXECUTING 1
EXECUTING 4
EXECUTING 4
EXECUTING 4
EXECUTING 4
EXECUTING 4
EXECUTING 4
EXECUTING 4
EXECUTING 4
EXECUTING 4
EXECUTING 4
EXECUTING 4
EXECUTING 4
EXECUTING 4
EXECUTING 4
EXECUTING 4
EXECUTING 4
EXECUTING 4
EXECUTING 4
EXECUTING 3
EXECUTING 1
EXECUTING 1
EXECUTING 1
EXECUTING 4
EXECUTING 1

after:
EXECUTING 1
EXECUTING 1
EXECUTING 4
EXECUTING 4
EXECUTING 4
EXECUTING 4
EXECUTING 4
EXECUTING 4
EXECUTING 4
EXECUTING 4
EXECUTING 4
EXECUTING 4
EXECUTING 4
EXECUTING 4
EXECUTING 4
EXECUTING 4
EXECUTING 4
EXECUTING 4
EXECUTING 4
EXECUTING 4
EXECUTING 4
EXECUTING 4
EXECUTING 1
EXECUTING 4

the parallelism of the last part of the job is indeed higher.
Unfortunately, as minux and josharian, I wasn't able to observe any significant speedup. For time go build -a -p 4 std there's no statistical difference between tip and the CL. For make.bash I observed a modest 1.5% speedup. I didn't try with all.bash.

Thanks for the null results. They're helpful. Deprioritizing.


Fix minor documentProps error

Line 24:
const { documentProps } = pageContext;
const title = (documentProps && documentProps.title) || 'Vite SSR app';

I believe this should be const { documentProps } = pageContext.exports;
Which also needs to be resolved downstream in ./types.ts

Indeed.
Thanks for the PR!


Report doesn't update on changes

Hi! I love this project, thank you for creating and open sourcing it.
I've installed the Lexi GitHub Action and it's working great. However, if you make changes to a pull request after opening it, although the action runs again, the report doesn't update, and there's no new report added.
I'm fairly new to using GitHub Actions, so maybe I just need to configure something there, but I'm unsure. Is there a way to update the displayed report every time the action runs? And is this something I should be doing in the config, or you need to be doing in the action itself?

Hi @StarfallProjects!
The scores displayed in the GitHub action represent the metrics from the most recent commit.
Lexi displays a report for each commit. I see now that we may need to document this, because it is a little bit hidden.
To view metrics on changes after each commit in a pull request: At the top right of the GitHub action, click edited, then select the commit report that you want to view.

Example of previous commit

Thank you!

We've added docs for this to the readme.
Thanks for reaching out and for the feedback @StarfallProjects!