Why does this crate use packed representation in a bunch of places?

A colleague of mine got this linker error today:
ld: warning: alignment (1) of atom 'l_anon.e8717737f2aafb2fc9f63ff91b30211b.2' is too small and may result in unaligned pointers
ld: warning: pointer not aligned at address 0x10464FB19 ('l_anon.e8717737f2aafb2fc9f63ff91b30211b.2' + 1 from /Users/[...]/libmatrix_sdk_ffi.a(const_panic-ef6c6c5eecaca335.const_panic.519cbb813ef9124e-cgu.8.rcgu.o))
ld: unaligned pointer(s) for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

First I thought that it seems really unlikely for const_panic to actually be related to the problem, but when looking at the codebase a bit I found a few uses of #[repr(packed)] as well as one Packed<&'a str> in PanicVariant that seems highly suspicious. What is the reason for using packed representation, in particular there? Is it just for efficiency?

What possible problem could this cause? Since I'm not using unsafe to pack the pointer, this can't cause UB (barring unsoundness bugs in rustc).

I'm far from an expert on linking, so I don't really know why the linker ever cares about alignment.
But separately, I wouldn't be surprised at all if #[repr(packed)] can (still) cause UB even without unsafe accesses, similar to #[no_mangle].

Well, #[no_mangle] triggers the unsafe_code lint while #[repr(packed)] doesn't, so it's considered safe by the compiler.
no_mangle example
https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=d9a41beab1c5737a04dc4820f54184c9
#![deny(unsafe_code)]

#[no_mangle]
fn foo(){}

error: declaration of a `no_mangle` function
 --> src/lib.rs:3:1
  |
3 | #[no_mangle]
  | ^^^^^^^^^^^^
  |
  = note: the linker's behavior with multiple libraries exporting duplicate symbol names is undefined and Rust cannot provide guarantees when you manually override them
note: the lint level is defined here
 --> src/lib.rs:1:9
  |
1 | #![deny(unsafe_code)]
  |         ^^^^^^^^^^^

#[repr(packed)] example
This example builds without triggering the lint:
https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=0bd9127194d0d9a1b14e35e976767f07
#![deny(unsafe_code)]

#[repr(packed)]
pub struct Foo(pub u8, pub u32, pub &'static str);

pub static F: Foo = Foo(3, 5, "hello");

Right.. Well the error from the issue description has now infected our CI and my small PR fixes it there as well.

CI run with const_panic patched to use this git repo, latest main git rev still reproduces the issue
CI run with my patch works

If you don't want to merge without actually finding out why it might be wrong, I can understand. Maybe my colleage who has started to implement their own linker has some insights, I'll ask them.

If you don't want to merge without actually finding out why it might be wrong

Yes, I want to know why it happens, to decide whether the PR is a fix for a const_panic bug or a workaround for a compiler bug.


syz-manager:  change vm count and process count without restarting syzkaller

Updating the config requires syzkaller to be completely restarted, which can take quite a while with many VMs running.
It would be nice to be able to reload the config file whilst syzkaller is still running.  If the VM/process counts have increased and other settings have stayed the same, it can just add/kill vms/processes.

dup of #2312


Code Formatting

Set up automated code formatting for the project (Prettier?)
Set up a GitHub action to check formatting on each commit (PR?)

Prettier and a linter to automate the formatting process would get this done pretty easy! There is also a vscode plugin for prettier. We should probably use ES Lint as it works for typescript and javacript.


Rollup failed to resolve import with monorepo

Describe the bug
I have a package (A) inside monorepo import lodash/get, and one package (B) has A as dependency, when building B, I faced that error.
Reproduction

Init monorepo with pnpm
create workspace A and workspace B
install lodash.get for A
init B with vitejs
install A for B
run pnpm build for B

System Info
System:
    OS: macOS 10.15.7
    CPU: (12) x64 Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
    Memory: 2.87 GB / 32.00 GB
    Shell: 5.7.1 - /bin/zsh
  Binaries:
    Node: 16.7.0 - /usr/local/bin/node
    Yarn: 1.22.5 - ~/.yarn/bin/yarn
    npm: 7.20.3 - /usr/local/bin/npm
    Watchman: 4.9.0 - /usr/local/bin/watchman
  Browsers:
    Brave Browser: <IP_ADDRESS>
    Chrome: 96.0.4664.55
    Edge: 96.0.1054.41
    Firefox: 94.0.2
    Safari: 15.1
  npmPackages:
    @vitejs/plugin-react-refresh: ^1.3.6 => 1.3.6
    vite: ^2.6.14 => 2.6.14

Used Package Manager
pnpm
Logs
[vite]: Rollup failed to resolve import "lodash/get" from "../../../libs/react-core-component/lib/Form/FormInput.js".
This is most likely unintended because it can break your application at runtime.
If you do want to externalize this module explicitly add it to
`build.rollupOptions.external`
error during build:
Error: [vite]: Rollup failed to resolve import "lodash/get" from "../../../libs/react-core-component/lib/Form/FormInput.js".
This is most likely unintended because it can break your application at runtime.
If you do want to externalize this module explicitly add it to
`build.rollupOptions.external`

Validations

[X] Follow our Code of Conduct
[X] Read the Contributing Guidelines.
[X] Read the docs.
[X] Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
[X] Make sure this is a Vite issue and not a framework-specific issue. For example, if it's a Vue SFC related bug, it should likely be reported to https://github.com/vuejs/vue-next instead.
[X] Check that this is a concrete bug. For Q&A open a GitHub Discussion or join our Discord Chat Server.
[X] The provided reproduction is a minimal reproducible example of the bug.

Why do you import lodash/get not lodash.get ?

@yoyo930021 oh, I just changed to lodash.get and it fixed. Because I remember I have other project that used vite and can build when using lodash/get too. Don't know why, also when pnpm start, I can use it without issue 😂😂😂


[Question]: Is there a way to have the system automatically choose whether to use the LLM or the RAG?

Question Validation

[X] I have searched both the documentation and discord for an answer.

Question
Now there is a scenario where a man enters a question and wants to get an answer, can this answer be automatically distributed to the RAG system to do an intent recognition function(All information is retrieved from local storage).
When the man enters another question, this answer is distributed to the LLM, which gives the answer.

@seanxuu this is pretty much what an agent does
https://docs.llamaindex.ai/en/stable/module_guides/deploying/agents/root.html


Question: Using electron service with WDIO in a script

Hello! I'm having an issue with trying to get the service up and running when using WDIO in a script running Jest. I've tried a couple of things, and the closest I got was doing the following:
Before test, start chromedriver:
const args = [       chromedriverPath,       '--remote-debugging-port=9515',       '--url-base=/',       '--verbose',     ];     const cdProcess = child_process.spawn(process.execPath, args);
Then create my remote:
const app = await remote({       logLevel: 'trace',       services: [         [           'electron',           {             appPath: appPath,             appName: appName,             chromedriver: {               port: 9515,               chromedriverCustomPath: chromedriverPath,               logFileName: 'wdio-chromedriver.log',             },           },         ],       ],       capabilities: {         browserName: 'chrome',         'goog:chromeOptions': {           binary: appExecutable,         },       },       port: 9515,     });
This will successfully start my electron application and let me do native WDIO things (app.saveScreenshot() works as expected), however I get the following error when trying to use electronApp:
TypeError: app.electronApp is not a function     > 90 |     const appName = await app.electronApp('getName');          |                               ^       91 |     console.log(appName);
I'm admittedly VERY new to WDIO so I'm fully aware that I may be missing something silly, but any advice I could get would be very much appreciated.
Thanks!

@arossetter thanks for filing the issue.
Unfortunately this service only works when using the wdio testrunner. I suggest to do that because your tests will look exactly the same as Jest test, given that WebdriverIO uses the same assertion library under the hood.
Let me know if you have further questions.


Sensitive terminology in Zowe

With the current climate, it was brought to my attention that it might be prudent to proactively avoid terms like whitelist/blacklist or master/slave in open source projects. Examples: https://github.com/rails/rails/issues/33677 https://www.postgresql.org/message-id/4DC14C0F-2211-48A5-A45E-3671BF97E9D6%40excoventures.com
Here are results from a quick github search for zowe: https://github.com/search?q=org%3Azowe+whitelist+OR+blacklist+OR+slave&type=Code

Given our target users, it doesn't help to follow the crowd without analyzing the familiarity and understand-ability of proposed terminology changes.
I'm curious about what terms veteran mainframers know, versus millennials, and how can we satisfy them both without invoking a protest.
One of the UI proposals we had in the beginning of Zowe was a form of "internationalization" in which you could switch the terminology of the UI around on the fly, such that some people would see "STCs" and "cylinders" while others would see "services" and "bytes", or similar.
My recommendation is to collect a list of terminologies the world is currently being sensitive about, and see what words with similar meanings exist. Then, send out a survey to find out what our end users understand, because for example, replacing whitelist/blacklist with rejectlist/acceptlist is bizarre to me. I've never seen rejectlist/acceptlist. On the other hand, I have seen include/exclude, and would find those to be equal in meaning. Let's collect a list of such things before making changes while the world is still uncertain as to what is hit or miss.

Thanks for the zowe search. I thought we had no use of 'slave', but we do with Jenkins!
I was just thinking about how some words we cannot change if they are a part of a library we use rather than our code itself. For example, nodejs https://nodejs.org/dist/latest-v12.x/docs/api/cluster.html#cluster_cluster_ismaster
Interestingly, their terminology is master&worker.
master has different meanings, such as with branches where master has nothing to do with slave.
so I'd say master alternatives should be broken down by topic:
master as a source code terminology:
primary
default
main
(weird svn terminology) trunk
release <- my favorite because we as a community agreed that work happens on "staging", and our current master branches are supposed to be untouched, just containing release quality code.
master & slave software references:
master and worker
leader and worker
leader and follower
...
primary and secondary is not valid. it implies a failover architecture, which is not what master & slave designs do. They are more about delegator & delegatee, or orchestrator & follower... and many other clunky words. leader, worker, and follower are all pretty simple to me.

I just ran a test, and it appears that git understands what branch is considered the default, so you can have master not exist without implicit git clone commands suddenly failing.
Not sure how far github has gone with renaming, as their instructions when making a new repo currently say this:
echo "# main-test" >> README.md
git init
git add README.md
git commit -m "first commit"
git remote add origin git@github.com:1000TurquoisePogs/main-test.git
git push -u origin master

I'd like to revisit this as part of the next ZLC call.  Broadcom feels very strongly about eliminating the use of inappropriate and insensitive terminology, especially focused on but not limited to those related to black/white and master/slave.  Our Mainframe Software Division is making this a priority as are the other divisions of Broadcom because it's the right thing to do.  I would imagine that a significant number of members and leaders within the Zowe community likely feel the same way.  I'd like to request that the ZLC provide guidance to the squads to complete their own analysis of the effort required and come back with an estimate of when they feel they would be able to complete it.   I'm not suggesting setting a deadline or causing anyone to miss any committed targets.  I'm simply arguing for guidance to make "language matters" a priority to all of the squads.  @armstro if you could please queue this up for next Wednesday I'd appreciate it.  Thank you!

Happy to add this to the next ZLC agenda - I guess I ask for some "prep" questions for the discussion to make this actionable -

I think Doc squad has already taken some actions on this - need to confirm and understand what has been done - maybe @nannanli or<EMAIL_ADDRESS>can report
Doc is only one aspect and the code itself is another - I am unclear what tools exists to help with the code (I'm assuming just like security scans or license scans there is (or will be) sensitivity scans?
I recall @jmertic thinking OMP might have a panel for guidance - maybe he can update us
Since we are getting ready to update zowe.org - would be good to try any tools or scans to try to catch issues before we go live
If there are not OMP or other public tools - can we see if any of our participating companies have something to contribute we could use
Is there some education we can offer the community - maybe a badge - to help increase awareness in the community (and not rely on tools to catch issues after the fact but prevent issues)

So happy to add to agenda - just want to provide useful guidance.

@armstro  The doc squad discussed the following terms and replacements that's suggested by IBM. We searched for these terms in the docs and found no occurrences in docs. However, master is a default term in GitHub (such as master branch but it's not paired with slave). Listing them here so they could be discussed in the ZLC call.

Certain uses of the term "master" are acceptable when it is not paired with the term "slave". If the term master is used in a context such as "Master Data Management" where there is no "Slave Data Management" counterpart, then that use of the term master is considered acceptable and not offensive. Additional examples where the term master is acceptable, assuming it is not used in conjunction with a related term including the word slave: masterpiece, Scrum Master

Discussed in today's ZLC.....squads were asked to assess in this PI the size the effort for any terminology changes in the source code for prioritization in future PI. Doc squad already well on the way of changes listed above.

Following on from @nannanli comment above.
On October 1st GitHub changed the default branch name from master to main, meaning any new repositories will have this as default. On the ZLC call someone also mentioned Git itself will make this change soon.
It would be worth each squad also investigating what would be required to make this change as well.

I will note that there is a launch of the Inclusive Naming Initiative today which might be useful for this effort to engage with. The first community meeting will be held via https://www.twitch.tv/cloudnativefdn. More details on the initiavite are at https://inclusivenaming.org/


[GSSOC'23] Update README with GSSoC Banner and Add "Back to Top" Link

Expected Behavior
I expected the banner in the README to accurately represent the GSSoC 23 event, showcasing the GSSoC logo, tagline, or related artwork instead of the "Hacktoberfest" banner.
Current Behavior
Currently, the README displays the "Hacktoberfest" banner instead of the expected GSSoC 23 banner. This creates a mismatch between the displayed content and the actual event the project is associated with.
Proposed Changes: Update README with GSSSoC Banner and Add "Back to Top" Link
Description
This issue involves updating the README file of the project. The current banner in the README displays "Hacktoberfest," which needs to be replaced with a banner representing "GSSSoC." Additionally, a "Back to Top" link should be added at the end of the README for improved navigation.
Steps to Implement
Replace the existing "Hacktoberfest" banner in the README with a "GSSSoC" banner that aligns with the project's branding.
Add a "Back to Top" link at the end of the README, enabling users to easily navigate back to the top of the document.
Expected Outcome
After implementing these changes, the README will accurately reflect the project's association with GSSSoC. The "Back to Top" link will provide convenient navigation for users browsing the document.

Hi , I am a contributor in GSSOC'23, Please assign this issue to me

It has been assigned to you, you can start working on it!

Hi @agamjotsingh18, I am a contributor in GSSOC'23, Please assign this issue to me

It has been assigned to you, you can start working on it!

Hi @agamjotsingh18, I am a contributor in GSSOC'23, Please assign this issue to me

Yes, will start working on it


word correct

Please sign the DCO as per the contribution guidelines.

Codecov Report
Patch coverage has no change and project coverage change: -0.36 :warning:

Comparison is base (f65a33d) 56.31% compared to head (2f11810) 55.95%.

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3418      +/-   ##
==========================================
- Coverage   56.31%   55.95%   -0.36%
==========================================
  Files         102      110       +8
  Lines        7281    11050    +3769
==========================================
+ Hits         4100     6183    +2083
- Misses       2535     4178    +1643
- Partials      646      689      +43

see 212 files with indirect coverage changes

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


add unit tests and made some code more concise

rather minor changes

Thanks for the cleanup, and clever corner routine :)


Add error messaging for trying to create a game without having a game selected.

Intent
This changeset adds a translation key for an error message that displays when a user attempts to create a new game through the "new game" modal without having a system selected.

Looks good, I'll merge this when the related PR merges.


Inconsistent progress indicator of print commands

Some print commands indicate progress with increasing number of users:
gam print users
Getting all Users in G Suite account (may take some time on a large account)...
Got 500 Users<EMAIL_ADDRESS>-<EMAIL_ADDRESS>Got 1000 Users<EMAIL_ADDRESS>-<EMAIL_ADDRESS>Got 1500 Users<EMAIL_ADDRESS>-<EMAIL_ADDRESS>...
But some print the value 500 all the time:
gam print aliases
Getting all User Aliases in G Suite account (may take some time on a large account)...
Got 500 Users<EMAIL_ADDRESS>-<EMAIL_ADDRESS>Got 500 Users<EMAIL_ADDRESS>-<EMAIL_ADDRESS>Got 500 Users<EMAIL_ADDRESS>-<EMAIL_ADDRESS>...
I believe the first one is much better, so I'd propose to change all progress indicators to that format.

Jay,
In my versions I've standardized on always showing running totals; do you want me to do the same in yours?
Ross

This is fixed in latest version, thanks Ross.


Menu reopening when using mask in text-input on range calendar

Describe the bug
When i use the v-mask to mask the field in date range, letting the user type the date without selecting it in the calendar, the menu keeps reopening when the date is selected and only closes on the blur event. Obs: I use the v-mask with the calendar and it works, but, it's the single calendar, not the range.
To Reproduce
Steps to reproduce the behavior:

Use v-mask in the date range
Use text-input prop in the date range
Use this mask on v-mask (v-mask="'##/##/#### - ##/##/####'")
Type a date and select it.

Expected behavior
It was expected that the calendar would close when selecting the date, without reopening and closing only in the blur.
Screenshots
Desktop & mobile (please complete the following information):

Chrome
4.5.1 and 5.1.2

Video below:
https://github.com/Vuepic/vue-datepicker/assets/78701182/27e75370-2342-4e6b-a5a7-807ae9a68986

Same issue occurred with v-maska package while using single date mode

Found little tricky solution: Close date picker menu programmatically once mask is completed. I think there is an issue with v-maska library.


[Bug]: Qwen/Qwen2-72B-Instruct 128k server down

Your current environment
PyTorch version: 2.3.0+cu121
Is debug build: False
CUDA used to build PyTorch: 12.1
ROCM used to build PyTorch: N/A
OS: Ubuntu Jammy Jellyfish (development branch) (x86_64)
GCC version: Could not collect
Clang version: Could not collect
CMake version: version 3.29.5
Libc version: glibc-2.35
Python version: 3.10.14 (main, May  6 2024, 19:42:50) [GCC 11.2.0] (64-bit runtime)
Python platform: Linux-5.10.134-15.al8.x86_64-x86_64-with-glibc2.35
Is CUDA available: True
CUDA runtime version: Could not collect
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration:
GPU 0: NVIDIA L20
GPU 1: NVIDIA L20
GPU 2: NVIDIA L20
GPU 3: NVIDIA L20
GPU 4: NVIDIA L20
GPU 5: NVIDIA L20
GPU 6: NVIDIA L20
GPU 7: NVIDIA L20
Nvidia driver version: 535.161.07
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True
CPU:
Architecture:                    x86_64
CPU op-mode(s):                  32-bit, 64-bit
Address sizes:                   52 bits physical, 57 bits virtual
Byte Order:                      Little Endian
CPU(s):                          128
On-line CPU(s) list:             0-127
Vendor ID:                       GenuineIntel
Model name:                      Intel(R) Xeon(R) Gold 6462C
CPU family:                      6
Model:                           143
Thread(s) per core:              2
Core(s) per socket:              32
Socket(s):                       2
Stepping:                        8
CPU max MHz:                     3900.0000
CPU min MHz:                     800.0000
BogoMIPS:                        6600.00
Flags:                           fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cat_l2 cdp_l3 invpcid_single intel_ppin cdp_l2 ssbd mba ibrs ibpb stibp ibrs_enhanced tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm rdt_a avx512f avx512dq rdseed adx smap avx512ifma clflushopt clwb intel_pt avx512cd sha_ni avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local split_lock_detect avx_vnni avx512_bf16 wbnoinvd dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp hwp_pkg_req avx512vbmi umip pku ospke waitpkg avx512_vbmi2 gfni vaes vpclmulqdq avx512_vnni avx512_bitalg tme avx512_vpopcntdq rdpid bus_lock_detect cldemote movdiri movdir64b enqcmd fsrm uintr md_clear serialize tsxldtrk pconfig arch_lbr amx_bf16 avx512_fp16 amx_tile amx_int8 flush_l1d arch_capabilities
Virtualization:                  VT-x
L1d cache:                       3 MiB (64 instances)
L1i cache:                       2 MiB (64 instances)
L2 cache:                        128 MiB (64 instances)
L3 cache:                        120 MiB (2 instances)
NUMA node(s):                    2
NUMA node0 CPU(s):               0-31,64-95
NUMA node1 CPU(s):               32-63,96-127
Vulnerability Itlb multihit:     Not affected
Vulnerability L1tf:              Not affected
Vulnerability Mds:               Not affected
Vulnerability Meltdown:          Not affected
Vulnerability Mmio stale data:   Not affected
Vulnerability Retbleed:          Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl and seccomp
Vulnerability Spectre v1:        Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:        Mitigation; Enhanced IBRS, IBPB conditional, RSB filling, PBRSB-eIBRS SW sequence
Vulnerability Srbds:             Not affected
Vulnerability Tsx async abort:   Not affected
Versions of relevant libraries:
[pip3] numpy==1.26.4
[pip3] nvidia-nccl-cu12==2.20.5
[pip3] torch==2.3.0
[pip3] transformers==4.41.2
[pip3] triton==2.3.0
[conda] numpy                     1.26.4                   pypi_0    pypi
[conda] nvidia-nccl-cu12          2.20.5                   pypi_0    pypi
[conda] torch                     2.3.0                    pypi_0    pypi
[conda] transformers              4.41.2                   pypi_0    pypi
[conda] triton                    2.3.0                    pypi_0    pypi
ROCM Version: Could not collect
Neuron SDK Version: N/A
vLLM Version: 0.5.0
vLLM Build Flags:
CUDA Archs: Not Set; ROCm: Disabled; Neuron: Disabled
GPU Topology:
GPU0    GPU1    GPU2    GPU3    GPU4    GPU5    GPU6    GPU7    CPU Affinity    NUMA Affinity   GPU NUMA ID
GPU0     X      PIX     NODE    NODE    SYS     SYS     SYS     SYS     0-31,64-95      0               N/A
GPU1    PIX      X      NODE    NODE    SYS     SYS     SYS     SYS     0-31,64-95      0               N/A
GPU2    NODE    NODE     X      PIX     SYS     SYS     SYS     SYS     0-31,64-95      0               N/A
GPU3    NODE    NODE    PIX      X      SYS     SYS     SYS     SYS     0-31,64-95      0               N/A
GPU4    SYS     SYS     SYS     SYS      X      PIX     NODE    NODE    32-63,96-127    1               N/A
GPU5    SYS     SYS     SYS     SYS     PIX      X      NODE    NODE    32-63,96-127    1               N/A
GPU6    SYS     SYS     SYS     SYS     NODE    NODE     X      PIX     32-63,96-127    1               N/A
GPU7    SYS     SYS     SYS     SYS     NODE    NODE    PIX      X      32-63,96-127    1               N/A
Legend:
X    = Self
SYS  = Connection traversing PCIe as well as the SMP interconnect between NUMA nodes (e.g., QPI/UPI)
NODE = Connection traversing PCIe as well as the interconnect between PCIe Host Bridges within a NUMA node
PHB  = Connection traversing PCIe as well as a PCIe Host Bridge (typically the CPU)
PXB  = Connection traversing multiple PCIe bridges (without traversing the PCIe Host Bridge)
PIX  = Connection traversing at most a single PCIe bridge
NV#  = Connection traversing a bonded set of # NVLinks
🐛 Describe the bug
I use vllm/vllm-openai:v0.5.0 by Distributed Inference and Serving on two L40s machine ， 8 cards per machine, a total of 16 cards for two machines
On head node
ray start --head
On worker nodes
ray start --address=xxx:6379
python -m vllm.entrypoints.openai.api_server --served-model-name Qwen2-72B-Instruct --trust-remote-code --tensor-parallel-size 16   --model /model/Qwen2-72B-Instruct --max-num-seqs 1
90000 token  no problem ，100000 token  server is down
then I got the following error:
ERROR:    Exception in ASGI application
Traceback (most recent call last):
File "/root/miniconda3/envs/vllm/lib/python3.10/site-packages/starlette/responses.py", line 265, in call
await wrap(partial(self.listen_for_disconnect, receive))
File "/root/miniconda3/envs/vllm/lib/python3.10/site-packages/starlette/responses.py", line 261, in wrap
await func()
File "/root/miniconda3/envs/vllm/lib/python3.10/site-packages/starlette/responses.py", line 238, in listen_for_disconnect
message = await receive()
File "/root/miniconda3/envs/vllm/lib/python3.10/site-packages/uvicorn/protocols/http/httptools_impl.py", line 553, in receive
await self.message_event.wait()
File "/root/miniconda3/envs/vllm/lib/python3.10/asyncio/locks.py", line 214, in wait
await fut
asyncio.exceptions.CancelledError: Cancelled by cancel scope 7f24f6169600
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/root/miniconda3/envs/vllm/lib/python3.10/site-packages/uvicorn/protocols/http/httptools_impl.py", line 399, in run_asgi
result = await app(  # type: ignore[func-returns-value]
File "/root/miniconda3/envs/vllm/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 70, in call
return await self.app(scope, receive, send)
File "/root/miniconda3/envs/vllm/lib/python3.10/site-packages/fastapi/applications.py", line 1054, in call
await super().call(scope, receive, send)
File "/root/miniconda3/envs/vllm/lib/python3.10/site-packages/starlette/applications.py", line 123, in call
await self.middleware_stack(scope, receive, send)
File "/root/miniconda3/envs/vllm/lib/python3.10/site-packages/starlette/middleware/errors.py", line 186, in call
raise exc
File "/root/miniconda3/envs/vllm/lib/python3.10/site-packages/starlette/middleware/errors.py", line 164, in call
await self.app(scope, receive, _send)
File "/root/miniconda3/envs/vllm/lib/python3.10/site-packages/starlette/middleware/cors.py", line 85, in call
await self.app(scope, receive, send)
File "/root/miniconda3/envs/vllm/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 65, in call
await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
File "/root/miniconda3/envs/vllm/lib/python3.10/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
raise exc
File "/root/miniconda3/envs/vllm/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
await app(scope, receive, sender)
File "/root/miniconda3/envs/vllm/lib/python3.10/site-packages/starlette/routing.py", line 756, in call
await self.middleware_stack(scope, receive, send)
File "/root/miniconda3/envs/vllm/lib/python3.10/site-packages/starlette/routing.py", line 776, in app
await route.handle(scope, receive, send)
File "/root/miniconda3/envs/vllm/lib/python3.10/site-packages/starlette/routing.py", line 297, in handle
await self.app(scope, receive, send)
File "/root/miniconda3/envs/vllm/lib/python3.10/site-packages/starlette/routing.py", line 77, in app
await wrap_app_handling_exceptions(app, request)(scope, receive, send)
File "/root/miniconda3/envs/vllm/lib/python3.10/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
raise exc
File "/root/miniconda3/envs/vllm/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
await app(scope, receive, sender)
File "/root/miniconda3/envs/vllm/lib/python3.10/site-packages/starlette/routing.py", line 75, in app
await response(scope, receive, send)
File "/root/miniconda3/envs/vllm/lib/python3.10/site-packages/starlette/responses.py", line 258, in call
async with anyio.create_task_group() as task_group:
File "/root/miniconda3/envs/vllm/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 680, in aexit
raise BaseExceptionGroup(
exceptiongroup.ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)

@youkaichao  Please help me check, I suspect it's a timeout issue. Thank you

Does it occur everytime? Do you have a request that can stably trigger the error?

i encountered the same problem. Both qwen2-72b and qwen1.5-110b produce the asyncio.exceptions.CancelledError after a few hours running.

Exactly the same problem I've been encountering since Jun 14. Maybe some semaphore issue?

Again it happened this morning. It seemed that the generation throughput was getting slower and slower and finally not responding. I guess there are some resources not released after request finished.

@junior-zsy We encountered the same problem when using 4 L40 GPUs for inference based on Qwen2-72B-Instruct-128K. We solved the problem by setting the following two parameters:
llm = LLM(
    ...,
    enable_chunked_prefill=True,
    max_num_batched_tokens=8192
)

You can adjust max_num_batched_tokens appropriately according to your GPU resources.

@junior-zsy We encountered the same problem when using 4 L40 GPUs for inference based on Qwen2-72B-Instruct-128K. We solved the problem by setting the following two parameters:
llm = LLM(
    ...,
    enable_chunked_prefill=True,
    max_num_batched_tokens=8192
)

You can adjust max_num_batched_tokens appropriately according to your GPU resources.

thank you

@youkaichao I have discovered the ultimate cause of this problem，The problem lies in run_engine_loop function
        has_requests_in_progress = False
        while True:
            if not has_requests_in_progress:
                logger.debug("Waiting for new requests...")
                await self._request_tracker.wait_for_new_requests()
                logger.debug("Got new requests!")

            # Abort if iteration takes too long due to unrecoverable errors
            # (eg. NCCL timeouts).
            try:
                async with asyncio_timeout(ENGINE_ITERATION_TIMEOUT_S):
                    has_requests_in_progress = await self.engine_step()
            except asyncio.TimeoutError as exc:
                logger.error(
                    "Engine iteration timed out. This should never happen!")
                self.set_errored(exc)
                raise
            await asyncio.sleep(0)

def start_background_loop(self) -> None:
        """Start the background loop."""
        if self.errored:
            raise AsyncEngineDeadError(
                "Background loop has errored already.") from self._errored_with
        if self.is_running:
            raise RuntimeError("Background loop is already running.")
        # Initialize the RequestTracker here so it uses the right event loop.
        self._request_tracker = RequestTracker()

        self._background_loop_unshielded = asyncio.get_event_loop(
        ).create_task(self.run_engine_loop())
        self._background_loop_unshielded.add_done_callback(
            partial(_log_task_completion, error_callback=self._error_callback))
        self.background_loop = asyncio.shield(self._background_loop_unshielded)

When a request triggers a timeout, a single exception can cause the entire service to crash. The raise passes an exception to the upper layer, but the upper layer's code does not try exception to catch the exception. In the end, the entire service crashes

@youkaichao I have discovered the ultimate cause of this problem，The problem lies in run_engine_loop function
        has_requests_in_progress = False
        while True:
            if not has_requests_in_progress:
                logger.debug("Waiting for new requests...")
                await self._request_tracker.wait_for_new_requests()
                logger.debug("Got new requests!")

            # Abort if iteration takes too long due to unrecoverable errors
            # (eg. NCCL timeouts).
            try:
                async with asyncio_timeout(ENGINE_ITERATION_TIMEOUT_S):
                    has_requests_in_progress = await self.engine_step()
            except asyncio.TimeoutError as exc:
                logger.error(
                    "Engine iteration timed out. This should never happen!")
                self.set_errored(exc)
                raise # If a timeout is triggered, the entire server crashes because there is no try exception caught on this exception
            await asyncio.sleep(0)

def start_background_loop(self) -> None:
        """Start the background loop."""
        if self.errored:
            raise AsyncEngineDeadError(
                "Background loop has errored already.") from self._errored_with
        if self.is_running:
            raise RuntimeError("Background loop is already running.")
        # Initialize the RequestTracker here so it uses the right event loop.
        self._request_tracker = RequestTracker()
       # Failure to capture exceptions resulted in program crash
        self._background_loop_unshielded = asyncio.get_event_loop(
        ).create_task(self.run_engine_loop())
        self._background_loop_unshielded.add_done_callback(
            partial(_log_task_completion, error_callback=self._error_callback))
        self.background_loop = asyncio.shield(self._background_loop_unshielded)

When a request triggers a timeout, a single exception can cause the entire service to crash. The raise passes an exception to the upper layer, but the upper layer's code does not try exception to catch the exception. In the end, the entire service crashes,I can solve the above problem by setting VLLM-ENGINE-ITERATION-TIMEOUT, but I don't understand why vllm code needs to be written like this, raise, but the upper layer did not capture it. Thank you

I have tried  setting VLLM-ENGINE-ITERATION-TIMEOUT=600, using "try-catch" in start_background_loop and abort this invalid request in run_engine_loop, but they don't make effect. Do you have any other idea about how to deal with this error?

@junior-zsy We encountered the same problem when using 4 L40 GPUs for inference based on Qwen2-72B-Instruct-128K. We solved the problem by setting the following two parameters:
llm = LLM(
    ...,
    enable_chunked_prefill=True,
    max_num_batched_tokens=8192
)

You can adjust max_num_batched_tokens appropriately according to your GPU resources.

Thanks, the method works for me.


Title based on html in page

I am wondering if it is possible to have this tool grab a block of html from a page to use in the title.
For example I have 100 windows open using salesforce, however all of the windows titles just say "case # "  I'd rather have it look into the web page and extract and display this html div tag to my tab's title is "TITLE OF MY CASE"

Hello @SeattleMatt,
Thanks for your feedback, your idea is very good and this will not break the current {title} variable.
I have something for you, what about this title: {#customer_name}?
This will retrieve the content of #customer_name, e.g. Google.
Actually, you can inject any CSS valid selector inside curly braces.
What do you think?
Bests

Yes, I believe this would work, in this example I would use the line
{#efhpTitle} in the Title field?
while the content of the page contains

Is this functionality already built in or are you proposing to add it?

I'm not sure why github closed this in my name, I can't seem to re-open.

Alright, this feature will be available in the next version 0.10.0.
I'll release it in few days.
Bests

Version 0.10.0 released!


server: spin servers for IPv6 dynamically from config

Aardvark config can contain list of both IPv4 and IPv6.
Following PR ensures that we spin servers for IPv6 as well
without ignoring.

@mheon @baude This should close IPv6 spin ups.


Change PR template to remind people to run gofmt and golangci-lint on their code before creating PR

Update CI to also run gofmt

I'll make the pipeline DM people who don't run golangci-lint 😡

Joke aside, I can add a git hook and people can't commit if the linter or formatter fails


Competitive Programming: Binary Search

Description
I want to make a documentation on Binary Search Algorithm.
Domain
Competitive Programming
Type of Contribution
Documentation
Code of Conduct

[X] I follow Contributing Guidelines & Code of conduct of this project.

/assign

/assign

I want to contribute in documentation part of it. Please assign this to me.

\assign Video


when use it in typescript ,it cannot be recognized correctly

My temporary solution is:
// shims-vue.d.ts
declare module "*.vue" {
    import { defineComponent } from "vue";
    const component: ReturnType<typeof defineComponent>;
    export default component;
}
// add
declare module "vue-draggable-next" {
    export { VueDraggableNext } from "vue-draggable-next/dist/src/index";
}

Just write like this👇
declare module 'vue-draggable-next';

My temporary solution is:
// shims-vue.d.ts
declare module "*.vue" {
    import { defineComponent } from "vue";
    const component: ReturnType<typeof defineComponent>;
    export default component;
}
// add
declare module "vue-draggable-next" {
    export { VueDraggableNext } from "vue-draggable-next/dist/src/index";
}

reply 👀

@bigmeow is something I have it on my plan for timing just  use declare module 'vue-draggable-next'

@anish2690 I think it should be specified in the types field of package.json to avoid error reporting.

I hope it to be recognized automatically

https://github.com/anish2690/vue-draggable-next/pull/4 added support for type definition


Cleanup list of third-party middlewares

Fix websocket/jwt middleware URL's
Remove outdated third-party middlewares (only those without updates since 2020/2021)

While doing this PR I noticed the fiberprometheus was out-of-date, submitted a PR to fix it
Rel https://github.com/ansrivas/fiberprometheus/pull/173

Just did the same with fiber_ldap middleware. Rel https://github.com/eozer/fiber_ldapauth/pull/26


Update spark client release to 0.7.3

Update build version
Update changelog

I think you can change the docs version here as well.

Done - usually we do it after the release - but I guess it is ok as it is 1 min away


rakudo: 2022.02 -> 2022.03

Description of changes
Updates rakudo and it's dependencies.
https://github.com/rakudo/rakudo/blob/2022.03/docs/ChangeLog

rakudo: 2022.02 -> 2022.03
nqp: 2022.02 -> 2022.03
moarvm: 2022.02 -> 2022.03

Things done

Built on platform(s)

[X] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
22.05 Release Notes (or backporting 21.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[ ] Fits CONTRIBUTING.md.

Tested compilation of the executables. They all appear to work fine, with the exception of *-debug, but this is not a regression. LGTM, would merge this if I had the permissions. :-)


Shahar pull request

Oh, great sloth watching us from above, we are motionless nothings without your super-slow guidance.
Approve my pull request  and with your slothly help all my code from now on will pass even the toughest of code reviews.


Modal + Scroll, the counter is not positioned correctly

If the plugin is used inside of bootstrap modal and the content of the body is scrollable, the counter is not positioned correctly.

Open the modal
Click on textarea (the counter appears)
Scroll down

I've a created a jsfiddle so you can reproduce the bug: https://jsfiddle.net/fbj14nd0/
Thank you.

You can use the option appendToParent: true which has been designed for these scenarios.
Here is an updated fiddle: https://jsfiddle.net/fbj14nd0/1/

Hi, thanks for your reply and solution. Didn't know the use of appendToParent. Have a great day!


CSS

#CSS
##User Story
As a bootcamp student
I want the prework notes to be structured on a webpage
So that I can easily find and read the information
##Acceptance Criteria
GIVEN Prework Study Guide website
WHEN I view the study guide
THEN I see a dark blue header and footer, and four boxes with a shadow

Added CSS style sheet to update HMTL file. Merged updated HTML into the main branch.


Hack in a postgres read function

oops, did not mean to make this on the upstream repo, apologies! :)


Update collate

Coverage remained the same at 68.953% when pulling 717c3d7c9f5d2e4a9e008c8740eab1b085d252e2 on collate into 9fe00b59a202355d71c56862e16428a730522c35 on develop.


Data cleanup and reorganization

Sample data may need some cleanup and reorganization
I think it will be better to have sample data close to something that can be use in backtesting production.
I think we should split data directory into data/bar and data/tick
I also think that we should use "round" date range for data
SP500TR.csv for example is from 2010-01-04 to 2016-06-22
maybe we should only have data up to 2015-12-31.
A possible organization could be
data
    datasource1
        bar
            D
                2010
                    SP500TR.csv
                    another_ticker (to define).csv
                2011
                    SP500TR.csv
                    another_ticker (to define).csv
                2012
                    SP500TR.csv
                    another_ticker (to define).csv
                2013
                    SP500TR.csv
                    another_ticker (to define).csv
                2014
                    SP500TR.csv
                    another_ticker (to define).csv
                2015
                    SP500TR.csv
                    another_ticker (to define).csv
        tick
            2015
                2015-02
                    2015-02-01
                        MSFT.csv
                        AMZN.csv
                        GOOG.csv

freq string being

'D' for daily
'H' for hourly
'15Min' for 15 minutes
...

like in Pandas DateOffsets
see pd.tseries.frequencies.to_offset('H').delta and ?pd.date_range
Moreover it will urge us to build price handler in a stream / iterate "fashion".

We need to be careful here as we should not be distributing data itself within the codebase. It will almost certainly be violating Terms Of Service of wherever it was taken from.
Instead we will need to have a post-install script that can download the data from Quandl/Yahoo or similar and then populate the directory structure as @femtotrader has suggested (which I do like a lot!).

We need some data for tests... so maybe just some "sample" data but with such a directory structure.

In that case we can generate it ourselves statistically.
I don't mind having chunks of data in the codebase, it's just proprietary data from other sources that will be a problem. We can replace these data sources with our own fake "stock" prices and then use these in our tests.


GPflow 2.0: ChangePoints

Introduction of a ChangePoints kernel to allow the identification of regime changes in 1d functions or timeseries. Examples provided in the associated notebook. I would welcome feedback on the design, and whether this would be something useful to have in the core library.
This is a port of previous PR https://github.com/GPflow/GPflow/pull/1065 to gpflow2.

You can also change the merge target for Pull Requests - no need to close and open a new one just to change that it should be merging into develop-2.0 instead of develop. :)

Sorry about this - I was a little hasty in deleting and re-opening!

@frgsimpson as per your comments from the previous PR:

Explanation of np.log(0.95) - I have simplified this so that the steepness parameter of the sigmoid can be passed in directly so things are a little clearer
Multiple kernels tests - Done
Move to pytest - Done
Hierarchical flattening of changepoints - I overrode the flattening behaviour for changepoints as it is not clear how to flatten in a sensible manner
Activation order - I have removed this in favour of direct specification via kernels
Efficient K_diag - Still TODO

Efficient K_diag - Done

@frgsimpson @awav All previous comments addressed - do you have any further thoughts?

Thanks for review @awav - I have resolved those comments. I've also added in an illustrative notebook to demo the kernel (had forgotten to git add this back in when I moved it to the new notebook location.

Hello @jahall, I apologize for closing your PR. It was closed automatically when I moved develop-2.0 to the develop. Could you re-open the PR? Thanks!

@awav @st-- Just wondering if you think this PR (and #1088) needs any further work.

LGTM, please address comments and I'm okay with merging it after. Good work, thanks @jahall!

LGTM, please address comments and I'm okay with merging it after. Good work, thanks @jahall!

Thanks @awav, have made appropriate changes.
I'm also keen to merge my generalized periodic kernel to v2.0 (see #1052 for v1.0 implementation). However, #1088 is a necessary first step for that, so am also interested in your feedback on that.


dev-cmd/pr-automerge: skip PRs with pre-release label

[x] Have you followed the guidelines in our Contributing document?
[x] Have you checked to ensure there aren't other open Pull Requests for the same change?
[x] Have you added an explanation of what your changes do and why you'd like us to include them?
[ ] Have you written new tests for your changes? Here's an example.

[ ] Have you successfully run brew typecheck with your changes locally?
[ ] Have you successfully run brew tests with your changes locally?

PRs with the pre-release label are always not ready for merge, so let's skip them by default.
See also: Homebrew/homebrew-core#128906.

Thanks @ZhongRuoyu!

PRs with the pre-release label are always not ready for merge, so let's skip them by default.

As mentioned in Slack the other day: I think we should move towards instead using draft PRs for those that aren't ready to merge. Making a PR draft/not does not affect CI status.


[Ignore] Spyglass CI workaround

Please ignore this PR, it is merely to force CI to build launcher with the two PRs Spyglass requires so that I can make a docker image for server hosts to test with.
I know, abuse much.

Setting to draft cause not intended to merge :P
(CI should still build on draft IIRC)


iPhone exporter for Godot 3.0

This is still a work in progress but the basics are there. @reduz as discussed on IRC, let me know what needs changing to get it to export textures in the correct format etc.
In order to use this you need the following template file:
https://www.dropbox.com/s/wovvh0wxavowyk9/iphone.zip?dl=0
This is just a sample file for now, it doesn't have the correct binaries in it, they are both 64bit only debug builds of todays master. What is important are the project files. Important for that person who will make a final version, the files are stripped of any user files, xcode will create those, make sure you strip them out of your copy too.
Note that I am using XCode 9 beta at the moment but I think the template will work from XCode 7 and up.
The exporter will ask the user for an xcodeproj file to export too.
The name you use will be used for the other folders and files as well to brand the project.
The folder in which this file is to be created will be used to extract the contents of the zip file in. Both the project files and plist files will be parsed for placeholders.
After the export is done xcode is started and the project file is loaded (this last step works only on Mac OS X and only if xcode is configured to be the default app to open xcodeproj files with)
Things left todo (possibly not as part of this PR):

export data pack with correct textures
investigate creating correct attributes file for App Store sandboxing (or leave this up to xcode?)
check completeness and requirements set in plist file
deal with icons

And for easier access:
iphone.zip

Should your iphone template obsolete and replace the current misc/dist/ios_xcode?

@akien-mga yes I would say the iphone template replaces the old ios_xcode files. That is what it is based on but with a number of placeholders that get replaced on export.
Obviously the binaries in my template eventually need to be replaced with actually working binaries :)


Update Bevy to 0.9.x

Some perf issues on macOS arm but not sure if related to update or not. will dig later.


gilbert watch on windows

I can run gilbert render and gilbert serve but gilbert watch fails
gilbert watch
Searching c:\users\hp\documents\cps\digital_site\venv2\lib\site-packages\gilbert\plugins for plugins...
Loaded plugin: collection
Loaded plugin: markdown
Loaded plugin: scss
Loaded plugin: yaml
Traceback (most recent call last):
  File "C:\Users\HP\Documents\CPS\digital_site\venv2\Scripts\gilbert-script.py", line 11, in <module>
    load_entry_point('gilbert==0.3.3', 'console_scripts', 'gilbert')()
  File "c:\users\hp\documents\cps\digital_site\venv2\lib\site-packages\gilbert\cli.py", line 95, in main
    args.func(args, site)
  File "c:\users\hp\documents\cps\digital_site\venv2\lib\site-packages\gilbert\cli.py", line 37, in handle_watch
    site.watch()
  File "c:\users\hp\documents\cps\digital_site\venv2\lib\site-packages\gilbert\site.py", line 143, in watch
    import inotify.adapters
  File "c:\users\hp\documents\cps\digital_site\venv2\lib\site-packages\inotify\adapters.py", line 11, in <module>
    import inotify.calls
  File "c:\users\hp\documents\cps\digital_site\venv2\lib\site-packages\inotify\calls.py", line 4, in <module>
    import inotify.library
  File "c:\users\hp\documents\cps\digital_site\venv2\lib\site-packages\inotify\library.py", line 8, in <module>
    instance = ctypes.cdll.LoadLibrary(_FILEPATH)
  File "C:\Users\HP\AppData\Local\Programs\Python\Python37\lib\ctypes\__init__.py", line 434, in LoadLibrary
    return self._dlltype(name)
  File "C:\Users\HP\AppData\Local\Programs\Python\Python37\lib\ctypes\__init__.py", line 356, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: [WinError 126] The specified module could not be found

The implementation for "watch" was written only for inotify compatible OSs as a matter of expediency.
Probably best to remove it, make it a plugin [hmm... need a plugin pattern for command line tools..] and document a better, more general purpose solution.

I think it makes sense to just keep it as is for now with a warning that it doesn't work on Windows

https://github.com/watchexec/watchexec
This seems to be the recommended general purpose solution. If you feel like giving it a go, it seems "watchexec -i docs gilbert render" would be the right invocation.
If it works, could you please provide some documentation?

That looks like a promising tool. Would be happy to write docs.

If you feel like giving it a go, it seems "watchexec -i docs gilbert render" would be the right invocation.

Great thanks, I have tried this out and it works fine. I'm happy to provide some docs


Updating Firewall Requirements to Include Outbound

Based on discussion with Chetan Desai, the outbound endpoints for the On-prem provisioning agent are the same for the Cloud Sync and HR provisioning agent: https://teams.microsoft.com/l/message/19:96cc8135752445e0857b872d60ce0e2a@thread.skype/1668601966388?tenantId=72f988bf-86f1-41af-91ab-2d7cd011db47&groupId=3e1eb424-0cb7-4665-b384-4fbfd3cd0f83&parentMessageId=1668601966388&teamName=Identity PM Collaboration&channelName=User Provisioning (ECMA%2C Cloud Sync)&createdTime=1668601966388&allowXTenantAccess=false

Learn Build status updates of commit a4e4365:
:x: Validation status: errors
Please follow instructions here which may help to resolve issue.

File
Status
Preview URL
Details

:x:Error

Details

Line 0, Column 0: [Error: download-failed] Download failed for file 'https://opdhsblobprod04.blob.core.windows.net/contents/1200f4420e254130ac029d2948482d9f/e0543683ec55a0a543edb97d79ff70a1'. Try closing and reopening the PR. If you get this Error again, file an issue.

For more details, please refer to the build report.
If you see build warnings/errors with permission issues, it might be due to single sign-on (SSO) enabled on Microsoft's GitHub organizations. Please follow instructions here to re-authorize your GitHub account to Learn Build.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
Note: Your PR may contain errors or warnings or suggestions unrelated to the files you changed. This happens when external dependencies like GitHub alias, Microsoft alias, cross repo links are updated. Please use these instructions to resolve them.
For any questions, please:Try searching the learn.microsoft.com contributor guidesPost your question in the Learn support channel

Learn Build status updates of commit 8e95932:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

articles/active-directory/app-provisioning/on-premises-application-provisioning-architecture.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the learn.microsoft.com contributor guidesPost your question in the Learn support channel

@billmath
Can you review the proposed changes? IMPORTANT: When the changes are ready for publication, add a #sign-off comment to signal that the PR is ready for the review team to merge.
#label:"aq-pr-triaged"
@MicrosoftDocs/public-repo-pr-review-team

#sign-off

@billmath
Thanks for reviewing! Note that GitHub doesn't support commands made in the diff review UI.
In the future, #sign-off comments should be made separately in the conversation view, or the PR might get overlooked.


Use NonDarwinLogger when running tests instead of os_log

'swift test' doesn't print os_log output to the command line. Use the  NonDarwinLogger that prints to stderr so we can view the log output in CI test runs.

@swift-ci Please test


Toml and maybe even yaml support

Google suggestions makes me think toml parsing support would be very popular, we might as well also do yaml, it shouldn't be much more work.
We should use the builtin toml parser when possible, and have optional dependency groups.

Hello,
Learning pydantic and pydantic settings.
I'm not used to work with optionnal dependencies. I used the same mecanism in pydantic with email-validator.
If the current code is OK for you I will add TOML,
I basically copied the docs example from Jsonsource (which I added too).
The builtin tomlib is only available from python 3.11, but the project is >=3.8.

=3.11 = tomlib
<3.11 = fallback toml reader lib.

Do you have any recommandation for the tomlib you want me to use ?

Hello,
We should be able to create Toml & Yaml source  by inheriting InitSettingsSource. Toml and Yaml parsing libraries generates a dict which the InitSettingsSource also uses. I added the source for Toml for reference.
Is there any downside to inheriting from InitSettingsSource?

@kjithin I update the PR, I guess it's better to use the base class you mentionned :).

Added in https://github.com/pydantic/pydantic-settings/pull/211


can't access kubeflow dashboard

Hi
I try to use kubeflow in remote server with SOCKS5 proxy
But I got a error about OIDC-gateway timeout
colud you help me?
command and ressult
kubectl logs oidc-gatekeeper-7b64c4d576-fv6gk -n kubeflow -c oidc-gatekeeper
time="2021-05-04T08:53:57Z" level=info msg="Config: &{ProviderURL:http://A.B.C.D.xip.io/dex ClientID:authservice-oidc ClientSecret:LH2CEUVGZXCJCGAAHCQ8BMLYZDGMQB OIDCAuthURL: RedirectURL:/authservice/oidc/callback OIDCScopes:[openid profile email groups] StrictSessionValidation:false AuthserviceURLPrefix:/authservice/ SkipAuthURLs:[/authservice/ /dex/] AuthHeader:Authorization HomepageURL:/authservice/site/homepage AfterLoginURL: AfterLogoutURL:/authservice/site/after_logout UserIDHeader:kubeflow-userid UserIDPrefix: UserIDClaim:email UserIDTokenHeader: Hostname: Port:8080 WebServerPort:8082 ReadinessProbePort:8081 CABundlePath: SessionStorePath:bolt.db SessionMaxAge:86400 ClientName:AuthService ThemesURL:themes Theme:kubeflow TemplatePath:[web/templates/default] UserTemplateContext:map[]}"
time="2021-05-04T08:53:57Z" level=info msg="Starting readiness probe at 8081"
time="2021-05-04T08:53:57Z" level=info msg="Starting server at :8080"
time="2021-05-04T08:53:57Z" level=info msg="Starting web server at :8082"
time="2021-05-04T08:54:07Z" level=error msg="OIDC provider setup failed, retrying in 10 seconds: Get http://A.B.C.D.xip.io/dex/.well-known/openid-configuration: dial tcp: lookup A.B.C.D.xip.io on <IP_ADDRESS>:53: read udp <IP_ADDRESS>:41720-><IP_ADDRESS>:53: i/o timeout"
time="2021-05-04T08:54:27Z" level=error msg="OIDC provider setup failed, retrying in 10 seconds: Get http://A.B.C.D.xip.io/dex/.well-known/openid-configuration: dial tcp: lookup A.B.C.D.xip.io on <IP_ADDRESS>:53: read udp <IP_ADDRESS>:40210-><IP_ADDRESS>:53: i/o timeout"
time="2021-05-04T08:54:47Z" level=error msg="OIDC provider setup failed, retrying in 10 seconds: Get http://A.B.C.D.xip.io/dex/.well-known/openid-configuration: dial tcp: lookup A.B.C.D.xip.io on <IP_ADDRESS>:53: read udp <IP_ADDRESS>:35025-><IP_ADDRESS>:53: i/o timeout"
time="2021-05-04T08:55:07Z" level=error msg="OIDC provider setup failed, retrying in 10 seconds: Get http://A.B.C.D.xip.io/dex/.well-known/openid-configuration: dial tcp: lookup A.B.C.D.xip.io on <IP_ADDRESS>:53: read udp <IP_ADDRESS>:58099-><IP_ADDRESS>:53: i/o timeout"
time="2021-05-04T08:55:27Z" level=error msg="OIDC provider setup failed, retrying in 10 seconds: Get http://A.B.C.D.xip.io/dex/.well-known/openid-configuration: dial tcp: lookup A.B.C.D.xip.io on <IP_ADDRESS>:53: read udp <IP_ADDRESS>:43772-><IP_ADDRESS>:53: i/o timeout"
time="2021-05-04T08:55:47Z" level=error msg="OIDC provider setup failed, retrying in 10 seconds: Get http://A.B.C.D.xip.io/dex/.well-known/openid-configuration: dial tcp: lookup A.B.C.D.xip.io on <IP_ADDRESS>:53: read udp <IP_ADDRESS>:45738-><IP_ADDRESS>:53: i/o timeout"
time="2021-05-04T08:56:07Z" level=error msg="OIDC provider setup failed, retrying in 10 seconds: Get http://A.B.C.D.xip.io/dex/.well-known/openid-configuration: dial tcp: lookup A.B.C.D.xip.io on <IP_ADDRESS>:53: read udp <IP_ADDRESS>:39714-><IP_ADDRESS>:53: i/o timeout"
time="2021-05-04T08:56:27Z" level=error msg="OIDC provider setup failed, retrying in 10 seconds: Get http://A.B.C.D.xip.io/dex/.well-known/openid-configuration: dial tcp: lookup A.B.C.D.xip.io on <IP_ADDRESS>:53: read udp <IP_ADDRESS>:33643-><IP_ADDRESS>:53: i/o timeout"
time="2021-05-04T08:56:47Z" level=error msg="OIDC provider setup failed, retrying in 10 seconds: Get http://A.B.C.D.xip.io/dex/.well-known/openid-configuration: dial tcp: lookup A.B.C.D.xip.io on <IP_ADDRESS>:53: read udp <IP_ADDRESS>:59579-><IP_ADDRESS>:53: i/o timeout"
time="2021-05-04T08:57:07Z" level=error msg="OIDC provider setup failed, retrying in 10 seconds: Get http://A.B.C.D.xip.io/dex/.well-known/openid-configuration: dial tcp: lookup A.B.C.D.xip.io on <IP_ADDRESS>:53: read udp <IP_ADDRESS>:42709-><IP_ADDRESS>:53: i/o timeout"
time="2021-05-04T08:57:28Z" level=error msg="OIDC provider setup failed, retrying in 10 seconds: Get http://A.B.C.D.xip.io/dex/.well-known/openid-configuration: dial tcp: lookup A.B.C.D.xip.io on <IP_ADDRESS>:53: read udp <IP_ADDRESS>:46672-><IP_ADDRESS>:53: i/o timeout"
time="2021-05-04T08:57:48Z" level=error msg="OIDC provider setup failed, retrying in 10 seconds: Get http://A.B.C.D.xip.io/dex/.well-known/openid-configuration: dial tcp: lookup A.B.C.D.xip.io on <IP_ADDRESS>:53: read udp <IP_ADDRESS>:34708-><IP_ADDRESS>:53: i/o timeout"
time="2021-05-04T08:58:08Z" level=error msg="OIDC provider setup failed, retrying in 10 seconds: Get http://A.B.C.D.xip.io/dex/.well-known/openid-configuration: dial tcp: lookup A.B.C.D.xip.io on <IP_ADDRESS>:53: read udp <IP_ADDRESS>:43310-><IP_ADDRESS>:53: i/o timeout"
time="2021-05-04T08:58:28Z" level=error msg="OIDC provider setup failed, retrying in 10 seconds: Get http://A.B.C.D.xip.io/dex/.well-known/openid-configuration: dial tcp: lookup A.B.C.D.xip.io on <IP_ADDRESS>:53: read udp <IP_ADDRESS>:42017-><IP_ADDRESS>:53: i/o timeout"
time="2021-05-04T08:58:48Z" level=error msg="OIDC provider setup failed, retrying in 10 seconds: Get http://A.B.C.D.xip.io/dex/.well-known/openid-configuration: dial tcp: lookup A.B.C.D.xip.io on <IP_ADDRESS>:53: read udp <IP_ADDRESS>:33622-><IP_ADDRESS>:53: i/o timeout"
time="2021-05-04T08:59:08Z" level=error msg="OIDC provider setup failed, retrying in 10 seconds: Get http://A.B.C.D.xip.io/dex/.well-known/openid-configuration: dial tcp: lookup A.B.C.D.xip.io on <IP_ADDRESS>:53: read udp <IP_ADDRESS>:35982-><IP_ADDRESS>:53: i/o timeout"
time="2021-05-04T08:59:28Z" level=error msg="OIDC provider setup failed, retrying in 10 seconds: Get http://A.B.C.D.xip.io/dex/.well-known/openid-configuration: dial tcp: lookup A.B.C.D.xip.io on <IP_ADDRESS>:53: read udp <IP_ADDRESS>:39350-><IP_ADDRESS>:53: i/o timeout"
time="2021-05-04T08:59:48Z" level=error msg="OIDC provider setup failed, retrying in 10 seconds: Get http://A.B.C.D.xip.io/dex/.well-known/openid-configuration: dial tcp: lookup A.B.C.D.xip.io on <IP_ADDRESS>:53: read udp <IP_ADDRESS>:58735-><IP_ADDRESS>:53: i/o timeout"
time="2021-05-04T09:00:08Z" level=error msg="OIDC provider setup failed, retrying in 10 seconds: Get http://A.B.C.D.xip.io/dex/.well-known/openid-configuration: dial tcp: lookup A.B.C.D.xip.io on <IP_ADDRESS>:53: read udp <IP_ADDRESS>:32825-><IP_ADDRESS>:53: i/o timeout"

inspection-report-20210504_175754.tar.gz

We should have fix out for this, can you try installing it again with microk8s on channel=1.20/edge

I try to so many thing about autorization.
But all the tries were failed....
I finally find a method which only use kubeflow pipeline dashboard.
In my case, I just use only kubeflow pipeline only, So I think kubeflow jupyter server, dex-auth have a lot of errors.
This is what I success to deploy kubeflow pipeline in my cluster
`
sudo snap install microk8s --classic --channel=1.20/edge
sudo usermod -a -G microk8s $USER
sudo chown -f -R $USER ~/.kube
su - $USER
microk8s status --wait-ready
microk8s enable dns storage gpu registry:size=100Gi
microk8s status --wait-ready
export PIPELINE_VERSION=1.5.0
kubectl apply -k "github.com/kubeflow/pipelines/manifests/kustomize/cluster-scoped-resources?ref=$PIPELINE_VERSION"
kubectl wait --for condition=established --timeout=60s crd/applications.app.k8s.io
kubectl apply -k "github.com/kubeflow/pipelines/manifests/kustomize/env/platform-agnostic-pns?ref=$PIPELINE_VERSION"
get url for access dashboard
kubectl port-forward -n kubeflow svc/ml-pipeline-ui 8080:80
`
After port-forwarding their are two method to access kubeflow pipeline dashboard
Local  Access
If your cluster(microk8s) installed in local, open your web browser.
Just go to URL <IP_ADDRESS>:8080
Remote Server Access
If your cluster installed in remote server and you need to remote access,
1. Open your terminal and enter ssh -CfND 9999 <username>@<network-server-ip>(for the proxy access)
2. Open your web browser,  set a SOCKS5 proxy.
3. Just go to URL '<IP_ADDRESS>:8080' !
I folllow up this construction => #1243
In my case, use Chrome browser not firefox, I use SwitchOmega which is chrome add-on.


The values in the node table of board db  is not available

In GitLab by @sokril on Oct 18, 2019, 11:50
follow the kubernetes code:
  # Check if the tree is dirty.  default to dirty
  if git_status=$("${git[@]}" status --porcelain 2>/dev/null) && [[ -z ${git_status} ]]; then
    KUBE_GIT_TREE_STATE="clean"
  else
    KUBE_GIT_TREE_STATE="dirty"
  fi

In GitLab by @sokril on Nov 8, 2019, 08:06
fix with pr#2219

In GitLab by @sokril on Nov 8, 2019, 08:29
closed via commit aa75ae63cacc774bb186609561617debc72d4753

In GitLab by @sokril on Nov 29, 2019, 03:12
mentioned in commit aa75ae63cacc774bb186609561617debc72d4753


Update relayer docs in docs/ibc/relayer.md

Summary
The information in those docs are likely outdated. It is currently unmaintained, so probably less confusing to remove that information entirely

For Admin Use

[ ] Not duplicate issue
[ ] Appropriate labels applied
[ ] Appropriate contributors tagged/assigned

Going to close this issue in favour of #50


ModuleNotFoundError: No module named 'edgetpu'

Doc trying to follow: https://coral.ai/docs/accelerator/get-started/#1c-on-windows
Host OS: Windows 10
Python3 version: 3.6.5

What I ran:
-Pre-configuration and runtime for the edge TPU on windows were made
-Code for facial recognition via a TFlite trained model.
What happened
from edgetpu.classification.engine import ClassificationEngine

ModuleNotFoundError: No module named 'edgetpu'

Are you running the example by following the instructions given at : https://coral.ai/docs/accelerator/get-started/#3-run-a-model-using-the-tensorflow-lite-api ?

@alitillawi
Hey! The example on the getting started page  should be using the tflite API, which is from this repo and shouldn't need the edgetpu package.
I supposed you are running this demo instead? In that case, you'd need to install thee edgetpu module, from this page and it is most likely this package that you'll need:
https://dl.google.com/coral/edgetpu_api/edgetpu-2.14.0-cp36-cp36m-win_amd64.whl

So I believe the main issue was that I had 2 python environments. I have removed one of them and repeated the whole get started process again also with the suggested model using git bash.
on the one side I am still getting on my program:
from edgetpu.classification.engine import ClassificationEngine

ModuleNotFoundError: No module named 'edgetpu'
and on the example program:
$ python classify_image.py --model models/mobilenet_v2_1.0_224_inat_bird_quant_edgetpu.tflite --labels models/inat_bird_labels.txt --input images/parrot.jpg
Traceback (most recent call last):
File "classify_image.py", line 122, in
main()
File "classify_image.py", line 100, in main
interpreter.allocate_tensors()
File "C:\Program Files\Python36\lib\site-packages\tflite_runtime\interpreter.py", line 243, in allocate_tensors
return self._interpreter.AllocateTensors()
File "C:\Program Files\Python36\lib\site-packages\tflite_runtime\tensorflow_wrap_interpreter_wrapper.py", line 110, in AllocateTensors
return _tensorflow_wrap_interpreter_wrapper.InterpreterWrapper_AllocateTensors(self)
RuntimeError: Internal: Unsupported data type in custom op handler: 0Node number 1 (EdgeTpuDelegateForCustomOp) failed to prepare.

@alitillawi Awesome, now it seems like we have 2 problems :)

ModuleNotFoundError: No module named 'edgetpu' means that either:

a) you don't have edgetpu module installed. If you have only installed everything from the getting started page, then edgetpu is definitely not installed. You'll have to install it here.
b) it is installed, but the python version you are using couldn't find it in it's path. I'm not sure the exact path that edgetpu was installed for your system, but for me it's here:
~ » python3 -c 'print(__import__("edgetpu").__path__)'                                                 1 ↵ vunam@penguin
['/usr/lib/python3/dist-packages/edgetpu']

So in that case, one way you can add it to your python program is like so:
import sys
sys.path.append('/usr/lib/python3/dist-packages/edgetpu')

But that's the path it was installed for my system, you may have to look around to find yours.

RuntimeError: Internal: Unsupported data type in custom op handler: 0Node number 1 (EdgeTpuDelegateForCustomOp) failed to prepare. is a new issue we just started seeing today. 1.14.1 is added pcie supports to our window driver, but didn't expected this to happen. If you try grabbing 1.14 (without the last .1) from here, it should works (Just found out about this in this issue: https://github.com/google-coral/tflite/issues/21).

FYI, for the second issue: RuntimeError: Internal: Unsupported data type in custom op handler: 0Node number 1 (EdgeTpuDelegateForCustomOp) failed to prepare.
We fixed it! Just re-download the newest package (1.14.1) and everything should works!
sossy again for all these issues :)

Thank you for the support so far!
Status update:

The sample project from the get sterted with the error below is now running:
RuntimeError: Internal: Unsupported data type in custom op handler: 0Node number 1 (EdgeTpuDelegateForCustomOp) failed to prepare.

What I did I have downloaded the whole rep. from https://github.com/google-coral/edgetpu, then I have run the scripts related to windows as well in the python directory the folder "edgetpu-2.14.0-py3.6.egg-info" was missing and it was added manually.
Now regarding my sample program I still no longer get the
ModuleNotFoundError: No module named 'edgetpu'.
No I am getting: an error related to the wrapper:
ModuleNotFoundError: No module named 'edgetpu.swig.edgetpu_cpp_wrapper'

it seems that I am missing the file edgetpu_cpp_wrapper.py

Update after using pip install "https:...edgetpu for my python version" --upgrade all the missing files and modules were replaced. And I was able to run custom module.
it seems that the GIT and other installation systems are somehow inconsistent concerning paths under windows.
Thanks again.

Awesome! Glad to know you got it all figured out :)
Sorry again, the getting started really should be a straight forward step, but I think there were a few factors in this case where we were in a middle of adding new features to our libedgetpu.dll.
FYI: the module from this repo won't work right out of the box, because our edgetpu modue is backed by c++ code, and it hasn't been built, that is why you are seeing the edgetpu.swig.edgetpu_cpp_wrappe issue. This is the swig code that translates the c++ to python. To build it, you can do something like:
make DOCKER_IMAGE=ubuntu:18.04  DOCKER_CPUS="k8" DOCKER_TARGETS=swig docker-build

and you'l find some dynamic libraries built for the python module :)
Anyhow, unless you're looking to modify the library, the best way is to install the module from here.


mocking for custom scalars

"graphql-tools": "^0.8.4",
1. I try the custom scalar and resolvermap, it works fine.
http://dev.apollodata.com/tools/graphql-tools/scalars.html#Date-as-a-scalar
schemas:
scalar Date

type Query {
  getCustomScalar: Date
}

also write a resolver for getCustomScalar
  getCustomScalar(){
     return new Date()
  }

resolver map for Date
import { GraphQLScalarType } from 'graphql';
import { Kind } from 'graphql/language';

const resolverMap = {
  Date: new GraphQLScalarType({
    name: 'Date',
    description: 'Date custom scalar type',
    parseValue(value) {
      return new Date(value); // value from the client
    },
    serialize(value) {
      return value.getTime(); // value sent to the client
    },
    parseLiteral(ast) {
      if (ast.kind === Kind.INT) {
        return parseInt(ast.value, 10); // ast value is always in string format
      }
      return null;
    },
  }),
};

2. I try the basic mock type, it works fine
like this:
export const IntMock = {
  Int: () => 666,
}

and use like this:
  addMockFunctionsToSchema({
    schema,
    mocks:   IntMock,
  })

3. but when I combine them together, it not works, and I try mock Date like this
export const DateMock = {
  Date: () => new Date(1995, 11, 17)
}

it said,
{
  "data": {
    "getCustomScalar": null
  },
  "errors": [
    {
      "message": "_Date is not a constructor",
      "locations": [
        {
          "line": 2,
          "column": 3
        }
      ],
      "path": [
        "getCustomScalar"
      ]
    }
  ]
}

console:
TypeError: _Date is not a constructor
    at Date (/Users/HaveF/Desktop/newTech/frontpage-server/data/resolvers/custom_scalars.js:22:15)
    at /Users/HaveF/Desktop/newTech/frontpage-server/node_modules/graphql-tools/src/mock.ts:185:51
    at resolveOrError (/Users/HaveF/Desktop/newTech/frontpage-server/node_modules/graphql/execution/execute.js:473:12)
    at resolveField (/Users/HaveF/Desktop/newTech/frontpage-server/node_modules/graphql/execution/execute.js:459:16)
    at /Users/HaveF/Desktop/newTech/frontpage-server/node_modules/graphql/execution/execute.js:273:18
    at Array.reduce (native)
    at executeFields (/Users/HaveF/Desktop/newTech/frontpage-server/node_modules/graphql/execution/execute.js:270:42)
    at executeOperation (/Users/HaveF/Desktop/newTech/frontpage-server/node_modules/graphql/execution/execute.js:211:10)
    at /Users/HaveF/Desktop/newTech/frontpage-server/node_modules/graphql/execution/execute.js:119:13
    at Object.execute (/Users/HaveF/Desktop/newTech/frontpage-server/node_modules/graphql/execution/execute.js:118:10)

Do I miss something?

Solved.
Confused by custom scalar Date and javascript build-in Date...

What did the final result look like? @HaveF


add latency property to RealReachability

@levey merged into master! nice feature! I think you already used this in your product?


[Breaking Change Request] [vm/ffi] Disallow invoking ffi methods with generics

Summary
Disallow invoking the following methods in dart:ffi with a generic and with Pointer with a generic type argument.
https://github.com/dart-lang/sdk/blob/897b1a2121748a3ee8c7b252b38c6efa6378888d/sdk/lib/ffi/ffi.dart#L24-L27
https://github.com/dart-lang/sdk/blob/897b1a2121748a3ee8c7b252b38c6efa6378888d/sdk/lib/ffi/ffi.dart#L63-L64
https://github.com/dart-lang/sdk/blob/897b1a2121748a3ee8c7b252b38c6efa6378888d/sdk/lib/ffi/ffi.dart#L535-L547
And introduce the following to reduce boilerplate:
/// Manages memory on the native heap.
abstract class Allocator {
  /// Allocates [numBytes] bytes of memory on the native heap.
  ///
  /// If [alignment] is passed, the allocated memory will be at least aligned
  /// to [alignment] bytes.
  ///
  /// Throws an ArgumentError on failure to allocate.
  Pointer<T> allocate<T extends NativeType>(int numBytes, {int? alignment});

  /// Releases memory on the native heap.
  ///
  /// Throws an ArgumentError on failure to free.
  void free(Pointer pointer);
}

/// Allocates `sizeOf<T>() * count` bytes of memory using `allocator.allocate`.
external Pointer<T> allocate<T extends NativeType>(Allocator allocator,
    {int count = 1});

Motivation
This will enable us to do two things.

Tree shaking of user-defined subclasses of Struct (https://github.com/dart-lang/sdk/issues/38721).
Optimize invocations of this code by directly inlining the size (https://github.com/dart-lang/sdk/issues/38648).

Impact
package:ffi's allocate<T extends NativeType>() will stop working.
User code using sizeOf, .ref, and elementAt generically will stop working. However, at this moment we are not aware of any code written in such way. This breaking change request is to assess whether code such as that is out there.
This requires changes API to take an int argument for the size or offset and let their callees invoke sizeOf with a non-generic type argument.
// Before breaking change.
final pointer = allocate<Int8>(count: 10);

// After breaking change (without boilerplate reduction).
final pointer = allocate<Int8>(sizeOf<Int8>() * 10);

Boilerplate reduction for allocation
Repeating the type argument is inconvenient. To address this we introduce the Allocator interface and allocate method to dart:ffi (see summary).
// Before breaking change.
final pointer = allocate<Int8>(count: 10);

// After breaking change.
final pointer = allocate<Int8>(allocator, count: 10);

Proposed changes: https://dart-review.googlesource.com/c/sdk/+/177705/2/sdk/lib/ffi/allocation.dart
Migration of package:ffi
We remove allocate from package:ffi.
We introduce a MallocAllocator which implements the Allocator interface and a const malloc.
class MallocAllocator implements Allocator {
  Pointer<T> allocate<T extends NativeType>(int numBytes);

  void free(Pointer pointer);
}

const malloc = MallocAllocator();

Proposed changes: https://github.com/dart-lang/ffi/blob/api/allocator/lib/src/allocation.dart
Migration of users of package:ffi
// Before breaking change.
final pointer = allocate<Int8>(count: 10);
free(pointer);

// After breaking change.
final pointer = allocate<Int8>(malloc, count: 10);
malloc.free(pointer);

Example migrations:

https://github.com/dart-lang/ffi/tree/api/allocator (the rest of the changed files)
https://dart-review.googlesource.com/c/sdk/+/177706

Discussion
Zero-initialize memory
We have two options for initializing memory to zero.

Add calloc besides malloc to package:ffi. This would use the calloc on all non-Windows OSes, and HEAP_ZERO_MEMORY flag for HeapAlloc on Windows.
Add a named argument bool zeroInitialize = false to allocate and Allocator.allocate.

In the first one, the implementor of the Allocator has control over whether something is zero-initialized. In the latter one, the caller of allocate has control over whether something is zero-initialized.
Requires-constant-type-arguments
The need to disallow .ref on Pointer<T extends Struct> stems from the fact that Dart does not support calling new T();
We could explore adding a pragma that would inline function bodies, or duplicate them with instantiated type arguments, as a more general solution to this problem.
@pragma('requires-constant-type-arguments')
T instantiateSomethingGeneric<T extends Object>(){
  // The following would be allowed because of the pragma.
  return new T();
}

class MyClass {}

void main(){
  final myClass = instantiateSomethingGeneric<MyClass>();
  print(myClass);
}

When exploring this we should check what this kind of logic would to to code size.
Thanks to @mraleph for this suggestion.

https://dart-review.googlesource.com/c/sdk/+/177705/8/sdk/lib/ffi/allocation.dart#24
@lrhn suggested using an extension method instead of a top level method for allocate:
// Before breaking change.
final pointer = allocate<Int8>(count: 10);
free(pointer);

// After breaking change with extension method `alloc` (`allocate` conflicts with instance method).
final pointer = malloc.alloc<Int8>(10);
malloc.free(pointer);

// After breaking change with extension method `call`.
final pointer = malloc<Int8>(10);
malloc.free(pointer);

The call option would result in the following with arena allocation and bump allocation (https://github.com/dart-lang/sdk/issues/38721#issuecomment-755912846):
final pointer = pool<Int8>(10);
pool.freeAll();

final pointer = bumpAllocator<Int8>(10);
bumpAllocator.free(pointer);

We have 3 options:

Allocator.allocate instance method + Malloc.alloc extension method.
Allocator.allocateBytes instance method + Malloc.allocate extension method.
Allocator.allocate instance method + Malloc.call extension method.

Any preferences?

https://dart-review.googlesource.com/c/sdk/+/177705/8/sdk/lib/ffi/allocation.dart#24
@lrhn suggested using an extension method instead of a top level method for allocate:
// Before breaking change.
final pointer = allocate<Int8>(count: 10);
free(pointer);

// After breaking change with extension method `alloc` (`allocate` conflicts with instance method).
final pointer = malloc.alloc<Int8>(10);
malloc.free(pointer);

// After breaking change with extension method `call`.
final pointer = malloc<Int8>(10);
malloc.free(pointer);

The call option would result in the following with arena allocation and bump allocation (https://github.com/dart-lang/sdk/issues/38721#issuecomment-755912846):
final pointer = pool<Int8>(10);
pool.freeAll();

final pointer = bumpAllocator<Int8>(10);
bumpAllocator.free(pointer);

We have 3 options:

Allocator.allocate instance method + Malloc.alloc extension method.
Allocator.allocateBytes instance method + Malloc.allocate extension method.
Allocator.allocate instance method + Malloc.call extension method.

Any preferences?

@dcharkes  Please make an announcement to https://groups.google.com/a/dartlang.org/g/announce
cc @Hixie @matanlurey @vsmenon for review and approval.

@dcharkes  Please make an announcement to https://groups.google.com/a/dartlang.org/g/announce
cc @Hixie @matanlurey @vsmenon for review and approval.

I have no opinion on this and thus no objection. I would encourage this change to be amply documented, in particular with the error message someone would see included in the documentation so that if they google for it they will find it.

I have no opinion on this and thus no objection. I would encourage this change to be amply documented, in particular with the error message someone would see included in the documentation so that if they google for it they will find it.

lgtm
fyi - @leafpetersen - if we expect more general use for compile-time known type params.

lgtm
fyi - @leafpetersen - if we expect more general use for compile-time known type params.

I don't work in this area, no comment.

I don't work in this area, no comment.

@dcharkes Please make an announcement to https://groups.google.com/a/dartlang.org/g/announce

Announcement sent: https://groups.google.com/a/dartlang.org/g/announce/c/kZOGrc6-rIg

@dcharkes Please make an announcement to https://groups.google.com/a/dartlang.org/g/announce

Announcement sent: https://groups.google.com/a/dartlang.org/g/announce/c/kZOGrc6-rIg

Approved

Approved

We have landed deprecation hints for invoking the methods with d74b2f46724cd48d0f4214fc38146010d137556d and f74c9d43d3adcb4bac75d9089e64ad34861afdcc.
Turning these hints into errors and making use of the static types to do tree shaking (https://github.com/dart-lang/sdk/issues/38721) and optimizations (https://github.com/dart-lang/sdk/issues/38648) is tracked in the respective issues. Closing this issue.

We have landed deprecation hints for invoking the methods with d74b2f46724cd48d0f4214fc38146010d137556d and f74c9d43d3adcb4bac75d9089e64ad34861afdcc.
Turning these hints into errors and making use of the static types to do tree shaking (https://github.com/dart-lang/sdk/issues/38721) and optimizations (https://github.com/dart-lang/sdk/issues/38648) is tracked in the respective issues. Closing this issue.

I am honest - I am Dart nebiew and for my first little hobby project
https://pub.dev/packages/dart_periphery
I use ffi to maniplate low level structs - e.g. i2c_msg for the I2C bus.
Currently I build complete native structs on the Dart side. It will possible to migrate this code according the new ffi changes?
Sorry for my missing knowlege to understand the upcoming changes in their full range.
/// Helper class mapped to the C struct `
class NativeI2Cmsg extends Struct {
  @Int16()
  int addr;
  @Int16()
  int flags;
  @Int16()
  int len;
  Pointer<Int8> buf;
  factory NativeI2Cmsg.allocate() => allocate<NativeI2Cmsg>().ref;
}

static Pointer<NativeI2Cmsg> _toNative(List<I2Cmsg> list) {
    final ptr = allocate<NativeI2Cmsg>(count: list.length);
    var index = 0;
    for (var data in list) {
      var msg = ptr.elementAt(index++);
      msg.ref.addr = data.addr;
      msg.ref.len = data.len;
      var flags = 0;
      if (data.flags.isNotEmpty) {
        for (var f in data.flags) {
          flags |= I2CmsgFlags2Int(f);
        }
      }
      msg.ref.flags = flags;
      msg.ref.buf = allocate<Int8>(count: data.len);
      if (data.predefined.isNotEmpty) {
        var count = 0;
        for (var value in data.predefined) {
          msg.ref.buf.elementAt(count++).value = value;
        }
      }
    }
    return ptr;
  }

Yes, just fire up the IDE with the latest beta release to get guidance from the analyzer hints.
/// Helper class mapped to the C struct `
class NativeI2Cmsg extends Struct {
  @Int16()
  int addr;
  @Int16()
  int flags;
  @Int16()
  int len;
  Pointer<Int8> buf;
}

Pointer<NativeI2Cmsg> _toNative(List<I2Cmsg> list) {
  final ptr = calloc<NativeI2Cmsg>(list.length);
  var index = 0;
  for (var data in list) {
    final msg = ptr.elementAt(index++);
    msg.ref.addr = data.addr;
    msg.ref.len = data.len;
    var flags = 0;
    if (data.flags.isNotEmpty) {
      for (var f in data.flags) {
        flags |= I2CmsgFlags2Int(f);
      }
    }
    msg.ref.flags = flags;
    msg.ref.buf = calloc<Int8>(data.len);
    if (data.predefined.isNotEmpty) {
      var count = 0;
      for (var value in data.predefined) {
        msg.ref.buf.elementAt(count++).value = value;
      }
    }
  }
  return ptr;
}


stepwise learning rate scheduler

Hello,
I am trying to manage LR scheduling with my training with pL. Main methods are relying on epoch-wise updates on LR, is there a way to reduce this to step-wise?

OS: [Linux]
Packaging [pip]
Version [1.0.4]

https://pytorch-lightning.readthedocs.io/en/latest/optimizers.html?highlight=scheduller
for example:
def configure_optimizers(self):
optimizer = Adam(...)
scheduler = {'scheduler': OneCycleLR(optimizer, ...),
'interval': 'step'}
return [optimizer], [scheduller]

I'll close this for now, feel free to re-open if you have more questions.
Also check out our forums: https://forums.pytorchlightning.ai/


iox-#208 add insert method

Pre-Review Checklist for the PR Author

[x] Code follows the coding style of CONTRIBUTING.md
[x] Tests follow the best practice for testing
[x] Changelog updated in the unreleased section including API breaking changes
[x] Branch follows the naming format (iox-#123-this-is-a-branch)
[x] Commits messages are according to this guideline

[x] Commit messages have the issue ID (iox-#123 commit text)
[x] Commit messages are signed (git commit -s)
[x] Commit author matches Eclipse Contributor Agreement (and ECA is signed)

[x] Update the PR title

Follow the same conventions as for commit messages
Link to the relevant issue

[x] Relevant issues are linked
[x] Add sensible notes for the reviewer
[x] All checks have passed (except task-list-completed)
[x] Assign PR to reviewer

Notes for Reviewer
Unlike the std::string insert method, a boolean is returned to indicate whether the insert was successful. Furthermore, insert will fail when count is greater than the size of the passed string.
Checklist for the PR Reviewer

[x] Commits are properly organized and messages are according to the guideline
[ ] Code according to our coding style and naming conventions
[ ] Unit tests have been written for new behavior

[ ] Each unit test case has a unique UUID

[x] Public API changes are documented via doxygen
[x] Copyright owner are updated in the changed files
[x] PR title describes the changes

Post-review Checklist for the PR Author

[ ] All open points are addressed and tracked via issues

References

Partially closes #208

As for string.inl:550:26: error: array subscript<PHONE_NUMBER>3709551615 is above array bounds of ‘char [11]’ [-Werror=array-bounds]
550 |     subString.m_rawstring[length] = '\0';
it could likely be fixed by obscuring the destination of the write using a volatile pointer like so
volatile char* p =  &subString.m_rawstring[length];
*p = `\0`;

Previosly the compiler assumed it could write to any 64 bit integer value length value here, but actually the size of the arry is known at compile time so it will complain. I consider this a bug due to very overeager reaction, if the compiler does not know enough it should stay silent (but some versions apparently do not).
With volatile the compiler cannot infer anymore what the size of the buffer is, hence it will not complain anymore. Without volatile it still might, as there is no aliasing of p (still worth a try if we dislike volatile).
However, this may reduce optimization potential to some extent (only minor here I think). Something has to give here though.


Fixed usability in BoxAuthentication Activity

sorry, this issue to #10
Added progress dialog and progress bar in BoxAuthentication

No longer needed the pull request.


arch linux/mingw64 compilation fails

in file:
https://github.com/doitsujin/dxvk/blob/master/src/d3d11/d3d11_include.h
in line 27: Needs to be ifdef instead of ifndef to successfully compile with mingw64 under ArchLinux, otherwise compilation fails with errors about redefinition of the symbols inside that preprocessor conditional block

Which mingw version are you using? I'm on Arch as well and the definitions are required for it to work with MinGW.

How are you building DXVK?
Literally just built from my dxvk-git AUR package and it builds fine.

which mingw version?
[dennis@DB2arch ~]$ pacman -Q | grep mingw
mingw-w64-binutils 2.29-1
mingw-w64-crt-git <IP_ADDRESS>80316-1
mingw-w64-gcc 7.3.0-1
mingw-w64-headers-git <IP_ADDRESS>80316-1
mingw-w64-winpthreads-git <IP_ADDRESS>80316-1
how I built?
I git cloned https://github.com/doitsujin/dxvk and then followed the instructions on that page. After the small change to that source file, compilation succeeded. Even tested it with Witcher3 and Kingdom Come: Deliverance which both seemed to work without issues.
"_MSC_VER" suggests it is not defined in any compiler other than Microsoft's so I am confused how those definitions should be required for mingw. Will I encounter runtime issues if I leave them out?

My setup is:
adria@Aegon ~ $ pacman -Q | grep mingw
mingw-w64-binutils 2.29-1
mingw-w64-configure 0.1-1
mingw-w64-crt 5.0.3-1
mingw-w64-gcc 7.3.0-1
mingw-w64-headers 5.0.3-1
mingw-w64-pkg-config 2-3
mingw-w64-winpthreads 5.0.3-1

I see you are using git headers. Maybe there's the issue?
Do you find the same problem if you try to build the dxvk-git AUR package?

See my edit. The real issue is that those definitions are already in the headers I'm using, so a proper fix would involve checking each "missing" definition (due to outdated mingw headers) individually, instead of relying on _MSC_VER and by that making assumptions of what is and what is not defined already.

How is that supposed to work? It is not possible to use #ifdef with typedefs.
Also, the definitions are not missing from MinGW headers, they are not recognized. Please don't ask me why though.

Good question. Probably involves testing against specific mingw(headers) version then instead of testing for the absence of the Microsoft compiler version. Or maybe it is somehow possible to run a short test compilation before the real build process and conditionally include a different header if that test fails with redefinition errors?
Apparently in the (newer) headers, the definitions are recognized, otherwise the compiler would not complain about redefinition.
Example output if I don't strip the (re)definitions from the file that comes with dxvk:
[dennis@DB2arch build.w64]$ ninja [1/31] Compiling C++ object<EMAIL_ADDRESS>FAILED<EMAIL_ADDRESS>/usr/bin/x86_64-w64-mingw32-g++ -Og -gdwarf-2 -Isrc/util/util@sta -Isrc/util -I../src/util -I.././include -fdiagnostics-color=always -pipe -Wall -Winvalid-pch -Wnon-virtual-dtor -std=c++1z -O3 -MD -MQ<EMAIL_ADDRESS>-MF<EMAIL_ADDRESS>-o<EMAIL_ADDRESS>-c ../src/util/com/com_guid.cpp In file included from ../src/util/com/../../d3d11/d3d11_interfaces.h:3:0,                  from ../src/util/com/com_guid.cpp:3: ../src/util/com/../../d3d11/d3d11_include.h:28:16: error: redefinition of ‘struct D3D11_FEATURE_DATA_THREADING’  typedef struct D3D11_FEATURE_DATA_THREADING {                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~ In file included from /usr/x86_64-w64-mingw32/include/d3d11_1.h:98:0,                  from ../src/util/com/../../d3d11/d3d11_include.h:5,                  from ../src/util/com/../../d3d11/d3d11_interfaces.h:3,                  from ../src/util/com/com_guid.cpp:3: /usr/x86_64-w64-mingw32/include/d3d11.h:1042:16: note: previous definition of ‘struct D3D11_FEATURE_DATA_THREADING’  typedef struct D3D11_FEATURE_DATA_THREADING {
If it is all too much trouble however, we might just forget about this issue for the moment though since I can successfully compile if I just leave those (re)defs.

I can reproduce the issue by using the -git headers.
Is it even possible to check for the header version?

Or maybe it is somehow possible to run a short test compilation before the real build process and conditionally include a different header if that test fails with redefinition errors?

That sounds like complete overkill for a problem that shouldn't exist in the first place. Apparently it works with up-to-date mingw - still building the latest version, will test later.

I searched a bit but could not find a way to check for header versions alone, only found the usual compiler version defs but those likely won't be sufficient and I agree that implementing a pre-compilation test would be too much since the original issue is resolved with later headers.
Also looked at the d3d11.h file from the master branch and from the (old) 5.0.3 branch which appear to be different. The old one I see does for example not define D3D11_FEATURE_DATA_THREADING (among many others).
(5.0.3: https://sourceforge.net/p/mingw-w64/mingw-w64/ci/v5.0.3/tree/mingw-w64-headers/direct-x/include/d3d11.h )
(master: https://sourceforge.net/p/mingw-w64/mingw-w64/ci/master/tree/mingw-w64-headers/direct-x/include/d3d11.h )
A pre-compilation test could involve parsing the first line:
/*** Autogenerated by WIDL 3.1 from direct-x/include/d3d11.idl - Do not edit /
vs.
/ Autogenerated by WIDL 1.6 from direct-x/include/d3d11.idl - Do not edit ***/
But anyway, if the latest headers make the _MSC_VER workaround obsolete, it should just be stripped instead of making any attempt to support outdated mingw headers.
Sorry to cause any trouble. I really like this project and hope that it will be included in a future version of Wine as an optional install-time choice to replace the default Wine D3D11 implementation.

Think is, by removing the workaround I'd break Ubuntu 17.10 again, and the non-git Arch packages. I'll more than happily do that if Ubuntu 18.04 actually ships with updated headers, but right now this would just annoy people.

Yeah, not worth the trouble for the time being then. And the issue is not too hard to fix locally.

CMake has CheckTypeSize() function to detect if structure is defined. Is meson has something similar?

CMake has CheckTypeSize() function to detect if structure is defined. Is meson has something similar?

has_type(typename) ?

The main problem is that headers are in w32-headers package which version is not equal to MinGW/GCC version.

has_type(typename)

more like has_header_symbol() but it crashes on my PC :D

I'm on arch and am able to compile 64 and 32 bit fine. Here's my mingq packages (from AUR i believe.)
[gloriouseggroll@shittywok dxvk]$ pacman -Qs mingw
local/mingw-w64-binutils 2.29-1 (mingw-w64-toolchain mingw-w64)
Cross binutils for the MinGW-w64 cross-compiler
local/mingw-w64-crt 5.0.3-1 (mingw-w64-toolchain mingw-w64)
MinGW-w64 CRT for Windows
local/mingw-w64-gcc 7.3.0-1 (mingw-w64-toolchain mingw-w64)
Cross GCC for the MinGW-w64 cross-compiler
local/mingw-w64-headers 5.0.3-1 (mingw-w64-toolchain mingw-w64)
MinGW-w64 headers for Windows
local/mingw-w64-winpthreads 5.0.3-1 (m

Should no longer be an issue when using MinGW 6.0.