fix: organize notifications page

start working on the notifications page

This is a great start. Let me give you a little context on what these will do. On the profile page, there will be a section for users to choose the types of resource they need. Each type will have have a subtopic or 2 and qualifications. Every level should have an other box for the user to choose and imput a short answer. Also, there should be a spot where any resources this user might have post will automatically be displayed from query functionality.
The resources page will have an area to post a resource with a modal template pop up for the organization to enter their name, the resource, contact info, and description. There should also be a section with the same options as the profile page, that these resource has to offer for help.
With both pages having almost the same exact multiple choice style of entering the data, it will make for cleaner data queries so the users will receive notifications when there are resources posted with the same boxes as his profile.
I'm waiting for the organizations to give me a bullet point list of the keywords we will use for this cross referencing. Does this make sense for you.
How would I go about making you co-owner of this repo, and if it's possible would you like for be a owner of this with me?
I was just discussing, O S I modals, three tier architecture and SQLite for a few hour's, so as you can imagine, I need to rest. I will be on in the morning while I sit in on an MPRN meeting. Have a good night @Ronnie018.


Uniform naming in makefiles of macros pointing to a Vim executable file

Problem:
All makefiles in use have different names for macros pointing to the
location of the Vim executable. And when you need to specify a different
location of the Vim file via the make program command line, you either have to
remember the name of the macro in the given makefile, or you have to open the
makefile to see the name of the macro.
This, as you may realize, is not very good for productivity and also creates
some confusion in the makefiles themselves.
Solution:
Use a common name for such a macro in the makefiles.
This macro name is "VIMPROG", similar to the v:progpath variable.
(in addition, the same name is used in the syntax checker Makefile and testdir).

Thanks.

It seems that, possibly these changes, have uncovered some
non-tags that make CI grumble (linux, normal, gcc, vimtags).
git blame -L1012,1012 runtime/doc/autocmd.txt
#25def2c8b8 (=?UTF-8?q?Magnus=20Gro=C3=9F?= 2021-10-22 18:56:39 +0100 1012)      :au ModeChanged *:[vV\x16]* let &l:rnu = mode() =~# '^[vV\x16]'

git blame -L442,442 runtime/doc/if_cscop.txt
#071d4279d6 (Bram Moolenaar 2004-06-13 20:20:40 +0000 442)       " Hitting CTRL-space *twice* before the search type does a vertical

git blame -L360,360 -L375,375 runtime/doc/usr_51.txt
#30ab04e16e (Bram Moolenaar 2022-05-14 13:33:50 +0100 360)   1   *typecorrect.txt*       Plugin for correcting typing mistakes
#30ab04e16e (Bram Moolenaar 2022-05-14 13:33:50 +0100 375)  16                                                   *typecorrect-settings*

However, running make locally as follows (Debian, v9.1.0532)
cd runtime/doc/
make clean tags

does not find any of them.

Apparently, instead of a freshly compiled Vim version
(src/vim), a packaged, pre-v9.0.0110 version is used
(Ubuntu 22.04).  CI is pleased now.
@chrisbra, are these changes enough for a PR?

Thanks! We don't need to increment the patch number for anything below runtime, so let me just include the change to runtime/doc/Makefile.


migrate a great feature from  Leanny/SeedSearcher && upgrade deps

Just a note: I will have time to review  and test this on the weekend

Thank you!


How to create istio-archive?

@howardjohn -
I see that this commit  got rid of the istio-archive target.
I tried finding a newer make target that generates istio-1.7.1-linux-amd64.tar.gz, but was not able to. How can I achieve this?

https://github.com/istio/release-builder has all the info and steps


CORS headers

See issue #12.
Merge #14 first!

Would it make sense to add schema definitions for "Access-Control-Allow-Origin" and "Content-Type" response headers in order to minimize redundancy? I also found that Open API 3.0.0 recommends: "If a response header is defined with the name "Content-Type", it SHALL be ignored."

I'm still working on it due to the missing authorization headers. It get's even more complicated, but I try to reduce redundancy with the next commit.


Prepare CI for Train

Add the Train projects to the cleanvm, repocheck, staging and submit
jobs. The Cloud:OpenStack:Train and Cloud:OpenStack:Train:Staging jobs
will need to be created and populated before these will work.

lets first get #3725 merged and passing before starting downstream CI efforts..

Can we revisit this? The rpm-packaging CI is set up and the projects now exist in OBS.


New  point of view

Thank you for your work. It works very well.
I have a question. How can I make the video from a new point of view.

Since we read the camera pose from transform_xx.json, you can give a new sequence of views by specifying another file, or interacting with gui (remains some bugs
unsolved). You can also apply additional transforms directly in the code

Thank you. This is works fine.


Fix add_partonomy.py

Original bug report and discussion: https://github.com/hubmapconsortium/portal-ui/issues/2218

For now, just revert to flat organ facets:

[ ] https://github.com/hubmapconsortium/portal-ui/pull/2222

Then:

[ ] Identify impediments to a strictly tree-like structure:

Systemic terms ("vasculature") should be excluded
There may be data entry errors where terms were picked that do not and cannot belong to the organ.
Some ontologies are by their nature DAGs rather than trees: For example, if "left lung" and "right lung" are distinguished, just having "alveoli" isn't enough to give a unique path.
The current structure has been described as a poly-tree: it needs to be reduced to a single tree.
Others?

[ ] Clean up the RUI-JSON, if needed.
[ ] Improve data entry tools, if possible.
[ ] Either

provide a ontology service endpoint that can trace a term back to the root, and call from add_partonomy.py
or make multiple API calls from add_partonomy.py
or provide an updated version of ccf-partonomy.jsonld that is strictly a tree structure, and reference it from add_partonomy.py.

I was coming here to say that the ccf-partonomy.jsonld is a tree, but Now I see in my code how there is an edge case that could have prevented that! We'll be updating the ccf with new data in the next two weeks, so I'll try to get that piece fixed in the process!
Also thanks for resurfacing this.

Questions from @computationdoc :

... and Portal UI is already built assuming the anatomy is a proper tree is my understanding from the problem description from Nils. If Portal UI does allow polyhierarchies and still counts correctly and searches correctly we can use ‘all shortest paths‘ rather than ‘single shortest path’ in neo4j so the difference in code is trivial to these tools so no big rewrite to allow polyhierarchies if we choose to.

For hierarchical faceting, the portal documents need to include anatomy_# fields like this:
{...
 'anatomy_0': ['body'],
 'anatomy_1': ['large intestine'],
 'anatomy_2': ['transverse colon'],
...}

This structure is dictated by the searchkit library we use to interact with elasticsearch.
There is no necessary limit on the depth... but in terms of usability, you may want to avoid intermediate levels.... In this example, introducing 'digestive tract' between 'body' and 'large intestine' may or may not be useful: It's a decision, and it shouldn't be my decision.
Different documents should agree on the depth of a particular term: If 'transverse colon' is at level 2 for one document, it should be on that same level for all documents.
These fields should be single valued-except at the lowest level. This is not a hard constraint, but if it is violated it limits what we can do with the data. ie: If intermediate levels are multi-valued, we can't use this for a bread-crumb trail in the UI.
I don't really care what the backing structure is, hierarchy or polyhierarchy, as long as this structure can be generated.

Can you point me to a current JSON for the format Portal UI is currently receiving or is transforming to - Is it a simple tree of ccf_annotations in JSON, and you are separately indexing from search the datasets to each distinct ccf_annotation to ‘know’ the facet counts?

Currently, the code downloads<EMAIL_ADDRESS>Code in add_partonomy.py parses and traverses this file, but I have no particular attachment to it: Feel free to revise, or throw it away, if more of the heavy lifting can be done in the API.

I have one question: is there any downside to all single valued - i.e. having multiple paths each identical except different lowest value (rather than multi valued at lowest level)? Obviously one could post hoc combine them but I’d just assume skip that step if it’s irrelevant since the process I’m envisioning creates a set of single-value paths from each unique value in the data to ‘body’ anyway.

If I understand correctly, you are asking if instead of
{...
 'anatomy_0': ['body'],
 'anatomy_1': ['large intestine'],
 'anatomy_2': ['transverse colon', 'ascending colon'],
...}

it could be something like:
{...
  'anatomy': [
    ['body', 'large intestine', 'transverse colon'],
    ['body', 'large intestine', 'ascending colon']
  ],
...}

If I'm misunderstanding, please provide an example of what you have in mind.
If my understanding is correct: This would be a great response from the API, and a great intermediate representation, but for the index, we need anatomy_0, anatomy_1, etc. To make the hierarchy useful in search, the levels need to be cleanly separated, so we can do a high-level search by constraining just anatomy_0 and _1... or progressively more constrained searches by adding _2, _3, etc.

Good news. 1. We were able to verify yesterday that the new ASCT+B ingests properly into KG so we can take advantage of better trees in the infrastructure. and 2. Found the algorithm needed in neo4j documentation to essentially result a tree with guarantee of each node at only one level, so don't have to create the algorithm (its embedded in this documentation - I think in the "sequence" section if I recall correctly):
https://neo4j-contrib.github.io/neo4j-apoc-procedures/index33.html#_expand_paths
So, bottom line, we can now build fairly simply a process that takes a list of codes/concepts (e.g., the set of AS from RUI data) from PROV and uses those to assemble the right "sub-graph" from KG without new algorithm work per se, just queries. We'll need to develop the proper spec and schedule the work still.

Update from Jesse:

Spoke to JS, as of now he is principally responsible. If we deliver it without using ASCTB, it can be done quickly. With it it's still not certain. Message to be sent out to figure out best course of action.


Is 'Root CA certificate' description in article correct?

On Portal, enrollment group has two options: 'CA certificate' or 'Intermediate Certificate'. Are you mapping 'Root CA certificate' to 'CA certificate' in this article? Usually 'Root CA certificate' is self-signed but here I think it is refer to the certificate that user upload and verified on DPS Portal. Can we call it a 'Root certificate' since it might be signed by other CA (may be it is purchased from Verisign)?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: e5095261-1589-340f-acb7-f4dbe414942a
Version Independent ID: 2fda5715-b67d-788f-a95f-21267a9b0fd1
Content: Security concepts in Azure IoT Hub Device Provisioning Service
Content Source: articles/iot-dps/concepts-security.md
Service: iot-dps
GitHub Login: @nberdy
Microsoft Alias: nberdy

@xiongyu0523 Thanks for the feedback! I have assigned the issue to the content author to evaluate and update as appropriate.

"Root certificate" is over-limiting because it refers to the self-signed terminal root of trust which is generally stored offline. We felt "CA certificate" is a more general term and reflects the fact that customers can upload any signing certificate to DPS (doesn't have to be the root).
@Alberto-Vega-MSFT please close.


Type inference of msg_send! with void returns does not work with nightly

This is probably not a bug that needs to be fixed in rust-objc, but a warning to developers since it causes runtime crashes with unclear errors.
Previously, both of these worked, assuming setThing: returned void:
let _ = msg_send![obj, setThing: 1];
msg_send![obj, setThing: 0];

They worked because Rust inferred unknown types as ().  As of recent Rust nightly builds, unknown types are inferred as a new '!' type (see: https://github.com/rust-lang/rust/issues/48950).
Unfortunately, Rust will build it and the crashes occur at runtime.  Depending on which features you have compiled rust-objc with, you either get a panic at the calling line, a panic in objc_exception, or a segfault.
The error messages are entirely unhelpful unless you build rust-objc with the verify_message feature, which catches it at compile-time.
Fix by forcing the type back to ():
let _: () = msg_send![obj, setThing: 1];

Any plans for a proper fix? This affects us greatly in metal-rs.

@kvark I believe the fix should be either in metal-rs: adding let () = before relevant msg_sends, or rustc changing what type inference defaults to on unconstrained types. Either way, looks like rustc will revert and avoid stabilising till ramifications are clearer https://github.com/rust-lang/rust/issues/49691

I didn't even know it would compile if you omit the let _: () = when using msg_send!! Sorry to hear this caused folks trouble. Unfortunately I'm not aware of anything that we can do in the objc crate to avoid this...
If the rustc change gets shipped again, it seems like the only way to fix would be to add the type annotations. I'll close this out because I'm not aware of any action objc can take, but if anyone has an idea let me know!

I haven't fully understood this issue. I thought this was unavoidable, but after @SimonSapin's comment https://github.com/rust-lang/rust/pull/65355#issuecomment-542166585, it turns out that the un-annotated usage only compiles because of a quirk in objc's macro and the type resolution.
For example, the following code will not compile:
mem::zeroed();

You get an error stating "type annotations needed". I would expect that a bare msg_send! like the following behaves the same way and would fail to compile:
msg_send![obj, release];

But as described above, that code is accepted! It infers a () return type from msg_send!. Why is this? Well, it turns out that the following code will compile:
if false {
    panic!("panic")
} else {
    mem::zeroed()
};

For some reason, the compiler sees that one branch is ! and the other is _ and decides that the type must be (). There is a comparable code structure inside the msg_send! macro which results in the same behavior. With the change to stabilize !, the compiler will instead infer that the type here is !, and undefined behavior ensues.
With that understanding, there is in fact something we can do here: change the structure of the code inside the macro so that bare msg_send! is not accepted, and correctly errors requiring an explicit return type. This will prevent users from relying on this quirk in type resolution while unblocking the Rust devs on stabilizing !.


Install_crate can be optional for most cases

In many cases cargo plugin is in the form of cargo-command so the crate name can be extracted from the command itself in most cases and there is no need to manually define it.

It is possible that this might cause a backward compatibility break so this change would result with a minor version bump


Replace github.com/ghodss/yaml with sigs.k8s.io/yaml

What type of PR is this:

/area refactoring
What does this PR do / why we need it:
The github.com/ghodss/yaml package is no longer actively maintained. There are numerous inquiries about the project's status on its issue tracker: https://github.com/ghodss/yaml/issues. sigs.k8s.io/yaml is a permanent fork of github.com/ghodss/yaml, which is actively maintained by Kubernetes SIG and widely used in K8s projects.
Since sigs.k8s.io/yaml was already a dependency before this pull request was made, we can remove 1 extra dependency by replacing github.com/ghodss/yaml with sigs.k8s.io/yaml.
Which issue(s) this PR fixes:

PR acceptance criteria:

[ ] Unit test

[ ] Integration test

[ ] Documentation

How to test changes / Special notes to the reviewer:

/unhold

/lgtm
Thanks @Juneezee for this change.

/ok-to-test


Improve text color & add text boxes color and shadow in Leading Blog page

When you select dark mode these text color blends with background color, which doesn't look good..

& this text box doesn't look good, I want to customize it by adding background color and shadow for both dark and light mode -

like this -

I want to Fixed all these issue properly.
@ANSHIKA-26  please assign me this task.

@ANSHIKA-26 assign me with level 2

#1099

& this text box doesn't look good, I want to customize it by adding background color and shadow for both dark and light mode -

@ANSHIKA-26
this text box doesn't look good, I want to customize it by adding background color and shadow for both dark and light mode
??


Can't connect to private TestRail with TestRailSDK version 0.9

Dear Sir,
In the latest version TestRailSDK in Maven repository: 0.9, I can connect to my personal TestRail of TestRail cloud, and get the information from the API of TestRail, for example, project name, project ID, Test suite name, Test suite ID.
But once I changed to my company private testrail ,I can't connect to it, I verified that I can connect to my private testrail with REST tool such as POSTMAN, but use the TestRailSDK with version 0.9, I got the error below:
Exception in thread "main" java.lang.RuntimeException: Connection is null (probably hit timeout), check parameters for [https://<IP_ADDRESS>/index.php?/api/v2/get_projects]
	at com.rmn.testrail.service.TestRailService.getRESTRequest(TestRailService.java:518)
	at com.rmn.testrail.service.TestRailService.getEntityList(TestRailService.java:134)
	at com.rmn.testrail.service.TestRailService.getProjects(TestRailService.java:179)

Could you advise how to fix the problem?

please close this issue, found the problem, and the problem is private testrail instance issue.


chore(main): release 0.4.0

:robot: I have created a release beep boop
0.4.0 (2023-07-10)
Features

ability to not write logs to disk if needed (#255) (69469c0)

This PR was generated with Release Please. See documentation.

:robot: Release is at https://github.com/Autodesk/pgbelt/releases/tag/v0.4.0 :sunflower:


Question on inputting as PDB/structure files rather than .fasta for multimer predictions

Hello,
We have been using Alphafold in order to predict protein:protein interactions using the multimer function. In this case, quite a lot of the processing time is taken up by folding a protein sequence for which we already know the structure. As we are more interested in protein complex formations and interactions, is there any way to input PDB files (or similar) rather than .fasta files and effectively bypass the protein folding steps?

This was also my question to AlphaFold: Could I ask if its possible to feed pdb files, i.e. model structures straight into alphafold multimer? This seems to be implied in your paper where Alphafold and cluspro structures are compared. If this is possible, could you give me some indication of how it is achieved.

I totally agree with the previous comment, it would be great to be able to do this for multimer interactions... Or at least to use DeepMind pdb database of 200millions proteins instead of refolding it each time... To be able to input a known A-B complex and search for how C is binding would also be great.


fix readme Configuration Tags for eventhub

This just fixes parsing of the readme. A Tag must be directly below a Configuration, not Suppression.

@ruowan, you approved these changes 11 days ago. Can you please merge it?

/azp run

/azp run

/azp run

/azp run


Default to https

Can you default the site to https? The site looks to support it and it can be manually changed from http://en.wheelsage.org/ to https://en.wheelsage.org/ but it would be nice if the site redirected it automatically.

Yup, it's time

http://wheelsage.org/ done


Fix mypy and test coverage issues on KPOA

Linked PR https://github.com/astronomer/astronomer-providers/pull/139

Facing issues during testing as we are unable to deploy DAG changes onto astro-cloud 5.0.1

@pankaj will try using astro-cloud 5.0.2 and update here

Created new deployment and test


[Feature]: vLLM does not work with Hugging Face ZeroGPU Spaces

Information about HF ZeroGPU Spaces can be found here: https://huggingface.co/zero-gpu-explorers
The environment and code for this issue is kept fully within this Hugging Face space, specifically the app.py for the expected working code for being able to run a chat with vLLM: https://huggingface.co/spaces/mgoin/vllm-zero-gpu
Your current environment
Interestingly, it seems like ZeroGPU Spaces really don't have GPUs available at startup. This is clearly an issue :)
Collecting environment information...
/usr/local/lib/python3.10/site-packages/torch/cuda/__init__.py:611: UserWarning: Can't initialize NVML
  warnings.warn("Can't initialize NVML")
PyTorch version: 2.1.2+cu121
Is debug build: False
CUDA used to build PyTorch: 12.1
ROCM used to build PyTorch: N/A

OS: Debian GNU/Linux 12 (bookworm) (x86_64)
GCC version: (Debian 12.2.0-14) 12.2.0
Clang version: Could not collect
CMake version: version 3.25.1
Libc version: glibc-2.36

Python version: 3.10.13 (main, Mar 12 2024, 12:16:25) [GCC 12.2.0] (64-bit runtime)
Python platform: Linux-5.10.192-183.736.amzn2.x86_64-x86_64-with-glibc2.36
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Architecture:                       x86_64
CPU op-mode(s):                     32-bit, 64-bit
Address sizes:                      46 bits physical, 48 bits virtual
Byte Order:                         Little Endian
CPU(s):                             96
On-line CPU(s) list:                0-95
Vendor ID:                          GenuineIntel
Model name:                         Intel(R) Xeon(R) Platinum 8275CL CPU @ 3.00GHz
CPU family:                         6
Model:                              85
Thread(s) per core:                 2
Core(s) per socket:                 24
Socket(s):                          2
Stepping:                           7
BogoMIPS:                           5999.98
Flags:                              fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq monitor ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single pti fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid mpx avx512f avx512dq rdseed adx smap clflushopt clwb avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves ida arat pku ospke
Hypervisor vendor:                  KVM
Virtualization type:                full
L1d cache:                          1.5 MiB (48 instances)
L1i cache:                          1.5 MiB (48 instances)
L2 cache:                           48 MiB (48 instances)
L3 cache:                           71.5 MiB (2 instances)
NUMA node(s):                       2
NUMA node0 CPU(s):                  0-23,48-71
NUMA node1 CPU(s):                  24-47,72-95
Vulnerability Gather data sampling: Unknown: Dependent on hypervisor status
Vulnerability Itlb multihit:        KVM: Mitigation: VMX unsupported
Vulnerability L1tf:                 Mitigation; PTE Inversion
Vulnerability Mds:                  Vulnerable: Clear CPU buffers attempted, no microcode; SMT Host state unknown
Vulnerability Meltdown:             Mitigation; PTI
Vulnerability Mmio stale data:      Vulnerable: Clear CPU buffers attempted, no microcode; SMT Host state unknown
Vulnerability Retbleed:             Vulnerable
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass:    Vulnerable
Vulnerability Spectre v1:           Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:           Mitigation; Retpolines, STIBP disabled, RSB filling
Vulnerability Srbds:                Not affected
Vulnerability Tsx async abort:      Not affected

Versions of relevant libraries:
[pip3] numpy==1.26.4
[pip3] torch==2.1.2
[pip3] triton==2.1.0
[conda] Could not collectROCM Version: Could not collect
Neuron SDK Version: N/A
vLLM Version: 0.3.3
vLLM Build Flags:
CUDA Archs: Not Set; ROCm: Disabled; Neuron: Disabled
GPU Topology:
Could not collect

🐛 Describe the bug
The ZeroGPU project claims that:

ZeroGPU Spaces should mostly be compatible with any PyTorch-based GPU Space.
Compatibilty with high level HF libraries like transformers or diffusers is slightly more guaranteed
That said, ZeroGPU Spaces are not as broadly compatible as classical GPU Spaces and you might still encounter unexpected bugs

The benefit of working well with ZeroGPU is that you can now get access to free GPUs for live vLLM spaces on HF, rather than paying an hourly price to host your vLLM demo. Currently they are using A100s so there are definitely capable GPUs available. The complexity of using this comes from the fact that this uses a sort of serverless or work-sharing structure where the GPU is quickly taken and released based on the application function call. It seems that vLLM breaks this contract with ZeroGPU because it directly allocates workers to devices using torch.cuda.set_device(self.device) during model load.
Because vLLM carefully allocates and manages GPU memory, it may be fundamentally incompatible with what ZeroGPU requires in order to provide GPUs for free for demos. Still, it's worth opening an issue since it was be convienent if it was a small fix and others may encounter this as the project ramps up.
Here is the output of the HF Space when trying to load a model, you can clearly see the CUDA must not be initialized in the main process on Spaces with Stateless GPU environment. error:
INFO 03-19 21:22:02 llm_engine.py:87] Initializing an LLM engine with config: model='NousResearch/Hermes-2-Pro-Mistral-7B', tokenizer='NousResearch/Hermes-2-Pro-Mistral-7B', tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=False, dtype=torch.bfloat16, max_seq_len=4096, download_dir=None, load_format=auto, tensor_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=False, kv_cache_dtype=auto, device_config=cuda, seed=0)
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
You set `add_prefix_space`. The tokenizer needs to be converted from the slow tokenizers
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Traceback (most recent call last):
  File "/home/user/app/app.py", line 34, in <module>
    model = LLM(model_id, max_model_len=MAX_INPUT_TOKEN_LENGTH)
  File "/usr/local/lib/python3.10/site-packages/vllm/entrypoints/llm.py", line 109, in __init__
    self.llm_engine = LLMEngine.from_engine_args(engine_args)
  File "/usr/local/lib/python3.10/site-packages/vllm/engine/llm_engine.py", line 391, in from_engine_args
    engine = cls(*engine_configs,
  File "/usr/local/lib/python3.10/site-packages/vllm/engine/llm_engine.py", line 128, in __init__
    self._init_workers()
  File "/usr/local/lib/python3.10/site-packages/vllm/engine/llm_engine.py", line 180, in _init_workers
    self._run_workers("init_model")
  File "/usr/local/lib/python3.10/site-packages/vllm/engine/llm_engine.py", line 1041, in _run_workers
    driver_worker_output = getattr(self.driver_worker,
  File "/usr/local/lib/python3.10/site-packages/vllm/worker/worker.py", line 85, in init_model
    torch.cuda.set_device(self.device)
  File "/usr/local/lib/python3.10/site-packages/torch/cuda/__init__.py", line 404, in set_device
    torch._C._cuda_setDevice(device)
  File "/usr/local/lib/python3.10/site-packages/torch/cuda/__init__.py", line 298, in _lazy_init
    torch._C._cuda_init()
  File "/usr/local/lib/python3.10/site-packages/spaces/zero/torch.py", line 133, in _cuda_init_raise
    raise RuntimeError(
RuntimeError: CUDA must not be initialized in the main process on Spaces with Stateless GPU environment.
You can look at this Stacktrace to find out which part of your code triggered a CUDA init

Interesting. I would say this is not a bug. Rather something creative need to be figure out to make vLLM's assumption of exclusive GPU access compatible with sharing. Potential candidate includes treating the block table as swappable/virtual spaces.

Hey is the fixed?Are Vllms working now on ZeroGPU?


process: hammer v2

Description
Fixes: https://github.com/dust-tt/tasks/issues/1582
Changes the way we split docs:

Simply operate at the chunk level relying on the fact that without query we get ~topk chunks (possibly a bit more if we have too much data, otherwise less) and group them in 32 groups. We use topK = 768 so each group have 12288 tokens at most (with a few injected tags), which is guaranteed to hold in context size for models we use for our assistants.
The fact that we do split in 32 groups helps the model generate more extracted data points and it also seems to follow objective well now.

Overall should create a much better experience. Risk is that we go full 32 parallel call to model providers at each extract execution. To be monitored.
Dust-app: https://dust.tt/w/78bda07b39/spaces/vlt_rICtlrSEpWqX/apps/953b79fe89
Risk
Described above, increased concurrency.
Deploy Plan

deploy front

@flvndvd yes. LLM block up to 32 concurrency: https://github.com/dust-tt/dust/blob/main/core/src/run.rs#L82

Chat is more relevant, same number: https://github.com/dust-tt/dust/blob/main/core/src/run.rs#L83


Escaping of first decimal digits in the current form - is this something what we expect?

If I understand correctly, with the current spec draft logic, we have:
const string = '40';
const escaped = RegExp.escape(string); // => '\\x40'
const re = new RegExp(escaped); // => /\x40/

re.test('40'); // => false
re.test('@'); // => true

Is this expected behavior?

It seems it should be something like \x340.

You're right, we should be hex encoding the the digit in this case.

Yes, the intention is to hex encode.


Search query results rows based on row filter ( keyword )

On Oracle Workbench after i search on a specific table i can then further filter as you can see on the image .
It is searching column by column every row and if it finds the specific keyword on any column it shows that row on the results . If it doesn't that row isn't visible on the results
I can't find that functionality on dbeaver ( is it missing or am i missing something ) ?

Hello @goxr3plus,
You can configure columns visibility using Configure filters button:

Please let me know if you have any questions left.

@ShadelessFox Hello again i checked now  and the customize filters is not exactly what we need .
It just filters which columns are matching by name the filter .
We need it to filter the columns by content so for example  we type the word a .
It will show all the rows that have in any column the letter a . So for example on the above given picture ( Row 1 has a on the column FirstName so row 1 will be visible , row 2 doesn't contain letter a on any column as i see so row 2 will not be visible on the results , row 3 has letter a on Andy so it will be visible , and so on .

Pardon me, I was thinking about column filtering. Thanks for explaining, though!
It seems like a cool feature to have.
As a workaround, you can:

Press Ctrl+F in the data viewer to search for any text you want
Enable Mark occurrences of the selected cell's value (preview)

Also waiting for this. It's the one feature I miss since I left Workbench and can save you so much time in the long run.
Could integrate it into this filter field but just make it that it searches globally in every column on the table:

You can vote for this issue with emoji under the issue description.


Does it support generated the Graph to the Graph database? like Neo4j?

Do you need to file an issue?

[X] I have searched the existing issues and this feature is not already filed.
[X] My model is hosted on OpenAI or Azure. If not, please look at the "model providers" issue and don't file a new one here.
[X] I believe this is a legitimate feature request, not just a question. If this is a question, please use the Discussions area.

Is your feature request related to a problem? Please describe.
Does it support generated the Graph to the Graph database? like Neo4j?
Describe the solution you'd like
Does it support generated the Graph to the Graph database? like Neo4j?
Additional context
Does it support generated the Graph to the Graph database? like Neo4j?

Duplicate of #433


fix(sqla): allow series limit without subquery support

SUMMARY
The recent PR #16660 broke series limiting for databases that don't support subqueries (Pinot and Solr). This moves the check for allows_subqueries down one level to the section where the subquery is constructed, thus allowing for a prequery to populate the series values to limit by if subqueries/joins aren't supported.
Note, however, that before #16660 we weren't checking for support for subqueries when constructing the limiting subquery. But since there are no db engine specs that allow joins but don't support subqueries, adding the check for subquery support shouldn't have any practical implications right now.
BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

TESTING INSTRUCTIONS

ADDITIONAL INFORMATION

[ ] Has associated issue:
[ ] Required feature flags:
[ ] Changes UI
[ ] Includes DB Migration (follow approval process in SIP-59)

[ ] Migration is atomic, supports rollback & is backwards-compatible
[ ] Confirm DB migration upgrade and downgrade tested
[ ] Runtime estimates and downtime expectations provided

[ ] Introduces new feature or API
[ ] Removes existing feature or API

lgtm, let's merge to fix the regression and add tests in a follow up?

SGTM - I'll add a follow-up test soon 👍


RunnableWithFallbacks doesn't implement streaming support

Checked other resources

[X] I added a very descriptive title to this issue.
[X] I searched the LangChain.js documentation with the integrated search.
[X] I used the GitHub search to find a similar question and didn't find it.
[X] I am sure that this is a bug in LangChain.js rather than my code.
[X] The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code
Sample code similar to the following should stream the results:
import { BedrockChat } from "@langchain/community/chat_models/bedrock";
import { concat } from "@langchain/core/utils/stream";
import type { AIMessageChunk } from "@langchain/core/messages";

function createModelWithFallbacks() {
  const primaryModel = new BedrockChat({
    model: "anthropic.claude-3-5-sonnet-20240620-v1:0",
    region: "us-east-1",
  });

   const secondaryModel = new BedrockChat({
     model: "anthropic.claude-3-haiku-20240307-v1:0",
     region: "us-east-1",
   });

  // Using `withFallbacks` causes streaming to not work
  return primaryModel.withFallbacks({
    fallbacks: [
      secondaryModel,
    ],
  });
}

export async function testStream() {
  const model = createModelWithFallbacks();

  // This will result in a call to `.invoke()` instead
  const stream = await model.stream("What color is the sky on Mars?");

  let gathered: AIMessageChunk | undefined = undefined;

  for await (const chunk of stream) {
    console.log(chunk);
    if (gathered === undefined) {
      gathered = chunk;
    } else {
      gathered = concat(gathered, chunk);
    }
  }

  console.log(gathered);
}

testStream();

However, currently invoke will be called on the model(s) and no streaming happens.
A sample patch to resolves the issue:
diff --git a/dist/runnables/base.cjs b/dist/runnables/base.cjs
index 2276cfd01b0ad7072094f634bbe7ec115189f20e..cf87a60ff48ceb57b6bcc0abfbf9e7db27a9c5f5 100644
--- a/dist/runnables/base.cjs
+++ b/dist/runnables/base.cjs
@@ -1699,6 +1699,73 @@ class RunnableWithFallbacks extends Runnable {
         await runManager?.handleChainError(firstError);
         throw firstError;
     }
+    async *_streamIterator(input, options) {
+        const callbackManager_ = await manager_js_1.CallbackManager.configure(options?.callbacks, undefined, options?.tags, undefined, options?.metadata);
+        const { runId, ...otherOptions } = options ?? {};
+        const runManager = await callbackManager_?.handleChainStart(this.toJSON(), _coerceToDict(input, "input"), runId, undefined, undefined, undefined, otherOptions?.runName);
+
+        let chunk;
+        let firstError = undefined;
+        let stream = undefined;
+        let concatSupported = false;
+        let finalOutput = undefined;
+        for await (const runnable of this.runnables()) {
+            concatSupported = true;
+            try {
+                stream = await runnable.stream(input, config_js_1.patchConfig(otherOptions, { callbacks: runManager?.getChild() }));
+                chunk = stream.next().value;
+
+                // Clear the error since we successfully got a chunk from this runnable
+                firstError = undefined;
+            } catch (e) {
+                if (firstError === undefined) {
+                    firstError = e;
+                }
+            }
+
+            // Check if we successfully got a chunk from this runnable
+            if (!firstError) {
+                // Use the stream from this runnable for the rest of the chunks
+                break;
+            }
+        }
+
+        if (firstError) {
+            await runManager?.handleChainError(firstError);
+            throw firstError;
+        }
+
+        if (chunk) {
+            yield chunk;
+        }
+
+        if (stream) {
+            try {
+                for await (chunk of stream) {
+                    if (concatSupported) {
+                        if (finalOutput === undefined) {
+                            finalOutput = chunk;
+                        } else {
+                            try {
+                                // eslint-disable-next-line @typescript-eslint/no-explicit-any
+                                finalOutput = concat(finalOutput, chunk);
+                            }
+                            catch (e) {
+                                finalOutput = undefined;
+                                concatSupported = false;
+                            }
+                        }
+                    }
+                    yield chunk;
+                }
+            } catch (e) {
+                await runManager?.handleChainError(e);
+                throw e;
+            }
+        }
+
+        await runManager?.handleChainEnd(finalOutput);
+    }
     async batch(inputs, options, batchOptions) {
         if (batchOptions?.returnExceptions) {
             throw new Error("Not implemented.");
diff --git a/dist/runnables/base.js b/dist/runnables/base.js
index d60358b742c78df8cb1edc7d3a872a1c383bc3ed..76e5ab8fcafbb5ffe134929411c09300970321b6 100644
--- a/dist/runnables/base.js
+++ b/dist/runnables/base.js
@@ -1683,6 +1683,73 @@ export class RunnableWithFallbacks extends Runnable {
         await runManager?.handleChainError(firstError);
         throw firstError;
     }
+    async *_streamIterator(input, options) {
+        const callbackManager_ = await CallbackManager.configure(options?.callbacks, undefined, options?.tags, undefined, options?.metadata);
+        const { runId, ...otherOptions } = options ?? {};
+        const runManager = await callbackManager_?.handleChainStart(this.toJSON(), _coerceToDict(input, "input"), runId, undefined, undefined, undefined, otherOptions?.runName);
+
+        let chunk;
+        let firstError = undefined;
+        let stream = undefined;
+        let concatSupported = false;
+        let finalOutput = undefined;
+        for await (const runnable of this.runnables()) {
+            concatSupported = true;
+            try {
+                stream = await runnable.stream(input, patchConfig(otherOptions, { callbacks: runManager?.getChild() }));
+                chunk = stream.next().value;
+
+                // Clear the error since we successfully got a chunk from this runnable
+                firstError = undefined;
+            } catch (e) {
+                if (firstError === undefined) {
+                    firstError = e;
+                }
+            }
+
+            // Check if we successfully got a chunk from this runnable
+            if (!firstError) {
+                // Use the stream from this runnable for the rest of the chunks
+                break;
+            }
+        }
+
+        if (firstError) {
+            await runManager?.handleChainError(firstError);
+            throw firstError;
+        }
+
+        if (chunk) {
+            yield chunk;
+        }
+
+        if (stream) {
+            try {
+                for await (chunk of stream) {
+                    if (concatSupported) {
+                        if (finalOutput === undefined) {
+                            finalOutput = chunk;
+                        } else {
+                            try {
+                                // eslint-disable-next-line @typescript-eslint/no-explicit-any
+                                finalOutput = concat(finalOutput, chunk);
+                            }
+                            catch (e) {
+                                finalOutput = undefined;
+                                concatSupported = false;
+                            }
+                        }
+                    }
+                    yield chunk;
+                }
+            } catch (e) {
+                await runManager?.handleChainError(e);
+                throw e;
+            }
+        }
+
+        await runManager?.handleChainEnd(finalOutput);
+    }
     async batch(inputs, options, batchOptions) {
         if (batchOptions?.returnExceptions) {
             throw new Error("Not implemented.");

Error Message and Stack Trace (if applicable)
No response
Description
(Modified description of the situation based on dosubot's description from a similar issue:)
The stream method implementation provided in the base class (from which other runnables inherit) is designed to work with individual Runnable instances. It leverages an asynchronous generator (_streamIterator) to yield results as they become available.
The RunnableWithFallbacks and similar constructs do not override or provide their own implementations of the stream method that would allow them to handle streaming appropriately across all constituent runnables. Without such an implementation, attempting to stream through a RunnableWithFallbacks defaults to the base class's streaming behavior, which does not implement streaming/call streaming on the actual primary or fallback models.
We did a quick local patch of the built files. I've added the patch code above even though it's not the best way to fix the issue. The patch code is just for reference.
System Info
> rush-pnpm show langchain

<EMAIL_ADDRESS>| MIT | deps: 16 | versions: 281
Typescript bindings for langchain
https://github.com/langchain-ai/langchainjs/tree/main/langchain/

keywords: llm, ai, gpt3, chain, prompt, prompt engineering, chatgpt, machine learning, ml, openai, embeddings, vectorstores

dist
.tarball: https://registry.npmjs.org/langchain/-/langchain-0.2.9.tgz
.shasum: 1341bdd7166f4f6da0b9337f363e409a79523dbb
.integrity: sha512-iZ0l7BDVfoifqZlDl1gy3JP5mIdhYjWiToPlDnlmfHD748cw3okvF0gZo0ruT4nbftnQcaM7JzPUiNC43UPfgg==
.unpackedSize: 4.0 MB

dependencies:
@langchain/core: >=0.2.11 <0.3.0  @langchain/textsplitters: ~0.0.0  js-tiktoken: ^1.0.12              jsonpointer: ^5.0.1               langsmith: ~0.1.30                openapi-types: ^12.1.3            uuid: ^10.0.0                     zod-to-json-schema: ^3.22.3
@langchain/openai: >=0.1.0 <0.3.0 binary-extensions: ^2.2.0         js-yaml: ^4.1.0                   langchainhub: ~0.0.8              ml-distance: ^4.0.0               p-retry: 4                        yaml: ^2.2.1                      zod: ^3.22.4

maintainers:
- nfcampos<EMAIL_ADDRESS>- jacoblee93<EMAIL_ADDRESS>- andrewnguonly<EMAIL_ADDRESS>- davidduong<EMAIL_ADDRESS>- hwchase17<EMAIL_ADDRESS>- basproul<EMAIL_ADDRESS>
dist-tags:
latest: 0.2.9     next: 0.2.3-rc.0

published 13 hours ago by jacoblee93<EMAIL_ADDRESS>
Platform:
mac
> node --version
Node v20.11.1

> rush-pnpm --version
pnpm 8.15.4

@dosu Seems like the streaming support should be implemented within Langchain, not by consumers. Please forward to a human for review.

So this will basically check if pulling the initial chunk throws an error, and falls back if it does? Otherwise, if there's a streaming error mid-stream, it'll throw?

Yes, that code is very similar to the Python implementation (which should be used as the basis for a real implementation for this - this version was a bit cobbled together and run through LLM + modeled after other methods in the Runnable* classes).

Seems reasonable - if you're up for making it a PR would appreciate that otherwise I can when I get a minute.

Just to comment - we would greatly value this fix/improvement as well.

Live in<EMAIL_ADDRESS>


Draggable render ref

The ref for Draggable's render function property is not actually passed, I checked the source code and the typescript signature also confirms this, yet the documentation suggests otherwise.
Draggable's render function should be passed a ref object like how Containers's render function is. I believe this is a bug.

I have the same issue the page broken when pass the ref to "react-windows" List as a child


Sync JSON files and headers with current master branch

Synced with godotengine/godot@b90ef52eceef51ff6ad58ca230b159642f984651.

@akien-mga I get holding off with the 3.0 and 3.1 branches until there is an official release, but I think its worth our while to keep master as in sync with Godot master as possible?


revert to exported - GrNewald-Human-10x3pv3

Dataset/group this task is for:
GrNewald-Human-10x3pv3 - Single-cell sequencing of human midbrain reveals glial activation and a Parkinson-specific neuronal state.

project uuid - 739ef78a-ba5d-4487-a013-9982db66d222
link to ingest UI

Relevant People

Wrangler: Rachel
Contributor<EMAIL_ADDRESS><EMAIL_ADDRESS>dev

Description of the problem with the dataset
need to add corresponding author details, please revert complete -> exported (or submitted? not sure the difference)

Reverted to valid

Reverted to valid


Compile main.cc with nvcc and g++ in cmake

Not straightforward to compile with nvcc and g++ the same target (main.cc). Currently is solved with making a temporal copy of the main.cc file: https://gitlab.cern.ch/ebrondol/clue/-/blob/master/src/CMakeLists.txt#L62

@fdplacido suggested to look at these codes:

https://gitlab.cern.ch/lhcb/Allen/-/blob/master/CMakeLists.txt#L408
https://gitlab.cern.ch/lhcb/Allen/-/blob/master/CMakeLists.txt#L426


Showing Error no "8" while running push notifications for iOS

I am using three devices.The following error code is repeated for three times
Error is: 8
Note  { encoding: 'utf8',
payload:
{ messageFrom: 'Colomer',
aps: { badge: 1, sound: 'notification-beep.wav', alert: [Object] } },
expiry: 0,
identifier: 0,
priority: 10,
retryLimit: -1,
device: { token: <Buffer 65 18 87 34 c8 98 9c 12 b9 16 68 b0 8d 5d 7f e2 31 2c 4c 42 21 19 84 16 01 8f aa 6f 76 a6 e9
6a> },
alert: { body: 'New offer from Colomer', 'action-loc-key': 'View' },
badge: 1,
sound: 'notification-beep.wav',
newsstandAvailable: undefined,
contentAvailable: undefined,
mdm: undefined,
compiled: true,
truncateAtWordEnd: false,
urlArgs: undefined,
'content-available': 1,
compiledPayload: '{"messageFrom":"Colomer","aps":{"badge":1,"sound":"notification-beep.wav","alert":{"body":"New offer
from Colomer","action-loc-key":"View"}}}' }

For future reference if somebody else ends up here, in my case this was caused by missing root CAs. A good guide explaining this error can be found here: https://stackoverflow.com/questions/23343910/verify-errornum-20-when-connecting-to-gateway-sandbox-push-apple-com

@Kuzmin how did you add the CA? Just combine it into your cert file? there is no option in node-apn to include a CA cert.

@medoix You need to install the CA as a root CA on your computer (and subsequently the server where this will be running), or download it and provide it to the ca option in node-apn Connection constructor. The ca option can be seen at https://github.com/argon/node-apn/blob/master/doc/connection.markdown
Either way, you will need to follow the verified answer on StackOverflow to ensure that you download the right certificates. (There are several similarly named CA's on entrust, the correct one should be named entrust_2048_ca.cer, and you can further verify that it's the right one by following the instructions on StackOverflow)


pyspider not crawling anymore after taskdb has grown

I am trying to find out why pyspider is not crawling anymore. The task db has 30 million records, 17mil pending and 13mil success.
After restarting all the pyspider components, it does a "SELECT taskid,project,schedule FROM site1 WHERE status = 1" and then nothing happens. No crawl, everything is idle, no mysql queries and server load is 0.
I cant really see what is wrong, right now I assume the taskdb is too big for pyspider to handle. I even tried running it on a 32gb memory server.
Do you have any idea what can go wrong in this scenario?  Thanks

The design of task db is about 10 million each project for mysql. I think it tasks too much time to load them into memory while starting up(you can CTRL-C the scheduler, it should break at Scheduler._load_tasks).
A workaround for this is to add a limit for the sql, eg SELECT taskid,project,schedule FROM site1 WHERE status = 1 LIMIT 100000. But you have to restart scheduler when all of the tasks loaded is done.

Another disadvantage of making the limit is when you are applying new tasks which is in ACTIVE status would take more time to de-duplicate.

I see. This is pretty big limitation for large crawls, but very interesting to solve. Is there a timeout value currently that makes it impossible to load the tasks? Because the memory of the server isnt the limitation.

"a timeout value" is a timeout for Scheduler._load_tasks? It may not a reliable solution as it depends on the performance of database.
To build a large capacity scheduler, memory is the limitation. There are two aspect for the limitation. First, users always prefer lower resource consumption. Second, the design of scheduler, tasks of one project must be stored in memory of the same host. That's say, even if you can distribute different project to different hosts, the tasks of the same project is limited by the memory capacity of a single server (about hundred millions). Only if design a scheduler that can distribute the tasks to different instance (which is too complex for pyspider).
Another limitation is the performance of database. Reflected in the time of loading tasks into memory and the performance of query and update new requests.
Currently solution is to use priority and handwritten script to reduce the tasks in ACTIVE status and the number of new requests. With distributed mongodb cluster, one of the project in a pyspider instance has 300 million tasks in it (but not more than 1 hundred thousand tasks in ACTIVE).
But if you do want to have that large number of ACTIVE tasks, not loading them into memory in one-time is a reasonable solution as the tasks couldn't be fetched in one day. It's used in the spider of Baidu too. We select the tasks in order of priority with the number that can be done in 24 hours every day. You can design such mechanism too.

If we somehow write the priority flag in the taskdb project table, can we make scheduler doing a series of queries looped with a limit ? So scheduler execute query every X time with a limit and gets new tasks to feed the fetchers. If we write priority into the table, we can still keep priority feature working. Isnt this better than a preset 24 hours limit?
The problem right now is that the scheduler is too greedy trying to get the whole dataset, but like you said.. this will always be limited by system specs (cpu, memory etc).
I like the idea of creating a external script to manage the tasks, but rather use this as a last resort if this cant be solved properly in pyspider.

The 24 hours limit is used because it's more easier to cooperate with other components (eg. new requests is not insert into database in real time, it is merged every 24 hours, before select tasks to feed fetcher).
I think with good design of script by using priority, the number of ACTIVE task can be controlled in a low
quantity. That's the reason scheduler is trying load every tasks greedily.
SELECT tasks FROM taskdb ORDER BY priority LIMIT 10000 and do it again when tasks all done is workable. But it needs modify the database scheme, that's the last thing I want to change.

Current these are the errors when restarting scheduling with 30mil+ tasks project.
[I 150212 13:02:21 base_connection:179] Connecting to <IP_ADDRESS>:5672
[I 150212 13:02:21 base_connection:179] Connecting to <IP_ADDRESS>:5672
[I 150212 13:02:21 base_connection:179] Connecting to <IP_ADDRESS>:5672
[I 150212 13:02:21 scheduler:395] loading projects
[D 150212 13:02:21 scheduler:83] project: site1 updated.
[E 150212 13:02:30 scheduler:405] 2013: Lost connection to MySQL server during query
    Traceback (most recent call last):
      File "/opt/pyspider/pyspider/scheduler/scheduler.py", line 400, in run
        self.run_once()
      File "/opt/pyspider/pyspider/scheduler/scheduler.py", line 384, in run_once
        self._update_projects()
      File "/opt/pyspider/pyspider/scheduler/scheduler.py", line 82, in _update_projects
        self._update_project(project)
      File "/opt/pyspider/pyspider/scheduler/scheduler.py", line 98, in _update_project
        self._load_tasks(project['name'])
      File "/opt/pyspider/pyspider/scheduler/scheduler.py", line 129, in _load_tasks
        self.taskdb.ACTIVE, project, self.scheduler_task_fields
      File "/opt/pyspider/pyspider/database/mysql/taskdb.py", line 84, in load_tasks
        tablename, what=fields, where=where, where_values=(status, )
      File "/opt/pyspider/pyspider/database/basedb.py", line 70, in _select2dic
        for row in dbcur:
      File "/usr/lib/python2.7/site-packages/mysql/connector/cursor.py", line 804, in fetchone
        row = self._fetch_row()
      File "/usr/lib/python2.7/site-packages/mysql/connector/cursor.py", line 782, in _fetch_row
        binary=self._binary, columns=self.description)
      File "/usr/lib/python2.7/site-packages/mysql/connector/connection.py", line 688, in get_row
        (rows, eof) = self.get_rows(count=1, binary=binary, columns=columns)
      File "/usr/lib/python2.7/site-packages/mysql/connector/connection.py", line 671, in get_rows
        rows = self._protocol.read_text_result(self._socket, count)
      File "/usr/lib/python2.7/site-packages/mysql/connector/protocol.py", line 309, in read_text_result
        packet = sock.recv()
      File "/usr/lib/python2.7/site-packages/mysql/connector/network.py", line 226, in recv_plain
        raise errors.InterfaceError(errno=2013)
    InterfaceError: 2013: Lost connection to MySQL server during query
[D 150212 13:02:30 scheduler:83] project: site1 updated.
[E 150212 13:08:38 scheduler:405] Can not reconnect to MySQL after 1 attempt(s): Unread result found.
    Traceback (most recent call last):
      File "/opt/pyspider/pyspider/scheduler/scheduler.py", line 400, in run
        self.run_once()
      File "/opt/pyspider/pyspider/scheduler/scheduler.py", line 386, in run_once
        self._check_request()
      File "/opt/pyspider/pyspider/scheduler/scheduler.py", line 256, in _check_request
        fields=self.merge_task_fields)
      File "/opt/pyspider/pyspider/database/mysql/taskdb.py", line 95, in get_task
        for each in self._select2dic(tablename, what=fields, where=where, where_values=(taskid, )):
      File "/opt/pyspider/pyspider/database/basedb.py", line 67, in _select2dic
        dbcur = self._execute(sql_query, where_values)
      File "/opt/pyspider/pyspider/database/basedb.py", line 35, in _execute
        dbcur = self.dbcur
      File "/opt/pyspider/pyspider/database/mysql/mysqlbase.py", line 21, in dbcur
        self.conn.ping(reconnect=True)
      File "/usr/lib/python2.7/site-packages/mysql/connector/connection.py", line 1015, in ping
        self.reconnect(attempts=attempts, delay=delay)
      File "/usr/lib/python2.7/site-packages/mysql/connector/connection.py", line 993, in reconnect
        raise errors.InterfaceError(msg)
    InterfaceError: Can not reconnect to MySQL after 1 attempt(s): Unread result found.

Yeah you are right, priority is very important with big crawls, I added priority later in the crawl. Thats why I have higher active/processed ratio. It was catching up, but slowly.
I found out the reason why scheduler is stopping in this case. Its a bug in python-connector-mysql 2.0.2 & 2.0.3.  This bug is occurs when a longblob column contains more than about 12 MB.
http://bugs.mysql.com/bug.php?id=74933
http://bugs.mysql.com/bug.php?id=74483
So even the latest available version of python-connector-mysql is not working. I made changes to the library and now the scheduler is working again.
But you have a point, when using priority properly and by controlling the number of ACTIVE tasks it should let the scheduler keep going.
So for now I think we can close this here and maybe write something in the wiki about the limitations. I will reopen issue when I bump into this problem, my other project is 500mil+ links.

That's really huge number, it could be a good use case of pyspider.
I will continue to consider how to reuse the code and make changes to Schedule and Database properly.


Cache socket port number in bolt class

The port number is (at least) queried for each debug log line. No matter if logging is enabled or not. Wrapping socket.getsockname() into a check whether debug logging is enabled or not, to only query it when necessary, made the driver slower for environments with a fast response (read: my local machine). Therefore, caching the port number is the only viable compromise. The downside is that this assumes that the port number never changes. As the driver is right now, it hands over all control over the socket to the Bolt class, so this assumption is safe. But it introduces complexity in the code base by adding the need to know about and adhere to this principle in future changes.
Background: a customer complained about the calls slowing down their deployment significantly. On my local machine it's actually really fast and commenting out the line in question (https://github.com/neo4j/neo4j-python-driver/blob/1ed96f94a7a59f49c473dadbb81715dc9651db98/neo4j/_async/io/_bolt5.py#L271 + in the other bolt versions) and timing a workload that hits that line frequently, didn't alter the driver's throughput in a measurable way. Timing this call in a Windows VM on my Linux machine showed it was 4 times slower. I didn't go as far as timing the whole driver in the VM.

The change is simple enough and looks ok to me. I'm wondering what the error would be if the port did happen to change without the cache knowing.


builtin command executeInlineValueProvider does not exist

Does this issue occur when all extensions are disabled?: Yes

VS Code Version: 1.81.1
OS Version: any

Steps to Reproduce:

in extension code, call
vscode.commands.executeCommand('vscode.executeInlineValueProvider',vscode.Uri.parse('c:/foo'),new vscode.Range(0, 0, 1, 1));
returned promise is rejected with Error: command '_executeInlineValueProvider' not found

I am developing an extension, and was not interested in this specific command, but noticed it was missing when I was looking for an example of a debug related built-in command.
I have no use for the command, I am just reporting it, because its in the the built-in commands wiki.

Looks like this was never implemented, the only commit that contains that string is https://github.com/microsoft/vscode/commit/e537fd9e3182ec03cf26d195fc17b184715628ec. @roblourens do you have extra context here, should this be removed or fixed up?

I don't have any extra context but I suppose it would make sense for that command to exist?

This feature request is now a candidate for our backlog. The community has 60 days to upvote the issue. If it receives 20 upvotes we will move it to our backlog. If not, we will close it. To learn more about how we handle feature requests, please see our documentation.
Happy Coding!

@jrieken do we automatically add these api commands for all language providers or only if they need to exist for some specific scenario?

For things in languages we have added them upfront but for other providers we don't have strict rules. It's fine to go by request or to treat it as FR

For verification, the only implementation of this API that I am aware of is in java-debug, so you can write an extension that calls executeCommand('vscode.executeInlineValueProvider' and call it while debugging java, or add a dummy provider like this
vscode.languages.registerInlineValuesProvider('javascript', {
	provideInlineValues(document, viewPort, context, token) {
		return [new vscode.InlineValueText(new vscode.Range(0, 0, 0, 1), 'hi')];
	},
})


Blockscout verification does not work

Component
Forge
Have you ensured that all of these are up to date?

[X] Foundry
[X] Foundryup

What version of Foundry are you on?
forge 0.2.0 (b44b045 2023-02-21T00:11:41.592203605Z)
What command(s) is the bug in?
forge verify-contract
Operating System
Linux
Describe the bug
It is not possible to verify contracts on any of the blockscout explorers right now.
Steps to reproduce:

Clone the one and only Multicall repo: https://github.com/mds1/multicall
forge build
forge v --verifier blockscout --verifier-url https://blockscout.com/eth/mainnet/api 0xcA11bde05977b3631167028862bE2a173976CA11 Multicall3 fails with

Submitting verification for [src/Multicall3.sol:Multicall3] "0xcA11bde05977b3631167028862bE2a173976CA11".
2023-02-21T19:58:25.991351Z ERROR etherscan: Failed to deserialize response: data did not match any variant of untagged enum ResponseData res="{\"message\":\"Something went wrong.\",\"result\":null,\"status\":\"0\"}"

forge v --verifier blockscout --verifier-url https://blockscout.com/eth/mainnet/api --compiler-version 0.8.12 --num-of-optimizations 10000000 0xcA11bde05977b3631167028862bE2a173976CA11 Multicall3 yeilds the very same error message above.

I think slightly related - i tried to verify on metis blockscout via
forge script scripts/Myscript.s.sol:Deploy --rpc-url metis --broadcast --ledger --legacy --mnemonic-indexes ${MNEMONIC_INDEX} --sender ${LEDGER_SENDER} --verify -vvvv

While this does not error it also doesn't do any verification.
The blockscout api for verification is compatible with etherscan afaik - only difference is that you don't need an api key, so i guess it should work.

Closing—duplicate of an existing newer verification issue and we've also merged a few improvements to blockscout verification which should have unblocked this.

The full command is:
$ forge script script/Load.s.sol:LoadScript --rpc-url $RPC_URL --private-key $DEPLOYER_PRIVATE_KEY --broadcast --verify --verifier blockscout --verifier-url $VERIFIER_URL


Name and Domain

We need to find a suitable name (STAC Index at the moment) and then register a domain for it.
cc @cholmes
Once that is done, enable HTTPS and create or update the Privacy Policy.

All done: stacindex.org


[WIP] Add lexicographical_topological_sort_py_cmp

This commit adds a new function lexicographical_topological_sort_py_cmp
that uses a python comparison to determine sort order.

This is mostly a test to see if doing this or the str comparison makes more sense

After doing some local testing with this function in qiskit-terra this function is significantly slower than the existing function. This took 265.5 seconds while for the same sorts for the lambda function version took 4 secs, and with #36 (and using that option) it took 3 secs.

The performance here is too slow to be worth pursuing this any further. Closing since it's not a realistic approach.


python3Packages.karton-autoit-ripper: init at 1.0.0

Motivation for this change
AutoIt script ripper for Karton framework
https://github.com/CERT-Polska/karton-autoit-ripper
Related to #81418
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[x] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 120742 run on x86_64-linux 1

4 packages built:

python38Packages.autoit-ripper
python38Packages.karton-autoit-ripper
python39Packages.autoit-ripper
python39Packages.karton-autoit-ripper

Result of nixpkgs-review pr 120742 at d8c45d87 run on aarch64-linux 1

2 packages marked as broken and skipped:

python38Packages.karton-autoit-ripper
python39Packages.karton-autoit-ripper

2 packages built successfully:

python38Packages.autoit-ripper
python39Packages.autoit-ripper

@fabaff please take a look why those packages are marked broken.

@fabaff please take a look why those packages are marked broken.

I think they have a dependency that's marked as broken on aarch64.


[DONT MERGE]Fix/temp soln for load metrics

Whether to use this solution will be decided later.
Temp solution for load metrics

Don't need this anymore.


Add a license because licenses are cool

Feel free to choose something other than MIT.

Wait wdym there's no list of checks PRs need to pass? :p
(that's a joke kekw)


Add Navbar, Footer and dynamic page switching

Steps that were taken to complete this project:

Add Linters for HTML/CSS.
Use best practices for HTML & CSS.
Update the README file.
Add Mark up to the .html file.
Used Bootstrap for styling
Add Form to save books.
Use Local storage to save user data.
Users can add books and can remove them from the list.
Using multiple classes
Add a navigation bar, footer and a timestamp
Create a new contact section
Created function to be used in onclick event listener which involves clicking the nav item to display the corresponding page.
Timestamp was created dynamically using Javascript.

I had fixed most of the linter issues in JS, however, one error that it was displaying stated that I could not use more than one class. Fixing this may lead to a change in the outcome of my code as I used a total of 3 classes and will be very time-consuming to change. So I then added a " /* eslint-disable */ " to the top of the script.js file so that it could pass the linter tests and avoid small errors that it was displaying. My code works and fulfils all the requirements of the project. Also, another linter error stated said I did not use the function created, but it was called in an onclick event.


allow statcounter it is just a website which track os and other stuff

Allow 'statcounter.com'

Thank you for this @userforsource.
This domain is listed by three of our curators
data/someonewhocares.org/hosts
data/yoyo.org/hosts
data/mvps.org/hosts

This won't change anytime soon.
Closing.


1.2105.3: healthcheck passes while rubber-soul not replicating and not trying to reconnect to rethinkdb

Do close if it's likely that this issue has already been addressed after 1.2105.3, and let me know which PR/commit would address the root cause, thanks.
In a prod k8s cluster we have a healthy rubber-soul pod that is not replicating to es.
At the time of inspection both es and rethink were up and serving dependant services just fine. rethink-proxy had been online since 2021-06-19T09:11:31+08:00 (had been restarted at that time).
last logs from in rubber-soul container are from the moment that rethink-proxy service was restarting (19/06/21), with no additional log entries between then and 13/07/21.
It appears that the re-connection attempt to rethink was never retried again (I believe it should retry until reconnected), yet the container did not start failing healthchecks.
This meant the issue went undetected until specific missing search data was noticed by end users.
Normal functionality of rubber-soul resumed after a MANUAL pod restart.
...
2021-06-15T20:43:26+08:00 level=DEBUG  time=2021-06-15T12:43:26Z source=rubber_soul.table_manager method=backfill model=Settings subcount=100
2021-06-15T20:43:26+08:00 level=DEBUG  time=2021-06-15T12:43:26Z source=rubber_soul.table_manager method=backfill model=Settings subcount=41
2021-06-15T20:43:26+08:00 level=DEBUG  time=2021-06-15T12:43:26Z source=rubber_soul.table_manager method=backfill model=Settings subcount=100
2021-06-15T20:43:26+08:00 level=INFO   time=2021-06-15T12:43:26Z source=rubber_soul.table_manager method=backfill model=Settings count=441
2021-06-19T09:11:29+08:00 level=ERROR  time=2021-06-19T01:11:29Z source=rethink_db.connection message="reconnecting"
2021-06-19T09:11:29+08:00 End of file reached (IO::EOFError)
2021-06-19T09:11:29+08:00   from ???
2021-06-19T09:11:29+08:00   from ???
2021-06-19T09:11:29+08:00   from ???
2021-06-19T09:11:29+08:00   from ???
2021-06-19T09:11:29+08:00   from ???
2021-06-19T09:11:29+08:00
2021-06-19T09:11:30+08:00 Unhandled exception in spawn: Error connecting to 'dev-rethinkdb-proxy.rethinkdb:28015': Connection refused (Socket::ConnectError)
2021-06-19T09:11:30+08:00   from ???
2021-06-19T09:11:30+08:00   from ???
2021-06-19T09:11:30+08:00   from ???
2021-06-19T09:11:30+08:00   from ???
2021-06-19T09:11:30+08:00   from ???
<END - no additional entries>

Made changes that likely fix this issue.
Please monitor for abnormality.
If this is not resolved, the issue will be deeper in the code, probably at the DB driver level.


Release 0.1.2

Release PR - merge when ready.
Steps:

Release the beta channel bundle from the artifacts.
Test beta. Make changes to release branch as necessary.
Release the stable channel.
Merge this PR.

Closed for another PR. #15


Various small changes + create systemd timers

Hi,
I've made some changes to your role that I think you might like. I have the use case where I want to run a script before running rsnapshot (I want to ensure the src/dst directories are mountpoints and mount a ceph snapshot). Given the script for the two backup points are not the same, I decided to split my two backup points into two different rsync instances and let systemd run the script as preExec. Furthermore, the playbook adds a systemd service and timers to let systemd handle when a backup should be run.
This PR makes the following changes:

Allow options to be set for each backup point ( see example further down )
Create systemd .service and .timers for hour, daily, weekly and hourly backups - can be disabled by setting "enable_systemd_timers" to false.
Allows disabling retain periods from the playbook by setting  the value to -1 ( see example )
Adds a variable for disabling file logging
Allows to change the rsync config path from the playbook ( nice if multiple rsync instances are run )

Example config:
tasks:
  - include_role:
       name: ansible-rsnapshot
     vars:
       - rsnapshot_systemd_postfix: "_projectX"
       - rsnapshot_config_file: /etc/rsnapshot_projectX.conf
       - rsnapshot_lockfile: "/var/run/rsnapshot_projectX.pid"
       - rsnapshot_snapshot_root: "/mnt/bkup_dst/"
       - rsnapshot_backup_points:
         - dir: '/mnt/ProjectX_src/'
           host: 'HostA'
           options: '+rsync_long_args=--no-relative'
       - rsnapshot_retain_monthly: 6
       - rsnapshot_retain_weekly: 4
       - rsnapshot_retain_daily: 7
       - rsnapshot_retain_hourly: -1 # note, hourly backups are not run)
       - rsnapshot_link_dest: 0
       - rsnapshot_systemd_preexec: "/usr/local/sbin/preScript.sh"
       - rsnapshot_systemd_postexec: "/usr/local/sbin/PostScript.sh"
       - rsnapshot_systemd_onfailure: "/usr/local/sbin/failureScript.sh"
       - rsnapshot_verbose: "3"

Please let me know if you don't like some of the changes.

Hi @vbotka ! Why haven't you merged this pull request yet?

@Exchizz can you do these changes?

Done - I'm not aware if any other variables should be moved.

@Exchizz can you do these changes?

Thank you for the comments, Dmitry! I've finally managed to start reviewing the patches.
Thank you for your contribution,  Mathias! Sorry for the delay. I'm working on it now. Let me complete the whole review. Some patches will be accepted right now. But, there must be also some changes. Please let me finish the review. I'll come back to you with the summary today.

Hi Mathias! I've accepted all PR into the 'devel' branch and made some changes:
* File renamed to tasks/systemd-timers.yml
* Variable 'enable_systemd_timers' renamed to 'rsnapshot_systemd_timers'
* Variable 'rsnapshot_systemd_timers' depends on OS. Moved to vars/defaults/*
* Default 'rsnapshot_systemd_timers: false'. Backward compatible. Do
  not break crontab installations.
* Debug updated
* Added tags rsnapshot_systemd_timers, rsnapshot_systemd_timers_*
* Truthy values changed to true/false because of .yamllint
* Cleanup, comments, and formatting
* Removed condition from 'Install rsnapshot timers'. Create all timers.
* Added 'Disable rsnapshot timers'
* Fixed conditions 'lookup/vars'
* Get and display timers when 'rsnapshot_debug=true'

I've tested the role with Ubuntu 20.04 with rsnapshot_systemd_timers: false only. Would it be possible to review the changes and potentially test it? Let me know if you have any questions.
Thank you,
-vlado

The problem with the custom configuration files for timers hasn't been resolved yet,e.g. rsnapshot_systemd_postfix: "_projectX". It will be necessary to separate the configuration of the timers from the default configuration, I think. Do you have any idea?

The problem with systemd configuration has been solved. The tasks systemd*, configure.yml, and templates have been updated to accept the configuration loop from the dictionary rsnapshot_systemd_conf. Please review the changes
    * systemd example in README updated
    * Changed structure of systemd config dictionary rsnapshot_systemd_conf d({})
    * Added var rsnapshot_snapshot_root_create d(true)
    * Renamed var -rsnapshot_systemd_timers to rsnapshot_systemd d(false)
    * Moved var rsnapshot_systemd from vars/defaults/* to default/main.yml
    * Loop enabled configure.yml and templates
    * Updated debug.yml
    * Renamed task to systemd.yml and split to systemd-conf.yml and systemd-service.yml
    * Tested with Ubuntu 20.04

The test with Ubuntu 20.04 failed to start the timers

failed: [plana] (item=monthly) => {"ansible_loop_var": "item", "changed": false, "item": "monthly", "msg": "Unable to start service /etc/systemd/system/rsnapshot_projectX-monthly.timer: Failed to start etc-systemd-system-rsnapshot_projectX\x2dmonthly.timer.mount: Unit etc-systemd-system-rsnapshot_projectX\x2dmonthly.timer.mount not found.\n"}

See the full log
Would it be possible to help me to resolve this problem?

Yes sure, I will get back to you

In some of the files, you prepend the variables with "conf." such as "conf.rsnapshot_config_file". The role fails to install due to not all variables have conf. prepended.
Do you want all variables to start with conf.... ?  :)

The problem with systemd configuration has been solved. The tasks systemd*, configure.yml, and templates have been updated to accept the configuration loop from the dictionary rsnapshot_systemd_conf. Please review the changes
    * systemd example in README updated
    * Changed structure of systemd config dictionary rsnapshot_systemd_conf d({})
    * Added var rsnapshot_snapshot_root_create d(true)
    * Renamed var -rsnapshot_systemd_timers to rsnapshot_systemd d(false)
    * Moved var rsnapshot_systemd from vars/defaults/* to default/main.yml
    * Loop enabled configure.yml and templates
    * Updated debug.yml
    * Renamed task to systemd.yml and split to systemd-conf.yml and systemd-service.yml
    * Tested with Ubuntu 20.04

The test with Ubuntu 20.04 failed to start the timers

failed: [plana] (item=monthly) => {"ansible_loop_var": "item", "changed": false, "item": "monthly", "msg": "Unable to start service /etc/systemd/system/rsnapshot_projectX-monthly.timer: Failed to start etc-systemd-system-rsnapshot_projectX\x2dmonthly.timer.mount: Unit etc-systemd-system-rsnapshot_projectX\x2dmonthly.timer.mount not found.\n"}

See the full log
Would it be possible to help me to resolve this problem?

(I'm not really sure how to add the commit properly to this pull request)
I can successfully install rsnapshot on ubuntu 20.04 when I add conf. to some of the variables (as mentioned above) and when applying the following commit to fix the systemd enable error:
9a4b410c7814ae766c2e3114369f4603994fd98f

Why do you create /usr/local/etc/rsnapshot.conf when /etc/rsnapshot_projectX.conf is also created ?

The main idea of all the changes I've made is to "run the role once" to configure both user-space (/usr/local/etc/rsnapshot.conf ) and systemd (/etc/rsnapshot_projectX.conf, when enabled). The default values of the user-space configuration are also used to configure systemd. The task configure.yml and the template rsnapshot.conf.j2 are shared. This makes both the usage and maintenance easier. By default the configuration of systemd is disabled (rsnapshot_systemd: false). This makes all the changes introduced by this feature backward compatible.

In some of the files, you prepend the variables with "conf." such as "conf.rsnapshot_config_file".
The role fails to install due to not all variables have conf. prepended.
Do you want all variables to start with conf.... ? :)

Short answer: No. The variable "conf" is dynamically created :)
Details:
See tasks/systemd.yml
- name: "systemd: Create configuration"
  include_tasks: systemd-conf.yml
  loop: "{{ rsnapshot_systemd_conf|dict2items }}"
  loop_control:
    loop_var: oitem
  vars:
    rsnapshot_config_file: "{{ '/etc/rsnapshot_' ~  oitem.key ~ '.conf' }}"
    postfix: "{{ oitem.key }}"
    conf: "{{ oitem.value }}"
  tags: rsnapshot_systemd_conf

The purpose of this task is to prepare the necessary local variables and loop the dictionary rsnapshot_systemd_conf.
See tasks/systemd-conf.yml
- name: "systemd-conf: Debug"
  ...
- name: "systemd-conf: Configure {{ postfix }}"
  include_tasks: configure.yml

The purpose of this task is to display the debug only and include configure.yml
See tasks/configure.yml
...
- name: "configure: Create configuration {{ rsnapshot_config_file }}"
  template:
    src: rsnapshot.conf.j2
    dest: "{{ rsnapshot_config_file }}"
    owner: "{{ conf.rsnapshot_config_file_owner|default(rsnapshot_config_file_owner) }}"
    group: "{{ conf.rsnapshot_config_file_group|default(rsnapshot_config_file_group) }}"
    mode: "{{ conf.rsnapshot_config_file_mode|default(rsnapshot_config_file_mode) }}"
    backup: "{{ rsnapshot_backup_conf }}"
    validate: "rsnapshot -c %s configtest"

The purpose of this task is to share the code to configure both user-space (default values) and systemd (values in the conf dictionary; coming from the the dictionary rsnapshot_systemd_conf).

The role fails to install due to not all variables have conf. prepended.

This should not happen. My intention is to "default" filter all conf.* attributes. Have I omitted any such variable? Would it be possible to share the error?

The role fails to install due to not all variables have conf. prepended.

This should not happen. My intention is to "default" filter all conf.* attributes. Have I omitted any such variable? Would it be possible to share the error?

ah I get it :)
TASK [ansible-rsnapshot : configure: Create directory {{ conf.rsnapshot_snapshot_root| default(rsnapshot_snapshot_root) }}] **************************************************************************************************************************************************************************************************
fatal: [box]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'conf' is undefined\n\nThe error appears to have been in '/...../ansible-rsnapshot/tasks/configure.yml': line 3, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: \"configure: Create directory {{ conf.rsnapshot_snapshot_root|\n  ^ here\nThis one looks easy to fix.  It seems that there is a value started\nwith a quote, and the YAML parser is expecting to see the line ended\nwith the same kind of quote.  For instance:\n\n    when: \"ok\" in result.stdout\n\nCould be written as:\n\n   when: '\"ok\" in result.stdout'\n\nOr equivalently:\n\n   when: \"'ok' in result.stdout\"\n"}

(I'm not really sure how to add the commit properly to this pull request)
dc445be

See tasks/systemd-service.yml how the variables "rsnapshot_systemd_name, postfix , conf" are used.

In the log of the test box I can see:

May 14 12:00:00 localhost systemd[370463]<EMAIL_ADDRESS>Failed to execute command: No such file or directory
May 14 12:00:00 localhost systemd[370463]<EMAIL_ADDRESS>Failed at step EXEC spawning /usr/local/sbin/preScript.sh: No such file or directory

It would help if you checkout "devel", create the scripts, and test it.

TASK [ansible-rsnapshot : configure: Create directory {{ conf.rsnapshot_snapshot_root| default(rsnapshot_snapshot_root) }}] **************************************************************************************************************************************************************************************************
fatal: [box]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'conf' is undefined\n\nThe error appears to have been in '/...../ansible-rsnapshot/tasks/configure.yml': line 3, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: "configure: Create directory {{ conf.rsnapshot_snapshot_root|\n  ^ here\nThis one looks easy to fix.  It seems that there is a value started\nwith a quote, and the YAML parser is expecting to see the line ended\nwith the same kind of quote.  For instance:\n\n    when: "ok" in result.stdout\n\nCould be written as:\n\n   when: '"ok" in result.stdout'\n\nOr equivalently:\n\n   when: "'ok' in result.stdout"\n"}

I can't reproduce the problem. I see no problem with the name of the task  split to 2 lines.
tasks/configure.yml
- name: "configure: Create directory {{ conf.rsnapshot_snapshot_root|
                                        default(rsnapshot_snapshot_root) }}"
  file:
    state: directory
    ...


Add header item "Content-Length“

Add header item "Content-Length" to response when shareFilesFromDirectory is used to share files.
This PR is related to this #380  .

@nichbar Thanks for taking the time to create this. Can you please rebase from stable and clean the PR.


helm: don't lookup namespaced resource when intsall

What this PR does / why we need it:
Improve  the process of helm upgrade, more info see: https://github.com/hwameistor/hwameistor/issues/117
Special notes for your reviewer:

This PR only improves the process of helm upgrade and avoids the problem of deleting pod.
There are two cluster level resources(clusterrole, clusterrolebinding) that retain the lookup judgment.

Does this PR introduce a user-facing change?

NONE

@alexzhc cc

yes, this issue applies to any parameter upgrade by helm.

Removed "if not lookup" from clusterrole and binding yamls

Another workaround
$ helm template helm/hwameistor \
        -s templates/post-install-claim-disks.yaml \
        --set storageNodes='{k8s-worker-3,k8s-worker-4,k8s-worker-5}' \
        | kubectl apply -f -


numalign: support multiple devices from same resource pool

Add support for multiple devices from the same extended resource pool,
which would be exported in an environment variable like:
PCIDEVICE_INTEL_COM_SRIOV=0000:03:02.1,0000:03:04.3

https://github.com/k8snetworkplumbingwg/sriov-network-device-plugin#pod-device-information
Signed-off-by: Phil Sphicas<EMAIL_ADDRESS>

thanks for the fix! I had limited use cases with multiple devices, and thus I limited the code to just the first occurrence, but there is no real reason to ignore the other devices.
In general, the code is starting to show its age and I'm starting to consider a rewrite from the ground up.


Customize header and footer

We are currently using openEQUELLA version 2019.2, and are working with Next Education Services on a government project.
We need to use the new UI template which offers responsive design.
It is not possible to add custom link or small images in the header / footer by changing the CSS stylesheet.
We believe that this functionality would be helpful for other openEQUELLA users.
Can openEQUELLA developers update this??”

Garnet here from Next Education Services, to share that we are collaborating with @aponsl.
We have shared current status of functionality based on the different openEQUELLA versions.  If anyone would like more context or information please let me know.
Thanks
Garnet

@aponsl @garnetberrry Could you please clarify your requirements with some further information and possibly screen shots of what you need to achieve? Thank you.

Hi @cathfitz
We would like to share a mock up showing a Header and Footer within responsive openEQUELLA UI 2019.2 as desired by Government end user.  This is important as the external links represent things like Legal and GDPR.
Requirements:
Responsive Header and Footer containing linkable pictures (e.g. logos) and/or linkable text
Mock up design:
Header, as you can see in the image, there are 2 logos and some text
Footer, there are 4/5 logos which are linkable along with some linkable text
*if possible enabling JAVA SCRIPT (or similar) in the Header / Footer would be favourable so that admin can create own link behaviour.
Example, like to other authentication pages or system triggers.
Image:


[stable/mysql] add MySQL password secret annotations

Is this a new chart

NOTE: We're experiencing a high volume of PRs to this repo and reviews will be delayed. Please host your own chart repository and submit your repository to the Helm Hub instead of this repo to make them discoverable to the community. Here is how to submit new chart repositories to the Helm Hub.

What this PR does / why we need it:
This PR adds annotations for MySQL password secret.
Which issue this PR fixes
(optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged)

fixes #

Special notes for your reviewer:
Checklist
[Place an '[x]' (no spaces) in all applicable fields. Please remove unrelated fields.]

[x] DCO signed
[x] Chart Version bumped
[x] Variables are documented in the README.md
[x] Title of the PR starts with chart name (e.g. [stable/mychartname])

/assign @olemarkus


Update celery, kombu for compatibility with pymongo 3.x

This fixes the following error seen in girder:
  ...
  File "/home/msmolens/dev/girder/plugins/romanesco/server/__init__.py", line 64, in schedule
'romanesco.run', job['args'], job['kwargs'])
  File "/home/msmolens/dev/python/girder.git/local/lib/python2.7/site-packages/celery/app/base.py", line 351, in send_task
reply_to=reply_to or self.oid, **options
  File "/home/msmolens/dev/python/girder.git/local/lib/python2.7/site-packages/celery/app/amqp.py", line 305, in publish_task
**kwargs
  File "/home/msmolens/dev/python/girder.git/local/lib/python2.7/site-packages/kombu/messaging.py", line 168, in publish
routing_key, mandatory, immediate, exchange, declare)
  File "/home/msmolens/dev/python/girder.git/local/lib/python2.7/site-packages/kombu/connection.py", line 440, in _ensured
return fun(*args, **kwargs)
  File "/home/msmolens/dev/python/girder.git/local/lib/python2.7/site-packages/kombu/messaging.py", line 180, in _publish
[maybe_declare(entity) for entity in declare]
  File "/home/msmolens/dev/python/girder.git/local/lib/python2.7/site-packages/kombu/messaging.py", line 111, in maybe_declare
return maybe_declare(entity, self.channel, retry, **retry_policy)
  File "/home/msmolens/dev/python/girder.git/local/lib/python2.7/site-packages/kombu/common.py", line 99, in maybe_declare
return _maybe_declare(entity)
  File "/home/msmolens/dev/python/girder.git/local/lib/python2.7/site-packages/kombu/common.py", line 110, in _maybe_declare
entity.declare()
  File "/home/msmolens/dev/python/girder.git/local/lib/python2.7/site-packages/kombu/entity.py", line 505, in declare
self.queue_declare(nowait, passive=False)
  File "/home/msmolens/dev/python/girder.git/local/lib/python2.7/site-packages/kombu/entity.py", line 531, in queue_declare
nowait=nowait)
  File "/home/msmolens/dev/python/girder.git/local/lib/python2.7/site-packages/kombu/transport/virtual/__init__.py", line 447, in queue_declare
return queue_declare_ok_t(queue, self._size(queue), 0)
  File "/home/msmolens/dev/python/girder.git/local/lib/python2.7/site-packages/kombu/transport/mongodb.py", line 115, in _size
return self.get_messages().find({'queue': queue}).count()
  File "/home/msmolens/dev/python/girder.git/local/lib/python2.7/site-packages/kombu/transport/mongodb.py", line 262, in get_messages
return self.client[DEFAULT_MESSAGES_COLLECTION]
  File "/home/msmolens/dev/python/girder.git/local/lib/python2.7/site-packages/kombu/transport/mongodb.py", line 256, in client
self._open()
  File "/home/msmolens/dev/python/girder.git/local/lib/python2.7/site-packages/kombu/transport/mongodb.py", line 179, in _open
use_greenlets=_detect_environment() != 'default',
  File "/home/msmolens/dev/python/girder.git/local/lib/python2.7/site-packages/pymongo/mongo_client.py", line 322, in __init__
username, password, dbase, opts)
  File "/home/msmolens/dev/python/girder.git/local/lib/python2.7/site-packages/pymongo/client_options.py", line 108, in __init__
options = dict([validate(opt, val) for opt, val in iteritems(options)])
  File "/home/msmolens/dev/python/girder.git/local/lib/python2.7/site-packages/pymongo/common.py", line 452, in validate
value = validator(option, value)
  File "/home/msmolens/dev/python/girder.git/local/lib/python2.7/site-packages/pymongo/common.py", line 100, in raise_config_error
raise ConfigurationError("Unknown option %s" % (key,))
ConfigurationError: Unknown option auto_start_request

I was also experiencing this error, thanks!
LGTM.

@msmolens I just gave you write access to this repo. Feel free to push the large green merge button once you get LGTM on this and any future branch, and feel free to push topic branches to the central repo. Welcome to Romanesco!


Adding covers and logos for 42 Lausanne coalitions

Hello !
I noticed that my coalition badge was not displayed on '42 Stats Card ', so I added the covers and badges of the coalitions on campus 42 Lausanne.
Thank you 😊

thanks for contributed!!!

@all-contributors please add @ricardoreves  for code, bug


Додаткова інфо

Knowledge
•	HTML, CSS, SCSS, Bootstrap
•	JS,   NODE, React, Angular,  Vue/VueX
•	GULP, WEBPACK
•	FIGMA,  PhotoSHop(нарезка), GITHUB
•	Mongo, Firebase, MySQL

Additional info


floj unit tests are hanging/failing

Unit tests get stuck here when running mvn clean package:
testTransactionsRetain(org.floj.core.ParseByteCacheTest)  Time elapsed: 0.039 sec  <<< ERROR!
java.nio.BufferUnderflowException
    at org.floj.core.ParseByteCacheTest.testTransaction(ParseByteCacheTest.java:373)
    at org.floj.core.ParseByteCacheTest.testTransactionsRetain(ParseByteCacheTest.java:153)

Running org.floj.core.PeerAddressTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec - in org.floj.core.PeerAddressTest
Running org.floj.core.PeerGroupTest

Running mvn clean package -DskipTests for now

Hi Chris, I couldn't get those unit tests to work with Bitcoinj, so I didn't update them for my part of the work, as it could be a couple of month long project; I just translated Bitcoinj to work with floj, but I didn't repair any of bitcoinj's inherent problems.

Okay, thanks, that's good to know. We'll (the floj community, I hope) attack them later.

In Bitcoinj, tags v0.14.7 and the master branch pass all tests for me without any changes; previous versions require skipping the PaymentSessionTest and ignoring testless projects thus: -Dtest='!PaymentSessionTest' -DfailIfNoTests=false  (The version tagged segwit also requires skipping KeyChainGroupTest, eg. -Dtest='!PaymentSessionTest,!KeyChainGroupTest' -DfailIfNoTests=false)
Currently in this project, there are 4 tests that hang, so we have to run the following in order to bypass those tests:
mvn package -Dtest='!FilteredBlockAndPartialMerkleTreeTests,!PeerGroupTest,!PeerTest,!TransactionBroadcastTest'
That gives the output: Tests run: 5119, Failures: 347, Errors: 153, Skipped: 4
I could see skipping those 4 tests (maybe they require additional, network-related infrastructure?), though I have a feeling they'll work after other unit tests are fixed.
@rengelking What version of bitcoinj did you fork (against which you couldn't get unit tests to work)?

Looking over the non-hanging test, I think close to all of them are related to the changes of parameters from Bitcoint to FLO. I've created a new branch, failing-tests-repair, that will be for fixing code or the tests to pass.


Override ports in Docker/Swarm mode

Do you want to request a feature or report a bug?
Feature
What did you do?
I might be totally missing something. Anyway, this not obvious from the docs.
Assume I have docker image that exposes a port 8080.
And I want to expose it via traefik at 80
With a file backend I would do it like this:
# traefik.toml
logLevel = "DEBUG"
defaultEntryPoints = ["http"]
[entryPoints]
  [entryPoints.http]
  address = ":80"

[file]
# rules
[backends]
  [backends.backend1]
    [backends.backend1.servers.server1]
    url = "http://<IP_ADDRESS>:8080"

[frontends]
  [frontends.frontend1]
  backend = "backend1"
  entrypoints = ["http"]
    [frontends.frontend1.routes.test_1]
    rule = "Host:sub.example.com"

But there is no way to express that with docker/swarm backend.
If there is, I will be happy to some docs around it.

Thanks for your interest in Traefik !
Every question in the issues means features less 😉.
The issue tracker is for reporting bugs and feature requests only.
For end-user related support questions, refer to one of the following:

Stack Overflow using the "traefik" tag
the Traefik community Slack channel