[WIP] str() to accept optional length

Intended to fulfil https://github.com/iovisor/bpftrace/issues/199.
It's pretty close to working, but I think I need to ask for help at this point.
Integer literals work:
bpftrace -e 'tracepoint:syscalls:sys_enter_write /pid == 23506/ { printf("<%s>\n", str(args->buf, 2));  }'
# type pwd into a terminal

<w>
<d>
# it printed 1 character instead of the 2 we requested; probably probe_read_str replaces final byte with a NULL. fixable.

But it doesn't seem to be able to use items from structs (e.g. args->count):
sudo bpftrace -e 'tracepoint:syscalls:sys_enter_write /pid == 6034/ { printf("<%s>\n", str(args->buf, args->count)); }'
Attaching 1 probe...
Error loading program: tracepoint:syscalls:sys_enter_write (try -v)

The verbose output shows the error:
R2 min value is negative, either use unsigned or 'var &= const'

I think R2 is the size arg to bpf_probe_read_str(void *dst, int size, const void *unsafe_ptr).
I think it wants me to prove that the size will always be positive.
This is despite the fact that I've already cast it to a 64-bit unsigned:
b_.CreateStore(
  b_.CreateIntCast(expr_, b_.getInt64Ty(), false),
  strlen);

I also tried using a bitmask:
b_.CreateStore(
  b_.CreateAnd(
    b_.CreateIntCast(expr_, b_.getInt64Ty(), false),
    0xffffffff),
  strlen);

The bitmask changed the error to:
R2 unbounded memory access, use 'var &= const' or 'if (var < const)'

I also tried using CreateMaxNum():
b_.CreateStore(
  b_.CreateMaxNum(
    b_.getInt64(0),
    b_.CreateIntCast(expr_, b_.getInt64Ty(), false),
    "ensure_positive"),
  strlen);

CreateMaxNum() changed the error to:
Segmentation fault

I also tried using a condition:
call.vargs->at(1)->accept(*this);
Value *len_u64 = b_.CreateIntCast(expr_, b_.getInt64Ty(), false);

Function *parent = b_.GetInsertBlock()->getParent();
BasicBlock *positive = BasicBlock::Create(module_->getContext(), "positive", parent);
BasicBlock *negative = BasicBlock::Create(module_->getContext(), "negative", parent);
BasicBlock *done = BasicBlock::Create(module_->getContext(), "done", parent);
b_.CreateCondBr(
  b_.CreateICmpUGE(len_u64, b_.getInt64(0), "if_positive"),
  positive,
  negative);

b_.SetInsertPoint(positive);
b_.CreateStore(len_u64, strlen);
b_.CreateBr(done);

b_.SetInsertPoint(negative);
b_.CreateStore(b_.getInt64(1), strlen);
b_.CreateBr(done);

b_.SetInsertPoint(done);

But this resulted in the same:
R2 min value is negative, either use unsigned or 'var &= const'

Does anybody here have any ideas on how to overcome this error?

Here's the -dd output:
https://gist.github.com/Birch-san/c478887d8db25f58a2b8280939349882

Everything you've tried makes sense to me, and I've followed a similar path when developing earlier features, where I've hit the same errors before. I wish I documented them in https://github.com/iovisor/bpftrace/blob/master/docs/internals_development.md -- someone should at some point: have a section for each error message and what it means with suggested solutions.
It helps to read https://github.com/torvalds/linux/blob/master/kernel/bpf/verifier.c . Eg:
		if (reg->umax_value >= BPF_MAX_VAR_SIZ) {
			verbose(env, "R%d unbounded memory access, use 'var &= const' or 'if (var < const)'\n",
				regno);
			return -EACCES;
		}

And BPF_MAX_VAR_SIZ (where's the E???) is (1 << 29). So yeah, that would suggest that using a bitmask is a good idea, but you're already trying that. Is the b_.CreateIntCast() necessary? Can we bitmask to expr_ directly?
What you wrote serves as good documentation for diagnosing this type of issues anyway, so if we figure out the solution it might be nice to copy what you wrote and put it in internals_development.md.

Thanks for the hint.
I tried omitting the 64-bit cast; no improvement.* I also tried moving the bit mask to be as late as possible (wrapping it around b_.CreateLoad(strlen)), so that it's definitely constraining the arg on b_.CreateProbeReadStr(). No improvement.*
*These changes randomized the error to one of the other error messages already reported. I didn't document exactly what change resulted in which error. Might be an idea for my next pass.
I've found the bpf self-test that reproduces the error:
R2 min value is negative, either use unsigned or 'var &= const'
https://github.com/torvalds/linux/blob/master/tools/testing/selftests/bpf/test_verifier.c#L7378
It's followed by other assertions on bounds, and finally a "correct bounds" test (which hopefully reveals how to make this thing work):
https://github.com/torvalds/linux/blob/master/tools/testing/selftests/bpf/test_verifier.c#L7419
I followed the history of the test (it's undergone some changes in wording) to when it was originally committed, with accompanying implementation code:
https://github.com/torvalds/linux/commit/06c1c049721a995dee2829ad13b24aaf5d7c5cce#diff-1d760ce3dbf1aad3a5e223501e8aaa7bR3622

Ah, I think I understand.
First, a recap... we start with this error:
R2 min value is negative, either use unsigned or 'var &= const'

The verifier wants proof that it's non-negative.
So, we bitmask with 0xFFFFFFFF, and we get to the next error:
R2 unbounded memory access, use 'var &= const' or 'if (var < const)'

So the verifier's satisfied with the minimum bound, but wants a maximum bound as well.
The snippet you posted was right on the money:
if (reg->umax_value >= BPF_MAX_VAR_SIZ) {
	verbose(env, "R%d unbounded memory access, use 'var &= const' or 'if (var < const)'\n",
		regno);
	return -EACCES;
}

And as you say: BPF_MAX_VAR_SIZ is 1 << 29. The upper limit exists to provide protection against overflows.
So, I think it is responding to the bitmask, and we just need to tweak the bitmask (or conditional branching) to ensure that the largest possible value is 1 << 29 - 1. Will give it a try soon.

okay, the smaller bitmask ((1ULL << 29) - 1 = 0x1FFFFFFF) certainly seems to have had an effect.
side-note: how to print unsigned 64-bit ints as hex.
We now get a new error:
invalid stack type R1 off=-72 access_size=536870911

536870911=0x1FFFFFFF (i.e. my bitmask)
Verbose output

I think it may be impossible to write a valid program.
Here's where the verifier kicks us out:
off = reg->off + reg->var_off.value;
if (off >= 0 || off < -MAX_BPF_STACK || off + access_size > 0 ||
    access_size < 0 || (access_size == 0 && !zero_size_allowed)) {
	verbose(env, "invalid stack type R%d off=%d access_size=%d\n",
		regno, off, access_size);
	return -EACCES;
}

We encounter...
invalid stack type R1 off=-72 access_size=536870911

...because off + access_size > 0.
The stack probably looks like this:

static int check_stack_boundary(struct bpf_verifier_env *env, int regno,   			int access_size, bool zero_size_allowed, struct bpf_call_arg_meta *meta)#L2032
static int check_helper_mem_access(struct bpf_verifier_env *env, int regno, int access_size, bool zero_size_allowed, struct bpf_call_arg_meta *meta)#L2087
static int check_func_arg(struct bpf_verifier_env *env, u32 regno, enum bpf_arg_type arg_type, struct bpf_call_arg_meta *meta)#L2260

The reason our access_size is so high (equals the bitmask) is because that's the unsigned maximum of the register:
err = check_helper_mem_access(env, regno - 1,
			      reg->umax_value,
zero_size_allowed, meta);

…and that's what gets passed all the way down into check_stack_boundary(…).
Ah, I think I'm starting to understand. probe_read_str writes into the BPF stack-allocated memory. I only allocated a string buffer of 64 bytes (STRING_SIZE) on the stack.
so I either need to parameterise the "length of string buffer", or tighten my bitmask to 64 to ensure the read string fits in the string buffer.

I tried doing dynamic allocations:
if (call.func == "str")
{
  AllocaInst *strlen = b_.CreateAllocaBPF(b_.getInt64Ty(), "strlen");
  b_.CreateMemSet(strlen, b_.getInt8(0), sizeof(uint64_t), 1);
  if (call.vargs->size() > 1) {
    call.vargs->at(1)->accept(*this);
    b_.CreateStore(b_.CreateAnd(expr_, 0x1fffffff), strlen);
  } else {
    b_.CreateStore(b_.getInt64(STRING_SIZE), strlen);
  }

  AllocaInst *buf = b_.CreateAllocaBPF(b_.getInt8Ty(), b_.CreateLoad(strlen), "str");
  b_.CreateMemSet(buf, b_.getInt8(0), b_.CreateLoad(strlen), 1);
  call.vargs->front()->accept(*this);
  b_.CreateProbeReadStr(buf, b_.CreateLoad(strlen), expr_);
  b_.CreateLifetimeEnd(strlen);
  expr_ = buf;
}

Alas:
str(args->buf, args->count) encounters a sefault. No -ve was able to be generated, but here's the -dd).
str(args->buf, 1) encounters invalid stack off=14 size=1. -ve here, -dd here.
str(args->buf, 64) works fine. 63 does not.
So, something (printf?)'s trying to begin a read (at least?) 64 bytes out of my string buffer, and in the 1–63 case: I'm not allocating enough string buffer to accommodate that.
Maybe that's the key to solving the segfault in the case of str(args->buf, args->count). Or maybe I'm barking up the wrong try.
also: it'd be educational to see what happens if I constrain args->count to a 0xFF bitmask, since a lot of things (my string buffer allocation, and perhaps printf) are working with 64.

constraining args->count to a 0x0000003F bitmask (e.g. <=63) passes the verifier.
Here's the -v, -dd, and output:
sudo bpftrace -e 'tracepoint:syscalls:sys_enter_write /pid == 6034/ { printf("%d:<%s>\n", args->count, str(args->buf, args->count)); }'
Running...
1:

1:<w>
1:<d>
1:<
>
30:</home/anon/git/bpftrace/build
>
116:<[01;>

anon@anon-VirtualBox:~/git/bpftrace/build$ pwd
/home/anon/git/bpftrace/build
anon@anon-VirtualBox:~/git/bpftrace/build$

I think part of the problem is a friction between semantic_analyser and codegen_llvm.
semantic_analyser is responsible for populating bpftrace_.printf_args_, which ultimately determines in codegen_llvm:

how many bytes we allocate per arg (when allocating the printf_args)
how many bytes we MEMCPY per arg (when writing the printf_args)

I don't think semantic_analyser should be in charge of that; it couldn't know up-front how big a changing string can be.
codegen_llvm could only learn the final size of the string by using call.vargs->at(n) and running the visitor on it. we do this already, but separately from measuring/allocating the struct. maybe we'll be forced to couple those steps.

Closed in favor of https://github.com/iovisor/bpftrace/pull/241


Sidebar link not working

Not exactly sure how the links in the sidebar menu are supposed to work. For me they do not work.
I've inserted a link for the Profile of the User menu as follows. And the link doesn't work. Upon clicking, it closes the "User" menu.
sidebar:
# Logo (from /images/logos/___.svg)
logo: fresh-square
sections:
- title: User
icon: user
links:
- text: Profile
url: https://www.google.com
- text: Account
url: /
- text: Settings
url: /

@raheems That appears to be an issue with the CSS Ninja theme itself, which I do not maintain. So unfortunately I'm not really sure how to fix it.

I would encourage you, though, to see if there's a JavaScript fix in fresh.js.


New version: Ferdium.Ferdium.Nightly version 6.7.1-nightly.14

[ ] Have you signed the Contributor License Agreement?
[ ] Have you checked that there aren't other open pull requests for the same manifest update/change?
[ ] This PR only modifies one (1) manifest
[ ] Have you validated your manifest locally with winget validate --manifest <path>?
[ ] Have you tested your manifest locally with winget install --manifest <path>?
[ ] Does your manifest conform to the 1.5 schema?

Note: <path> is the name of the directory containing the manifest you're submitting.

Microsoft Reviewers: Open in CodeFlow

Hi @Alphrag,
The package didn't pass a Defender or similar security scan. This might be a false positive and we can rescan tomorrow.
(Automated response - build 694.)

Automatic Validation ended with:

Installation failed with exit code -1978335187 2024-01-31 20:08:49.782 [FAIL] Installer failed security check. Url: https://github.com/ferdium/ferdium-app/releases/download/v6.7.1-nightly.14/Ferdium-win-AutoSetup-6.7.1-nightly.14-ia32.exe Result: 0x80004005

(Automated response - build 694.)

/AzurePipelines run

Hi @Alphrag,
The package didn't pass a Defender or similar security scan. This might be a false positive and we can rescan tomorrow.
(Automated response - build 695.)

Automatic Validation ended with:

Installation failed with exit code -1978335187 2024-02-02 02:07:34.858 [FAIL] Installer failed security check. Url: https://github.com/ferdium/ferdium-app/releases/download/v6.7.1-nightly.14/Ferdium-win-AutoSetup-6.7.1-nightly.14-ia32.exe Result: 0x80004005

(Automated response - build 695.)

/AzurePipelines run


Raw HTTP swagger

How to auto generate swagger for raw http request?
Is there any way to manually define the swagger?

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 45f70e36-c511-0d20-e809-2085b81bab9e
Version Independent ID: 7e13d7ec-01fd-ecd9-05de-8fb333564ec8
Content: Author entry script for advanced scenarios - Azure Machine Learning entry script authoring
Content Source: articles/machine-learning/v1/how-to-deploy-advanced-entry-script.md
Service: machine-learning
Sub-service: mlops
GitHub Login: @dem108
Microsoft Alias: sehan

@congkun-ck
Thanks for your feedback! We will investigate and update as appropriate.

@RamanathanChinnappan-MSFT
Thanks. I figured out from the code. It can be specified by putting swagger[n].json under AML_APP_ROOT.


DSD-78 DEV-14564 - Upgrades module to use terraform 13

https://datatamr.atlassian.net/browse/DEV-14564

:tada: This PR is included in version 1.0.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


fix(bootstrap): get correct radio value through queryselector

fix #1187

I know this is old, but the material version seems to have the same problem

@Nielsb85 could you please fill a new issue with a reproduction example. Thanks!


bug: Missing Dependency when running

Describe the bug
Hello,
I followed the instructions on github (nothing more) and when I try to run it with the following command:
sudo docker run -it --rm -p 3000:3000 google-flan-t5-xl-service:53fd1e22aa944eee1fd336f9aee8a437e01676ce serve
I'm getting the following error.
Error: [bentoml-cli] serve failed: Failed loading Bento from directory /home/bentoml/bento: Failed to import module "generated_flan_t5_service": No module named 'orjson'
To reproduce
No response
Logs
`sudo docker run -it --rm -p 3000:3000 google-flan-t5-xl-service:53fd1e22aa944eee1fd336f9aee8a437e01676ce serve
`

`Error: [bentoml-cli] `serve` failed: Failed loading Bento from directory /home/bentoml/bento: Failed to import module "generated_flan_t5_service": No module named 'orjson'`

Environment
python:3.9.17
bentoml:1.0.22
openllm:0.1.8

This is strange. Did you run into any issue when running containerize?

You shouldn't run openllm with sudo first and foremost, try again without sudo

It’s not related to sudo.
Having same error without sudo

This seems like the weirdest bug to me. orjson is a core dependency and should already include during install openllm in the container?
Would you mind running bentoml containerize ... --opt progress=plain and dump all of the output to a text file and send it here? Thanks.

I discover a bug on main and I will release a new version patch version promptly

can you try with 0.1.9 to see if you still run into this issue?

Still the same issue

Still the same issue.
We installed everything from scratch on another vm as well, doesn’t work

Hey there, I have identified the issue and will have a patch release soon. sorry for the trouble

Did you still run into this issue?

I was having the same error. Upgraded to latest version (openllm, version 0.1.17). Build step with --overwrite and containerize step with --opt no-cache. It was solved.


StorageLoader: add tcpKeepAlive=true to JDBC for long-running COPYs via NAT

Otherwise we get the dread:

An I/O error occured while sending to the backend

For people who find this in the future: don't forget to configure your client's kernel to use TCP keep-alive correctly too:

http://tldp.org/HOWTO/TCP-Keepalive-HOWTO/usingkeepalive.html
http://docs.aws.amazon.com/redshift/latest/mgmt/connecting-firewall-guidance.html

JDBC settings: https://jdbc.postgresql.org/documentation/head/connect.html

Update: I'm not sure the TCP keep-alive settings in the kernel make a material difference...


Renaming will silently fail when program subtly doesn't typecheck

We've have this function:
let rec recur = (i) => {
  if (i) {
    0
  } else {
    recur(i, 10)
  }
};
let a = recur(1);

When trying to rename it to something else, the recursive call does not get renamed. This seems a bit bad because it destroys the trust I'd put in renaming tools. This kind of stuff would cause some very-hard to find bugs.
I'm guessing the reason is that the type checker doesn't know that that call is the same as the one being renamed.

The renaming functionality in the server is currently driven by the responses merlin gives to the occurrences query so this is something that would need to be addressed upstream. cc @let-def
Can you open an issue about it in the merlin repo?

Closing this since I've created the relevant issue upstream.


Tutorial Page mendix-onboarding.md Issue. PROD

Tutorial issue found: https://github.com/sap-tutorials/Tutorials/blob/master/tutorials/mendix-onboarding/mendix-onboarding.md contains invalid primary tag.
Your tutorial was not created. Please double-check primary tag property.
Each tutorial md-file shall have primary tag provided above. Example:
---
title: Text Bundles within Node.js SAP HANA applications
description: Working with text bundles in Node.js
primary_tag: products>sap-hana
tags: [ tutorial>intermediate, products>sap-hana, products>sap-hana--express-edition ]
---
Affected server: PROD

I've made the changes proposed.

Thank you for your feedback. The issue seems to be resolved, so I am closing the issue.
If you still have questions, feel free to reopen the issue.


Thread monitoring container that is stopped is not terminated properly

To reproduce:

start docker-zabbix-sender
start container
few seconds later, stop the container and waits for its termination.
send SIGINT signal (with CTRL-C for instance) to docker-zabbix-sender
The thread collecting stats for the dead container is not immediately stopped. Instead it ungracefully read-timeout a few seconds later.

Here is the stack I got:
Exception in thread happy_banach:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner
    self.run()
  File "/usr/local/lib/python2.7/dist-packages/docker_zabbix_sender-0.1a-py2.7.egg/docker_zabbix_sender/stats.py", line 63, in run
    for stats in stream:
  File "/usr/local/lib/python2.7/dist-packages/docker_py-1.0.0-py2.7.egg/docker/client.py", line 166, in _stream_helper
    data = reader.read(1)
  File "/usr/lib/python2.7/dist-packages/urllib3/response.py", line 174, in read
    data = self._fp.read(amt)
  File "/usr/lib/python2.7/httplib.py", line 543, in read
    return self._read_chunked(amt)
  File "/usr/lib/python2.7/httplib.py", line 585, in _read_chunked
    line = self.fp.readline(_MAXLINE + 1)
  File "/usr/lib/python2.7/socket.py", line 476, in readline
    data = self._sock.recv(self._rbufsize)
timeout: timed out

Should be fixed by #5


Run a memory profiler on kytosd  with a AmLight production-like scenario and observe high allocations

Typically in development with a ring or linear,3 scenario with just a few EVCs, kytosd consumes 150-200 MB.
However, at AmLight, on docker ps the container was using 806.7MiB / 9.727GiB, the docker image is based on debian and is running probably a few more processes, but it shouldn't likely have that much difference. So, this needs a bit of research, either way, it'd be great to also have a development baseline with linear, 3, and amlight e2e topology running kytosd with a memory profile like https://github.com/bloomberg/memray for some time and then observe high allocations and map any potential issues that might be surfacing.
❯ ps_mem -p $(pgrep kytosd)
 Private  +   Shared  =  RAM used       Program

153.8 MiB +   1.4 MiB = 155.2 MiB       kytosd
---------------------------------
                        155.2 MiB
=================================

It'll set this a priority_medium without a target since it's not immediately impacting, eventually, it'll get picked up probably on 2023.2 scope too, we'll see.

I decided to watch the memory usage while running kytos with the e2e tests, as I thought it would be a good stress test of kytos. For me the Memory usage hovered around 200 to 300MiB.
dbfff37e7190   kytos-end-to-end-tests-kytos-1   2.86%     323.9MiB / 3.016GiB   10.49%    124MB / 168MB     134MB / 86.8MB    271

However, it should be noted this was from an image built for e2e tests relevant to my PRs, and not the official amlight image. Ill try running e2e tests again with the amlight docker image, and see if I get any different behaviour there.

Running e2e with the amlight image in particular is getting me around 200 to 250 MiB of memory usage.
CONTAINER ID   NAME                             CPU %     MEM USAGE / LIMIT     MEM %     NET I/O           BLOCK I/O       PIDS
af892da51559   kytos-end-to-end-tests-kytos-1   8.36%     243.6MiB / 3.016GiB   7.89%     86.4MB / 62.1MB   0B / 66MB       221

I'm trying to run kytosd with scalene but the interactive terminal keeps on closing so I can't keep the program running in the foreground. I think I remember in the past suggesting we move away from the requirement of the interactive terminal while running as a foreground process, specifically for running in docker containers, as then the docker container's lifetime could be directly tied to kytos.

@Ktmi, it's a good thing that on e2e at least you're seeing around 200-250 MB of RAM used. Maybe a future step is also to see if in a long running type of exec (over a few days while still running tests) in a server it stays that way too.
Regarding the memory profiler, it might be worth trying out other ones that might still be compatible with kytosd in foreground as it is. Other than that, you might also try to run it in the background mode. Yes, we do have an issue in our backlog to optionally run kytosd in the foreground without the embedded ipython shell https://github.com/kytos-ng/kytos/issues/104 but we'll likely not prioritize in the short term.
@italovalcy, would you also happen to have a chart of RAM usage of this container in prod? It'd be great to know if that docker stats sample mem usage was just showing a spike (and eventually garbage collection starts doing its job) or if it indeed might be leaking. That way @Ktmi can also confirm with his findings based on local e2e tests exec and also with the memory profiler that's he's trying to use. Thanks.


Sync submodule cmake

After merging this PR, we can proceed to a new release to update conda accordingly.
@cmastalli Do you agree?

Just by curiosity, why would conda need a release here ?

For packaging issues on the CMake aspects


i3 pin-workspaces behaviour for a named monitor output.

What type of PR is this? (check all applicable)

[ ] Refactor
[x] Feature
[ ] Bug Fix
[ ] Optimization
[x] Documentation Update
[ ] Other: Replace this with a description of the type of this PR

Description
Adds a new configuration for module i3:

pin-output - If pin-workspaces is set, will pin workspaces associated with the named monitor instead of the current monitor

Related Issues & Documents
Closes #1625
Documentation (check all applicable)

[x] This PR requires changes to the Wiki documentation (describe the changes)
[ ] This PR requires changes to the documentation inside the git repo (please add them to the PR).
[ ] Does not require documentation changes

Documentation to be added to i3 module page - under description of pin-output configuration:
; Pin workspaces defined on monitor 'DP-1' instead of the same output as the bar
;
; Default: empty
pin-output = DP-1

Should pin-output imply pin-workspaces? Seems more user-friendly.


Uniprot and Uniprot Somatic "incorrectly" rescued by beta API

While production has removed uniprot and uniprot somatic, the new ETL has rescued them.
The main reason for this is that the new ETL ingests whatever is in the files, but the uniprot file contains 3 datasources in the same file: uniprot, uniprot_somatic and uniprot_literature. This is not currently affecting FE, but it's displaying some data that we
I'm inclined to propose the solution that @mkarmona  implemented in the past. Subset the input file to only include uniprot_literature before ingesting it into the pipeline.
Long term, we will create the file with the right content, but we are still exploring the data to diagnose the value of it.

@AsierGonzalez provided the filtered file and the pipeline will pick the appropriate file in upcoming runs

After re run the ETL with the proper evidences a new output was generated
gs://ot-snapshots/etl/outputs/20.11/evidences/succeeded

This looks perfect @cmalangone :
❯ gsutil ls gs://ot-snapshots/etl/outputs/20.11/evidences/succeeded
gs://ot-snapshots/etl/outputs/20.11/evidences/succeeded/
gs://ot-snapshots/etl/outputs/20.11/evidences/succeeded/_SUCCESS
gs://ot-snapshots/etl/outputs/20.11/evidences/succeeded/sourceId=cancer_gene_census/
gs://ot-snapshots/etl/outputs/20.11/evidences/succeeded/sourceId=chembl/
gs://ot-snapshots/etl/outputs/20.11/evidences/succeeded/sourceId=clingen/
gs://ot-snapshots/etl/outputs/20.11/evidences/succeeded/sourceId=crispr/
gs://ot-snapshots/etl/outputs/20.11/evidences/succeeded/sourceId=europepmc/
gs://ot-snapshots/etl/outputs/20.11/evidences/succeeded/sourceId=eva/
gs://ot-snapshots/etl/outputs/20.11/evidences/succeeded/sourceId=eva_somatic/
gs://ot-snapshots/etl/outputs/20.11/evidences/succeeded/sourceId=expression_atlas/
gs://ot-snapshots/etl/outputs/20.11/evidences/succeeded/sourceId=gene2phenotype/
gs://ot-snapshots/etl/outputs/20.11/evidences/succeeded/sourceId=genomics_england/
gs://ot-snapshots/etl/outputs/20.11/evidences/succeeded/sourceId=intogen/
gs://ot-snapshots/etl/outputs/20.11/evidences/succeeded/sourceId=ot_genetics_portal/
gs://ot-snapshots/etl/outputs/20.11/evidences/succeeded/sourceId=phenodigm/
gs://ot-snapshots/etl/outputs/20.11/evidences/succeeded/sourceId=phewas_catalog/
gs://ot-snapshots/etl/outputs/20.11/evidences/succeeded/sourceId=progeny/
gs://ot-snapshots/etl/outputs/20.11/evidences/succeeded/sourceId=reactome/
gs://ot-snapshots/etl/outputs/20.11/evidences/succeeded/sourceId=slapenrich/
gs://ot-snapshots/etl/outputs/20.11/evidences/succeeded/sourceId=sysbio/
gs://ot-snapshots/etl/outputs/20.11/evidences/succeeded/sourceId=uniprot_literature/

I reload the index: uniprot_literature. The other two indices (uniprot and uniprot_somatic) are removed.
config.conf updated.


[Feature] Make it easier or document how to change the default port from 8080

Hi, port 8080 is quite common, and chances are it's used by other servers or WebUI.
Changing it in the docker-compose-dev.yaml isn't enough, because it's also hard-coded in other files, such as:
(LLocalsearch) 10:50:43 piero@gl753vd  LLocalsearch $ grep -R 8080 *
backend/Dockerfile.dev:EXPOSE 8080
backend/apiServer.go:	fmt.Println("Server started at http://localhost:8080")
backend/apiServer.go:	http.ListenAndServe(":8080", nil)
backend/Dockerfile:# Expose port 8080 to the outside world
backend/Dockerfile:EXPOSE 8080
grep: backend/tmp/main: binary file matches
docker-compose.dev.yaml:      - '8080:8080'
node_modules/vite/dist/node/index.d.ts:     * @example `http://<IP_ADDRESS>:8080`
node_modules/sade/readme.md:  .example('--port 8080 --etag')
node_modules/sade/readme.md:    $ sirv --port 8080 --etag
vite.config.ts:                target: 'http://backend-dev:8080',
...

Thank you,
Piero

Ah you're right, I missed that. Exposing the backend is no longer needed anyways. The frontend proxies request to /api to the backend (see vite config). Will remove this :)

I think I should really add a GitHub ci action to build proper containers and not let people build the project from source

Looks like it wasn't my backend that was exposed, but the searXNG container. Fixed that :)
Now only the frontend port (3000) should be exposed, changing that one should work via the docker-compose file


FolderChooserDialog.Builder not work in fragment

Hi
I am using this  code in fragment but error in this section that is bold
FolderChooserDialog.Builder(this or getActivtiy() or getContext())

https://github.com/afollestad/material-dialogs/issues/863


Same Sync Committees Can Span Multiple Periods

In Altair:
https://github.com/ethereum/eth2.0-specs/blob/dev/specs/altair/beacon-chain.md#get_sync_committee_indices
    base_epoch = Epoch((max(epoch // EPOCHS_PER_SYNC_COMMITTEE_PERIOD, 1) - 1) * EPOCHS_PER_SYNC_COMMITTEE_PERIOD)

The base_epoch will always be 0 for the first 2 committee periods.
If epoch is in
0 <= epoch < EPOCHS_PER_SYNC_COMMITTEE_PERIOD then:
(max(epoch // EPOCHS_PER_SYNC_COMMITTEE_PERIOD, 1) - 1) will equal 0.
If epoch is in
EPOCHS_PER_SYNC_COMMITTEE_PERIOD <= epoch < 2*EPOCHS_PER_SYNC_COMMITTEE_PERIOD then:
(max(epoch // EPOCHS_PER_SYNC_COMMITTEE_PERIOD, 1) - 1) will still equal 0.
While not an issue for mainnet as we are on epoch 36000, it might be relevant for any new devnet/testnets spun up. If this is an acceptable compromise at genesis then the issue can be ignored, thought it might be worth bringing up given how long lasting sync committees are.

The sync committee is always applied to both current and next in the fork transition processing. This is for simplicity in calculation and allows for not having a truncated committee duty (instead you get a longer duty) in the event that a fork is not on a sync committee period boundary
closing!


Duplicated url with AutoroutePart

Repro:

Create a content type with AutoroutePart.
Add two content items with the same url.

The second url should have a suffix or the editor should prevent this duplication.

In Orchard 1 if programmatically you store a content item (new or modified) with same url as other one that exists it arise problems cause both have same alias. So, then you need to add code to prevent it.
It would be interesting in Orchard 2 this is checked by the framework every time you store a content item with autoroute part no matter the scenario (editing it through admin UI, through recipe or programmatically) . So, the suffix would be added in the case it is detected a duplicate no matter the scenario. I mean you don't need to worry about that also when you add/edit a content item programmatically.

@lukaskabrt your fix added a new 'Published' column to the AutoroutePartIndex table so there was an error if as me, your instance has been created before this change.
It is ok because we are in Beta but next time, for that kind of change, please add an UpdateFrom1 function in Migrations.cs

I told him to do. He had made a migration, but it was useless as the old values would not be populated.
I know I have to provide a way to mark as index as stale so that it gets rebuilt. I will provide it soon, probably when I work on projections which will need something similar.


Risk: deploying too late or too soon

#14 Issue by smattingly,
See the notes on the QA risk issue. Similar ideas apply here.
Other risks related to deployment are usually connected with specific characteristics of the client organization, its processes, work schedules, etc.

A lot of student work could not be properly tracked until the system is up and running.

@abed12345 While that's a valid point, I don't think it is a risk-- I don't think it hurts our chances of delivering the system. Maybe you see it differently.

same

reviewed

reviewed

reviewed

reviewed


kafka: custom configuration

User provided custom configuration support

For Combined clusters

Single custom config secret

For Topology clusters

Separate custom config secret for brokers & controllers

Restrict some configurations for customization so that it doesn't break the cluster like - cluster ID, quorum voters address etc.


Correct IdP initiated single sign on

Resolves #302 and #304

@quartje @thijskh

:+1: for 302 and :+1: for 304


OOM error on a 384 VRAM GB 8x RTX A6000

OOM error on a 384 VRAM GB 8x RTX A6000
System information and environment

OS Platform and Distribution : Linux Ubuntu 1804
Python version: 3.8.10
CUDA version: 11.2
NCCL version:
cupy version: 11.2
GPU model and memory: 8x RTX A6000

Hi, did you solve your problem?

closed due to inactivity


Create new desktop app testing methods

Since the installer builder is now going to be the desktop app, with many major changes. This means that we need to methods and procedures to doing these tests.
Things that may/may not be needed:

Test template, as tests will likely be less frequent and possibly on request
Alterations/rewrite of QA book section.

New/Changed Aspects:

How many sims/what sims should be tested
What non sim aspects should be tested (locale, zooming, etc.)
Should old versions be kept
How often should routine tests be done (likely depends in part on how often builds happen)
How many/what platforms should be done
How will these builds be accessed to test

@KatieWoe, I recently installed Ubuntu (the most common Linux distribution) on my Linux laptop. If we ever need non-VM Linux testing for the Desktop app, I can do it.

@KatieWoe, I recently installed Ubuntu (the most common Linux distribution) on my Linux laptop. If we ever need non-VM Linux testing for the Desktop app, I can do it.

Great! At the moment, if I recall correctly, they may be having trouble getting a Linux distribution made, so no need at the moment.

Great! At the moment, if I recall correctly, they may be having trouble getting a Linux distribution made, so no need at the moment.

Most things here have been worked out. A test template will be made for https://github.com/phetsims/qa/issues/1143


Give more relevance to ationbar buttons

The current graphics are a bit "pale". We agree on having a clean and light approach but I think the current list of labels are a bit too week and action buttons don't have the right relevance. We're in the editor, so they should be the most prominent actions for the user, but in this moment the menu button have higher relevance.
We should make some tests, maybe restore the border for the buttons, use more contrast for the font color...

@giohappy as you suggested

much better IMHO, what do you think @allyoucanmap ?


Add /65 addressing limitation

Add the following limitation into the doc:
In case of wide IPv6 CIDRs (range≤/64) only the first /65 range is addressable (e.g. from x:x:x:x::0 to x:x:x:x:7fff:ffff:ffff:ffff).
Closes #83

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 40.68%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
443

Relevant Lines:
1089

💛  - Coveralls


dropdownAutoWidth support

support for dropdownAutoWidth attribute from select2 would be appreciated.

:+1:

+1! is that on the roadmap?

I'd also like to see this

This is better styled from CSS if possible, or modifications to at least do it that way.
Also commenters: please use the GitHub reactions instead, and refrain from comments that add no useful information/signal.

:+1:


⚠ Fix lint issues: govet linter

Please check if the PR fulfills these requirements

[ n/a] Tests for the changes have been added (for bug fixes / features)
[x] PR title follows the guidelines defined in  https://github.com/ossf/scorecard/blob/main/CONTRIBUTING.md#pr-process

What kind of change does this PR introduce? (Bug fix, feature, docs update, ...)
Code quality improvement

What is the current behavior? (You can also link to an open issue here)
This PR fixes issues from the govet linter. With current settings, the only checks enabled are for field alignment and invalid struct field tags. The fieldalignment check will look for structs that could take up less memory if the fields are re-ordered. It can also give the optimal ordering which is what I used for this commit.
Part of issue #173.

What is the new behavior (if this is a feature change)?
n/a

Does this PR introduce a breaking change? (What changes might users need to make in their application due to this PR?)
Yes. Currently, if the requested output is JSON and a check does not contain any details, the output will look like:
{"Name":"Signed-Releases","Pass":false,"Confidence":10,"Details":null}
With this change, "Details" will be omitted if empty:
{"Name":"Signed-Releases","Pass":false,"Confidence":10}
The omitempty tag was added to CheckResult.Details in #344, but it was not formatted correctly, so it did not have its intended effect.
An end user would have to change their application to check to make sure Details is present in the JSON before reading from it. If we think the benefit from omitting this field is not worth the breaking change, I can remove that commit from this PR.

Thank you! for taking all the warnings.
IMO For now let’s not have any breaking changes on the JSON.
I will let @azeemshaikh38 @inferno-chromium and @oliverchang  also chime in.


Fix error in password example

This forwards changes from https://github.com/marc2332/freya/commit/e37f66390671552235ab216eefeb62040cc846af to the password example.

My bad.. ty! Perhaps I could add a carco check just for examples in the CI...

I agree. I think testing the doctests would be good too.


Styles annotation with IDE or VSCode

Hey! First of all, I want to say thank you for this loader!
I'm using https://github.com/Jimdo/typings-for-css-modules-loader in a project with TS codebase and I'm starting to use your package for Rescript. But I can't understand how to enable integration with VSCode or Jetbrains Webstorm.
I have something like that (example) but I haven't autocompleted or styles annotation in my editors. My structure of component:

│── Button
│   │── Button.bs.js
│   │── Button.css
│   │── Button.re
│   │── ButtonStyles.bs.js
│   └── ButtonStyles.re

And my component:
let styles: ButtonStyles.definition = [%raw {| require("./ Button.css") |}];

[@ react.component]
let make = () => {
   <button className={Cn.append(styles##button, styles##test)}>
     {"Test" -> React.String}
   </button>
}

Do you know how to enable autocomplete? Or editors don't support this one yet?

As far as I know, [%raw] syntax doesn't work with syntax highlight.


The custom font cannot be loaded by UIFontPickerViewController.

For example, I installed Monaco by other font app, however, The Code App's UIFontPickerViewController can't found it.
I think UIFontPickerViewController only provides system font by default. The following step may fix this problem:
Add Fonts capability in  iOS app’s target, and select Use Installed Fonts.
Please see the following article: https://developer.apple.com/documentation/xcode/configuring-custom-fonts

#527 #804

Thanks for the heads up. I have added this entitlement and a new build will be available on TestFlight soon.

Custom fonts are shown but not loaded properly even with the entitlement. Investigation needed.

@EpsIotaPi Can you confirm if the latest commit fixes the issue? You can try it on TestFlight as soon as the build is available.

Thank you for your great work! I can confirm the latest commit already fixes this issue!


Consider using pyproject-metadata for metadata schema

I stumbled across the project pyproject-metadata which provides a nice dataclass that represents the core metadata portion of pyproject.toml, so I wanted to make a note of it in case it makes sense to use it here.

Sounds definitely worth investigating… why re-invent the wheel?


Install over private network

RancherOS Version: (ros os version)
Currently using 1.5.1 but any version I guess
Where are you running RancherOS? (docker-machine, AWS, GCE, baremetal, etc.)
Baremetal
Hi there,
Thanks for the amazing work, I have been using rancher since 1.2 and it keeps being great and better!
I am currently setting up a large volume installation process over a private network.
I am using iPXE +http server so I can easily provide any iso/vmlinuz/initrd/cloud-config etc...
Booting process is all fine and working well. But I want to install ros on those computers' disk.
For now the installation process is downloading rancher/os from github.com repository but I would like to avoid that and stay in my private network as much as possible.
I understand that the installation is looking from an iso on /dev/sr0
I am able to provide this iso (Custom rancheros.iso with preloaded images) over the network, I can mount it on /dev/sr0 but it doesn't seem to be enough (I suppose it's because /dev/sr0 is already mounted)
Would you have any suggestion here ? I can PXE from the initrd of my custom Rancher OS on probably preload rancher/os in there but I am not sure it will work.
I have another question which might have been covered in https://github.com/rancher/os/issues/2697 (But I am not 100% clear on it). Is there anyway to add preloaded images during installation process to the system being installed (So I can access those images straightaway at first boot.)
Thanks for your time & help.
Joris

If you want to autoformat the disk when booting by iPXE, you should add the autoformat part to kernel cmdline:
rancher.state.autoformat=[/dev/sda,/dev/vda] rancher.state.dev=LABEL=RANCHER_STATE rancher.state.wait

However, this does not install the bootloader to disk, so you cannot upgrade ROS.

If you want to install completely to disk, you should not use rancher.state.autoformat.
You can push rancher/os image to your registry, then use ros  install -i  myrepo/os:xxx to install.

Hi @niusmallnan thanks for your answer
I indeed want to fully install ROS to the disk with the bootloader (The machine will be moved afterward with a less reliable network). I'll look into hosting ros image in my registry.
Also is it possible to copy preloaded images to a "soon to be installed" ros while installing it? (I would like the preloaded images to be on my system before the first boot, I guess I can just mount /dev/sdaX and put a tar of docker images in /var/lib/rancher/preload/system-docker but I'm wondering if there is another way)
Thanks!

@minjatJ
You can add a cloud-init datasource like url, then add this source to RancherOS via iPXE.
You can add some scripts via cloud-init to help install to disk automatically:
#cloud-config
write_files:
  - path: /etc/rc.local
    permissions: "0755"
    owner: root
    content: |
      #!/bin/bash
      wait-for-docker
      if [ -e /dev/disk/by-label/RANCHER_STATE ]; then
          # installed to disk
          ...
      else
          # not install to disk
          # install, but don't reboot
          # mount the STATE device, and save the preload images.
      fi

Yes I'am already doing it with the following config
#cloud-config
hostname: rancherospxe
ssh_authorized_keys:
  - ssh-rsa AAAB....

write_files:
  - path: /etc/rc.local
    permissions: "0755"
    owner: root
    content: |
      #!/bin/bash
      (
      # ROS install will complain if /dev/sr0 doesn't exists
      mkdir /dev/sr0
      wait-for-docker
      echo y | ros install -f -c /cloud-config.yml -d /dev/sda
      ) &
  - path: /cloud-config.yml
    permissions: "0700"
    owner: root
    content: |
      #cloud-config
      hostname: freshros
      ssh_authorized_keys:
        - ssh-rsa AAAAB....
      write_files:
        - container: network
          path: /var/lib/iptables/eth1/rules.sh
          permissions: "0755"
          owner: root:root
          content: |
            #!/bin/bash
            set -ex
            # Open ports for rancher-agent registration
            iptables -I INPUT -p tcp --dport 6443 -j ACCEPT
            iptables -I INPUT -p tcp --dport 2376 -j ACCEPT
            # the last line of the file needs to be a blank line or a comment
        - path: /opt/install/install.sh
          permissions: "0544"
          owner: root
          content: |
            # Create VPN container
            system-docker run --restart=always --name vpn -d --cap-add=NET_ADMIN --net=host --device /dev/net/tun -v /opt/vpn:/vpn dperson/openvpn-client

            # Register rancher-agent
            docker run -d --privileged --restart=unless-stopped --net=host -v /etc/kubernetes:/etc/kubernetes -v /var/run:/var/run rancher/rancher-agent:v2.2.3 --server {.RancherServer} --token {.RancherAgentToken}  --etcd --controlplane --worker

            echo "VPN & Agent installed"
            touch /var/lib/rancher/install_on_disk.done
        - path: /etc/rc.local
          permissions: "0755"
          owner: root
          content: |
            #!/bin/bash
            (
            wait-for-docker

            # If vpn & agent are not installed yet
            if [ ! -e /var/lib/rancher/install_on_disk.done ];then
              /opt/install/install.sh >> /var/log/install.log 2>&1
            else
              # Need to manually start the system-docker vpn
              system-docker start vpn
              echo "All already installed" >>  /var/log/install.log
            fi
            ) &
        - path: /opt/vpn/vpn.ovpn
          owner: root
          permission: "0400"
          content: |
            client

            tls-client
            auth SHA256
            cipher AES-256-CBC
            remote-cert-tls server
            ....
      rancher:
        network:
          interfaces:
            l0:
            eth0:
              dhcp: true
            eth1:
              dhcp: true
              post_up:
              - /var/lib/iptables/eth1/rules.sh

I am not very happy with those wait-for-docker and manual checks but I can't find another way to do it. Would be great if we can wait for network (I tried while ! wget etc... but seems to be hanging)
Also after a manual reboot the system-docker container vpn (Started with restart=always) is not coming back automatically, I have to force a system-docker start vpn
Anyway those are minor issues that I should be able to fix soon.
Thanks for your help!

You can try custom-system-services for your vpn container. https://rancher.com/docs/os/v1.x/en/installation/system-services/custom-system-services/#labels
We don't have wait-for-network. Here are some you can refer to, I often use it.
    # check default routes
    for i in {1..10}; do
        # IPV4 UG: 0003
        # IPV6 UG: 00450003
        gateway=`cat /proc/net/route | grep 0003 | wc -l`
        gateway_v6=`cat /proc/net/ipv6_route | grep 00450003 | wc -l`
        if [[ $gateway -ge 1 || $gateway_v6 -ge 1 ]]; then
            ready="true"
            break
        else
            info "waiting for the default routes"
            sleep 1
        fi
    done

Yes I tried custom-system-services but I need access to the network in order to get the docker image but system services are launched before network (If I understand correctly). I also tried by create a service.yml in /var/lib/rancher/conf/vpn.yml then enable & up in rc.local but it seems that after a reboot I have to manually start it up again (Wondering if it's the intended behaviour or I might have miss something).
Thanks for the network snippet, that's very handy.
I'll close this ticket as I'm able to do pretty much everything I need.
Btw my company is trying to contact rancher but there isn't much reply so far (Slack/Forum/Contact page) any tip on who/where to contact for a potential high volume usage ?

Oh yeah and I didn't go for custom-system-service for my vpn container because of this issue https://github.com/rancher/os/issues/2776

@minjatJ I forwarded your message, maybe my colleague will find you soon.


Add Gateway's name to RemoteNickFormat

In order to support extra use cases we should add the {GATEWAY} tag to the RemoteNickFormat string which would be replaced by the value of the name= field from a gateway's configuration.
This is very useful when you are forwarding, for example, multiple channels from one chat to a single channel on another one (one-way). It will help you identify the source channel of a message on the target chat.

Thanks for your contribution!
Looks fine to me, but I think the matterclient/matterclient.go change shouldn't be there?

Hey. Sorry for the late reply. Was off on holiday last week. The change in matterclient/matterclient/go is just a drive-by fix that my linters were complaining about. The formatting directive is not correct. On second look my fix is not even enough as there are two formatting directives but only one input argument.
Should the whole thing be:
	if m.GetUser(data.UserId) == nil {
		m.log.Infof("User %s is not known, ignoring message %s", data.UserId, data.Message)
		return
	}

? In that case I would indeed not need to change the formatting directive.

Will also add the necessary documentation changes for the new {GATEWAY} tag in the RemoteNickId format.


[Loggable] Double logging on an entity which extends an abstract entity.

In my project I have developed an AbstractEntity to save having to rewrite the same functionality many times. The class contains status (published, draft, unpublished) and privacy (private, public) options.
I then have my main entities extend this class, i.e. Product.
The problem I have is that I want to log when the status and privacy are changed, as well as attributes on the Product entity. It appears that because I've added the versioned annotation to attributes in both classes that two entries are added into the db.
AbstractEntity.php
/**
 * @ORM\HasLifecycleCallbacks()
 */
abstract class AbstractEntity
{
    /**
     * @ORM\Column(type="string")
     * @Gedmo\Versioned
     */
    protected $status;

    /**
     * @ORM\Column(type="string")
     * @Gedmo\Versioned
     */
    protected $privacy;

Product.php
/**
 * @ORM\Entity
 * @ORM\Table(name="product")
 * @ORM\HasLifecycleCallbacks()
 * @Gedmo\Loggable(logEntryClass="\Noisegate\CmsBundle\Entity\Log\ProductLog")
 */
class Product extends AbstractEntity
{
    /**
     * @ORM\Column(type="string")
     * @Gedmo\Versioned
     */
    protected $name;

    /**
     * @ORM\Column(type="string")
     * @Gedmo\Versioned
     */
    protected $slug;

ProductLog.php
/**
 * @ORM\Entity(repositoryClass="Gedmo\Loggable\Entity\Repository\LogEntryRepository", readOnly=true)
 * @ORM\Table(
 *      name="log_product",
 *      indexes={
 *          @ORM\Index(name="log_class_lookup_idx", columns={"object_class"}),
 *          @ORM\Index(name="log_date_lookup_idx", columns={"logged_at"}),
 *          @ORM\Index(name="log_user_lookup_idx", columns={"username"}),
 *      }
 * )
 */
class ProductLog extends AbstractLogEntry
{

}

Is there a correct way to use loggable when extending classes?

the result is same if you add MappedSuperclass annotation? but if it allows abstract classes probably it needs to be fixed.

I tried adding @MappedSuperclass to my AbstractEntity class however the issue still occurs.
namespace MyApp\CmsBundle\Entity;

use Gedmo\Mapping\Annotation as Gedmo;
use Doctrine\ORM\Mapping as ORM;

/**
 * @ORM\HasLifecycleCallbacks()
 * @ORM\MappedSuperclass
 */
abstract class AbstractEntity
{

It appears I had configuration conflict:
gedmo.listener.loggable:
        class: Gedmo\Loggable\LoggableListener
        tags:
            - { name: doctrine.event_subscriber, connection: default }
        calls:
            - [ setAnnotationReader, [ "@annotation_reader" ] ]

and
stof_doctrine_extensions:
    default_locale: en
    orm:
        default:
            loggable: true

I have removed the Gedmo config and it's working as expected now.

well, you had two listeners doing the same :)


 U: .

Console output
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6446/1/console

versions
git=2.34.1 maven=3.9.8

Powered by
Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 16s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

_ master Compile Tests _

+1 :green_heart:
mvninstall
3m  2s

master passed

+1 :green_heart:
mvnsite
6m 32s

master passed

+1 :green_heart:
spotless
0m 49s

branch has no errors when running spotless:check.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
2m 52s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
mvnsite
6m 16s

the patch passed

+1 :green_heart:
xmllint
0m  0s

No new issues.

+1 :green_heart:
spotless
0m 45s

patch has no errors when running spotless:check.

_ Other Tests _

+1 :green_heart:
asflicense
0m 15s

The patch does not generate ASF License warnings.

22m 11s

Subsystem
Report/Notes

Docker
ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6446/1/artifact/yetus-general-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/6446

Optional Tests
dupname asflicense mvnsite codespell detsecrets xmllint spotless

uname
Linux 21ef54d1d407 5.4.0-200-generic #220-Ubuntu SMP Fri Sep 27 13:19:16 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master / 6c18fd1a23d1ef345079cbb97cd47a16b3484871

Max. process+thread count
83 (vs. ulimit of 30000)

modules
C: . U: .

Console output
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6446/1/console

versions
git=2.34.1 maven=3.9.8 xmllint=20913

Powered by
Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.


pars.py doesn't work anymore

pars.py doesn't work anymore
Got this since february 2022
Traceback (most recent call last):
File "C:\TG-INVITE\40\TGPars-main\pars.py", line 79, in
all_participants = client.get_participants(target_group, aggressive=True)
File "C:\Users\Singularity\AppData\Local\Programs\Python\Python310\lib\site-packages\telethon\sync.py", line 39, in syncified
return loop.run_until_complete(coro)
File "C:\Users\Singularity\AppData\Local\Programs\Python\Python310\lib\asyncio\base_events.py", line 641, in run_until_complete
return future.result()
File "C:\Users\Singularity\AppData\Local\Programs\Python\Python310\lib\site-packages\telethon\client\chats.py", line 507, in get_participants
return await self.iter_participants(*args, **kwargs).collect()
File "C:\Users\Singularity\AppData\Local\Programs\Python\Python310\lib\site-packages\telethon\requestiter.py", line 113, in collect
async for message in self:
File "C:\Users\Singularity\AppData\Local\Programs\Python\Python310\lib\site-packages\telethon\requestiter.py", line 74, in anext
if await self._load_next_chunk():
File "C:\Users\Singularity\AppData\Local\Programs\Python\Python310\lib\site-packages\telethon\client\chats.py", line 222, in _load_next_chunk
results = await self.client(self.requests)
File "C:\Users\Singularity\AppData\Local\Programs\Python\Python310\lib\site-packages\telethon\client\users.py", line 30, in call
return await self._call(self._sender, request, ordered=ordered)
File "C:\Users\Singularity\AppData\Local\Programs\Python\Python310\lib\site-packages\telethon\client\users.py", line 80, in _call
raise MultiError(exceptions, results, requests)
telethon.errors.common.MultiError: ([None, FloodWaitError('A wait of 29 seconds is required (caused by GetParticipantsRequest)'), FloodWaitError('A wait of 29 seconds is required (caused by GetParticipantsRequest)'), None, FloodWaitError('A wait of 29 seconds is required (caused by GetParticipantsRequest)'), None, FloodWaitError('A wait of 29 seconds is required (caused by GetParticipantsRequest)'), FloodWaitError('A wait of 29 seconds is required (caused by GetParticipantsRequest)'), FloodWaitError('A wait of 29 seconds is required (caused by GetParticipantsRequest)'), FloodWaitError('A wait of 29 seconds is required (caused by GetParticipantsRequest)'), FloodWaitError('A wait of 29 seconds is required (caused by GetParticipantsRequest)'), FloodWaitError('A wait of 29 seconds is required (caused by GetParticipantsRequest)'), FloodWaitError('A wait of 29 seconds is required (caused by GetParticipantsRequest)'), FloodWaitError('A wait of 29 seconds is required (caused by GetParticipantsRequest)'), FloodWaitError('A wait of 29 seconds is required (caused by GetParticipantsRequest)'), FloodWaitError('A wait of 29 seconds is required (caused by GetParticipantsRequest)'), FloodWaitError('A wait of 29 seconds is required (caused by GetParticipantsRequest)'), FloodWaitError('A wait of 29 seconds is required (caused by GetParticipantsRequest)'), FloodWaitError('A wait of 29 seconds is required (caused by GetParticipantsRequest)'), FloodWaitError('A wait of 29 seconds is required (caused by GetParticipantsRequest)'), FloodWaitError('A wait of 29 seconds is required (caused by GetParticipantsRequest)'), FloodWaitError('A wait of 29 seconds is required (caused by GetParticipantsRequest)'), FloodWaitError('A wait of 29 seconds is required (caused by GetParticipantsRequest)'), FloodWaitError('A wait of 29 seconds is required (caused by GetParticipantsRequest)'), FloodWaitError('A wait of 29 seconds is required (caused by GetParticipantsRequest)'), FloodWaitError('A wait of 29 seconds is required (caused by GetParticipantsRequest)')], [<telethon.tl.types.channels.ChannelParticipants object at 0x000001A94182DE40>, None, None, <telethon.tl.types.channels.ChannelParticipants object at 0x000001A940ABE9E0>, None, <telethon.tl.types.channels.ChannelParticipants object at 0x000001A941838A00>, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None], [<telethon.tl.functions.channels.GetParticipantsRequest object at 0x000001A940970F10>, <telethon.tl.functions.channels.GetParticipantsRequest object at 0x000001A940970FD0>, <telethon.tl.functions.channels.GetParticipantsRequest object at 0x000001A940971090>, <telethon.tl.functions.channels.GetParticipantsRequest object at 0x000001A940971150>, <telethon.tl.functions.channels.GetParticipantsRequest object at 0x000001A940971210>, <telethon.tl.functions.channels.GetParticipantsRequest object at 0x000001A9409712D0>, <telethon.tl.functions.channels.GetParticipantsRequest object at 0x000001A940971390>, <telethon.tl.functions.channels.GetParticipantsRequest object at 0x000001A940971450>, <telethon.tl.functions.channels.GetParticipantsRequest object at 0x000001A940971510>, <telethon.tl.functions.channels.GetParticipantsRequest object at 0x000001A9409715D0>, <telethon.tl.functions.channels.GetParticipantsRequest object at 0x000001A940971690>, <telethon.tl.functions.channels.GetParticipantsRequest object at 0x000001A940971750>, <telethon.tl.functions.channels.GetParticipantsRequest object at 0x000001A940971810>, <telethon.tl.functions.channels.GetParticipantsRequest object at 0x000001A9409718D0>, <telethon.tl.functions.channels.GetParticipantsRequest object at 0x000001A940971990>, <telethon.tl.functions.channels.GetParticipantsRequest object at 0x000001A940971A50>, <telethon.tl.functions.channels.GetParticipantsRequest object at 0x000001A940971B10>, <telethon.tl.functions.channels.GetParticipantsRequest object at 0x000001A940971BD0>, <telethon.tl.functions.channels.GetParticipantsRequest object at 0x000001A940971C90>, <telethon.tl.functions.channels.GetParticipantsRequest object at 0x000001A940971D50>, <telethon.tl.functions.channels.GetParticipantsRequest object at 0x000001A940971E10>, <telethon.tl.functions.channels.GetParticipantsRequest object at 0x000001A940971ED0>, <telethon.tl.functions.channels.GetParticipantsRequest object at 0x000001A940971F90>, <telethon.tl.functions.channels.GetParticipantsRequest object at 0x000001A940972050>, <telethon.tl.functions.channels.GetParticipantsRequest object at 0x000001A940972110>, <telethon.tl.functions.channels.GetParticipantsRequest object at 0x000001A9409721D0>])

Possible solution to the problem. Changed the aggressive parameter in line 79 to False. In this case, the pars limit will be 10k people.
all_participants = client.get_participants(target_group, aggressive=False)

В файле pars.py
79 строка
all_participants = client.get_participants(target_group, aggressive=True)
Замени на
all_participants = client.get_participants(target_group)
100% решение


HUE 4.6 [Filebrowser] Create/Touch file in Cyrillic directory

Hello I use Hue™ 4.5.0 and Hue™ 4.6.0
When I try create file in directory with Cyrillic patch (/hdfs/user/vkuryata/тест/file.txt)
I will find that file in other directory.
EGG.:

Create Cyrillic directory:

Create file "test.txt" in new directory with Cyrillic name

In step 2 hue make dir "/user /vkuryata / %C3%91%C2%82%C3%90%C2%B5%C3%91%C2%81%C3%91%C2%82/" (/ user/ vkuryata/ ÑÐµÑÑ) and put here file "test.txt"

I got ERROR:
File "/usr/share/local/hue/build/env/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/core/handlers/exception.py", line 41, in inner     response = get_response(request)   File "/usr/share/local/hue/build/env/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/core/handlers/base.py", line 249, in _legacy_get_response     response = self._get_response(request)   File "/usr/share/local/hue/build/env/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/core/handlers/base.py", line 187, in _get_response     response = self.process_exception_by_middleware(e, request)   File "/usr/share/local/hue/build/env/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/core/handlers/base.py", line 185, in _get_response     response = wrapped_callback(request, *callback_args, **callback_kwargs)   File "/usr/share/local/hue/build/env/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/utils/decorators.py", line 185, in inner     return func(*args, **kwargs)   File "/usr/share/local/hue/apps/filebrowser/src/filebrowser/views.py", line 213, in view     stats = request.fs.stats(path)   File "/usr/share/local/hue/desktop/core/src/desktop/lib/fs/proxyfs.py", line 148, in stats     return self._get_fs(path).stats(path)   File "/usr/share/local/hue/desktop/libs/hadoop/src/hadoop/fs/webhdfs.py", line 314, in stats     res = self._stats(path)   File "/usr/share/local/hue/desktop/libs/hadoop/src/hadoop/fs/webhdfs.py", line 303, in _stats     json = self._root.get(path, params, headers)   File "/usr/share/local/hue/desktop/core/src/desktop/lib/rest/resource.py", line 142, in get     return self.invoke("GET", relpath, params, headers=headers, allow_redirects=True, clear_cookies=clear_cookies)   File "/usr/share/local/hue/desktop/core/src/desktop/lib/rest/resource.py", line 87, in invoke     log_response=log_response)   File "/usr/share/local/hue/desktop/core/src/desktop/lib/rest/resource.py", line 124, in _invoke     log_length and len(resp.content) > log_length and "..." or "" if resp else "" UnicodeDecodeError: 'ascii' codec can't decode byte 0xd1 in position 79: ordinal not in range(128)

There are two error here as the logging is erroring and hiding the initial HDFS message. This was fixed in master.
There was some changes in master for better URL but it might still not work. 4.7 release should be out next week or we could try on master to confirm.

When I try Create new dir with Cyrillic name I got Error:

[06/Apr/2020 17:34:30 +0300] exception    ERROR    Internal Server Error: /filebrowser/mkdir
Traceback (most recent call last):
File "/usr/share/local/hue/build/env/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/core/handlers/exception.py", line 41, in inner
response = get_response(request)
File "/usr/share/local/hue/build/env/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/core/handlers/base.py", line 249, in _legacy_get_response
response = self._get_response(request)
File "/usr/share/local/hue/build/env/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/core/handlers/base.py", line 187, in _get_response
response = self.process_exception_by_middleware(e, request)
File "/usr/share/local/hue/build/env/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/core/handlers/base.py", line 185, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/usr/share/local/hue/build/env/lib/python2.7/site-packages/Django-1.11.22-py2.7.egg/django/utils/decorators.py", line 185, in inner
return func(*args, **kwargs)
File "/usr/share/local/hue/apps/filebrowser/src/filebrowser/views.py", line 1173, in mkdir
return generic_op(MkDirForm, request, smart_mkdir, ["path", "name"], "path")
File "/usr/share/local/hue/apps/filebrowser/src/filebrowser/views.py", line 1102, in generic_op
op(*args)
File "/usr/share/local/hue/apps/filebrowser/src/filebrowser/views.py", line 1171, in smart_mkdir
request.fs.mkdir(request.fs.join(urllib_unquote(path), urllib_unquote(name)))
File "/usr/share/local/hue/desktop/core/src/desktop/lib/fs/proxyfs.py", line 178, in mkdir
return self._get_fs(path).mkdir(path, *args, **kwargs)
File "/usr/share/local/hue/desktop/libs/hadoop/src/hadoop/fs/webhdfs.py", line 458, in mkdir
success = self._root.put(path, params, headers=headers)
File "/usr/share/local/hue/desktop/core/src/desktop/lib/rest/resource.py", line 189, in put
allow_redirects=allow_redirects, clear_cookies=clear_cookies)
File "/usr/share/local/hue/desktop/core/src/desktop/lib/rest/resource.py", line 87, in invoke
log_response=log_response)
File "/usr/share/local/hue/desktop/core/src/desktop/lib/rest/resource.py", line 124, in _invoke
log_length and len(resp.content) > log_length and "..." or "" if resp else ""
UnicodeDecodeError: 'ascii' codec can't decode byte 0xd1 in position 79: ordinal not in range(128)

if I modify: /usr/share/local/hue/apps/filebrowser/src/filebrowser/views.py and add sys.setdefaultencoding('utf8'),  I don't got any Error messages in error.log, but I can't create files in Cyrillic patch.
...
**vi /usr/share/local/hue/apps/filebrowser/src/filebrowser/views.py**
import sys
reload(sys)
sys.setdefaultencoding('utf8')
...

utf-8 is the default https://github.com/cloudera/hue/blob/master/desktop/conf/pseudo-distributed.ini.tmpl#L157
What is the trace of the new error?


Backport serialization fix for replicator

[X] You've read the Contributor Guide and Code of Conduct.
[ ] You've included unit or integration tests for your change, where applicable.
[ ] You've included inline docs for your change, where applicable.
[ ] There's an open issue for the PR that you are making. If you'd like to propose a change, please open an issue to discuss the change or find an existing issue.

Backport of a serialization fix from Lucene.
Description
This backports a serialization fix in the upstream Lucene project to avoid arbitrary type instantiation when errors occur.
Upstream commit: https://github.com/apache/lucene/commit/b4b153f64fb0420c6e28ddea4c442e119458756c

Needs one more change, will reopen


Single Server FAR

Moves find all references to single server

Going to merge, but if there's any follow up please feel free to comment and I'll do in another PR


[T4A5][T11-B2]Pankaj Bhootra

Ready for review

@PankajB1997 Some comments added. Please close the PR after reading comments.


SDK

The same as before. About placing licenses headers policy. Some of the file have the header, some do not.
for example
https://github.com/neutron-org/neutron-sdk/blob/chore/sdk-preparations/packages/neutron-sdk/src/interchain_queries/queries.rs
and next to it
https://github.com/neutron-org/neutron-sdk/blob/chore/sdk-preparations/packages/neutron-sdk/src/interchain_queries/register_queries.rs
is it ok?

Here, Apache states that:

Include a copy of the Apache License, typically in a file called LICENSE, in your work, and consider also including a NOTICE file that references the License.

To apply the Apache License to specific files in your work, attach the following boilerplate declaration, replacing the fields enclosed by brackets "[]" with your own identifying information. (Don't include the brackets!) Enclose the text in the appropriate comment syntax for the file format. We also recommend that you include a file or class name and description of purpose on the same "printed page" as the copyright notice for easier identification within third-party archives.

Since all files in this repository are licensed as Apache 2.0, there is no reason to include declaration headers in source files, LICENSE and NOTICE files are sufficient.


Improved documentation for viewing Fluo logs

May want to document the issues with logback vs log4j when running in Yarn, with some background on this issue.

We also need to document that it would be best if observers used slf4j when doing their own logging.  That could be a new issue/PR


Update GlobalTransactionAutoConfiguration.java

applicationName + "-fescar-service-group" 改为 applicationName + "-seata-service-group"

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

#801

Hi, could you please sign the CLA?

could you pls modify fescar to seata in seata-module like https://github.com/alibaba/spring-cloud-alibaba/blob/master/spring-cloud-alibaba-seata/src/main/java/com/alibaba/cloud/seata/feign/SeataFeignClient.java#L71.

@fangjian0423 #804


Indexing mapped type with union key returns bad function union

Bug Report
🔎 Search Terms
generic, mapped types, parameter, argument, union, never
🕗 Version & Regression Information
This is the behavior in every version I tried, and I reviewed the FAQ for entries about mapped types
(Note: versions before 3.9.7 in the playground report the same error, just with a different message)
⏯ Playground Link
Playground link with relevant code
💻 Code
type Token = { type: "A" } | { type: "B" }; //| { type: "C" } | { type: "D" } | { type: "E" } | { type: "F" } | { type: "G" };
type TokenHandler<T> = (token: Token & { type: T }) => any;
type TokenMap = {
    [T in Token["type"]]: TokenHandler<T>;
};

const tokenHandlers = { A(token) {} /* ... */ } as TokenMap;
const token = { type: "A" } as Token;

// Resolves to TokenHandler<"A"> | TokenHandler<"B">
// and becomes (tok: never) => void
// instead of the expected (tok: Token & { type: "A" | "B" }) => any
const handler = tokenHandlers[token.type];
handler(token); // error: Argument of type 'Token' is not assignable to parameter of type 'never'.
// And if we add more token types we get error: Expression produces a union type that is too complex to represent.

Making the function itself generic solves the error, but now we lose typing for the token parameter
Playground
type Token = { type: "A"; a: string } | { type: "B"; b: number };
type TokenHandler<T> = <T>(token: Token & { type: T }) => any;
type TokenMap = {
    [T in Token["type"]]: TokenHandler<T>;
};

const tokenHandlers = {
    // error: Property 'a' does not exist on type 'Token & { type: T; }'.
    A(token) { return token.a; }
    // ...
} as TokenMap;
const token = { type: "A" } as Token;

// Resolves to TokenHandler<"A"> | TokenHandler<"B">
// and becomes <"A" | "B">(tok: Token & { type: "A" | "B" }) => any
const handler = tokenHandlers[token.type];
handler(token); // ok

The only workaround I can think of is to cast every time it's indexed
// Resolves to TokenHandler<"A" | "B">
// and becomes (tok: Token & { type: "A" | "B" }) => any
const explicitCast = tokenHandlers[token.type] as TokenHandler<typeof token.type>;
explicitCast(token); // ok

// Avoids having to use cast everywhere, but adds runtime overhead
function getHandler<T extends Token["type"]>(tokenType: T) {
    return tokenHandlers[tokenType] as TokenHandler<T>;
}
const functionCast = getHandler(token.type);
functionCast(token);

🙁 Actual behavior
TokenMap['A' | 'B'] resolves to (tok: never) => any
🙂 Expected behavior
TokenMap['A' | 'B'] should resolve to (tok: Token & { type: "A" | "B" }) => any

Given that you have TokenHandler<"A"> | TokenHandler<"B">, this being a union implies the program doesn't know which one it is - so the only value you can safely call it with is Token & { type: "A" & "B" }--of which no value exists.  Hence never.  If it were Token & { type: "A" | "B" } then I could take tokenHandlers[token.type] and freely pass it the wrong type with no complaint from the compiler.

This looks like a duplicate of #30581, and the fix as described/implemented in #47109 involves refactoring to something like
type TokenTypes = "A" | "B" | "C"
type Token<K extends TokenTypes = TokenTypes> = { [P in K]: { type: P } }[K];
type TokenHandlers = {
    [K in TokenTypes]: (token: Token<K>) => any;
}

const tokenHandlers = { A(token) { } /* ... */ } as TokenHandlers;
const token = { type: "A" } as Token;

function handleToken<K extends TokenTypes>(token: Token<K>) {
    const handler = tokenHandlers[token.type];
    handler(token); // okay
}

handleToken(token); // okay

Playground link

Thanks @jcalz, I knew this was a job for correlated unions but couldn’t remember what those were actually called in order to search for the relevant issue. 😅

It took me a while to understand what's going on in this case, but it's the closest to the situation I had so I'll share here too:
const suites = {
  key1(_arg0: 11, _arg2: 12) {},
  key2(_arg0: 21) {},
  key3(_arg0: 31, _arg2: 32) {},
  key4(_arg0: 41, _arg2: 42) {},
  key5(_arg0: 51, _arg2: 52) {},
  key6(_arg0: 61, _arg2: 62) {},
  key7(_arg0: 71) {},
  key8(_arg0: 81, _arg2: 82) {},
};

type Tests = typeof suites;
type TestSuites = {
  [key in keyof Tests]: (
    setup: TestSetups[key],
    option: TestOptions[key]
  ) => void;
};
type TestSetups = {
  [key in keyof Tests]: Parameters<Tests[key]>[0];
};
type TestOptions = {
  [key in keyof Tests]: Parameters<Tests[key]>[1];
};

const setups: TestSetups = {
  key1: 11,
  key2: 21,
  key3: 31,
  key4: 41,
  key5: 51,
  key6: 61,
  key7: 71,
  key8: 81,
};

const options: TestOptions = {
  key1: 12,
  key2: undefined,
  key3: 32,
  key4: 42,
  key5: 52,
  key6: 62,
  key7: undefined,
  key8: 82,
};

(Object.keys(suites) as Array<keyof TestSuites>).forEach(
  <T extends keyof TestSuites>(key: T) => {
    const suite: TestSuites[T] = suites[key];
    suite(setups[key], options[key]);
  }
);

playground
If I understand what's going on, it's because you can't rely on the original object you're using for the map, but have to create an intermediary type that also uses the keys to determine the type, thus "linking" the relationship of the types.
Thanks a lot @jcalz !


Add finalizer attribute hook to validate a deserialized structure

In https://github.com/servo/rust-url/pull/252, @Manishearth is writing a serde deserializer for urls since parsing a string can be expensive. However, to do this safely from untrusted sources, they need to validate that the data is semantically correct. We unfortunately don't have an easy way of doing this without writing a custom deserializer. It'd be pretty simple to do as an attribute, however. Here's my idea:
#[derive(Serialize, Deserialize)]
#[serde(finalizer="check_greater_than_10")]
struct GreaterThan10 {
    n: usize,
}

fn greater_than_10<D>(item: GreaterThan10) -> Result<GreaterThan10, D::Error>
    where D: Deserializer
{
    if item.n > 10 {
        Ok(item)
    } else {
        Err(Error::invalid_value("less than or equal to 10"))
    }
}

This seems relevant: https://github.com/Keats/validator. Let's make sure to support this library in a nice way.

My hacky workaround is to

don't derive Deserialize for my main data type "DataType"
copy my main data type to a "FakeDataType" which has the same fields, and make this one derive Deserialize
implement Deserialize for DataType by:

invoking deserialize to FakeDataType
validate the result
convert FakeDataType to DataType.

It's a dumb hack but at least it allows me to avoid rewriting all the deser code manually.

#626 has another use case.
I am on board with implementing exactly the approach in @erickt's comment.

Another use-case from irc by @alteous

I would like to obtain a maximum integer value from a tree of structures. The context is these values are offsets into arrays, and I want to check that they are all valid ahead of time. The integer values are wrapped in Index types.

I believe I could also use this for an enum which is either "unknown" or some value fitting a specific pattern, but both are strings.

I have the same usecase as in #626. I want to load shapes, and I also want to store some precomputed attributes on them that are derived from the other fields (for example radius_squared = radius * radius).
I don't want to serialize or deserialize these fields, but I want them to be present in the finished structs. The shapes could be deeply nested in another structure that is loaded, so it's very messy to try to go down the structures in a mutable fashion and set it on the structs manually.

Perhaps a silly question, but why an attribute instead of a trait?  I pictured:
trait SerdeFinalize: Sized {
    fn finalize<E:Error>(self) -> Result<Self, E>;
}

#[derive(Serialize, Deserialize)]
#[serde(Finalize)]
struct GreaterThan10 {
    n: usize,
}

impl SerdeFinalize for GreaterThan10 {
    fn finalize<E:Error>(self) -> Result<Self, E> {
        if self.n > 10 {
            Ok(self)
        } else {
            Err(E::invalid_value(Unexpected::Unsigned(self.n as u64), &"a number above 10"))
        }
    }
}

(And maybe #[serde(Finalize)] would no longer be needed once specialization is stable and everything could get a default no-op finalizer that's overridable if desired.)

I've been working on something related to this for derive_builder. My assumption has been that the two-step process:

Gather data from some outside source into an intermediate struct
Finalize the domain type (perform semantic validations, generate cached values, RAII, etc.)

should be split across two different types, and that the sidecar type could be generated through a combination of macros, i.e. deriving a Builder which in turn derives Deserialize. In that world, field-level validations would be handled through types, rather than the attributes in the validator project.
I've run into a few hurdles so far:

I don't know of a clean way to invoke TryFrom<...> that doesn't require writing manual deserialization code. This makes fallible conversions to domain-specific types (like "Credit Card" or "Email") very tedious to write. I'm hoping that the stabilization of TryFrom will pave the way for a more reusable solution.
Surfacing errors of many different types to the outer caller requires boxing or other approaches that complicate error inspection.
Getting multiple errors during a single deserialization isn't possible.

Cross-referencing https://github.com/Keats/validator/issues/22.
It looks like one of the main concerns there is how to bubble up structured/detailed errors when validation/finalization fails. If anybody has usecases/opinions, it would be nice to feed them back there.

There seem to be two situations for this:

After deserialization, perform an infallible transformation `FnOnce(T) -> T
After deserialization, perform a fallible transformation FnOnce(T) -> Result<T, E: serde::de::Error>

From a naming perspective, these seem exactly like map and and_then. It probably shouldn't be valid to declare both on one type/field.
@lucab I've been going back and forth on the errors front. Having that function return a serde Error is really useful for composition with this feature; maybe users can handle that conversion in their own code?

Any progress on this lately?

It would be nice to have a hook like the one mentioned by @scottmcm. With validator, I could do something like
#[derive(Debug, Validate, Deserialize)]
#[serde(Finalize)]
struct SignupData {
    #[validate(email)]
    mail: String,
    #[validate(url)]
    site: String,
    #[validate(length(min = "1"), custom = "validate_unique_username")]
    #[serde(rename = "firstName")]
    first_name: String,
    #[validate(range(min = "18", max = "20"))]
    age: u32,
}

// This
impl SerdeFinalize for SignupData {
    fn finalize<ValidationErrors>(self, deserialization_errors: Option<HashMap<Cow<String>, serde::de::Error>>) -> Result<Self, ValidationErrors> {
      if let Some(de_errors) = deserialization_errors {
           // format errors and return without doing any validation?
      }
      // the current Validate::validate fn would be impl here instead
    }
}

// And use it later
let data: SignupData = serde_json::from_str(data)?;

This way it's a one step action for users. The part I'm not sure about is if a user wants to hook into the serde error messages to add i18n for example or to have only one kind of error. I believe https://docs.serde.rs/serde/de/trait.Error.html would be enough for that but those errors will need to be passed to whatever Finalize function is so the validation library can format them as seen in the snippet above (type completely made up and probably incorrect).

We are brainstorming some ways to generalize this in #1118.

@bstrie's workaround is pretty good. For custom structs, you can do make a shadow type, implement deserialize on it, and do validations within the TryFrom implementation.
use serde::Deserialize;
use serde_json;
// The target is to not allow deserialization if option1 & option2 are none
#[derive(Deserialize, Debug)]
#[serde(try_from = "MyTypeShadow")]
pub struct MyType {
    option1: Option<usize>,
    option2: Option<usize>,
}

// The shadoww type only has to implement Deserialize
#[derive(Deserialize)]
pub struct MyTypeShadow {
    option1: Option<usize>,
    option2: Option<usize>,
}

pub struct MyTypeValidationError;

// The error type has to implement Display
impl std::fmt::Display for MyTypeValidationError {
    fn fmt(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
        write!(formatter, "option1 and option2 cannot be null")
    }
}

impl std::convert::TryFrom<MyTypeShadow> for MyType {
    type Error = MyTypeValidationError;
    fn try_from(shadow: MyTypeShadow) -> Result<Self, Self::Error> {
        let MyTypeShadow { option1, option2 } = shadow;
        if option1.is_none() && option2.is_none() {
            return Err(MyTypeValidationError);
        }
        // Any other validations
        Ok(MyType { option1, option2 })
    }
}

fn main() {
    // This will return an Err
    println!("{:?}", serde_json::from_str::<MyType>(r##"{}"##));
    // This will work
    println!(
        "{:?}",
        serde_json::from_str::<MyType>(r##"{"option1": 20}"##)
    );
}

Would a PR implementing the approach in the OP (https://github.com/serde-rs/serde/issues/642#issue-196247375) be accepted (a finalizer= attribute on the struct level), or is this issue blocked on something else?

A finalizer fn for deserialization is also useful for when you were forced to do a #[serde(skip)] on a field due to it being from a 3rd party lib with no serialization support, so you have to reinitialize the data once the struct is deserialized

Another use case: Discord API libraries serenity and twilight. Some JSON payloads omit important data from nested objects because it can be inferred from top-level fields. But when deserializing into a Rust type, we want to include that important data in the nested object itself.
Serenity's current workaround is a manual Deserialize impl that deserializes into a serde_json::Value, patches that, and then continues to deserialize into the target type (1, 2, 3)
Twilight's current workaround is a full blown 710LOC manual Deserialize impl (1)

Just skimmed over this ticket, and IIUC, there's no built-in support for this in serde 1.0.x, correct?
The TryFrom pattern in https://github.com/serde-rs/serde/issues/642#issuecomment-683276351 looks like a great option for my use case (which is post-deserialization validation).
Suggestion: add that pattern to the examples on https://serde.rs . I just glanced at the examples ToC and didn't see an entry that was obviously about this pattern. I'm going to take a stab at adding that content to the website in a PR.


上传大文件时会出现CONNECT_TIMEOUT

上传文件时，我设置的connectTimeout = 10s
上传时间超过10秒就会CONNECT_TIMEOUT，但实际建立连接没有超时
定位问题发现是request.close();超时
看实现是不是把上传文件的时间也算到了connectTimeout里？
Future future = request.close();
if (options.connectTimeout > 0) {
  future = future.timeout(Duration(milliseconds: options.connectTimeout));
}
HttpClientResponse responseStream;
try {
  responseStream = await future;
} on TimeoutException {
  _throwConnectingTimeout();
}

Hi guys, sorry my Chinese is limited to Google Translate, but it seems I am also having the same issue. When I am uploading 1.5 MB file (to Google Storage in my case) I am getting CONNECT_TIMEOUT exception even when upload succeeds. I am also getting this pretty consistently.
The upload code is quite simple:
await _dio.put("/some/upload/url",
        data: file.readAsBytesSync(),
        options: Options(
          //10 minutes
          sendTimeout: 600000
        )
      );

I have also tried setting larger timeout to base Dio options:
//10 minutes
dio.options.connectTimeout = 600000;
dio.options.receiveTimeout = 600000;
dio.options.sendTimeout = 600000;

The upload itself actually succeds as I can see the uploaded files on the server and the file is healthy (not corrupted).
The exception stacktrace:
#0      DioMixin._dispatchRequest (package:dio/src/dio.dart:963:7)
<asynchronous suspension>
#1      DioMixin._request._interceptorWrapper.<anonymous closure>.<anonymous closure>.<anonymous closure> (package:dio/src/dio.dart:849:37)
#2      DioMixin.checkIfNeedEnqueue (package:dio/src/dio.dart:1117:22)
#3      DioMixin._request._interceptorWrapper.<anonymous closure>.<anonymous closure> (package:dio/src/dio.dart:846:22)
#4      new Future.<anonymous closure> (dart:async/future.dart:176:37)
#5      _rootRun (dart:async/zone.dart:1180:38)
#6      _CustomZone.run (dart:async/zone.dart:1077:19)
#7      _CustomZone.runGuarded (dart:async/zone.dart:979:7)
#8      _CustomZone.bindCallbackGuarded.<anonymous closure> (dart:async/zone.dart:1019:23)
#9      _rootRun (dart:async/zone.dart:1184:13)
#10     _CustomZone.run (dart:async/zone.dart:1077:19)
#11     _CustomZone.bindCallback.<anonymous closure> (dart:async/zone.dart:1003:23)
#12     Timer._createTimer.<anonymous closure> (dart:async-patch/timer_patch.dart:23:15)
#13     _Timer._runTimers (dart:isolate-patch/timer_impl.dart:398:19)
#14     _Timer._handleMessage (dart:isolate-patch/timer_impl.dart:429:5)
#15     _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:168:12)


fix : User token authentication issue (#124) and pressing enter on the signin page should sign the user #123

Issue #123
How I have resolved it ?
Whenever a user is typing password, if the user presses enter then I am calling handleSubmit function which is resolving the issue.
HandleSubmit was expecting a form event e, which is used to prevent the default behaviour of forms. I have  made that parameter as optional because here we are not using  so no need for e.preventDefault
Issue #124
I have corrected the url of NEXT_PUBLIC_BASE_URL_LOCAL in .env.example from 8000 to 3000

Can u fix the merge conflicts please

@hkirat I have fixed the conflicts

Thankyou! /bounty $10


Enable Messaging With Bot in Teams

I have added my Bot in the app catalogue of my company. When some users try to contact the chatbot, they see the message error that I attached in this issue.
> "Conversation with this bot is not supported"

But when one new user contacts the bot, this error doesn´t appear and they can interact with the chatbot without any problems,
Can someone tell me why this happens?

@AndresCgaehd  - thank you for submitting feedback.
I think the best way forward is if you open a service ticket in your tenant so this can get resolved ASAP. Based on the outcome let me know if it is something that can be called out in the docs.
Please follow this link to contact support for business products:  https://docs.microsoft.com/office365/admin/contact-support-for-business-products
Please keep us posted here on the resolution so we can feed whatever information you discover into the content.

@officedocsbot close

Having the same issue


Ability to copy selected text

Is your feature request related to a problem? Please describe.
I'm actually very surprised that this is not a thing, as it's something that sounds super essential to me: actually being able to copy the text you've selected.
Describe the solution you'd like
I would like to be able to actually properly copy the text I've selected, without it somehow including line numbers and random spaces and line breaks.
Describe alternatives you've considered
As mentioned in various other issues, related to copying text, a potential solution is to provide a separate way to copy the content, i.e. via a button or by overriding onCopy, but that doesn't really solve my problem. As an end user, I'd expect that I have the ability to selectively copy text, copying the full content can be extra annoying, especially if you're trying to copy a single file out of a massive file...

Are you using Safari?

No, I use Vivaldi, which is a Chromium-based browser. Your question actually inspired me to try in Firefox, and to my surprise, it works there. Not quite sure what's going on...

Update: tried in Google Chrome as well, same issue, so this appears to be a thing in (most) Chromium-based browsers.

On Edge and Chrome (both Chromium based) the copying seems to work fine for me. Sounds like you're on Linux?

On Edge and Chrome (both Chromium based) the copying seems to work fine for me. Sounds like you're on Linux?

Yeah, but a friend of mine who uses Chrome on Windows told me he has the same issue. For the record, I have wrap long lines and line numbers enabled.


what is the usage of unum?

Hi, I am wondering what is the uniform number and what is the use of it? I tried to look at the document, but didn't find a place to explain the purpose of this value.

Each player has a uniform number, ranging from 1-11. (1 will be, at least in some respects, treated as the goalie; see #34 for some discussion of this.) It is used in identifying them for two of the high-level actions, Pass (to a teammate) and Mark Player (vs a particular opponent).


How to reinitiate flip?

Hello,
I need to know if it is possible to reinitiate .flip namespace on a selector after I have turned it off with
$(this).off(".flip");
I have already tried
$( this ).flip({
          trigger: 'hover'
        });

but with no success.

Same question here.

Same question here

http://stackoverflow.com/questions/43595432/reinitiate-flip-js/43598450#43598450
See this thread - the answer worked for my issue.


Update [content].tsx

Imported HeadingsTemplate file and added it alphabetically in the correct place between the layout tags

Thanks @EmmaDawsonDev for merging my contribution in the main😎😎
It was a nice experience!!


Otomi chart install returns error when there is no kms blob in input values.yaml

Describe the bug:
When there is no kms in values.yaml for otomi chart, it returns an error:
Error: template: otomi/templates/sops-secrets.yaml:1:17: executing "otomi/templates/sops-secrets.yaml" at <.Values.kms.sops>: nil pointer evaluating interface {}.sops

To Reproduce

remove kms blob from input values
install the otomi chart

Expected behavior: (a clear and concise description of what you expected to happen)
it should install the chart and otomi without kms

will be fixed when #613 lands

ok


Missing Filename in shader compilation error

Sometimes when com,piling shaders with glslang the error messages do not mention the file name.
@GabeRundlett

I didn't realize this was an opened issue!
Fixed in d4c41e5051fac8ed197a4b0d0280c35ce2353fc3


BC2021w1. Return record from table procedure

Describe the bug
When i create a procedure on the table or tableextension and return the record this doesn't work
To Reproduce
this returns empty record
tableextension 50100 "PurchaseLineExt" extends "Purchase Line" {     procedure FindFirstRec(): Record "Purchase Line"     begin          Rec.FindFirst();     end; }
this returns first record
tableextension 50100 "PurchaseLineExt" extends "Purchase Line" {     procedure FindFirstRec() PurchLine: Record "Purchase Line"     begin          PurchLine.FindFirst();     end; }
Expected behavior
Should return Rec
Screenshots
5. Versions:

AL Language:
7.0.426055
Business Central:

Please clarify.
 tableextension 50100 "PurchaseLineExt" extends "Purchase Line" {      procedure FindFirstRec(): Record "Purchase Line"      begin          Rec.FindFirst();      end;  }
": Record "Purchase Line"" just says, that the return type of the function is a Record of Type Purchase Line.
This return type has no connection to the Rec.
To get the desired functionality add a "exit(Rec);" after the FindFirst.
 tableextension 50100 "PurchaseLineExt" extends "Purchase Line" {      procedure FindFirstRec(): Record "Purchase Line"      begin          Rec.FindFirst();         exit(Rec);     end;  }

Do you mean the first one? Why should it return Rec?? Rec means the global current record, not the result of the function.

To get the desired functionality add a "exit(Rec);" after the FindFirst.

Sure, but then doing that is pointless, since the Rec.FindFirst() already changed the state of the instance on which the method was called, so they can just use that, instead of also copying it to the result.

To get the desired functionality add a "exit(Rec);" after the FindFirst.

Sure, but then doing that is pointless, since the Rec.FindFirst() already changed the state of the instance on which the method was called, so they can just use that, instead of also copying it to the result.

In case of TableExtensions you are right.
In case of PageExtensions returning the Rec would be good as it would make the difference between:
myPage.FindFirstRec();
myPage.GetRecord(myRec);
and
MyRec := MyPage.FindFirstRec();
Also in the example provided by dkatson the return type of the function is set, so it should return something.

I am closing this issue as this is the expected behavior. The other comments on the issue provide more information on how to handle this scenario.


fix: storage IO panel not working

Description
Describe the big picture of your changes here to communicate to the maintainers why we should accept this pull request.
If it fixes a bug or resolves a feature request, be sure to link to that issue.
Fix storage IO panels in kube-prometheus Grafana dashboards as "decimals": -1 isn't the correct value that breaks this panel.
Type of change
What type of changes does your code introduce to the kube-prometheus? Put an x in the box that apply.

[ ] CHANGE (fix or feature that would cause existing functionality to not work as expected)
[ ] FEATURE (non-breaking change which adds functionality)
[X] BUGFIX (non-breaking change which fixes an issue)
[ ] ENHANCEMENT (non-breaking change which improves existing functionality)
[ ] NONE (if none of the other choices apply. Example, tooling, build system, CI, docs, etc.)

Changelog entry
Please put a one-line changelog entry below. Later this will be copied to the changelog file.

- Fix storage IO panels in kube-prometheus Grafana dashboards

@dragoangel yaml is generated from jsonnet so you would need to make any changes there

Hi @philipgough, sorry but I can't find file where this dashboards are described in jsonnet format


Store state trie in SQLite

This means that writes of blocks and writes to state can be kept in sync, because the write barriers that SQLite provides will apply to both forms of data. This should eliminate inconsistencies between the two data stores.
Note that I also renamed the SQL database from blockdata.db to db.sqlite3.

Does this mean that a node running with this commit will have to start from genesis/checkpoint and rebuild its state?


pipectl: 0.2.2 -> 0.3.0

Motivation for this change
Upgrdes
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[x] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
22.05 Release Notes (or backporting 21.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

Why the backport?

@risicle I don't see any breaking changes so I assume v0.3.0 is just for prestige, therefore a backport makes sense to me
@Ferdi265 is that so?

I usually use the patch version for bugfixes, and the minor version for new features with no intended breaking changes. Once I'm comfortable switching to 1.0.0, major will be breaking changes, minor will be compatible features, patch will be bugfixes.
Technically, 0.3.0 changes the default temp directory, which might possibly be interpreted as not 100% compatible, but that is just pedantic nitpicking :)

Oh! Could it be that the last wl-mirror update that added pipectl as an optional dependency did not get backported? If that is so then this failure would make sense (no pipectl to cherrypick this commit into)

Yep, nixos-21.11 still has wl-mirror at 0.5.0, so pipectl is not present either