Aijuh/199/log level selector

Finish implementing log level selector as @kangz12345 requested as a week goal.
Doubly checked comments and grammer issue.
I will fix code right away if I missed something.
This closes #199

Could you attach the screenshot of the GUI?

Fixed mistakes.


Data structure changes

Includes:

parcel parsing
monitoring mode, so no output is created
few more tests and error handling in areas that creaked under pressure

Don't review this one until #9 and #16 are merged

@wpears, can you elaborate on what you mean by monitoring mode, so no output is created?

When retriever is called with no bucket or directory, it will fetch files from their sources and pipe them through the checkHash module, but nowhere else. Once all the hashes are checked, it will report on the results, but won't actually manipulate the files or send them anywhere. So this would be a good mode to daemonize such that we can find out when data is updated without constantly sending bytes over the wire to S3

Got it.  Makes sense.  Thanks.


Orphaned resources handling

Description
Provides a structure where cluster resources can be added.
When calling the update function, all provided resources will be patched and the deployed resources which are not provided will be deleted.
This is work in progress! The implementation is not suitable for all operators and will be changed.

The implementation is already tested in:

stackabletech/hbase-operator#215
Integration tests:
https://github.com/stackabletech/airflow-operator/tree/orphaned_resources_handling
https://github.com/stackabletech/druid-operator/tree/orphaned_resources_handling
https://github.com/stackabletech/hdfs-operator/tree/orphaned_resources_handling
https://github.com/stackabletech/hive-operator/tree/orphaned_resources_handling
https://github.com/stackabletech/kafka-operator/tree/orphaned_resources_handling
https://github.com/stackabletech/nifi-operator/tree/orphaned_resources_handling
https://github.com/stackabletech/opa-operator/tree/orphaned_resources_handling
https://github.com/stackabletech/superset-operator/tree/orphaned_resources_handling
https://github.com/stackabletech/trino-operator/tree/orphaned_resources_handling
https://github.com/stackabletech/zookeeper-operator/tree/orphaned_resources_handling

@adwk67 will have a look at spark-k8s-operator if there is anything to do.
Review Checklist

[ ] Code contains useful comments
[ ] (Integration-)Test cases added (or not applicable)
[ ] Documentation added (or not applicable)
[ ] Changelog updated (or not applicable)
[ ] Cargo.toml only contains references to git tags (not specific commits or branches)

Once the review is done, comment bors r+ (or bors merge) to merge. Further information

It would be great if you could also add Support for Secrets, as the trino-operator creates a Secret used as a shared secret https://trino.io/docs/current/security/internal-communication.html#configure-shared-secret

Otherwise LGTM

It would be great if you could also add Support for Secrets, as the trino-operator creates a Secret used as a shared secret https://trino.io/docs/current/security/internal-communication.html#configure-shared-secret

If I understood the source code of the Trino operator correctly then there is one shared secret per Trino cluster. This secret has an owner reference to the cluster and is removed when the cluster is deleted. I do not see how this secret could become orphaned.

If I understood the source code of the Trino operator correctly then there is one shared secret per Trino cluster. This secret has an owner reference to the cluster and is removed when the cluster is deleted. I do not see how this secret could become orphaned.

You're absolutely right!

bors merge


[Bug] OpenAI Compatible Prompt Template Error

Checklist

[x] 1. I have searched related issues but cannot get the expected help.
[x] 2. The bug has not been fixed in the latest version.
[x] 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.
[x] 4. If the issue you raised is not a bug but a question, please raise a discussion at https://github.com/sgl-project/sglang/discussions/new/choose Otherwise, it will be closed.
[x] 5. Please use English, otherwise it will be closed.

Describe the bug
I noticed that the prompt template that is applied is incorrect because it does not parse the text correctly. More specifically, in the openai_api/adapter.py, we use Huggingface to tokenize:
 prompt_ids = tokenizer_manager.tokenizer.apply_chat_template(
                    request.messages, tokenize=True, add_generation_prompt=True
                )

But, the request messages is still in Pydantic form so then it gets incorrectly tokenized by the huggingface tokenizer.
For example, if the request.messages is:
[ChatCompletionMessageUserParam(role='user', content=[ChatCompletionMessageContentImagePart(type='image_url', image_url=ChatCompletionMessageContentImageURL(url='https://raw.githubusercontent.com/sgl-project/sglang/main/assets/logo.png', detail='auto')), ChatCompletionMessageContentTextPart(type='text', text='Describe this image')]), ChatCompletionMessageGenericParam(role='assistant', content='This image is really fun.'), ChatCompletionMessageUserParam(role='user', content='Tell me a story about the image')]

We get the following prompt and prompt ids:
[128000, 128006, 882, 128007, 271, 58, 16047, 34290, 2097, 2831, 1945, 5920, 5930, 1151, 1843, 2975, 518, 2217, 2975, 28, 16047, 34290, 2097, 2831, 1945, 3222, 6659, 1151, 2485, 1129, 1059, 52027, 916, 14, 2034, 75, 34796, 14, 2034, 5317, 15711, 22158, 29647, 3592, 518, 7872, 1151, 3989, 47359, 13149, 34290, 2097, 2831, 1199, 5920, 5930, 1151, 1342, 518, 1495, 1151, 75885, 420, 2217, 52128, 128009, 128006, 78191, 128007, 271, 2028, 2217, 374, 2216, 2523, 13, 128009, 128006, 882, 128007, 271, 41551, 757, 264, 3446, 922, 279, 2217, 128009, 128006, 78191, 128007, 271]

<|begin_of_text|><|start_header_id|>user<|end_header_id|>

[ChatCompletionMessageContentImagePart(type='image_url', image_url=ChatCompletionMessageContentImageURL(url='https://raw.githubusercontent.com/sgl-project/sglang/main/assets/logo.png', detail='auto')), ChatCompletionMessageContentTextPart(type='text', text='Describe this image')]<|eot_id|><|start_header_id|>assistant<|end_header_id|>

This image is really fun.<|eot_id|><|start_header_id|>user<|end_header_id|>

Tell me a story about the image<|eot_id|><|start_header_id|>assistant<|end_header_id|>

I have a suspicion that this is because huggingface does not know that we are using Pydantic datamodels and so it just looks at the content section of the messages and parses it which is just a string-ified version of the Pydantic data model.
Reproduction
python -m sglang.launch_server --model-path lmms-lab/llama3-llava-next-8b --port=30000
Client script:
import openai
import time

working_messages = [
    {
            "role": "user",
            "content": [
                {
                    "type": "image_url",
                    "image_url": {
                        "url": "https://raw.githubusercontent.com/sgl-project/sglang/main/assets/logo.png"
                    },
                },
                {"type": "text", "text": "Describe this image"},
            ],
            },
            {
             "role": "assistant",
             "content": "This image is really fun."
            },
            {
             "role": "user",
             "content": "Tell me a story about the image"
            }
]

client = openai.Client(api_key="EMPTY", base_url="http://<IP_ADDRESS>:30000/v1")
start = time.time()
response = client.chat.completions.create(
    model="llama3-llava-next-8b",
    messages=working_messages,
    temperature=0.3,
    max_tokens=512,
    stream=True,
)

for i, chunk in enumerate(response):
    if chunk.choices[0].delta.content is not None:
         print(chunk.choices[0].delta.content, end="")

Environment
Python: 3.9.19 (main, Mar 21 2024, 17:11:28) [GCC 11.2.0]
CUDA available: True
GPU 0: NVIDIA L4
GPU 0 Compute Capability: 8.9
CUDA_HOME: /usr/local/cuda
NVCC: Cuda compilation tools, release 12.1, V12.1.105
CUDA Driver Version: 535.183.01
PyTorch: 2.4.0+cu121
sglang: 0.2.14.post2
flashinfer: 0.1.6+cu121torch2.4
triton: 3.0.0
transformers: 4.44.2
requests: 2.32.3
tqdm: 4.65.0
numpy: 1.23.5
aiohttp: 3.9.5
fastapi: 0.109.2
hf_transfer: 0.1.8
huggingface_hub: 0.24.6
interegular: 0.3.3
packaging: 23.0
PIL: 9.2.0
psutil: 6.0.0
pydantic: 2.8.2
uvicorn: 0.22.0
uvloop: 0.19.0
zmq: 26.0.3
vllm: 0.5.5
multipart: 0.0.9
openai: 1.42.0
anthropic: 0.34.1
NVIDIA Topology:
        GPU0    CPU Affinity    NUMA Affinity   GPU NUMA ID
GPU0     X      0-15    0               N/A

Legend:

  X    = Self
  SYS  = Connection traversing PCIe as well as the SMP interconnect between NUMA nodes (e.g., QPI/UPI)
  NODE = Connection traversing PCIe as well as the interconnect between PCIe Host Bridges within a NUMA node
  PHB  = Connection traversing PCIe as well as a PCIe Host Bridge (typically the CPU)
  PXB  = Connection traversing multiple PCIe bridges (without traversing the PCIe Host Bridge)
  PIX  = Connection traversing at most a single PCIe bridge
  NV#  = Connection traversing a bonded set of # NVLinks

Hypervisor vendor: KVM
ulimit soft: 1048576

If you use lmms-lab/llama3-llava-next-8b, you should not use the chat template in huggingface tokenizer. This is because the chat template in huggingface tokenizer does not support image.
The correct way is to use the custom chat template for llava-next. You can specify it when you launch the server.
https://github.com/sgl-project/sglang/blob/55f5976b42d736f3dfe2f8f9b91a6536c212744a/README.md?plain=1#L246-L247


Method Not Allowed when i use post

First Check

[X] I added a very descriptive title to this issue.
[X] I used the GitHub search to find a similar issue and didn't find it.
[X] I searched the FastAPI documentation, with the integrated search.
[X] I already searched in Google "How to X in FastAPI" and didn't find any information.
[X] I already read and followed all the tutorial in the docs and didn't find an answer.
[X] I already checked if it is not related to FastAPI but to Pydantic.
[X] I already checked if it is not related to FastAPI but to Swagger UI.
[X] I already checked if it is not related to FastAPI but to ReDoc.

Commit to Help

[X] I commit to help with one of those options 👆

Example Code
from typing import List
from fastapi import FastAPI
from uuid import UUID,uuid4
from models import User, Gender, Role

app = FastAPI()

db: List[User] = [
    User(
        id=uuid4(),
        first_name="John",
        last_name="Doe",
        gender=Gender.male,
        roles=[Role.user]
         ),
    User(
        id=UUID("3f4f106b-1758-4e1f-a5ac-205ad4a177bf"),
        first_name="Natalia",
        last_name="Landofdawn",
        gender=Gender.female,
        roles=[Role.user,Role.admin]
         )
]

@app.get('/')
async def root():
    return {"hello":"world"}

@app.get('/api/v1/users')
async def fetch_users():
    return db

@app.post("/api/v1/users'")
async def register_user(user: User):
    db.append(user)
    return {"id":user.id}

Description
hello sir i have some problem when i try use method post result {
"detail": "Method Not Allowed"
}
here is screenshoot

this is my code
main.py
`from typing import List
from fastapi import FastAPI
from uuid import UUID,uuid4
from models import User, Gender, Role
app = FastAPI()
db: List[User] = [
User(
id=uuid4(),
first_name="John",
last_name="Doe",
gender=Gender.male,
roles=[Role.user]
),
User(
id=UUID("3f4f106b-1758-4e1f-a5ac-205ad4a177bf"),
first_name="Natalia",
last_name="Landofdawn",
gender=Gender.female,
roles=[Role.user,Role.admin]
)
]
@app.get('/')
async def root():
return {"hello":"world"}
@app.get('/api/v1/users')
async def fetch_users():
return db
@app.post("/api/v1/users'")
async def register_user(user: User):
db.append(user)
return {"id":user.id}`
models.py
`from typing import List
from fastapi import FastAPI
from uuid import UUID,uuid4
from models import User, Gender, Role
app = FastAPI()
db: List[User] = [
User(
id=uuid4(),
first_name="John",
last_name="Doe",
gender=Gender.male,
roles=[Role.user]
),
User(
id=UUID("3f4f106b-1758-4e1f-a5ac-205ad4a177bf"),
first_name="Natalia",
last_name="Landofdawn",
gender=Gender.female,
roles=[Role.user,Role.admin]
)
]
@app.get('/')
async def root():
return {"hello":"world"}
@app.get('/api/v1/users')
async def fetch_users():
return db
@app.post("/api/v1/users'")
async def register_user(user: User):
db.append(user)
return {"id":user.id}`
Operating System
Windows
Operating System Details
Windows 10
FastAPI Version
fastapi 0.89.0
Python Version
Python 3.11.0
Additional Context
No response

I think it's because your route here: @app.post("/api/v1/users'") has an extra apostrophe. Edit it to @app.post("/api/v1/users") and should work.

I think it's because your route here: @app.post("/api/v1/users'") has an extra apostrophe. Edit it to @app.post("/api/v1/users") and should work.

Thank you sir it's working now


Hotfix #68 - Fixed exception message on missing dependency for RouteCollector

Fixes #68
Use RouteCollector instead of non-existsing PathBasedRoutingMiddleware in factory in case of missing dependencies.

[x] Are you fixing a bug?

[x] Detail how the bug is invoked currently.
[x] Detail the original, incorrect behavior.
[x] Detail the new, expected behavior.
[x] Base your feature on the master branch, and submit against that branch.
[x] Add a regression test that demonstrates the bug, and proves the fix.
[x] Add a CHANGELOG.md entry for the fix.

Thanks, @xtreamwayz


fix for CatchCN CLM4.5 fail on SLES15 GNU

During SLES15 testing we got CatchCN clm 4.5 model run fail due to floating-point exception.
Issue happens during specific calculation where exponent is too large.
Error is in this calculation:
https://github.com/GEOS-ESM/GEOSgcm_GridComp/blob/e2b79bdd24df30ab49889596f8fd97f4ab7271fb/GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSsurface_GridComp/GEOSland_GridComp/GEOScatchCN_GridComp/GEOScatchCNCLM45_GridComp/CLM45/CNFireMod.F90#L695-L697
Issue: https://github.com/GEOS-ESM/GEOSgcm_GridComp/issues/896
@gmao-jkolassa this fix works and results in round off difference for CatchCN. Maybe there is better way to do it I am not sure.
For GCM this is zero diff trivial since fix is in CLM4.5 module.
fyi: @gmao-rreichle @rdkoster

@gmao-jkolassa :  We'd like to get feedback the following:

Is there a specific reason why fire_m_fac(4,6) was set to the ridiculously large value (10.e15), which ends up in the exponent in the fire_m_tmp calculation?
Would you be ok with the new value (10.e5)?
A better solution is probably to set fire_m_fac(4,6) to no-data, then avoid the calculation of exp(-fire_m_fac*...) altogether and set fire_m_tmp to whatever you think is appropriate in this case [or prescribe the result of exp(-fire_m_fac*...) for the PFTs in question]

@biljanaorescanin Thank you for implementing this fix. I am happy with this solution for now.
@gmao-rreichle , to answer your questions:

Is there a specific reason why fire_m_fac(4,6) was set to the ridiculously large value (10.e15), which ends up in the exponent in the fire_m_tmp calculation?

The fire_m_fac parameter controls the sensitivity of combustibility to the soil moisture conditions. The large value of 10e15 was chosen to address excessive amounts of fires simulated for PFTs 4 and 6. At the time we tested a few different values for fire_m_fac and found that only by limiting the occurrence of fire in PFTs 4 and 6 to very dry conditions were we able to get a somewhat reasonable match to observations.

Would you be ok with the new value (10.e5)?

Because of the above issue and many other problems with the CLM4.5 fire code that we found, we decided to no longer conduct fire simulations with this code and instead moved forward the implementation of Catchment-CN5.1, with its (arguably) improved fire code. That is to say, no one is currently using Catchment-CN4.5 for fire simulations and so I think setting fire_m_fac to 10.e5 to avoid the issues that were encountered during testing is fine.

A better solution is probably to set fire_m_fac(4,6) to no-data, then avoid the calculation of exp(-fire_m_fac*...) altogether and set fire_m_tmp to whatever you think is appropriate in this case [or prescribe the result of exp(-fire_m_fac*...) for the PFTs in question]

I agree that there probably is a better way to address this issue. At the time we were discussing to also vary the parameters controlling the sensitivity of the combustibility to the atmospheric humidity and atmospheric temperature instead of just altering the soil moisture sensitivity alone (which led to having to set it to this very large value). But because of all the other issues mentioned above, we never did test this and instead decided to move on to a newer model. For the same reason and given that Catchment-CN5.1 is around the corner, I am not sure whether it makes sense to spend a lot of time on coming up with a better fix. But, I will let @rdkoster chime in on that point.

For my part, I don't see the value in putting time and effort into CatchmentCN-4.5.  We had high hopes in the beginning, but the value of that version never materialized - even the NCAR people threw up their hands in regard to it and moved on.  I'm not sure who would ever want to use it.  -- Randy
From: Jana Kolassa @.>
Sent: Wednesday, February 7, 2024 2:06 PM
To: GEOS-ESM/GEOSgcm_GridComp @.>
Cc: Koster, Randal D. (GSFC-6101) @.>; Mention @.>
Subject: [EXTERNAL] [BULK] Re: [GEOS-ESM/GEOSgcm_GridComp] fix for CatchCN CLM4.5 fail on SLES15 GNU (PR #898)
CAUTION: This email originated from outside of NASA.  Please take care when clicking links or opening attachments.  Use the "Report Message" button to report suspicious messages to the NASA SOC.
@biljanaorescaninhttps://github.com/biljanaorescanin Thank you for implementing this fix. I am happy with this solution for now.
@gmao-rreichlehttps://github.com/gmao-rreichle , to answer your questions:
Is there a specific reason why fire_m_fac(4,6) was set to the ridiculously large value (10.e15), which ends up in the exponent in the fire_m_tmp calculation?
The fire_m_fac parameter controls the sensitivity of combustibility to the soil moisture conditions. The large value of 10e15 was chosen to address excessive amounts of fires simulated for PFTs 4 and 6. At the time we tested a few different values for fire_m_fac and found that only by limiting the occurrence of fire in PFTs 4 and 6 to very dry conditions were we able to get a somewhat reasonable match to observations.
Would you be ok with the new value (10.e5)?
Because of the above issue and many other problems with the CLM4.5 fire code that we found, we decided to no longer conduct fire simulations with this code and instead moved forward the implementation of Catchment-CN5.1, with its (arguably) improved fire code. That is to say, no one is currently using Catchment-CN4.5 for fire simulations and so I think setting fire_m_fac to 10.e5 to avoid the issues that were encountered during testing is fine.
A better solution is probably to set fire_m_fac(4,6) to no-data, then avoid the calculation of exp(-fire_m_fac*...) altogether and set fire_m_tmp to whatever you think is appropriate in this case [or prescribe the result of exp(-fire_m_fac*...) for the PFTs in question]
I agree that there probably is a better way to address this issue. At the time we were discussing to also vary the parameters controlling the sensitivity of the combustibility to the atmospheric humidity and atmospheric temperature instead of just altering the soil moisture sensitivity alone (which led to having to set it to this very large value). But because of all the other issues mentioned above, we never did test this and instead decided to move on to a newer model. For the same reason and given that Catchment-CN5.1 is around the corner, I am not sure whether it makes sense to spend a lot of time on coming up with a better fix. But, I will let @rdkosterhttps://github.com/rdkoster chime in on that point.

Reply to this email directly, view it on GitHubhttps://github.com/GEOS-ESM/GEOSgcm_GridComp/pull/898#issuecomment-1932691696, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AMXBXCPJVRDL4ERW6UCQDJ3YSPGCNAVCNFSM6AAAAABC6F5DJKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMZSGY4TCNRZGY.
You are receiving this because you were mentioned.Message ID: @.***>

Per @gmao-jkolassa's report of older results from the time when the original 10e15 numbers were implemented, this change would impact the science output of CNCLM45 and thus cannot be merged without proper science evaluation.  Since NCAR advises that CLM45 should not be used anymore, the plan is to disable CNCLM45 in GEOS going forward by disallowing LSM_CHOICE=3 in the setup of the GCM and the LDAS.  This will be done in separate PRs.


Clean build of v1.5.12 fails due to v8

What's wrong
A clean build on M2 Mac using Docker fails due to v8.
How to reproduce
git clone --recurse-submodules git@github.com:timeplus-io/proton.git

cd proton

./docker/packager/packager --package-type binary --docker-image-version clang-16 --proton-build --enable-proton-local --output-dir `pwd`/build_output --compiler clang-16-darwin

Error message
Build fails with
ninja: error: 'contrib/v8-cmake/bytecode_builtins_list_generator', needed by 'contrib/v8-cmake/generated/builtins-generated/bytecodes-builtins-list.h', missing and no known rule to make it

I probably know something.. sorry about this.
the --compiler clang-16-darwin will jump to cross compile,
we hard code some build binary for the v8.
would you like to use directly bare metal build instead of docker?
https://github.com/timeplus-io/proton/blob/develop/BUILD.md#bare-metal-build-5
the proton server final is a static library. the fresh build basically need  about 90 minutes.
also you can download the binary or pull the docker directly to take a try proton.

Thanks, I am aware that the binary exists and I am already using it. I am trying to build proton to be able to work on the source code. The build also fails under wsl and on a ubuntu machine. I will try again, document the process and then make a bug ticket.

really appreciate your trial. I check the code again,  the "docker build for macOS" issue may not have a quick fix.
Would you mind trying the bare-metal build instead? the macOS use the dockered low level is based on arm/x86 linux.
Regarding the WSL / Ubuntu build failure, this seems to be an issue.
Some of our colleagues already use WSL for their daily development and debugging.

Bare-metal build worked for me after upgrading xcode to version 15.4.

glad to hear that. There's a current issue with macOS where the stacktrace only works in debug mode, but there's no stacktrace when you throw an exception directly (ClickHouse already has a patch to fix this, but I don't have the time to back port it).
Apart from this problem, macOS also doesn't have jemalloc && sanitizer. thoes are some potential issues we currently encountered.


RFC: Block decoration ordering

An idea for an API to explicitly order block decorations that occur at the same screen row in a TextEditor. This came out of our work in https://github.com/atom/github/pull/1913.

View rendered docs/rfcs/004-decoration-ordering.md

FYI: My current plan is to merge this at COB today (Eastern time) unless there are any objections.


How to log in an account from the back-end

Hi!
Great work on Play-Pac4J, we're happily outsourcing part of our authentication using Pac4J :smile: !
Quick question: we want to be able to log in a user using an email link (e.g. confirm email link after registration or a 'magic link'). What's the best way to do so using Pac4J?

Currently, play-pac4j is based on pac4j v1.7. I'm going to upgrade it with pac4j v1.8 in the next few days. It's going to be easier with pac4j v1.8.
The registration part is not done by pac4j: a user fills a form and he is created into your database (for example) and a token is generated. Then, a link with this token is sent to the user email.
On a specific url, you need to define a ParameterClient which will get the provided token and pass it to your SpecificTokenAuthenticator which will fetch the user profile from the database based on the token. Saves this user profile in the current credentials and use a AuthenticatorProfileCreator: the user will be authenticated this way.
With pac4j v1.7, it would be harder: you would need to create the ParameterClient on your own, a specific Credentials also to pass the user profile to the ProfileCreator and the AuthenticatorProfileCreator.

Right! I hadn't seen the ParameterClient before! Seems suitable for our use case. Our token authenticator is ready to go :smile: , so I guess we'll just wait for 1.8! Great timing!
Thanks for your clear answer.


[opt](inverted index) unified optimization judgment to prevent omissions

https://github.com/apache/doris/pull/38027

run buildall


enable retry create pod from file

What this PR does / why we need it: Enable retry "kubectl apply -f" commands
Which issue this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): fixes #
Special notes for your reviewer:
If applicable:

[ ] documentation
[ ] unit tests
[ ] tested backward compatibility (ie. deploy with previous version, upgrade with this branch)

Release note:

enable retry create pod from file

not needed


Added a tutorial to the readme

This commit will add more examples to install and configure the theme and templates. Additionally, I added a quickstart example

I think you need to merge it to your code base so it gets integrated, since there are no github actions set up, such that the merge will get automatically done if you approve it.

Do I just click "Merge pull request"?

Yes. There should not be any merge conflicts occring, since I only added the readme and pulled the latest changes.


GraphQL Get: Query all properties if none are given

Currently get queries require to list all properties that should be returned and if no properties are given, the query returns an error.
Instead of returning an error, the client should get all non-reference properties from the schema if no properties are set and include those by default in the query.

let's wait for GRPC with that


Issuing scammer warning to moderators

In the event that a scammer posts an ETH or BTC address, it could be an alternative idea to message moderators with the scammers account name, rather than post a loud public message in the channel.

I would rather not spam the moderators too much, but perhaps simply send a message to the person writing the address saying their message was deleted because it contained an address? The idea behind the public message was to make sure other users were aware that it was fake or to remind them to not trust addresses

Sure, I understand. Personally i would like my channel to be calm and not announce scammers too loudly, but make admins/mods aware. If there are many attacks then the channel could get noisy. Perhaps we can work on making some of these settings configurable.

That's totally fair. Perhaps we could do ;

Send a private message to user saying they shouldn't post this
Warn admins/mods if the user tries to share >= 5 times (if the user is spamming).

Discrete, doesn't spam the mods/admins all the time when people post stuff by mistake, but will still warn them if someone is spamming addresses.

yes sounds good, although maybe a configurable threshold? for some, twice is spamming for others 5 is too much.

Yep, sounds good

Progress made ;
Created an #scambot-internal channel where the bot posts stuff, like deleted messages, new users, etc. Will add posting ETH addresses soon, so user will be warned privately and team admins (and some mods) will be able to see the messages containing the ETH that was deleted.

Implemented ;
ETH addresses don't trigger a warning, instead the message is posted in #scambot-internal and a warning is sent via DM to the user at fault. #scambot-internal needs to be a private channel where the bot is present.


feat(logger): clean old logs every hour

We can probably close this in favor of #1690, right?  We could clean at an interval in that one as well, if you think it's necessary, but I think it's fine without it.


Change interval of scraping the metrics

what is the default  scraping of the vcenter?
is there a way to change it?
i am having big vcenter and one scrape takes ~ 1 minutes and service get to crash very often

Please read the README as it states where to change this.


UI Recheck are not working since triggers is being silly while parsing the event from GH

Event JSON

{
  "action": "rerequested",
  "check_run": {
    "id":<PHONE_NUMBER>,
    "node_id": "MDg6Q2hlY2tSdW4yNTI4ODUyODc2",
    "head_sha": "d5cb4a13749b577f27bdc43166154bb6c2aa48ae",
    "external_id": "",
    "url": "https://api.github.com/repos/chmouel/scratchmyback/check-runs/2528852876",
    "html_url": "https://github.com/chmouel/scratchmyback/runs/2528852876",
    "details_url": "https://console-openshift-console.apps.paac.devcluster.openshift.com/k8s/ns/scratch-my-back/tekton.dev~v1beta1~PipelineRun/scratch-my-back-pr-9trkl/logs",
    "status": "completed",
    "conclusion": "failure",
    "started_at": "2021-05-07T15:47:38Z",
    "completed_at": "2021-05-07T15:47:48Z",
    "output": {
      "title": "😞 CI Run: Failed",
      "summary": "Tekon CI has failed.",
      "text": "\n| Status | Duration | Name |\n| --- | --- | --- |\n✅ Succeeded | 8 seconds | task-from-remote\n✅ Succeeded | 8 seconds | task-from-tektondir\n✅ Succeeded | 7 seconds | task-referenced-internally\n❌ Failed | 8 seconds | task-spec\n\n
\n\n\n 🗒 More detailed status\n
Name:              scratch-my-back-pr-9trkl\nNamespace:         scratch-my-back\nService Account:   pipeline\nTimeout:           1h0m0s\nLabels:\n tekton.dev/pipeline=scratch-my-back-pr-9trkl\n tekton.dev/pipeline-ascode-branch=main\n tekton.dev/pipeline-ascode-owner=chmouel\n tekton.dev/pipeline-ascode-repository=scratchmyback\n tekton.dev/pipeline-ascode-sender=chmouel\n tekton.dev/pipeline-ascode-sha=d5cb4a13749b577f27bdc43166154bb6c2aa48ae\n\nStatus\n\nSTARTED         DURATION    STATUS\n9 seconds ago   8 seconds   Failed\n\nMessage\n\nTasks Completed: 4 (Failed: 1, Cancelled 0), Skipped: 0 (\"step-task-spec\" exited with code 1 (image: \"registry.access.redhat.com/ubi8/ubi-minimal@sha256:5cfbaf45ca96806917830c183e9f37df2e913b187aadb32e89fd83fa455ebaa6\"); for logs run: kubectl -n scratch-my-back logs scratch-my-back-pr-9trkl-task-spec-kjg9d-pod-cw5pq -c step-task-spec\n)\n\nResources\n\n No resources\n\nParams\n\n NAME       VALUE\n repo_url   https://github.com/chmouel/scratchmyback\n revision   d5cb4a13749b577f27bdc43166154bb6c2aa48ae\n\nResults\n\n No results\n\nWorkspaces\n\n NAME     SUB PATH   WORKSPACE BINDING\n source   ---        VolumeClaimTemplate\n\nTaskruns\n\n NAME                                                        TASK NAME                    STARTED         DURATION    STATUS\n scratch-my-back-pr-9trkl-task-from-tektondir-n86f4          task-from-tektondir          9 seconds ago   8 seconds   Succeeded\n scratch-my-back-pr-9trkl-task-referenced-internally-bm7p8   task-referenced-internally   9 seconds ago   7 seconds   Succeeded\n scratch-my-back-pr-9trkl-task-spec-kjg9d                    task-spec                    9 seconds ago   8 seconds   Failed\n scratch-my-back-pr-9trkl-task-from-remote-kh9kr             task-from-remote             9 seconds ago   8 seconds   Succeeded\n
\n\n\n",
      "annotations_count": 0,
      "annotations_url": "https://api.github.com/repos/chmouel/scratchmyback/check-runs/2528852876/annotations"
    },
    "name": "Tekton Pipeline as Code CI",
    "check_suite": {
      "id":<PHONE_NUMBER>,
      "node_id": "MDEwOkNoZWNrU3VpdGUyNjc4MTg3OTc1",
      "head_branch": "tektonci",
      "head_sha": "d5cb4a13749b577f27bdc43166154bb6c2aa48ae",
      "status": "queued",
      "conclusion": null,
      "url": "https://api.github.com/repos/chmouel/scratchmyback/check-suites/2678187975",
      "before": "b865bbe983f9a0cedd2512122b3756af2edaa3e8",
      "after": "d5cb4a13749b577f27bdc43166154bb6c2aa48ae",
      "pull_requests": [
        {
          "url": "https://api.github.com/repos/chmouel/scratchmyback/pulls/1",
          "id": 617467616,
          "number": 1,
          "head": {
            "ref": "tektonci",
            "sha": "d5cb4a13749b577f27bdc43166154bb6c2aa48ae",
            "repo": {
              "id": 359117785,
              "url": "https://api.github.com/repos/chmouel/scratchmyback",
              "name": "scratchmyback"
            }
          },
          "base": {
            "ref": "main",
            "sha": "4b8bbc927fae0c4dda82de8d69346f268e68e6e5",
            "repo": {
              "id": 359117785,
              "url": "https://api.github.com/repos/chmouel/scratchmyback",
              "name": "scratchmyback"
            }
          }
        }
      ],
      "app": {
        "id": 112592,
        "slug": "pipelines-as-code",
        "node_id": "MDM6QXBwMTEyNTky",
        "owner": {
          "login": "openshift-pipelines",
          "id": 57996262,
          "node_id": "MDEyOk9yZ2FuaXphdGlvbjU3OTk2MjYy",
          "avatar_url": "https://avatars.githubusercontent.com/u/57996262?v=4",
          "gravatar_id": "",
          "url": "https://api.github.com/users/openshift-pipelines",
          "html_url": "https://github.com/openshift-pipelines",
          "followers_url": "https://api.github.com/users/openshift-pipelines/followers",
          "following_url": "https://api.github.com/users/openshift-pipelines/following{/other_user}",
          "gists_url": "https://api.github.com/users/openshift-pipelines/gists{/gist_id}",
          "starred_url": "https://api.github.com/users/openshift-pipelines/starred{/owner}{/repo}",
          "subscriptions_url": "https://api.github.com/users/openshift-pipelines/subscriptions",
          "organizations_url": "https://api.github.com/users/openshift-pipelines/orgs",
          "repos_url": "https://api.github.com/users/openshift-pipelines/repos",
          "events_url": "https://api.github.com/users/openshift-pipelines/events{/privacy}",
          "received_events_url": "https://api.github.com/users/openshift-pipelines/received_events",
          "type": "Organization",
          "site_admin": false
        },
        "name": "Pipelines as Code",
        "description": "Pipelines as code application to validate your repositories with OpenShift Pipelines",
        "external_url": "https://github.com/openshift-pipelines/pipelines-as-code/",
        "html_url": "https://github.com/apps/pipelines-as-code",
        "created_at": "2021-04-27T12:15:44Z",
        "updated_at": "2021-04-27T12:25:57Z",
        "permissions": {
          "administration": "read",
          "checks": "write",
          "contents": "write",
          "issues": "write",
          "members": "read",
          "metadata": "read",
          "organization_plan": "read",
          "pull_requests": "write"
        },
        "events": [
          "commit_comment",
          "issue_comment",
          "pull_request",
          "pull_request_review",
          "pull_request_review_comment",
          "push"
        ]
      },
      "created_at": "2021-05-07T15:47:24Z",
      "updated_at": "2021-05-07T15:53:17Z"
    },
    "app": {
      "id": 112592,
      "slug": "pipelines-as-code",
      "node_id": "MDM6QXBwMTEyNTky",
      "owner": {
        "login": "openshift-pipelines",
        "id": 57996262,
        "node_id": "MDEyOk9yZ2FuaXphdGlvbjU3OTk2MjYy",
        "avatar_url": "https://avatars.githubusercontent.com/u/57996262?v=4",
        "gravatar_id": "",
        "url": "https://api.github.com/users/openshift-pipelines",
        "html_url": "https://github.com/openshift-pipelines",
        "followers_url": "https://api.github.com/users/openshift-pipelines/followers",
        "following_url": "https://api.github.com/users/openshift-pipelines/following{/other_user}",
        "gists_url": "https://api.github.com/users/openshift-pipelines/gists{/gist_id}",
        "starred_url": "https://api.github.com/users/openshift-pipelines/starred{/owner}{/repo}",
        "subscriptions_url": "https://api.github.com/users/openshift-pipelines/subscriptions",
        "organizations_url": "https://api.github.com/users/openshift-pipelines/orgs",
        "repos_url": "https://api.github.com/users/openshift-pipelines/repos",
        "events_url": "https://api.github.com/users/openshift-pipelines/events{/privacy}",
        "received_events_url": "https://api.github.com/users/openshift-pipelines/received_events",
        "type": "Organization",
        "site_admin": false
      },
      "name": "Pipelines as Code",
      "description": "Pipelines as code application to validate your repositories with OpenShift Pipelines",
      "external_url": "https://github.com/openshift-pipelines/pipelines-as-code/",
      "html_url": "https://github.com/apps/pipelines-as-code",
      "created_at": "2021-04-27T12:15:44Z",
      "updated_at": "2021-04-27T12:25:57Z",
      "permissions": {
        "administration": "read",
        "checks": "write",
        "contents": "write",
        "issues": "write",
        "members": "read",
        "metadata": "read",
        "organization_plan": "read",
        "pull_requests": "write"
      },
      "events": [
        "commit_comment",
        "issue_comment",
        "pull_request",
        "pull_request_review",
        "pull_request_review_comment",
        "push"
      ]
    },
    "pull_requests": [
      {
        "url": "https://api.github.com/repos/chmouel/scratchmyback/pulls/1",
        "id": 617467616,
        "number": 1,
        "head": {
          "ref": "tektonci",
          "sha": "d5cb4a13749b577f27bdc43166154bb6c2aa48ae",
          "repo": {
            "id": 359117785,
            "url": "https://api.github.com/repos/chmouel/scratchmyback",
            "name": "scratchmyback"
          }
        },
        "base": {
          "ref": "main",
          "sha": "4b8bbc927fae0c4dda82de8d69346f268e68e6e5",
          "repo": {
            "id": 359117785,
            "url": "https://api.github.com/repos/chmouel/scratchmyback",
            "name": "scratchmyback"
          }
        }
      }
    ]
  },
  "repository": {
    "id": 359117785,
    "node_id": "MDEwOlJlcG9zaXRvcnkzNTkxMTc3ODU=",
    "name": "scratchmyback",
    "full_name": "chmouel/scratchmyback",
    "private": false,
    "owner": {
      "login": "chmouel",
      "id": 98980,
      "node_id": "MDQ6VXNlcjk4OTgw",
      "avatar_url": "https://avatars.githubusercontent.com/u/98980?v=4",
      "gravatar_id": "",
      "url": "https://api.github.com/users/chmouel",
      "html_url": "https://github.com/chmouel",
      "followers_url": "https://api.github.com/users/chmouel/followers",
      "following_url": "https://api.github.com/users/chmouel/following{/other_user}",
      "gists_url": "https://api.github.com/users/chmouel/gists{/gist_id}",
      "starred_url": "https://api.github.com/users/chmouel/starred{/owner}{/repo}",
      "subscriptions_url": "https://api.github.com/users/chmouel/subscriptions",
      "organizations_url": "https://api.github.com/users/chmouel/orgs",
      "repos_url": "https://api.github.com/users/chmouel/repos",
      "events_url": "https://api.github.com/users/chmouel/events{/privacy}",
      "received_events_url": "https://api.github.com/users/chmouel/received_events",
      "type": "User",
      "site_admin": false
    },
    "html_url": "https://github.com/chmouel/scratchmyback",
    "description": "Back scratching",
    "fork": false,
    "url": "https://api.github.com/repos/chmouel/scratchmyback",
    "forks_url": "https://api.github.com/repos/chmouel/scratchmyback/forks",
    "keys_url": "https://api.github.com/repos/chmouel/scratchmyback/keys{/key_id}",
    "collaborators_url": "https://api.github.com/repos/chmouel/scratchmyback/collaborators{/collaborator}",
    "teams_url": "https://api.github.com/repos/chmouel/scratchmyback/teams",
    "hooks_url": "https://api.github.com/repos/chmouel/scratchmyback/hooks",
    "issue_events_url": "https://api.github.com/repos/chmouel/scratchmyback/issues/events{/number}",
    "events_url": "https://api.github.com/repos/chmouel/scratchmyback/events",
    "assignees_url": "https://api.github.com/repos/chmouel/scratchmyback/assignees{/user}",
    "branches_url": "https://api.github.com/repos/chmouel/scratchmyback/branches{/branch}",
    "tags_url": "https://api.github.com/repos/chmouel/scratchmyback/tags",
    "blobs_url": "https://api.github.com/repos/chmouel/scratchmyback/git/blobs{/sha}",
    "git_tags_url": "https://api.github.com/repos/chmouel/scratchmyback/git/tags{/sha}",
    "git_refs_url": "https://api.github.com/repos/chmouel/scratchmyback/git/refs{/sha}",
    "trees_url": "https://api.github.com/repos/chmouel/scratchmyback/git/trees{/sha}",
    "statuses_url": "https://api.github.com/repos/chmouel/scratchmyback/statuses/{sha}",
    "languages_url": "https://api.github.com/repos/chmouel/scratchmyback/languages",
    "stargazers_url": "https://api.github.com/repos/chmouel/scratchmyback/stargazers",
    "contributors_url": "https://api.github.com/repos/chmouel/scratchmyback/contributors",
    "subscribers_url": "https://api.github.com/repos/chmouel/scratchmyback/subscribers",
    "subscription_url": "https://api.github.com/repos/chmouel/scratchmyback/subscription",
    "commits_url": "https://api.github.com/repos/chmouel/scratchmyback/commits{/sha}",
    "git_commits_url": "https://api.github.com/repos/chmouel/scratchmyback/git/commits{/sha}",
    "comments_url": "https://api.github.com/repos/chmouel/scratchmyback/comments{/number}",
    "issue_comment_url": "https://api.github.com/repos/chmouel/scratchmyback/issues/comments{/number}",
    "contents_url": "https://api.github.com/repos/chmouel/scratchmyback/contents/{+path}",
    "compare_url": "https://api.github.com/repos/chmouel/scratchmyback/compare/{base}...{head}",
    "merges_url": "https://api.github.com/repos/chmouel/scratchmyback/merges",
    "archive_url": "https://api.github.com/repos/chmouel/scratchmyback/{archive_format}{/ref}",
    "downloads_url": "https://api.github.com/repos/chmouel/scratchmyback/downloads",
    "issues_url": "https://api.github.com/repos/chmouel/scratchmyback/issues{/number}",
    "pulls_url": "https://api.github.com/repos/chmouel/scratchmyback/pulls{/number}",
    "milestones_url": "https://api.github.com/repos/chmouel/scratchmyback/milestones{/number}",
    "notifications_url": "https://api.github.com/repos/chmouel/scratchmyback/notifications{?since,all,participating}",
    "labels_url": "https://api.github.com/repos/chmouel/scratchmyback/labels{/name}",
    "releases_url": "https://api.github.com/repos/chmouel/scratchmyback/releases{/id}",
    "deployments_url": "https://api.github.com/repos/chmouel/scratchmyback/deployments",
    "created_at": "2021-04-18T10:57:45Z",
    "updated_at": "2021-04-29T10:19:59Z",
    "pushed_at": "2021-05-07T15:47:24Z",
    "git_url": "git://github.com/chmouel/scratchmyback.git",
    "ssh_url": "git@github.com:chmouel/scratchmyback.git",
    "clone_url": "https://github.com/chmouel/scratchmyback.git",
    "svn_url": "https://github.com/chmouel/scratchmyback",
    "homepage": null,
    "size": 21,
    "stargazers_count": 0,
    "watchers_count": 0,
    "language": null,
    "has_issues": true,
    "has_projects": true,
    "has_downloads": true,
    "has_wiki": true,
    "has_pages": false,
    "forks_count": 0,
    "mirror_url": null,
    "archived": false,
    "disabled": false,
    "open_issues_count": 1,
    "license": {
      "key": "gpl-3.0",
      "name": "GNU General Public License v3.0",
      "spdx_id": "GPL-3.0",
      "url": "https://api.github.com/licenses/gpl-3.0",
      "node_id": "MDc6TGljZW5zZTk="
    },
    "forks": 0,
    "open_issues": 1,
    "watchers": 0,
    "default_branch": "main"
  },
  "sender": {
    "login": "chmouel",
    "id": 98980,
    "node_id": "MDQ6VXNlcjk4OTgw",
    "avatar_url": "https://avatars.githubusercontent.com/u/98980?v=4",
    "gravatar_id": "",
    "url": "https://api.github.com/users/chmouel",
    "html_url": "https://github.com/chmouel",
    "followers_url": "https://api.github.com/users/chmouel/followers",
    "following_url": "https://api.github.com/users/chmouel/following{/other_user}",
    "gists_url": "https://api.github.com/users/chmouel/gists{/gist_id}",
    "starred_url": "https://api.github.com/users/chmouel/starred{/owner}{/repo}",
    "subscriptions_url": "https://api.github.com/users/chmouel/subscriptions",
    "organizations_url": "https://api.github.com/users/chmouel/orgs",
    "repos_url": "https://api.github.com/users/chmouel/repos",
    "events_url": "https://api.github.com/users/chmouel/events{/privacy}",
    "received_events_url": "https://api.github.com/users/chmouel/received_events",
    "type": "User",
    "site_admin": false
  },
  "installation": {
    "id": 1234,
    "node_id": "MDIzOkludGVncmF0aW9uSW5zdGFsbGF0aW9uMTY1NTYzODI="
  }
}

Fixed in https://github.com/openshift-pipelines/pipelines-as-code/blob/main/config/401-trigger-issue-recheck.yaml


Where is the source code?

Did I miss something obvious? I'm trying to debug an Exception coming from the Aml.Engine NuGet package DLL and I would like to have a glance at the source to debug.

Source code is available only for AutomationML members. Contat<EMAIL_ADDRESS>


Fixed argument passing when run with sudo

The ARGS array does not exist in this script, so probably just a copy/paste issue from somewhere else.

Thanks!


Getting a generic run fail that appears to be due to the PR being too big

Describe the bug
Our Gitstream actions fail after running for about 2 minutes on this specific PR. Judging by the 413 HTTP fail code, I'm guessing the PR is too large (completely fair, it is). Ideally, Gitstream would notify us of the fail reason somewhere so we don't have to check the logs.

Expected behavior
When a PR is so big that it causes a service failure, a fail reason is given in the check notes.
Additional context
More of a "nice to have" than a true bug, feel free to handle it accordingly.

Thanks for reporting this @kyle3s. We'll add this to our queue of potential issues to debug. Would you mind sharing info about how large the PR is so we can more accurately test this?

Hi @kyle3s
Our team has recently implemented updates to our infrastructure to better support large files in pull requests.
Could you please confirm that this problem no longer persists on your end?
Thanks!

Hi Pavel,
In my case I haven't actually seen this error in quite a while because I
modified my gitstream.cm file to ignore the HTML snapshots that were
causing the issue and that fixed it.
Thanks,
Kyle
On Sun, Dec 17, 2023, 5:40 AM Pavel Vaks @.***> wrote:

Hi @kyle3s https://github.com/kyle3s
Our team has recently implemented updates to our infrastructure to better
support large files in pull requests.
Could you please confirm that this problem no longer persists on your end?
Thanks!
—
Reply to this email directly, view it on GitHub
https://github.com/linear-b/gitstream/issues/227#issuecomment-1859133738,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AVUEG7BFFGUVZ4CV7GFDOBDYJ3DZTAVCNFSM6AAAAAAZ433ZP6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNJZGEZTGNZTHA
.
You are receiving this because you were mentioned.Message ID:
@.***>

Oh, I see. Nice solution.
Anyway, as we have now added support for large files, I'm closing this issue. Please feel free to reopen it if this problem reoccurs.
Best,
Pavel.


Possible bug in split

It seems like either there's a missing note at: https://doc.rust-lang.org/regex/regex/struct.Regex.html#method.split or a bug in split. If you use a non-capturing group in the pattern, the character that the non-capturing group matches isn't included in the results.
fn split_with_escapes() {
        let re = Regex::new(r"(?:[^\\])\.").unwrap();
        let fields:Vec<String> = re.split(r"a.b\.c").map(|s|s.replace(r"\.",".")).collect();

        assert_eq!(vec!["a","b.c"], fields);
    }

assertion failed: '(left == right)' (left: '["a", "b.c"]', right: '["", "b.c"]')

I don't see any bug here. Note this part of the documentation:

Namely, each element of the iterator corresponds to text that isn't matched by the regular expression.

In particular, your regex (?:[^\\])\. matches a. in a.b\.c.

If you use a non-capturing group

It doesn't seem like a non-capturing group is relevant here. Removing the capturing group from your regex doesn't change its observable behavior.

Sorry I was conflating look-around with non-capturing. You're right.


Don't inject PHX_SERVER by default

Hi, just poking around the code and I thought it was strange that by default the children have PHX_SERVER injected.I have a use case where I need the children to be running the phoenix server.
My understanding is the caller can decide if they want to start the Phoenix endpoint in their supervision tree (via FLAME.Parent.get()), so I don't understand why this environment variable must be injected and not be overridable. It gives the caller more flexibility to let them figure out if the endpoint should even be started and how they want to configure the server.

Honestly, I have copied this code from the FLAME.FlyBackend without giving it much thought. Maybe @chrismccord could enlighten us on why this ENV is set to  false on the FlyBackend?

It probably makes sense to disable it by default but I agree it should be overridable. Especially because the endpoint may start other "services". @nickdichev-firework, what in particular do you need from the endpoint?

Thanks @josevalim for jumping in. Maybe we can just remove the ENV var from the Enum.reject() (as you did in this PR) but leave it set to false by default (undo that change)? This would make it overridable.

@mruoss Great, I will make those changes today.
@josevalim I have a service which implements a FLAME-like runtime without requiring the parents/children to be clustered. The children serve a phoenix API for clients running in our kubernetes cluster to interact with a membrane pipeline. We explicitly chose to have the clients talk HTTP/websocket directly to the children.
We are evaluating FLAME to replace our in house solution, its interesting to think about proxying these calls through the parent in the context of the FLAME cluster :)

Seems like I added this early on, but the release of the container will still exec the server overlay, which sets PHX_SERVER=true, so I don't think this default does even what it was aiming to do, and I can confirm the correct way to prevent the endpoint from starting would be to handle it in your sup tree, so ship it!


add evaluationviews for vft and bnt

all evaluation views should also have a "generate PDF" buttons for quick export of the results


Object

[Enter feedback here]
This article fails to define what is or is not an object. This reference should be explicit to aid the customer and technical resources to understand what makes up the count explicitly. Secondary searches on object yield docs using unaligned language for resource or objectID based resources. To an outside reader this is confusing and provides no focused understand of what an object is and how it adds to the object count issue above.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 201e070f-b494-437a-2a44-945ea17928fd
Version Independent ID: 6625800d-f778-2c57-d5b0-31e27c8703ba
Content: Error (This company has exceeded the number of objects that can be synchronized) displays in a directory synchronization report - Active Directory
Content Source: support/azure/active-directory/exceed-number-objects-synced.md
Service: active-directory
GitHub Login: @genlin
Microsoft Alias: genli

@timmedina， Thanks for the feedbacks. We will investigate and improve the article.

I see this was closed but I am not seeing a relevant statement of what an object is. This reads like a user or group are the few targets that are categorized as an object in Azure. This is not the case on a base technical level. Can this be reopened and modified to bring the language inline to the actual definition of Object to Azure?


feat(connector): MSK connection by AWS IAM

I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.
What's changed and what's your intention?
Use AWS IAM to build the connection for MSK.
Doc will be added later.

Checklist

[ ] I have written necessary rustdoc comments
[ ] I have added necessary unit tests and integration tests
[ ] I have added test labels as necessary. See details.
[ ] I have added fuzzing tests or opened an issue to track them. (Optional, recommended for new SQL features #7934).
[ ] My PR contains breaking changes. (If it deprecates some features, please create a tracking issue to remove them in the future).
[ ] All checks passed in ./risedev check (or alias, ./risedev c)
[ ] My PR changes performance-critical code. (Please run macro/micro-benchmarks and show the results.)

[ ] My PR contains critical fixes that are necessary to be merged into the latest release. (Please check out the details)

Documentation

[ ] My PR needs documentation updates. (Please use the Release note section below to summarize the impact on users)

Release note
If this PR includes changes that directly affect users or other significant modifications relevant to the community, kindly draft a release note to provide a concise summary of these changes. Please prioritize highlighting the impact these changes will have on users.

Didn't dive into details.
Seems the only user-facing change AWS_MSK_IAM  is a new value option for the property sasl_mechanism.

Seems the only user-facing change AWS_MSK_IAM  is a new value option for the config sasl_mechanism

Also users can provide ak/sk and assume role arn. Just like kinesis and pulsar source.


Decouple connection pool from HTTP client

Purpose

Is to decouple the connection pool from HTTP client.

Approach

By default HTTP clients will share a global pool but they also has a way to configure their own pool

Automation tests

Unit tests Added

Security checks

Followed secure coding standards in http://wso2.com/technical-reports/wso2-secure-engineering-guidelines? yes
Ran FindSecurityBugs plugin and verified report? yes
Confirmed that this PR doesn't commit any keys, passwords, tokens, usernames, or other secrets? yes

Codecov Report

Merging #13355 into master will increase coverage by <.01%.
The diff coverage is 80.51%.

@@             Coverage Diff              @@
##             master   #13355      +/-   ##
============================================
+ Coverage     63.98%   63.99%   +<.01%
  Complexity      659      659
============================================
  Files          2147     2148       +1
  Lines         94210    94301      +91
  Branches      12406    12414       +8
============================================
+ Hits          60281    60345      +64
- Misses        29035    29060      +25
- Partials       4894     4896       +2

Impacted Files
Coverage Δ
Complexity Δ

...nalang/net/http/clientendpoint/InitGlobalPool.java
100% <100%> (ø)
0 <0> (?)

...main/java/org/ballerinalang/net/http/HttpUtil.java
72.18% <72.54%> (-1.17%)
0 <0> (ø)

...et/http/clientendpoint/CreateSimpleHttpClient.java
88.63% <94.44%> (+2.92%)
0 <0> (ø)
:arrow_down:

...a/org/ballerinalang/bre/bvm/WorkerDataChannel.java
80.86% <0%> (-7.41%)
0% <0%> (ø)

...org/ballerinalang/bre/bvm/WaitCallbackHandler.java
82.88% <0%> (-0.91%)
0% <0%> (ø)

...et/http/actions/httpclient/AbstractHTTPAction.java
78.29% <0%> (-0.86%)
0% <0%> (ø)

...mantics/analyzer/StreamsQuerySemanticAnalyzer.java
87.47% <0%> (+0.13%)
0% <0%> (ø)
:arrow_down:

...inalang/compiler/desugar/StreamingCodeDesugar.java
96.73% <0%> (+0.35%)
0% <0%> (ø)
:arrow_down:

... and 5 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3c0bc10...3a92f31. Read the comment docs.


Update pybids to latest release

Weird, we are already requiring pybids >= 0.3.0, which is the latest.
Should 0.3.0 include not traversing hidden folders already?

(#810)


Fix having clause compilation

This PR fixes bad compilation of having clause when there is more than one having component. Example:
Query:
new Query("Table1").Having("Column1", ">", 1).Having("Column2", "=", 1);

Compiled query:
SELECT * FROM [Table1] HAVING [Column1] > 1, AND HAVING [Column2] = 1

With tix fix it would compile to:
SELECT * FROM [Table1] HAVING [Column1] > 1 AND [Column2] = 1

Thanks merged to master

Thanks for merging. Could you please update the nuget package too?

Yes a new version will be released soon, for the moment, you can use the nightly build from myget
https://sqlkata.com/docs#nightly-builds


mcap_record connection header information like mcap convert captures

It turns out mcap convert captures per-publisher connection information, including latching state and the md5sum (so doing a manual conversion isn't necessary, but may as well hold on to that):
mcap list channels mcap_convert_from_bag_example.mcap
id  schemaId  topic                                     messageEncoding  metadata                                                                           `
...
18  1         /tf_static                                ros1             {"callerid":"/some_static_transform_publisher","latching":"1","md5sum":"94810edda583a504dfda3829e70d7eec","topic":"/tf_static"}
...
25  1         /tf_static                                ros1             {"callerid":"/robot_state_publisher","latching":"1","md5sum":"94810edda583a504dfda3829e70d7eec","topic":"/tf_static"}

The latching entry may be missing entirely (therefore it's not latched), and the callerid, but looks like md5sum and topic (redundantly) is there in the channel metadata.
This means there does need to be a per-publisher channel in the mcaps, can mcap_record easily support that?
mcap_play will have to create multiple publishers as well, instead of aggregating everything for each topic.

Done in https://github.com/lucasw/mcap_tools/commit/508611bb1aa1d795a8e139ca0a35d2a39bd85e1a - at least as much as possible, need to revisit once there are per-publisher Channels.


Add getBatchCount for SQL DML clauses

Fixes #1700

Looks good

Agree, it's a backport after all.


Develop

Coverage increased (+0.08%) to 83.073% when pulling b42a8d9becd9d54ae6890c2c4670beb5b75ddef8 on develop into 28def36b7554d7ec87328e7949ff7242e11aad46 on master.


Programmatically Tags broken

Hi,
I'm using v-model but it seems it doesn't work when the list of tags is updated from the outside.
Example :

myTags: {{myTags}}

    <tags-input v-model="myTags"
                placeholder="Add tag here"></tags-input>
    <button @click="populateMyTags">Add Tags</button>
    <button @click="resetMyTags">Reset</button>

import Vue from 'vue/dist/vue.esm.js';
import TagsInput from '@voerro/vue-tagsinput';
Vue.component('tags-input', TagsInput);

var xxx = new Vue({
    el: '#sandbox',
    data: {
        myTags: []
    },
    methods: {
        populateMyTags: function() {
            this.myTags = ["foo", "bar"];
        },
        resetMyTags: function() {
            this.myTags = [];
        }
    }
});

As you should see, Add Tags and  Reset buttons update only the HTML Text but it doesn't update the list of tags displayed in <tags-input>.

Just tried it out, works fine for me.

I had the same issue and this is what I did to solve it..
app.temp_array.push(tag);
app.selectedTags=app.temp_array;

Seems like pushing directly does not change the tags for some reason..


'model_reset.pth' in second stage

when I try to run iMTFA's second training stage using configs/coco-experiments/mask_rcnn_R_50_FPN_ft_fullclsag_cos_bh_base.yaml, how can i get the model_reset.pth

I have found the same problem! Do you solve it? Thx


[Feature] Show non-online time on users

(copied from the same issue on Rocket.Chat)
(This is something I'm about to do a PR on and have started the coding already)
Given that I am a user
I would like on Direct Messages and the Member list to see how long a user has been in a non-online status e.g.
(away icon) User A 15 minutes
(offline icon) User B 2 days
(online icon) User C
This is a critical feature for some corporate users for adoption of Rocket.Chat
Note this also will require a PR to Konnecty/meteor-user-presence

I have a PR #47  in for this feature. Could you please review and add it in if you agree with the new functionality?

Withdrawing PR and request as another user suggested a way to do what I wanted without a change to the base package.


Document or automate the update process for WASM binaries

When writing #494, I once again found myself unable to update the WASM grammar for OCaml, despite the instructions in CONTRIBUTING.md. I struggled with Docker configurations, manual ad hoc copy/pastes, and achieved nothing in the end.
I think we need to either automate the process, or make it fool-proof.

I struggled with Docker configurations

I think the process launches a container, rather than you having to run everything inside a container.
That said, I agree: it took a while to work out, the first time I did it. Having it part of the CI build would be ideal, if it can be triggered only when the dependency changes.

Resolved with #610


Feature Request: Move image when trained and update name

Hello,
first of all, thank you for your amazing work!
I would like to propose this function: When in the config the setting save.unknown is true (default) and I train an unknown image in the matches section it would be great, if the image is automatically moved/copied to .storage/train/{matchname}.
It would also be nice if a "modified" (trained/untrained) image would be reevaluated again, so that the unknown images are not unknown again or if known images are corrected with the right name.
I'm using the latest beta as of now.
What do you think about this?

Hey @mr-manuel, thank you for checking out my project and for the feature suggestions. I think it makes sense and is a great idea so I took a stab at implementing the reprocessing functionality.
Any images you train from the matches page will be copied and used on the training page. This was already built in and should be working. I believe that is what you were describing for the first function, right?
Here's a little gif demonstrating how I'm handling the reprocessing of the images. Currently you have to do it on an image by image basis. There is a button in the bottom right of every match image. Clicking this will reprocess that image and update the results.
In this gif you can see the first image was for an unknown user. I train it for myself, then reprocess the image to show it has the updated results.

Hi @jakowenko, thank you for your fast answer!
First function: Yes that was exactly that what I was describing. But somehow this don't work. The images don't get copied after training in the matches section to the train folder. In the logs I can't find any error. Probably something with permissions, even if Double Take managed to create the folders? For testing purpose i also changed the folder permissions to 777 and file permissions to 666, but no change.
I also copied manually the files to the train folders. Double Take recognized them only after a restart of the container (could be normal since I did not use the upload function), but afterwards they disappeared under matches, even if the files are still in the matches folder.
# ls -l matches/
total 1232
-rw-r--r-- 1 root root 97861 Sep  2 17:07 007ba212-f92d-4fe4-ba27-b1b3e4bcf230.jpg
-rw-r--r-- 1 root root 75459 Sep  2 19:00 0dc61d88-f05e-4b04-8486-eedd6c3e1d95.jpg
-rw-r--r-- 1 root root 61028 Sep  3 08:58 17a0f83b-9799-429a-9e81-a19bfe2feef1.jpg
-rw-r--r-- 1 root root 62722 Sep  3 09:04 296e64c6-2446-4883-8696-43c72785f01e.jpg
-rw-r--r-- 1 root root 61813 Sep  2 17:04 35130771-dd4a-4646-84a2-326adb21bb57.jpg
-rw-r--r-- 1 root root 92624 Sep  2 17:08 372d6139-792c-4b1b-b565-cdeb734ffb36.jpg
-rw-r--r-- 1 root root 60431 Sep  2 17:04 4dd1b597-8114-4e1e-bd06-507f802bf433.jpg
-rw-r--r-- 1 root root 66416 Sep  2 16:09 5223581e-fff2-4b63-a29a-612308236a8f.jpg
-rw-r--r-- 1 root root 60520 Sep  3 07:25 56c34887-2db4-45c3-aae7-4af0d8e4d0bd.jpg
-rw-r--r-- 1 root root 53066 Sep  2 15:08 570a8b8c-52a1-46b4-ac61-e39ada3b9ab5.jpg
-rw-r--r-- 1 root root 74920 Sep  2 18:10 63c61ccb-85ac-4309-abfd-fd243b67702f.jpg
-rw-r--r-- 1 root root 50653 Sep  2 18:02 772e9dde-918c-42f3-ad70-5a0cc72dd518.jpg
-rw-r--r-- 1 root root 84548 Sep  2 18:21 7b8cb32b-4b90-41b3-9194-08bb7e20ad53.jpg
-rw-r--r-- 1 root root 35699 Sep  2 14:50 8898c0e2-efc1-442c-8bf5-39832b845312.jpg
-rw-r--r-- 1 root root 53623 Sep  3 08:58 b70f134b-f095-4a52-9147-db16e50ff0db.jpg
-rw-r--r-- 1 root root 33573 Sep  2 20:32 da6b9e60-dba2-475e-9c7a-94fb244894eb.jpg
-rw-r--r-- 1 root root 70110 Sep  2 18:10 e0f7a097-ccba-4a4f-b05f-23a5a6d762d0.jpg
-rw-r--r-- 1 root root 55067 Sep  3 08:55 e5320594-dbf6-41ba-a288-d7d60a70d7c9.jpg
-rw-r--r-- 1 root root 73094 Sep  2 17:09 fef65f4b-46d3-4369-ade9-9633ed4876c5.jpg

# ls -l train/ashley/
total 248
-rw-rw-rw- 1 root root 97861 Sep  2 17:07 007ba212-f92d-4fe4-ba27-b1b3e4bcf230.jpg
-rw-rw-rw- 1 root root 92624 Sep  2 17:08 372d6139-792c-4b1b-b565-cdeb734ffb36.jpg
-rw-rw-rw- 1 root root 60520 Sep  3 07:25 56c34887-2db4-45c3-aae7-4af0d8e4d0bd.jpg

# ls -l train/manuel/
total 200
-rw-rw-rw- 1 root root 53066 Sep  2 15:08 570a8b8c-52a1-46b4-ac61-e39ada3b9ab5.jpg
-rw-rw-rw- 1 root root 74920 Sep  2 18:10 63c61ccb-85ac-4309-abfd-fd243b67702f.jpg
-rw-rw-rw- 1 root root 70110 Sep  2 18:10 e0f7a097-ccba-4a4f-b05f-23a5a6d762d0.jpg

Second function: Thank you very much

The file name changes when it's copied, just to make sure it doesn't overwrite an existing file in the trianing folder. The file sizes for manuel and ashely seem to match up to file sizes in the matches folder. Are you sure when you go to the Train page in the UI that you don't see the one that was just trained?

In my case the files aren't copied. Therefore I copied the files manually for testing. After I train an image from the Matches tab I can't find them under the Train tab.

I go to the Train tab -> select ashley on the top left -> select the pictures with ashley on it -> click on train on the top left -> confirm with yes -> after a few seconds the "2 files trained for ashley" message appears on the bottom left. Now I check the train folder for ashley but it's still empty.

I manually copy the files to the Train folders in filesystem. After restarting the docker container the images arent't displayed anymore in the Maches tab, even if the files are still in the folder.

Hm, that's strange. Here's a GIF of how it works on my computer. You can see the train folder is empty and also shows no results on the UI. When I train an image it copies it over then when I go to the train page on the UI the trained image is there as well.
What version of Double Take are you running?

I see. Very strange. Can I enable advanced logging somehow?
I‘m using the latest beta released 6 hours ago.

I don't have any advanced logging in there at the moment. There is a messages.log file that you could send me on discord if you'd like. That should include all the logs from the system. I can see if anything stands out to me in there.
Very strange though, I haven't heard of this happening to anybody else.

I should be able to get some verbose logs in early next week. If you aren't able to figure out what's going on over the weekend, hopefully some more logs will help us debug.

I think now I got the problem. I'm publishing Double Take with a reverse proxy (Traefik). When I train over HTTP (http://doubletake.local) all works fine. But when I'm training over HTTPS with a internal certificate (https://doubletake.local) it does not work and I get this error in the logs:
21-09-04 10:22:20 error: save url error: unable to verify the first certificate
Error: unable to verify the first certificate
    at TLSSocket.onConnectSecure (_tls_wrap.js:1514:34)
    at TLSSocket.emit (events.js:400:28)
    at TLSSocket._finishInit (_tls_wrap.js:936:8)
    at TLSWrap.ssl.onhandshakedone (_tls_wrap.js:708:12)

Maybe we need an option to ignore untrusted certificates?

If I put the containers in an isolated network without access to the outside (so no external DNS available, access only through reverse proxy) I get this error:
21-09-04 17:38:44 error: save url error: getaddrinfo EAI_AGAIN doubletake.local
Error: getaddrinfo EAI_AGAIN doubletake.local
    at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:69:26)
21-09-04 17:38:49 error: save url error: getaddrinfo EAI_AGAIN doubletake.local
Error: getaddrinfo EAI_AGAIN doubletake.local
    at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:69:26)
21-09-04 17:38:54 error: save url error: getaddrinfo EAI_AGAIN doubletake.local
Error: getaddrinfo EAI_AGAIN doubletake.local
    at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:69:26)

This tells me that the saving mechanism is not working internally but an internal process is calling the hostname from the HTTP request. Is this an expected behavior? In my eyes it should use localhost or a host/ip from the config file. What do you think about it?

Hey @mr-manuel, sorry I was out of town for the weekend.
You should be able to disable the self signed certificate warnings by adding this to your docker-compose.yml file.
environment:
    NODE_TLS_REJECT_UNAUTHORIZED: 0

The training endpoint takes a full image path, to make it dynamic for images that live outside of Double Take along with images from within. But this seem to cause more problems when trying to train from within the UI itself, especially when reverse proxies and self signed certs are involved. I'll tweak this logic and hopefully that will fix your issue. Rather than passing a full image path, I'll pass just the filename to the API and let the API find and copy the image rather than downloading.
Right now the image path is in this format for you: https://doubletake.local/api/storage/matches/UUID-.jpg, so if the container can't resolve this address, then that error would make sense.

I just pushed a new beta build which should fix this for you. The API will copy the image now instead of trying to download it. This should fix the errors you were getting due to the reverse proxy and certs.
Let me know when you test this out if it works.

No problem. Thank you very much!
I tested it and it works. It even do not copy an image again, if it was already copied. Before it copied an image over and over again, if I was training the same image multiple times.
In the beta from about 30 minutes ago I found some "HTTP 421 Redirection" issues, but now with the latest beta (released 5 minutes ago) it did not happen again.
Problem solved, thanks again.

Just wanted to let you know I added support for log levels now. More of a heads up for future stuff that may come up 😄


docs(README): patch README import statements

problem statement
import statements are biffed
solution
patch
discussion
it's been a long while since i've been to reactabular.  looks like a lot has changed!  time to dig in

Codecov Report

Merging #322 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #322   +/-   ##
=======================================
  Coverage   53.53%   53.53%
=======================================
  Files          21       21
  Lines         467      467
  Branches      102      102
=======================================
  Hits          250      250
- Misses        185      189    +4
+ Partials       32       28    -4

Impacted Files
Coverage Δ

...ages/reactabular-virtualized/src/calculate-rows.js
17.14% <0%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2279b9c...1651543. Read the comment docs.

Thanks for the fixes. I published the changes.
I'll work with Artem tomorrow to get everything to React 16. The implementation has evolved a lot for sure. 👍


Clarify "cd into the release directory"

I'm not entirely familiar with using command prompt. Could you give a specific step by step on how to extract and cd into the release directory?

Shift + Right click in de folder were de anirip.exe is located and then select open command prompt here

https://ss64.com/nt/cd.html
Here's a good run-down of what CD does.


Delete Discord Server List Server Links

We currently have a ban on advertisement, but I've noticed a few people getting around this by listing their server on disboard and then sending that link instead. I propose we do something similar to how we block discord invites to block these links. (Honestly this might be overkill, but I'm pretty sure advertisement isn't allowed). If given the option, I'd be fine with implementing this.

I've added https://disboard.org/ to our blacklist filter, so moderators will be pinged if it is posted.
I don't think it's necessary to hardcode this, as it doesn't happen too often.
Thank you for the suggestion!


componenet : testamonial

Build testamonials
Structure
title : "what people say"
text  "bla bla bla"
image "/upload/background.png"
items
Title: " England is awesome to raid"
text :"Lots of pretty places to visit and money to bring back home 10/10 recommend"
image  "/upload/person.png"
name : "Ragnar"
name: "King"

added


Missing word

In https://inria.github.io/scikit-learn-mooc/python_scripts/cross_validation_train_test.html

[...] cross-validation is a bit than the natural scale [...]

A word is missing after bit, I don't know which one though.

I guess "smaller" maybe @glemaitre can confirm.
Just so you know you can have hypothesis links like this: https://hyp.is/kL_S_ktkEeus3p-zfmZjNw/inria.github.io/scikit-learn-mooc/python_scripts/cross_validation_train_test.html
There is an icon to get this link at the bottom right:

Yes "smaller" it is. The mean generalization error is 45 k$ and the std of the natural scale is 115 k$.


Add nullable support to Elastic.CommonSchema.Serilog

Introduce TryGetScalarString as extention to TryGetScalarPropertyValue that does not assume ScalarProperty.Value is never null if returned.
Further more this enables stricter null checks.

Hi, Is this released already? if so, in which version? Thanks


WIP Bots

Workflow Design Board
Upstream Issue
Most of this is just a bunch of churn to make the json schema more explicitly typed so that other things can more easily consume it.
TODO:

[x] Change cargo-vet to be bottable

[x] emit extra context in the json output
[x] ? add a --with-metadata flag to be able to pass a cached cargo-metadata on stdin(?)/file(?) to operate "headless"

[x] create a Github App/Action that runs cargo vet --output-format=json-full

[x] have the bot send the json blob to our webapp and get back some kind of transcation id
[x] have the bot post a comment on the repo with the issue / link to webapp with that id
[ ] have the bot(?) able to run cargo-vet certify in CI at the webapp's request?

[x] create a webapp

[x] have a simple database (redis?)
[x] render the json report
[x] have a form for the values in certify
[x] have the webapp auth the user with github to be able to do things on their behalf
[ ] have the webapp command the bot(?) to do a certify on submit

From chat:
ok so I got a public cargo-dist repo setup at: https://github.com/axodotdev/cargo-dist
it's basically the template but with things tweaked to be cargo-* friendly. binary does nothing.
I also setup cargo-vet ci as per the book (wasmtime uses this task): https://github.com/axodotdev/cargo-dist/blob/main/.github/workflows/ci.yml#L95-L97
but pointing at my PR branch (and using --output-format=json-full): https://github.com/axodotdev/cargo-vet/pull/1
(As a side-effect the cargo-vet binary caching is broken so the tasks take extra long to fetch+build it from scratch every time.)
Here is a failing dependabot PR because it updates a dep to a version not audited in cargo-vet: https://github.com/axodotdev/cargo-dist/pull/7
Here is a the failing CI task and the JSON it vomits out: https://github.com/axodotdev/cargo-dist/actions/runs/3245691708/jobs/5323548466

The biggest hurdle to overcome at this point is creating the GH app/bot and hooking this failing CI into that, as per the Design Board

I've been prototyping out some actions stuff over at https://github.com/Gankra/gh-action-tests/ just so I'm the only one suffering under 10 million failed CI runs.
Current Status

✅ cargo-vet-check

✅ runs cargo vet --locked --output-format=json-full, and on failure (for PRs)...
✅ adds fields to the json about the repo/pr
✅ curls the json to cargo-vet.axo.dev
✅ posts a comment on the PR linking the appropriate page on cargo-vet.axo.dev
❌ does this under a generic "github ci" account and not "cargo-vet" (fine, whatever for MVP)
🚧 on-push trigger currently disabled because I'm using to develop the stuff below...

🚧 cargo-vet-certify

🚧 get-audits job mocks getting JSON from "somewhere"

here's the current json which works as a pseudo-schema

✅ cargo-vet-certify job sets up cargo-vet and gets base64 json from get-audits then invokes a certify.js action
✅ certify.js decodes the base64 inputs and runs cargo vet certify in a loop
❌ commit + push changes to branch

currently I just run git diff to test that the certify invocations worked
still reading docs, mildly confused

Future Work

We need to trigger the cargo-vet-certify workflow remotely (Nikkita)

Apparently the API we have available can be passed inputs so we can maybe remove the get-audits job altogether and just invoke the cargo-vet-certify job directly?

Need to make a commit and push to the PR branch! (Gankra)

Need to do this "as" the bot? Maybe naturally happens based on the way we trigger the job?
Do we need to use octokit to make the commit or can we legit run git commit -am ...?
Need to figure out the push API

Examples

cargo-vet-check action linking a working url to do audits at
cargo-vet-certify action producing a useful git-diff

🎊 the mvp is working!
Changes

cargo-vet-check now does not submit comments, but instead bundles up the failure in an artifact zip

the new cargo-vet-check-submit-failure task responds to cargo-vet-check failing, unpacks the zip, and submits the problem/comment

this is necessary to have the PR sandboxed (readonly) when doing things based on its code while still letting us do things like post comments (write)

a fuck ton of shell-scripty stuff is replaced with "github-script" which is just node.js programs inline in the yaml with some implicit imports

cargo-vet-certify now commits and pushes

and works cross-repo by triggering the action on the repo the PR is FROM

this has some weird permissions implications

TODO

We need to work out exactly what to do in the cross-repo commit+push situation.

There is a permission for repo owner editing branches of cross-repo PRs and this permission is granted by default!
However it seems like triggering the action is an extra permission? investigation required
If this is unsolveable we could do something like "open a PR against the PR"..?

We need to work out the UX for when two people submit audits at the same time (or one person submits, then resubmits).

Thankfully no data is "lost" because we save the form state
We could rebase and retry?

May result in duplicate audits being committed

We could run the checkout action later to minimize the time when racing can happen
??? something smarter ???

There isn't any feedback on the PR that the background tasks are running.

cargo-vet-check-submit-failure
cargo-vet-certify

Have the comments+commits be done by our bot and not the generic github actions account?


Making POST on 'refresh' permamently deregisters the service from Eureka

I have config-server registered as a eureka-server client, another config-client also registered as a eureka-server client, config-server using native call to load the local file config properties. Each time, when I POST a /bus/refresh call to config-server, the config-client get the updated properties but immediately they both get permanently unregistered with Eureka-server unless we restarted them.  by the way, the config-server and config-client both listened a local rabbitmq server.
I see this issue in almost the latest version which have the dependency management like below
<dependencyManagement>
	<dependencies>
		<dependency>
			<groupId>org.springframework.cloud</groupId>
			<artifactId>spring-cloud-dependencies</artifactId>
				<version>Dalston.RELEASE</version>
			<type>pom</type>
			<scope>import</scope>
		</dependency>
	</dependencies>
</dependencyManagement>

Also tried the version Dalston.RC1, no luck
I also see a lot of people talked about this issue before, like
https://github.com/spring-cloud/spring-cloud-netflix/issues/1652
or spring-cloud/spring-cloud-netflix#421
They all marked as fixed. Which version has the fix in it?

Eureka-server config:
server.port=1111
eureka.client.register-with-eureka=false
eureka.client.fetch-registry=false
eureka.client.serviceUrl.defaultZone=http://localhost:${server.port}/eureka/
Config-server config:
spring.application.name=config-server
server.port=7001
eureka.client.serviceUrl.defaultZone=http://localhost:1111/eureka/
spring.profiles.active=native
management.security.enabled=false
security.basic.enabled=false
config-client config: in bootstrap.properties
spring.application.name=ahspace
server.port=7002
eureka.client.serviceUrl.defaultZone=http://localhost:1111/eureka/
spring.cloud.config.discovery.enabled=true
spring.cloud.config.discovery.serviceId=config-server
spring.cloud.config.profile=dev
spring.cloud.config.failFast=true
spring.rabbitmq.host=localhost
spring.rabbitmq.port=5672
spring.rabbitmq.username=guest
spring.rabbitmq.password=guest
management.security.enabled=false
security.basic.enabled=false
Three of apps used the basic and only necessary dependencies. used the same dependency management as above.

Three apps used the parent as below:

org.springframework.boot
spring-boot-starter-parent
1.5.2.RELEASE

For Eureka server: only one dependency:

org.springframework.cloud
spring-cloud-starter-eureka-server

For Config server: only has these three dependencies

org.springframework.cloud
spring-cloud-config-server

	<dependency>
		<groupId>org.springframework.cloud</groupId>
		<artifactId>spring-cloud-starter-eureka</artifactId>
	</dependency>

	<dependency>
		<groupId>org.springframework.cloud</groupId>
		<artifactId>spring-cloud-starter-bus-amqp</artifactId>
	</dependency>

For Config client: only has these three dependencies

org.springframework.cloud
spring-cloud-starter-config

org.springframework.cloud
spring-cloud-starter-eureka

org.springframework.cloud
spring-cloud-starter-bus-amqp

@AmberHuang05 if you'd like us to look further, please provide a project on github or zipped up, not a partial project in badly formatted comments.

https://github.com/AmberHuang05/config-setting
have 3 zip files. Are they enough? Also I set up a local rabbitmq server, all default installation and setting
step1: start the rabbitmq server
step2: start the discovery-server
step3: start the config-server
step4: start the config-client
step5: check the http://localhost:1111, eureka monitor page, we can see both config-server and config-client instances were registered.
step6: send a POST request to config-server, like curl -X POST http://localhost:7001/bus/refresh
step7: go back to http://localhost:1111 page, see config-server and config client instances were deregistered permanently from eureka server.

My guess is this is some kind of weird timing issue.  If I start setting breakpoints and debugging through the code, I will notice the server will successfully reregister with the server.  However without the breakpoint the reregistration does not happen correctly.

The problem appears to be that during the refresh event the CloudDiscoveryClient is destroyed and the shutdown method in Eureka client is called.  This unregisters the client from Eureka but since the EurekaClient bean is used anywhere after the refresh event the EurekaClient bean is not recreated so we do not reregister the client.  We need to force the EurekaClient to be recreated in order to make sure the client is reregistered.

I just uploaded a new zip in https://github.com/AmberHuang05/config-setting
This sample did not use rabbitmq or message bus. It only contains Eureka server, config server and config client, both config-server and config-client registered to Eureka server and after we send out a refresh post message to config client instance, the config-client instance was deregistered to Eureka server permanently

Waiting for fix

There's currently no timeline for Dalston.SR1

got the same issue, waiting for the fix

Waiting for fix

Should be this week (if not, next).

Waiting for fix

Waiting for fix too. Any update or release date?

@ryanjbaxter Thanks for the info

@ryanjbaxter Thanks, it's fixed.

yes! Thanks alll, it's fixed, i used recent pom.
<dependency>             <groupId>org.springframework.cloud</groupId>             <artifactId>spring-cloud-dependencies</artifactId>             <version>Dalston.SR1</version>             <type>pom</type>             <scope>import</scope>         </dependency>

I'm still use Dalston.Release version.
How to fix it in my project? Upgrade is not an option .

Then you won't get the fix

@spencergibb
You are right, I'll try to upgrade version to SR4.


Pull Requests must pass the CI and be internally validated in order to be merged.

/regen-doc

/generate-screenshots


Add example usage code in README

The README file needs to be updated with example code demonstrating how to use the extension for a common use case.
Secondly, we should copy the example usage code into a unit test to ensure that the code works correctly even when the extension source code changes.

@rly can't they refer to the notebook?

I think the example in the README serves a different purpose than the notebook. The example in the readme should be short and sweat, just showing how simple it is to use the extension and what it can do. The notebook then goes into much more detail to do a step-by-step walkthrough.

I agree with @oruebel . Something like https://github.com/rly/ndx-events and https://github.com/rly/ndx-csd and https://github.com/catalystneuro/ndx-sound

#20


Comma in wrong position in readme :bing

Hmmm... You're right that the comma's location doesn't make sense, but I doubt Bing allows 500,000 queries daily when there's also a 125,000 yearly limit. We probably need to update the limits based on current Bing documentation.

Looking at the bing documentation:
Phone: Can exceed 125,000 cumulative billable transactions per calendar year.
Public Windows app and Windows Phone apps: Can exceed 50,000 cumulative billable transactions within any 24-hour period.

So looks like it it's supposed to be 50,000


[FEA] Fingerprint tables that have joins and HashAggregates

Is your feature request related to a problem? Please describe.
This issue is to add ability to create representative datasets based on statistics of input datasets that are part of join operation.
This issue is create a prototype and to track exploratory work to get the statistics that is required to generate representative data for joins and HashAggregates(if there are spills)
Reference issue: https://github.com/NVIDIA/spark-rapids/issues/11239

[ ]  Get the statistics from the input datasource(tables) which does the join operation and save it to output file. Start with parquet file.
[ ]  From the eventlog get the join type, buildside(if applicable), number of input rows, number of output rows after the join operation.
[ ]  Generated data based on the information from above 2 steps. i.e it should take into stats and the join information in generating the dataset.
[ ]  Investigate the statistics that are important if there are spills in HashAggregate operation.

Currently, I am working on a few issues related to datasourceInfo like #1172
I believe this would change the current implementation of extracting datasources and the details of SQLInfo


Drop the libsystemd dependency

It turns out that all of the libsystemd functions are actually pretty simple. There is no need to pull in a 4 MB library.

Marking this as fixed since it was done a few releases ago already


np.exp does not return inf when overflow

For example, np.exp(100) will return 100 while it should return inf.

That's a bit awkward. I'll take a look.

Sorry @hawkinsp ! It seems that the problem is resolved after I rebuild jax.

I can actually reproduce this, though.  It works fine with fast math mode disabled:
$ XLA_FLAGS=--xla_cpu_enable_fast_math=false ipython
Python 3.7.2 (default, Jan 16 2019, 11:36:28)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.2.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: from jax import numpy as np

In [2]: np.exp(100.)
/Users/phawkins/p/jax/jax/lib/xla_bridge.py:167: UserWarning: No GPU found, falling back to CPU.
  warnings.warn('No GPU found, falling back to CPU.')
Out[2]: array(inf, dtype=float32)

So I think the right fix is to disable fast math mode by default, something also tracked in #276 .

Thanks for confirming it! I first thought that it is dtype mess in my previous build. I just get this issue again. :(

Note you can work around by setting the environment variable XLA_FLAGS=--xla_cpu_enable_fast_math=false.

Got it! Thanks @hawkinsp !

This was fixed as part of a recent Jaxlib update (0.1.13). Let us know if you see more problems.

Thanks for notifying me @hawkinsp !


Add support for sending raw byte arrays

For a project, I have to support a protocol which uses raw bytes for communication. I found it to be nearly impossible to do this with the existing "send" method as it does some character set encoding magic. So I've added a new method which can be used to send raw byte arrays.

This solution will split the data first into 20Byte pices:
sendBytes(data) {
        // Return rejected promise immediately if data is empty.
        if (!data) {
            return Promise.reject('Data must be not empty');
        }

        if (!(data instanceof Uint8Array)) {
            return Promis.reject('Data must be Uint8Array')
        }

        // Split data to chunks by max characteristic value length (this._maxCharacteristicValueLength).
        var chunks = [], i;
        for (i = 0; i < data.length; i += this._maxCharacteristicValueLength){
            chunks.push(data.slice(i, i + this._maxCharacteristicValueLength));
        }

        // Return rejected promise immediately if there is no connected device.
        if (!this._characteristic) {
            return Promise.reject('There is no connected device');
        }

        // Write first chunk to the characteristic immediately.
        let promise = this._characteristic.writeValue(chunks[0]);

        // Iterate over chunks if there are more than one of it.
        for (let i = 1; i < chunks.length; i++) {
            // Chain new promise.
            promise = promise.then(() => new Promise((resolve, reject) => {
                // Reject promise if the device has been disconnected.
                if (!this._characteristic) {
                    reject('Device has been disconnected');
                }

                // Write chunk to the characteristic and resolve the promise.
                this._characteristic.writeValue(chunks[i]).
                    then(resolve).
                    catch(reject);
            }));
        }
        return promise;
    }
}

I appreciate your help, but this repository is wrong target for bluetooth-terminal changes, sorry guys.


DynamicLinqTypeAttribute does not support custom structs

Here is what to include in your request to make sure we implement a solution as quickly as possible.
1. Description
I create a custom struct for combine decimal with unit types. named as Measurable. The struct can convert numbers with many unit types.
2. Exception
Does not generate any exception. Because, I cant set the [DynamicLinqType] declaration.
Please change the code like this:
Path to file : src/System.Linq.Dynamic.Core/CustomTypeProviders/DynamicLinqTypeAttribute.cs
line 6:  [AttributeUsage(AttributeTargets.Class | AttributeTargets.Struct | AttributeTargets.Enum, AllowMultiple = false, Inherited = false)]

https://github.com/zzzprojects/System.Linq.Dynamic.Core/pull/579

Thank you for your response. With this development, it will be possible to use 'attribute' for 'struct'.


Any documentation or examples?

Hey I'm interested in using this library, but I can't seem to find any documentation or examples of how to use it. Are these available somewhere?

Unfortunately we have no documentation. This lib has been developed for our needs, bun feel free to ask any questions.                                                                                Отправлено с моего смартфона BlackBerry 10.                                                                                От: Justin MaierОтправлено: пятница, 29 мая 2015 г., 20:58Кому: agileharbor/shipStationAccessОтветить: agileharbor/shipStationAccessТема: [shipStationAccess] Any documentation or examples? (#1)Hey I'm interested in using this library, but I can't seem to find any documentation or examples of how to use it. Are these available somewhere?
—Reply to this email directly or view it on GitHub.

Ok, Thanks @DmytroSV


ALL ISSUES

"Issues" packaging order

Back-ticks additions
Links to classes and methods where back-ticks are used
Fix types defined in each parameter description where inconsistencies are present
Fix unclear descriptions

doc specifications changes ideas

could the parameter description types be updated to be code-formatted instead of italicized?
maybe instead of ", default 'infer'" to finish descriptions, we could use " (default: 'infer')" or some such thing
do default values need to be listed in the full list before they are specified as actual default? I.e. for header, the default is 'infer' but that isn't explained in the main list of options.
how should "pandas" be written? Capital or no, code-format or no?

read_csv

api change ideas

header should be header_row to align with index_col
could date_format be moved closer to parse_dates?
delim_whitespace - redundant to delimiter = '\s+'; should this be removed?
maybe combine true_values and false_values into single parameters that accepts {True: ..., False: ...}?
why does usecols ignore order? should that be changed? Or is it driven by performance optimization
why nrows but no ncols?
(research) why have true_values, false_values, and na_values; can't converters handle all this?
is there a simpler subset than na_filter, keep_default_na, na_values
parse_dates has a lot of types; is there a way that could be minimized?
could thousands and decimal be replaced by a number format string?
why do some parameters use _ and others don't? E.g., skiprows, skipinitialspace, skipfooter ... skip_blank_lines?? Could those be changed?
inconsistent use of "col" vs "column"
skipinitialspace may be better as lstrip to match the python string syntax?
index_col - why isn't None used to force skipping the first column and 'infer' used as default (to be consistent with column)?
could we define types from typing instead of vague things like sequence or list-like or array-like?

check into

index_col - when you use strings, do you have to assign the renamed versions or the original ones?
cache_dates - why would you want False ever?

errors

skiprows - should be iterable instead of list-like
(research) true_values and false_values are just "list, optional" whereas na_values is "scalar, str, list-like or dict, optional"
verbose - no explanation about how
keep_default_na - don't need to say "if ... is True" on each bullet point (factor this out)
comment - should specify "str (length 1)" like escapechar and lineterminator
thousands, decimal, comment - all should have "str (length 1)" (to be consistent with escapechar, quotechar, lineterminator)
delim_whitespace - "if True, nothing should be in delimiter: does this mean "should" or "can't"? Does it fail if you do it wrong?
delim_whitespace - the `\t' is not rendering properly (it's being expanded out to a tab-sized white space instead of the specifier
"See Also" section - read_csv is shown there; should be removed.
filepath_or_buffer - initial sentence is improper; replace with something like "filepath, url, or file object associated with the data source"
sep - should be elaborated on
sep, delimiter - what happens if both are specified with different values?
delimiter - should be optional instead of default None right?
index_col - says both "optional" and "default None" but should just say "optional"
header - maybe specify sequence instead of list because order matters (and to align with header)
usecols - maybe specify set instead of list because order is ignored
dtype - the types are not real types there (I think), i.e., "dict of column -> type"; should be specified properly
delim_whitespace - should say "nothing should be set for delimiter or sep"
lineterminator - "only valid with C parser" - change to be engine rather than "parser" to be specific. (CHECK: is this what's implied?)
header - should str be included in the types since default is 'infer' or is that technically a Literal anyway?
dtype_backend - "defaults to..." should be replaced with " default 'numpy nullable'"

clarity ideas

Don't think we need to elaborate so much on the unorderedness in usecols

read_clipboard

api changes

dtype_backend - should not be included in the signature since **kwargs associated with read_csv is already specified and it's listed there (CHECK: make sure the description and defaults are the same)

errors

dtype_backend - "defaults to..." should be replaced with " default 'numpy nullable'"
sep - default has a formatting error (shows as 's+')
sep - should not be "string or regex delimiter". Just say "regex delimiter" (implies string and specifies that it will be interpreted as regex rather than plain text)
**kwargs - should specify everything from read_csv except filepath_or_buffer (CHECK: are there also others that don't apply?")

The dtype_backend thing in the read_clipboard is probably fine. Looks like it's there as a defined named argument for it so whatever.


tree: support custom tree icon

Please make sure these boxes are checked before submitting your PR, thank you!

[x] Make sure you follow Element's contributing guide (中文 | English | Español | Français).
[x] Make sure you are merging your commits to dev branch.
[x] Add some descriptions and refer relative issues for you PR.

[Feature Request] Add icon to node on Tree Component

这个PR 什么时候能够合并到主分支啊

需要这个PR

这是要合并的意思🤔