Handling Error Responses in Login API with OpenAPI Specification

Description
I wrote a login API, and when I enter the wrong password, it sends back a status code 400 along with a message. However, I know that the OpenAPI specification doesn't directly include the raise HTTPException feature. Is there a workaround for this?
Problem Details

Current Behavior: When a user enters the wrong password, the API responds with a status code 400 and a corresponding error message.
Issue: The OpenAPI specification does not inherently support the raise HTTPException used in FastAPI for error handling. This means that the error handling as implemented does not directly translate to OpenAPI documentation.

Example Code
Here is a simplified example of the current implementation:
from fastapi import FastAPI, HTTPException

app = FastAPI()

@router.post("/login/access-token")
def login_access_token(
    session: SessionDep,
    form_data: Annotated[OAuth2PasswordRequestForm, Depends()],
) -> Token:
    """
    Get an access token for authentication.

    This endpoint authenticates a user's credentials and returns an access token
    if the credentials are valid.

    - `session`: SQLModel database session dependency.
    - `form_data`: OAuth2PasswordRequestForm object containing username and password.

    ## Returns:
    - `Token`: A Token object containing the access token and token type.

    ## Raises:
    - `HTTPException`: If the provided username or password is incorrect.
    """
    user = authenticate(
        session=session, username=form_data.username, password=form_data.password
    )
    if not user:
        raise HTTPException(
            status_code=status.HTTP_400_BAD_REQUEST,
            detail="Incorrect email or password",
        )
    access_token_expires = timedelta(minutes=settings.ACCESS_TOKEN_EXPIRE_MINUTES)
    access_token = create_access_token(user.user_id, access_token_expires)
    return Token(access_token=access_token, token_type="Bearer")

Desired Outcome
To have the OpenAPI documentation accurately reflect the API's behavior, including the handling of incorrect passwords and the resulting status code 400 response.
Question
Is there a workaround to properly document this behavior in the OpenAPI specification?
Reproducible example or configuration
No response
OpenAPI specification (optional)
"responses": {
          "200": {
            "description": "Successful Response",
            "content": {
              "application/json": {
                "schema": {
                  "$ref": "#/components/schemas/Token"
                }
              }
            }
          },
          "422": {
            "description": "Validation Error",
            "content": {
              "application/json": {
                "schema": {
                  "$ref": "#/components/schemas/HTTPValidationError"
                }
              }
            }
          }
        }

System information (optional)
No response

Yes @sagardwivedi, but that is a FastAPI question. Had the OpenAPI specification been correct, then it would be an issue with this package

Okay, I will raise the question with FastAPI. @mrlubos, thank you for your response!

Should I close the issue here?

I am trying to custom set the error message if i get the response 400
if (response.status === 400) {
      return setError(error?.detail)
}

but i am getting (property) detail?: ValidationError[] | undefined in error?.detail
but I am getting from the backend

It doesn't matter what is returned from the backend. OpenAPI specification is the contract describing the API. It's up to the server to then value that contract. In your case, there are scenarios you're not describing in the contract as you can see from your spec

same problem....
@sagardwivedi Will you bring up the FastAPI issue? Please provide a link

https://github.com/fastapi/fastapi/discussions/11882#discussion-6973409

I think it would be correct to always return the server's response status code in the case of an error. This makes more sense, as it provides the actual response status from the server.

@IgorKha I understand your point, but after thoroughly checking the FastAPI documentation, I couldn't find an alternative method. The @hey-api/openapi-ts library relies on openapi.json to generate client, and unfortunately, the openapi.json doesn't include the raise HTTPException.

Yes, I understand; I read the discussion about FastAPI. I realize this is an issue specific to their implementation. However, I would still like to ask the author of this library to consider providing an option to return the actual server response status code in case of an error, regardless of what's in the OpenAPI specification. In my opinion, this would be quite versatile and convenient for everyone.
Thank you :)

@IgorKha if you're using the new standalone client packages, you have access to the raw response object. Is that what you need?

Yes, I understand; I read the discussion about FastAPI. I realize this is an issue specific to their implementation. However, I would still like to ask the author of this library to consider providing an option to return the actual server response status code in case of an error, regardless of what's in the OpenAPI specification. In my opinion, this would be quite versatile and convenient for everyone. Thank you :)

@mrlubos

I use @hey-api/client-axios, but I also tested @hey-api/client-fetch. Indeed, the raw response contains the server status, but the issue is that you have to remember this additional step while working. This workaround does address the problem, though it's not as native as I'd like. It seems I'll have to adjust the backend 🥲
Anyway, thank you for the great tool! 😄

@IgorKha feel free to propose a different implementation!

I use @hey-api/client-axios, but I also tested @hey-api/client-fetch. Indeed, the raw response contains the server status, but the issue is that you have to remember this additional step while working. This workaround does address the problem, though it's not as native as I'd like. It seems I'll have to adjust the backend 🥲
Anyway, thank you for the great tool! 😄

FastAPI actually provides a solution to this in the documentation. When you set a responses parameter in your path operation decorators, the other response codes will show up in the OpenAPI spec and openapi-ts will capture them.


Remove database create logic inside lift, fail fast if the database doesn't exist.

Right now lift has all sorts of logic to create a database. This won't work for every datasource and it'd be better to create datasouces in prisma init or the prisma studio flows, where we can ask the user some questions and hold their hand.
If the user doesn't use the prisma init flow, we'll expect them to bring their own database (BYOD).
Instead Lift commands will check if it can connect to the database and fail fast if it can't connect. This will resolve #26 and it will make the migration engine code cleaner.
Thoughts @schickling ?

My rust code is here:
https://github.com/prisma/prisma-engine/pull/34
Next week working with @timsuchanek to use this in lift/photon.

Shipped in latest alpha.


Update User Guide for findo

Lets update user guide for find order command

Codecov Report

Merging #162 (e02b6ce) into master (7a2df27) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master     #162   +/-   ##
=========================================
  Coverage     66.10%   66.10%
  Complexity      576      576
=========================================
  Files           105      105
  Lines          2015     2015
  Branches        229      229
=========================================
  Hits           1332     1332
  Misses          604      604
  Partials         79       79

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7a2df27...e02b6ce. Read the comment docs.


Session Time Out

As a request from HR we need a bot to be closed after 15 mins of inactivity. Basically the session should be destroyed, or even better a timeout trigger to be called, where we can define our own logic to deal with session data, re-authentication, saying sorry to the user  etc..
Is there any mechanism in place for this yet? If not any work arounds? I did raise a question about using a trigger that allows you to check authorization on each type of activity, that I could use for this, but that's also not answered yet: #3379

Hi @cxbuilders. There may be some workarounds to make this scenario possible, however this capability isn't supported nicely in the SDK. I'll transfer to the appropriate project for continued discussion as a feature suggestion.


Dnd month date header fix

When using DnD, the date header on month views is not clickable despite there being an <a> tag that should take you to the drilldown view when clicked. I traced this down to the pointer-events styling done in the DnD addon's styles.less.

Apologies. This contained code from my other PR that has been approved. I'll make a different one.


signWithParams Throws 'No certificates were found that met all the given criteria'

Summary
signWithParams isn't working as expected. Running signtool directly from the command line works fine, i.e.., signtool.exe sign /s MY /n Publisher /fd sha256 /u "Code Signing" /tr http://tsa.starfieldtech.com .\setup.msi works
Repro Steps

Make sure the certificate exists
Add signWithParams property to config

...
signWithParams: '/s MY /n Publisher /fd sha256 /u "Code Signing" /tr http://tsa.starfieldtech.com',
...

@dapperdandev did you find a solution?

Nope. Just ended up dropping the signtool.exe binary into a vendor folder and running via child_process.exec. This was a while ago though and there are probably better workarounds.

@L11R  Nope. Just ended up dropping the signtool.exe binary into a vendor folder and running via child_process.exec. This was a while ago though and there are probably better workarounds.

@dapperdandev it seems we will end up the same.


Fix typo in init.pp

Pull request acceptance prerequisites:

[x] Signed the CLA (if not already signed)
[x] Rebased/up-to-date with base branch
[ ] Tests pass
[x] Updated CHANGELOG.md with patch notes (if necessary)
[x] Updated CONTRIBUTORS (if attribution is requested)

Those failures are on me; the typo fix should be safe. Thank you for your contribution, @wolfgang-lausenhammer!


Unitful example

Hi, in the Unitful example with the numerical values you are using, the mass unit should be kg instead of g
g=5.9722e24/6.3781e6**2*6.67430e-11=9.798
Refs:

https://en.wikipedia.org/wiki/Earth_mass
https://en.wikipedia.org/wiki/Gravitational_constant
https://en.wikipedia.org/wiki/Earth_radius

Thanks for the report! Should be fixed in #8.


[RFC] Replace Ruby with Kotlin

As discussed in #310 this pull request introduces a new Kotlin based plugin system as well as a much, much better plugin build system.
Major planned features:

[x] Debugging support (with IntelliJ or Eclipse's step-through debugger for example)
[x] Unit test support
[ ] Integration test framework
[x] Full editor integration (auto-completion, error reporting, etc.)
[ ] Runtime plugin reloading (in development mode only)

For a more detailed explanation of what we get with kotlin and this new plugin system see the discussion in #310.
As well as the plugin system itself, there is a lot of plugins written in Ruby left to port.  A list of these is below.

[ ] areas
[x] bank
[ ] chat

[ ] privacy
[x] private-messaging

[ ] cmd

[ ] animate
[ ] bank
[ ] item
[ ] lookup
[ ] messaging
[ ] npc
[ ] punishment
[ ] skill
[ ] teleport

[ ] combat
[ ] consumables
[ ] dialogue
[ ] dummy
[ ] emote-tab
[ ] entity

[ ] attributes
[ ] mob

[ ] extension
[ ] following
[ ] walk-to

[x] spawning

[ ] location

[x] al-kharid
[x] edgeville
[x] falador
[x] lumbridge
[ ] tutorial-island
[x] varrock

[ ] logout
[ ] navigation

[ ] door

[ ] player-action
[ ] quest
[ ] run
[ ] shops
[ ] skill

[ ] fishing
[ ] herblore
[ ] magic
[ ] mining
[ ] prayer
[ ] runecraft

[x] util

During this conversion I think we should go through and fix any open plugin issues left over from Ruby (such as the bugs that have been found, etc)

Could also add woodcutting, since it's just mining with a few different ids, I could do it myself as soon as mining is done.

@ryleykimmel ACK
@Promises I'd rather we didn't add new work onto this already monolithic task.  However, am happy to merge code into it as we go if people would like to take on plugin work themselves.

@Promises feel free to work on wc yourself and we'll offer code review + happily accept a pull request
@garyttierney util might not need to exist, need to think about what stuff to put in the (kotlin version of) bootstrap

@Major- right.  Probably some core things we want to create bindings for in the bootstrap.  Some thoughts:

Scheduled tasks - currently only used for following - per-player scheduled task or action might work better
Message Listeners - already done
Event Listeners - todo
Attribute bindings - todo - I think the bootstrap should provide this since attributes are very "core" (we need to persist them).
Game entity lookups

I'll help convert some plugins today

Seems that plugins with identical file names can't be used even if they are in other packages.
I had a file called bank.plugin.kts for banking commands and it wouldn't load because of bank plugin's identically named bank.plugin.kts file

@garyttierney ^

Cx

Lots of work done on testing now.  Any thoughts so far?  @Major- @ryleykimmel

I have created a merge request for a port of the mining plugin over to Kotlin.
If it is all good and approved I will work on porting over more plugins.
Also @Promises, if this goes through, could you possibly use it for wood cutting?

@tlf30  its pretty much the same as mining, so i'll take a look.

@Promises I had some free time so I build a woodcutting plugin. I do not have all of the ID perfect yet. I was wondering if you could post on the PR some ID numbers for trees/stumps if you had them.
Thanks,
Trevor

With kotlins extension properties I don't think we need the attributes plugin anymore, although we do need to consider how we serialize extension data.

What is left for work to get this merged?

@tlf30 At long last, not much! We want comprehensive plugin tests (#361), ways to make the testing plugins more ergonomical (#382, #395), and documentation for developing plugins (#393).

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@56cc934). Click here to learn what that means.
The diff coverage is 22.68%.

@@            Coverage Diff            @@
##             master     #338   +/-   ##
=========================================
  Coverage          ?   18.42%
  Complexity        ?      662
=========================================
  Files             ?      636
  Lines             ?    11069
  Branches          ?     1641
=========================================
  Hits              ?     2039
  Misses            ?     8823
  Partials          ?      207

Impacted Files
Coverage Δ
Complexity Δ

.../org/apollo/game/message/impl/ArrowKeyMessage.java
0% <ø> (ø)
0 <0> (?)

...o/game/message/impl/ForwardPrivateChatMessage.java
0% <ø> (ø)
0 <0> (?)

...org/apollo/game/message/impl/KeepAliveMessage.java
0% <ø> (ø)
0 <0> (?)

...apollo/game/message/impl/EnteredAmountMessage.java
0% <ø> (ø)
0 <0> (?)

...a/org/apollo/game/message/impl/CommandMessage.java
0% <ø> (ø)
0 <0> (?)

.../apollo/game/message/impl/MouseClickedMessage.java
0% <ø> (ø)
0 <0> (?)

game/src/main/java/org/apollo/Service.java
0% <ø> (ø)
0 <0> (?)

...game/message/handler/BankButtonMessageHandler.java
0% <ø> (ø)
0 <0> (?)

...rg/apollo/game/message/handler/MessageHandler.java
100% <ø> (ø)
1 <0> (?)

...essage/handler/DialogueContinueMessageHandler.java
0% <ø> (ø)
0 <0> (?)

... and 176 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 56cc934...754343a. Read the comment docs.


[CI] Include build badge in README

Include a build badge in the project’s README to show the build status of the latest PR / commit that was merged into the project’s main branch.

@stiiifff this can potentially be misleading. If users were to fork this project they'd see the badge of the original repository (this one) instead of their fork. In a normal repository that is not meant to be forked this is not a problem, but in this case it might be. Should we still do it?


Add CI via Travis and code coverage via nyc and coveralls

I mainly want to add this for selfish reasons in case I want to offer more contributions. :)
Will need to:

Sign in to coveralls.io and enable it for this repo
Add the Travis CI service on this repo and add a COVERALLS_REPO_TOKEN env var to your settings in https://travis-ci.org/replicatedhq/dockerfilelint
Enjoy watching PRs and pushes go green!

Might also be a good idea to add code linting (or at least an .editorconfig file). I like standard since it's a one-stop shop, but then you'd lose semicolons.
FYI, here's the initial nyc report with 17 passing tests:
----------------|----------|----------|----------|----------|----------------|
File            |  % Stmts | % Branch |  % Funcs |  % Lines |Uncovered Lines |
----------------|----------|----------|----------|----------|----------------|
 lib/           |    95.76 |    93.59 |      100 |    95.76 |                |
  checks.js     |    97.37 |     96.3 |      100 |    97.37 |         83,149 |
  run_checks.js |    92.86 |     87.5 |      100 |    92.86 |       24,35,52 |
----------------|----------|----------|----------|----------|----------------|
All files       |    95.76 |    93.59 |      100 |    95.76 |                |
----------------|----------|----------|----------|----------|----------------|

lgtm, coveralls and travis-ci added!


Test errors: FloatingPoint not defined (julia 0.5.0)

I'm planning on using Regression to do multinomial least squares, but when I tried to load the command after using Regression, it wasn't found. Naturally, I tried to test the package, it returns:
ERROR: LoadError: LoadError: LoadError: LoadError: LoadError: UndefVarError: FloatingPoint not defined
 in include_from_node1(::String) at ./loading.jl:488 (repeats 2 times)
 in eval(::Module, ::Any) at ./boot.jl:234
 in require(::Symbol) at ./loading.jl:415
 in include_from_node1(::String) at ./loading.jl:488
 in eval(::Module, ::Any) at ./boot.jl:234
 in require(::Symbol) at ./loading.jl:415
 in include_from_node1(::String) at ./loading.jl:488
 in macro expansion; at /home/david/.julia/v0.5/Regression/test/runtests.jl:12 [inlined]
 in anonymous at ./<missing>:?
 in include_from_node1(::String) at ./loading.jl:488
 in process_options(::Base.JLOptions) at ./client.jl:262
 in _start() at ./client.jl:318
while loading /home/david/.julia/v0.5/EmpiricalRisks/src/common.jl, in expression starting on line 35
while loading /home/david/.julia/v0.5/EmpiricalRisks/src/EmpiricalRisks.jl, in expression starting on line 84
while loading /home/david/.julia/v0.5/Regression/src/Regression.jl, in expression starting on line 6

On a side note, I've tried GLM, GLMnet, and Lasso, and it looks to me like none of these had multinomial logistic regression implemented. Do you know if this is true?

I second the issue on Julia v0.5.

I don't get this error anymore when on the master branches of Regression.jl and EmpiricalRisks.jl
Still more work was necessary to get tests passing, I have two open PRs now for that:
https://github.com/lindahua/EmpiricalRisks.jl/pull/19
#18


Seafoam Islands Item not Highlighted

There's a regular item on the 2nd floor near ladder #6 that isn't highlighted by the "Highlight Regular Items" option.

Thanks for reporting! This should now be fixed


Async notification of metadata changes [moved]

This is Issue 1155 moved from a Google Code project.
Added by 2012-11-02T18:25:38.000Z by<EMAIL_ADDRESS>Please review that bug for more context and additional comments, but update this bug.
Original labels: Type-Enhancement, Priority-Low, v2.1.0
Original description
OrientDB should provide a way to asynchronously notify the clients when the metadata changes on the server.

Option for an automatic metadata reload is also a nice to have.

This may improve consistency when one client changes the metadata while the database is used by many other clients. It also makes periodic reloads (which is needed because of the previously described case) unnecessary.

Hi,
This is implemented in 3.0, will be relased in 3.0 final.
Closing


GDPR compliance - Google Analytics

A fair warning - the search site uses Google Analytics, but does not offer a confirmation or opt out prompt, which is in violation of GDPR rules.

I think this is a front end issue. @brenoferreira assigning you, but can you please move it to the front end repo as well?

thank you @theduke ! if you have any more info, please let us know while we look into it.


Close connections after receiving an InternalServerError

For a long time, we've been working around #408, an upstream issue where connections could go into a permanently-unhealthy state by responding to all notifications with an InternalServerError after replacing an expired authentication token. We hesitated to build a fix into Pushy itself for several reasons:

We had reason to believe that the issue would receive a timely upstream fix, and so it was best not to commit too hard to a client-side fix.
It wasn't obvious that the same solution would work for everybody.
Users had a viable workaround in that they could close and re-open connections if things got stuck in an unhealthy state.

Now that we've moved to a connection-pooling model, users no longer have direct control over connections. We've also gone a looooooong time without an upstream fix, and the new connection model allows for easy behind-the-scenes connection replacement. This means that we have a more acute need for a client-side fix and fewer reasons to hold off.
Long story short: this pull request introduces behavior where clients will quietly replace unhealthy connections behind the scenes, which fixes #408.

I was digging into this a bit earlier today and came up with an idea that might be easier to implement.  The APNS Spec says we are supposed to create a new auth token after 1 hour of use of the previous token.  So it seems to me that in TokenAuthenticationApnsClientHandler.java we should be storing both the "this.authenticationToken" but also this "this.authenticationTokenIssuedAt".  Then in "getheadersForPushNotification()" we modify the test from:
if (this.authenticationToken == null) {
to something like:
if ((this.authenticationToken == null) ||
(new Date().getTime() - this.authenticationTokenIssuedAt.getTime()) > (3600L * 1000L)) {
This is just off the top of my head.  I'm not a java person so please pardon if my time calculation is ugly, but I think it shows the idea.  I'd probably put the issuedAt inside the authenticationToken rather than as another value in the  TokenAuthenticationApnsClientHandler, etc.
This does not really obviate the need for handling the server error case, but I think with the above and given the APNS spec we might reduce the number of times we have to handle it and have to do two round-trips for the push notification that happens to hit the expired token.
Thoughts?

@ksquaredkey Thanks for the suggestion! If I'm understanding right, the core of your suggestion is that we should proactively generate new tokens at the one-hour mark instead of waiting for the server to tell us that tokens have expired. That was actually something we did early on (but never shipped), but we couldn't guarantee that the server wouldn't expire tokens at some other time (i.e. they might change the the expiration period to 30 minutes instead of an hour), and so we'd need to leave the regenerate-on-expiration code in place anyhow. Rather than having two pathways to achieve the same thing, we decided to keep it simple.
To be clear, Pushy handles this all internally and automatically re-sends notifications with expired tokens. Callers should never know that it even happened. It's true that means we'll double-send some notifications, but even if we're looking at a slow round-trip time of ~100 milliseconds, that means we're "vulnerable" for 0.1 seconds / 3600 seconds ~= 0.003% of the time, and that seems pretty okay to me.
Am I addressing what you're getting at, or am I missing your point?

…either way, I think the need for this change is pressing, and revising our overall strategy for token expiration is more than we should take on as part of this effort. I'm going to go ahead and merge/ship this change.
@ksquaredkey I'd love to learn more about your suggestion, but let's plan to address it as a separate thing. Thanks!

Sounds good.  I'll work up a proof-of-concept as a separate pull request.


Classifier or regressor?

@tlienart
This doesn't look right - classifier and regression in same line of code?
LogisticClassifier_ = SKLM.LogisticRegression
@sk_clf mutable struct LogisticClassifier <: MLJBase.Probabilistic

This is from linear-classifiers.jl.
Similar code appears later for other models.

Okay, nevermind. LogisticRegression is a classifier.


'block scoped' should be 'function scoped'

Here 'block scoped' should be 'function scoped'

Probably shouldn't be Conversely with this change either.

Thanks, it was originally written function but at some point I must have accepted a PR without taking enough care :sweat_smile:


[Bug]: ROI extractor returns error "tuple index out of range"

What happened?
I am working with this code base to convert 6 subjects:
https://github.com/catalystneuro/tye-lab-to-nwb/tree/main/src/tye_lab_to_nwb/ast_ophys
I created individual convert_session.py files for each subject.  Three of them are working correctly and convert my data as expected.
However, one subject returned an error, and I am not sure what to look for in my data in order to correct it.
Steps to Reproduce
git clone https://github.com/catalystneuro/tye-lab-to-nwb
cd tye-lab-to-nwb
conda env create --file make_env.yml
conda activate tye-lab-to-nwb-env
python  ~/tye-lab-to-nwb/src/tye_lab_to_nwb/ast_ophys/convert_session_jackalope.py

Traceback
Source data is valid!
/home/lkeyes/anaconda3/envs/tye_lab_to_nwb_env/lib/python3.11/site-packages/pymatreader/utils.py:238: UserWarning: Complex objects (like classes) are not supported. They are importedeffort base but your mileage will vary.
  warn('Complex objects (like classes) are not supported. '
Traceback (most recent call last):
  File "/home/lkeyes/Projects/GIT/tye-lab-to-nwb/src/tye_lab_to_nwb/ast_ophys/convert_session_jackalope.py", line 207, in <module>
    session_to_nwb(
  File "/home/lkeyes/Projects/GIT/tye-lab-to-nwb/src/tye_lab_to_nwb/ast_ophys/convert_session_jackalope.py", line 107, in session_to_nwb
    converter = AStOphysNWBConverter(source_data=source_data)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/lkeyes/anaconda3/envs/tye_lab_to_nwb_env/lib/python3.11/site-packages/neuroconv/nwbconverter.py", line 65, in __init__
    self.data_interface_objects = {
                                  ^
  File "/home/lkeyes/anaconda3/envs/tye_lab_to_nwb_env/lib/python3.11/site-packages/neuroconv/nwbconverter.py", line 66, in <dictcomp>
    name: data_interface(**source_data[name])
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nadata/snlkt/home/lkeyes/Projects/GIT/tye-lab-to-nwb/src/tye_lab_to_nwb/ast_ophys/interfaces/cnmfe_matlab_segmentationinterface.py", line 12, in __init__
    super().__init__(file_path=file_path)
  File "/home/lkeyes/anaconda3/envs/tye_lab_to_nwb_env/lib/python3.11/site-packages/neuroconv/datainterfaces/ophys/basesegmentationextractorinterface.py", line 20, in __init__
    self.segmentation_extractor = self.get_extractor()(**source_data)
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nadata/snlkt/home/lkeyes/Projects/GIT/tye-lab-to-nwb/src/tye_lab_to_nwb/ast_ophys/extractors/cnmfe_matlab_segmentationextractor.py", line 42, in __init__
    self._image_masks = self._transform_image_masks()
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nadata/snlkt/home/lkeyes/Projects/GIT/tye-lab-to-nwb/src/tye_lab_to_nwb/ast_ophys/extractors/cnmfe_matlab_segmentationextractor.py", line 71, in _transform_image_masks
    num_rois = self.get_num_rois()
               ^^^^^^^^^^^^^^^^^^^
  File "/home/lkeyes/anaconda3/envs/tye_lab_to_nwb_env/lib/python3.11/site-packages/roiextractors/segmentationextractor.py", line 278, in get_num_rois
    return trace.shape[1]
           ~~~~~~~~~~~^^^
IndexError: tuple index out of range

Operating System
Linux
Python Executable
Conda
Python Version
3.8
Package Versions
environment_for_issue.txt
Code of Conduct

[X] I agree to follow this project's Code of Conduct
[X] Have you ensured this bug was not already reported?

It seems that the trace for that example does not have two dimensions, maybe only one roi? I think that @weiglszonja is the one that worked with that code base, could you take a look?

It seems that the trace for that example does not have two dimensions, maybe only one roi? I think that @weiglszonja is the one that worked with that code base, could you take a look?

That would be my first guess too. @laurelrr can you check the shape of the traces in the extractor?
from tye_lab_to_nwb.ast_ophys.extractors.cnmfe_matlab_segmentationextractor import CnmfeMatlabSegmentationExtractor

extractor = CnmfeMatlabSegmentationExtractor(file_path="path/to/problematic_file.mat")
# Check the original trace shape, should be [num_rois x num_frames]
print(extractor._dataset_file["C"].shape)
print(len(extractor._dataset_file["C"].shape))

If this returns a one dimensional array then we have to fix the code in the custom extractor .
Let me know if this is the case and I'll help to fix it.

I ran into the same error at the extractor line:
Python 3.11.2 (main, Mar 27 2023, 23:42:44) [GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from tye_lab_to_nwb.ast_ophys.extractors.cnmfe_matlab_segmentationextractor import CnmfeMatlabSegmentationExtractor
>>> extractor = CnmfeMatlabSegmentationExtractor("/snlkt/ast/Miniscope/expAnalysis/20211206_CACO_ALL_DISC/CACO3_DISC6/analysis/msCam_MC/G2B1_C4-6886_L_jackalope_DISC6UNP+TONES_neuron.mat")
/home/lkeyes/anaconda3/envs/tye_lab_to_nwb_env/lib/python3.11/site-packages/pymatreader/utils.py:238: UserWarning: Complex objects (like classes) are not supported. They are imported on a best effort base but your mileage will vary.
  warn('Complex objects (like classes) are not supported. '
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/nadata/snlkt/home/lkeyes/Projects/GIT/tye-lab-to-nwb/src/tye_lab_to_nwb/ast_ophys/extractors/cnmfe_matlab_segmentationextractor.py", line 42, in __init__
    self._image_masks = self._transform_image_masks()
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nadata/snlkt/home/lkeyes/Projects/GIT/tye-lab-to-nwb/src/tye_lab_to_nwb/ast_ophys/extractors/cnmfe_matlab_segmentationextractor.py", line 71, in _transform_image_masks
    num_rois = self.get_num_rois()
               ^^^^^^^^^^^^^^^^^^^
  File "/home/lkeyes/anaconda3/envs/tye_lab_to_nwb_env/lib/python3.11/site-packages/roiextractors/segmentationextractor.py", line 278, in get_num_rois
    return trace.shape[1]
           ~~~~~~~~~~~^^^
IndexError: tuple index out of range

Not sure if this is helpful or not but I loaded the file into Matlab and here is what is in it:

  struct with fields:

                    A: [90240×1 double]
               A_prev: [90240×6 double]
                    C: [0.3787 0.3574 0.4006 0.3782 0.3933 0.4259 0.3824 0.3880 0.4590 0.3978 0.3650 0.4191 0.4809 0.3841 0.3219 0.3921 0.3981 0.3856 0.3413 0.3595 0.4127 0.3992 … ]
               C_prev: [6×15218 double]
                C_raw: [0.3787 0.3574 0.4006 0.3782 0.3933 0.4259 0.3824 0.3880 0.4590 0.3978 0.3650 0.4191 0.4809 0.3841 0.3219 0.3921 0.3981 0.3856 0.3413 0.3595 0.4127 0.3992 … ]
                    S: [1×15218 double]
               kernel: [1×1 struct]
                    b: {3×4 cell}
                    f: {3×4 cell}
                    W: {3×4 cell}
                   b0: {3×4 cell}
               b0_new: [240×376 double]
              options: [1×1 struct]
                    P: [1×1 struct]
                   Fs: 15
                 file: '/nadata/snlkt/ast/Miniscope/expAnalysis/20211206_CACO_ALL_DISC/CACO3_DISC6/analysis/msCam_MC/G2B1_C4-6886_L_jackalope_DISC6UNP+TONES_msCamComb_MC.mat'
          frame_range: [1 15218]
                  ids: 2
                 tags: 0
                   Cn: [240×376 double]
                  PNR: [240×376 double]
                 Coor: {4×1 cell}
    neurons_per_patch: []
                   Df: []
                 C_df: []
                 S_df: []
              batches: {}
              file_id: []

Thank you @laurelrr, this is helpful indeed. I opened https://github.com/catalystneuro/tye-lab-to-nwb/pull/57 to fix this, can you check if this works for your data? Feel free to comment on the pull request if you have any questions. I can also test it if you share this file with me.

This was fixed in the tye-lab-to-nwb repo. Closing it now.
@weiglszonja @laurelrr  let me know if I am missing something


Problems with the @ alias when developing vue.js projects

VSCode Version: 1.41.1(user setup)
OS Version: Windows 10

Steps to Reproduce:
1.When I use the alias @ in the vue.js project, it points by default to the first project in the vs code work bar, so it is reported incorrectly.
2. Items are not affected unless they are in the first column.

Does this issue occur when all extensions are disabled?: Yes/No

Can you share your tsconfig?
/needsMoreInfo

Can you share your tsconfig?
/needsMoreInfo

{
  "compilerOptions": {
    "target": "esnext",
    "module": "esnext",
    "strict": true,
    "jsx": "preserve",
    "importHelpers": true,
    "moduleResolution": "node",
    "experimentalDecorators": true,
    "esModuleInterop": true,
    "allowSyntheticDefaultImports": true,
    "sourceMap": true,
    "baseUrl": ".",
    "types": [
      "webpack-env"
    ],
    "paths": {
      "@/*": [
        "src/*"
      ]
    },
    "lib": [
      "esnext",
      "dom",
      "dom.iterable",
      "scripthost"
    ]
  },
  "include": [
    "src/**/*.ts",
    "src/**/*.tsx",
    "src/**/*.vue",
    "tests/**/*.ts",
    "tests/**/*.tsx"
  ],
  "exclude": [
    "node_modules"
  ]
}


(newb) Beginner questions

Hi there - so I've read the docs and watched the video - all good, but I still have some very beginner questions.
Endless event streams?
If I have an event stream into which I push state from a regular AJAX update (for example), will that just grown "forever"? Or is the idea that the event stream only notifies active consumers and then forgets the message. If I cause a new listener to be attached to the DOM will that get all existing events or just new events?
Signals/Vars or Event streams?
Do you generally react to Signals/Vars or event streams? I understand the technical difference I think (stream is, well, a stream of things, Signals/Vars are simple values. Are Signals generally projections of the event stream?
Integrating with React (specifically antd)
In your video it shows how to integrate with React using a ScalaJS class representing the React component. I know ScalaJS-react and ScalablyTyped have antd wrappers - do I simply import the ScalaJS-react and then add the ScalablyTyped antd mappings and then "use" them as ScalaJS classes? Is there any prior art? I've confusing myself about the wiring up of the different eco-systems...
display:none or don't render
In my React SPAs I only render the current "page". Your examples/videos include both returning null and also setting display:none. Do you have a preference? Do you render all pages and display:none all but the active page or only build the active page?
Testing
This might be a "it's just ScalaJS use whatever" but do you have any pointers for testing, or is it more a case of seeding the event stream/Signals/Vars and then testing the rendering?
I wasn't kidding about the (newb) :-), and please forgive any dumb questions. Laminar looks perfect for what I am thinking about using it for, but I will need to integrate with external libraries (antd, Highcharts etc.)
Thanks!

Endless event streams?

As you suspected, streams only notify whatever consumers they have the moment they receive the event. After that, the event is forgotten. Signals on the other hand always keep a reference to their current value, which is basically the last event that the signal emitted. So when you add a new consumer to a signal (e.g. by means of addObserver), it will not only be notified about future events, but will also immediately receive the signal's current value (if you don't want that, add the consumer to signal.changes instead). So as you see neither signals nor streams "grow", over time they are the same size as when they started.

Signals/Vars or Event streams?

Signals are often times accumulations of event values, but in general as a rule of thumb, you should use signals when it makes sense conceptually (it's state that you need to remember, rather than an event you need to immediately react to). For example if you have a form with input, you might want to store its state in a Var, so that you can sample that Var's value when you later submit the form. Or, if you're making an Ajax request, it will produce an event stream of responses, but if you want to show a "loading" screen while you're waiting for the response, it makes sense to convert the response stream to a signal using toWeakSignal. Then you know that if your signal contains None, the response hasn't arrived yet.

Integrating with React (specifically antd)

I get similar questions occasionally, but I personally just don't use Laminar and React together so I don't know / don't remember the details. Consider asking this in our gitter or maybe in scalajs react gitter, because, this is more of a React / ScalablyTyped question. In the video I showed which React methods need to be called at which points in time and how to wire that with Laminar. Whether those methods are native React or wrapped in some interface, doesn't really matter to us, it should do the same thing anyway. Basically you might want to get a working React + ScalablyTyped setup first, then it should be obvious how to integrate it from Laminar in the manner that the video suggests.

display:none or don't render

display null is the javascript way of resetting display prop to its original value (after you've set it to none in Laminar). An element's original display value is usually block, but not always, could be inline or table-whatever, and I can't be bothered with such specifics so I just use null. Sue me :p
display: none should only be used on smaller components, certainly not on whole pages, that would be a waste of performance. To render only the current active page efficiently you should use a router like Waypoint or frontroute.

Testing

You can check Laminar's own tests. Seeding observables and checking the structure of the rendered DOM is basically what we're doing in there. You can also simulate user clicks and stuff like that. Personally I'm not really a believer in unit testing of frontend components. Feels too fragile, requires lots of mocking, and doesn't cover the stuff that breaks most often in my experience, such as visual layout and styles. I prefer testing hot paths end-to-end, with a headless browser. That kind of setup is not specific to laminar so there's lots of resources about it.
Laminar is very easy to integrate with third party components especially when they're implemented in pure JS. Stuff like highcharts should work great. Web components seem to work well too. For react-based libs you'll need to deal with their big heavy interfaces, but not much we can do about that on our side.
PS I'm gonna move this to github discussions, feel free to respond there!


  Errors should be connection-local rather than throwing to whatever might catch, so it should be a whole sale improvement and is unlikely to affect anyone whose syscalls work.
Risk Level: medium
Testing: updated tests
Docs Changes: n/a
Release Notes: n/a
https://github.com/envoyproxy/envoy-mobile/issues/176

/retest


Feature Request: Support for Azure Private DNS Zone per Azure Region

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description
The module currently does not support per region Azure Private DNS Zones.
Is your feature request related to a problem?
When working with PaaS services with built-in DR capabilities (geo-redundant storage accounts, SQL DB failover groups, etc.), multiple region Private Endpoints are mandatory.
https://learn.microsoft.com/en-us/azure/cloud-adoption-framework/ready/azure-best-practices/private-link-and-dns-integration-at-scale#private-link-and-dns-integration-in-hub-and-spoke-network-architectures.
The module currently only deploys a single set of Azure Private DNS Zones.

Describe the solution you'd like

A toggle to enable DNS zones to be deployed per virtual network hub location.
Since Private DNS zones are global resources, they need to be in different resource groups. These resource groups will need to have region specific naming.
Virtual network links to the hub will need to link to their respective Private DNS Zone. e.g. East US hub will link to the East US private DNS Zone.

Things to consider:
Related Azure policies may be affected.

Additional context
Here is a great document from Adam Stuart detailing the complexities of Multi-region use of Azure Private Link
https://github.com/adstuart/azure-privatelink-multiregion

Hi,
We do not plan to add this feature here. We will instead be working on a dedicated module for private DNS zones, which can achieve the scale that you are looking for.


UI Bug when zoom in/ zoom out browser

Description the bug
UI Bug when zooming150% in the browser

All list screen

Buy (Network Selection, Service Selection )
Export to devices
[Receive assets
Send NFT
Staking

Select a network
Select a validator
Staked result
Unstake
Compound
Claim reward
Withdraw

Resolve the issue on the v1.0.2


Hostname for storybook not localhost anymore

We just upgraded to 2.20.0 and see some problems with Storybook starting under a different hostname. Before it was always localhost now it seems to be the machine name. Even when -h <IP_ADDRESS> or -h localhost is passed. I am still investigating why this is happening with the new version.

Hi @joscha, thanks for reporting this. It's a bug recently crept into Storybooks, I'm working on the fix right now.

Released v2.20.1 with a patch. It should work correctly now.
This happened when I mistakenly removed the STORYBOOK_ prefix from config environment variables (because the same prefix is used to send data to the client). [source]

nice one, thank you @mnmtanish

just verified: this fixed the problem!


Update Blog “test1”

Automatically generated by Netlify CMS

👷 Deploy Preview for quiet-chimera-c1d66f processing.

Name
Link

🔨 Latest commit
0527bf7b2473575fcff0b3805d20a1a0b85d79bc

🔍 Latest deploy log
https://app.netlify.com/sites/quiet-chimera-c1d66f/deploys/64e24ce07758c900085739a5


Suggestion: Add UI distinction to tell tabs are tabs

Contributions

[X] I have read the contribution docs, and understand what's expected of me.

What page(s) or areas on docs.getdbt.com are affected?
https://docs.getdbt.com/reference/resource-configs/meta#designate-a-source-column-as-containing-pii is an easy example page because it contains an example at the top of the page that has a lot of "tab" options.

This is in Chrome on Mac Ventura.
What changes are you suggesting?
As a first time user it's hard to tell that Models, Sources, Seeds, etc. are tabs and that I should click on the tab to see a specific implementation. As I've continued to use the docs over time I still miss that there are other tabs I could refer to. I think adding some kind of box around the title of each tab would help visually identify that those words indicate tabs and that you should click on them.
Additional information
No response

Looks like we can customize tabs to make them clearer to users

I love this idea! I was just thinking about this when I reviewed a PR. @mirnawong1 thanks for doing that research!


add support for an enum key transformer

hey there!
as you mentioned in #183, i'd also love to have a key transformer for enums as well, so that i can camelCase csharp enum values.
thank you for your hard work, as always.

Thank you! I'll add in next version.

Fixed in 66b9418


Fix message acknowledgment configuration to handle 'on_failure' option correctly

When user set configure ack for message.
Ex:
  def handle_failed(messages, _context) do
    messages
    |> Enum.map(fn msg ->
      Message.configure_ack(msg, on_failure: :discard)
    end)
  end

Filter does not work as expected because the Enum.filter always return true.
Fix: Using message_ack_options[:on_failure] by compare with :requeue atom.

Hey @trchopan,
Thanks for taking the time to report the bug and submit a PR, really appreciate it!
Turns out there was another issue or two with the message ack'ing logic; I've bundled all the fixes into https://github.com/elliotekj/off_broadway_memory/pull/8 and will close this.
Thanks again!


Horizontal flipping isn't happening for walking

The same arm is moving, not both.

The player seems to be fine.

Ah. Oak was getting unflipped constantly...


selenium/standalone-chromium is crashing randomly likely due to less shared memory compared to Docker Desktop

Describe the bug
Browser in selenium/standalone-chromium is crashing randomly during automated tests likely due to small amount of shared memory, this works fine in Docker Desktop.
The problem is orbStack documentation does not include any information relevant to /dev/shm
Workaround is to add --disable-dev-shm-usage Chrome startup switch, this is a common practice when using Gitlab docker runners.
To Reproduce
Anything that uses selenium/standalone-chromium will likely fail more often in orbStack than normal Docker Desktop install.
EDIT: note I was using following, I wonder if that might be causing the problem, I'll do some more testing later
volumes:
- /dev/shm:/dev/shm
Expected behavior
It might be probably enough to just document this limitation.
It would be ideal if there was a setting to configure max available shared memory in the containers.
See --shm_size option in https://docs.docker.com/reference/cli/docker/container/run/
Diagnostic report (REQUIRED)
OrbStack info:
Version: 1.7.0
Commit: c5439c5612cdb5febbbc69169a4f3b11cdada6c2 (v1.7.0)
System info:
macOS: 14.6.1 (23G93)
CPU: arm64, 8 cores
CPU model: Apple M2
Model: Mac14,2
Memory: 24 GiB
Full report: https://orbstack.dev/_admin/diag/orbstack-diagreport_2024-08-28T22-41-22.676212Z.zip

Screenshots and additional context (optional)
No response

64 MiB is the standard default for Docker Engine on Linux, so you should use --shm-size. We've increased the host /dev/shm size, however, so -v /dev/shm:/dev/shm should also work on newer versions of OrbStack.

Released in v1.7.1.

tested in my setup, works now, thanks a lot


Python Courses

I have added few free python courses, from well known webSites.

Have you checked preview of your commit? Please check one time. Please make sure you format text very well so that other users can easily read it.

Hi @PSanni, thank you for the contribution!
Regarding your commits, there is a formatting issue!
We request you to send another PR with a single commit with following proper formatting. You can take this file as a reference https://github.com/GDGAhmedabad/Learning-Resources/blob/master/android/Useful_links.md

Got it.

Closing this PR! Looking forward for your next PR and awesome contribution


Statefulset: unresponsive container, locked in pending state

Hello there,
kube-aws version used for the cluster: v0.9.5-rc.6
Statefulsets becomes unresponsive randomly
Sometimes (randomly it seems), one of the containers become unresponsive (impossible to exec, other services timeout when trying to reach it, and no CPU/MEM info available on the dashboard).
I'm forced to kill them with --grace-period=0 --force (killing them softly put them in deleting state forever)
This happens with or without mounted volume
Then, restarting pods with EBS mounted fails to restart
The following happen when an EBS volume is mounted and I kill the unresponsive pod:
Then once killed, the new pod is locked in Pending state. Only way to solve it for now is to drop the Statefulset, remove the volumes and the AWS EBS disk, and and then to create the Statefulset again.
After a while:
Unable to mount volumes for pod "mysql-0_preprod(f73a4d10-202d-11e7-9e20-0676cd8703f5)": timeout expired waiting for volumes to attach/mount for pod "preprod"/"mysql-0". list of unattached/unmounted volumes=[mysql-data default-token-gndwg]
kubelet ip-xx-xx-xx-xx.eu-west-1.compute.internal
warning
Error syncing pod, skipping: timeout expired waiting for volumes to attach/mount for pod "preprod"/"mysql-0". list of unattached/unmounted volumes=[mysql-data default-token-gndwg]
kubelet ip-xx-xx-xx-xx.eu-west-1.compute.internal

The statefulset used:
apiVersion: v1
kind: Service
metadata:
  namespace: preprod
  name: mysql
  labels:
    app: mysql
spec:
  ports:
    - name: mysql
      port: 3306
  selector:
    app: mysql
---
apiVersion: apps/v1beta1
kind: StatefulSet
metadata:
  namespace: preprod
  name: mysql
spec:
  serviceName: "mysql"
  replicas: 1
  template:
    metadata:
      labels:
        app: mysql
    spec:
      terminationGracePeriodSeconds: 10
      securityContext:
        fsGroup: 999
      containers:
      - name: mysql
        imagePullPolicy: Always
        image: mariadb:latest
        ports:
        - containerPort: 3306
          name: mysql
        volumeMounts:
        - name: mysql-data
          mountPath: /var/lib/mysql
        env:
        - name: MYSQL_ROOT_PASSWORD
          valueFrom:
            secretKeyRef:
              name: mysql
              key: password
        - name: MYSQL_DATABASE
          valueFrom:
            secretKeyRef:
              name: mysql
              key: db
  volumeClaimTemplates:
  - metadata:
      name: mysql-data
      annotations:
        volume.beta.kubernetes.io/storage-class: "ebs"
    spec:
      accessModes: [ "ReadWriteOnce" ]
      resources:
        requests:
          storage: 25Gi

And storage class used:
kind: StorageClass
apiVersion: storage.k8s.io/v1beta1
metadata:
  name: ebs
provisioner: kubernetes.io/aws-ebs
parameters:
  type: gp2

kubectl describe pvc (while pod locked in Pending state) :
Name:		mysql-data-mysql-0
Namespace:	preprod
StorageClass:	ebs
Status:		Bound
Volume:		pvc-e66446ff-0fe5-11e7-a814-0676cd8703f5
Labels:		app=mysql
Capacity:	25Gi
Access Modes:	RWO
No events.

How can I recover my Statefulset without recreating it? (I would like to not lose data)
update: restarting the worker node fixes this issue
Informations
region: eu-west-1
kube-aws used: v0.9.5-rc.6

Could you paste kubectl get pv pvc-e66446ff-0fe5-11e7-a814-0676cd8703f5 -o yaml?

As I can't reproduce the issue easily (have to wait until something fails), I can't post it right now. Will post it as soon as the issue appears again


Lokalise: Translations update

This pull request was initiated by Lokalise (user Maximilian Goldschmidt) at 2023-12-14 14:52:16

French done
Thursday, December 14, 2023
09:25 (GMT-05:00)


Adds form elements

Adds a button for indicating boolean states inside forms

This is also the first PR with jest-snapshots-svg, and renames consignment files from kebab-case to snake_case

Fails

:no_entry_sign:

No CHANGELOG added.

Warnings

:warning:

Please assign someone to merge this PR, and optionally include people who should review.

:warning:

Could not find stories using these components:

Info

  Generated by :no_entry_sign: dangerJS

OK, this is now good to go - you can see the components in "Consignments: Style" - they have no interactions

Maxim already has a PR to review, @mzikherman doesn't


Fix memory leak

Hi.
I've been trying to run CHIME on my environment but continuously faced out of memory issues on small sized cache settings.
My physical server has 256GB DRAM which I thought is quite enough to run client side operations.
Looking into some code snippets, I think memory leak is occurring at TreeCache::add_to_cache(InternalNode* page) line 135.
After successful CAS from a non-null pointer to a new pointer, the original non-null pointer is not correctly freed.
At least in my environment, I have confirmed that the out of memory issue is removed after applying this change.

Thanks for pointing out the bug! I have merged your commit.


Skip hanging spawn tests

What does this PR do?
Since we haven't found a solution for all loggers, we've decided to skip these tests under the circumstances where they hang. This was attempted in #10826
That is torch<1.9 and Python>=3.8.
We don't expect this to be a problem for users as they won't be spawning as many processes together as we do when we run pytest.
Closes #10805
Does your PR introduce any breaking changes? If yes, please list them.
None
Before submitting

[ ] Was this discussed/approved via a GitHub issue? (not for typos and docs)
[x] Did you read the contributor guideline, Pull Request section?
[x] Did you make sure your PR does only one thing, instead of bundling different changes together?
[n/a] Did you make sure to update the documentation with your changes? (if necessary)
[n/a] Did you write any new necessary tests? (not for typos and docs)
[n/a] Did you verify new and existing tests pass locally with your changes?
[x] Did you list all the breaking changes introduced by this pull request?
[n/a] Did you update the CHANGELOG? (not for typos, docs, test updates, or internal minor changes/refactorings)

PR review
Anyone in the community is welcome to review the PR.
Before you start reviewing make sure you have read Review guidelines. In short, see the following bullet-list:

[x] Is this pull request ready for review? (if not, please submit in draft mode)
[x] Check that all items from Before submitting are resolved
[x] Make sure the title is self-explanatory and the description concisely explains the PR
[x] Add labels and milestones (and optionally projects) to the PR so it can be classified

Did you have fun?
Nope
cc @borda

@carmocca should we add the pytest-timeout also to the regular CPU tests?

should we add the pytest-timeout also to the regular CPU tests?

We totally can, but we haven't seen any hangs there at the moment. The current failures we have are caused by the MacOs tests actually hitting the global timeout because they are slow.

Ok, I commented this only because I saw them timeout and feared that skipping the tests had global impact on others.

The "Slow CI" addition might alleviate that since the worker tests are particularly slow on MacOS.

Finally 😓


DatePicker component

A DatePicker component already exist, but at the moment it is complicated to play with.
Suggestions/inspiration to implement a better DatePicker component:

use-date-input
Material UI
React Day Picker v8
Ant Design

Just saw this tweet https://twitter.com/devongovett/status/1539269528512647171?t=rp-JfBruOG5LS1t1Cz2_Hw&s=19


how can i verify uReplicator is 1-1 Partition Mapping replicator

i have tested uReplicator ,but is seems not 1-1 partiton replicator,src kafka and des kafka  message distribute is not the same.
<[appdeploy@cnsz22VLK12969:/home/appdeploy/kafka_2.12-<IP_ADDRESS>/bin]$sh kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list <IP_ADDRESS>:9098 --topic wktest05 --time -1
wktest05:8:490
wktest05:2:490
wktest05:5:490
wktest05:4:490
wktest05:7:490
wktest05:1:490
wktest05:9:491
wktest05:3:491
wktest05:6:492
wktest05:0:491
[appdeploy@cnsz22VLK12969:/home/appdeploy/kafka_2.12-<IP_ADDRESS>/bin]$sh kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list <IP_ADDRESS>:9098 --topic wktest05 --time -1
wktest05:8:491
wktest05:2:491
wktest05:5:490
wktest05:4:490
wktest05:7:490
wktest05:1:490
wktest05:9:490
wktest05:3:491
wktest05:6:491
wktest05:0:491

What's your worker start command?

OK,here is my conmands；
controller command: nohup ./bin/start-controller.sh -port 9000 -helixClusterName testMirrorMaker -zookeeper <IP_ADDRESS>:2181 -enableAutoWhitelist true -srcKafkaZkPath <IP_ADDRESS>:2181/kafka/test01 -destKafkaZkPath <IP_ADDRESS>:2181/kafka/test02 > controller.log 2>&1 &
work conmmand: nohup ./bin/start-worker.sh --consumer.config ./config/consumer.properties --producer.config ./config/producer.properties --helix.config ./config/helix.properties > worker.log 2>&1 &
thses properties as follows:
<[appdeploy@cnsz22VLK12969:/home/appdeploy/uReplicator-master/uReplicator-Distribution/target/uReplicator-Distribution-pkg/config]$grep -v # consumer.properties | grep -v '^$'
zookeeper.connect=<IP_ADDRESS>:2181/kafka/test01
zookeeper.connection.timeout.ms=30000
zookeeper.session.timeout.ms=30000
group.id=kloak-mirrormaker-test
consumer.id=kloakmms01-sjc1
partition.assignment.strategy=roundrobin
socket.receive.buffer.bytes=1048576
fetch.message.max.bytes=8388608
queued.max.message.chunks=5
auto.offset.reset=smallest
[appdeploy@cnsz22VLK12969:/home/appdeploy/uReplicator-master/uReplicator-Distribution/target/uReplicator-Distribution-pkg/config]$grep -v # producer.properties | grep -v '^$'
bootstrap.servers=<IP_ADDRESS>:9098,<IP_ADDRESS>:9098,<IP_ADDRESS>:9098
client.id=kloak-mirrormaker-test
producer.type=async
compression.type=none
key.serializer=org.apache.kafka.common.serialization.ByteArraySerializer
value.serializer=org.apache.kafka.common.serialization.ByteArraySerializer
[appdeploy@cnsz22VLK12969:/home/appdeploy/uReplicator-master/uReplicator-Distribution/target/uReplicator-Distribution-pkg/config]$grep -v # helix.properties | grep -v '^$'
zkServer=localhost:2181
instanceId=testHelixMirrorMaker01
helixClusterName=testMirrorMaker

Please tell me how to implement 1-1 Partition Mapping replicator ;Thanks a lot!!

When you start worker, you need an extra option --dstzk.config dstzk.properties. Content of that file is just enable=true.

I have added the extra option --dstzk.config dstzk.properties when i started the worker;and i tested again,it seems not to work,Number of messages per partition is not the same between the src and dst:
<sh kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list <IP_ADDRESS>:9098 --topic wktest09 --time -1
wktest09:8:191
wktest09:2:191
wktest09:5:190
wktest09:4:190
wktest09:7:190
wktest09:1:190
wktest09:9:190
wktest09:3:191
wktest09:6:192
wktest09:0:192
$sh kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list <IP_ADDRESS>:9098 --topic wktest09 --time -1
wktest09:8:190
wktest09:2:190
wktest09:5:190
wktest09:4:191
wktest09:7:191
wktest09:1:191
wktest09:9:191
wktest09:3:191
wktest09:6:191
wktest09:0:191>
the conent of the dstzk.properties:
<cat config/dstzk.properties
enable=true>

Sorry, dstzk.properties needs another line:
enable=true
zkServer=<dst zk path>

zkServer is the zk path your broker uses.

Nice,it's well done. Thank u very much! but i still have two questions:
1,Why does the number of synchronized partitions change while I do nothing,I set it to 5, but after a while it becomes 10:
<curl -X POST -d '{"topic":"wktest10", "numPartitions":"5"}' http://localhost:9000/topics
curl http://localhost:9000/topics/wktest10
{"externalView":{"0":["testHelixMirrorMaker01"],"1":["testHelixMirrorMaker01"],"2":["testHelixMirrorMaker01"],"3":["testHelixMirrorMaker01"],"4":["testHelixMirrorMaker01"]},"idealState":{"0":["testHelixMirrorMaker01"],"1":["testHelixMirrorMaker01"],"2":["testHelixMirrorMaker01"],"3":["testHelixMirrorMaker01"],"4":["testHelixMirrorMaker01"]},"serverToNumPartitionsMapping":{"testHelixMirrorMaker01":5},"serverToPartitionMapping":{"testHelixMirrorMaker01":["0","1","2","3","4"]},"topic":"wktest10"}
curl http://localhost:9000/topics/wktest10
{"externalView":{"0":["testHelixMirrorMaker01"],"1":["testHelixMirrorMaker01"],"2":["testHelixMirrorMaker01"],"3":["testHelixMirrorMaker01"],"4":["testHelixMirrorMaker01"],"5":["testHelixMirrorMaker01"],"6":["testHelixMirrorMaker01"],"7":["testHelixMirrorMaker01"],"8":["testHelixMirrorMaker01"],"9":["testHelixMirrorMaker01"]},"idealState":{"0":["testHelixMirrorMaker01"],"1":["testHelixMirrorMaker01"],"2":["testHelixMirrorMaker01"],"3":["testHelixMirrorMaker01"],"4":["testHelixMirrorMaker01"],"5":["testHelixMirrorMaker01"],"6":["testHelixMirrorMaker01"],"7":["testHelixMirrorMaker01"],"8":["testHelixMirrorMaker01"],"9":["testHelixMirrorMaker01"]},"serverToNumPartitionsMapping":{"testHelixMirrorMaker01":10},"serverToPartitionMapping":{"testHelixMirrorMaker01":["0","1","2","3","4","5","6","7","8","9"]},"topic":"wktest10"}

2.How to query the synchronous state? Is there any monitoring method?    Looking forward to your reply. Thank you again!

How many partitions are in source cluster? You enabled  -enableAutoWhitelist true, so controller will auto whitelist the missing partitions.

What do you mean by "synchronous state"?

Thank you for your reply;

there are 10 partitions in source cluster.
2.I mean how do you monitor replication from source cluster to destination cluster;Any tools here?

In this case, -enableAutoWhitelist true will auto whitelist the remaining 5 partitions.

You can add -javaagent:./bin/libs/jmxtrans-agent-1.2.4.jar=config/jmxtrans.xml and check the metrics in jmxtrans.xml. Please see the uReplicator-Worker/README.md after we merge #220. We will add instructions for controller later.

thank U,I see.

hello,I have one more question,Does uReplicatou support kafka_2.10-<IP_ADDRESS>?At first, the source cluster and target cluster I tested were both 0.8 version, but failed; Is there a  kafka version requirement for uReplicator?What is the minimum required  kafka version of Ureplicator? Looking forward to your reply. Thank you again!

Please try kafka-<IP_ADDRESS> branch.

hello,Is there a difference between kafka-<IP_ADDRESS> branch and  master ? When I use kafka <IP_ADDRESS> branch, it doesn't seem to replicator successfully; The worker and controller start command are the same as before.

here are errors:
1、start controller error:
2019-03-06T20:40:58.020+0800: Total time for which application threads were stopped: 0.0005317 seconds, Stopping threads took: 0.0000629 seconds
2019-03-06T20:40:58.146+0800: Total time for which application threads were stopped: 0.0005706 seconds, Stopping threads took: 0.0002429 seconds
[2019-03-06 20:40:58,159] ERROR Error writing backup to the file idealState-backup-env (com.uber.stream.kafka.mirrormaker.controller.core.FileBackUpHandler:49)
[2019-03-06 20:40:58,159] ERROR Failed to take backup, with exception: java.lang.NullPointerException (com.uber.stream.kafka.mirrormaker.controller.core.ClusterInfoBackupManager:72)
2.start work with the option --dstzk.config ./config/dstzk.properties error:
[2019-03-06T18:50:42.271+0800: Total time for which application threads were stopped: 0.0124421 seconds, Stopping threads took: 0.0000455 seconds
[2019-03-06 18:50:42,347] INFO Starting mirror maker (kafka.mirrormaker.MirrorMakerWorker$:68)
Exception in thread "main" joptsimple.UnrecognizedOptionException: 'dstzk.config' is not a recognized option
at joptsimple.OptionException.unrecognizedOption(OptionException.java:93)
at joptsimple.OptionParser.handleLongOptionToken(OptionParser.java:402)
at joptsimple.OptionParserState$2.handleArgument(OptionParserState.java:55)
at joptsimple.OptionParser.parse(OptionParser.java:392)
at kafka.mirrormaker.MirrorMakerWorker$.main(MirrorMakerWorker.scala:126)
at kafka.mirrormaker.MirrorMakerWorker.main(MirrorMakerWorker.scala)
at com.uber.stream.kafka.mirrormaker.starter.MirrorMakerStarter.main(MirrorMakerStarter.java:45)
3 start work without --option --dstzk.config ./config/dstzk.properties error:
19-03-06T20:43:44.392+0800: Total time for which application threads were stopped: 0.0003483 seconds, Stopping threads took: 0.0000616 seconds
kafka.consumer.ConsumerTimeoutException
at kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:69)
at kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:33)
at kafka.utils.IteratorTemplate.maybeComputeNext(IteratorTemplate.scala:66)
at kafka.utils.IteratorTemplate.hasNext(IteratorTemplate.scala:58)
at kafka.mirrormaker.MirrorMakerWorker$MirrorMakerThread.run(MirrorMakerWorker.scala:303)

OK，i have  implemented replicator through branch kafka-<IP_ADDRESS>;But it doesn't support  the option --dstzk.config;so it's NOT 1-1 Partition Mapping replicator; how solve this? Looking forward to your reply. Thank you again!

Do you need 1-1 partition mapping? It's fine to replicate without 1-1 mapping. We don't support that in <IP_ADDRESS> branch. If you want that, you can try to patch the diff.

Here is the diff for 1-1 mapping
https://github.com/uber/uReplicator/commit/e3e4e46462bda28895e2e2b80aa7fd70b7ae83db#diff-09d062719a93718d910837ffcd461991
I can try to make a patch over the weekend, but there is no guarantee.

Thank you for your answers. Yes, we need  1-1 partition mapping.I'll try to patch the diff,But this should be difficult for me because I'm not a developer.

https://github.com/uber/uReplicator/commit/e3e4e46462bda28895e2e2b80aa7fd70b7ae83db#diff-09d062719a93718d910837ffcd461991

https://github.com/uber/uReplicator/pull/222


Register provider via policy Part Two

Services

changeanalysis
chaos
cognitiveservices
computefleet
containerservicefleet
cosmosdbforpostgresql

Description
Please add an informative description that covers that changes made by the pull request and link all relevant issues.
If an SDK is being regenerated based on a new swagger spec, a link to the pull request containing these swagger spec changes has been included above.
All SDK Contribution checklist:

[ ] The pull request does not introduce [breaking changes]
[ ] CHANGELOG is updated for new features, bug fixes or other significant changes.
[ ] I have read the contribution guidelines.

General Guidelines and Best Practices

[ ] Title of the pull request is clear and informative.
[ ] There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

[ ] Pull request includes test coverage for the included changes.

/azp run java - changeanalysis - mgmt - tests

/azp run java - cognitiveservices - mgmt - test

/azp run java - cognitiveservices - mgmt - tests

/azp run java - computefleet - mgmt - tests

/azp run java - containerservicefleet - mgmt - tests

API change check
API changes are not detected in this pull request.


please add github as repo url like openosrs

hi, please add github as repo url

This can already be done if you follow the structure.
I've setup an example here https://github.com/Arnuh/RuneLiteHijack-PluginHub which can be used via the url https://raw.githubusercontent.com/Arnuh/RuneLiteHijack-PluginHub/master/
I'll also replace the usage of my domain with a github repo as to reduce any risk of using this project.

oke thanks man. that great hehe


🐞 - Sheet is closing on click on content in firefox

Which @taiga-ui/* package(s) are the source of the bug?
addon-mobile
Please provide a link to a minimal reproduction of the bug
https://taiga-ui.dev/components/sheet
Is this issue blocking you?
Blocking
Description
Sheet with {closeable: true} option is closing on click on content.
Angular version
12.2.16
Taiga UI version
2.78.0
Which browsers have you used?

[ ] Chrome
[X] Firefox
[ ] Safari
[ ] Edge

Which operating systems have you used?

[ ] macOS
[X] Windows
[ ] Linux
[ ] iOS
[X] Android

I've spent two days straight trying to fix this and my verdict is, unfortunately, Firefox for mobile is just bad. It works differently than Chrome and even Safari when it comes down to CSS Scroll snap and has some other issues that had me pulling my hair out. The best we can do is disable snapping in mobile Firefox. I can no longer spend resources on this due to low usage of this browser. Getting everything to work properly in mobile Firefox would require a complete overhaul of Sheet and approach taken to implement it which is relying on native browser scrolling, rather than computing CSS transforms.


Peak-driver-linux module not built

Hi,
I try to build the peak-linux-driver.
When I call bitbake core-image-rt, I can see that the tasks of the recipe peak-linux-driver are executed but pcan.ko is not added to the rootfs and I cannot find any file pcan.ko in the tmp directory (and subdirectories).
Did someone experience the same issue?
Vincent

Hi @VincentDaanen.
I believe the issue is that core-image-rt does not IMAGE_INSTALL the peak-linux-driver. But on the other hand, then it shouldn't build it either.
Did you find a peak-linux-driver.rpm in tmp/deploy/<arch>/rpm (assuming you use rpm output)? If you have the rpm, you can list the files in it by issuing rpm -qlp <file>. It could be that the rpm also gets a kernel-module prefix.
If you don't have the RPM, or anyway if it's not included on your image, you should create a new image that reuses core-image-rt and adds peak-linux-driver. I haven't tried the below so the path might be slightly off.
require meta/recipes-rt/images/core-image-rt.bb

IMAGE_INSTALL += peak-linux-driver

I just made a test-build myself and there seems to be some issue with the recipe. Yocto believes the build succeeds, but the log says otherwise:
DEBUG: Executing shell function do_compile
NOTE: make -j 12 NET=NETDEV_SUPPORT KERNEL_LOCATION=/home/pelux/pelux-master/build/tmp/work-shared/intel-corei7-64/kernel-source KERNEL_PATH=/home/pelux/pelux-master/build/tmp/work-shared/intel-corei7-64/kernel-source KERNEL_VERSION=4.9.81-intel-pk-standard CC=x86_64-pelux-linux-gcc  -fuse-ld=bfd LD=x86_64-pelux-linux-ld.bfd  AR=x86_64-pelux-linux-ar  O=/home/pelux/pelux-master/build/tmp/work-shared/intel-corei7-64/kernel-build-artifacts KBUILD_EXTRA_SYMBOLS=
make[1]: Entering directory `/home/pelux/pelux-master/build/tmp/work/intel_corei7_64-pelux-linux/peak-linux-driver/7.9-r0/peak-linux-driver-7.9/driver'
Makefile:102: *** "Can't find /home/pelux/pelux-master/build/tmp/work-shared/intel-corei7-64/kernel-source/include/linux/version.h".  Stop.
make[1]: Leaving directory `/home/pelux/pelux-master/build/tmp/work/intel_corei7_64-pelux-linux/peak-linux-driver/7.9-r0/peak-linux-driver-7.9/driver'
make[1]: Entering directory `/home/pelux/pelux-master/build/tmp/work/intel_corei7_64-pelux-linux/peak-linux-driver/7.9-r0/peak-linux-driver-7.9/driver'
Makefile:102: *** "Can't find /home/pelux/pelux-master/build/tmp/work-shared/intel-corei7-64/kernel-source/include/linux/version.h".  Stop.
make[1]: Leaving directory `/home/pelux/pelux-master/build/tmp/work/intel_corei7_64-pelux-linux/peak-linux-driver/7.9-r0/peak-linux-driver-7.9/driver'
make[1]: Entering directory `/home/pelux/pelux-master/build/tmp/work/intel_corei7_64-pelux-linux/peak-linux-driver/7.9-r0/peak-linux-driver-7.9/lib'
make[1]: warning: jobserver unavailable: using -j1.  Add `+' to parent make rule.
# ln -sf ../driver/pcan.h pcan.h
x86_64-pelux-linux-gcc  -fuse-ld=bfd src/libpcan.c -fPIC -shared -O2 -Wall -Wl,-soname,libpcan.so.0 -lc -I. -I../driver -DNO_RT -o libpcan.so.0.6
src/libpcan.c:38:10: fatal error: stdlib.h: No such file or directory
 #include <stdlib.h>
          ^~~~~~~~~~
compilation terminated.
make[1]: *** [libpcan.so.0.6] Error 1
make[1]: Leaving directory `/home/pelux/pelux-master/build/tmp/work/intel_corei7_64-pelux-linux/peak-linux-driver/7.9-r0/peak-linux-driver-7.9/lib'
make[1]: Entering directory `/home/pelux/pelux-master/build/tmp/work/intel_corei7_64-pelux-linux/peak-linux-driver/7.9-r0/peak-linux-driver-7.9/test'
make[1]: warning: jobserver unavailable: using -j1.  Add `+' to parent make rule.
x86_64-pelux-linux-gcc  -fuse-ld=bfd src/receivetest.c src/common.c -I. -I../lib -I../driver -L../lib -o receivetest   -lpcan -DNO_RT
src/receivetest.c:40:10: fatal error: stdio.h: No such file or directory
 #include <stdio.h>
          ^~~~~~~~~
compilation terminated.
src/common.c:34:10: fatal error: stdio.h: No such file or directory
 #include <stdio.h>
          ^~~~~~~~~
compilation terminated.
make[1]: *** [receivetest] Error 1
make[1]: Leaving directory `/home/pelux/pelux-master/build/tmp/work/intel_corei7_64-pelux-linux/peak-linux-driver/7.9-r0/peak-linux-driver-7.9/test'
DEBUG: Shell function do_compile finished

I'll look into it more!

Assigning this to @GordanM temporarily, since I won't be able to work on this issue anymore.

Alright @tobsan , thanks!


7.x fix completion for sub-command names

shell completion for subcommands with custom names is broken.
here is a test and a fix.
this is the same as #1529 but for the 7.x maintenance branch. changes were cherry-picked from #1529.

Decided I'll just be releasing 8.0 rather than another 7.x bugfix just for this, so closing this one.


Weird deprecation warning about deriveCodec

Hi,
I have this class definition:
@ConfiguredJsonCodec
final case class DBLog[A](
  id: UUID,
  tableName: String,
  recordID: UUID,
  verb: CrudVerb,
  snapshot: Option[A],
  createdAt: DateTime
)

With Circe 0.12.1 I'm getting this warning at compile time:
DBLog.scala:12:2: method deriveCodec in object semiauto is deprecated (since 0.12.0): Use deriveConfiguredCodec
[warn] @ConfiguredJsonCodec
[warn]  ^

And this might be because @ConfiguredJsonCodec uses deprecated methods in its implementation:
https://github.com/circe/circe/blob/master/modules/generic/shared/src/main/scala/io/circe/generic/util/macros/JsonCodecMacros.scala#L89
Could this be solved somehow?
I like to keep -Xfatal-warnings on 🙂

Hmm, yeah, let me take a look.


DBZ-7421 Improve incremental snapshot performance with increasing number of collections to snapshot

a possibile solution for https://issues.redhat.com/browse/DBZ-7421
Before
Benchmark                             (numberOfTableToSnapshot)   Mode  Cnt       Score   Error  Units
IncrementalSnapshotContextPerf.store                          1  thrpt    2  962999.691          ops/s
IncrementalSnapshotContextPerf.store                          5  thrpt    2  118903.880          ops/s
IncrementalSnapshotContextPerf.store                         10  thrpt    2   31712.744          ops/s
IncrementalSnapshotContextPerf.store                         50  thrpt    2     786.632          ops/s
IncrementalSnapshotContextPerf.store                        100  thrpt    2     195.274          ops/s

After some code optimization
Benchmark                             (numberOfTableToSnapshot)   Mode  Cnt       Score   Error  Units
IncrementalSnapshotContextPerf.store                          1  thrpt    2  926594.940          ops/s
IncrementalSnapshotContextPerf.store                          5  thrpt    2  130743.643          ops/s
IncrementalSnapshotContextPerf.store                         10  thrpt    2   30547.231          ops/s
IncrementalSnapshotContextPerf.store                         50  thrpt    2     873.143          ops/s
IncrementalSnapshotContextPerf.store                        100  thrpt    2     222.739          ops/s

With the cache
Benchmark                             (numberOfTableToSnapshot)   Mode  Cnt        Score   Error  Units
IncrementalSnapshotContextPerf.store                          1  thrpt    2  1794620.143          ops/s
IncrementalSnapshotContextPerf.store                          5  thrpt    2  1984367.094          ops/s
IncrementalSnapshotContextPerf.store                         10  thrpt    2  1792400.614          ops/s
IncrementalSnapshotContextPerf.store                         50  thrpt    2  1985691.291          ops/s
IncrementalSnapshotContextPerf.store                        100  thrpt    2  1911970.545          ops/s

Even if improving the code that serializes the data collections to JSON, the performance was not good. Adding a cache seems the best approach here.
With this implementation the serialize code will not affect the store method that is called during the event IncrementalSnapshotChangeRecordReceiver but will affect the methods that modify the dataCollectionsToSnapshot variable, but the impact in this case is limited to the number of data collections to snapshot and not at every events.
@Naros, @jpechane @vjuranek @jcechace  WDYT?

@mfvitale I think the solution seems to be right! I'd just prefer now as both the attributes are linked together not to manipulate them directly in the methods but instead provide specific methods that will change both of them when needed. WDYT?

Maybe it is necessary just for clear scenario.

@mfvitale I think it is good, thanks!
Theoretically it should be delegation (HAS-A) not inheritance (IS-A) relationship but I think it is good pragmatic solution.

@mfvitale I think it is good, thanks! Theoretically it should be delegation (HAS-A) not inheritance (IS-A) relationship but I think it is good pragmatic solution.

Isn't it?

@jpechane @Naros seems something eligible for a backport, WDYT?

@mfvitale Applied, thanks!


fix code typo in widget-behavior.md

Fixes an example code snippet that incorrectly passed a string argument to args, not a tuple. @Snehan
fixes #1037
Contribution License Agreement
By submitting this pull request you agree that all contributions to this project are made under the Apache 2.0 license.

Thanks for the catch! I switched it to a list so it's a little more readable with fewer nested parentheses. 😄 Thanks again!


when I compile CUDA function for LSS from BEVDepth，it repears  the problem like followings:

when I compile CUDA function for LSS from BEVDepth，it repears  the problem like followings:
No CUDA runtime is found, using CUDA_HOME='/usr/local/cuda-11.3/'
Traceback (most recent call last):
File "setup.py", line 44, in
sources_cuda=['src/voxel_pooling_forward_cuda.cu'],
File "setup.py", line 27, in make_cuda_ext
raise EnvironmentError('CUDA is required to compile MMDetection!')
OSError: CUDA is required to compile MMDetection!
How can I solve it ?

Thank you for your interests.
Does your device have a GPU? Is nvidia driver installed?   Is "/usr/local/cuda-11.3/" actually a directory?

yes, I have both

---- Replied Message ----

From
<EMAIL_ADDRESS>

Date
06/05/2023 19:22

To
<EMAIL_ADDRESS>

Cc
***<EMAIL_ADDRESS>

Subject
Re: [OpenDriveLab/ThinkTwice] when I compile CUDA function for LSS from BEVDepth，it repears  the problem like followings: (Issue #1)

Thank you for your interests.
Does your device have a GPU? Is nvidia driver installed?   Is "/usr/local/cuda-11.3/" actually a directory?
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID<EMAIL_ADDRESS>

Hi, I think the issue might be with mmdetection:
Do you check whether you install the mmdetection right?

Yes, let me see if I can solve this problem. Thank you for your trouble

---- Replied Message ----

From
<EMAIL_ADDRESS>

Date
06/05/2023 23:06

To
<EMAIL_ADDRESS>

Cc
***<EMAIL_ADDRESS>

Subject
Re: [OpenDriveLab/ThinkTwice] when I compile CUDA function for LSS from BEVDepth，it repears  the problem like followings: (Issue #1)

Hi, I think the issue might be with mmdetection:
Do you check whether you install the mmdetection right?
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID<EMAIL_ADDRESS>

Hi, the bug might be related with the recent large refactor of mmcv.
FYI, I use mmcv==1.7.0, mmdet==2.28.2, mmsegmentation==0.30.0, mmdet3d=1.0. Please try with the exact version of pytorch and mmcv. Hope this could be helpful :)


Change icon when alarm is pressed

When the Teltonika tracker alarm button is pressed, currently the popup displays the details in red. We could also change the icon to match, by for example adding a red outline or a red overlay to the 'normal' selected tracker icon. An example is shown below.
The way to implement this would be to add another check in the onSelect (like the id.includes("sp_") ) but on the alarm event.

Additionally, one could change the icon for the user's own tracked icon to make it clear where they are. For this, the sent ID can be checked against the marker's received ID, and if they are the same, we generate the marker with another icon.


Merge slimmed Hit and Track into devel

This certainly has bugs, mkFit efficiency is low and main seems slow.  It also disables validation and seeding code that still needs conversion.  However, it at least gets the bulky changes out for review.

Hi @dan131riley, from a visual inspection the changes look fine to me. Let me know if you need help debugging the low efficiency issue.

I found two large problems, one with mapping the hits on the simtrks correctly, which affected sMatrix because we sort the hits into layerHits; added a new function to remap from MC truth hits to the simulated hits.  Second problem was a relatively trivial issue with offsets into the Hit structure getting messed up.
I also ported over some "end - itrack" fixes from the mt2 branch.  I believe MkFitter::FindCandidates() still has problems in this area, I added a couple of assertions (currently commented out) that trigger when the # of tracks isn't a nice number, where it tries to add hits to non-existent seedidx entries.
Efficiency of MkBuilder::FindTracks() is still lower than expected, but not as dramatically, so I think this should be merged while work on that and validation continues.


Asking for suggestions for my use cases

Hi there, I was at KCD UK when I found out about this project; I immediately found it interesting and wanted to experiment a little bit with it.
Now I've tried adding the paranoia action in a project I'm maintaining where unfortunately I was already adding lots of cerfiicates so I now find myself with having three options:

remove all the certs and find a better way to add only a specific subset of certs (I'm still studying whether this is actually feasible but I'm worried it won't)
audit every single certificate and add them to the allow list so that, after the audit, I have 100% control and understanding over the certificates I'm using
allow any certificate that is installed (like I did in this PR or better by allowing all the current fingerprints) but it practically means I'm not actually using the tool so I'm just wasting action resources without adding any security with the risk of having to patch the paranoia.yml file anytime a new certificate is issued.

I believe that I should stick with option 1, however if that's not feasible what do you think would be the best option in your experience?
Also, assuming that option 1 is not feasible and that auditing +100certificate will be a pain, do you think option 3 can still be useful? I believe it can if paranoia fails the check for some unexpected issues (such as expired certificates, etc) do you see that coming? or have any plan for some interesting features that may be letting me adopt the tool even with such options at hand?
Thanks in advance!
Cheers!

Hey @fredmaggiowski, thanks for coming along at KCD UK, and for trying out Paranoia.
It's a hard problem to be sure. For me I'd look at alternatives first.

Does the tool need network access at all? If it's a server it probably doesn't need CA certs, It only needs them if it's acting as a TLS client, or if it's planning to use client authentication (which probably needs a custom CA anyway).
Can you use trust-manager, or some other way of distributing trust at runtime?

If either of thsoe are true, you could omit the certs alltogether.
Failing that, if you know you only need access to some sites, like specific APIs, then you could bundle just those certs (there's a risk if the developer of those sites changes their CA though — such as moving to Cloudflare).
If you do need generic certs, you're realisticly not going to be able to audit all 100+ certs in the normal bundle by yourself. Honestly the built-in OS bundles are not bad — it's just that they get outdated quickly.
As for what Paranoia can do, there's already a paranoia inspect command that gives a bit more detail about each CA. It's not all that automated though. I'm super interested in thoughts here!

Hi @JamesLaverack sorry for my late reply, thank you for this thorough answer!
Yes the service needs network access but I guess we could look into trust-manager, when running in k8s it may be a viable alternative (even though the service may also run outside k8s therefore someone would have to ship the necessary CA some other way, which may still be acceptable imho).
I've tried using paranoia inspect and it gave me an interesting output, currently in my bundle there are 12 certificates with issues
inspect output
Certificate CN=Cybertrust Global Root,O=Cybertrust\, Inc
┣ 🚨 expired ( expired on 2021-12-15T08:00:00Z, 52 weeks 23 hours since expiry)
┗ 🚨 removed from Mozilla trust store, comments: June 2015: DigiCert acquired this root cert from Verizon.
Certificate CN=DST Root CA X3,O=Digital Signature Trust Co.
┣ 🚨 expired ( expired on 2021-09-30T14:01:15Z, 1 year 10 weeks since expiry)
┗ 🚨 removed from Mozilla trust store, no reason given
Certificate CN=E-Tugra Certification Authority,OU=E-Tugra Sertifikasyon Merkezi,O=E-Tuğra EBG Bilişim Teknolojileri ve Hizmetleri A.Ş.,L=Ankara,C=TR
┗ ⚠ expires soon ( expires on 2023-03-03T12:09:48Z, 11 weeks 1 day until expiry)
Certificate CN=GeoTrust Primary Certification Authority - G2,OU=(c) 2007 GeoTrust Inc. - For authorized use only,O=GeoTrust Inc.,C=US
┗ 🚨 removed from Mozilla trust store, no reason given
Certificate CN=GlobalSign,OU=GlobalSign Root CA - R2,O=GlobalSign
┣ 🚨 expired ( expired on 2021-12-15T08:00:00Z, 52 weeks 23 hours since expiry)
┗ 🚨 removed from Mozilla trust store, comments: Ownership transferred to GTS:
https://bug1325532.bmoattachments.org/attachment.cgi?id=8844281
Certificate CN=Hellenic Academic and Research Institutions RootCA 2011,O=Hellenic Academic and Research Institutions Cert. Authority,C=GR
┗ 🚨 removed from Mozilla trust store, no reason given
Certificate CN=Hongkong Post Root CA 1,O=Hongkong Post,C=HK
┗ ⚠ expires soon ( expires on 2023-05-15T04:52:29Z, 21 weeks 3 days until expiry)
Certificate CN=QuoVadis Root Certification Authority,OU=Root Certification Authority,O=QuoVadis Limited,C=BM
┣ 🚨 expired ( expired on 2021-03-17T18:33:33Z, 1 year 38 weeks since expiry)
┗ 🚨 removed from Mozilla trust store, no reason given
Certificate CN=Sonera Class2 CA,O=Sonera,C=FI
┣ 🚨 expired ( expired on 2021-04-06T07:29:40Z, 1 year 36 weeks since expiry)
┗ 🚨 removed from Mozilla trust store, no reason given
Certificate CN=Staat der Nederlanden EV Root CA,O=Staat der Nederlanden,C=NL
┗ 🚨 expired ( expired on 2022-12-08T11:10:28Z, 6 days 20 hours since expiry)
Certificate OU=Trustis FPS Root CA,O=Trustis Limited,C=GB
┗ 🚨 removed from Mozilla trust store, comments: Email from the CA April 26, 2021: no new certificates have been issued since January 2018 and all expired as of January 2021.  We have a project in place to revoke any associated relying certificates which is currently being scheduled.
Certificate CN=VeriSign Universal Root Certification Authority,OU=VeriSign Trust Network+OU=(c) 2008 VeriSign\, Inc. - For authorized use only,O=VeriSign\, Inc.,C=US
┗ 🚨 removed from Mozilla trust store, no reason given
Found 129 certificates total, of which 12 had issues

I'm thinking about removing these certificate and put them in the rejection list in my .paranoia.yaml configuration to make sure I don't have them anymore, this may be a nice starting point in using the tool.
On a side note It would be nice to have fingerprints in both paranoia export and inspect commands to make it easier to find out where are certificate located and use the fingerprints for exceptions.
What do you think?


Develop

You need to commit your changes first. Read https://help.github.com/desktop/guides/contributing/committing-and-reviewing-changes-to-your-project/#platform-mac. Then open another pull request, but request to merge them into develop, not master.


HexGrid: allow alternating columns to have n/n-1 items

In HexGrid, the even number columns are shifted by half the item size. Currently each column has the same number of items. This means that sometimes the odd number columns could fit a second item to better fill the space.
before (y = 2)

after (y = 2.5)

Great suggestion, I have incorporated the functionality in β23|280  you now can enter e.g  e=5.1 or 4.9  for cutting the top or bottom. And fixed the help according your contribution.


Create list component

User Story:
As a user,
I want to have my information clearly arranged,
so i can read the data quickly
AC:

[ ] create list component
[ ] show the data from the api

Does this apply for all different sites (e.g. when looking at the mockups the different states to click through) or only the first one where you see the different assets in a list?

This list is for detail view (3rd page)


[🚧] Record Patches

Patch Description
This PR starts work on implementing record patches as described in #266.
Notes
This mostly works, but we run into some weird issues with conversion checking. For example, the following ought to typecheck:
def testing (A Z : type) (B : A → type)
            (p : Z → sig (x : A) (bx : B x))
            (z : Z) : sig (x : A) (bx : B x) as x [ x .= {p z} @ x | bx .= {p z} @ bx ] := p z

However, we get the following error:
  Expected sig (x : el {ext el/out A #t {_x => el/out {el/out p z} @ x}})

           (bx : el {ext el/out {el/out Z {el/out {el/out B p} @ x}} #t {
                     _x => el/out {el/out B p} @ bx}})

            =
           sig (x : el el/out A)

           (bx : el el/out {el/out Z z})

I believe I know what is missing! Stay tuned 😎


Enable quick exploration of Rhino Security Labs AWS PrivEsc methods

Description:

Describe your idea.  Please be detailed.  If a feature request, please
describe the desired behavior, what scenario it enables, and how it
would be used.

As discussed in this cartography meeting, we should use the data and relationships in cartography to quickly reveal attack techniques described in this article: https://rhinosecuritylabs.com/aws/aws-privilege-escalation-methods-mitigation/
We could also adapt Rhino Labs' existing script: https://github.com/RhinoSecurityLabs/Security-Research/blob/master/tools/aws-pentest-tools/aws_escalate.py; see "Scanning for Permission Flaws: aws_escalate" in that article.
For example, we could add a CreateAccessKeyVulnerable=True field to the :AWSPrincipal node to quickly identify targets accessible with a given AWS Access Key, or something else. (this is a bad example, but hopefully you get the idea of reading this article and finding neat techniques that we could quickly expose in the graph).

In general I think this can be a huge added value for organisations. Being able to statically "pentest" infra based on Cartography's data and quickly identify potential issues is a great proposition IMHO.


Git conflicts

I don't know how parsers for other languages handle the case, when a file has git conflicts, but Elixir knows about git and if you run the module below, then Elixir reports:
** (SyntaxError) test.exs:5:1: found an unexpected version control marker, please resolve the conflicts: <<<<<<< HEAD:lib/my_module.ex

Example:
defmodule MyModule do
  defstruct [:a, :b, :c]

  @type t :: %__MODULE__{
<<<<<<< HEAD:lib/my_module.ex
      a: binary(),
      b: integer(),
=======
      a: float(),
      b: boolean(),
>>>>>>> 816a01c71 (feat: use new library [JIRA-111]):lib/my_module.ex
      c: boolean
  }
end

And here how it is rendered in neovim

It would be great if tree sitter knows about git as well and doesn't parse git conflict metadata as regular Elixir terms

As far as I know, no tree-sitter parsers handle version control markers. I don't think tree-sitter parsers should look for these since they aren't part of the language (the Elixir tokenizer recognizes them but bails if it sees them), the work would be duplicated between any parsers since markers can end up in a source file for any language, and they would be tricky to parse successfully. They would be similar to parsing comments but they could be within multi-line strings.
We might be able to add extras rules for them so they would be recognized but it wouldn't work in all cases (multi-line strings) and they wouldn't be parsed structurally: you wouldn't be able to highlight the deleted changes in red and the added lines in green for example.
It's something we discussed how to support in Helix and the current thinking is that we will solve it manually rather than with tree-sitter. That being said, it might be possible to support this with plain tree-sitter by writing a version control marker parser that injects Elixir or whatever language into the non-marker parts. This would be more expensive and slower though since most of the document would be an injection and it would require combined injections. (Last I looked they were not fully supported in nvim but that may have changed.)

Thank you for response. Totally agree, it is better to rely on plugins (in case of neovim).
I'll close the issue.

As far as I know, no tree-sitter parsers handle version control markers.

This is correct. You could perhaps jury-rig something, but in general if version control markers are a thing you need to worry about, you probably want a multiple-phase parser.


113/standalone-map-menu-remaining

Fixes #113
This PR adds the download and study details content to the standalone map menu.
Note that we might want to consider preventing menu items from being unmounted, since some of them may require service calls. This would effectively cache those calls, which will make switching between menu items faster.
ETA: I also tried to make padding and width styles consistent with the wider menu entries (filter, download, notes, and study details).

That's what I was thinking. I'll make another issue for that.


Update deploy.yml

Update the heap size for building in CI
Changelog
New
Changed

Set NODE_OPTIONS to increase heap size in deploy workflow

Removed

cc @alisonjoseph


Additional powershell script for running client installation

Starting with CA and then PTF. Looking to use the individual silent install scripts versus the .bat file delivered with DPK. The DPK script doesn't have good options for silent install.

Split out client install and puppet apply steps to separate provisioning scripts.

PTF install isn't working.


Add shebang line to make app run on python3

I don't think you need this. In your virtualenv, python should be python 3 anyway.
@mknecht @zkanda Thoughts?

Yes I agree, for what reason your doing this @admiral96 ?

Most python scripts have shebangs... and also just making sure they'd use python 3 on other computers

@admiral96 when you're right, you gotta fight!  (and it rhymes, too!) :) even when @zkanda and @dashmug disagree :p
you are right that scripts should have a shebang line so they can be easily run on UNIX. But only scripts, i.e. modules that are used by being run directly. Not regular Python modules that are used by being imported. Figure out which ones are scripts and then give those proper shebang lines.
@dashmug is right that proper shebang lines should make as few assumptions about the environment as possible. Basically, you're trying to say “run this as a python script” and let the environment figure out if that means the current virtualenv, a user-installed python or a system installation.
=> I expect a new PR that makes the shebang right in the scripts. :)


Work through existing K tutorial

Work through the tutorial at https://github.com/kframework/k/tree/master/k-distribution/k-tutorial through 1_basic (whatever is present at the moment, it's a work in progress). Make sure to do all the exercises, they are intended to help train you in the "parlance" of working with K. Please write down any issues you find with the wording of the tutorial, or where you think it could be clarified. Open a pull request into K repository with your fixes and clarifications.
Max

[x] Lesson 1
[x] Lesson 2
[x] Lesson 3
[x] Lesson 4
[x] Lesson 5
[x] Lesson 6
[x] Lesson 7
[x] Lesson 8
[x] Make notes about issues you run into in the tutorial.

Henry

[x] Lesson 1
[x] Lesson 2
[x] Lesson 3
[x] Lesson 4
[x] Lesson 5
[x] Lesson 6
[x] Lesson 7
[ ] Lesson 8
[ ] Make notes about issues you run into in the tutorial.

https://github.com/kframework/k/issues/1998


Add e2e tests for all commands

The package CLI and CLI e2e are missing e2e tests.

@BioPhoton will add more details.

Closing as we decided this change doesn't make sense right now:

see comments in #199