cf env app-name var-name to get a single environment variable

When viewing environment variables for an app, it is often overwhelming to have to parse the entire collection of environment variables (e.g. to be able to pipe a single variable's value to another process). I find myself doing a cf ssh so I can echo $ENV_VAR | my-other-process. I would like to be able to do something like cf env my-app var-name | my-other-process to avoid the SSH step.

Hi Dan,
What is the use case for wanting to pipe a single env var to another process? Why not several, or even the whole shebang? (E.g. to start an app locally with the same environment variables)
cf env currently prints system provided vars, user provided vars, staging and running env var groups, which I believe is not very scripting friendly, so I want to understand how we can improve it for all its users.
In your case, the CLI may not necessarily know which particular ENV_VAR. I'm sure the backend has rules that define which takes priority, but we'd prefer not to duplicate that business logic in the CLI.
Cheers,
Dies Koper
CF CLI PM

The use case is usually to get a service binding's connection information so I can inspect it myself. I think the bigger problem about making it hard to grep is that the standard output has multiple JSON documents and plenty of plaintext. If STDOUT were a single JSON blob, I could pass that to something like jq and easily get the information I needed, without the CLI needing to include any additional business logic.

Without being able to get a single env var with out put and having non json text in the output it is pretty much impossible to use with scripting in a clean way. Would love to be able to grab binded services variables using this command.

Have to agree with this, it makes scripting a pain, choose JSON or KEY=VALUE, don't output both at once.
Ideally there would be a switch to toggle between KEY=VALUE or JSON.

Agreed, the current mix of plaintext / JSON-objects makes scripting really hard if not impossible.

Indeed.
Here is what I needed to do to get some environment variables from the "User Provided" section
export $(cf env ${CFAPP} | \
  awk '/User-Provided:/{flag=1;next}/^$/{flag=0}flag' | \
  grep -e 'AWS_ACCESS_KEY_ID' -e 'AWS_SECRET_ACCESS_KEY' -e 'AWS_STORAGE_BUCKET_NAME' | sed "s/\: /\=/g")

Which is straightforward..ish, to grab variables from the system-provided part, you have to do a similar thing and pipe via jq, which is even more hassle.
There needs to a way to grab either or both sections in key=value.

Here is the small snippet to list all the environment variables of PCF:
cf ssh <App_Name> -c "python -c 'import os, json; print(json.dumps({key:value for (key,value) in os.environ.items()}, indent = 2))'"
{
"INSTANCE_GUID": "315c18fb-e031-4e6e-67c4-84d1",
"CF_INSTANCE_PORT": "61064",
"CF_INSTANCE_KEY": "/etc/cf-instance-credentials/instance.key",
"USER": "vcap",
"INSTANCE_INDEX": "0",
.
.
.
.
"PWD": "/home/vcap",
"VCAP_APP_PORT": "8080"
}
}

Related: https://github.com/cloudfoundry/cli/issues/1415

With the GA of the v7 CLI, we're no longer actively developing against the v6 line.  With an interest the overall hygiene of the CLI project we're closing this  issue.
If this issue is still occurring in v7 please feel free to comment and re-open. Thank you!

Please Re-Open.
It still outputs a mix of plain text and JSON which is still hard to extract values - even with regular expressions.
Backward Compatibility - if needed
For backward compatibility reasons you may add this behaviour only by giving an additional parameter to get the JSON output.
Meaning --json has to be given for STDOUT a JSON object:
cf env myappname --json
Suggested solution - as already mentioned:

use one single JSON as STDOUT
and additionally give an option to get a single property value:

cf env myappname --json -p propertyname
Examples (with Backward Compatibility):
Using:
$ cf env myappname --json

must return valid JSON Object as STDOUT:
{
    VCAP_SERVICES: {
        "application-logs": [ ... ],
        "connectivity": [ ... ],
        ...
    },
    VCAP_APPLICATION: {
        "application_id": "30051990-0000-0000-0000-33cf333e3f33",
        ...
    },
    "User-Provided": {
        TENANT_HOST_PATTERN: "xyz",
        ...
    }
}

Specifying a property name:
$ cf env myappname --json -p VCAP_APPLICATION

must return given property value as valid JSON STDOUT
{
    "application_id": "30051990-0000-0000-0000-33cf333e3f33",
    ...
}

⚠ Please Note:
"..." in Example-JSON object stands for the remaining properties and values of the valid object.

Please Re-Open.
It still outputs a mix of plain text and JSON which is still hard to extract values - even with regular expressions.
Backward Compatibility - if needed
For backward compatibility reasons you may add this behaviour only by giving an additional parameter to get the JSON output. Meaning --json has to be given for STDOUT a JSON object:
cf env myappname --json
Suggested solution - as already mentioned:

use one single JSON as STDOUT
and additionally give an option to get a single property value:

cf env myappname --json -p propertyname
Examples (with Backward Compatibility):
Using:
$ cf env myappname --json

must return valid JSON Object as STDOUT:
{
    VCAP_SERVICES: {
        "application-logs": [ ... ],
        "connectivity": [ ... ],
        ...
    },
    VCAP_APPLICATION: {
        "application_id": "30051990-0000-0000-0000-33cf333e3f33",
        ...
    },
    "User-Provided": {
        TENANT_HOST_PATTERN: "xyz",
        ...
    }
}

Specifying a property name:
$ cf env myappname --json -p VCAP_APPLICATION

must return given property value as valid JSON STDOUT
{
    "application_id": "30051990-0000-0000-0000-33cf333e3f33",
    ...
}

⚠ Please Note: "..." in Example-JSON object stands for the remaining properties and values of the valid object.

This seems like a very helpful setting. Otherwise its quite tedious to get single values from the arbitrary response format.

Hello,
it would be very helpful if this issue would be reopened. Everytime we deploy something to the CloudFoundry Space, the environment variables change.
In order to develop locally but still have a connection to the CF Space (and the attached backends) we need one of those variables in our local .env file and change it a little bit before that.
For now we are using some js that does some nasty string operations so having a json output would be very helpful.

Hello friends, I am also facing the same issue, since some credentials of services expires and need to be regenerated often so want to retrive the env for Vcap_services>
Still this issue is not resolved ?


(feat) Add WandB

WandB stands for Weights & Biases, it is a tool for visualizing and tracking ML experiments, from datasets to production ML models.

GitHub Repository
Website

Hi Nour,
thank you very much for your feedback and the PR. This list focusses on tooling that is open source / open core. I am pretty sure the WandB repository is only the client for the proprietary WandB web plattform.
Although WandB is a great tool and also offers some functionality for DCAI, it doesn't fit the scope of the list.
Best,
Stefan


Close button mis-aligns header text

https://github.com/samaritanministries/dash-modal.js/blob/b3244a19e7066c1492f3ada6ecb81f6d973ed980/styles/dash-modal.scss#L56
Add margin-left: -2rem to the close button to bring the text back.
Also add top: 0.75rem instead of 1rem to better position.

Fixed this with v3.0.3


chore: fix tekton patch func-deploy to be as clustertask

Fix missing patch for func-deploy Task to be threat as ClusterTask on midstream.

/approve
/lgtm


Add Launch to camera calibration tutorial

Alternative strategy, I am creating a docker container for the camera calibration app. The app runs directly via Python having sourced built ROS packages and installed python deps.

Strategy of deploying via Docker container employed due to it resolving multiple issues including getting correct python dependencies, a local build of the workspace is more complex.


Update Cache Interface keys method to return IterableIterator

Update Cache Interface keys method to return IterableIterator<Key>
Motivation & Background
As it's currently typed, the Cache interface that is required for users to create and provide their own cache implementations mandates that the keys method return an IterableIterator<string>. I believe this to be incorrectly typed, or at the very least inconveniently typed and a burden for consumers. I think that the correct type should be IterableIterator<Key> given that the get, set, and delete methods all require a Key as input.
Consider the default implementation provided by swr:
// Default cache provider
const [cache, mutate] = initCache(new Map()) as [Cache<any>, ScopedMutator<any>]

This effectively the same as :
const cacheMap: Cache<any> = new Map<Key, State<any, any>>()
const [cache, mutate] = initCache(cacheMap) as [Cache<any>, ScopedMutator<any>]

Which results in a type error:
Type 'Map<Key, State<any, any>>' is not assignable to type 'Cache<any>'.
     The types returned by 'keys()' are incompatible between these types.
       Type 'IterableIterator<Key>' is not assignable to type 'IterableIterator<string>'.
         Type 'Key' is not assignable to type 'string'.
           Type 'undefined' is not assignable to type 'string'.

This puts a burden on users/consumers of swr when implementing their own cache, especially if the underlying data structure is a Map.
My use case is creating a custom LRU Cache to back swr, se the following gist of its implementation: https://gist.github.com/dmmulroy/33c31bef81554b099726b3a680dad974
Because keys is typed as IterableIterator<string> I have to cast.
/**
   * Returns an iterable iterator over the keys in the cache.
   *
   * @returns {IterableIterator<string>} - An iterator of the cache keys.
   */
  public keys(): IterableIterator<string> {
    return this.cache.keys() as IterableIterator<string>
  }

An Alternative Approach
An alternative approach would be to update the Cache interface to the following:
export interface Cache<Data = any> {
  keys(): IterableIterator<string>
  get(key: string): State<Data> | undefined
  set(key: string, value: State<Data>): void
  delete(key: string): void
}

This is actually may be more likely to be accurate and reflect what occurs at run time. As far as I can tell, any time the cache is accessed by swr internally the key is always serialized into a string via the serialize^1 function prior to access.
You can see that any time the cache is accessed[^2], it's accessed via callbacks returned from createCacheHelper[^3] in which the SWRGlobalState is referenced/retrieved and used get to the setter callback from initCache[^4] which is stored in global state. In all instances (you can search by reference to confirm), the Getter and Setter callbacks are passed string keys which have been serialized.
In addition, I believe this may actually be worth considering because unstable_serialize[^5] is publicly exported. I assume this is to aid consumers/users if they choose to directly access/mutate the cache (despite explicitly being called out in the docs as anti pattern[^6]) as seen in #2216.
[^2]: Getter Callback, Setter Callback, Previous value in Setter, Setter Callback from SWRGlobalState
[^3]: https://github.com/vercel/swr/blob/main/_internal/utils/helper.ts#L32
[^4]: https://github.com/vercel/swr/blob/main/_internal/utils/cache.ts#L26
[^5]: https://github.com/vercel/swr/blob/main/core/use-swr.ts#L679
[^6]: https://swr.vercel.app/docs/advanced/cache#modify-the-cache-data

I also just noticed that the docs are incorrect as well and list the type of Cache as the following:
interface Cache<Data> {
  get(key: string): Data | undefined
  set(key: string, value: Data): void
  delete(key: string): void
  keys(): IterableIterator<string>
}

Thanks for the PR !
I think we should take the alternative approach here.
It is more simple and the fact is that we only use string key here.

Awesome - I'll be happy to make those changes and get them pushed up tomorrow or sunday.

Changes have been made, take a look when you have the chance @promer94 🚀


Remove QueryString Const

jape can't handle constants in DecodeForm, it panics with panic: unhandled expr type (*ast.SelectorExpr). We should fix it in the analyzer but until that's fixed I decided to remove the query string constants since we were hardly consistent about using constants for query string params anyway.

I opened a PR for jape here: https://github.com/SiaFoundation/jape/pull/12


Voice consistency

The voice of the bot is not consistent when deploying on different machines.

not an issue. changed the voice by using different voice from speechSynthesis module


Add span links to messaging system tracing

What does this PR do?
Closes #2489:

[x] Adding basic span links capabilities
[x] Add span link to polling spans in all messaging plugins
[ ] Kafka: when there is an active transaction before the beginning of the iteration - create a message-handling span for each record with a single link to the sending span, if such exists. Otherwise- create a transaction per record
[ ] SpringAmqpBatchMessageListenerInstrumentation - same as Kafka (if makes sense)
[ ] AWS Lambda functions triggered by SNS/SQS: create one transaction always and add span links
[ ] Add to changelog and elsewhere in documentation if required

:green_heart: Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2022-05-03T12:09:29.680+0000

Duration: 35 min 5 sec

Test stats :test_tube:

Test
Results

Failed
0

Passed
2322

Skipped
15

Total
2337

:green_heart: Flaky test report
Tests succeeded.
:robot: GitHub comments
To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.

run benchmark tests : Run the benchmark tests.

run jdk compatibility tests : Run the JDK Compatibility tests.

run integration tests : Run the Agent Integration tests.

run end-to-end tests : Run the APM-ITs.

run windows tests : Build & tests on windows.

run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

run integration tests

run integration tests


X offset of a text placed on a line is ignored

On the text-style-linestring-nice test, there is an example of a text placed on a line with placement: 'line' and offsetX: -10: https://github.com/openlayers/openlayers/blob/7c50a849cbccb379b7d3c9f11d024739aa7b8bc6/test/rendering/cases/text-style-linestring-nice/main.js#L85-L92
However, the offsetX attribute does not play any role, the value seems to be ignored.
The rendering test does not fail as the expected image does not contain the offset either.

I think the idea for this test was to test for offsetY, not offsetX. offsetX is indeed not supported for text along lines. It could be implemented though. So if you want to give it a shot, feel free to submit a pull request.


fix: emit update event of plugin API by scene property update

Emit update  event of plugin API by scene property update

A problem was found in which the sceneProperty referenced by the plugin API remained out of date even if the plugin was successfully notified of an update of the sceneProperty. Fixing this will take some time, so it will be done at the same time as the layer system will be revamped in the future.


[Feature Request]: Goals+ By Game

What feature would you like to see?
Similar to how we can use get_team_xgoals() to pull xgoal data by team, season, and game, it would be awesome to see goals+ split out by those same fields.
I think it would be helpful for overall analysis- I'm working on building out a prediction model currently based on xgoals and would love to see if goals+ would improve accuracy or any underlying metrics.
No worries on time especially with the season ending somewhat soon - goals+ is a wishlist metric and thought I would at least ask! Thanks again for the great work on this package and overall data

Hey @nielsenz, sorry it's take so long to respond to this. This would be a great feature to have but we'd most likely have to implement this on the API first and then propagate it to itscalledsoccer. We'll add it to our backlog and let you know when it's available

@nielsenz Actually, we do have this available already for player and goalkeeper goals added. You just need to specify the split_by_games argument like so:
from itscalledsoccer.client import AmericanSoccerAnalysis

asa_client = AmericanSoccerAnalysis()
goals_added_by_game = asa_client.get_player_goals_added(league="mls", season_name="2023", split_by_games=True)


can not find dnn_semantic_segmentation_ex.h

I can not find where is 'dnn_semantic_segmentation_ex.h' header file ?
neither did my compiler?
why?

In the examples folder.


autodetect shader uniforms

new ShaderReloader method that finds all uniforms not set by processing and creates a new control element for each

maybe custom annotation


Merge main branch into dev-1.26

Merge main branch into dev-1.26 to keep in sync, part of the Docs activities in the release process, see here.
CC 1.26 Docs Team: @cathchu  @mickeyboxell @katmutua @Rishit-dagli
cc @reylejano

clean sync
/approve

/lgtm


chore: Debian 12 as base image in Dockerfile drift-labs/gateway#65

lgtm, feel free to include the curl installation too

lgtm, feel free to include the curl installation too

done

ty for PR!


Reshaping Tensor in custom Layer with unspecified Batch Size

Hello,
I'm writing a Maxout layer or Feature Pooling layer for both backends. The batch size is not fixed.
In Theano it's pretty straightforward:
import keras.backend as K
class FeaturePool(Layer):
        ...
        def call(self, x, mask=None):
                # in Theano, self.axis=1, pool_size=2 then the channel of the output will be halved
                # input shape: (None, n_channel, col, row)
                input_shape = tuple(x.shape)
                num_feature_maps = input_shape[self.axis]
                num_feature_maps_out = num_feature_maps // self.pool_size

                # pool shape: (None, num_feature_maps_out, pool_size, col, row)
                pool_shape = (input_shape[:self.axis] +
                              (num_feature_maps_out, self.pool_size) +
                              input_shape[self.axis+1:])

               input_reshaped = K.reshape(x, pool_shape)
               # and the final output will be a max function over the pool_size dimension
               return K.max(input_reshaped, axis=self.axis + 1)

But when implementing it for tensorflow backend, the input shape will be defined as
input_shape = x.get_shape()

which is a TensorShape and when I'm doing the reshape, an error occurs saying that the reshape is not possible because the shape contains None.
I'd definitely need this layer in tensorflow but currently haven't found a solution, any suggestion will be greatly appreciated.

This is a TensorFlow question. The answer is to use:
shape = K.shape(x)
pool_shape = tf.pack([shape[0], ...])  # Here you can mix integers and symbolic elements of `shape`
input_reshaped = K.reshape(x, pool_shape)

Group normalization in keras need to handle the reshape consists of None dimension.

Group normalization in keras need to handle the reshape consists of None dimension.

Any details to solve None dimension?

Keras contrib has support for Group Normalization with None support.


Change alpha to beta in source ip docs

Just substitute alpha for beta and remove the parts about enabling the alpha annotation.
Will probably send a better concepts/ doc as a follow up, but this is the bare-minimum.

This change is

tech LGTM


make it work

Thank you very much, I'll release a new update asap :)

Thank you very much, I'll release a new version asap :)

Version 0.1.2 released in the Chrome Web Store. Should be rolled out soon to everyone.


feature: Fetch First 10 Rows

Feature scope
CLI (options, error messages, logging, etc.)
Description
Hello,
Im usning meltano CLI, im facing one issue since many days, the issue is i want to fetch just 10 rows of table but they return me all the rows how i can over come from this? im try filers, select is working but this thing is not working have any solution of this?
Thanks

Hi @AbdullahFiveRiver!
I'm closing this as a duplicate of https://github.com/meltano/meltano/issues/8080. Do comment here or in the other issue if you disagree. Please leave a 👍 there if that's the use case you're looking for.


plotnine issue

Hi,
just wanted to run your exploration notebook but plotnine cannot be imported since it loads matplotlib._contour which does not exist (in version 3.6.0).
ModuleNotFoundError: No module named 'matplotlib._contour'
See also here: link
If I did anything wrong with the setup, please let me know.
Thanks in advance!

Thanks for flagging this @mertensu !
Uh oh, looks like this follows from a breaking change in matplotlib 3.6.0, also released 2 days ago.  There is an active issue in plotnine, opened 2 days ago: https://github.com/has2k1/plotnine/issues/619
@ivanorsolic , could you please pin the plotnine and maltplotlib versions in environment.yml to pre-break versions until this is fixed? Generally for reproducibility it would be a good idea to pin all versions while we're at it :)

See https://github.com/worldstrat/worldstrat/pull/7

Thanks @mertensu ! Closing as fixed by #7 .


List available features

Very simple script listing the rules present in the repository. Output is a CSV file, which can be parsed to Excel quite nicely:

I can make an .exe file out of the script of course.

@evandroAlfieri That makes sense. It happens quite often already that a PR evolves to a bigger discussion and then remains open indeed .. Let's merge it then
@JTurbakiewicz I've approved the changes. If I'm right you can merge the PR when you resolve the conversations (1 approving review).


Want different users to see different data based on their unique API key that is used in Custom connector

I have a custom connector and it gets data back from an API. Each user whitin the company has an API key which brings back different levels of data. I have a report and I want it to be able to change based on which user is viewing it. So if a person with a low level key logged in and viewed the report in the application they would only see as much data as the API key allows them. I have embedded this report using App owns data method but everyone is viewing the data using my API key which is used in the Gateway. Anyone know how I can get the report to be able to refresh the data for that user that logged in and show them the data that they have access too based on the API key. Row Level security won't work. The data has to load for each user using their unique key

This isn't possible today at all through import, and requires special coding in the service on a per-connector basis to work with Direct Query.

Hi Curt,
Thanks very much for responding. Can you give me any additional information on how to go about the special coding in the service on a per-connector basis to work with Direct Query. I'm only an undergraduate software developer at the moment so I don't have too much experience in this area. Any advice would be appreciated
Regards,
Darragh

From: Curt Hagenlocher @.>
Sent: 02 April 2022 14:03
To: microsoft/DataConnectors @.>
Cc: Darragh Casserly @.>; Author @.>
Subject: Re: [microsoft/DataConnectors] Want different users to see different data based on their unique API key that is used in Custom connector (Issue #485)
This isn't possible today at all through import, and requires special coding in the service on a per-connector basis to work with Direct Query.
—
Reply to this email directly, view it on GitHubhttps://github.com/microsoft/DataConnectors/issues/485#issuecomment-1086635920, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AYPIATXHJSG567IP7BYJZKLVDBASBANCNFSM5SBM23TQ.
You are receiving this because you authored the thread.Message ID: @.***>

I should have just said that it's not possible at all, as the practical distinction is basically irrelevant: you as a third-party have no way of using any kind of single-signon or single-signon-adjacent mechanism other than via Windows authentication in the Gateway.


HDDS-2999. Move server-related shared utilities from common to framework

What changes were proposed in this pull request?
hdds-common project is shared between all the client and server projects. hdds-server-framework project is shared between all the server side services.
To reduce unnecessary dependencies (to Hadoop, for example) we can move all the server-side related classes (eg. rocksdb layer, certificate tools) to the framework from the common.
We don't need the rocksdb utilities and certificate tools on the client side.
What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-2999
How was this patch tested?
With maven build. This is only a compile level restructure of the code

@anuengineer Next step forward to a Hadoop independent common. Please review if you have time (no logic has been changed just moving methods and classes between locations...)

Also, making tools depend on server-framework seems a bit awkward, but we can improve it later.

Agree. Earlier I proposed to rename all the proto files to use three categories:

client protocols (eg. ozone-client, hdds-client proto): for the types which should be available from the client
server protocols (for services between servers, like the cert client, can be blocked on the firewall from outside)
admin protocols (for protocols used by admin cli tools)

We can organize the dependencies based on similar principals.
As of now we have:

common (shared between servers and clients)
framework (shared between servers)

We can create separated project for the shared classes between tools (cli) and servers ("admin" classes)

common (shared between servers, clients, tools)
framework (formerly common-serverside) (shared between servers)
common-tools (shared between servers and tools)

I am not sure about the dependency hierarchy between common-tools and common but we can do it in the near future...


Data for radar not displaying (nlhrw)

I tried to checkout the data present in baltrad for nlhrw in spring 2024 ( https://crow.aloftdata.eu/#/?radar=nlhrw&date=2024-04-14&interval=24&timedisplay=UTC&vpiMode=mtr&vpColorScheme=viridis&lang=en ). For me the data is not displaying (tried two browsers), while the data seems to be present: https://aloftdata.eu/browse/?prefix=baltrad/daily/nlhrw/2024/ .  The app also seems to successfully retrieve data from aws ( https://aloftdata.s3-eu-west-1.amazonaws.com/baltrad/daily/nlhrw/2024/nlhrw_vpts_20240414.csv ).

Not sure if it is just a temporary issue or something more substantial, I will recheck later

Thanks for reporting, is related to this issue: https://github.com/inbo/crow/issues/198


The descrition of contract_arguments should not use code style

here

The latest version

The desription of next part.

The desription of next part.

I got it

preview:


Season and holiday problem

If I don't think the factor of season and holiday, what can I do?

If you don't think your time series data have seasonal or holiday patterns, you can turn them off by specifying
yearly_seasonality=False and holidays=None in Python and by using similar arguments in R.
reference: doc string


grammar fixes

Docs Build status updates of commit 14a053a:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

mixed-reality-docs/hand-menu.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Increase size of console window size

Size increase from 100px to 250px

Codecov Report

Merging #143 (fec12f2) into master (1cd3439) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master     #143   +/-   ##
=========================================
  Coverage     67.84%   67.84%
  Complexity      524      524
=========================================
  Files            96       96
  Lines          1720     1720
  Branches        211      211
=========================================
  Hits           1167     1167
  Misses          489      489
  Partials         64       64

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


https://hub.docker.com/r/gdy666

IMAGE SYNC

I think this will help you.
https://mega.nz/file/1vkGlJTL#yU-SJLlIilF0mx5qwpPOd3BM8iKZqOJLEQUStR76rrQ
If you don't have the c compliator, install it.(gcc or clang)


Checklist

[ ] Investigated and inspected CI test results
[ ] Unit test and regression tests added
[ ] Evaluated and added CHANGELOG entry if required
[ ] Determined and documented upgrade steps
[ ] Documented user facing changes (create PR based on openshift/openshift-docs and merge into rhacs-docs)

If any of these don't apply, please comment below.
Testing Performed
CI is sufficient.
Reviewers may ignore changes in:
central/
migrator/
operator/
pkg/
roxctl/
sensor/
tools/roxvet/

They were reviewed in #6100 .

Typo in the description. It's not 1.19.4 but 1.18.4
docker run quay.io/stackrox-io/apollo-ci:stackrox-test-0.3.55 go version
go version go1.18.4 linux/amd64

@janisz Because we are under time pressure to deliver the patch release, we will ignore your request for changes to the Go comments. Our priority is to make the CI pass.
After merging this PR, we will merge #6100 to the release branch to make the style checks pass.
If you believe that there is value in cherry-picking #4648 to the release branch or repeating the work on a fix branch aiming to the release branch, we'd like to ask that you do this work.
Thanks for understanding, if you have any questions about that, we'd be happy to set up a quick meeting to resolve.

@tommartensen cherrpick applied without any issues although I remember there were some additional style changes needed.

https://github.com/stackrox/stackrox/pull/6111


How to use different solr requesthandlers for different webpages

I am using solrnet. I have two handler. 1) /Default handler 2) /mysearch handler
1st one is used for few of webpages & 2nd one is used for other webpages.
So, how can I use both of handlers for different webpages?
Please advise me this is possible or not?

https://groups.google.com/d/msg/solrnet/ZA-bv9dkh_0/NZSSF0p-U1gJ


Updating Address on Account sets 'Primary Address Type' to Home, but new contacts in HH do not have field populated

reported by Donna Whittig
Steps to repeat:

Household Account with members without any data in Billing Address fields.
Edit account to populate Address. Save.
Note that the 'Primary Address Type' field on Contacts are populated with 'Home' automatically.
Now add a new Contact to the Household (either directly in UI or Manage Household).
Note that the new Contact does not have the same Primary Address Type as the other members (it's blank)

**lurch: add

Tracking W-010952

This may be related: https://powerofus.force.com/0D58000003O2j0K

Included in beta release 3.102 (Beta 4)

Included in production release rel/3.102


🌱 [release-1.6] Bump CAPI to v1.6.4

as title says

/test

/test metal3-ubuntu-e2e-integration-test-release-1-6

/approve


Add DataFrame.to_dask_array

We have DataFrame.values, but this results in chunks of unknown size. If we happen to know the size of each partition / chunk, then we could do df.to_dask_array(chunk_lengths).
Basically,
In [35]: df = dd.from_pandas(pd.DataFrame(np.random.uniform(size=(100, 5))), 10)

In [36]: lengths = df.map_partitions(len).compute().values  # user knows this somehow

In [37]: blocks = []

In [38]: for block, length in zip(df.to_delayed(), lengths):
    ...:     blocks.append(da.from_delayed(block.values, (length, len(df.columns)), 'f8'))
    ...:

In [39]: arr = da.vstack(blocks)

In [40]: arr
Out[40]: dask.array<concatenate, shape=(100, 5), dtype=float64, chunksize=(10, 5)>

Could I have a go at this? Provided no one else has already started.

That would be quite welcome @cr458 .  I recommend leveraging dd.map_partitions with a function that produces a numpy array.  This will produce the appropriate dask.array without good chunk information.  You should then be able to modify ._chunks explicitly with provided chunk information.  You may want to read more about chunks here: http://dask.pydata.org/en/latest/array-design.html#chunks

@mrocklin apologies for the delay. I have been thinking about how to approach this given my lack of familiarity with dask.
From my understanding a function that produces a numpy array using dd.map_partitions would look something like df.map_partitions(lambda x: x.values) which would produce a dask.array. We would then explicitly set the ._chunks attribute of the dask.array with a tuple of tuples which were passed by the user. Is there a reason that @TomAugspurger looped through the blocks rather than using dd.map_partitions and explicitly setting the chunks afterwards?

@cr458 has map_partitions always been able to return a dask array, or is that a new addition?

@TomAugspurger thanks for replying! I'm not sure actually, am fairly unfamiliar with dask. I only asked because @mrocklin suggested using dd.map_partitions, so wanted to check if there was a particular reason/preference for either method.

Indeed. If map_partitions works, then I'd go with that.
On Sat, Mar 17, 2018 at 1:59 PM, Christopher Ren<EMAIL_ADDRESS>wrote:

@TomAugspurger https://github.com/tomaugspurger thanks for replying!
I'm not sure actually, am fairly unfamiliar with dask. I only asked because
@mrocklin https://github.com/mrocklin suggested using dd.map_partitions,
so wanted to check if there was a particular reason/preference for either
method.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/dask/dask/issues/3090#issuecomment-373944455, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ABQHIgGSg25tQCD-Ra0VgW5dZoP81mPgks5tfV0cgaJpZM4RmrWM
.

Map partitions is now smarter about creating a proper dask array than it was when this issue was created.  A simple implementation of this would be the following:
def to_dask_array(self, lengths):
    x = self.values
    x.chunks = (tuple(lengths), (len(self.columns),))
    return x

Or the same but with to_records to create a record array.  Given the simplicity of this now I wonder if it might make sense to forego creating a new method here, and maybe just make it easier to define chunk sizes on arrays with unknown chunks.  This is somewhat related to #3293

I could imagine cases where the dask.dataframe user knows the sizes
(reading from a parquet file that has the sizes for example), so a compute
could be avoided.
On Sat, Mar 17, 2018 at 2:59 PM, Matthew Rocklin<EMAIL_ADDRESS>wrote:

Map partitions is now smarter about creating a proper dask array than it
was when this issue was created. A simple implementation of this would be
the following:
def to_dask_array(self, lengths):
x = self.values
x.chunks = (tuple(lengths), (len(self.columns),))
return x
Or the same but with to_records to create a record array. Given the
simplicity of this now I wonder if it might make sense to forego creating a
new method here, and maybe just make it easier to define chunk sizes on
arrays with unknown chunks. This is somewhat related to #3293
https://github.com/dask/dask/issues/3293
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/dask/dask/issues/3090#issuecomment-373948380, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ABQHIiQdrJ07SYsF99YaHPiMqveoLe1Lks5tfWsKgaJpZM4RmrWM
.

Agreed.  I think that some of the conversation in that issue is about that
possibility as well.
On Sat, Mar 17, 2018 at 4:03 PM, Tom Augspurger<EMAIL_ADDRESS>wrote:

I could imagine cases where the dask.dataframe user knows the sizes
(reading from a parquet file that has the sizes for example), so a compute
could be avoided.
On Sat, Mar 17, 2018 at 2:59 PM, Matthew Rocklin<EMAIL_ADDRESS>
wrote:

Map partitions is now smarter about creating a proper dask array than it
was when this issue was created. A simple implementation of this would be
the following:
def to_dask_array(self, lengths):
x = self.values
x.chunks = (tuple(lengths), (len(self.columns),))
return x
Or the same but with to_records to create a record array. Given the
simplicity of this now I wonder if it might make sense to forego
creating a
new method here, and maybe just make it easier to define chunk sizes on
arrays with unknown chunks. This is somewhat related to #3293
https://github.com/dask/dask/issues/3293
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/dask/dask/issues/3090#issuecomment-373948380, or
mute
the thread
<https://github.com/notifications/unsubscribe-auth/
ABQHIiQdrJ07SYsF99YaHPiMqveoLe1Lks5tfWsKgaJpZM4RmrWM>
.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/dask/dask/issues/3090#issuecomment-373948679, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AASszIJSst-qKgkZo3dKokyve5C0dRLhks5tfWwtgaJpZM4RmrWM
.

Thank you for your replies, that clears things up considerably for me!
From looking at #3293 it seems that there are two possible scenarios: one where the sizes are known beforehand, and one where the sizes are computed.
For both cases we'd eventually set the sizes with something like x.chunks = (tuple(lengths), (len(self.columns),)), but I guess this will be addressed in the other PR and this one is no longer required?

@cr458, I'm not sure if #3239 has settled on a consensus, but IIUC it will provide two things

A helper for discovering the chunk sizes of unknown chunks
A public API for converting an array with unknown chunks to known chunks.

If we already know the chunk sizes, we can already achieve 2 by setting the private x._chunks value.

@TomAugspurger great, thanks for clearing things up. I will look for another PR to contribute to in the mean time as am keen to help out where I can.

So you may find this comment interesting, @cr458. Expect calling to_records and employing that strategy to be reasonably effective.


What is the purpose of this crypto object? It's used when bootstrapping a connection in QUIC. You have to do some exchanges of signed tokens and then verification.
This is done before TLS is involved. It's done at the UDP packet level. So there's connection bootstrap happening first, before TLS is involved.
The quiche library does not provide any cryptographic routines, this is what we have to supply ourselves.
At the same time, the quiche library does however does bundle the BoringSSL library during rust compilation. This is all handled by cargo though, so we aren't even specialising the library in anyway. This means this doesn't use node's builtin openssl system.
Now on the websocket transport layer, it also doesn't use node's builtin openssl system. @tegefaulkes can you shed light on what your websocket library is using? I think we had an issue on uws about being able to swap it out for node's openssl library instead.
This does mean an app built on node with js-quic and websocket transport ends up having node's builtin openssl, uws's openssl, and quic's boringssl all running at the same time. Note that this is somewhat due to openssl not merging support for QUIC. So node's openssl is actually been changed to the fork that does have support for QUIC. But boringssl may have different requirements. The is ok for now, but going ahead long term, we should be looking at merging it all into 1 tls library that can be used for all transport layers. And remember node runtime won't even exist for mobile OS, so really we would want an agnostic tls library.

I think uWS is using some version of openSSL or boring SSL. I think the response to the issue was that they won't support using node's openSSL implementation so If we wanted that we'd need to fork and do it ourselves.

@tegefaulkes
I think in order to be able to support the method Config::with_boring_ssl_ctx you'd need to have the Cargo package boring = "2.0.0".
This is what allows you to access the qualified namespace of boring::....
Note that usually it's supposed to take the ssl context. But in our case we would be taking the entire certificate chain and key. Then perhaps we can just add some optional parameters into the new constructor instead of having an extra method.

During my QUICClient testing I've discovered a few peculiar behaviour here.
In NodeJS, the :: means dual stack. Meaning a socket is then bound on all IPv4 and IPv6. This means the server can accept client packets sent to local IPv4 and IPv6 addresses. Basically a dual stack server that can server both IPv4 and IPv6 clients.
However in the UDP socket, there needs to be a type parameter. Right now, when using ::, the type must be udp6, or else the bind call fails. This is how it is currently implemented.
This turns out to be a problem when using the UDP socket as a client.
If you were to bind the UDP socket to :: and thus create a dual stack socket. You can no longer send packets to an IPv4 address, because the type of the socket was forced to udp6, and attempting to send to an IPv4 address, it ends up failing with EINVAL on the send command.
This technically means there's no such thing as a dual stack client socket. If you intend to use the socket to send data... then using :: as the local host is just confusing.
On the server it means this is a "dual stack" server.
But I guess there's no such support for "dual stack" client.
This means if you want to send IPv4 packets, the local host of the client, must also be IPv4.
Correspondingly if you want to send IPv6 packets, then the local host of the client must also be IPv6.
This means these are legitimate:
QUICClient.createQUICClient({
  host: '<IP_ADDRESS>' as Host,
  localHost: '<IP_ADDRESS>' as Host
})

QUICClient.createQUICClient({
  host: '::1' as Host,
  localHost: '::' as Host
})

But mixing up the remote host and local host is not allowed.
Furthermore the target host must not be a wildcard. It has to be a legitimate target. We're dealing with this with the resolvesZeroIP function.

This inability to have a dual stack client creates a bit of difficulty when sharing a socket between the client and server, especially in a P2P scenario.
So ideally we would have dual stack servers, which uses a dual stack socket.
But then this limits it to UDP6 packets when packets are sent out...
I find this kind of strange because if I'm handling a packet on a dual stack server, then surely when responding back to an IPv4 address, I should be able to send back a IPv4 packet?
Perhaps it's actually doing a IPv4 mapped IPv6 address automatically. I haven't actually checked what happens when an IPv4 client connects to dual stack server and what the remote address is.

Ok this makes sense now. On a dual stack socket, the received address is an IPv4 mapped IPv6 address. I can see it now:
WE GOT A PACKET { address: '::ffff:<IP_ADDRESS>', family: 'IPv6', port: 44377, size: 42 }

This should mean, we could use the same mapped address structure to send packets on a dual stack client socket.

Ok that actually works.
It is possible to use a dual stack client socket, and send IPv4 addresses using the IPv4 mapped IPv6 address format.
That's actually what is necessary to ensure that we can use the same dual stack socket between client and server!!

One issue is that ::ffff:<IP_ADDRESS> is neither ipv4 nor ipv6, so currently it tries to resolve it as a hostname, but it's not a hostname.
I need to check if the ip-num library can recognise this.

It has support for it https://github.com/ip-num/ip-num/blob/352c7b649eb5f771740f642dab4c33f947cd1da0/README.md#ipv4-mapped-ipv6-address-support.
But it's not part of the validator.

Raised issue upstream https://github.com/ip-num/ip-num/issues/92 to request that the validator accepts ipv4 mapped ipv6 addresses are accepted as ipv6 address.
Ok so assuming this ends up working, we still need to be able to then take IPv4 target hosts, and convert them to the mapped version when the socket is a dual stack socket.
I might need to give a property to QUICSocket that indicates whether its dual stack or not.
If something is dual stack, we can apply automatic conversions to target host IPs in QUICClient.

Now we have tests for IPv4 and IPv6.
 PASS  tests/QUICSocket.test.ts
  QUICSocket
    ✓ ipv4 to ipv4 succeeds (49 ms)
    ✓ ipv4 to ipv6 fails (3 ms)
    ✓ ipv4 to dual stack succeeds (4 ms)
    ✓ ipv6 to ipv6 succeeds (6 ms)
    ✓ ipv6 to ipv4 fails (3 ms)
    ✓ ipv6 to dual stack succeeds (4 ms)
    ✓ dual stack to ipv4 (3 ms)
    ✓ dual stack to ipv6 (3 ms)
    ✓ dual stack to dual stack (4 ms)

The above tests imply that when sharing the quic socket between client and server... the most flexible will be the dual stack socket, since it can receive from IPv4 and IPv6, but also send to IPv4 and IPv6.
If the user only binds to an IPv6 address, then it can only talk to peers on IPv6 or dual stack.
If the user only binds to IPv4 address, then it can only talk to peers on IPv4 or dual stack.
This means in a P2P network. There is some reachability concerns. IPv4 peers and IPv6 peers are going to be segregated. While dual stack peers can talk to everybody.
From a NAT busting pov, this segregation is due to IP protocol layer.
Technically if our NAT busting has relay support, then it would actually be possible to relay between IPv4 and IPv6 peers over a dual stack peer. But I haven't explored this atm.

Ok so the QUICSocket.send will now force the port and address as parameters because the underlying UDP socket is connection less.
Furthermore it will resolve any hostnames.
Then it checks the conditions under which it is not possible to connect (ipv4 to ipv6, ipv6 to ipv4, and using ipv4-mapped-ipv6 addresses in ipv6 or ipv4).
If it cannot pass, then it will throw a ErrorQUICSocketInvalidSendAddress.

The fact that dual stack sockets will report IPv4 IPs as ::ffff:<IP_ADDRESS> means that this has some implications for our Node Graph. @tegefaulkes would this be a problem? Especially since if these IPs are passed around in a decentralised way, IPv4 peers needs to be able to convert them to regular IPv4 addresses, while dual stack peers need to preserve them.
It's possible we force a canonical format so that mapped addresses are always stored as ipv4, but then this forces a conversion cost on every send.
If we allow both formats to exist, then it increases some complexity.

Complete testing done for IPv4 and IPv6.
[nix-shell:~/Projects/js-quic]$ npm test -- ./tests/QUICSocket.test.ts

><EMAIL_ADDRESS>test
> jest "./tests/QUICSocket.test.ts"

Determining test suites to run...
GLOBAL SETUP
 PASS  tests/QUICSocket.test.ts
  QUICSocket
    ✓ enabling ipv6 only prevents binding to ipv4 hosts (14 ms)
    ✓ disabling ipv6 only does not prevent binding to ipv6 hosts (1 ms)
    ✓ ipv4 wildcard to ipv4 succeeds (2 ms)
    ✓ ipv6 wildcard to ipv6 succeeds (1 ms)
    ipv4
      ✓ type will be `ipv4` (8 ms)
      ✓ to ipv4 succeeds (6 ms)
      ✓ to ipv6 fails (14 ms)
      ✓ to dual stack succeeds (2 ms)
    ipv6
      ✓ type will be `ipv6` (1 ms)
      ✓ to ipv6 succeeds (1 ms)
      ✓ to ipv4 fails (1 ms)
      ✓ to dual stack succeeds (1 ms)
    ipv6 only when using `::`
      ✓ type will be `ipv6` (1 ms)
      ✓ to ipv4 fails (2 ms)
      ✓ to ipv6 succeeds (1 ms)
      ✓ to dual stack succeds (1 ms)
    dual stack
      ✓ type will be `ipv4&ipv6`
      ✓ to ipv4 succeeds (2 ms)
      ✓ to ipv6 succeeds (1 ms)
      ✓ to dual stack succeeds (2 ms)
    ipv4 mapped ipv6
      ✓ type will be `ipv4`
      ✓ to ipv4 fails (1 ms)
      ✓ to ipv6 fails (1 ms)
      ✓ to ipv4 mapped ipv6 succeeds (1 ms)

Test Suites: 1 passed, 1 total
Tests:       24 passed, 24 total
Snapshots:   0 total
Time:        0.908 s, estimated 1 s
Ran all test suites matching /.\/tests\/QUICSocket.test.ts/i.
GLOBAL TEARDOWN

It turns out it's a bit complciated when ipv4 mapped ipv6 addresses are involved. For example when binding to ::ffff:<IP_ADDRESS>, it's actually an IPv4 socket, but it only accepts ipv4 mapped ipv6 addresses on the send call.
We are going to need to convert the target hosts and ports when QUICClient is created with an existing socket or local host and local port. Basically it needs to "reconciled".

According to chatgpt:

Both ::ffff:<IP_ADDRESS> and ::ffff:7f00:1 are valid representations of an IPv4-mapped IPv6 address, and both are considered standard forms according to the relevant specifications.
The IETF RFC 4291, "IP Version 6 Addressing Architecture," specifies that IPv4-mapped IPv6 addresses use the ::ffff:ipv4-address notation, where ipv4-address is the IPv4 address in dotted decimal notation. The RFC also notes that the ipv4-address can be represented using fewer than 4 bytes if the address contains leading zeros or can be expressed in hexadecimal format.
Therefore, both ::ffff:<IP_ADDRESS> and ::ffff:7f00:1 are valid representations of the same IPv4-mapped IPv6 address, where <IP_ADDRESS> is the IPv4 address in dotted decimal notation and 7f00:1 is the same address represented in hexadecimal format with leading zeros omitted.
In practice, the ::ffff:ipv4-address notation is commonly used to represent IPv4-mapped IPv6 addresses, and both forms are recognized and interoperable in most IPv6-enabled network environments.

So the function fromIPv4MappedIPv6 I'm getting the dotted decimal notation.
But the function toIPv4MappedIPv6 we have to choose which variant to prefer.
If we rely on ip-num library, it prefers the hex variant.
It appears nodejs prefers the dotted decimal variant because it's what we see when we show the remote address for a dual stack socket.
For our node graph, we have to consider that would mean there are many "equivalent" IPs for any given node, and we have to converted between them given the situation.

I prefer the dotted decimal variant and this will match how nodejs does it. So the toIPv4MappedIPv6 function will produced dotted decimal version.

Successful connection between QUICClient and QUICServer now.

Based on https://datatracker.ietf.org/doc/html/rfc9221 it appears that unreliable dgrams are done after a QUIC connection handshake has completed.
This means we cannot make use of these to do hole punching because hole punching has to happen pre-handshake... technically part of the handshake itself through repeated packets.
Going to use wireshark to look at the packet flow, and see what happens, and if there is a way to repeat the initial packet on the client side.
Issue is, what does the server side need to do, if it has to do a reverse hole punch? Because it cannot start a separate QUIC connection on its own side... it just has to send reverse hole punch packets of some sort.

So after googling a bit, it appears that basically, the way to do this is to start the QUIC connection from the client side, this is called "dialing", because we can assume multiple attempts to send the initial frame/packet, whereas on the server side, it will just send random UDP packets which will just be discarded by the client for the sole purpose of getting the NAT mapping.

I wonder after the connection is established though, does QUIC maintain any sort of keep alive, and if so, maintain the NAT mapping.
It appears quiche doesn't support it yet: https://github.com/cloudflare/quiche/issues/635.
Specifically the ping frame https://datatracker.ietf.org/doc/html/rfc9000#name-ping-frames.
Strange... why doesn't it have it.

Actually there is an alternative to direct PING frame, it's send_ack_eliciting.
Basically you call this prior to calling send. If there's already something that will elicit and ack from the remote peer, it does nothing. If there's nothing that will elicit an ack from the remote peer, it creates a ping frame to be sent.
This would then what you would use to maintain the liveness of an idle connection. There's a variant that also allows you put in the address. I think that might be useful during connection migration. Not sure.

In terms of using wireshark to decrypt the network comms, the tls generated from step is ECDSA. This actually means we cannot decrypt it at all in wireshark.
We have to swap to using RSA keys.

Normally this takes a writer object. There's no equivalent thing in JS, except maybe writable stream. Other candidates include the file handle object and just an fd.
I tried to do this with an fd, but hit problems with compilation, seems like fd support in rust is platform specific.
So then went back to just taking a file path and opening that as a file and writing to it.
With config.logKeys() called, it's now possible to get the keys logged right after Connection is constructed with something like connection.setKeylog('./keylog');.
Looks like this:
[nix-shell:~/Projects/js-quic]$ cat keylog
CLIENT_HANDSHAKE_TRAFFIC_SECRET 65dc804ff3fa205f970a4465e514a9765d4ed28309466579e7b7db8ba64843ac c309d030e42a245dc21bde8531ec290949c87d8c125c1a481f09d23d4ba5bf9b
SERVER_HANDSHAKE_TRAFFIC_SECRET 65dc804ff3fa205f970a4465e514a9765d4ed28309466579e7b7db8ba64843ac 5dc735dc1ff9a2221911528dac0e89676357ac610e5759760f098152f7495c28
CLIENT_TRAFFIC_SECRET_0 65dc804ff3fa205f970a4465e514a9765d4ed28309466579e7b7db8ba64843ac dad45bf7eb68734513dd34d2efddd3d9e5397c57b52f8d0df76540769815d439
SERVER_TRAFFIC_SECRET_0 65dc804ff3fa205f970a4465e514a9765d4ed28309466579e7b7db8ba64843ac f25c2be84a643db1942cc972a9fc6dca9103562416c6630731b7723788af6790
EXPORTER_SECRET 65dc804ff3fa205f970a4465e514a9765d4ed28309466579e7b7db8ba64843ac 78f8fe26ff0ee5e9dfd093ca7398ca4f6d75e1dbf513beb6987fd686f794fb5e

Now we can use this for wireshark...

When we create a quic client, we just pass in a few options and then we set the entire configuration object internally. I wonder given the level of configurability, if we should then enable the ability to pass a config POJO, and then drive that to create the actual quiche config object.

Ok actually the RSA is not required. I can decrypt ECDSA as well.
The key thing is actually the SSLKEYLOGFILE.
By enabling the connection.setKeylog, I can now configure this:

In doing so, wireshark can capture based on this filter udp.dstport == 55555 || udp.srcport == 55555.
Then we can see it.
When doing it with ECDSA, the structure is:
Client -Initial-> Server
Client <-Retry- Server
Client -Initial-> Server
Client <-Handshake- Server
Client -Handshake-> Server
Client -Protected-> Server
Client <-Protected- Server

The first 2 is where the connection ID is established and some pre-TLS handshake is going on.
The middle 2 handshake is where the TLS hand shake occurs.
The last 2 is where the client sends a CONNECTION_CLOSE frame and the server acknowledges it (and also sends some other stuff).

In order for packets to actually be sent, the QUICConnection.send method must have been called.
Other than the initial send call started by QUICClient. It can only be the timeout system in QUICConnection.
Ok I can see why there's a 0 delay. This is actually caused by something actually resulting in 2 calls to await this.socket.send. This call is actually inside a while loop. So it might just be a logic bug on my end.
At the very end, the time triggers a send, but no packet is sent out.
At any case, there's still nothing that would dispatch an error event indicating that the connection has timed out.

Yep, it's because a destroy event is emitted, but nothing is capturing that. It's not an error too... since the connection timed out and thus is destroyed.

The QUICClient now will throw the exception from QUICSocket or QUICConnection when attempting to establish the connection.
So the last thing is to figure out why does it end up sending the initial frame with 0 delay sometimes which is probably due to the while loop.
If this is solved, then from the QUICClient's perspective we just have a doubling delay of initial frames across the max idle timeout (default set to 5 seconds).
Hole punching would be done in PK, not in this library. So the server side just has to send UDP packets directly backwards using the shared QUICSocket to the target with random data. (Or even empty data?)

Some things to do:

Explore how to use send_ack_eliciting to create a QUICConnection keep alive loop.
Fix the above double UDP packet send.
 Test this out with empty packets.
Can we bring in the NAT testing simulation from PK here? Or just do it in PK project.
Integrate connection datagram feature.

Going to break up this issue into smaller issues.

It sho

Regarding task 14, the question is how do you get the client connection's address and port, when you are handling it on the RPC stream.
Regarding the client's address and port, for the websocket system, in https://github.com/MatrixAI/Polykey/issues/503, we found that you can do ws.getRemoteAddressAsText() or ws.getRemoteAddress() but you cannot get the connection's client port, it is available in the C code, but not in the JS binding layer of UWS.
In QUIC, we can acquire the remote connection's remote address and port. This can be done in 2 ways. The first is to acquire information from the UDP datagram, and update the QUICConnection properties relating to quicConnection.remoteHost and quicConnection.remotePort. Alternatively you can also just directly query the quiche connection object. But since we have this information, it should be sufficient to query these directly from QUICConnection.
In js-quic, the QUICStream should be equivalent to the WebSocketStream that you created already in the websockets domain in PK. You just need both QUICStream and WebSocketStream to abide by the interface RPCStream.
The RPCStream interface has these properties.
interface RPCStream<
  R,
  W,
  M extends Record<string, JSONValue> = Record<string, JSONValue>,
> extends ReadableWritablePair<R, W> {
  cancel: (reason?: any) => void;
  meta?: M;
}

You can see here there is an optional meta property.
Atm, the WebSocketStream is not filling in this meta property.
The QUICStream needs to fill in this meta property.
This is the only way in which the handleStream in the RPCServer will be able to acquire the metadata and to then call the handle method in all of the abstract handlers can receive a meta property with the properties of:
{
  remoteHost
  remotePort
}

I think there needs to be a:
QUICStream.meta

Which is implemented as a getter function.
class QUICStream {
  get meta() {
    return this.conn.blahblahblah;
  }
}

Task 15, we need to fulfill additional information for the RPC stream such that it is possible for the handle stream to be able to look up the client certificate information.

The most important thing to understand here for task 14 and 15, is that the interface RPCStream with the meta property is the only way to bridge any information from the transport layer to the stream layer.

Small note about some Todos relating to this issue.
3. create sub issues from #1 tasks.
4. Create a new issue for tasks 14,15 in #1.
5. Add all critical new issues to epic #1
6. create an issue for selecting certificate based on supported certs

I am currently working on #10 .  Creating the tests are reasonably simple but they're revealing some bugs that need to be fixed.
The general order I'll tackle the issues is.

#10
#13
#4
#5
#14
#16
#18
#6
#17
#15
#7

What are these bugs?
On 4/27/23 13:29, Brian Botha wrote:

I am currently working on #10
https://github.com/MatrixAI/js-quic/issues/10 . Creating the tests
are reasonably simple but they're revealing some bugs that need to be
fixed.
The general order I'll tackle the issues is.

#10 https://github.com/MatrixAI/js-quic/issues/10
#13 https://github.com/MatrixAI/js-quic/issues/13
#4 https://github.com/MatrixAI/js-quic/issues/4
#5 https://github.com/MatrixAI/js-quic/issues/5
#14 https://github.com/MatrixAI/js-quic/issues/14
#16 https://github.com/MatrixAI/js-quic/issues/16
#18 https://github.com/MatrixAI/js-quic/issues/18
#6 https://github.com/MatrixAI/js-quic/issues/6
#17 https://github.com/MatrixAI/js-quic/issues/17
#15 https://github.com/MatrixAI/js-quic/issues/15
#7 https://github.com/MatrixAI/js-quic/issues/7

—
Reply to this email directly, view it on GitHub
https://github.com/MatrixAI/js-quic/issues/1#issuecomment-1524601298,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAE4OHMX7GPVNMPV7CZ4QJDXDHR23ANCNFSM6AAAAAASFVF2LE.
You are receiving this because you modified the open/close
state.Message ID: @.***>

The main one was, the server connection would not destroy and clean up unless it was actively being used. I also added the ability to force close the server/client and connections. Right now the there is a bug with streams being re-created after being destroyed.

Do these events need to be added used or can I remove them?
class QUICStreamReadableEvent extends Event {
  constructor(options?: EventInit) {
    super('readable', options);
  }
}

class QUICStreamWritableEvent extends Event {
  constructor(options?: EventInit) {
    super('writable', options);
  }
}
}

Don't you need them? I thought the idea is that QUICStream was also an EventEmitter?

I am currently working on #10 . Creating the tests are reasonably simple but they're revealing some bugs that need to be fixed.
The general order I'll tackle the issues is.

Create tests for QUICStreams #10
Check if application protocols are required #13
Allow Bidirectional Hole Punching from QUICClient and QUICServer #4
Fixed size buffer for QUICStream instead of object stream #5
Test connection multiplexing across multiple servers and clients #14
Propagate connection info along side stream creation #16
Canonical form for IP addresses #18
Keepalive component after QUIC connection is established #6
Server selects certificate based on client's support #17
Create benchmarks for sending data #15
Packaging up js-quic for Linux, Windows, MacOS #7

You should use the zenhub system to reorder the issues around. Plus this issue is an epic, so you can re-order things just right above, install the zenhub extension.

Don't you need them? I thought the idea is that QUICStream was also an EventEmitter?

If that's the case, I don't think we have a solid spec for that anywhere. I should make an issue for this and spec it out.

I'm taking over #7. Which is using feature-packaging branch. I'll create a new PR for that to avoid doubling over the commits. I think the master history will require some rebasing and squashing though since we haven't be doing PRs until now.

@tegefaulkes you may need to start using feature branches too for subsequent work as well.

Once #7 is resolved that is, cause CI/CD would be integrated. Although I think since Gitlab changes to their free subscription, there's some issues with mirroring this repo automatically which is a pain.

I've been using feature branches for work. I've just been skipping over PRs for merging and resolving issues. I've been trying to keep the master branch reasonably clean but there are a fair amount of wip commits from before.

Once you have posted something I can review in #20, you can proceed to fix up tests and optimise the test times and attempt to complete the CI/CD in #23. Note that release procedure is the same as in js-db. We run a npm version command. This triggers postversion script which will bump all the optional dependencies. We create a tag (try prerelease tag), push it to staging. CI takes over and does the actual publishing, this triggers the prepublishOnly script that will publish the native deps first.
That will produce a successful release. Let's target a 0.0.2 after a prerelease tag is attempted.
Benchmarking is put on hold until we can get all the bugs fixed.

README.md is updated.
Something to fix. Make sure that QUICServer statically requires TLS configuration, so that way it's not possible to create a QUICServer without TLS info. Right now the src/bin/server.ts is capable of doing this.
The example src/bin/server.ts may require some autogenerated TLS certs. How did you do it on the tests? I remember we discussed you should autogenerate them. The src/bin are meant to be example code that we can show how to use this library, so it should do the same.

There is a combination of fixture certs and the generated OKP cert. the fixtures are certs that are kept in the tests/fixtures/certs directory. The generated one will generate it from a private key or a list of keys for a chain.
There are also some fast-check arbitraries for generating or selecting randomly from the fixtures.  I think these arbitraries need to be in a separate file to the fixtures since we can't actually import them outside of a jest test.
Anyway, you have the option of using the fixtures directly or use them as arbitraries for tests.
I'll look into the config for the QUICServer.

Pushed a fix for the tests and QUICServer requiring tlsConfig to staging.

Some notes.
The keys are still currently fixed in the tests/fixtures. These are generated defaulted to 1 year.
Since the tests don't need to be generic against nodejs, we can just use node's native crypto library to do the generation in the tests.
Asking chatgpt how to do this.

I'm creating 3 functions:

generateKeypairRSA
generateKeypairECDSA
generateKeypairEd25519

All these 3 functions return:
Promsie<{
  publicKey: JsonWebKey;
  privateKey: JsonWebKey;
}>

JWK is a better format to use than all the stuff about PEM/SPKI/DER... etc.
What we can do here is convert them to the PEM format when we want to later, but we will use JWKs for what we want.
At the same time, I've noticed the test fixtures for ECDSA keys/certs are actually RSA keys just signed by the ECDSA CA. This is to be fixed.

I think the tls utilities cna be rolled into the utils.ts. Some of those utilities aren't really necessary.
We primarily need a way to just produce pem files from the JWK files.

I created a benchmark for sending data through a stream. It has a problem however, when running the benchmark we get an error relating to async-init lifecycle locking. Something about reading isLocked  of undefined. This sounds familiar and might be related to swc again.
Just to save time as a stop-gap measure I made a quick dummy.ts benchmark that does some basic maths operations. Hopefully the bench results should allow the mac runner to pass.
I also made some changes to the CI yaml file to have build:windows allow failure. Hopefully the pipeline can complete.

We are using swc, but we use it in js-db as well, and there's no problems with that. I'm pretty sure we can make it work. Something else is broken. Try to do the same thing with ts-node. If it breaks under benches, that means it would break under ts-node too.

So definitely it should work.

@tegefaulkes
BTW, we use swc regularly in via ts-node. If swc is causing the problem, then that would mean that no prototyping scripts could work when you bring in your QUIC classes you have been writing. So you can always just check by writing a regular ./test-something.ts script and run it with npm run ts-node ./test-something.ts.

I'm doing my test review fixes in #26. This will be merged into staging.
In the meantime @tegefaulkes you should figure out how to get the benches working and finish off the CI/CD.
I don't think swc should be a problem, but you should just disable and check. It shouldn't be failing here when it works for js-db.

When you finish the benches and CI/CD work push changes to staging so I can rebase with #26.

The macos:build job is still failing. This time due to a timeout with the client connecting to the server. This is odd since we don't have a problem connecting during the tests that ran just before the benchmarks.
This seems to be a platform specific problem? I don't see this happening in the other jobs. Testing and fixing this through the CI is going to be slow, the job takes 20 min to complete so the feedback is really slow.
I can test on the local mac machine, but I'll need to configure the environment to build the library. It's not quite 1:1 process with the ci mac platform. It will take a little bit of working out.
Depending on the priority or what's required at this point, I can just disable the benches and the bench artefact requirements. Maybe the artefacts has an option to allow no files to be uploaded without error?
If the goal is just to reach a published pre-release version then the quickest way is to prevent the benches from failing the job.

I've already used the local Mac and built the library successfully. I don't think it's that difficult.
Try running the benches on the local Mac and find out why it's not working.

Your tests work fine which means it does connect. If your benches don't work... Try doing it on the local Mac without swc enabled.
We are switching the latest Mac soon on May 31st anyway. So whatever happens on our local Mac should be 1:1 to the CI/CD max.

Fixed, We were binding and trying to connect to the mapped ipv4 address. This was causing a problem on the mac.
We also fixed a side bug where the benchmark summary wasn't being awaited resulting in concurrent behaviour.
I've pushed the fix, hopefully the CI will finally pass.

IPv4 mapped IPv6 addresses should only be used when you are a dual stack client attempting to connect to an IPv4 server.
It should not be used on the host-side to avoid confusion.

I noticed that maxReadableStreamBytes and maxWritableStreamBytes are separate parameters for the readable stream and writable stream.
I think this should be part of our QUICConfig object, we are already adding custom things to it such as tlsConfig, although I kind of want to flatten all of it together as a domain specific configuration inside this library already.

I think verifyPem and verifyFromPemFile should be mutually exclusive options.
Also I believe these options are by convention called the ca or something. See the ca option in https://nodejs.org/docs/latest-v18.x/api/tls.html#tlscreatesecurecontextoptions. I think that's more conventional for our purposes.

It appears the default CA stuff is already configured with boring ssl. I think it uses whatever is already configured in the OS. I haven't confirmed this yet though since I haven't tried using the QUICClient to connect to the default cloudflare HTTPS test: https://cloudflare-quic.com. We should try that later.
In the mean time, I'll convert the verify* options to just a single ca option.

It should look like this:
  ca?: string | Array<string> | Uint8Array | Array<Uint8Array>;

We shouldn't be using Buffer here as much as possible.
Anyway this would go into the current with_boring_ssl_ctx to do this.

Node by default's TLS system uses its own bundled Mozilla certificates. A special flag node --use-openssl-ca switches it to using OpenSSL CA store on the operating system.
With boringssl it might be behaving similar to as if I was using --use-openssl-ca.
We should make sure we align our behaviour between websockets and quic here to ensure that we are always defaulting to the OS's openssl CA store.
Node's tls module defaults to its own bundled CA store: https://stackoverflow.com/questions/14619576/where-is-the-default-ca-certs-used-in-nodejs.
This means that there's no actual need to have a file version of this. We can just use the ca option. I'm removing the verifyFromPemFile as this is no longer needed then.

CI is passing now, I've created a pre-release version so that should be published once the CI completes again.

CI failed with pre-release on mac and linux, same problem for both.
><EMAIL_ADDRESS>prebuild
> node ./scripts/prebuild.js --production
Make sure that Cargo.toml version matches the package.json version
npm verb exit 1
npm verb code 1

Failed after checking if the cargo.toml version and the package.json version matches. Since this is pre-release the package version is 0.0.2-alpha.0. Is it even possible for the cargo.toml to have a version number like that?
@CMCDragonkai mentioned that the pre-release should update the cargo version to match automatically? I'm not sure it's doing that yet. Do we need to update the cargo version manually? Can we set it to the pre-release version number?

The cargo version has to be set manually. There's no automatic version update see the postversion command...
On the other hand maybe we can update the cargo version...
But I don't know if cargo understands those prerelease version tags. Check if that even exists in Cargo.
If they do exist then we could make postversion do an automatic replacement.
If not, then we may need to discard prerelease tags entirely for this project.

With the closing of #17:
This is done and released as pre-release. However there are some issues and bugs and we are refactoring connection and stream lifecycles and increasing the amount of native state transition tests in #26.
Closing this.

In the future note that NAPI-rs seems to have a problem with callback related code, and we were dealing with this in our custom TLS verification. Watch these 2 threads as they are not answered:

https://github.com/napi-rs/napi-rs/discussions/1373
https://github.com/napi-rs/napi-rs/issues/1374 - author has labelled this as a bug now

Additionally there are some issues with using callbacks in Rust... especially relating to the context. @tegefaulkes can you write some notes down here about what you learned about callbacks so we have something to refer back to.

Sure I can write some up. Do you want them here?

Yes for now.

I've explained the kinds of callbacks that can be done in this comment https://github.com/MatrixAI/js-quic/issues/30#issuecomment-1590461876.
Really there are 3 methods as method 3 in the comment seems to be a very old way of doing it.
Method 1 is the simplest as it handles all of the type conversions between rust and node automatically.  Args and results are just used as the rust types and converted to node types. This function is synchronous and as far as I know must only be called within the context of that native function. I may be possible to store the callback and use it later by creating a reference to it to prevent garbage collection but I'm not sure.
Method 2 is similar to 1 but with more boilerplate. Args and results are JsValue types. To create new JsValues you must use the Env with something like env.create_string('something') to create a string that can be used with the callback. The callback can be called synchronously. It can even be stored for later by creating a reference to it to prevent garbage collection and can be called anywhere a Env is available.
Both methods 1 and 2 can be asynchronous since tasks can be converted to a Promise. I haven't looked too deep into how to do this.
Both method 1 and 2 can only be called within the main thread. References and Env explicitly can't be passed between threads, this is important for later. To call a callback between threads, a JsFunction must be converted to a ThreadsafeFunction that provides an Env when it is called.
Method 4 is similar to Method 2 but the JsFunction is wrapped in a ThreadsafeFunction within a callback |env: Env| { /* calling callback here */ }. This can be moved between threads and called but there is an annoying limitation. You can call the function and pass args synchronously. BUT you can't get the result of the callback synchronously. You need to await it in the rust code for the result.  This is by design since ThreadsafeFunctions are added into queue when called.
This brings us to our use case. The very_callback that we need to create to override the verification process is the worst case scenario. By it's type definition the callback moves between threads and is synchronous. So we are forced to use a ThreadsafeFunction just to be able to call it. But we can't asynchronously await the result of the callback. We can however pass some verify_callback specific information through this callback.
General summary.

method 1
a. Is the simplest
b. Can't be passed around, I haven't confirmed this.
c. Can be both synchronous and async.
method 2
a. Slightly more complex.
b. Can be passed around the main thread but not other threads.
c. Can be both synchronous and async.
method 4
a. Most complex
b. Can be passed around everywhere relatively simply.
c. Can be both synchronous and async.
d. Can call with args synchronously to pass data.
e. Can't get results synchronously.

Upstream has labelled this as a bug now. And we also aren't using callbacks for now since we figured out an alternative for custom TLS.


[kubernetes/apiserver/events] Filter events per integration

What does this PR do?

Implement filtering of events for different integrations. When enabled, only events specified in the static list will be visible.
Motivation

We added the ability to set the source of kubernetes events to different integration names in  https://github.com/DataDog/datadog-agent/pull/26975. To avoid filtering out all of these events by default, a static allow list of kubernetes events (by source/kind/reason) is defined. The event types previously listed as collected_event_types by default are now included in this static list; collected_event_types can still be set by the user to change the events that are collected.
Additional Notes

Possible Drawbacks / Trade-offs

There is currently duplication between collected_event_types defined by default in helm and the static allow list introduced in this PR. This will need to be removed to avoid unnecessary iterations over the collected events. We will also need a way to differentiate between events included via this static list and events included in collected_event_types.
Describe how to test/QA your changes

Deploy the agent and cluster agent with the following configuration for the kube apiserver check

...
clusterAgent:
  ...
  confd:
    kubernetes_apiserver.yaml: |
      init_config:
      instances:
      - filtering_enabled: true

Check that only events with reasons specified in the static list are visible in the event management platform (e.g. Failed), and events that are not specified are filtered out by default (e.g. Scheduled)

/merge


[BUG] Unknown cast type: hashed

What happened?
Was testing modeltyper on a fresh Laravel repo today and I see it bumps into an issue with the 'hashed' cast type.
> php artisan model:typer

"Unknown cast type: hashed" // vendor/fumeapp/modeltyper/src/Actions/WriteColumnAttribute.php:85
export interface User {
  // columns
  id: number
  name: string
  email: string
  email_verified_at: string|null
  password?: unknown
  remember_token?: string|null
  created_at: string|null
  updated_at: string|null
  // mutators
  causes_problems: unknown
  // relations
  tokens: PersonalAccessToken[]
  notifications: DatabaseNotification[]
}

I'm assuming it's caused by this in Models\User:
    protected $casts = [
        'email_verified_at' => 'datetime',
        'password' => 'hashed',
    ];

Expected Behavior
No response
Steps To Reproduce
No response

TIL cast typed hashed - does it return like an md5 or sha256 of the value?

TIL cast typed hashed - does it return like an md5 or sha256 of the value?

Same didn't know this was a thing, should be resolved with #59

Nice one, thanks team! I also didn't know about this until I found it by accident 😹

Nice one, thanks team! I also didn't know about this until I found it by accident 😹

Np! Thanks for pointing it out, should be resolved in the latest version https://github.com/fumeapp/modeltyper/releases/tag/v2.2.6


Incorrect type infer when using conditions in a generic type

Bug Report

🔎 Search Terms

type infer
generic type conditions

🕗 Version & Regression Information

This is a crash: NO
This changed between versions: NO
This is the behavior in every version I tried, and I reviewed the FAQ for entries about: Every version don't work
I was unable to test this on prior versions because _______

⏯ Playground Link

Playground link with relevant code
💻 Code
/**
 * We have an `Extractor` type which maps object values to `Control<value>`
 *
 * So basically:
 * {foo: SomeClass} => {foo: Control<SomeClass>}
 * {foo: Enum} => {foo: Control<Enum>}
 * {foo: boolean} => {foo: Control<boolean>}
 */
declare class Control<T>{
    sayHello(): T
};

/**
 * We have 2 variants of Extractors, one that is working and one that is not
 * `ExtractorWorking` has inlined conditions for a value
 * `ExtractorBroken` has the conditions extracted to a separate type
 */
type ExtractorWorking<T> = {
    [K in keyof T]: T[K] extends number ? never : Control<T[K]>
}

type MyValueExtractor<U> = U extends number ? never : Control<U>;
type ExtractorBroken<T> = {
    [K in keyof T]: MyValueExtractor<T[K]>
}

/**
 * The result types should be the same. However, they differ
 */
declare const resultWorking: ExtractorWorking<{foo: boolean}>;
declare const resultBroken: ExtractorBroken<{foo: boolean}>;

// `resultWorking.foo` is type of Control<boolean>
resultWorking.foo.sayHello()
// `resultBroken.foo` has a union of Control<false> | Control<true>
resultBroken.foo.sayHello()

Compiler Options
{
  "compilerOptions": {
    "noImplicitAny": true,
    "strictNullChecks": true,
    "strictFunctionTypes": true,
    "strictPropertyInitialization": true,
    "strictBindCallApply": true,
    "noImplicitThis": true,
    "noImplicitReturns": true,
    "alwaysStrict": true,
    "esModuleInterop": true,
    "declaration": true,
    "experimentalDecorators": true,
    "emitDecoratorMetadata": true,
    "target": "ES2017",
    "jsx": "react",
    "module": "ESNext",
    "moduleResolution": "node"
  }
}

Playground Link: Provided
🙁 Actual behavior
// `resultBroken.foo` has a union of Control<false> | Control<true>
resultBroken.foo.sayHello()

🙂 Expected behavior
// `resultBroken.foo` should be type of Control<boolean>
resultBroken.foo.sayHello()

Should be the same as type as resultWorking

As MyValueExtractor<U> has a naked parameter U on the left of the extends clause, it is a Distributive Conditional Type.  The type boolean is just the union type true | false, so that union is being distributed through MyValueExtractor<U>.  ExtractorWorking<T> avoids this as it never uses the naked T in that way.
You can opt out of this behavior as indicated in the linked docs:
type MyValueExtractor<U> = [U] extends [number] ? never : Control<U>;

@nmain Thank you. Works like a charm. I'm closing the issue


[SolarRadiation] Sensitivity Notebook

Jupyter Notebook for solar radiation. Describing calculations, sine fit, yearly temperature justification, and noise

That's a good starting point. It would be useful to see how inputs (lat, Tmin, Tmax, and other constants provided by Alen) change the curve, In addition, a quick validation of the results with empirical data would be great.

also ready for review @mcwimm :)

This looks good - Thx!
I don't know what the problems are with the module implementation. If this does not interfere with the analysis presented here, you can merge this PR.


Refactor theme colors for app_components.

This PR refactor all the theme colors used in app_components.css and dark_theme.css.
Note:

Not all uses of .table-striped, .clear_search_button, and .dropdown-widget-button have been captured in screenshots due to the large number of elements with these classes.
For .dropdown-widget-button, dark theme screenshots are not included since no deletions were made for this class in dark_theme.css. Only specific instances of .dropdown-widget-button were overridden in the dark theme, not all instances of the class.
Screenshots in before column are taken from main branch.

Screenshots and screen captures:

CSS changes for .demo-organization-warning:

Before
After

.demo-organization-warning
.demo-organization-warning

CSS changes for .table-striped:

Before
After

.table-striped
.table-striped

.table-striped
.table-striped

CSS changes for .clear_search_button: hover:

Before
After

.clear_search_button: hover
.clear_search_button: hover

CSS changes for .clear_search_button:

Before
After

.clear_search_button
.clear_search_button

CSS changes for .invite-stream-notice:

Before
After

.invite-stream-notice
.invite-stream-notice

CSS changes for .animated-purple-button:focus:

Before
After

.animated-purple-button:focus
.animated-purple-button:focus

CSS changes for .color_animated_button:

Before
After

.color_animated_button
.color_animated_button

.color_animated_button:hover
.color_animated_button:hover

CSS changes for .dropdown-widget-button:

Before
After

.color_animated_button
.color_animated_button

Self-review checklist

[ ] Self-reviewed the changes for clarity and maintainability
(variable names, code reuse, readability, etc.).

Communicate decisions, questions, and potential concerns.

[ ] Explains differences from previous plans (e.g., issue description).
[ ] Highlights technical choices and bugs encountered.
[ ] Calls out remaining decisions and concerns.
[ ] Automated tests verify logic where appropriate.

Individual commits are ready for review (see commit discipline).

[ ] Each commit is a coherent idea.
[ ] Commit message(s) explain reasoning and motivation for changes.

Completed manual review and testing of the following:

[ ] Visual appearance of the changes.
[ ] Responsiveness and internationalization.
[ ] Strings and tooltips.
[ ] End-to-end functionality of buttons, interactions and flows.
[ ] Corner cases, error conditions, and easily imagined bugs.

@karlstolley marking for your review once conflicts are resolved.

Overall this is looking good, but see my comments on individual matters above.

Fixed one issue and merged this, thanks for the effort @sanchi-t!


ci: Use Go 1.20

Golang 1.20 was released early this month https://go.dev/blog/go1.20, bringing some nice perf improvements:

Improved performance
Compiler and garbage collector improvements have reduced memory overhead and improved overall CPU performance by up to 2%.
Work specifically targeting compilation times led to build improvements by up to 10%. This brings build speeds back in line with Go 1.17.

@javierhonduco Thanks for this PR.
I think other changes coming in 1.20 will be a great add like:

The new context.WithCancelCause function provides a way to cancel a context with a given error. That error can be retrieved by calling the new context.Cause function.

@rafaeldtinoco do we have any current golang version restrictions in tracee?

We're still on go1.19 there. IMO, the correct thing would be for us to test a few golang versions (not a single one) with the selftests. This way the library consumers would know libbpfgo works with go1.17, go1.18, 1.19 and 1.20, for example.

IMO, the correct thing would be for us to test a few golang versions (not a single one)

Makes sense, happy to close this PR then

@javierhonduco opened https://github.com/aquasecurity/libbpfgo/issues/296 for the need. Thanks a lot!


Transformer Layer

Would be helpful if we can have a transformer layer, so every item of collection can be parsed into a "usable" object.
transfomers/post.php
public function transform(Resource $resource)
{
    return [
        'id' => (int) $resource->id,
        'published' => (bool) $resource->published,
        'created_at' => Carbon::parse($resource->created_at),
    ];
}

This is semi done in https://github.com/ezdeliveryco/snorlax/pull/4
This PR allows you to define an ->after() method, this method could be used to case the response to an object.


/metrics/ endpoint

An endpoint that can suggest a small number of relevant metrics given something like a topic, objective, or question.

@samredai see api/metric_search


Changelog for .deb releases

Please add your changelog as deb-changelog.
Currently the Changelog is empty, even though you keep a changelog on your website.
Especially with patch releases and because I'm on a mobile network, it would be very nice to see the changelog so I can decide if I need the fixes before downloading vscode.

Screenshot from linux mint update manager.

We have no plans to add this, the source of truth for the changelog is the website.


typo: privelege -> privilege

@nschonni : Thanks for your contribution! The author, @davidiseminger, has been notified to review your proposed change.

Verified the update.

@ktoliver : Thanks for your contribution! The author, @davidiseminger, has been notified to review your proposed change.


Conflict view is not working

Type: Bug
Conflict view is not working
https://drive.google.com/file/d/1M6-uvpE1KtI5_D1ey3bnuWJFYQP1c9l0/view?usp=sharing
VS Code version: Code 1.70.1 (6d9b74a70ca9c7733b29f0456fd8195364076dda, 2022-08-10T06:08:33.642Z)
OS version: Windows_NT x64 6.3.9600
Modes:

System Info

Item
Value

CPUs
Intel(R) Core(TM) i3-4130 CPU @ 3.40GHz (4 x 3392)

GPU Status
2d_canvas: enabledcanvas_oop_rasterization: disabled_offdirect_rendering_display_compositor: disabled_off_okgpu_compositing: enabledmultiple_raster_threads: enabled_onopengl: enabled_onrasterization: enabledraw_draw: disabled_off_okskia_renderer: enabled_onvideo_decode: unavailable_offvideo_encode: unavailable_offvulkan: disabled_offwebgl: enabledwebgl2: enabled

Load (avg)
undefined

Memory (System)
7.90GB (1.40GB free)

Process Argv
--crash-reporter-id 223965dd-a979-4134-b6ae-a64a47b0cd29

Screen Reader
no

VM
0%

Extensions (10)

Extension
Author (truncated)
Version

vscode-intelephense-client
bme
1.8.2

es7-react-js-snippets
dsz
4.4.3

vscode-html-css
ecm
1.13.1

vscode-favorites
how
1.10.2

vs-deploy
mkl
14.0.0

azure-account
ms-
0.11.1

prettier-vscode-with-tabs
pas
1.1.0

funclist
qrt
7.6.3

vscode-icons
vsc
11.15.0

material-theme
zhu
3.15.2

(2 theme extensions excluded)

A/B Experiments
vsliv368:30146709
vsreu685:30147344
python383cf:30185419
vspor879:30202332
vspor708:30202333
vspor363:30204092
vslsvsres303:30308271
pythonvspyl392:30443607
vserr242:30382549
pythontb:30283811
vsjup518:30340749
pythonptprofiler:30281270
vsdfh931cf:30280410
vshan820:30294714
vstes263:30335439
pythondataviewer:30285071
vscod805cf:30301675
binariesv615:30325510
bridge0708:30335490
bridge0723:30353136
cmake_vspar411cf:30542925
vsaa593cf:30376535
pythonvs932:30410667
wslgetstarted:30449410
cppdebug:30492333
pylanb8912:30545647
vsclangdc:30486549
c4g48928:30535728
hb751961:30543456
dsvsc012:30540252
azure-dev_surveyone:30548225

Same problem here, conflict view/highlighter is not showing up anymore (no config change whatsoever)

Is it broken or just a horrible UI change? Either way it's been unusable.

Also not working here in the case of Subversion conflicts. These conflicts were correctly detected and highlighted up until 1.70 when they broke. Neither 1.70.1 and 1.70.2 recovery releases have fixed this.

/duplicate
Most of the old experience will be restored in the next update.

Thanks for creating this issue! We figured it's covering the same as another one we already have. Thus, we closed this one as a duplicate. You can search for similar existing issues. See also our issue reporting guidelines.
Happy Coding!

But it's not about the old
Its happening with the new merge ui as well


updateQueries updates all instances of a parameterized query, not the "appropriate" one

I have a query that gets the graph of a  specificproject, like this:
export const projectQuery = gql`
query projectQuery($projectId: Int!) {
  project(id: $projectId) {
    id archived
    address{shortDisplay}
    contract {client description originalPrice
      variations {id, statusId, reasonId, description}
    }
    primaryContact {contactId}
  }
}

and a mutation that adds a variation to a specific project like this:
 mutate({variables: {variation: variation, projectId: projectId},
                updateQueries: {
                    // Make sure that variation list is updated with the new variation
                    projectQuery: (prev, {mutationResult}) => {
                        console.log("createVariation projectQuery updater", prev, mutationResult)
                        return update(prev, {
                          project: {
                            contract: {
                              variations: {$push: [mutationResult.data.createVariation]}}}
                        })
                    }
                }
            })

The problem is that I can see this updateQueries function being called for every instance of the projectQuery that is in the store when the mutation runs.
This results in the information for every cached project being updated - so they all wrongly get a variation added to them, when only the specific one should have.
Debugging this issue raised another unexpected result: updateQueries is called for every instance of the query in the store - even ones that are identical.    I have two components in a render tree that each make the projectQuery for the project that is on the route.   This results in updateQueries being called twice even when the application has only ever looked at one project.  That's quite surprising.
So:

Is it a bug that updateQueries doesn't handle selection of correctly parameterised instances of the query to operate on?
Is it a bug that the same query is cached multiple times (and thus updated multiple times by a given mutation)?

Please see issue #903. It basically mentions similar issues.

Another related one #902

I agree that the two referenced issues are about the same topic.  #902 seems like the clear-cut bug part of this, and hopefully will get resolved quickly (the multiple instances of the same query in the store)
Unfortunately, #903 (parameterised queries all getting updated) got contaminated with a separate question about optimistic responses, and in the end I found it hard to understand which issue got the attention of the developers, and which was left dangling :(
This issue is specifically about the fact that all instances of a parameterized query get updated.

@GreenAsJade The behavior you described is actually the intended behavior, as unintuitive as it may seem at first. updateQueries simply updates a query by name, so if there are multiple instances of that query, updateQueries will run for each of them. The reason is that updateQueries was designed as a pretty flexible way to update any currently active query, not just the current one (that's why it's called updateQueries and not updateQuery).
What that means is that you have to deal with possible overlap in the cache and make sure the update is only pushed to the array once, etc. This is also true when you have a mutation and a subscription that update the same array in the cache, so I'd say it's not that surprising.
I agree that it might be worth adding in react-apollo that updates just the current query. The problem with that though is that we'd have to deal with all sorts of race conditions. We'll get around to doing that eventually, but for now you have to do it yourself. 😬
That said, you may also want to consider using the result reducers as documented here: http://dev.apollodata.com/react/cache-updates.html#resultReducers
Also note that you can check the matching parameters yourself, they are passed into updateQueries in the options parameter:
(parameter) options: {
    mutationResult: Object;
    queryName: Object;
    queryVariables: Object;
}

@helfer the problem is not that it updates multiple queries but that it updates multiple queries that are stopped long time ago and not active. I do not think it is by design?

@yurigenin  "Lingering 'inactive' queries" is not actually the problem addressed by this issue.  I think that is "related but different", and covered by #902.
Here I am reporting that a mutation on an object of a specific id triggers updateQueries on all queries of all objects of that type (not just the specific one that was mutated).
As I type this, I realise that with the current API it couldn't be any other way: we can only tell updateQueries the name of the query that we want to update, but not the parameters that were involved.
This means that every cached query of a given type (name) will have the updateQueries function run on it.
To me this does not make sense.   My app makes two queries:
       projectQuery(1)
       projectQuery(2)

Then I do a mutation on the content of project number 2.   "Clearly" I should be able to say "updateQueries please update projectQuery(2)"
Therefore, I think that this issue is actually an "enhancement request", to make the updateQueries functions able to take parameters of the original query, and thus only be activated on those.
Note that in the mean time there is a "workaround", which is to check in prev whether this mutation should apply to this instance of the query.
I think the documentation needs to mention this, and I may submit a PR for that before too long.

@GreenAsJade  You can do it already. There is a queryVariables parameter passed to your reducer. There you can check which values your variables have for this reducer call and only reduce state for those results that correspond to the current variable values.

That sounds good.  Where is it documented?

@GreenAsJade I don't think it is but you can discover a lot by tracing the call stack in the debugger.

@GreenAsJade here's the signature of the second argument to the updateQueries reducer:
(parameter) options: {
    mutationResult: Object;
    queryName: Object;
    queryVariables: Object;
}

As I write this I realize that we really need to improve our docs for this... -> https://github.com/apollostack/react-docs/issues/112