build: BPFMAN_IMG & BPFMAN_AGENT_IMG to overwrite image

The bpfman-operator is setup to allow BPFMAN_IMG to overwrite the default bpfman image, and BPFMAN_AGENT_IMG to overwrite the bpfman-agent image. However, the Makefile is leveraging kustomize. kustomize can replace an image string in a yaml when it knows the k8s object layout, but these images are passed via a ConfigMap, which is opaque data. So the current implementation doesn't work. kustomize does have a ConfigMapGenrator, which can replace the contents of a ConfigMap.
The change is to:

Change the kustomization.yaml to use a configMapGenerator.
Use sed to replace the default images with those passed in (or just the default image if none were passed in). The kustomize edit set image command doesn't work.  The sed command is changing the file content, so rename kustomization.yaml to kustomization.yaml.env and piped the changes to kustomization.yaml.
Add kustomization.yaml to .gitignore so changes aren't tracked by git.

To reproduce, run something like:
BPFMAN_IMG=quay.io/billy99/bpfman-test:pr-1150 make run-on-kind

Then use kubectl describe pod -n bpfman bpfman-daemon-xxxxx to view the container image that was loaded.

NOTE: make run-on-kind is currently not working. As a temp workaround, the BPFMAN_AGENT_IMG and BPFMAN_OPERATOR_IMG in Makefile need a -amd64 appended to them. I didn't fix that because I think it needs a little more discussion on the correct fix.

NOTE: make run-on-kind is currently not working. As a temp workaround, the BPFMAN_AGENT_IMG and BPFMAN_OPERATOR_IMG in Makefile need a -amd64 appended to them. I didn't fix that because I think it needs a little more discussion on the correct fix.

WDYT about the following to fix -amd64 tag issue ?
.PHONY: load-images-kind
 load-images-kind: ## Load bpfman-agent, and bpfman-operator images into the running local kind devel cluster.
-       kind load docker-image ${BPFMAN_OPERATOR_IMG} ${BPFMAN_AGENT_IMG} --name ${KIND_CLUSTER_NAME}
+       kind load docker-image ${BPFMAN_OPERATOR_IMG}-${MULTIARCH_TARGETS} ${BPFMAN_AGENT_IMG}-${MULTIARCH_TARGETS} --name ${KIND_CLUSTER_NAME}

NOTE: make run-on-kind is currently not working. As a temp workaround, the BPFMAN_AGENT_IMG and BPFMAN_OPERATOR_IMG in Makefile need a -amd64 appended to them. I didn't fix that because I think it needs a little more discussion on the correct fix.

WDYT about the following to fix -amd64 tag issue ?
.PHONY: load-images-kind
 load-images-kind: ## Load bpfman-agent, and bpfman-operator images into the running local kind devel cluster.
-       kind load docker-image ${BPFMAN_OPERATOR_IMG} ${BPFMAN_AGENT_IMG} --name ${KIND_CLUSTER_NAME}
+       kind load docker-image ${BPFMAN_OPERATOR_IMG}-${MULTIARCH_TARGETS} ${BPFMAN_AGENT_IMG}-${MULTIARCH_TARGETS} --name ${KIND_CLUSTER_NAME}

That's probably fine as long as that's the only place it is needed.


Cant browse file if password is not saved in server config

If I dont save the password but leave the field blank, it ask me for the password to establish the connection. It say connection ok, but I cant browse the folders. If I store the password on the ftp or sftp connection I get the list of files and all works good. Did I miss something when the password field is blank or it's a bug ?

Can you elaborate on which package you're entering a password into? Nothing in atom requires a password

Sorry, it's for the remote-edit package. I open a case for this package.


Make middlewares wrap handlers rather than run before and after

Currently middlewares and afterwares are handled in the vk/middleware.go file augmentHandler method. The way this works is that it calls the middlewares first, one by one, and finishes with them, then calls the actual core handler function, and then calls a bunch of other afterwares, one by one.
An architecture diagram would look like this:
Request -> (mw1) -> (mw2) -> (core handler) -> (aw1) -> (aw2) -> Response

Instead middlewares should be wrapping the other handlers. That way we can construct a layered structure that simplifies code and is easier to reason about, plus it would remove distinction between a middleware (used here to mean running code that should happen before the core handler does anything with it), and afterware (used here to mean running code that should happen after the core handler is done with the request).
The target structure should look like this:
Request ->                                                                                -> Response
           ( mw1 ->                                                    -> mw1, but after code )
                    ( mw2 ->                  -> mw2, but after code )
                             ( core handler )

A 💎 : https://justinas.org/alice-painless-middleware-chaining-for-go


light.range > 0

In the function SerializeGltfLight(), the serialization of the light range should be done only if the range is bigger than 0.
Something like this could fix it:
  if(light.range > 0)
  {
    SerializeNumberProperty("range", light.range, o);
  }

Reference: https://github.com/KhronosGroup/glTF/tree/master/extensions/2.0/Khronos/KHR_lights_punctual
Many thanks for fixing so quickly all the previous issues.

Thanks! Now fixed!


QuotaMgmtClient does not raise exception for response 429 (Too many requests)

Package Name: azure-mgmt-quota
Package Version: 1.1.0
Operating System: Windows 11
Python Version: 3.11.9

Describe the bug
The Azure Quota Management SDK (QuotaMgmtClient) does not raise an exception when receiving a 429 (Too Many Requests) response from the API when requesting quota increases. Instead the response code is shown in the terminal output, this makes it difficult to handle rate limiting scenarios properly
To Reproduce
Steps to reproduce the behavior:

Install library
pip install azure-mgmt-quota
Run code snippet using QuotaMgmtClient (Simplified code snippet)

try:
        client = QuotaMgmtClient(
            credential=DefaultAzureCredential(),
        )

        poller = client.quota.begin_create_or_update(
            resource_name=quota_sku,
            scope=f"subscriptions/{subscription_id}/providers/{provider}/locations/{quota_region}",
            create_quota_request={
                "properties": {
                    "limit": {
                        "limitObjectType": "LimitValue",
                        "value": quota_new_limit,
                    },
                    "name": {"value": quota_sku},
                }
            },
        )
        # Periodically check the status and try to access the result
        while not poller.done():
            logging.info(f"poller status: {poller.status()}")
            try:
                intermediate_result = poller.result(timeout=1)
                logging.info(f"Intermediate result: {intermediate_result}")

                return intermediate_result
            except (HttpResponseError, ServiceRequestError) as e:
                logging.error(
                    f"Failed to make the request - feel free to retry. But the specifics are here: {e}"
                )
                raise e
            except TimeoutError:
                logging.info("poller not complete yet, waiting...")
            time.sleep(10)  # Wait for 10 seconds before checking again

        # Once the poller is complete, access the final result
        final_result = poller.result()
        logging.info(f"Final result: {final_result}")

        logging.info("Successfully requested quota increase.")

        return final_result
    except (HttpResponseError, ServiceRequestError) as e:
        logging.error(
            f"Failed to make the request - feel free to retry. But the specifics are here: {e}"
        )
        raise e

Observe that when the API returns a 429 status, no exception is raised.

Expected behavior
When the API returns a 429 status code, the SDK should raise an HttpResponseError with a status code of 429. This will allow developers to catch and handle rate limiting scenarios appropriately.
Screenshots

Additional context
The current behaviour makes it challenging to implement proper rate limiting in applications using this SDK. It's especially problematic in serverless environments like Azure Functions where a timeout can occur without properly handling the error

Thank you for the feedback @AdrianM10 . We will investigate and get back to you asap.

Hi @AdrianM10, by the swagger definition here, quota has defined another exception model to raise instead of raising HttpResponseError. Please use from azure.mgmt.quota.models import ExceptionResponse to catch error info.

Hi @ChenxiJiang333, I've updated my exception handling to catch the "ExceptionResponse" instead of "HttpResponseError". Despite these changes, I'm still not able to catch the 429 response. No exception is caught or raised.

Hi @AdrianM10, not able to reproduce the error to catch locally, could you take a try at from azure.core.exceptions import ServiceResponseError ?

Hi @ChenxiJiang333, still the same. That also doesn't catch the error.

Hi @ChenxiJiang333, still the same. That also doesn't catch the error. To reproduce the error you'll need to send the same request multiple times for a quota increase in quick succession.

Would there be any error message I wonder? Have you tried except Exception as e:  logging.error(e) to check whether there would be an exception raised when receive the response 429? If there's not, then it might be the issue of the package.

Still not even with except Exception as e, just to test I intentionally sent a malformed payload, this returns an error (Response status 400) see below:
{
  "id": null,
  "message": "(InvalidResourceName) Name standardBSFamil is not valid resource name.\nCode: InvalidResourceName\nMessage: Name standardBSFamil is not valid resource name."
}

Looks like somehow the 429 response in particular isn't being treated as an exception.

Hi @AdrianM10, has found the root. Response 429 follows the retry policy, so it won't raise any exception but to resend the request after a specified time.

Hi @ChenxiJiang333, is there any sample code that I can reference to access the response status, response headers / object once the call has been made ? I had spent a week or so prior to raising the issue and had no luck.

I have been unable to use any method that can access the HTTP response object, is there any example code or docs I can use as reference?

Hi @AdrianM10, I have found a way to disable the retry policy and raise the exception.
Try update the code to:
        poller = client.quota.begin_create_or_update(
            resource_name=quota_sku,
            scope=f"subscriptions/{subscription_id}/providers/{provider}/locations/{quota_region}",
            create_quota_request={
                "properties": {
                    "limit": {
                        "limitObjectType": "LimitValue",
                        "value": quota_new_limit,
                    },
                    "name": {"value": quota_sku},
                }
            },
            retry_total = 0
        )

Hi @ChenxiJiang333, Thank you so much for your help and for providing this solution! I really appreciate your time and effort in addressing my issue.


Need help!!!

Tmuxinator is great. Sir, I saw your this code. But I want to run 4 VPS and multiple commands with a single tmux screen. like this image. So, what will my config file look like? Give me a demo. please, please, please

name: sample
root: ~/

windows:
  - stats:
    - ssh<EMAIL_ADDRESS>    - tail -f /var/log/stats.log
  - logs:
      layout: main-vertical
      panes:
        - logs:
          - ssh<EMAIL_ADDRESS>          - cd /var/logs
          - tail -f development.log

This should be enough to go on:
name: 869
windows:
    - editor:
        layout: tiled
        panes:
          - one:
            - echo one
          - two:
            - echo two
          - three:
            - echo three
          - four:
            - echo four


feat(storage): introduce reclaim_table_write_throughput

I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.
What's changed and what's your intention?
After https://github.com/risingwavelabs/risingwave/pull/18806
if a table stops writing (silently), it will still be misclassified as high throughput, which will prevent the merge strategy.
Therefore, the PR introduces a recliam mechanism to reclaim expired table throughput statistic, consider the following case

When table stops being written, the table throughput statistic will be recliamed.
When the throughput statistic of a table is empty, there are two scenarios

The throughput statistic has expired and has been reclaimed (no throughput, can be merged)
The table is in the creating phase, and the first ckpt has not yet completed. (will not be merged)

Checklist

[ ] I have written necessary rustdoc comments
[ ] I have added necessary unit tests and integration tests
[ ] I have added test labels as necessary. See details.
[ ] I have added fuzzing tests or opened an issue to track them. (Optional, recommended for new SQL features #7934).
[ ] My PR contains breaking changes. (If it deprecates some features, please create a tracking issue to remove them in the future).
[x] All checks passed in ./risedev check (or alias, ./risedev c)
[ ] My PR changes performance-critical code. (Please run macro/micro-benchmarks and show the results.)

[ ] My PR contains critical fixes that are necessary to be merged into the latest release. (Please check out the details)

Documentation

[ ] My PR needs documentation updates. (Please use the Release note section below to summarize the impact on users)

Release note
If this PR includes changes that directly affect users or other significant modifications relevant to the community, kindly draft a release note to provide a concise summary of these changes. Please prioritize highlighting the impact these changes will have on users.

PTAL @hzxa21 @zwang28


feat(guides): Update SABnzbd guide

Pull Request
Purpose

Update the SABnzbd guide to be UpToDate with 4.3
Approach

[x] Updated the images to match the new/changed settings.
[x] Added improved descriptions of several suggested settings, thnx to @thezoggy

Open Questions and Pre-Merge TODOs

Requirements

[x] These changes meet the standards for contributing.
[x] I have read the code of conduct.

jesus trash.....lol

yeah it was a plain copy/paste from zoggy with a few minor changes on my side

jesus trash.....lol

yeah it was a plain copy/paste from zoggy with a few minor changes on my side

I mean i got the basic message, but as far as grammar and shit goes you know how it is :D

i often use chatgpt
grammar check
text
but even then you come with changes.
If you approved the changes we wait till the weekend to merge because of the merge freeze we have

i often use chatgpt grammar check text but even then you come with changes.
If you approved the changes we wait till the weekend to merge because of the merge freeze we have

Yea, I know. It's not really as much the grammar "rules" that I come with as much as the way people talk in English and "flow" of the sentences. It's kind of a subjective thing, but I've gotten pretty good at writing and particularly with docs. It's mostly an experience thing and it's my native language.
I'm not trying to make you feel bad about it, as I said most of what was written wasn't horrible, just as far as ease of reading and flow of the sentences it had room to adjust.

jesus trash.....lol

this made it sound different lol

jesus trash.....lol

this made it sound different lol

oh shit. yea....no.....it's his name :P I didn't mean it like THAT....just a lot of changes lolol

Thnx for the review. added the do not merge label because of the freeze we've active at the moment


locale option not working

Hi,
I met a bug using the locale option, the words don't load and I have this error in my console:
(missing: http://localhost:8080/js/vuecal.common.i18n/fr.js)
    at requireEnsure (webpack-internal:///./node_modules/vue-cal/dist/vuecal.common.js:107:26)
    at Array.map (<anonymous>)
    at webpackAsyncContext (webpack-internal:///./node_modules/vue-cal/dist/vuecal.common.js:1595:34)
    at VueComponent.loadLocale (webpack-internal:///./node_modules/vue-cal/dist/vuecal.common.js:8939:36)
    at VueComponent.created (webpack-internal:///./node_modules/vue-cal/dist/vuecal.common.js:9839:10)
    at invokeWithErrorHandling (webpack-internal:///./node_modules/vue/dist/vue.runtime.esm.js:1853:57)
    at callHook (webpack-internal:///./node_modules/vue/dist/vue.runtime.esm.js:4213:7)
    at VueComponent.Vue._init (webpack-internal:///./node_modules/vue/dist/vue.runtime.esm.js:4998:5)
    at new VueComponent (webpack-internal:///./node_modules/vue/dist/vue.runtime.esm.js:5144:12)
    at createComponentInstanceForVnode (webpack-internal:///./node_modules/vue/dist/vue.runtime.esm.js:3280:10)```

Could it be linked with the fact that https://github.com/antoniandre/vue-cal/blob/5831735c00462eceff2799296bcb9a17b2d6c8d0/src/vue-cal/index.vue#L365
has been modified to fix a babel bug, and due to operator precedence the Webpack module method doesn't work as expected?
If not, could you confirm that the locale option is still working with a recent Vue v2 setup?

Thanks in advance.

Bonjour @c4ffein,
I can confirm the locale is still working with the latest vue-cal on Vue 2.
You can check this demo: https://codepen.io/antoniandre/pen/yLJWBbK?editors=1010
It is also working on the documentation page which is Vue 2.
https://antoniandre.github.io/vue-cal/#ex--internationalization
How did you load the locale? A correct path should be vue-cal/dist/i18n/fr.js.
Cheers.

By the way I just released version 3.9.0 where you can provide a fully custom locale.
Not what I recommend, but if in your project you have no other choice, then here it is: #352.

Hi again,
I just realized I made the import in the wrong file. Sorry for wasting your time and thanks for your support.


Update GSoC-interest.md

Extremely sorry.
I just realised that I actually open sourced my proposal........

I see no problem in making your proposal public (in fact, I was expecting that many more people would be doing so), but I respect your intentions. So, I just accepted your pr. Feel free to remove the Google doc too (the link is still in the history of the git repo).


Marking error datasets and warnings/caveats

Have an ERROR.txt or something in the dataset & on disk to indicate dataset should be skipped for processing.

See https://github.com/terraref/reference-data/issues/218
For data that a human has recognized as being in error (e.g. w/ blurry FLIR data, point clouds clipped at some height)

add a text file named "ERROR" with optional content / explanation / pointer to a github issue, could be a set of key: value pairs, perhaps in yaml that get parsed directly to json metadata
have an extractor that finds these files and adds a tag "quality"{ "ERROR" = "TRUE", "description":"", "key2":"value2"
add a a general rule perhaps at the level of extractors, or perhaps at the level of RabbitMQ that says any time an error (file or flag) is found, skip processing the dataset.

For example, for all FLIR data in 2017
quality:
  status: ERROR
  description: Sand and water contaminated FLIR Camera lens so temperature values are invalid
  url: https://github.com/terraref/reference-data/issues/182

duplicate of #557

there are different severities of error - FLIR 2017 is prominent example, but the stair-stepping on the laser3D data is not so cut and dry and may still have valuable data in it

Proposed script will add new metadata entry from Maricopa Field user with body like:
{
"quality": "ERROR",
"description": "Sand and water contaminated FLIR Camera lens so temperature values are invalid",
"url": https://github.com/terraref/reference-data/issues/182"
}

Other status could be WARNING, ADVISORY, etc. We would also write a corresponding yaml file to the globus directory with those contents as suggested, perhaps at the day level rather than repeated for entire dataset? or do we want it repeated at the dataset (timestamp) level?

We should also add a file named "ERROR" that contains the description and url to the affected dataset. I think repeating this at the dataset level will be good. There may be use for having a tag at a higher level, but that would be in addition to the dataset level flag.

My script is prepared to generate the YAML files & metadata, however due to the raw_data directories being owned by dlebauer I am unable to write into them. We can discuss how to handle this during meeting... probably one of:

run as sudo and chmod the yaml files when created so they are owned by dlebauer consistently with the others
have dlebauer run the script

I am glad that the raw data folder is locked down. I am not sure it makes sense for me to be the folder owner (as opposed to a user or group like ‘terraref’ but ... the idea is that we don’t touch the raw_data folder.
In the end, the key requirement is that any data that have known errors (or other issues) are clearly labeled as such.
It makes sense (at this point) to have to use sudo to touch the raw_data folder, if we should ever touch it at all. But maybe there is a ‘better’ way to handle this. Certainly none of the existing files should be touched, but allowing the same user that transfers the files to be able to create a new file would also seem reasonable.
For the FLIR, we did process the data to Level 1. Is the plan to also add an error file to the level 1 data?

Script is running now. Will close this when completed.
I would argue that in the FLIR case we don't add an error to Level_1 data, the goal was to flag the raw so that in the future we dont even process these erroneous datasets. I would make an argument for deleting level 1 + data from this time period for FLIR to be consistent with that.

standardized taxonomy for different error cases - ERROR that should not be sent through processing vs. WARNING vs. other classes

Max will write up some documentation/wiki once this is done to propose a standard approach to handling this.

support ability to explicitly define files (vs entire dataset) - 'all' could be default value.

Created #589 to follow this.


[BugFix] Fix process fail when select from load_tracking_logs specify not exist job_id

What type of PR is this：

[x] BugFix
[ ] Feature
[ ] Enhancement
[ ] Refactor
[ ] UT
[ ] Doc
[ ] Tool

Which issues of this PR fixes ：

Fixes #
Problem Summary(Required) ：

Checklist:

[x] I have added test cases for my bug fix or my new feature
[ ] This pr will affect users' behaviors
[ ] This pr needs user documentation (for new or modified features or behaviors)

[ ] I have added documentation for my new feature or new function

Bugfix cherry-pick branch check:

[x] I have checked the version labels which the pr will be auto backported to target branch

[x] 3.0
[ ] 2.5
[ ] 2.4
[ ] 2.3

run all

[FE PR Coverage Check]
:disappointed: fail : 0 / 2 (00.00%)
file detail

path
covered_line
new_line
coverage
not_covered_line_detail

:large_blue_circle:
com/starrocks/service/FrontendServiceImpl.java
0
2
00.00%
[1915, 1916]


Don't force resuming continuation on same thread

I assume these can be continued on other threads. If not, a comment should be added instead.

You are correct, well spotted. Thanks.

You are correct, well spotted. Thanks.


Security warnings

npm install gives
found 6 vulnerabilities (1 moderate, 4 high, 1 critical)
run npm audit fix to fix them, or npm audit for details
npm audit gives
                   === npm audit security report ===

                             Manual Review
         Some vulnerabilities require your attention to resolve

      Visit https://go.npm.me/audit-guide for additional guidance

Critical        Prototype Pollution
Package         handlebars
Patched in      >=4.0.14 <4.1.0 || >=4.1.2
Dependency of   hbs
Path            hbs > handlebars
More info       https://npmjs.com/advisories/755
High            Prototype Pollution
Package         handlebars
Patched in      >=4.3.0
Dependency of   hbs
Path            hbs > handlebars
More info       https://npmjs.com/advisories/1164
Moderate        Denial of Service
Package         handlebars
Patched in      >=4.4.5
Dependency of   hbs
Path            hbs > handlebars
More info       https://npmjs.com/advisories/1300
High            Arbitrary Code Execution
Package         handlebars
Patched in      >=4.5.2
Dependency of   hbs
Path            hbs > handlebars
More info       https://npmjs.com/advisories/1316
High            Arbitrary Code Execution
Package         handlebars
Patched in      >=4.5.3
Dependency of   hbs
Path            hbs > handlebars
More info       https://npmjs.com/advisories/1324
High            Prototype Pollution
Package         handlebars
Patched in      >=4.5.3
Dependency of   hbs
Path            hbs > handlebars
More info       https://npmjs.com/advisories/1325
found 6 vulnerabilities (1 moderate, 4 high, 1 critical) in 203 scanned packages
6 vulnerabilities require manual review. See the full report for details.
P

In the express-app I'm using hbs which depends on this vulnerable handlebars. Not sure what I can do to resolve it, I have the latest hbs package now.
Looks like hbs is aware and are tracking this: https://github.com/pillarjs/hbs/issues/185

Thanks for the report, I've updated the dependencies in 613924f2726fe3ac089e98601efc5fd78de65f54

There is a pull request on HBS that has been sitting for 27 days. Absolutely unbelievable. Upstream has limited any comments, issues and pull requests to collaborators which is again unbelievable.


Project creation form remembers funding sources

The project creation form when cancelled, seems to remember associated funding sources.
Steps:

Open create a new project form
Add some funding sources
Go back to dashboard
Return to form

Expected:
Fresh/empty form
Observed:
Contains funding sources from previous form

Opened by mistake. Closing.


add 'zeroctl list template' command

Once we have the flow described in #460 in place, we'll need a way to list all the templates available in a storage cluster, this command should also works against the 0-hub to be able to list all templates available to imports.

I suppose that means we somehow store a list of vdisk ids, which were created using zeroctl import template?

Because we already have zeroctl list vdisks, but that lists all vdisks, there is no filter on templates, as templates are just a concept, it's not visible in the storage at all what is a template and what is not.

Currently the way to create a templates is way to complex and cumbersome, so I think the creation of a template should also be handled by the zeroctl. Like that we can ensure that a certain format for the template is respected and list can then filter base on the name of the disk ?

@zaibon you are referring to creating them using zeroctl import template, no? Like I said, as long as they are created using zeroctl import template, and any other (new) zeroctl command in the future, we can maintain a set value in the ardb metadata server.

@zaibon you are referring to creating them using zeroctl import template

No I'm not. I'm referring to the creation of the template on the hub. See https://github.com/zero-os/home/blob/master/docs/tutorials/Create_a_vdisk_Boot_Image.md
The process described in this document should be reviewed and I think zeroctl should play a role there. Now it's all too manual, complex and cumbersome. No user would ever do that.

Anything the nbdserver does in those steps can easily be done using the zeroctl tool with some new kind of zeroctl create template command. However we would need to find a way to convert an image to to our storage blocks (which is now done using qemu-img convert). I don't think there is a standard Go solution available for that, so that might be quite a lot of work (to support such a image creation facility) if you want to automate all that into a zeroctl command...

@zaibon I created a seperate issue #463 for it. Which would provide a zeroctl create template for this purpose. For now I've put it under the Roadmap milestone as the beta-1 is already quite packed as it is. But if you really need it in beta-1, feel free to move it (but I can't promise it won't delay a release, as the beta-1 milestone is very packed as it is, so I wouldn't recommend doing so).

@zaibon in an earlier comment I proposed to add some new data (an ARDB list/set) which would collect the templates stored on an ARDB server. However I don't think this is a good idea because:

It would make all other commands which have to interact with that data also more complex (as it would need to ensure that data is also copied/deleted/moved/created/exported);
Templates are a high-level concept, outside the bounds of 0-Disk;

However, in all this time that 0-Disk is used by 0-Orchestrator I've noticed that the Cairo team already prefixes templates at all times with the prefix template:.
So here is what I'll do in a PR to resolve this issue:

Add an optional argument to the ListVdisks command (this can be nil):

func ListVdisks(cluster ardb.StorageCluster, pred func(string) bool) ([]string, error)

When it's not nil it will used as an extra filter.

Add a optional flag --name which takes a Regexp String, which we can use to than use the (*Regexp).MatchString as the predicate mentioned above;

This solution allows you to list all templates and more. Without requiring to store any new data (which is a bad idea, as it would only making everything more error prone).

This will essentially allow you to do zeroctl list vdisks <clusterID> --name '^template:', which will give you all templates on that cluster (note that this command is pretty slow if you have a lot of data on a server though).


encodeFunctionCall

I'm trying to replicate some code made on Ionic, which uses web3.js.
At a certain point, it calls web3.eth.abi.encodeFunctionCall but as far as I've looked throught the library it doesn't contain any similar feature.
Here's the snippet I'm trying to mirror:
const transaction = Object.assign({}, config_1.config.basicTransaction);
    transaction.gasLimit = 600000;
    const publicKeyCallData = web3.eth.abi.encodeFunctionCall(config_1.config.contractsAbi.registry.addKey, [publicKey]);
    transaction.data = web3.eth.abi.encodeFunctionCall(config_1.config.contractsAbi.manager.identity, [publicKeyCallData]);
    transaction.to = config_1.config.manager;
    return transaction;

So far I have this, trying to replicate the first encodeFunctionCall:
        let walletAddress = EthereumAddress(walletAddress)! // Your wallet address
        let contractMethod = "addKey" // Contract method you want to write
        let contractABI = """
<JSON with ABI>
"""
        let contractAddress = EthereumAddress("<Address>")!
        let abiVersion = 2 // Contract ABI version
        let parameters : [AnyObject] = [publicKey.toHexString() as AnyObject] // Parameters for contract method
        let extraData: Data = Data() // Extra data for contract method
        let contract = w3.contract(contractABI, at: contractAddress, abiVersion: abiVersion)!
        var options = TransactionOptions.defaultOptions
        options.from = walletAddress
        options.gasPrice = .manual(BigUInt(0))
        options.gasLimit = .manual(BigUInt(600000))
        let tx = contract.method(contractMethod, parameters: parameters, extraData: extraData, transactionOptions: options)

if let ethTransaction = try? tx?.assemble(), let hashSignature = ethTransaction.encode(forSignature: true, chainID: nil)?.toHexString(){
            print(hashSignature)
            return hashSignature
        }else {
            throw Error.pubKeyData
        }

I've tried to use assemble and encode the transaction trying to replicate encodeFunctionCall, which then I need to add as parameter to the second method, but this fails with "Failed to fetch gas estimate" and I'm clueless wether this is an issue with how I coded that function or with the Ionic - Swift translation I'm developing.
Can you elaborate on how to use web3.eth.abi.encodeFunctionCall on this library or if it's not supported at all?

Hello. Now you can use this function:
https://github.com/matter-labs/web3swift/blob/e36e767eb2de26decb0c157aeef2d023a1175076/Sources/web3swift/EthereumABI/ABIEncoding.swift#L136
Close for now. Feel free to comment issue if you still need help. 👍


Error in documentation for starting docker

In documentation it is said to use docker compose up -d. Howerver, it should be docker-compose up -d
To Reproduce
Steps to reproduce the behaviour:

following the documentation as is currently written;
typing docker compose up -d
press enter
get error unknown shorthand flag: 'd' in -d

Actual behaviour
unknown shorthand flag: 'd' in -d
OS information (IMPORTANT)

OS:  Ubuntu
Palworld Version:  N/A
Docker Image Version:  0.32.0

Hardware information (IMPORTANT)
N/A
docker-compose.yml contents (IMPORTANT)
---
services:
  palworld:
    image: thijsvanloef/palworld-server-docker:latest
    restart: unless-stopped
    container_name: palworld-server
    stop_grace_period: 30s  # Set to however long you are willing to wait for the container to gracefully stop
    ports:
      - 8211:8211/udp
      - 27015:27015/udp  # Required if you want your server to show up in the community servers tab
    environment:
      PUID: 1000
      PGID: 1000
      PORT: 8211  # Optional but recommended
      PLAYERS: 32  # Optional but recommended
      SERVER_PASSWORD: ""  # Optional but recommended
      MULTITHREADING: "true"
      RCON_ENABLED: "true"
      RCON_PORT: 25575
      TZ: "UTC"
      ADMIN_PASSWORD: ""
      COMMUNITY: "false"  # Enable this if you want your server to show up in the community servers tab, USE WITH SERVER_PASSWORD!
      SERVER_NAME: "Arm-Test"
      SERVER_DESCRIPTION: "Work God Dammit"
      EXP_RATE: 1.5
      PAL_CAPTURE_RATE: 1.5
      PAL_SPAWN_NUM_RATE: 1.5
      PLAYER_AUTO_HP_REGEN_RATE: 1.25
      PLAYER_AUTO_HP_REGEN_RATE_IN_SLEEP: 5
      PAL_AUTO_HP_REGEN_RATE: 1.25
      PAL_AUTO_HP_REGEN_RATE_IN_SLEEP: 5
      BUILD_OBJECT_DAMAGE_RATE: 0.25
      BUILD_OBJECT_DETERIORATION_DAMAGE_RATE: 0
      COLLECTION_DROP_RATE: 1.5
      COLLECTION_OBJECT_RESPAWN_SPEED_RATE: 0.5
      ENEMY_DROP_ITEM_RATE: 1.5
      DEATH_PENALTY: 1
      BASE_CAMP_WORKER_MAX_NUM: 20
      GUILD_PLAYER_MAX_NUM: 32
      PAL_EGG_DEFAULT_HATCHING_TIME: 0.5
      WORK_SPEED_RATE: 2
      ENABLE_NON_LOGIN_PENALTY: "false"
      COOP_PLAYER_MAX_NUM: 32
    volumes:
      - ./palworld:/palworld/

Bug Report Checklist

[y] I searched for my issue on the Known issues page
[y] I searched for my issue in the Closed issues
[y] I have included all relevant information required above.
[n - I don't know the first thing about docker, hence why I was following the documentation exactly as it was written] I am willing to fix this bug

Hi @T-h-o-r-p-e It seems like you are using compose v1 (docker-compose). This version of docker compose has been deprecated.
I would suggest updating your docker version and using Compose V2: https://docs.docker.com/compose/migrate/


Sonatype OSS Index Analyzer analysis receiving HTTP 500 and failing the build

Describe the bug
Sonatype OSS Index Analyzer analysis is throwing an exception because it receives an HTTP 500 response when getting the component report. This causes the Maven plugin mojo to fail, which causes the build to fail after the check goal.
This only started happening today, with no changes to this particular project (it's built on a schedule), so it's using the same dependencies that succeeded last week.
Version of dependency-check used
Maven plugin version 5.0.0.
Log file
When running Maven with debug and error flags, the exception in this gist is logged. There is nothing else relevant logged when running Maven with the -X -e flags.
Expected behavior
The error should not occur and the build should complete successfully.
Additional context
Issue (unsurprisingly) disappears when analysis is disabled. This may possibly be an issue only with certain dependencies. A separate project failed in the same way when when attempting to generate a report for org/springframework/spring-beans/4.3.15.RELEASE/spring-beans-4.3.15.RELEASE.jar. All dependency jars are retrieved from Maven central.
Both of these projects successfully built last week (20 June 2019) with no changes to their configuration and using the exact same dependencies (eg. the local m2 repository still has the same jars and metadata.)
Maven configuration of plugin:
<plugin>
  <groupId>org.owasp</groupId>
  <artifactId>dependency-check-maven</artifactId>
  <version>5.0.0</version>
  <configuration>
    <format>ALL</format>
    <skipProvidedScope>true</skipProvidedScope>
    <skipRuntimeScope>true</skipRuntimeScope>
    <assemblyAnalyzerEnabled>false</assemblyAnalyzerEnabled>
  </configuration>
  <executions>
    <execution>
      <goals>
        <goal>check</goal>
      </goals>
    </execution>
  </executions>
</plugin>

Duplicate of #2017

Unfortunately - when an external service is reporting a 500 error there is not much the dependency-check code can do other than report the service is failing.  It appears to be back up and running again.

It would be better if it didn't fail the whole build -- or do I need to file a feature request for that?

@roddy, I agree. @jeremylong, is there a reason it needs to fail the build?

@jeremylong - I possibly phrased my initial post poorly. The bug I was reporting wasn't that the external service was failing; it was that the maven plugin was failing the build due to these circumstances. If that's the intended behavior it needs to be documented somewhere, because it was entirely unexpected on this side of things. At most I would have expected this to flip the status to unstable similar to a Maven unit test failure.

With the case of dependency analysis - just because a build passed last week does not mean it should pass this week - new vulnerabilities may have been published. ODC failed because it was not able to complete its checks.  If you want to make the build more stable you can configure <failOnError>false</failOnError>.

Failing the build because vulnerabilities are found, or vulnerability threshholds are crossed makes sense. But that's not the same as having a vulnerability database/API go down and being able to not contact it. In my case, you could publish a million new vulnerabilities which all trigger in my service, and I'm not going to fail my build, because that how I've got it configured (which is the default, coincidentally.) Conflating the use cases isn't helping us in this case.
<failOnError> is a useful configuration but overly broad, I think. Not knowing what other "error" cases there might be, I don't want to change it on a whim. Destabilizing the build (eg. making it unstable like a failed unit test) would have been more useful and allow us to have been alerted that something is wrong with the build, but still allow the thing to build.

I completely understand what you are asking for - I just need help figuring out how to achieve the goal.  We absolutely need to start putting the errors into the report so they are more visible. That would allow you to set failOnError to false, let the build pass, and then after the fact determine if the scan was successful.  Any other ideas?


[Feature] Migrate pkg tests to Ginkgo

Describe the solution you'd like
At the moment, we are using two different test frameworks:

Ginkgo/Gomega based tests for our APIs and controllers
Go tests for our pkgs

Ideally we should converge and migrate the pkg tests to ginkgo as well.
Describe alternatives you've considered
Keep using Go tests for pkg, it doesn't feel right to have 2 test frameworks in the same project. Besides, the pkgs could benefit a lot from the Ginkgo table tests and the Gomega assertions.

Hi! Can I take this one?

ginkgo seems to be integrating well with go based tests, no need to migrate for now!


Query on diagram of template areas

I think there may be a mistake in this diagram of the template areas.
In the text on the page templates page it states that beforeContent sits outside of main. It also states that main overrides main.
In the below diagram, beforeContent is shown as being within the main override. Based on the text, I think it should be outside it - that main wraps content only.

@edwardhorsford Thanks for raising this. The confusion is caused by two similarly named elements: block:main in the template macro and <main> in the HTML.
You can either use:

block:beforeContent(which sets content outside <main> element) and block:content(which is wrapped within <main>)
block:main that overrides both block:beforeContent and <main>

So the documentation and the diagram are correct but it's true that the naming could cause some confusion. From an implementation point of view, it made sense calling the template element block:main as it overrides block:beforeContent but maybe less so from a markup point of view. To fix this, we could rename block:main to block:supersede or some such. However we would anticipate this being a breaking change for many of our users so we need to balance the inconvenience caused by making such a change against the benefits gained.

Yeh just to note I find this confusing too

Thanks @hannalaakso.
Based on what you've said, I think the text "Override the default  element." is either incorrect or could do with further clarification. It sounds like it also supersedes block:beforeContent.

We've updated the wording based on your feedback, if you run into any more issues in the future feel free to re-open this or raise a new issue thanks Ed :)

Thanks @nickcolley, that covers my confusion.


local CLI cache

We experience some issues because of too many api requests, e.g.: #225 , #164 . These could be tackled my implementing a local cache.
Does anyone has experience with or suggestions for a local file based caching library/approach?

This project seems to be a neat cache:
https://github.com/grantjenks/python-diskcache
Also also does a comparison with other similar projects in its README. I don't have any experience, however :/


Padding of column string slice fails when last column of file has empty values

Description
I have an .xlsx file and for the last column, some values are empty cells. Normally, the row that is returned (a slice of strings) would be padded with an additional nil value (empty string) to represent an empty column - this works if I use an xlsx file that contains a column with some empty cells where the column is not the last column. However, if the column is the last column the padding of the slice fails.
Steps to reproduce the issue:

Create an xlsx file with two columns, where at least one of the rows contains an empty cell for the second column.
Read as usual, run GetRows() on the file object.
Range over rows and print any row where length of the row != 2 (the expected length).

Describe the results you received:
The rows with empty values were not padded with an empty string within the returned column slice of strings.
Describe the results you expected:
The rows with empty values would be padded with an empty string within the returned column slice of strings.
 In which case I can close this issue happily 👍

Thanks for your issue. I have tested with the following spreadsheet and it works well.

A
B
C

1
A1

2
A1
B2
C3

3
A3
B3

4
A4
B4
C4

5
A5
B5

package main

import (
    "fmt"

    "github.com/360EntSecGroup-Skylar/excelize"
)

func main() {
    f, err := excelize.OpenFile("Book1.xlsx")
    if err != nil {
        fmt.Println(err)
        return
    }
    // Get all the rows in the Sheet1.
    rows, err := f.GetRows("Sheet1")
    for _, row := range rows {
        for _, colCell := range row {
            fmt.Print(colCell, "\t|")
        }
        fmt.Println()
    }
}

Output:
A1  |
A2  |B2  |C3  |
A3  |B3  |
A4  |B4  |C4  |
A5  |B5  |

The GetRows API will retrieve not empty cells for each row, but it not will be skipping the cell with a blank string, so the length of each row returned by GetRows maybe not consistent. If you have any questions, let me know.

I close this issue and welcome left comments if you have any more questions.

had the same issue, this library I'm using to guesstimate a csv delimiter does so by comparing recurring separator patterns, which means it is not happy when encountering (quick example)
;;;
;
;;

Here's a quick solution for anyone wanting to feed f.GetRows to cw.WriteAll in a 'square' shape:
func padOutRows(rows [][]string) [][]string {
	columnCount := 0
	for _, r := range rows {
		if len(r) > columnCount {
			columnCount = len(r)
		}
	}
	for i, r := range rows {
		rows[i] = r[:columnCount]
	}
	return rows
}

@xuri I understand why it usually makes sense to skip empty rows (in face, I like how 'human' this library is in its approach to data),  but maybe offer it as an option?

had the same issue, this library I'm using to guesstimate a csv delimiter does so by comparing recurring separator patterns, which means it is not happy when encountering (quick example)
;;;
;
;;

Here's a quick solution for anyone wanting to feed f.GetRows to cw.WriteAll in a 'square' shape:
func padOutRows(rows [][]string) [][]string {
	columnCount := 0
	for _, r := range rows {
		if len(r) > columnCount {
			columnCount = len(r)
		}
	}
	for i, r := range rows {
		rows[i] = r[:columnCount]
	}
	return rows
}

@xuri I understand why it usually makes sense to skip empty rows (in face, I like how 'human' this library is in its approach to data),  but maybe offer it as an option?


fix(module:input): detect textarea-count config changes

PR Checklist
Please check if your PR fulfills the following requirements:

[x] The commit message follows our guidelines: https://github.com/NG-ZORRO/ng-zorro-antd/blob/master/CONTRIBUTING.md#commit
[ ] Tests for the changes have been added (for bug fixes / features)
[ ] Docs have been added / updated (for bug fixes / features)

PR Type
What kind of change does this PR introduce?

[x] Bugfix
[ ] Feature
[ ] Code style update (formatting, local variables)
[ ] Refactoring (no functional changes, no api changes)
[ ] Build related changes
[ ] CI related changes
[ ] Documentation content changes
[ ] Application (the showcase website) / infrastructure changes
[ ] Other... Please describe:

What is the current behavior?
The textarea-count component doesn't respond to nzMaxCharacterCount changes until the textarea changes
Issue Number: N/A
What is the new behavior?
The textarea-count component updates immediately when nzMaxCharacterCount changes
Does this PR introduce a breaking change?
[ ] Yes
[x] No

Other information

Sorry for taking so long to review, if you are still dealing with this issue, please make sure the CI passes after syncing the master codes :)


Frame closures less in terms of Obj-C and C blocks

Fixes: rdar://118579466

@krilnon Yes, the goal with this kind of text is to give folks a list of other names that they might be familiar with, and I guess I didn't read it as saying closures in Swift aren't like closures in other languages.  Can you think of a wording that doesn't have the omission issue?

Something like "Closures in Swift are similar to anonymous functions, lambdas, and closures in other languages" seems reasonable. The extra item in the list acknowledges that you might have run across the same term in another language and that they're at least conceptually similar.

Your latest commit looks great!


Issue when using with RETableViewManager

I'm trying to combine the REMenu with RETableViewManager, but when I try to touch on the button that navigate to the controller use RETableViewManager, the app crashed, the Reason is: "reason: 'UINavigationControllers are not allowed in a navigation controller"
Is this an issue?
Thanks!

Two RECommonFunctions.


Allow end user to see systems in a deploy group

For example - what servers am I going to be deploying to if I choose 'misc'?
Could be a tooltip or small list under the name...

dry-run support exists in Dreadnot, but isn't used by our installations.
https://github.com/racker/chef/blob/master/site-cookbooks/ele-dreadnot/templates/default/dn-prod-settings.js.erb#L17


emplace

https://en.cppreference.com/w/cpp/container/vector/emplace

Do this before #10

use std::move_backward to move to the right https://en.cppreference.com/w/cpp/algorithm/move_backward, also in #10


Add description to Atom entry

Related to issue #928

Should we classify it as a both "open source" and "hackable"? From my perspective, "open source"  is implied if you say something is "hackable".
Therefore, I would suggest going with a simple "Hackable text editor" as the description.

I thought it was important to emphasize that Atom is open source. I consider that it is a key aspect of Atom's developmet.


CC:Tweaked/Restitched Compat

Is this compatible with Tweaked and its ability to directly connect to chests with wired modems?

Currently only compatible with Plethora's ability to control chests, nothing has been updated properly to take advantage of the newer chests features, but it is planned.


Serialization - in reference to issue #36

Coverage increased (+0.08%) to 96.536% when pulling d470ce5a8959b6d84c6da1341fb030392b022a1a on Flood1993:serialization into 92a7c1d4d0364585ae864148291523665dc076ad on opensistemas-hub:master.

Codecov Report

Merging #42 into master will increase coverage by -0.41%.

@@            Coverage Diff             @@
##           master      #42      +/-   ##
==========================================
- Coverage   94.74%   94.34%   -0.41%
==========================================
  Files           7        7
  Lines         819      866      +47
  Branches      118      135      +17
==========================================
+ Hits          776      817      +41
- Misses         29       30       +1
- Partials       14       19       +5

Impacted Files
Coverage Δ

osbrain/init.py
100% <100%> (ø)
:white_check_mark:

osbrain/address.py
95.83% <75%> (-1.95%)
:x:

osbrain/agent.py
90.35% <88%> (-0.52%)
:x:

osbrain/nameserver.py
99.09% <ø> (+1.8%)
:white_check_mark:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 92a7c1d...33cfa23. Read the comment docs.

Coverage decreased (-0.3%) to 96.11% when pulling 33cfa238054267c5bbb1b1b5a40f3be8eb6824dc on Flood1993:serialization into 92a7c1d4d0364585ae864148291523665dc076ad on opensistemas-hub:master.

Lowered the complexity of the functions. Also, the tests for checking message passing without serialization now use raw ZMQ sockets.
However, there is an issue which I have not been able to track down. In order for the tests to run OK, I've had to add explicit calls to close the ZMQ socket and destroy the ZMQ context at the end of each of the following tests (located in osbrain/tests/test_agent.py):

test_reqrep_without_serialization
test_pushpull_without_serialization
test_pubsub_without_serialization

The code that manages the freeing of the resources are the lines:
socket.close()
context.destroy()
In order to reproduce the issue, first, checkout this commit 33cfa23. Then, run pytest -k _without_serialization. The tests should run successfully. If you remove the resource freeing calls from each function, the tests won't pass (I'm getting stuck after passing successfully test_pushpull_without_serialization and test_reqrep_without_serialization, but when running the last one, test_pubsub_without_serialization, it hangs up).
Note that that without the resource freeing calls, each test runs OK individually.

Coverage decreased (-0.5%) to 95.991% when pulling ab3b212a702a55a7f111de8482fd81cbe21cc550 on Flood1993:serialization into 92a7c1d4d0364585ae864148291523665dc076ad on opensistemas-hub:master.

Coverage decreased (-0.6%) to 95.991% when pulling bd439c83ad9eb792c26be213796d537347a7464a on Flood1993:serialization into 3d00404223035db38514b22f76af43ca4809e0b8 on opensistemas-hub:master.

Thanks for the changes!
Would you mind...?

Removing the pytest file.
Moving agent tests to a new file (i.e.: test_agent_serialization.py).
Having a look at why lines 847, 848, 864, 865 are not being covered.

Thanks a lot! :blush:

Coverage decreased (-0.5%) to 96.105% when pulling e7f8c599ec0ec293b12aa25375372f1fd5a6c569 on Flood1993:serialization into 437c303ecf846a04a1d25fab2da1eb050ae73328 on opensistemas-hub:master.

Codecov Report

Merging #42 into master will not impact coverage by -0.83%.

@@            Coverage Diff             @@
##           master      #42      +/-   ##
==========================================
- Coverage   94.74%   93.92%   -0.83%
==========================================
  Files           7        7
  Lines         819      873      +54
  Branches      118      134      +16
==========================================
+ Hits          776      820      +44
- Misses         29       34       +5
- Partials       14       19       +5

Impacted Files
Coverage Δ

osbrain/init.py
100% <100%> (ø)
:white_check_mark:

osbrain/address.py
95.83% <75%> (-1.95%)
:x:

osbrain/agent.py
89.63% <83.05%> (-1.24%)
:x:

osbrain/nameserver.py
99.09% <ø> (+1.8%)
:white_check_mark:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9484113...fb100f8. Read the comment docs.

Coverage increased (+0.1%) to 96.564% when pulling 1b209f26f1b2321a2729968a4e3c34c54a524cf5 on Flood1993:serialization into 948411356f25922ed7b00a4ff2754ec5b18f9948 on opensistemas-hub:master.

Coverage increased (+0.1%) to 96.564% when pulling 2df0a6755accf5aa7fc25eb2e942b42dc88e11e6 on Flood1993:serialization into 948411356f25922ed7b00a4ff2754ec5b18f9948 on opensistemas-hub:master.

Coverage increased (+0.2%) to 96.698% when pulling 9d75547b5142f5139ab4226453ab92d9d2cd73a4 on Flood1993:serialization into 948411356f25922ed7b00a4ff2754ec5b18f9948 on opensistemas-hub:master.

Codecov Report

Merging #42 into master will increase coverage by 0.29%.

@@            Coverage Diff             @@
##           master      #42      +/-   ##
==========================================
+ Coverage   94.74%   95.04%   +0.29%
==========================================
  Files           7        7
  Lines         819      848      +29
  Branches      118      126       +8
==========================================
+ Hits          776      806      +30
+ Misses         29       28       -1
  Partials       14       14

Impacted Files
Coverage Δ

osbrain/init.py
100% <100%> (ø)
:white_check_mark:

osbrain/address.py
97.91% <100%> (+0.13%)
:white_check_mark:

osbrain/agent.py
91.09% <97.72%> (+0.23%)
:white_check_mark:

osbrain/nameserver.py
99.09% <ø> (+1.8%)
:white_check_mark:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9484113...9d75547. Read the comment docs.


python: pipenv 11.9.0 -> 2018.5.18

Updates pipenv to latest version. Also, 11.9.0 is broken on nixos.
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[x] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[x] Fits CONTRIBUTING.md.

See also #38655

Ah, I see.


Add support for Tinkerbell as a platform

Similar to currently supported bare-metal platform, we should add support for https://github.com/tinkerbell

WIP PR is here: https://github.com/kinvolk/lokomotive/pull/392. The Terraform part for controllers is implemented.


POD-602 queries that use joins or includes+references should talk to the slav…

…e by default
https://zendesk.atlassian.net/browse/POD-602
that should make our worst queries a lot faster / have less impact ...
@osheroff @gabetax
/cc @jonmoter @bhouse

Looks like this is failing on the rails 3.2 gemfile tests.

all green now, review plz

:+1:


docs: replace "drizzle-orm/planetscale" with "drizzle-orm/planetscale…

use the current path which is "drizzle-orm/planetscale-serverless"

Thanks!


Added reg cert, fixed hashing of VKeys

Description
Adds reg cert, which behaves similarly to delegate c nothing nothing d, except that it doesn't require witnessing.
Adds HSVKey, which stores the verification key together with the hash that's provided during translation in conformance testing. This fixes the issue where the value of the VKey itself is used as the hash, which caused conformance tests to fail.
related: https://github.com/IntersectMBO/cardano-ledger/pull/4780
closes https://github.com/IntersectMBO/formal-ledger-specifications/issues/615
Checklist

[x] Commit sequence broadly makes sense and commits have useful messages
[ ] Any semantic changes to the specifications are documented in CHANGELOG.md
[x] Code is formatted according to CONTRIBUTING.md
[ ] Self-reviewed the diff

@Soupstraw before merging, could you please add descriptions for each substantial change to the CHANGELOG.md?  (especially anything that changes the ledger semantics)


Doesn't work when watching

When using watch in sage9 theme using yarn start, the production assets in dist/ are cleaned up,
hence blade-svg-sage can't find the SVGs and the page fails to load.
How can it be configured to look for the SVGs in assets directory instead?
I have to use this workaround while watching:
add_filter('bladesvg_image_path', function () {
    return '/srv/www/web/app/themes/the-theme/resources/assets/images/svg/icon';
});

@Log1x: Related roots discourse discussion: https://discourse.roots.io/t/asset-lookup-while-watching/12640

I'm going to rewrite and fix a lot of these issues here shortly as I plan on using this in a project I'm currently working on.
I think it went in the wrong direction with a couple of the recent commits. There are a few things to consider:

I merged a commit making it check the manifest for the final file name, but really; an SVG that's being read from the filesystem doesn't need to be cache busted.
I'm unsure if we should read the SVG from dist or keep it defaulted to assets since, like the above, it's being read by the file system. At that point, it'd come down to if the assets folder is being pushed to the production server or not. But related to the roots discourse discussion above, whether the SVG's should actually live with the other assets while being used inline, is another point that needs to be addressed.

v2.0.0 should take care of this.


Update dependencies

fs2-core 2.1.0
cats-collections-core 0.9.0

I haven't updated any of the plugins but would be happy to if you'd like?

Thanks a lot! Yeah upgrading plugins would be great. Tbh it would also be nice to have at least Scala-steward setup, but I have had very little time to give love to upperbound lately

I'll update the plugins and have a stab at setting up scala steward (if I can do that as the non-owner).


Fixed XSS #1

Fixed XSS #1
Issues: Cross-Site Scripting allows people to inject malicious javascript code to steal user data.
Please check if the PR fulfills these requirements

[x] It's submitted to right branch according to our branching model
[x] It's right issue type on title
[x] When resolving a specific issue, it's referenced in the PR's title (e.g. fix #xxx[,#xxx], where "xxx" is the issue number)
[ ] The commit message follows our guidelines
[x] Tests for the changes have been added (for bug fixes/features)
[ ] Docs have been added/updated (for bug fixes/features)
[ ] It does not introduce a breaking change or has description for the breaking change

Description
Bountry URL: https://www.huntr.dev/bounties/2-npm-tui-grid/
Issue: XSS
Fixed
Payload: 55'<img src=x onerror=alert(1)>
paste payload on any custom editor field and click outside to view javascript execution

Thank you for your contribution to TOAST UI product. 🎉 😘 ✨

👋 Hello, @js87zz. @Aravindha1234u has opened a PR to us with a fix for a potential vulnerability in your repository. To view the vulnerability, please refer to the bounty URL in the first comment, above. If you want this fix in your repository, a PR will automatically open once you comment:
@huntr-helper - LGTM

☎ Need further support?
Come and join us on our community Discord!

❓@js87zz - want more fixes like this?
Copy this snippet into your README.md for more vulnerability fixes in the future:
[![huntr](https://cdn.huntr.dev/huntr_security_badge_mono.svg)](https://huntr.dev)


[CT-2271] [Feature] Compute seed file hashes incrementally

Is this your first time submitting a feature request?

[X] I have read the expectations for open source contributors
[X] I have searched the existing issues, and I could not find an existing issue for this feature
[X] I am requesting a straightforward extension of existing dbt functionality, rather than a Big Idea better suited to a discussion

Describe the feature
This relates to a conversion on Slack which resulted in issue https://github.com/dbt-labs/dbt-core/issues/7117 and corresponding PR.
If we allow dbt to hash larger seed files I think these should be computed incrementally to avoid allocation of too much memory. This will be most important in CI where memory is more limited.
My idea is to add another classmethod to the FileHash dataclass which has the same signature as from_contents. This method can then be used specifically for seed file hashes and from_contents can be continued to be used alongside it.
@classmethod
  def from_path(cls, path: str, name="sha256") -> "FileHash":
      """Create a file hash from the file at given path.
      """
      pass

Describe alternatives you've considered
The alternative is to go with what we have which could spike memory usage for those who override the default limit and have a lot of large seed files. Yes this could be an anti pattern but there's no reason to limit the user like that here.
Who will this benefit?
I would argue that the default limit should be increased from 1 MB but as we have the environment variable override possibility in above PR I would be fine with keeping the default at 1 MB for now.
Therefore, the most benefit are more advanced projects which will override the 1 MB limit with the environment variable.
Are you interested in contributing this feature?
Yes, I will create the PR
Anything else?
No response

Thanks for opening this issue and the associated PR @noppaz ! Excited to see you and @acurtis-evi collaborate and combine the complementary pieces.
Related to https://github.com/dbt-labs/dbt-core/issues/6875
Current behavior
From the caveats to state comparison for seeds:

dbt stores a file hash of seed files that are <1 MiB in size. If the contents of these seeds is modified, the seed will be included in state:modified.

Proposed behavior
Lightly edited from here:

add a DBT_MAXIMUM_SEED_SIZE env variable (expressed in MiB)
default to 1
setting DBT_MAXIMUM_SEED_SIZE to 0 would remove the limit entirely

i.e., if the derived constant MAXIMUM_SEED_SIZE is 0, then there is no limit to the seed size

compute file hashes incrementally for larger seed files

Functional approval
@jtcohen6 indicated the following in the discussion in Slack:

Ok! I'm not strictly opposed to either/both of:

making this limit configurable (you accept slowness as trade-off)
updating the logic to better handle larger files — if one of you would be interested in contributing that (not something we'd be prioritizing soon ourselves)

Acceptance criteria

enable configuration of the maximum seed size

I'm guessing we'll want a corresponding click CLI option for any new environment variable (like here and here)

compute hash incrementally to support memory constrained environments
open an issue here to describe the new behavior

e.g. update this documentation about hashes for seeds

PR https://github.com/dbt-labs/dbt-core/pull/7125 aims to solve two issues so the linking in the development pane to the right doesn't link to the PR atm.

I've added an issue for documentation updates here https://github.com/dbt-labs/docs.getdbt.com/issues/2958


Expose p2p signature collection errors and ignore errors at correct level

Issue Link:

@eswarasai this should help with your tests.

@lucasvo This would be covered by testworld, right now there is no way to test this other than mocking, which doesn't makes sense for a p2p request.


shebang and executable paths incorrect in gitbash

When creating a rez environment with gitbash on Windows, the shebang and executable paths use a single slash which gets escaped and causes the paths to be incorrect.
Environment

OS "windows 10"
Rez version "2.109.0"
Rez python version "3.7"

To Reproduce

Open a gitbash shell on Windows
Run a rez-env

Expected behavior
Taken into a rez environment.
Actual behavior
Command not found error

I just noticed the gitbash plugin, but I can't seem to get it to work. No matter what I do I get:
C:\Users\hughetop\AppData\Local\Temp\rez_context_b1y1cklv\rez-shell.sh: line 5: c:\program: No such file or directory
The space in "program files" is messing things up, and I can't figure out what the syntax needs to be. Doubling up the backslashes like in the pull request doesn't seem to fix it. Trying to escape the space makes windows think it's another directory. Using the "/c/program\ files/..." syntax doesn't work either. This has been wracking my brain for months, but I really want to get gitbash working so I don't have to use cmd or powershell anymore.

Good point about the spaces. I think the solution might be to use DOS short paths:
Eg. /c/Program Files/Git/bin/bash.exe -> /c/PROGRA~1/Git/bin/bash.exe. I'm doing some testing now.

We had a bit of a discussion about this on Slack already. This needs to be fixed for good as this is only the default but other paths might also need escaping. We need proper cross platform escaping to really fix this in my eyes.

(Adding quotes works in gitbash but not in rez. I think that they are stripped by rez when sanitizing?)

Interesting, I tried using that DOS shortpath and it got a different error. So my find_executable is just:
return r'C:\PROGRA~1\Git\bin\bash.exe'.replace('\\', '\\\\')
and it'll recognize gitbash, but it doesn't seem to start bash properly. The bash commands aren't available.
$ rez-env --shell gitbash python-3.9
bash: mkdir: command not found

You are now in a rez-configured environment.

resolved by hughetop@********, on Fri Apr 29 12:52:12 2022, using Rez v2.103.4

requested packages:
python
~platform==windows           (implicit)
~arch==AMD64                 (implicit)
~os==windows-10.0.19042.SP0  (implicit)

resolved packages:
python-3.9.10  \\some\network\path\packages\ext\python\3.9.10
>
hughetop@******** MINGW64 /c/Users/hughetop
$ ls
bash: ls: command not found

It seems to be because the path is using ; as pathsep, not :
$ echo $PATH
/c/rez/Scripts/rez:C:\Program Files\Python39;C:\Program Files\Python39\Scripts\;C:\Program Files\Git\cmd;/mingw64/bin;/usr/bin;/c/Users/hughetop/bin;C:\rez\Scripts\rez

There were other unrelated syspaths in there but I removed them to shorten the example. Whereas if I exit rez and echo $PATH:
$ echo $PATH
/c/rez/Scripts/rez:/c/Users/hughetop/bin:/mingw64/bin:/usr/local/bin:/usr/bin:/bin:/mingw64/bin:/usr/bin:/c/Users/hughetop/bin:/c/Program Files/Python39/Scripts:/c/Program Files/Python39:...

It's closer than I've gotten before, but I still can't get it to handle the paths correctly.

Ah good point. Had not thought about pathsep

Just to give some quick background, when I was working on getting gitbash to work, I made some changes to the local rez install in rez/resolved_context.py:
executor_path = executor.env.PATH.value()
executor_path = re.sub(r'([a-zA-Z]):\\', r'/\g<1>/', executor_path)
executor_path = executor_path.replace('\\', '/')
executor_path = executor_path.replace(';', ':')
executor.env.PATH = executor_path

Essentially the PATH variable needs to be POSIX but PYTHONPATH needs to be DOS. It's very weird and was a huge pain in the rear.

PYTHONPATH needs to be DOS style because the python process is a windows process. PATH needs to be POSIX because of bash.
This path replacement is very fragile and is one reason why I was against it (and I'm still is).

Some reading: https://www.msys2.org/docs/filesystem-paths/

So, git-bash is an example of a shell that requires path normalization - when in a bash shell, (eg) C:\foo must be referred to as /c/foo in cases such as PATH (actually it seems to work with win paths when I test it, but utils like which give weird results). Before normalization was added, paths like C:\pkgs\foo\1.2.3/bin were being constructed, and the mix of differing slashes is just odd. Recently, this issue has come up also - https://github.com/AcademySoftwareFoundation/rez/issues/1321. Ironically, it might mean that we need to enforce forward slashes on win, which we would do as part of path normalization.
So I don't see how we can avoid normalization - if we do, we're simply making package commands unintentionally platform and/or shell- specific (and the whole idea is not to be!). However, PYTHONPATH is a case where an env-var is going to be ingested by a win proc that knows/cares nothing about the shell it was launched from, so normalizing that path is absolutely a problem.
Is this not just a case then of changing the default value of the new setting https://github.com/AcademySoftwareFoundation/rez/wiki/Configuring-Rez#pathed_env_vars to ["PATH"] instead of ["*PATH"]?
In any case, I don't think fixing this is as simple as adding the skip_root_normalization setting as per https://github.com/AcademySoftwareFoundation/rez/pull/1303 (I'm also not clear on why/how that PR fixes the issue described here - please explain this in the PR!).
Currently I'm a little confused about where we are on this as there seem to be multiple overlapping things going on:

This issue says there's an escaping issue causing shebang not to work in git-bash, yet the unit tests were passing already? Why are they passing for git-bash?
Someone else has mentioned a problem with space in path ala C:\Program Files, is that the actual root of the problem? We should add a unit test for this.
We've highlighted cases where normalization should/should not occur (ie the PATH/PYTHONPATH example), is that a question of changing default pathed_env_vars value, or is there a more fundamental problem here? Also, since the unit tests didn't pick this up, we should update them so they do - we've identified a problem, we should add a test to fail on that problem rather than just jumping in and fixing it, lest the fix regress in a later version.

Thinking on this further, I think there's some more things to take into consideration:

Currently, {root} or this.root in a package commands is giving the shell-normalized path. But I can see this being a problem for (eg) env.PYTHONPATH.append("{root}/python"), since as mentioned previously, PYTHONPATH should be in the form expect by the platform, not the shell. We need tests to pick this up. I bet they aren't now though because I didn't want python to be a requirement of the unit tests, since this would mean needing to provide a rezified python. Not sure what to do about this.
We might need to have the concept of platform path normalization vs shell path normalization (consider that we may have to fwd-slash on win wrt #1321). Perhaps we need to have separate pathed_env_vars config settings - one global, one per shell (all optional). The shell-specific setting would apply shell path norm, but the global would apply platform norm (and shell would be higher priority).

Pinging this thread as would be good to get movement.
@JeanChristopheMorinPerso - you mention that "This path replacement is very fragile and is one reason why I was against it (and I'm still is)." - but the git-bash shell requires this. Is there something else you had in mind?
@amorphousWaste - clarification still needed on #1303 as to specifically what problem it fixes and how. Is it actually the space in path (ala Program Files) causing the problem?
It seems to me that PATH might be the only var that needs to be shell-normalised. For all other cases (eg PYTHONPATH), it's platform normalization that should happen, and it seems likely that this could simply default to pathlib.Path.as_posix behaviour.
What I have in mind is something like this (config settings):
# apply platform-specific path normalization
platform_pathed_env_vars = [
  "*PATH"
]

# apply shell-specific path normalization
# note that most shells simply apply platform normalization (git-bash is a notable exception)
shell_pathed_env_vars = [
  "PATH"
]

@nerdvegas I completely forgot to explain what I was thinking about. Sorry for that.
I think we could probably give https://www.msys2.org/docs/filesystem-paths/ a read (and I encourage everyone to do it). That's the first thing I'd do. I have a feeling it's a "solved" problem. I'd removed the pathed_env_vars setting all together at the global level and rely on cygpath to convert paths for us. It even has a --path that is specifically designed for converting path lists in environment variables. It knows about the mapping between windows drive and posix mounts and it probably handles corner cases that we haven't thought about or we simply don't know they even exist.

It seems to me that PATH might be the only var that needs to be shell-normalised.

Unfortunately not. What about rez packages that contain bash scripts that don't call any windows executable that would stil require custom environment variables that would contain paths?
We should probably still need to take into considerations that not just PATH might need sanitization, which means we could have a per shell setting, or maybe something like this:
def commands():
    # Instead of "env.MY_CUSTOM_PATH_ENV = 'value here', we could add a new
    # 'set' method on the variables where a user could explicitly mark an environment variable as
    # safe to convert/sanitize. This would cover cases where a package contains bash scripts
    # that rely on custom environment variables that are not path lists and or
    # that does not contains the name "PATH".
    env.MY_CUSTOM_ENV_VAR.set('{root}/something', convert=True)

    # Same for append and prepend.
    env.MY_CUSTOM_ENV_VAR_OTHER.append('asd', convert=True)
    env.MY_CUSTOM_ENV_VAR_OTHER_2.prepend('asd', convert=True)

Going explicit like this would allow packages to control how environment variables should be handled. Though, I'm not sure how doable this is as I haven't played much with the internals of rex. And also note that I haven't put a huge amount of thoughts into this convert=True thing. But there is one big advantage with this approach and it's that a studio wouldn't endup with rotten/bloated rez configs filled with environment variable names that are all defined in packages that are no more used. I'm thinking of the cases where studios are afraid of removing something because they don;'t know what depends on that.
I also think the responsibility of sanitizing paths should be delegated to shell plugins. This means bash, powershell and others wouldn't be affected by any logic required by git-bash.
This would make the implementation less fragile IMO. I'd particularly be happy with a solution that uses cygpath. I'd also be happy if PATH is automatically taken care of by the git-bash plugin (if handled with cygpath).
Lastly we could reach out to the cygwin and or msys2 folks for guidance on how this could be handled.

Cheers for the link jc I just checked it out.
Generally I think we're on the right path (pun intended). A few things:

Agreed wrt default cygpath-like behaviour for handling paths. However, cygpath itself is yet another dependency, and we either would need to compile from source or rely on a system installation - neither of which I would want to do. I'm not aware of any python equivalent implementations either. We'll have to figure out what to do here, maybe just our own simplistic impl will be good enough for the moment.
We still need pathed_env_vars though. Cygpath itself has an analogy (MSYS2_ARG_CONV_EXCL), because we can't treat everything as a path (I imagine our existing *PATH default will suffice).
Agreed on the idea of convert, I have been thinking about this also, However I would call this shell=<bool>, that seems intuitive.
Sanitising paths already is delegated to shell plugins (the impl is there anyway), but yes the setting needs to be per-shell. Impl will need reworking also - specifically stuff like this.root is automatically normalized atm, and I don't think we should be doing that. What do we call the shell-specific setting? Perhaps we can just call it shell_pathed_env_vars.

One more thing. Perhaps instead of this shell=True approach, we could do something more like this (which would also match an existing convention): https://github.com/AcademySoftwareFoundation/rez/wiki/Package-Commands#literal-strings. We also really should make a distinction between platform path norm (ala cygpath) and shell norm (ala gitbash). So for eg:
def commands():
    env.PATHISH.append(as_path("{root}/bin"))
    env.FOO_PLUGIN_PATH.append(as_shell_path("{root}/foo"))

I created a new PR because I messed up the commits on the old one here: https://github.com/AcademySoftwareFoundation/rez/pull/1364
This should fix the issues with gitbash as well as addressing the concerns with pathing and using the config for that. Pathing has been moved to the individual shell plugins with different options depending on whether the plugin needs to use a shell path or differently formatted path. The config is not used at all. Please let me know if these are the sought after changes or if more work is needed.

Apologies for not looking into this yet, have not had the time recently.
I'm not clear on where we're at - my last comment in this thread was left hanging and there was no further discussion. To reiterate and tl;dr from memory:

we should make a distinction between pathed vars and shell-pathed vars
pathed vars should be a global setting; shell-pathed vars should be a per-shell setting
pathed vars should be normalized as per cygpath behaviour (posix paths are like /a/b/c; win paths are like C:/a/b/c )
path normalization should never be forced - if both settings are empty/null, no normalization occurs
normalization should not be automatically applied to internal vars such as this.root (as it is currently)

I want to be clear on agreed approach before reviewing code.
Thx
A

It makes sense to me, I've just decided to move away from gitbash so it's not as relevant for me anymore.

Hi @nerdvegas, thanks for looking into it again. I've been testing the PR and from a gitbash user's perspective, behavior has been largely addressed with the current changes except for a couple minor things which I've relayed to @amorphousWaste.
Briefly, what I'm experiencing is that the new posix paths that the gitbash plugin is normalizing to '/a/b/c' with ':' pathseps are working well except for PYTHONPATH where

OK, I think this PR should now address all the issues with gitbash pathing. It can be configured in the config.py file and implemented on a per-shell basis or not at all.

@amorphousWaste please keep the discussions related to the PR in the PR. It's hard to keep track of comments related to PRs when they are made in issues.

I never brought it to finish but I worked on this a while back:
https://github.com/AcademySoftwareFoundation/rez/issues/737
It adds missing semantics for variables to the configuration, so that platform cases can be handled centrally for each environment variable.
It can also handle some of the problems mentioned, like CMAKE behaving non-standard in Windows (uses semicolon as separator but expects posixpath-like paths) etc.
    "CMAKE_MODULE_PATH": {
        "parent_variables": "clear",

        "pathsep": ";",
        "modify_op": "posixpath",
    },

It also solves the issue of certain variables needed to be inherited or cleared from a calling environment, where rez variables land (before or after inherited variables), can make platform dependent choices, and most importantly gives that control back to the user so we don't have to hard-code anything while we can still provide sane standards.
Maybe worth revisiting the idea because I remember that we pulled back and forth to fix windows env var issues, but the problem is that certain clients have certain expectation towards certain environment variables, and they don't always coincide with what the operating system expects.
It would actually support mentioned functions to normalize/append in a particular way, but would be configurable and over-writable. Centrally because having the logic of how to add to environment variables in individual packages is a maintenance nightmare and should not differ for a single variable/environment anyway. I liked this approach better than extending the append and prepend functionality.

Hey @bfloch , thanks for remembering us of that issue! I'll have to carve some time out to read it carefully (with https://github.com/AcademySoftwareFoundation/rez/pull/739 too).

Centrally because having the logic of how to add to environment variables in individual packages is a maintenance nightmare and should not differ for a single variable/environment anyway. I liked this approach better than extending the append and prepend functionality.

Extending append or prepend or adding new functions would allow to make packages portable. It would allow to mark a string as a "shell" string, which would basically instruct the shell plugins to convert them as it sees fit. It's good to note that the packages wouldn't be aware of shell implementations. They would only mark a variable as "need to be converted properly".

Re-reading your comment - I think it is the same idea.
The difference is the level: I don't believe that every environment variable is responsibility of the shell as not every variable follows the shell concatenation and normalization rules.
My proposal spawned off the CMAKE_MODULE_PATH case which independent of platform/shell behaves by their rules.
Similarly I believe we have variables that behave under certain rules that either follow a platform (or shell).
What I wanted to achieve is similar: Have the packages only define their intent: Add to the variable - and let a global config handle the rest of understanding what it means. The global default for un-specified variables means that the semantics do not change much.
Maybe it is worth adding something along "ShellDependent" to #739 but I think that the approach of a config is not that bad.
Since environment variables are by definition a shared resource - the nice thing is that they do generally apply to all packages.
In the unlikely event that packages need to override the rules there is already a mechanism which is config overrides in packages. For package portability - I think having the choice of what the rules for normalization and concatenation are left to the config is actually helpful. If at any point we have generic packages we would be careful to ship them with reasonable defaults in the config - while system maintainers might choose to change them.
Hope you find some time to re-read both PRs. I might check if I can find some time to revisit them if the approach is deemed sound by the TSC.
Another aspect it addresses is to define how to handle parent process variables - which is outside of the scope of this issue, but also worth exploring.


186 filter to date method in gtfsinstance

Description

bbox_filter_gtfs optionally filters to a list of dates present within the gtfs calendar.
Fixes #186
Motivation and Context

Reducing GTFS archives to valid features (stops, shapes etc) on a given date appears to reduce the likelihood of encountering r5py errors when building transport network. It is currently thought that these errors are encountered due to specific fast travel issues, although further investigation and reproduction of the encountered error is ongoing.
Type of change

[ ] Bug fix (non-breaking change which fixes an issue)
[X] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)

bbox_filter_gtfs will continue to work as before, although specifying a populated list will run the gtfs_kit.feed.restrict_to_date() logic.
How Has This Been Tested?

Defensive checks and test suite included. Also a test marked with runinteg to ensure the r5py trasport network can be built from the filtered GTFS file.
Test configuration details:

OS: macos
Python version: Python 3.9.13
Java version: openjdk 11.0.19 2023-04-18 LTS
Python management system: pip

Advice for reviewer

Upon submission I noticed the test runners failing. After reviewing the fails, it became apparent that the current pytest invocation was incorrect and breaking. The changes to the CI workflows fix that and remove duplicated tests in all-os.tests.yml, where tests were being run twice:

pytest --runinteg
pytest

Checklist:

[X] My code follows the intended structure of this project
[X] I have performed a self-review of my own code
[X] I have commented my code, particularly in hard-to-understand areas
[X] I have made corresponding changes to the documentation
[X] My changes generate no new warnings
[X] I have added tests that prove my fix is effective or that my feature works
[X] New and existing unit tests pass locally with my changes
[ ] Any dependent changes have been merged and published in downstream modules

Additional comments

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (6f9ed81) 97.79% compared to head (19db3da) 97.82%.

Additional details and impacted files
@@            Coverage Diff             @@
##              dev     #190      +/-   ##
==========================================
+ Coverage   97.79%   97.82%   +0.02%
==========================================
  Files          14       14
  Lines        1179     1193      +14
==========================================
+ Hits         1153     1167      +14
  Misses         26       26

Flag
Coverage Δ

unittests
97.82% <100.00%> (+0.02%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Files
Coverage Δ

src/transport_performance/gtfs/gtfs_utils.py
100.00% <100.00%> (ø)

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

Hi @r-leyshon . Great work on this. Everything is working as expected. I have a few small comments however they aren't huge things to clear up.
On the r5py stuff, from eyeballing it, it looks good to me. We might however want to get someone else to quickly run the test on their machine to ensure it works.

@SergioRec @ethan-moss can anyone help complete the review by running the tests locally on their machine?

@CBROWN-ONS I have implemented all your suggestions except the final comment, which unless I have misunderstood is already the case. Many thanks for the good catches once again. Did you feel you needed a more in depth review of the tests or does a passing CI suite do enough to demonstrate the behaviour is as expected?

@CBROWN-ONS I have implemented all your suggestions except the final comment, which unless I have misunderstood is already the case. Many thanks for the good catches once again. Did you feel you needed a more in depth review of the tests or does a passing CI suite do enough to demonstrate the behaviour is as expected?

Hi Rich. After reviewing your comments on the matter, I agree that we are fine not to accept a parameter for date format while filtering the GTFS.
Great work on the other changes. Regarding the r5py stuff, I'm happy to merge as the CI demonstrates that these tests pass and are operational. Let me know if you agree with this, and I will merge this into dev.

@CBROWN-ONS I have implemented all your suggestions except the final comment, which unless I have misunderstood is already the case. Many thanks for the good catches once again. Did you feel you needed a more in depth review of the tests or does a passing CI suite do enough to demonstrate the behaviour is as expected?

Hi Rich. After reviewing your comments on the matter, I agree that we are fine not to accept a parameter for date format while filtering the GTFS.
Great work on the other changes. Regarding the r5py stuff, I'm happy to merge as the CI demonstrates that these tests pass and are operational. Let me know if you agree with this, and I will merge this into dev.

Good to go from my end, thanks again.


CI: Retry linkinator

What
Checklist

[ ] I have read and agree to Contributor Guide and the Code of Conduct
[ ] I've included a screenshot or gif (if applicable)

PR Build Summary: https://build.rerun.io/pr/2299

Docs preview: https://rerun.io/preview/7c4f91f/docs

…well, except it doesn't work :P

Sorry, should've turned this into a draft. I'm in the process of forking linkinator-action and adding retry option directly into it

Okay, now it hopefully will retry a few times before failing. Of course that requires an actual failure and despite retrying the workflow a few times, it hasn't failed yet, so we'll just have to wait and see. I did also submit the changes as a PR to upstream https://github.com/JustinBeckwith/linkinator-action/pull/157


Error with "az_ngw_ids" output when nat gateway is disabled

I have got the following error when trying to create only public subnets with no NAT gateways :

Error: Error refreshing state: 1 error(s) occurred:

module.public_subnets.output.az_ngw_ids: zipmap: count of keys (3) does not match count of values (0) in:

${zipmap(var.availability_zones, matchkeys(aws_nat_gateway.public..id, aws_nat_gateway.public..tags.AZ, var.availability_zones))}

There is a mismatch with zipmap because there are (here) 3 availability zones and matchkeys returns an empty list.
Here is my config :
data "aws_availability_zones" "az" {}

module "vpc" {
    source     = "git::https://github.com/cloudposse/terraform-aws-vpc.git?ref=master"
    namespace  = "${local.prefix}"
    name       = "vpc"
    stage      = "${var.environment}"
    cidr_block = "<IP_ADDRESS>/16"
}

module "public_subnets" {
    source              = "git::https://github.com/cloudposse/terraform-aws-multi-az-subnets.git?ref=master"
    namespace           = "${local.prefix}"
    stage               = "${var.environment}"
    name                = "${local.project}"
    availability_zones  = "${data.aws_availability_zones.az.names}"
    vpc_id              = "${module.vpc.vpc_id}"
    cidr_block          = "${module.vpc.vpc_cidr_block}"
    type                = "public"
    igw_id              = "${module.vpc.igw_id}"
    nat_gateway_enabled = "false"
}

Of course, if I comment out the az_ngw_ids output, everything is fine but it doesn't seem like a long term solution. I'm quite new to Terraform's syntax. Do you think about a workaround or a fix for this issue ?
Thanks

@PiWhy
It's probably because of the new version of Terraform you are using.
the module was tested some time ago, but they changed a lot in Terraform recently.
we are looking into that and will let you know ASAP.
thanks for testing!


代码运行报错

你好，当我运行代码时遇到如下错误，这个我不知道怎么解决，能帮助我一下吗？
Traceback (most recent call last):
File "/root/autodl-tmp/PAT/experiments/launch_scripts/../main.py", line 130, in
app.run(main)
File "/root/miniconda3/lib/python3.10/site-packages/absl/app.py", line 308, in run
_run_main(main, args)
File "/root/miniconda3/lib/python3.10/site-packages/absl/app.py", line 254, in _run_main
sys.exit(main(argv))
File "/root/autodl-tmp/PAT/experiments/launch_scripts/../main.py", line 105, in main
attack.run(
File "/root/autodl-tmp/PAT/experiments/launch_scripts/../../llm_attacks/base/attack_manager.py", line 1633, in run
attack.run(
File "/root/autodl-tmp/PAT/experiments/launch_scripts/../../llm_attacks/base/attack_manager.py", line 1012, in run
def_control, def_loss = self.defense_step(
File "/root/autodl-tmp/PAT/experiments/launch_scripts/../../llm_attacks/gcg/gcg_attack.py", line 309, in defense_step
self.prompts[k].update_benign_idx()
File "/root/autodl-tmp/PAT/experiments/launch_scripts/../../llm_attacks/base/attack_manager.py", line 674, in update_benign_idx
self.benign_idx = np.random.permutation(len(self._benign_prompts))
AttributeError: 'GCGPromptManager' object has no attribute '_benign_prompts'. Did you mean: 'get_benign_prompt'?

Could you provide your complete running prompt in the bash script?

Do you run the following command?
bash run_gcg_multiple.sh vicuna 15 0.2 0

The initialization of _benign_prompts can be found at attack_manager.py line 606 . Could you please print some intermediate variables here to help debug?

The parameter benign_file is not passed into attack = attack_lib.IndividualPromptAttack in main.py，Is the reason that the condition at line 605 in attack_manager.py is not being met？

作者您好，我已成功运行代码，非常感谢您的回复，希望您天天开心。

Thank you for your interest in our work🙂, if you have any other questions or find any bugs, please feel free to communicate.


fix gumballs bug by readding target to session

@pjbreaux
Issues:
Fixes #413
Problem: I introduced a bug by removing the "make target name" from
the TEST_SESSION name.  This caused tests to collide across cloud-type
and device type in autolog results, which broke gumballs.
Analysis: readding the target name fixes the issue
NOTE:  @pjbreaux thanks for diagnosing this so quickly!

Jeff, I was able to take this over.


provide detached GPG sig with releases

Gotta dust off my key and remember / google how to do this again.

Started doing this in the last release (1.1.1?)


Pipeline nodes with multiple output edges

I think I've got most of the DAG code working now.
First I used cargo xtask convert examples/sine/Runefile -o examples/debugging/Runefile.yml to convert our Sine Runefile to its YAML equivalent:
image: runicos/base

pipeline:
  # All the normal stages from our sine demo
  rand:
    capability: RAND
    outputs:
      - type: F32
        dimensions: [1]
  mod360:
    proc-block: "hotg-ai/rune#proc_blocks/modulo"
    inputs:
      - rand
    outputs:
      - type: F32
        dimensions: [1]
    args:
      modulus: 360.0
  sine:
    model: "./sinemodel.tflite"
    inputs:
      - mod360
    outputs:
      - type: F32
        dimensions: [1]
  serial:
    out: serial
    inputs:
      - sine

Then I added on a debug_output stage which will take the output from rand, mod360, and sine and log it to a separate serial channel
  debug_output:
    out: serial
    inputs:
      - rand
      - mod360
      - sine

Now when I rune build and rune run it I can see all the normal logging output
[2021-05-21T09:10:44.492Z INFO  rune::run] Running rune: examples/debugging/debugging.rune
[2021-05-21T09:10:44.514Z DEBUG rune_wasmer_runtime] Loading image
[2021-05-21T09:10:44.514Z DEBUG rune_wasmer_runtime] Instantiating the WebAssembly module
[2021-05-21T09:10:44.515Z DEBUG rune_wasmer_runtime] Loaded the Rune
[2021-05-21T09:10:44.515Z DEBUG rune_wasmer_runtime] Running the rune
[2021-05-21T09:10:44.515Z INFO  runicos_base::image] Serial: {"type_name":"f32","channel":1,"elements":[4.8617184e-2],"dimensions":[1]}

(channel 1 is the serial output)
Plus another line which logs the intermediate results as expected (indented for clarity)
[2021-05-21T09:10:44.515Z INFO  runicos_base::image] Serial: [
    {"type_name":"f32","channel":3,"elements":[-1.7984427e-24],"dimensions":[1]},
    {"type_name":"f32","channel":3,"elements":[-1.7984427e-24],"dimensions":[1]},
    {"type_name":"f32","channel":3,"elements":[4.8617184e-2],"dimensions":[1]}
]

I just ran rune build against all the Runefiles in the test-runes repo (commit e287eb3) and there were no errors so I think this is good to merge.


Remove Caddy from the list of users

Fixes #278

We liked Starlark btw. It could still make an interesting Caddy plugin, and someday is still on my list to explore :)

Obsoleted by #289.


TSServer holds file locks on build output, preventing 'clean'

TypeScript Version:  3.2.1

Search Terms:
tsserver, file lock, delete, clean, rimraf
Code
No code.
When running TSServer (as part of VSCode) with project references and an outDir specified (such that downstream project references reference the .d.ts files in the outDir), TSServer will sometimes hold locks on files, preventing build processes from "cleaning" the outDir, such as by using rimraf
Expected behavior:
Cleaning happens without error
Actual behavior:
Error: EPERM: operation not permitted, unlink 'D:\vso\client-web\obj\datetime\packages', for example, though any of various EPREM errors can occur. Closing VSCode or instructing it to restart the TSServer always resolves the issue.

It's not a 100% repro, so of course it's being difficult now. 😔 I'll try to get you a log.

Experiencing the same issue

Is there another workaround besides restarting VS Code?

Restarting just the TS Server usually fixes the issue as well.

Same issue when trying to run npm ci while vscode is running.
npm WARN prepare removing existing node_modules/ before installation
npm ERR! path <project dir>\node_modules\@types
npm ERR! code EPERM
npm ERR! errno -4048
npm ERR! syscall unlink
npm ERR! Error: EPERM: operation not permitted, unlink '<project dir>\node_modules\@types'
npm ERR!  { Error: EPERM: operation not permitted, unlink '<project dir>\node_modules\@types'
npm ERR!   cause:
npm ERR!    { Error: EPERM: operation not permitted, unlink '<project dir>\node_modules\@types'
npm ERR!      errno: -4048,
npm ERR!      code: 'EPERM',
npm ERR!      syscall: 'unlink',
npm ERR!      path: '<project dir>\\node_modules\\@types' },
npm ERR!   isOperational: true,
npm ERR!   stack: 'Error: EPERM: operation not permitted, unlink \'<project dir>\\node_modules\\@types\'',
npm ERR!   errno: -4048,
npm ERR!   code: 'EPERM',
npm ERR!   syscall: 'unlink',
npm ERR!   path: '<project dir>\\node_modules\\@types' }

it seems the TSService has a lock on the @types folder
OS
Windows
tsc -v
3.2.1
npm -v
6.7.0

This is duplicate of #29407 and marking as such