feat : Remote OCA bundle resolver

Support for OCA bundle files stored remotely (via http)
Acceptance Criteria:

Given an base URL (e.g.: https://raw.githubusercontent.com/bcgov/aries-oca-bundles/main/OCABundles/), the wallet should be able to discover/download/cache OCA bundle files as needed. Each wallet implementation may provide a different base URL.
A wallet implementation (e.g.: BC Wallet) may provide a list of bundle IDs(?) to warm up cache during app initialization

GET /all.json
  {
    "<schema_id> or <cred_def_id>":{},
    "<schema_id> or <cred_def_id>":{},
  }

GET /<schema_id>/bundle.json
  {
     "captureBase": {},
     "overlays": [
        {"capture_base": ""}
      ]
  }

GET /<schema_id>/<cred_def_id>/bundle.json
  {
     "captureBase": {},
     "overlays": [
        {"capture_base": ""}
      ]
  }

I would sugest the [GET] /all.json to return a map instead of a array, so the resolver knows what it's getting.

Just opened a draft for the remote OCA bundle resolver and the extraction of the current bifold OCA bundle resolver.

@thiagoromanos , some related conversation in bcgov/aries-oca-bundles#22 . @swcurran is suggesting an index file that is cacheable for quick lookup. Any thoughts on that?

Here is what the generated file is like:
https://github.com/bcgov/aries-oca-bundles/blob/main/ocabundles.json

I think the index file is a great idea. That doesn't excludes the "/all.json" endpoint, right? (fetch each bundle at startup would be slower than fetching a single larger file)

Here is what the generated file is like:
https://github.com/bcgov/aries-oca-bundles/blob/main/ocabundles.json

The remote resolver is implemented and merged into bifold (#891). There is still room for upgrades, but it is able to read the index file mentioned by @swcurran and load bundles from it (just like oca explorer).


feat(comp): Dynamic completion of charts

Now targeted for the master branch, in replacement of #6706.
What this PR does / why we need it:
This PR enables dynamic shell completion for charts. This includes chart references, local charts, and URLs.

Chart references are completed using helm search repo <specified repo> to look for possible choices.
Local charts are completed using shell file completion.
URLs only have their prefix completed (see example).

The following commands are supported:
helm show <subcmd> <TAB>      - completes to chart (ref, path, url)
helm pull <TAB>               - completes to chart (ref, url)

helm install [NAME] <TAB>     - completes to chart (ref, path, url)
helm template [NAME] <TAB>    - completes to chart (ref, path, url)
helm upgrade <TAB>            - completes to release name
helm upgrade <release> <TAB>  - completes to chart (ref, path, url)

Of note is the handling the case where no prefix is specified for completion of charts as shown in the first example below.
Example 1:
> helm install name <TAB>
# provides the choices (based on my configured repos)
./      /     astronomer/   file://     gitlab/     http://    https://     stable/

In the above example:

./ and / are shown as a hint to the user that charts on the file system can be specified
astronomer/, gitlab/ and stable/ are repos currently configured
file://, http:// and https:// are the URL prefixes a user can use

Example 2:
> helm install name sta<TAB>
# uses helm repo list to complete to:
> helm install name stable/

> helm install name stable/nginx-<TAB>
# uses helm search to complete to:
stable/nginx-ingress         stable/nginx-ldapauth-proxy  stable/nginx-lego

Special notes for your reviewer:
This PR has been tested for backwards compatibility using the acceptance-testing repo.
New tests have been prepared in https://github.com/helm/acceptance-testing/pull/56

Thanks for merging!
One question: helm search repo is dramatically slower than helm search hub.  Is this normal? (I haven't looked at the code). The completion would be more user friendly if helm search repo was as fast as helm search hub I believe @mattfarina  worked on the hub part?


UVI Request

--- UVI JSON ---
{
  "vendor_name": "Linux",
  "product_name": "Kernel",
  "product_version": "versions from  to before v4.14.253",
  "vulnerability_type": "unspecified",
  "affected_component": "unspecified",
  "attack_vector": "unspecified",
  "impact": "unspecified",
  "credit": "",
  "references": [
    "https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=9b6b2db77bc3121fe435f1d4b56e34de443bec75"
  ],
  "extended_references": [
    {
      "type": "commit",
      "value": "9b6b2db77bc3121fe435f1d4b56e34de443bec75",
      "note": "fixed"
    }
  ],
  "reporter": "joshbressers",
  "reporter_id": 1692786,
  "notes": "",
  "description": "isdn: cpai: check ctr->cnr to avoid array index out of bound\n\nThis is an automated ID intended to aid in discovery of potential security vulnerabilities. The actual impact and attack plausibility have not yet been proven.\nThis ID is fixed in Linux Kernel version v4.14.253 by commit 9b6b2db77bc3121fe435f1d4b56e34de443bec75. For more details please see the references link."
}
--- UVI JSON ---

This issue has been assigned UVI-2021-1001960


Withdraw to a crypto address.

Could you add a test and document it in the README?

Was just going to do this myself :), does it really need a separate functions though? Could just handle it in the one withdraw method with different validation based on the params?

This looks good, only needs another rebase

Sorry, didn't see that this was rebased. Thanks a lot!


Bug fix: when using parking-env with Grayscale obs

When I try to use parking-env with "GrayscaleObservation", I use the config:
`config = {
   "observation": {
       "type": "GrayscaleObservation",
       "observation_shape": (128, 64),
       "stack_size": 4,
       "weights": [0.2989, 0.5870, 0.1140],  # weights for RGB conversion
       "scaling": 1.75,
   },
   "policy_frequency": 2

}`,
It return bugs that the reward and the info cannot be calculated correctly, than It try to view the source code of the gym and see the bug: the reward and info in parking-env need KinematicsGoal obs to calculate. So I add a specialized obs observation_type_parking to replace the obs in the reward and the info calculation in parking-env.

Hi @McflyWZX
Thank you for the bug report and proposed fix!
The only nit I would have is that AbstractEnv shouldn't contain the env-specific code you added. Thankfully it is easy to just move it, and I have written a quick patch for that, but I get a permission denied when trying to push it to this PR (have you unchecked Allow edits from maintainers?)
It's fine though: I can just merge this PR and push the fix afterwards.
Many thanks!

Actually, in hindsight I found it a bit convoluted to compute the reward/terminal signals from the observation. They are now directly computed from the environment state, which seems more natural to me.

Actually, in hindsight I found it a bit convoluted to compute the reward/terminal signals from the observation. They are now directly computed from the environment state, which seems more natural to me.

I see it and it's nice and elegant code. One thing more is: should the termination condition add the "Out of bounds condition"? I don't know but it's helpful for me, sometimes the car run out of the map, it seems useless to take action in that case?
I add below code to parking-env file
    def _is_terminal(self) -> bool:
        .....
        out = abs(obs[0]['observation'][0]) > 0.4 or abs(obs[0]['observation'][1]) > 0.25
        return time or crashed or success or out

Yes, I agree that adding an out of bound condition (either as terminal state or penalty) seems like a good idea.
I would use the vehicle object directly rather than the observation, though (so that the observation can be changed freely):
def _is_terminal(self) -> bool:
        .....
        in = -40 < self.vehicle.position[0] < 40 and -25 < self.vehicle.position[1] < 25
        return time or crashed or success or not in

https://github.com/eleurent/highway-env/commit/138680da624fd74c5d9cfaa06116c1f140630ffc does break the compute_reward() api (and notably all the HER implementations...)

Ah shoot, I forgot about the GoalEnv api. I will revert for now, and fix it asap.


Gradio UI: Add colors to NPMI Dataframe

My plotly idea didn't pan out 😂
Instead, I am displaying the NPMI data as an html table and then adding a drop down component where users can determine how they would like to sort the table.
Happy to change the colormap used to color the cells as well based on the user research you have done. Just went with the pandas default as a PoC!

This still does not provide for sorting, so isn't the end functionality we need. But is gorgeous and great to have it in gradio! So, I'm merging in to open up the opportunity for @EziOzoani and @NimaBoscarino to easily work with this branch as well.


Force update resource sets

Description of changes:
Builds on top of #526 and implements the first part: re-applying the objects managed by the ClusterResourceSet
This should take care of vsphere syncer and driver images, which prior to this didn't ever get updated
I added comments on the code explaining why each action is needed and what should be the long term solution
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: g-gaston
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [g-gaston]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: danbudris, g-gaston
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [danbudris,g-gaston]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@g-gaston: The following test failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

eks-anywhere-e2e-presubmit
fa3d18f2265951280c8ef190440de850f50a237e
link
/test eks-anywhere-e2e-presubmit

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@g-gaston: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

eks-anywhere-e2e-presubmit
fa3d18f2265951280c8ef190440de850f50a237e
link
/test eks-anywhere-e2e-presubmit

eks-anywhere-attribution-files-presubmit
fa3d18f2265951280c8ef190440de850f50a237e
link
/test eks-anywhere-attribution-files-presubmit

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@g-gaston: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

eks-anywhere-e2e-presubmit
fa3d18f2265951280c8ef190440de850f50a237e
link
/test eks-anywhere-e2e-presubmit

eks-anywhere-attribution-files-presubmit
fa3d18f2265951280c8ef190440de850f50a237e
link
/test eks-anywhere-attribution-files-presubmit

eks-anywhere-mocks-presubmits
fa3d18f2265951280c8ef190440de850f50a237e
link
/test eks-anywhere-mocks-presubmits

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@g-gaston: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

eks-anywhere-e2e-presubmit
fa3d18f2265951280c8ef190440de850f50a237e
link
/test eks-anywhere-e2e-presubmit

eks-anywhere-attribution-files-presubmit
fa3d18f2265951280c8ef190440de850f50a237e
link
/test eks-anywhere-attribution-files-presubmit

eks-anywhere-mocks-presubmits
fa3d18f2265951280c8ef190440de850f50a237e
link
/test eks-anywhere-mocks-presubmits

eks-anywhere-presubmit
fa3d18f2265951280c8ef190440de850f50a237e
link
/test eks-anywhere-presubmit

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/lgtm


Require release notes and migration guides in the Bevy repo as part of PRs

I also like the idea of frontloading this work. Currently I'm liking the idea of putting release note markdown somewhere in the Bevy repo and requiring it as part of the review process. The trick is that we can't / shouldn't enforce this everywhere, so how do we build the process in such a way that (1) we don't encumber non-feature changes / changes that don't need release notes and (2) we don't accidentally drop features by not forcing them through the process.
Originally posted by @cart in https://github.com/bevyengine/bevy/discussions/16431#discussioncomment-11442803

There's a CI action that checks that PRs with the label M-Needs-Migration-Guide have the proper section in their description
In the same way, we could check on a label if the release note markdown file is present on label M-Needs-Release-Note
I think it's OK to require the file to be present, but a PR shouldn't be blocked on its content being final

@mockersf.

I agree with this approach! Rather than relying on the creation of a new file, we should instead validate if a file in the appropriate folder was changed at all, to allow for flexibility in the cases where we need to update / merge into an existing release note / migration guide.
We'll need to coordinate with the bevy-website repo here, and move the contents of https://github.com/bevyengine/bevy-website/tree/main/release-content into the Bevy repo (at least going forward). Then, as we prepare the 0.16 notes, we need to change the release note / migration guide website building to use those folders as the source of truth somehow. That point was discussed more in https://github.com/bevyengine/bevy/discussions/16430.
There's other valuable work to do here, like making sure short codes work and sample code can be compiled, but that should be saved for follow-up. To start, I just want a structure for people to write notes in the bevy repo.

Rather than relying on the creation of a new file, we should instead validate if a file in the appropriate folder was changed at all, to allow for flexibility in the cases where we need to update / merge into an existing release note / migration guide.

We can check that the PR number was mentioned in a file in that folder

We'll also have to decide how to handle this folder content during RC: do we empty it once we created the release branch? or when we finally release the new version? what happens to PR needing a release note merged between those two dates?

My preference is currently to leave it in the Bevy repo permanently, and then build the site from the repo like we do for bevy_assets. Failing that though, I guess we should only remove them once the release is cut, and copy them over after each release candidate??


Windows (ARM64) Binary

I’ve only just realized that this package does not ship with an ARM64 wasmtime library for Windows - checking up stream it doesn’t appear there is any native release either, despite there being an AARCH64 wheel.
Any chance at getting an experimental cross-compilation build within this package?
Tertiary report here

Hi @andrewmd5. We'll need upstream Wasmtime to build the C API package for AARCH64 Windows first; once that artifact is available, I can update wasmtime-dotnet's build to include it.

I opened an issue here.

I was looking at the workflows and it looks like builds are already produced for this triple?
https://github.com/bytecodealliance/wasmtime/blob/d256e25065d339b922093c28333779a155a61ccb/.github/workflows/main.yml#L381

Indeed, we appear to cargo check it in CI, but traps are non-functional (I would guess we're missing aarch64 unwind information generation for Windows, but I haven't looked at that in quite some time) and tests expect traps to work.
So it looks like this would be blocked until someone has the time to implement support for Windows aarch64.


Cannot install runhouse on static cluster

I have an Ubuntu  (24.04 - LTS) machine on my local network that I would like to use as a run.house cluster. Here's a minimal snippet to reproduce:
import runhouse as rh

def get_platform(a=0):
    import platform
    return platform.platform()

cluster = rh.cluster(
    name="rh-cluster",
    host="hostname.local",  # hostname or ip address,
    ssh_creds={"ssh_user": "user", "ssh_private_key": "~/.ssh/id_rsa"},
)

if __name__ == "__main__":
    remote_get_platform = rh.function(get_platform).to(cluster)
    print(remote_get_platform())

However, I get the following error

Logs + traceback
00:38:10.554378 | Cluster rh-cluster is up, but the Runhouse API server may not be up.
INFO | 2024-08-17 00:38:10.554551 | Restarting Runhouse API server on rh-cluster.
INFO | 2024-08-17 00:38:10.555490 | Running command on rh-cluster: runhouse --version
INFO | 2024-08-17 00:38:10.737680 | Running command on rh-cluster: ray --version
INFO | 2024-08-17 00:38:10.929170 | Running command on rh-cluster: python3 -m pip install ray==2.34.0
error: externally-managed-environment

× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
    python3-xyz, where xyz is the package you are trying to
    install.

    If you wish to install a non-Debian-packaged Python package,
    create a virtual environment using python3 -m venv path/to/venv.
    Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
    sure you have python3-full installed.

    If you wish to install a non-Debian packaged Python application,
    it may be easiest to use pipx install xyz, which will manage a
    virtual environment for you. Make sure you have pipx installed.

    See /usr/share/doc/python3.12/README.venv for more information.

...
...

Traceback (most recent call last):
  File "/Users/adit/code/cradle/tech_testing/.pixi/envs/default/lib/python3.12/site-packages/requests/adapters.py", line 667, in send
    resp = conn.urlopen(
           ^^^^^^^^^^^^^
  File "/Users/adit/code/cradle/tech_testing/.pixi/envs/default/lib/python3.12/site-packages/urllib3/connectionpool.py", line 843, in urlopen
    retries = retries.increment(
              ^^^^^^^^^^^^^^^^^^
  File "/Users/adit/code/cradle/tech_testing/.pixi/envs/default/lib/python3.12/site-packages/urllib3/util/retry.py", line 474, in increment
    raise reraise(type(error), error, _stacktrace)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/adit/code/cradle/tech_testing/.pixi/envs/default/lib/python3.12/site-packages/urllib3/util/util.py", line 38, in reraise
    raise value.with_traceback(tb)
  File "/Users/adit/code/cradle/tech_testing/.pixi/envs/default/lib/python3.12/site-packages/urllib3/connectionpool.py", line 789, in urlopen
    response = self._make_request(
               ^^^^^^^^^^^^^^^^^^^
  File "/Users/adit/code/cradle/tech_testing/.pixi/envs/default/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request
    response = conn.getresponse()
               ^^^^^^^^^^^^^^^^^^
  File "/Users/adit/code/cradle/tech_testing/.pixi/envs/default/lib/python3.12/site-packages/urllib3/connection.py", line 464, in getresponse
    httplib_response = super().getresponse()
                       ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/adit/code/cradle/tech_testing/.pixi/envs/default/lib/python3.12/site-packages/sentry_sdk/integrations/stdlib.py", line 129, in getresponse
    rv = real_getresponse(self, *args, **kwargs)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/adit/code/cradle/tech_testing/.pixi/envs/default/lib/python3.12/http/client.py", line 1428, in getresponse
    response.begin()
  File "/Users/adit/code/cradle/tech_testing/.pixi/envs/default/lib/python3.12/http/client.py", line 331, in begin
    version, status, reason = self._read_status()
                              ^^^^^^^^^^^^^^^^^^^
  File "/Users/adit/code/cradle/tech_testing/.pixi/envs/default/lib/python3.12/http/client.py", line 292, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/adit/code/cradle/tech_testing/.pixi/envs/default/lib/python3.12/socket.py", line 720, in readinto
    return self._sock.recv_into(b)
           ^^^^^^^^^^^^^^^^^^^^^^^
urllib3.exceptions.ProtocolError: ('Connection aborted.', ConnectionResetError(54, 'Connection reset by peer'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/adit/code/cradle/tech_testing/.pixi/envs/default/lib/python3.12/site-packages/runhouse/resources/hardware/cluster.py", line 631, in check_and_call
    self.client.check_server()
  File "/Users/adit/code/cradle/tech_testing/.pixi/envs/default/lib/python3.12/site-packages/runhouse/servers/http/http_client.py", line 250, in check_server
    resp = session.get(
           ^^^^^^^^^^^^
  File "/Users/adit/code/cradle/tech_testing/.pixi/envs/default/lib/python3.12/site-packages/requests/sessions.py", line 602, in get
    return self.request("GET", url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/adit/code/cradle/tech_testing/.pixi/envs/default/lib/python3.12/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/adit/code/cradle/tech_testing/.pixi/envs/default/lib/python3.12/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/adit/code/cradle/tech_testing/.pixi/envs/default/lib/python3.12/site-packages/requests/adapters.py", line 682, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionResetError(54, 'Connection reset by peer'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/adit/code/cradle/tech_testing/.pixi/envs/default/lib/python3.12/site-packages/runhouse/resources/hardware/cluster.py", line 645, in call_client_method
    return check_and_call()
           ^^^^^^^^^^^^^^^^
  File "/Users/adit/code/cradle/tech_testing/.pixi/envs/default/lib/python3.12/site-packages/runhouse/resources/hardware/cluster.py", line 642, in check_and_call
    raise ConnectionError(f"Check server failed: {e}.")
ConnectionError: Check server failed: ('Connection aborted.', ConnectionResetError(54, 'Connection reset by peer')).

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/adit/code/cradle/tech_testing/runhouse_test.py", line 17, in <module>
    remote_get_platform = rh.function(get_platform).to(cluster)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/adit/code/cradle/tech_testing/.pixi/envs/default/lib/python3.12/site-packages/runhouse/resources/functions/function.py", line 95, in to
    return super().to(
           ^^^^^^^^^^^
  File "/Users/adit/code/cradle/tech_testing/.pixi/envs/default/lib/python3.12/site-packages/runhouse/resources/module.py", line 485, in to
    env = env.to(system, force_install=force_install)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/adit/code/cradle/tech_testing/.pixi/envs/default/lib/python3.12/site-packages/runhouse/resources/envs/env.py", line 238, in to
    system.call(key, "_install_reqs", reqs=new_env.reqs)
  File "/Users/adit/code/cradle/tech_testing/.pixi/envs/default/lib/python3.12/site-packages/runhouse/resources/hardware/cluster.py", line 1064, in call
    return self.call_client_method(
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/adit/code/cradle/tech_testing/.pixi/envs/default/lib/python3.12/site-packages/runhouse/resources/hardware/cluster.py", line 660, in call_client_method
    self.restart_server()
  File "/Users/adit/code/cradle/tech_testing/.pixi/envs/default/lib/python3.12/site-packages/runhouse/resources/hardware/cluster.py", line 879, in restart_server
    self._sync_runhouse_to_cluster(
  File "/Users/adit/code/cradle/tech_testing/.pixi/envs/default/lib/python3.12/site-packages/runhouse/resources/hardware/cluster.py", line 492, in _sync_runhouse_to_cluster
    raise ValueError(
ValueError: Error installing runhouse on cluster <rh-cluster> node <cradletr1.local>
Sentry is attempting to send 2 pending events

My understanding is it trying to use the base python, but Ubuntu is not letting it do a pip install in the base python environment. How can I tell it to use a virtual environment? Don't see any docs for this.

I have a temporary solution which is manually creating the env and starting the runhouse server on the machine - is that the way to go?
Also: is it possible to have multiple ssh creds, one for each ip address in the cluster if I pass in a list?

Hey Adit thanks for reaching out! You can create an env and send it to the cluster independently, or create it as part of sending the function to the cluster in the .to() call. By sending the env to the cluster Runhouse will construct and cache the env (along with its specified packages, env vars, or secrets). We also support conda envs.
We have some more general info on Envs here.
For the example provided you could do something like:
cluster_env = rh.env(reqs=[<packages to install>], name="my_env")
remote_get_platform = rh.function(get_platform).to(cluster, env=cluster_env)

Let us know if this helps.
As for SSH creds, we don’t currently support multiple SSH creds for the cluster factory. Just confirming for your use case each cluster IP would have its own set of unique creds?

Hi @jlewitt1, thanks for the response!
Regarding the env, I'm referring not to the Kaleidoscope concept of an env, but rather the python env in which the runhouse package is originally installed. When I tried to initialize a local ubuntu machine as a cluster, it tried to use the base pip (not in any virtual environment). Check out PEP 668, new versions of Ubuntu don't allow package installs on the base pip which is why this error is happening (see my traceback as well as https://www.omgubuntu.co.uk/2023/04/pip-install-error-externally-managed-environment-fix)
Yes, the use case is mainly that different clusters might have different usernames. The credentials is an easy fix since I can just use an SSH key.
Regarding the Env or CondaEnv, I ran into an install issue due to what seems like a bug in the code, see #1167.
Appreciate your help!

Gotcha ok thx for clarifying - we've actually talked about adding virtualenv or pyenv as additional Runhouse supported env types, but until then manually creating the env as you did sounds like the best way around that.
For supporting a list of creds, that's probably something we could support, do you currently have multiple VMs that you are trying to string together? curious to hear more about the use case
Also we're pushing a fix to the condaEnv issue shortly, thx for flagging!

Appreciate it!
The use-case for the multiple credentials is we want to make an ad-hoc cluster out of on-prem machines, and possibly scale up to cloud VMs as well, and these may have different creds.

Makes sense, that's also something we've talked about - assuming all the clusters are in the same VPC we could probably support that, otherwise would require a bit of hoop jumping to allow each of the worker nodes to communicate with the head node. Happy to keep you posted on the timing of that feature!
Also the conda issue is now fixed, you can install from main if that's a blocker for now until the next release


Fix identical link generation prevention for the "Defined in" section (docs)

Follow-up of #6280.
Preventing duplicates in types.cr doesn't seems to work. See: Float32.
But it works now by preventing them in generator.cr.

Is there an exaplanation why the existing solution doesn't work as expected?

I guess, in the end it doesn't really matter though. The implementation is equivalent as it applies the correct result to Crystal::Doc::Type#location. So, as long as it works, it's fine by me.

Hmm I guess it's because at the time in the add_location method, the filename and line_number instance variables of the Location aren't yet correctly modified. But they are modified now in the relative_location before the any?. For example here's how filename gets modified before the any? iteration:
filename = filename[1..-1] if filename.starts_with? File::SEPARATOR
filename = filename[4..-1] if filename.starts_with? SRC_SEP

And all these modifications of filename and line_number don't happen before the any? iteration in add_location. That's why it didn't work in add_location I think.


Ensure MsBuild and VisualStudio build solution correctly

MsBuild and VS do not work same way to build a solution.
MsBuild is building correctly in face of project order.
VisualStudio has a lot of problem - it probably scans the projects for <ProjectReference> and just do not care about  when building project graph (this is a bug).
Note that with VS2013 forcing build order fixed the problem,
With VS2015, this trick does not work anymore.

Build correctly with msbuild with correctly build order.


V0.8.8 tls backport

A backport of commits https://github.com/seanmonstar/reqwest/commit/11f8588989f6bea0c8daa59cd4ae2a91cf451dd2 and https://github.com/seanmonstar/reqwest/commit/c417d6dab823324cde59e2badeb861a3e677e958 in order to support more recent native-tls to address #336. These commits include upgrading hyper to 0.12.

It seems to break the current API. It changes some public API and it is a problem for a 0.8.x release.

True; changes related to the hyper upgrade end up having an impact on the public API here. Closing this PR as unsuitable for the 0.8 branch; I see there is a feature on master hyper011 to "ease transition", perhaps incorporating those changes will allow for this backport while maintaining compatibility.

It seems that both hyper-tls and tokio-tls would need upgrades for a native-tls upgrade to happen. They require newer hyper though and hyper stuff is being reexported, so we can't upgrade it. Thus we'd have to find a way to backport the native-tls upgrade onto older hyper-tls and tokio-tls versions. This would be a giant endeavour and not sure whether it would work out at all.
I think it'd be best to ask @seanmonstar for a soon 0.9 reqwest release so that users don't have to use it from git (some can't because they are publishing to crates.io).


Enable to filter application by its piped

What this PR does / why we need it:
Support for piped filtering of applications list with Web UI and pipectl
Which issue(s) this PR fixes:
Fixes #4212
Does this PR introduce a user-facing change?:
View example
Each application name is the same as piped name.

NONE

Cool, how about removing "All" as well as Application Name because it's not the kind of application.

Do you mean Piped name, not Application name?
The list of Piped names is implemented in Select, not Autocomplete like Application name, so it seemed difficult to specify all Piped without All.

If we filter applications with piped, how do the matched applications line up?
If it's lined up following the updated time, I suppose you need to create a custom index.

I will investigate it. Thank you for nice review.

Do you mean Piped name, not Application name?
Sorry, I mean I thought it's better not to use the "All" property, as well as the labels filter.

@knanao
Please review this PR when you have time:bow:

https://user-images.githubusercontent.com/60760935/224869782-d044ef9f-b61b-4c05-be68-54d940b1b54c.mov


Add Publishers component from SLDS

Add the Publishers component from SLDS according to the spec:
https://lightningdesignsystem.com/components/publishers/
I will submit a props proposal soon.

I shall get this merged

I shall get this merged

@pulkonet Is it ok if we do the vertical-navigation component next, instead?  If so, I'll replace this issue with vertical-navigation for this eval period.

Yes, it is completely fine with me.

For vertical-navigation, it looks like we have the base variant, but we need to modify the props to support the examples listed here:
https://lightningdesignsystem.com/components/vertical-navigation/#Examples
And also, we need to add the Radio Group variant.  Do you want to come up with the props proposal for this? Thanks!

I will open an issue for it, and add a props proposal. I shall start working on this once I post a working PR for TrialBar though.

Moving to GSoC Evaluation period two.


Fix design-time build for Antlr-generated files

Fixes https://github.com/OmniSharp/omnisharp-vscode/issues/1822
This change may fix several issues, but it particular it fixes files not appearing in the OmniSharpWorkspace that were generated through Antlr. Essentially, we had failed to set the BuildingInsideVisualStudio property when running a design-time build in OmniSharp. Antlr uses this property to determine whether the antlr design-time grammar compilation target runs (link).
In addition, there a couple of other tweaks in this change that are somewhat unrelated:

There was a type in the ProvideCommandLineArgs property.
We should run two targets: Compile and CoreCompile. This is needed to produce items from ProvideCommandLineArgs.

cc @sharwell as he may find this amusing. :smile:

CI is failing because I need to update some other tests that are now getting a couple of additional temporary files. Will do that after lunch. 😀

I've fixed the tests. CoreCompile generates a few 0-length temporary files that are unnecessary. FWIW, these are also specifically removed in Roslyn's MSBuildWorkspace.

OK. After chatting with @sharwell, it sounds like there are some known issues with the Antlr4 code generators that can make it fail on OSX (e.g. https://github.com/tunnelvisionlabs/antlr4cs/issues/244 and https://github.com/tunnelvisionlabs/antlr4cs/issues/243).
I'm going to disable this test on non-Windows for now.

I'll work to get a build up that allows the test to be enabled on all platforms.

I'll work to get a build up that allows the test to be enabled on all platforms.

No rush. I know you're super-busy. :smile:


New service type for oauth_callback_authz

Our services types are CAS and OAuth. Please add one more for "OAuth Callback Authorize". The type description in the JSON would be "oauth_callback_authz".
No changes to the form needed. Just a new type. OAuth Callback Authorize = CAS as far as changes go. Other Oauth fields should hide.

Easily done.

Is the service type fixed values or generated from the db? If fixed, I need to make fields for them in the messages table.

Fixed


Print kernel sizes too

This only concerns convolution layers. But this change will enable
computing theoretical flops just looking at the dot graph.
Not sure if anyone else will find this useful but discard if you think is not
useful.

@googlebot I signed it!

Will you be able to add a test case for the use case described? The test is also to make sure this change is compatible with existing use case.

@pavithrasv, to add test, I'll add some help. Do you know how I can test changes in python files without building Tensorflow from source? Currently, I just changed the python files in the lib folder of TensorFlow installed via pip.
I'll greatly appreciate the help :).

It has been 16 days with no activity and the awaiting response label was assigned. Is this PR still valid? Assigning the stalled label. Please comment to reassure me that this is still being worked on.

@vmiheer Any update on this PR, please. Thanks!

I am currently busy and I won't be able to work on it.
Closing it, Thank you so much @pavithrasv for help.


Banner added

What is the change?
Banner is added in the readme
Related issue?
close: #745
Checklist:
Before you create this PR, confirm all the requirements listed below by checking the checkboxes [x]:

[yes] Have you followed the Contribution Guidelines while contributing.
[yes] Have you checked there aren't other open Pull Requests for the same update/change?
[yes] Did you lint your code before making the Pull Request?
[] Have you made corresponding changes to the documentation?
[] Your submission doesn't break any existing feature.
[] Have you tested the code before submission?

do add proper level tag

Okay, I understood that. I thought of something else. As you want me to design new banner with previous format, I think first we should work on the logo, so that same logo goes in the banner, right? Thanks for such a good explanation :)

Closing the PR for now, comment on this to reopen PR if required


bug: fix order of exports for module

See issue described here: https://stackoverflow.com/a/76127619/1070523
Throwing error in next:
Module not found: Default condition should be last one
  12 |   useTheme,
  13 | } from "@mui/material";
> 14 | import { CardSwiper, CardEnterEvent, CardEvent, CardData } from "react-card-swiper";
  15 | import { Close, Star, Favorite, Replay } from "@mui/icons-material";
  16 | import { collectionsState } from "../state";
  17 | import { useRecoilState } from "recoil";

https://nextjs.org/docs/messages/module-not-found

Fix is to set correct order.

Thank you for your contribution :)


net/http: BenchmarkClientServerParallelTLS64 sometimes reports an error

Please answer these questions before submitting your issue. Thanks!
What version of Go are you using (go version)?
go version devel +cc155eb
Does this issue reproduce with the latest release?
Yes.
What operating system and processor architecture are you using (go env)?
GOARCH="arm64"
GOBIN=""
GOCACHE="/home/*/.cache/go-build"
GOEXE=""
GOHOSTARCH="arm64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/username/go"
GORACE=""
GOROOT="/home/username/go-temp"
GOTMPDIR=""
GOTOOLDIR="/home/username/go-temp/pkg/tool/linux_arm64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build212786936=/tmp/go-build -gno-record-gcc-switches"
What did you do?
I found this error in the log of our daily job (cd  /src ;   go test -count 3 -timeout 180m -run=^$ -bench=. ... ).
Running the command "go test -bench BenchmarkClientServerParallelTLS64" alone can also reproduce this error, but the error sometimes appears, sometimes does not appear, you may need to run it several times.
What did you expect to see?
PASS
What did you see instead?
I have seen two kinds of error messages. When executing command "go test -bench BenchmarkClientServerParallelTLS64" , the error message appears:
goos: linux
goarch: arm64
pkg: net/http
BenchmarkClientServerParallel4-32                  30000             37850 ns/op           12324 B/op         88 allocs/op
BenchmarkClientServerParallel64-32                 10000            307570 ns/op           13278 B/op         96 allocs/op
BenchmarkClientServerParallelTLS4-32               20000            112708 ns/op           41859 B/op        433 allocs/op
BenchmarkClientServerParallelTLS64-32           2018/03/25 13:47:49 http: TLS handshake error from <IP_ADDRESS>:58119: read tcp <IP_ADDRESS>:34212-><IP_ADDRESS>:58119: use of closed network connection
20000            106537 ns/op           43259 B/op        454 allocs/op
PASS
ok      net/http        47.081s
When running all the benchmarks ( cd  /src ;   go test -count 3 -timeout 180m -run=^$ -bench=. ... ) , in addition to the above error, another error may also occur., namely:
15:10:47 BenchmarkClientServerParallelTLS64-64          	Build timed out (after 10 minutes). Marking the build as aborted.
15:20:47 Build was aborted
I do not understand what the problem is. How did it come about? Can anyone explain it to me, thank you!

Hi, does anyone know the problem?

I can confirm this bug in my box
Linux hikey13 4.15-hikey #1 SMP PREEMPT Sat Mar 10 22:41:04 UTC 2018 aarch64 GNU/Linux
goos: linux
goarch: arm64
pkg: net/http
BenchmarkClientServerParallelTLS64-8    2018/03/29 14:04:27 http: TLS handshake erro
r from <IP_ADDRESS>:49120: read tcp <IP_ADDRESS>:33467-><IP_ADDRESS>:49120: use of closed net
work connection
2018/03/29 14:04:27 http: TLS handshake error from <IP_ADDRESS>:49090: read tcp 127.0.0
.1:33467-><IP_ADDRESS>:49090: use of closed network connection
2018/03/29 14:04:27 http: TLS handshake error from <IP_ADDRESS>:49180: read tcp 127.0.0
.1:33467-><IP_ADDRESS>:49180: use of closed network connection
2018/03/29 14:04:27 http: TLS handshake error from <IP_ADDRESS>:49042: read tcp 127.0.0
.1:33467-><IP_ADDRESS>:49042: use of closed network connection
    2000            798913 ns/op           60430 B/op        604 allocs/op
PASS
ok      net/http        3.318s

@mengzhuo Yes, the issue also exists on x86.

@mengzhuo I'm not sure, the error message comes from tlsConn.Handshake(), I don't know why there is a connection at this stage after the server is closed.

@FiloSottile Do you know the cause of this problem? I have been investigating this issue for some time, but I still do not understand it. Can you give me some suggestions?

This is the other error information I've observed.
15:22:49 BenchmarkClientServerParallel64-96             	    2000	    668230 ns/op	   21286 B/op	     124 allocs/op
15:22:51 BenchmarkClientServerParallel64-96             	    3000	    418305 ns/op	   20977 B/op	     122 allocs/op
15:22:52 BenchmarkClientServerParallelTLS4-96           	    2000	    897250 ns/op	   65399 B/op	     620 allocs/op
15:22:55 BenchmarkClientServerParallelTLS4-96           	2018/04/08 14:22:58 http: TLS handshake error from <IP_ADDRESS>:38978: read tcp <IP_ADDRESS>:39659-><IP_ADDRESS>:38978: use of closed network connection
15:22:58     2000	    878092 ns/op	   43781 B/op	     417 allocs/op
15:22:58 BenchmarkClientServerParallelTLS4-96           	    1000	   1015730 ns/op	   72051 B/op	     693 allocs/op
15:22:59 BenchmarkClientServerParallelTLS64-96          	    5000	    345304 ns/op	   78196 B/op	     755 allocs/op
15:23:04 BenchmarkClientServerParallelTLS64-96          	    5000	    211130 ns/op	   71296 B/op	     683 allocs/op
15:23:08 BenchmarkClientServerParallelTLS64-96          	   10000	   1616486 ns/op	   74765 B/op	     731 allocs/op
15:23:27 BenchmarkServer-96                             	    5000	    313404 ns/op	    2273 B/op	      20 allocs/op
15:23:29 BenchmarkServer-96                             	    5000	    315469 ns/op	    2273 B/op	      20 allocs/op
15:23:32 BenchmarkServer-96                             	    5000	    308680 ns/op	    2257 B/op	      20 allocs/op
15:23:36 --- FAIL: BenchmarkClient
15:23:36 	main_test.go:140: Test appears to have leaked a timing out dial:
15:23:36 		internal/poll.runtime_pollWait(0xfffe56cf8750, 0x77, 0xfffe56cf8758)
15:23:36 			/home/root/ci-scripts/golang/src/runtime/netpoll.go:173 +0x3c
15:23:36 		internal/poll.(*pollDesc).wait(0x4011bab598, 0x77, 0x4016f55400, 0x4016f55438, 0x4014bbba01)
15:23:36 			/home/root/ci-scripts/golang/src/internal/poll/fd_poll_runtime.go:85 +0x90
15:23:36 		internal/poll.(*pollDesc).waitWrite(0x4011bab598, 0x4011bab500, 0x0, 0x0)
15:23:36 			/home/root/ci-scripts/golang/src/internal/poll/fd_poll_runtime.go:94 +0x30
15:23:36 		internal/poll.(*FD).WaitWrite(0x4011bab580, 0x40001b0010, 0x40001b0010)
15:23:36 			/home/root/ci-scripts/golang/src/internal/poll/fd_unix.go:446 +0x2c
15:23:36 		net.(*netFD).connect(0x4011bab580, 0x8a4200, 0x40001b0010, 0x0, 0x0, 0x89fee0, 0x4014bbbac0, 0x0, 0x0, 0x0, ...)
15:23:36 			/home/root/ci-scripts/golang/src/net/fd_unix.go:152 +0x1e0
15:23:36 		net.(*netFD).dial(0x4011bab580, 0x8a4200, 0x40001b0010, 0x8a4f20, 0x0, 0x8a4f20, 0x400c7e3c50, 0x5895e8, 0x401514bbf0)
15:23:36 			/home/root/ci-scripts/golang/src/net/sock_posix.go:142 +0x98
15:23:36 		net.socket(0x8a4200, 0x40001b0010, 0x830ae7, 0x3, 0x2, 0x1, 0x0, 0x0, 0x8a4f20, 0x0, ...)
15:23:36 			/home/root/ci-scripts/golang/src/net/sock_posix.go:93 +0x124
15:23:36 		net.internetSocket(0x8a4200, 0x40001b0010, 0x830ae7, 0x3, 0x8a4f20, 0x0, 0x8a4f20, 0x400c7e3c50, 0x1, 0x0, ...)
15:23:36 			/home/root/ci-scripts/golang/src/net/ipsock_posix.go:141 +0xbc
15:23:36 		net.doDialTCP(0x8a4200, 0x40001b0010, 0x830ae7, 0x3, 0x0, 0x400c7e3c50, 0x0, 0x0, 0x0)
15:23:36 			/home/root/ci-scripts/golang/src/net/tcpsock_posix.go:62 +0x78
15:23:36 		net.dialTCP(0x8a4200, 0x40001b0010, 0x830ae7, 0x3, 0x0, 0x400c7e3c50, 0x177502b8388f53, 0x221c2526, 0x40001b0010)
15:23:36 			/home/root/ci-scripts/golang/src/net/tcpsock_posix.go:58 +0xb0
15:23:36 		net.dialSingle(0x8a4200, 0x40001b0010, 0x4011bab500, 0x8a1d00, 0x400c7e3c50, 0x0, 0x0, 0x0, 0x0)
15:23:36 			/home/root/ci-scripts/golang/src/net/dial.go:547 +0x2e0
15:23:36 		net.dialSerial(0x8a4200, 0x40001b0010, 0x4011bab500, 0x4007084530, 0x1, 0x1, 0x0, 0x0, 0x0, 0x0)
15:23:36 			/home/root/ci-scripts/golang/src/net/dial.go:515 +0x188
15:23:36 		net.(*Dialer).DialContext(0xad7480, 0x8a4200, 0x40001b0010, 0x830ae7, 0x3, 0x40091395e0, 0xf, 0x0, 0x0, 0x0, ...)
15:23:36 			/home/root/ci-scripts/golang/src/net/dial.go:397 +0x4f0
15:23:36 		net/http.(*Transport).dial(0x400c19c2d0, 0x8a4200, 0x40001b0010, 0x830ae7, 0x3, 0x40091395e0, 0xf, 0x474a38, 0x400fb8e900, 0x200000003, ...)
15:23:36 			/home/root/ci-scripts/golang/src/net/http/transport.go:906 +0x124
15:23:36 		net/http.(*Transport).dialConn(0x400c19c2d0, 0x8a4200, 0x40001b0010, 0x0, 0x40001ec700, 0x5, 0x40091395e0, 0xf, 0x40056d9c00, 0x4003398101, ...)
15:23:36 			/home/root/ci-scripts/golang/src/net/http/transport.go:1119 +0x23c
15:23:36 		net/http.(*Transport).getConn.func4(0x400c19c2d0, 0x8a4200, 0x40001b0010, 0x40045d0a80, 0x40085acae0)
15:23:36 			/home/root/ci-scripts/golang/src/net/http/transport.go:956 +0x44
15:23:36 		created by net/http.(*Transport).getConn
15:23:36 			/home/root/ci-scripts/golang/src/net/http/transport.go:955 +0x26c

Looks like this is still an issue. Besides being a bug with this specific benchmark, it's a slight annoyance since it corrupts standard library benchmark output, therefore it does not parse correctly with standard tools (golang.org/x/perf/storage/benchfmt). For some reason the error log is also going to stdout rather than stderr. Easy to work around, but it would be nice if we could clean this up.
$ go test -run 'none^' -bench . -benchtime 10ms net/http
goos: linux
goarch: amd64
pkg: net/http
BenchmarkCookieString-8                      	   23854	       539 ns/op	     176 B/op	       1 allocs/op
...
BenchmarkClientServerParallel64-8            	     532	     77916 ns/op	   12270 B/op	      86 allocs/op
BenchmarkClientServerParallelTLS4-8          	2020/05/02 14:14:00 http: TLS handshake error from <IP_ADDRESS>:57680: read tcp <IP_ADDRESS>:40263-><IP_ADDRESS>:57680: use of closed network connection
2020/05/02 14:14:00 http: TLS handshake error from <IP_ADDRESS>:57672: read tcp <IP_ADDRESS>:40263-><IP_ADDRESS>:57672: use of closed network connection
2020/05/02 14:14:00 http: TLS handshake error from <IP_ADDRESS>:57626: read tcp <IP_ADDRESS>:40263-><IP_ADDRESS>:57626: use of closed network connection
2020/05/02 14:14:00 http: TLS handshake error from <IP_ADDRESS>:57660: write tcp <IP_ADDRESS>:40263-><IP_ADDRESS>:57660: use of closed network connection
2020/05/02 14:14:00 http: TLS handshake error from <IP_ADDRESS>:57690: read tcp <IP_ADDRESS>:40263-><IP_ADDRESS>:57690: use of closed network connection
2020/05/02 14:14:00 http: TLS handshake error from <IP_ADDRESS>:57658: read tcp <IP_ADDRESS>:40263-><IP_ADDRESS>:57658: use of closed network connection
2020/05/02 14:14:00 http: TLS handshake error from <IP_ADDRESS>:57644: write tcp <IP_ADDRESS>:40263-><IP_ADDRESS>:57644: use of closed network connection
2020/05/02 14:14:00 http: TLS handshake error from <IP_ADDRESS>:57654: write tcp <IP_ADDRESS>:40263-><IP_ADDRESS>:57654: use of closed network connection
2020/05/02 14:14:00 http: TLS handshake error from <IP_ADDRESS>:42360: read tcp <IP_ADDRESS>:33587-><IP_ADDRESS>:42360: use of closed network connection
2020/05/02 14:14:00 http: TLS handshake error from <IP_ADDRESS>:42436: read tcp <IP_ADDRESS>:33587-><IP_ADDRESS>:42436: use of closed network connection
2020/05/02 14:14:00 http: TLS handshake error from <IP_ADDRESS>:42440: read tcp <IP_ADDRESS>:33587-><IP_ADDRESS>:42440: use of closed network connection
2020/05/02 14:14:00 http: TLS handshake error from <IP_ADDRESS>:42448: read tcp <IP_ADDRESS>:33587-><IP_ADDRESS>:42448: use of closed network connection
2020/05/02 14:14:00 http: TLS handshake error from <IP_ADDRESS>:42424: read tcp <IP_ADDRESS>:33587-><IP_ADDRESS>:42424: use of closed network connection
2020/05/02 14:14:00 http: TLS handshake error from <IP_ADDRESS>:42442: read tcp <IP_ADDRESS>:33587-><IP_ADDRESS>:42442: use of closed network connection
2020/05/02 14:14:00 http: TLS handshake error from <IP_ADDRESS>:42420: read tcp <IP_ADDRESS>:33587-><IP_ADDRESS>:42420: use of closed network connection
2020/05/02 14:14:00 http: TLS handshake error from <IP_ADDRESS>:42444: read tcp <IP_ADDRESS>:33587-><IP_ADDRESS>:42444: use of closed network connection
2020/05/02 14:14:00 http: TLS handshake error from <IP_ADDRESS>:42446: read tcp <IP_ADDRESS>:33587-><IP_ADDRESS>:42446: use of closed network connection
2020/05/02 14:14:00 http: TLS handshake error from <IP_ADDRESS>:42422: write tcp <IP_ADDRESS>:33587-><IP_ADDRESS>:42422: use of closed network connection
2020/05/02 14:14:00 http: TLS handshake error from <IP_ADDRESS>:42438: write tcp <IP_ADDRESS>:33587-><IP_ADDRESS>:42438: use of closed network connection
      54	    196088 ns/op	  104960 B/op	    1317 allocs/op
BenchmarkClientServerParallelTLS64-8         	2020/05/02 14:14:00 http: TLS handshake error from <IP_ADDRESS>:60164: write tcp <IP_ADDRESS>:37981-><IP_ADDRESS>:60164: use of closed network connection
2020/05/02 14:14:00 http: TLS handshake error from <IP_ADDRESS>:60166: write tcp <IP_ADDRESS>:37981-><IP_ADDRESS>:60166: use of closed network connection
2020/05/02 14:14:00 http: TLS handshake error from <IP_ADDRESS>:36562: read tcp <IP_ADDRESS>:39223-><IP_ADDRESS>:36562: use of closed network connection
      63	    253756 ns/op	  115646 B/op	    1459 allocs/op
BenchmarkServer-8                            	     156	     73795 ns/op	    2495 B/op	      20 allocs/op
...


Add support for checking stats for tar::File

Add stat() and lstat() methods for returning std::io::FileStat values for flexible checking in tar::Files.

I've just pushed a somewhat significant revamp of the API, and all information like this is currently exposed through Header. There's no way to return a raw fs::Metadata now as there's no constructor, and the files don't really have the precise equivalent of this to justify returning it right now. As a result I'm going to close this as I believe it's sufficiently covered by Header


Add SMA Muhammadiyah 1 Jakarta to List

@samdgea Please provide us with:

the school official website URL
the school street address, including city and country
an URL of a page on the official website where the school offers an IT-related long-term (>1 year) course
a proof, which shows that the school recognizes the domain you are submitting as an official email domain for the students.

@samdgea You might miss my previous comment. Please react.

@samdgea I am sorry, this request remains unanswered for a while. I have to close it without merging. Please reopen and comment if you have an update regarding the aforementioned issue. Thank you for your understanding!

@philipto hi, sorry i missed your comment, i did not receive the notification

the school official website URL: https://smamuhammadiyah1jkt.sch.id
the school street address, including city and country: Jalan Kramat Raya No.49 RT.03/RW.04 Kec. Kramat Kel. Senen - Central Jakarta - Indonesia
an URL of a page on the official website where the school offers an IT-related long-term (>1 year) course Here
a proof, which shows that the school recognizes the domain you are submitting as an official email domain for the students. Here


Issues upgrading/installing newest 2.1.0 on my Netgate 1100

I do have a Netgate 1100 and I already did have the older API 2.0.3 installed. Now I want to update to the newest 2.1.0 version, but this fails. I have depicted the output from the different steps below.
Checking the current version:
[24.03-RELEASE][admin@router]/root: pfsense-restapi version
pkg-static: Unable to open '/usr/local/etc/pkg/repos//pfSense.conf':No such file or directory
pfSense-pkg-RESTAPI-2.0_3
Name           : pfSense-pkg-RESTAPI
Version        : 2.0_3
Installed on   : Tue Jul 30 10:39:10 2024 CEST
Origin         : sysutils/pfSense-pkg-RESTAPI
Architecture   : FreeBSD:15:*
Prefix         : /usr/local
Categories     : sysutils
Licenses       : APACHE20
Maintainer<EMAIL_ADDRESS>WWW            : UNKNOWN
Comment        : pfSense REST API package
Annotations    :
Flat size      : 6.16MiB
Description    :
The missing REST API package for pfSense.

With reference to https://pfrest.org/INSTALL_AND_CONFIG/ and when I do an update, then I get this:
[24.03-RELEASE][admin@hinet]/root: pfsense-restapi update
Searching for available updates... pkg-static: Unable to open '/usr/local/etc/pkg/repos//pfSense.conf':No such file or directory
pkg-static: Unable to open '/usr/local/etc/pkg/repos//pfSense.conf':No such file or directory
 No updates available.

Hmm.. OK. Then I just did an uninstall:
[24.03-RELEASE][admin@router]/root: pfsense-restapi delete
pkg-static: Unable to open '/usr/local/etc/pkg/repos//pfSense.conf':No such file or directory
pfSense-pkg-RESTAPI-2.0_3: missing file /usr/local/www/api/v2/documentation/index.php
pfSense-pkg-RESTAPI-2.0_3: missing file /usr/local/www/api/v2/schema/index.php
pkg-static: unlinkat(usr/local/www/api/v2/schema/): No such file or directory
pkg-static: unlinkat(usr/local/www/api/v2/): No such file or directory
pkg-static: unlinkat(usr/local/www/api/): No such file or directory
pkg-static: unlinkat(usr/local/www/api/v2/documentation/): No such file or directory
pkg-static: unlinkat(usr/local/www/api/v2/): No such file or directory
pkg-static: unlinkat(usr/local/www/api/): No such file or directory
Updating database digests format: . done
Checking integrity... done (0 conflicting)
Deinstallation has been requested for the following 1 packages (of 0 packages in the universe):

Installed packages to be REMOVED:
        pfSense-pkg-RESTAPI: 2.0_3

Number of packages to be removed: 1

The operation will free 6 MiB.
[1/1] Deinstalling pfSense-pkg-RESTAPI-2.0_3...
Removing RESTAPI components...
Menu items... done.
Loading package instructions...
Configuration... done.
[1/1] Deleting files for pfSense-pkg-RESTAPI-2.0_3: .........
[1/1] Deleting files for pfSense-pkg-RESTAPI-2.0_3...
[1/1] Deleting files for pfSense-pkg-RESTAPI-2.0_3... done
Restarting webConfigurator... done.

A doublecheck confirmed that the package was no longer installed:
[24.03-RELEASE][admin@router]/root: pfsense-restapi version
pfsense-restapi: Command not found.

But now when I want to install the new 2.1.0 version (after a reboot) then I get this error:
[24.03-RELEASE][admin@router]/root: pkg-static -C /dev/null add https://github.com/jaredhendrickson13/pfsense-api/releases/latest/download/pfSense-2.7.2-pkg-RESTAPI.pkg
pkg-static: An error occured while fetching package

I don't think I am doing anything wrong here, but please correct me if I am wrong 😁

I do have a Netgate 1100 and I already did have the older API 2.0.3 installed. Now I want to update to the newest 2.1.0 version, but this fails. I have depicted the output from the different steps below.
Checking the current version:
[24.03-RELEASE][admin@router]/root: pfsense-restapi version
pkg-static: Unable to open '/usr/local/etc/pkg/repos//pfSense.conf':No such file or directory
pfSense-pkg-RESTAPI-2.0_3
Name           : pfSense-pkg-RESTAPI
Version        : 2.0_3
Installed on   : Tue Jul 30 10:39:10 2024 CEST
Origin         : sysutils/pfSense-pkg-RESTAPI
Architecture   : FreeBSD:15:*
Prefix         : /usr/local
Categories     : sysutils
Licenses       : APACHE20
Maintainer<EMAIL_ADDRESS>WWW            : UNKNOWN
Comment        : pfSense REST API package
Annotations    :
Flat size      : 6.16MiB
Description    :
The missing REST API package for pfSense.

With reference to https://pfrest.org/INSTALL_AND_CONFIG/ and when I do an update, then I get this:
[24.03-RELEASE][admin@hinet]/root: pfsense-restapi update
Searching for available updates... pkg-static: Unable to open '/usr/local/etc/pkg/repos//pfSense.conf':No such file or directory
pkg-static: Unable to open '/usr/local/etc/pkg/repos//pfSense.conf':No such file or directory
 No updates available.

Hmm.. OK. Then I just did an uninstall:
[24.03-RELEASE][admin@router]/root: pfsense-restapi delete
pkg-static: Unable to open '/usr/local/etc/pkg/repos//pfSense.conf':No such file or directory
pfSense-pkg-RESTAPI-2.0_3: missing file /usr/local/www/api/v2/documentation/index.php
pfSense-pkg-RESTAPI-2.0_3: missing file /usr/local/www/api/v2/schema/index.php
pkg-static: unlinkat(usr/local/www/api/v2/schema/): No such file or directory
pkg-static: unlinkat(usr/local/www/api/v2/): No such file or directory
pkg-static: unlinkat(usr/local/www/api/): No such file or directory
pkg-static: unlinkat(usr/local/www/api/v2/documentation/): No such file or directory
pkg-static: unlinkat(usr/local/www/api/v2/): No such file or directory
pkg-static: unlinkat(usr/local/www/api/): No such file or directory
Updating database digests format: . done
Checking integrity... done (0 conflicting)
Deinstallation has been requested for the following 1 packages (of 0 packages in the universe):

Installed packages to be REMOVED:
        pfSense-pkg-RESTAPI: 2.0_3

Number of packages to be removed: 1

The operation will free 6 MiB.
[1/1] Deinstalling pfSense-pkg-RESTAPI-2.0_3...
Removing RESTAPI components...
Menu items... done.
Loading package instructions...
Configuration... done.
[1/1] Deleting files for pfSense-pkg-RESTAPI-2.0_3: .........
[1/1] Deleting files for pfSense-pkg-RESTAPI-2.0_3...
[1/1] Deleting files for pfSense-pkg-RESTAPI-2.0_3... done
Restarting webConfigurator... done.

A doublecheck confirmed that the package was no longer installed:
[24.03-RELEASE][admin@router]/root: pfsense-restapi version
pfsense-restapi: Command not found.

But now when I want to install the new 2.1.0 version (after a reboot) then I get this error:
[24.03-RELEASE][admin@router]/root: pkg-static -C /dev/null add https://github.com/jaredhendrickson13/pfsense-api/releases/latest/download/pfSense-2.7.2-pkg-RESTAPI.pkg
pkg-static: An error occured while fetching package

I don't think I am doing anything wrong here, but please correct me if I am wrong 😁

vercel-0.2.1.tar.gz

[24.03-RELEASE][admin@router]/root: pfsense-restapi delete
pkg-static: Unable to open '/usr/local/etc/pkg/repos//pfSense.conf':No such file or directory
pfSense-pkg-RESTAPI-2.0_3: missing file /usr/local/www/api/v2/documentation/index.php
pfSense-pkg-RESTAPI-2.0_3: missing file /usr/local/www/api/v2/schema/index.php
pkg-static: unlinkat(usr/local/www/api/v2/schema/): No such file or directory
pkg-static: unlinkat(usr/local/www/api/v2/): No such file or directory
pkg-static: unlinkat(usr/local/www/api/): No such file or directory
pkg-static: unlinkat(usr/local/www/api/v2/documentation/): No such file or directory
pkg-static: unlinkat(usr/local/www/api/v2/): No such file or directory
pkg-static: unlinkat(usr/local/www/api/): No such file or directory
Updating database digests format: . done
Checking integrity... done (0 conflicting)
Deinstallation has been requested for the following 1 packages (of 0 packages in the universe):
Installed packages to be REMOVED:
pfSense-pkg-RESTAPI: 2.0_3
Number of packages to be removed: 1
The operation will free 6 MiB.
[1/1] Deinstalling pfSense-pkg-RESTAPI-2.0_3...
Removing RESTAPI components...
Menu items... done.
Loading package instructions...
Configuration... done.
[1/1] Deleting files for pfSense-pkg-RESTAPI-2.0_3: .........
[1/1] Deleting files for pfSense-pkg-RESTAPI-2.0_3...
[1/1] Deleting files for pfSense-pkg-RESTAPI-2.0_3... done
Restarting webConfigurator... done.

Why do you repost the same as I just posted? Can you delete this? :-)

This doesn't look like an issue with the package itself, something's up with the pkg environment. What happens if you run pkg-static update?

[24.03-RELEASE][admin@routert]/root: pkg-static update
Updating pfSense-core repository catalogue...
pfSense-core repository is up to date.
Updating pfSense repository catalogue...
pfSense repository is up to date.
All repositories are up to date.

While I do this, I suddenly remember why this is a problem - I must use the pfSense-24.03-pkg-RESTAPI.pkg and not the pfSense-2.7.2-pkg-RESTAPI.pkg. I think I got carried away with the easy installation instruction and simply copy/pasted the one-liner in the instruction 😂 Maybe it would be a good idea just to highlight this for people like me - something like:

Installing the package
The pfSense REST API package is built just like any other pfSense package and can therefor be installed easily using pkg from the pfSense command line:
pkg-static -C /dev/null add https://github.com/jaredhendrickson13/pfsense-api/releases/latest/download/pfSense-2.7.2-pkg-RESTAPI.pkg

Or if you are using Netgate products then:

pkg-static -C /dev/null add https://github.com/jaredhendrickson13/pfsense-api/releases/latest/download/pfSense-24.03-pkg-RESTAPI.pkg

With the correct install command, then it of course works just fine:
[24.03-RELEASE][admin@router]/root: pkg-static -C /dev/null add https://github.com/jaredhendrickson13/pfsense-api/releases/latest/download/pfSense-24.03-pkg-RESTAPI.pkg
Fetching pfSense-24.03-pkg-RESTAPI.pkg: 100%  783 KiB 801.8kB/s    00:01
Installing pfSense-pkg-RESTAPI-2.1...
Extracting pfSense-pkg-RESTAPI-2.1: 100%
Saving updated package information...
done.
Loading package configuration... done.
Configuring package components...
Loading package instructions...
Custom commands...
Menu items... done.
Writing configuration... done.
Building endpoints... done.
Building privileges... done.
Building forms... done.
Building privileges... done.
Generating OpenAPI documentation... done.
Restoring REST API configuration... done.
Starting REST API settings sync... not configured.
Configuring schedule for \RESTAPI\Caches\AvailablePackageCache... done.
Configuring schedule for \RESTAPI\Caches\RESTAPIVersionReleasesCache... done.
Initiating webConfigurator restart... done.

And pfsense-restapi version gives this:
[24.03-RELEASE][admin@router]/root: pfsense-restapi version
pfSense-pkg-RESTAPI-2.1
Name           : pfSense-pkg-RESTAPI
Version        : 2.1
Installed on   : Sat Aug 31 15:00:12 2024 CEST
Origin         : sysutils/pfSense-pkg-RESTAPI
Architecture   : FreeBSD:15:*
Prefix         : /usr/local
Categories     : sysutils
Licenses       : APACHE20
Maintainer<EMAIL_ADDRESS>WWW            : UNKNOWN
Comment        : pfSense REST API package
Annotations    :
Flat size      : 6.42MiB
Description    :
The missing REST API package for pfSense.

Closing this topic, with an urge to clarify the install command for the two options 😁

Trs: (Tanpa subjek)
Periksa Gopay Customer Saldo Rp 100.000 by Share Treats!Hanya
Rp112.550!Dapatkan di Lazada sekarang! | https://s.lazada.co.id/s.tY70E
14114212257
Periksa Gopay Customer Saldo Rp 100.000 by Share TreatsDiskon 1%!Hanya
Rp111.424!Dapatkan di Lazada sekarang! | https://s.lazada.co.id/s.tbIut
085939873260
088286732823
lazada | https://s.lazada.co.id/s.tbHav
7682918584_ID-14114212257
Periksa Gopay Customer Saldo Rp 100.000 by Share Treats!Hanya
Rp112.550!Dapatkan di Lazada sekarang! | https://s.lazada.co.id/s.tYhG5
lazada | https://s.lazada.co.id/s.tY802
Pada tanggal Sab, 31 Agu 2024 20.08, Dennis @.***>
menulis:

@.***/root: pkg-static update
Updating pfSense-core repository catalogue...
pfSense-core repository is up to date.
Updating pfSense repository catalogue...
pfSense repository is up to date.
All repositories are up to date.
While I do this, I suddenly remember why this is a problem - I must use
the pfSense-24.03-pkg-RESTAPI.pkg and not the
pfSense-2.7.2-pkg-RESTAPI.pkg. I think I got carried away with the easy
installation instruction and simply copy/pasted the one-liner in the
instruction 😂 Maybe it would be a good idea just to highlight this for
people like me - something like:
Installing the package
The pfSense REST API package is built just like any other pfSense package
and can therefor be installed easily using pkg from the pfSense command
line:
pkg-static -C /dev/null add
https://github.com/jaredhendrickson13/pfsense-api/releases/latest/download/pfSense-2.7.2-pkg-RESTAPI.pkg
Or if you are using Netgate products then:
pkg-static -C /dev/null add
https://github.com/jaredhendrickson13/pfsense-api/releases/latest/download/pfSense-24.03-pkg-RESTAPI.pkg
With the correct install command, then it of course works just fine:
@.***/root: pkg-static -C /dev/null add https://github.com/jaredhendrickson13/pfsense-api/releases/latest/download/pfSense-24.03-pkg-RESTAPI.pkg
Fetching pfSense-24.03-pkg-RESTAPI.pkg: 100%  783 KiB 801.8kB/s    00:01
Installing pfSense-pkg-RESTAPI-2.1...
Extracting pfSense-pkg-RESTAPI-2.1: 100%
Saving updated package information...
done.
Loading package configuration... done.
Configuring package components...
Loading package instructions...
Custom commands...
Menu items... done.
Writing configuration... done.
Building endpoints... done.
Building privileges... done.
Building forms... done.
Building privileges... done.
Generating OpenAPI documentation... done.
Restoring REST API configuration... done.
Starting REST API settings sync... not configured.
Configuring schedule for \RESTAPI\Caches\AvailablePackageCache... done.
Configuring schedule for \RESTAPI\Caches\RESTAPIVersionReleasesCache... done.
Initiating webConfigurator restart... done.
And pfsense-restapi version gives this:
@./root: pfsense-restapi version
pfSense-pkg-RESTAPI-2.1
Name           : pfSense-pkg-RESTAPI
Version        : 2.1
Installed on   : Sat Aug 31 15:00:12 2024 CEST
Origin         : sysutils/pfSense-pkg-RESTAPI
Architecture   : FreeBSD:15:
Prefix         : /usr/local
Categories     : sysutils
Licenses       : APACHE20
Maintainer     : @.*
WWW            : UNKNOWN
Comment        : pfSense REST API package
Annotations    :
Flat size      : 6.42MiB
Description    :
The missing REST API package for pfSense.
Closing this topic, with an urge to clarify the install command for the
two options 😁
—
Reply to this email directly, view it on GitHub
https://github.com/jaredhendrickson13/pfsense-api/issues/545#issuecomment-2322892556,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/BK5LEHFHAQMGVDASA3RXMHLZUG545AVCNFSM6AAAAABNNVPN32VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMRSHA4TENJVGY
.
You are receiving this because you commented.Message ID:
@.***>


Retiring this API next week

Just an FYI for anyone watching this repo
USDA has an API now, maybe its not great, but running this costs money and there really isn't that much use anyway

@sckott - I'm having trouble accessing the USDA API. Do you have any information about it?

see https://github.com/USDA/USDA-APIs/issues/7#issuecomment-557136892

Thanks so much!
On Thu, Sep 10, 2020 at 8:55 AM Scott Chamberlain<EMAIL_ADDRESS>wrote:

see USDA/USDA-APIs#7 (comment)
https://github.com/USDA/USDA-APIs/issues/7#issuecomment-557136892
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/sckott/usdaplantsapi/issues/15#issuecomment-690305322,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADI66DB2TE4F2JZYUSSSWN3SFDLGZANCNFSM4P7TXTFQ
.


Is it safe to iterate task list without any lock?

I'm not sure whether it's safe to traverse this list without any lock. I examined all usages and most of them are protected by rcu_read_lock() or read_lock(&tasklist_lock).
https://github.com/PacktPublishing/Linux-Kernel-Programming/blob/7c72a6e8cba86b6c527ce6467872488dcea26253/ch6/foreach/prcs_showall/prcs_showall.c#L51

Hi @iwoeruhsdfwoi , thanks for your comment.
You're right reg the need for protection.
Do understand though, that this is a book where the reader makes progress as (s)he covers each chapter.
The locking / synchronization details are left to the last two chapters...
In fact, this statement in the book in Ch 6 (pg 311 of the e-book) hints at this:

...
We print the name of the thread as well (by looking up the comm member of
the task structure). You might question why we don't use the get_task_comm() routine to obtain the task's name here; the short
reason: it causes a deadlock! We shall explore this (and how to avoid it) in detail in the later chapters on kernel synchronization. For now, again, we just do it the simple way.
...

This is again mentioned in Ch 12 pg 668.
As you've written reg this, I'll put relevant info in a comment in these examples. Thanks.


[Bug]: BigQuery Storage Write API does not fully obey ignoreUnknownValues

What happened?
Writing to BigQuery with BigQueryIO in STORAGE_WRITE_API mode leads to loss of rows if any individual datapoint in it is invalid, even when using ignoreUnknownValues. The rows are piped instead to the failed output collection and dropped entirely.
This is related to #25228 which together mean it's quite common (at least in my use-case!) for rows to be lost. In addition #25233 which makes it harder to detect.
As an example I get the following error for an optional field even with ignoreUnknownValues set:
Exception: org.apache.beam.sdk.io.gcp.bigquery.TableRowToStorageApiProto$SchemaDoesntMatchException: Unexpected value :36183.0, type: class java.lang.Double. Table field name: <snip>, type: INT64'}

I think this could be a relatively easy fix; instead of throwing a SchemaDoesNotMatch exception at the end of singularFieldToProtoValue we could return null if FieldDescriptor.isOptional is true.
Similarly some over throws would need to change to check if the field is optional. e.g.: https://github.com/apache/beam/blob/634b0453469b66ee4c135aca48b02d2425916f36/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/TableRowToStorageApiProto.java#L692-L697
Apologies if this is not a bug, but I personally classify it as "unexpected behaviour" having moved from the Streaming API where this behaviour is supported.
I'd be happy to open a PR here, but have never contributed here and Java isn't my primary language, so it may be a little slow...
Issue Priority
Priority: 2 (default / most bugs should be filed as P2)
Issue Components

[ ] Component: Python SDK
[X] Component: Java SDK
[ ] Component: Go SDK
[ ] Component: Typescript SDK
[X] Component: IO connector
[ ] Component: Beam examples
[ ] Component: Beam playground
[ ] Component: Beam katas
[ ] Component: Website
[ ] Component: Spark Runner
[ ] Component: Flink Runner
[ ] Component: Samza Runner
[ ] Component: Twister2 Runner
[ ] Component: Hazelcast Jet Runner
[ ] Component: Google Cloud Dataflow Runner

agree that the behavior should be consistent among different writing methods. @slilichenko @apilloud @ahmedabu98 @lukecwik who have involed in #24366 for opinion.

Thank you!
I've got a branch in my fork which resolves this as well as #25228. It's probably not up-to-scratch but I can open a PR if it would help?
Fix for this issue: https://github.com/bastewart/beam/commit/ef09d344678c7e6b3dc21c2492147ce7fd327e55
Fix for #25228: https://github.com/bastewart/beam/commit/38601213f81896444c60dd9e590f8a795358d09a

Doing what you suggest makes sense but we'll also need to fill in all the type conversions that we should be doing otherwise users who had failures in the past will now have successes with missing data.
@reuvenlax is there a type map that we need to support for storage write API conversions before we can enable ignoreUnknownValues fully here?

ignoreUnknownValues is defined to trigger if an unknown field name is referenced, not if there is a type conversion error.
Is there an ask to have a mode that simply nulls out invalid fields (if the field is nullable) and continues to write the record? That seems like a reasonable feature request, though I think ignoreInvalidValues is a more-appropriate name.

ignoreUnknownValues is defined to trigger if an unknown field name is referenced, not if there is a type conversion error.
Is there an ask to have a mode that simply nulls out invalid fields (if the field is nullable) and continues to write the record? That seems like a reasonable feature request, though I think ignoreInvalidValues is a more-appropriate name.

Sorry, you are right! I was slightly confused about the behaviour of BigQuery in this instance. You're absolutely right that ignoreUnknownValues only skips unrecognised field names, and does not skip invalid data points.
The real issue I'm running into is #25228. As @lukecwik says it's a type conversion problem.
I'll close this issue as, as you say, it's a different feature request to ask for ignoreInvalidValues. It's also not a feature that BQ itself offers 👍


Modularize the project and publish to NPM

Modularize the project
Description

Transform every file in nunuStudio to module
Configure webpack to build editor and runtime (prod and dev).
Publish the editor to NPM

Platform

Web Version [X]
Windows [X]
Linux [X]

Implemented!


Add a 'semigroup' parameter to reducers

When True, identity values are covered up by None. This is on by default for min and max (though it can be turned on or off for any reducer).
This feature is mentioned in #58, but ByteMaskedArray (the subject of that issue) is a prerequisite for this parameter. It's much easier to cover up values with None when you have a ByteMaskedArray than with an IndexedOptionArray.
PR #143 will be providing ByteMaskedArray, so that's a prerequisite for this issue. This issue might even be added as a checkbox for that PR. (Oh, it already is.)

Oh, and the reason why I'm writing this: #70 (argmin and argmax) has this issue as a prerequesite. This issue has to be resolved before work can start on that.


Lacking of Blockchain environment config

fdpStorage field is initialized with the defaults for reaching out RPC server for blockchain connection and ENS.
There should be a set of environments (currently Goerli and fdp-play) in form of an enum flag that provides defaults for the granular configuration (also should be possible to set by flags, at least the RpcUrl) of the ensOptions of the FdpStorage instance.

lets merge this


Fix morphTo options.query

Don't use same query builder for all candidates
Introduction
fetching event gets same query builder for all morphTo candidates.
Motivation
We're using on('fetching', ..) to add deleted_at check for relations, but this fails for morphTo relations, since opts.query is the same regardless of candidate.
Proposed solution
Remove query from options prior to sending it to sync, solves this issue.
Current PR Issues
No issues known.
Alternatives considered
No alternatives considered.

@ricardograca Added a passing test (finally) that fails if I remove my fix.

@ricardograca I changed the test setup to better fit the existing structure, please check to see if it makes sense.

@ricardograca Great! Thanks. When can we expect the next version to be published?


Replace nosetests with py.test and move tests inside the package

Switched to py.test because nose is not developed anymore. Their own site recommends using py.test.
The tests are now part of the fatiando package and are installed with it in the fatiando.tests package. This means that we can run the tests on an installed version of the code, not just the one in the repository. Users can run python -c "import fatiando; fatiando.test()" after install to run our full test suite and make sure everything works.
For development, we'll have to use:

Install the code from your clone in editable mode using pip install -e .. You don't have to set PYTHONPATH anymore with this. Alternatively use make develop.
Edit the code, make new tests, etc
Run the tests using py.test --doctest-modules --pyargs fatiando. The --pyargs tells py.test to use the version installed (which is the code from repo because of 1. above). Alternatively use make test.
Check the test coverage with py.test --doctest-modules --cov=fatiando --pyargs fatiando or make coverage.

Extra things you'll need installed are pytest and pytest-cov, both available in Anaconda.
Developers: You'll need to update your branches after this gets merged. You should then move all new tests into fatiando/tests.
Checklist:

[x] Update instructions in Developer Guide
[x] Tell users to run python -c "import fatiando; fatiando.test()" after install to test.
[x] Make tests for new code (at least 80% coverage)
[x] Create/update docstrings
[x] Include relevant equations and citations in docstrings
[x] Docstrings follow the style conventions
[x] Code follows PEP8 style conventions
[x] Code and docs have been spellchecked
[x] Include new dependencies in doc/install.rst, requirements.txt, environment.yml, and README.rst
[x] Documentation builds properly (run make docs locally)
[x] Changelog entry (leave for last)

Notification for @fatiando/core-devs.

interesting... nose tests goood byee...

@eusoubrasileiro :smile: there isn't much of a difference from what we use in Fatiando. But I took this opportunity to move the tests into the fatiando package, which I was meaning to do for quite some time. Also forced me to convert some bad doctests into proper tests.

I am your disciple! :+1: Nice

@fatiando/core-devs I'm merging this now. Please update your branches to get the new test structure. Let me know if you need any help updating/merging.


[Project-Organizer] feature request - easy to add certain facets

Use case:  I want to create a Project-Organizer folder for all projects from a certain researcher or from a certain lab or on a certain topic.  Perhaps allow for results from an elasticsearch to generate a new folder, or to allow check boxes to add to a folder.

Will be consider when we enhance the project organizer. Thanks.


As a cold wallet, where is your online terminal?

Please tell me your official website address, thank you

you can download sallet one on appale store or google play.

you can download sallet one on appale store or google play.

if you want to know more information, please visit our official website: https://www.salletone.com


Do you want to translate into java programming

Do you want to translate into java programming 😁😁🙏🙏🙏🙏

https://github.com/JicuNull/WhatsJava


Support Ruby 2.3

I found that a build against ruby-head was failing.
I changed 2 parts to fix failing specs.
Use block instead of to_proc
Test suite stops at $stdin.noecho(&:gets) with Ruby 2.3.
When I changed it to $stdin.noecho { |stdin| stdin.gets }, it worked somehow.
But I don't know why... :(
Force ERB in Thor::Actions to use String buffer
#capture depends on ERB's internal buffer. Since ERB's default buffer is changed to Array buffer in https://github.com/ruby/ruby/pull/1143, I fixed a buffer of Thor::Actions' erb to String buffer.

see issue on Ruby Issue Tracking System

I think it is better to split first commit to another PR.

I think it is better to split first commit to another PR.

I think so too. Done https://github.com/erikhuda/thor/pull/513.

:heart_eyes:

Thanks for making this change in advance of the Ruby 2.3 release. It looks like the build is passing without merging https://github.com/erikhuda/thor/pull/513. Can that be closed?

Thank you for merging this!

It looks like the build is passing without merging #513. Can that be closed?

Not sure. While builds jobs are finished in https://travis-ci.org/erikhuda/thor/builds/97829708, a build for ruby-head (Ruby 2.3) haven't been finished yet https://travis-ci.org/erikhuda/thor/jobs/97829718.
I guess it needs to update a test or implementation to finish all tests in Ruby 2.3.


invalidate cache for category stats market is created and trade histo…

…ry when order is filled
https://github.com/AugurProject/augur/issues/8295
https://github.com/AugurProject/augur/issues/8327

@Mergifyio backport dev

@mergifyio backport dev


Default value derived from entry fields

A feature request! It would be amazing to have default values derived from other fields in an entry in the same way a slug or default asset location is generated, i.e. entering {city}, {state} in the "Default" field could derive the values automatically once imported

This has been added in the latest master, thanks for the feature request!


Incompatibility with pollen. Grindstone disenchanting doesnt work, consumes book (maybe other features too, did not check)

https://gist.github.com/DDZiroWan/fa9c133472096631cba92162b46d90bc

Pollen seems to make significant changes to how the grindstone screen handler works.


VSCode deletes itself during update

Possibly a duplicate of https://github.com/Microsoft/vscode/issues/61390
On closing VSCode, an automatic update was triggered. Part way through the update, a dialog popped up claiming it could not delete a file because it was still in use and that I should close VSCode if it was still open or see if a virus scanner was accessing the file. VSCode was closed and no virus scan was happening at the time so I told it to try again. It failed again with the same error so I cancelled the update and tried to relaunch VSCode. At this time, Windows gives me an error that the target of my shortcut no longer exists. Following where the shortcut should have been pointing, I found the same mostly deleted file structure mentioned in https://github.com/Microsoft/vscode/issues/61390.

VSCode Version: Unknown, version check not possible once application deleted itself.
OS Version: Windows 10, version 1803, build 17134.345

Steps to Reproduce:

Close VSCode with an update pending.

Does this issue occur when all extensions are disabled?: Unknown. Application deleted itself preventing further testing. The following extensions were installed before self-deletion:
ascii-unicode-excape (0.1.0)
C/C++ (0.19.0)
hexdump for VSCode (1.7.0)
highlight-words (0.0.8)
Material Icon Theme (3.6.0)
Partial Diff (1.4.0)
Perforce for VS Code (3.1.0)
Python (2018.9.2)
Shell launcher (0.2.0)
Unicode code point of current character (0.1.1)
Update log is attached.
vscode-inno-updater-1540913125.log
Workaround:
In searching my temp directory for the update logs, I found the following file "%TMP%\vscode-update-user-x64\CodeSetup-stable-7f3ce96ff4729c91352ae6def877e59c561f4850.exe". Running this exe directly seems to have reinstalled vscode and preserved my user settings.

/duplicate #60697


Mockup form success state

form success could be in the structure of simple message at bottom of the form, and/or in a notification if the page is redirected.

Various states ( + a couple variations ) shown here.
https://projects.invisionapp.com/boards/8Q36SNPE6ASNW/

+1 to error/success messages below the button, button text changing and highlighting a newly created element when appropriate. Instead of adding notices in the header, I vote we stick with the progress bar and not automatically redirect on form submit so the user gets a chance to see a response in context.

thanks for feedback, sounds good.

@na3d @abemassry feel free to comment as well

closing for now, can be reopened as needed


fix load game crash when no save files are present

Fixes #222
Changes proposed in this pull request.

This codes checks for the game .dat files, and if they don't exist notify the user by the new world being created anyway.

It's probably not the most elegant solution but it works :)
Happy for feedback.

Sorry Phil. Mad's PR came first and your globals need to be updated now. I'm sure it will be easy to do.

This is the next PR and is ready to merge when the conflicts are sorted. You can see the global names need to be updated.

Midnight. Bedtime. Whoever gets to resolve this first can feel free to merge it immediately. The change is minor so zero risk. Thx.

sorry, I tested it but didn't realized that the code still is flawed because previously it would always show the "cant load game" message, and now it won't load the game because there's already a file found. I've reverted to the previous behaviour, but it's still bugged that way. I shouldn't have merged it but I believed it would be fixed with the changes. I'll go ahead and fix it so that everything works as expected to have a working code base in main.

Thanks for spring that out!


grpc.base=false does not delete grpc files if opensource.license=9

When running gt new and using the defaults and setting grpc.base to false the files are deleted properly.
When doing the same but setting opensource.license to 9 (no license) the files are still present after generating.

The issue is probably related to the return nil in https://github.com/SchwarzIT/go-template/blob/8da3b5668345d10bbcb3a874a759d237b4480d5d/pkg/gotemplate/new.go#L296-L307 which exits the loop as soon as one option is not set.
Still not sure what the exact problem is since this case should not occur.

Closed with #73


Fix #19

Multi-line signal and constant declarations now indent

Thanks for the change. Could you please change the VHDLFormatter.ts file?

Ok thats done


Add rake tasks to un/archive a course

Motivation and Context

This PR adds several rake tasks to archive, unarchive, and delete a course and all its content. The relevant tasks are:

markus:archive:course
markus:unarchive:course
markus:remove:course

The following rake tasks were also moved:

markus:archive -> markus:archive:files
markus:unarchive -> markus:unarchive:files

The archive task works by copying the database content for all records associated with a course to csv files (one per table) and all files on disk associated with the course as is. These files are saved as a tar.gz archive on disk.
The remove task works by deleting the records associated with the course from the database and deleting all files on disk associated with the course.
The unarchive task works by copying the content of the csv files to a temporary database, running any pending migrations on the data and then copying the data from that database into the database used by the current MarkUs instance. Then the archived files are copied back to the expected location on disk. In order to ensure that there are no conflicts between identifiers the unarchive task does the following:

new ids are assigned to all records and the foreign keys are updated to match
for records that are not directly associated to the course being unarchived but may be referred to by a foreign key, the correct record to refer to is identified by inspecting the IDENTIFIER constant defined on the class. This is done because ids may have changed since the course was archived. For example, a role may refer to a user and the correct user is identified by inspecting the user_name column.

In order for archiving an entire course to work, there were a few adjustments that needed to be made to the source code as well. These include:

the order that test groups are displayed in the UI was determined by storing test group ids in the assignment_properties.autotest_settings field as a json string. This acted as a foreign key reference to the test_groups that should be displayed and in which order. Updating this field within the json string when the test group ids changed during unarchiving was going to be very error prone so I updated this so that the order is stored in the test_groups table instead. If a test group has a non-null tester_index then it is displayed on the Automated Testing page in ascending order as determined by the tester_index value.
File storage locations were often stored as constants which meant that they were assigned and frozen when the model classes were initialized. In order to copy archived files to their correct location on disk, we need to be able to dynamically (and temporarily) update the file location by updating the Settings.file_storage values. In order to do this, I moved the file storage locations out of constants and into methods that will be executed dynamically.
We rely on identifying which models are directly associated with a course based on whether they respond to :course. Two new models (LtiLineItem and LtiService) needed to have a has_one :course association added. Note that this is also required if we ever create a controller for either of these models since we use this same method to verify that the course in the url is correct.
In order to determine which order to load records into the database, we assume that there are no circular dependencies between tables (ie. a graph where nodes are tables and edges are foreign key references between tables contains no cycles). However, there was one cycle present where the peer_reviews table has a reference to results and results had an optional reference back to peer_reviews. In order to resolve this I removed the (I believe) unnecessary reference to peer_reviews from results. This change is not included in this PR and is in #6397 instead. Because of this, this PR requires #6397 to be pulled in first.

Your Changes

Description:
Type of change (select all that apply):

[x] New feature (non-breaking change which adds functionality)
[x] Refactoring (internal change to codebase, without changing functionality)

Testing

Questions and Comments (if applicable)

Checklist

[x] I have performed a self-review of my own code.
[x] I have verified that the pre-commit.ci checks have passed.
[x] I have verified that the CI tests have passed.
[x] I have reviewed the test coverage changes reported on Coveralls.
[x] I have added tests for my changes.
[x] I have updated the Changelog.md file.
[x] I have made changes to the documentation and linked to that pull request below.

Pull request to make documentation changes (if applicable)

Closing pending further review.