Unhandled Exception when runnning PS Agent using C# System.Management.Automation

Empire Version
2.2
OS Information (Linux flavor, Python version)
Kali 2017.2, Python 2.7.13
Expected behavior and description of the error, including any actions taken immediately prior to the error. The more detail the better.
I want to run Empire-generated Powershell code using C# System.Management.Automation API similarly to PowerPick. Step to reproduce:

Use a simple PoC such as https://gist.github.com/bneg/bf8c05664324e3efeb1fb05902152a20
Paste the launcher generated PS code to the stager string
Build with Visual Studio on a win 7
The agent checks in, but crashes when I send the first command with the following error.

Screenshot of error, embedded text output, or Pastebin link to the error
Unhandled Exception: System.Management.Automation.CmdletInvocationException: Exc
eption calling "ToUInt32" with "2" argument(s): "Index was out of range. Must be
 non-negative and less than the size of the collection.
Parameter name: startIndex" ---> System.Management.Automation.MethodInvocationEx
ception: Exception calling "ToUInt32" with "2" argument(s): "Index was out of ra
nge. Must be non-negative and less than the size of the collection.
Parameter name: startIndex" ---> System.ArgumentOutOfRangeException: Index was o
ut of range. Must be non-negative and less than the size of the collection.
Parameter name: startIndex
   at System.ThrowHelper.ThrowArgumentOutOfRangeException(ExceptionArgument argu
ment, ExceptionResource resource)
   at System.BitConverter.ToUInt32(Byte[] value, Int32 startIndex)
   at ToUInt32(Object , Object[] )
   at System.Management.Automation.MethodInformation.Invoke(Object target, Objec
t[] arguments)
   at System.Management.Automation.DotNetAdapter.AuxiliaryMethodInvoke(Object ta
rget, Object[] arguments, MethodInformation methodInformation, Object[] original
Arguments)

Any additional information
I tried using sharppick.exe from PowerPick (https://github.com/PowerShellEmpire/PowerTools/tree/master/PowerPick) but it crashes in the same way.

@epinna Have you tried running an empire agent on the box outside of powerpick?

Yes, the same agent runs correctly if not launched with System.Management.Automation.

@epinna I would look into exception handling in c# and see if that helps. Also, try to avoid base64 encoding the stager. Just pass the plain text version to c#. This issue isn't really related to empire.


Aresite

@bgruening Running out of travis time again, any idea what we can do?

We need to kindly ask travis to extend our build time.
@jfallmann do you mind putting this tool under rnateam?

@bgruening, I don't mind, will do, should I ask Travis or do you have a red line already ;)

I ask kindly, next time its your turn ;)

So we have more build time by default - and voila it turns green.

Very cool, guess that helps a lot! Thx for your effort!


Security testing... Please ignore z zuirec

Security testing... Please ignore z zuirec

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Missing "ibutsu_utils/_version.py"

In pyproject.toml "ibutsu_utils/_version.py" is specified but there is no such file. Please add this file or replace
[tool.hatch.build.hooks.vcs]
version-file = "ibutsu_utils/_version.py"

with
[tool.hatch.version]
source = "vcs"

This is an auto-generated file. In future, I want to pull the version in from this file and display it in various places.


Emmet Update Tag does not work with JSX, but works file with HTML

From @caiolopes in https://github.com/Microsoft/vscode/issues/4962#issuecomment-279769034
The feature "updateTag" is not working when language mode is JavaScript React, only HTML.. And it is so useful for adding a class, etc, in an already existing component.

Found the root cause in emmet:
The updateTag action becomes a no-op if it cannot find a context : https://github.com/emmetio/emmet/blob/328812c879b56104b7c2bf580d79650a7c20f05b/lib/action/updateTag.js#L112
And it cannot find the context due to https://github.com/emmetio/emmet/blob/328812c879b56104b7c2bf580d79650a7c20f05b/lib/utils/action.js#L163
Allowed syntaxes should allow syntaxes that extend html as well.

The fix will be available in tomorrow's Insider build


Plate Stuck on Setup Screen

I have successfully paired my plate with HA and have downloaded the blue prints. I have tried to run automations to get the device fully set up and functioning, but it is stuck on the Wifi Connected/MQTT Connected Screen. I have tried rebooting the device and updating the firmware, but am not having any luck. Can someone please assist?
Thanks

is there a mqtt error code on the display?
what does your broker shows in the log?

No error on the display. It says wifi and mqtt are both connected.
This is the log when I reboot the device:
[08:32:53] WARNING: [WARN] Not found homeassistant on local database
1618489973: Socket error on client , disconnecting.
1618490256: New connection from <IP_ADDRESS> on port 1883.
1618490256: Socket error on client , disconnecting.
1618490258: New connection from <IP_ADDRESS> on port 1883.
1618490258: Socket error on client , disconnecting.
1618490260: New connection from <IP_ADDRESS> on port 1883.
1618490260: Socket error on client , disconnecting.
1618490264: New connection from <IP_ADDRESS> on port 1883.
1618490264: Socket error on client , disconnecting.
1618490272: New connection from <IP_ADDRESS> on port 1883.
1618490272: Socket error on client , disconnecting.
1618490288: New connection from <IP_ADDRESS> on port 1883.
[08:38:09] WARNING: [WARN] Not found homeassistant on local database
1618490289: Socket error on client , disconnecting.
1618490321: New connection from <IP_ADDRESS> on port 1883.
1618490321: Socket error on client , disconnecting.
1618490385: New connection from <IP_ADDRESS> on port 1883.
1618490385: Socket error on client , disconnecting.
1618490496: Client peskoeplate-e8db84c5e5db disconnected.
1618490498: New connection from <IP_ADDRESS> on port 1883.
[08:41:39] INFO: [INFO] found npeskoe on Home Assistant
1618490499: New client connected from <IP_ADDRESS> as peskoeplate-e8db84c5e5db (p2, c1, k30, u'npeskoe').
1618490505: New connection from <IP_ADDRESS> on port 1883.
1618490505: Socket error on client , disconnecting.
1618490625: New connection from <IP_ADDRESS> on port 1883.
[08:43:46] WARNING: [WARN] Not found homeassistant on local database
1618490626: Socket error on client , disconnecting.
1618490746: New connection from <IP_ADDRESS> on port 1883.
1618490746: Socket error on client , disconnecting.
1618490866: New connection from <IP_ADDRESS> on port 1883.
1618490866: Socket error on client , disconnecting.

Try this link@https://community.home-assistant.io/t/there-is-no-homeassistant-local-user-in-the-database/204207

Thank you very much. @damanti-me That fixed it.


travis: add 1.9.x to go versions

Coverage remained the same at 92.026% when pulling 8d919450c5a6d7f3c7526fdae8dfae83608ee962 on dvrkps:patch-1 into ebd5c6cd5ecbd85dfb00e7860b0af96e0e95bcdd on yl2chen:master.


JOSS Review: Input data format conversion

JOSS Review issue
This is not a critical review issue but I came across it while reviewing:
Is your feature request related to a problem? Please describe.
Reading phylogenies stored in foreign file formats seems not to be supported.
Describe the solution you'd like
I'm running stochastic simulations and store the resulting trajectories in a newick tree file or in a json document. Now I want to analyse my data with phylotracks. How can I get the data into phylotracks?

Thanks so much for this suggestion! @mmore500 has added support for this via his alifedata-phyloinformatics-convert library, which is designed to handle translation among the format used by phylotrackpy and more standard bioinformatics formats. We have added documentation to the quickstart guide explaining how to use it. I'm going to close this for now, but if you see ways the documentation on it could be improved let me know!


Fix example_batch  DAG to check for job queue status.

In example_batch.py was failing at certain times because disable_job_queue_func was not waiting for the job queue to be disabled.  This PR adds a check for the job queue to be disabled before it moves to the next task.
closes #413

https://github.com/astronomer/astronomer-providers/pull/500

PR title needs to summarize the change you are doing. Can you please correct it?

PR title needs to summarize the change you are doing. Can you please correct it?

Updated the PR title.

not there yet. It needs to be concise.

PR title needs to summarize the change you are doing. Can you please correct it?

Updated the PR title.

not there yet. It needs to be concise.

Changed it again.


Enhance resource comparison for collections and blobs

This update enhances the logic for comparing resources in code
generation, especially for collections (maps and arrays) and blob
types. The previous method used nil checks, but they weren't
helpful for these types.
The revised code introduces a new function, fastCompareTypes,
to generate Go code that quickly compares two objects of the same
type. For collections, it uses the built-in len function to
swiftly spot length differences before diving into detailed checks.
For blobs, the code now employs the bytes.Equal function for
straightforward byte array comparisons.
This enhancement aims to make the generated comparison logic more
direct, readable, and efficient, especially when dealing with
collections and blobs in AWS SDK shapes.
Signed-off-by: Amine Hilaly<EMAIL_ADDRESS>By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

/hold

/retest

/lgtm

/unhold


[4.6] CLOUDSTACK-8984: VPC Network offerings tab missing from UI

This is a regression from commit af2f21894ce061faadc8cec29b901719303a29dc
added vpcofferings to the select list
Before the change (from the bug):

After the change:

@karuturi Thanks! Will test soon :-)

@karuturi @remibergsma This is the second PR I look into and find that Travis is nor running any Marvin tests!
As I said in PR #997, fire few Travis buidls say this:
The command "travis_wait 30 sleep 30" exited with 0.
$ ./tools/travis/script.sh $TESTS
./tools/travis/script.sh: line 42: nosetests: command not found
+------+--------+
| Test | Result |
+======+========+
+------+--------+

And the rest say this:
/home/travis/build.sh: line 108:  2376 Terminated              travis_jigger $! $timeout $cmd
The command "travis_wait 30 sleep 30" exited with 0.
$ ./tools/travis/script.sh $TESTS
Skipping tests ... SUCCESS !

Ping @karuturi @remibergsma @miguelaferreira
Code LGTM, because it's very simple. But I will run our set of tests agains this PR - although it mentions only UI fixes - and will check the UI as well, but manually.
Cheers,
Wilder

Ping @karuturi
Testing your PR, but when I checked the UI I still got the old list: no VPC offering. Do I need to change anything else in order to get it displayed?

Git Log:
commit 9a2af2f73713e0965577fd5e6f81931fead17bd3
Author: Rajani Karuturi<EMAIL_ADDRESS>Date:   Wed Oct 28 12:35:52 2015 +0530

    CLOUDSTACK-8984: VPC Network offerings tab missing from UI

    This is a regression from commit af2f21894ce061faadc8cec29b901719303a29dc

    added vpcofferings to the select list

Cheers,
Wilder

Ping @karuturi @remibergsma
I delete the Jetty cache and restarted the Management Server, but still did not get the new item on the list.
Cache location:

@wilderrodrigues  can you clear browser cache (cmd +shift + del in firefox) or reload the page without cache using shift+f5 and then try?

Hi @karuturi
It worked!
I removed the files under Jetty cache, stopped the management server, run mvn clean install again and restarted it. It's all fine!

Thanks!
Cheers,
Wilder

Ping  @remibergsma @DaanHoogland @borisroman @wido @miguelaferreira
Thanks, @karuturi!  :) LGTM :+1:
Test results:

Hardware required: false
Management Server + MySQL on CentOS 7.1
One KVM host on CentOS 7.1
ACS Agent + Common RPMs built from source

Test start/stop of router after addition of one guest network ... === TestName: test_01_start_stop_router_after_addition_of_one_guest_network | Status : SUCCESS ===
ok
Test reboot of router after addition of one guest network ... === TestName: test_02_reboot_router_after_addition_of_one_guest_network | Status : SUCCESS ===
ok
Test to change service offering of router after addition of one guest network ... === TestName: test_04_chg_srv_off_router_after_addition_of_one_guest_network | Status : SUCCESS ===
ok
Test destroy of router after addition of one guest network ... === TestName: test_05_destroy_router_after_addition_of_one_guest_network | Status : SUCCESS ===
ok
Test to stop and start router after creation of VPC ... === TestName: test_01_stop_start_router_after_creating_vpc | Status : SUCCESS ===
ok
Test to reboot the router after creating a VPC ... === TestName: test_02_reboot_router_after_creating_vpc | Status : SUCCESS ===
ok
Tests to change service offering of the Router after ... === TestName: test_04_change_service_offerring_vpc | Status : SUCCESS ===
ok
Test to destroy the router after creating a VPC ... === TestName: test_05_destroy_router_after_creating_vpc | Status : SUCCESS ===
ok
Test router internal advanced zone ... === TestName: test_02_router_internal_adv | Status : SUCCESS ===
ok
Test restart network ... === TestName: test_03_restart_network_cleanup | Status : SUCCESS ===
ok
Test router basic setup ... === TestName: test_05_router_basic | Status : SUCCESS ===
ok
Test router advanced setup ... === TestName: test_06_router_advanced | Status : SUCCESS ===
ok
Test stop router ... === TestName: test_07_stop_router | Status : SUCCESS ===
ok
Test start router ... === TestName: test_08_start_router | Status : SUCCESS ===
ok
Test reboot router ... === TestName: test_09_reboot_router | Status : SUCCESS ===
ok
test_privategw_acl (integration.smoke.test_privategw_acl.TestPrivateGwACL) ... === TestName: test_privategw_acl | Status : SUCCESS ===
ok
Test reset virtual machine on reboot ... === TestName: test_01_reset_vm_on_reboot | Status : SUCCESS ===
ok
Test advanced zone virtual router ... === TestName: test_advZoneVirtualRouter | Status : SUCCESS ===
ok
Test Deploy Virtual Machine ... === TestName: test_deploy_vm | Status : SUCCESS ===
ok
Test Multiple Deploy Virtual Machine ... === TestName: test_deploy_vm_multiple | Status : SUCCESS ===
ok
Test Stop Virtual Machine ... === TestName: test_01_stop_vm | Status : SUCCESS ===
ok
Test Start Virtual Machine ... === TestName: test_02_start_vm | Status : SUCCESS ===
ok
Test Reboot Virtual Machine ... === TestName: test_03_reboot_vm | Status : SUCCESS ===
ok
Test destroy Virtual Machine ... === TestName: test_06_destroy_vm | Status : SUCCESS ===
ok
Test recover Virtual Machine ... === TestName: test_07_restore_vm | Status : SUCCESS ===
ok
Test migrate VM ... SKIP: At least two hosts should be present in the zone for migration
Test destroy(expunge) Virtual Machine ... === TestName: test_09_expunge_vm | Status : SUCCESS ===
ok
Test Remote Access VPN in VPC ... === TestName: test_vpc_remote_access_vpn | Status : SUCCESS ===
ok
Test VPN in VPC ... === TestName: test_vpc_site2site_vpn | Status : SUCCESS ===
ok
Test to create service offering ... === TestName: test_01_create_service_offering | Status : SUCCESS ===
ok
Test to update existing service offering ... === TestName: test_02_edit_service_offering | Status : SUCCESS ===
ok
Test to delete service offering ... === TestName: test_03_delete_service_offering | Status : SUCCESS ===
ok
Test create VPC offering ... === TestName: test_01_create_vpc_offering | Status : SUCCESS ===
ok
Test VPC offering without load balancing service ... === TestName: test_03_vpc_off_without_lb | Status : SUCCESS ===
ok
Test VPC offering without static NAT service ... === TestName: test_04_vpc_off_without_static_nat | Status : SUCCESS ===
ok
Test VPC offering without port forwarding service ... === TestName: test_05_vpc_off_without_pf | Status : SUCCESS ===
ok
Test VPC offering with invalid services ... === TestName: test_06_vpc_off_invalid_services | Status : SUCCESS ===
ok
Test update VPC offering ... === TestName: test_07_update_vpc_off | Status : SUCCESS ===
ok
Test list VPC offering ... === TestName: test_08_list_vpc_off | Status : SUCCESS ===
ok
test_09_create_redundant_vpc_offering (integration.component.test_vpc_offerings.TestVPCOffering) ... === TestName: test_09_create_redundant_vpc_offering | Status : SUCCESS ===
ok

----------------------------------------------------------------------
Ran 40 tests in 9039.022s

OK (SKIP=1)
(END)

Cheers,
Wilder

LGTM as well!

Ping @bhaisaab you may want to backport this small fix to 4.5

@remibergsma good idea, will do thanks

@remibergsma just checked this does not apply for 4.5, 4.5 has no prefilter in that section
(see screenshot on 4.5)


More issues installing from docker

Still having problems (follow up to #69)
Step 2 : ADD https://raw.githubusercontent.com/begriffs/haskell-vim-now/master/install.sh /install.sh
Downloading 3.309 kB

 ---> 5fa7f448c073
Removing intermediate container 0c9f057c93a7
Step 3 : RUN /bin/bash /install.sh && rm -r /root/.cabal
 ---> Running in b1b3d6a5b5b5
--- Installer requires Stack. Installation instructions:
--- https://github.com/commercialhaskell/stack#how-to-install

I have stack installed in my machine.
{9:47}~/Downloads/haskell-vim-now:master ✓ ➭ which stack
/Users/mitochon/Applications/ghc-7.10.2.app/Contents/bin/stack
{9:47}~/Downloads/haskell-vim-now:master ✓ ➭ command -v stack >/dev/null && echo $?
0

Does stack need to be part of the Dockerfile?

Ah yes, I just switched the project over to using Stack. In fact at this moment I was testing that it works in various Linux distros. We will need to update the Dockerfile to install stack. Would you like to help out and send a pull request?

Using the above changes was able to complete the setup.
However there's also another issue with the last command but is not a hard failure so will leave it alone for now. Looked around a little bit couldn't find the .cabal directory in the resulting docker container.
--- Setting git to use fully-pathed vim for messages...
--- Configuring codex to search in stack...
rm: cannot remove ‘/root/.cabal’: No such file or directory
The command '/bin/sh -c /bin/bash /install.sh && rm -r /root/.cabal' returned a non-zero code: 1


Dynamic column

Hello,
Your lib is great.
I've been rendering tables in few minutes.
Would it be possible to render column dynamically ?
The use case is : my users would be able to select the data they want on a report and the app would render the column related to the user selection.

Duplicate of #15


Replace get_user_profile_by_email with get_user.

@djangopdx thanks for working on this!  It looks like you accidentally committed some files in the .gitignore; I'm not entirely sure how.   Can you clean that up?
Let folks know if you need any help.

I have updated the commit can you please check?


Move shape functions with Enums

Leftover from https://github.com/openxla/stablehlo/pull/825.
FftOp::inferReturnTypeComponents(): has FftType from StablehloEnums.td
RngOp::verify(): hasRngDistribution from StablehloEnums.td

How about we turn enums into a bunch of boolean parameters, similarly to what we've done in inferTriangularSolveOp?

How about we turn enums into a bunch of boolean parameters, similarly to what we've done in inferTriangularSolveOp?

Good suggestion!

I've just noticed that there's still rngInferReturnTypeComponents still lives in StablehloOps.cpp. Let's move it to TypeInference.h as well.


Please build on trusty

https://github.com/emergent-design/libpsinc/blob/2f79d3757132d3780325f76c7d2fcc26863e394a/.gitlab-ci.yml#L95
Please do not build on Xenial because it is not the oldest still-supported LTS release, which means that the resulting binaries will not run on all still-supported LTS releases. Please consider building on trusty (the currently oldest still-supported LTS release) for increased compatibility.
Thanks.

Firstly, thank you for linuxdeployqt, it greatly simplified the process of bundling our application!
Secondly, trusty support is not really necessary at the moment - this library is for communicating with a specialist piece of hardware which is only supported on xenial and above. The Qt application included acts as both a tool and example for the library, the AppImage was recently added for the sole purpose of providing the manufacturer with an easy to run instance for testing hardware. The AppImage version is not even deployed to the public downloads at this time.

OK, thanks for the clarification.

Since xenial is now the oldest still-supported Ubuntu release, it is now a reasonable target.


Can not get tabs or model jquery plugins work in jquery 1.7

I only use tabs and model plugins however others maybe affected. None work in the latest stable version of Jquery released 2 days ago.

works for me... what's not working for you? can you provide a jsfiddle pls? thanks!

Sorry for the late response. It seems I was wrong in that it does work in jsfiddle for me however not in my current project. I will do a little more research then update the ticket.

k closing for now - reopen if you come across something


Karpenter 1.0 Helm post-install kubectl pod does not have ARM64 arch version available

Description
Observed Behavior:
When attempting to rollout karpenter in an ARM based EKS cluster, the post-install pod that's trying to pull a bitnami/kubectl image from public ECR does not have an ARM64 version available.
Expected Behavior:
An ARM64 version of all containers used for Karpenter should be available.
Reproduction Steps (Please include YAML):

Deploy the karpenter Helm chart in an ARM-based cluster
See karpenter controller pods come up
See karpenter-post-install pods fail to pull with the following error:

Warning  Failed     2m37s (x4 over 3m59s)  kubelet            Failed to pull image "public.ecr.aws/bitnami/kubectl:1.30@sha256:13a2ad1bd37ce42ee2a6f1ab0d30595f42eb7fe4a90d6ec848550524104a1ed6": rpc error: code = NotFound desc = failed to pull and unpack image "public.ecr.aws/bitnami/kubectl@sha256:13a2ad1bd37ce42ee2a6f1ab0d30595f42eb7fe4a90d6ec848550524104a1ed6": no match for platform in manifest: not found

Versions:

Chart Version: 1.0
Kubernetes Version (kubectl version): 1.29

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Just noticed this has been fixed in more recent versions, closing this out.


Fixed defect in retryInterval, improved logging

Retrying endpoints works when hitting localhost:27754 which is the endpoint for device. localhost:27753 is flakey
Added logging for IWDP and log that says when IWDP starts
Calls process.stop() on IWDP if it fails, just in case the process actually started and we don't want to leave it running
Removed startIWDP capability (that goes in XCUITestDriver)

(please reveiw @imurchie @jlipps)

Published to 1.17.1


Manually terminate the Py4JServer during engine shutdown

Why are the changes needed?

Due to the Py4JServer initiating with a non-daemon thread, there is a possibility of it impeding the engine's termination. Therefore, it is imperative to manually terminate the Py4JServer during engine shutdown.
"Thread-23" #96 prio=5 os_prio=0 cpu=7.93ms elapsed=187532.67s tid=0x00007fee840cf000 nid=0x8f runnable  [0x00007fedca6bf000]
   java.lang.Thread.State: RUNNABLE
	at<EMAIL_ADDRESS>Method)
	at<EMAIL_ADDRESS>Source)
	at<EMAIL_ADDRESS>Source)
	at<EMAIL_ADDRESS>Source)
	at py4j.GatewayServer.run(GatewayServer.java:685)
	at<EMAIL_ADDRESS>Source)

How was this patch tested?

[ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

[ ] Add screenshots for manual tests if appropriate

[x] Run test locally before make a pull request

the start and stop need to be syncronized

the start and stop need to be syncronized

The start function has already been synchronized in the caller's code, shall we need to refine this?
https://github.com/apache/kyuubi/blob/d9e14f239d5f27c3149519b70665fe36feaf0b3f/externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/kyuubi/engine/spark/operation/ExecutePython.scala#L240-L253

the start and stop need to be syncronized

updated.

Thanks, merged to master/1.7


Message: Request ended prematurely due to a closed connection Stack Trace: undefined

Describe the bug
&caption when esmbot takes too long when the when the when the sus (Trigger)
Message: Request ended prematurely due to a closed connection(Error)
Stack Trace: undefined
Steps to reproduce

Go to '...'
Click on '....'
Scroll down to '....'
See error

Expected behavior
To display a caption when esmbot takes too long when the when the when the sus
Screenshots
No
Self-hosted instance?
Yes
Additional context
No

Check the image API logs, are there any errors in there?

Closing until OP responds.


Unify Constrain Errors

Problem
This TODO was added in https://github.com/noir-lang/noir/pull/4101. This is due to having two different strategies for resolving constrain instruction error messages which can be seen in this enum (https://github.com/noir-lang/noir/blob/afcb385daa572f990178eda51faf10dc20acd2d0/compiler/noirc_evaluator/src/ssa/ir/instruction.rs#L5730). We have two different strategies as sometimes when doing SSA codegen we want to include a constrain with a message, but we can't easily codegen a call to resolve_assert_message in the same way we do for user provided assertion messages. Thus, we have this assert_messages map for resolving messages specified by the compiler and separate calls to resolve_assert_message for handling assert messages specified by the user.
Happy Case
We should unify the strategy for specifying errors and error types. Ideally we would remove the assert_messages map from the Circuit type. We most likely could instead store errors and their respective types on the ABI rather than on the circuit. Upon circuit failure we can then use the ABI for fetching the appropriate error.
Some example pseudocode written by @TomAFrench from when we were originally planning #4101.
// This defines a class of errors, e.g. overflows
// This avoids us duplicating the definition of how to format this error message many times.
struct ErrorType {
    format_string: String
    types: Vec<AbiType>
}

// This defines a specific instance.
struct Error {
    type_id: u32
    inputs: Vec<(string, Range<Witness>)>
}

struct ABI {
    ...
    error_types: HashMap<u32, ErrorType>,
    errors: HashMap<u32, Error>
    ...
}

Unifying these constrain errors would essentially require full structured errors as part of the ABI. We could then have followup work to enable custom errors inside Noir similar to https://soliditylang.org/blog/2021/04/21/custom-errors/.
Alternatives Considered
No response
Additional Context
No response
Would you like to submit a PR for this Issue?
No
Support Needs
No response

Dynamic assert payloads
Current state

Consider the following code containing a "dynamic" assert payload:
fn option_expect<T, Err>(opt: Option<T>, err: Err) -> T {
    assert(opt.is_some(), err);
    opt.unwrap_unchecked()
}

fn main(option: Option<u8>) -> pub u8 {
    option_expect(option, "option is none")
}

vs the static assert payload version of the code:
fn main(option: Option<u8>) -> pub u8 {
    assert(option.is_some(), "option is none");
    option.unwrap_unchecked()
}

This is the generated SSA for the static version:
Initial SSA:
acir(inline) fn main f0 {
  b0(v0: u1, v1: u8):
    v3 = call f1(v0, v1)
    constrain v3 == u1 1 "option is none"
    v6 = call f2(v0, v1)
    return v6
}

And this one for the "dynamic" version:
Initial SSA:
acir(inline) fn main f0 {
  b0(v0: u1, v1: u8):
    v12 = call f1(v0, v1, [u8 111, u8 2⁴×7, u8 116, u8 105, u8 111, u8 110, u8 2⁵, u8 105, u8 115, u8 2⁵, u8 110, u8 111, u8 110, u8 101])
    return v12
}
acir(inline) fn option_expect f1 {
  b0(v0: u1, v1: u8, v2: [u8; 14]):
    inc_rc v2
    v4 = call f2(v0, v1)
    v8 = call f2(v0, v1)
    inc_rc v2
    constrain v4 == u1 1 call f3(v2, v8)
    v10 = call f4(v0, v1)
    dec_rc v2
    return v10
}
acir(inline) fn is_some f2 {
  b0(v0: u1, v1: u8):
    return v0
}
brillig fn resolve_assert_message f3 {
  b0(v0: [u8; 14], v1: u1):
    inc_rc v0
    v2 = not v1
    jmpif v2 then: b1, else: b2
  b1():
    call v3(v0, [u8 123, u8 34, u8 107, u8 105, u8 110, u8 100, u8 34, u8 58, u8 34, u8 115, u8 116, u8 114, u8 105, u8 110, u8 103, u8 34, u8 44, u8 34, u8 108, u8 101, u8 110, u8 103, u8 116, u8 104, u8 34, u8 58, u8 49, u8 52, u8 125], u1 0)
    jmp b2()
  b2():
    dec_rc v0
    return
}
acir(inline) fn unwrap_unchecked f4 {
  b0(v0: u1, v1: u8):
    return v1
}

The dynamic version has undergone some modifications before conversion to SSA. Let's explain what are those:
Modifications at HIR
        if matches!(
            assert_message_expr,
            Expression { kind: ExpressionKind::Literal(Literal::Str(..)), .. }
        ) {
            return Some(self.resolve_expression(assert_message_expr));
        }

        let is_in_stdlib = self.path_resolver.module_id().krate.is_stdlib();
        let assert_msg_call_path = if is_in_stdlib {
            ExpressionKind::Variable(Path {
                segments: vec![Ident::from("internal"), Ident::from("resolve_assert_message")],
                kind: PathKind::Crate,
                span,
            })
        } else {
            ExpressionKind::Variable(Path {
                segments: vec![
                    Ident::from("std"),
                    Ident::from("internal"),
                    Ident::from("resolve_assert_message"),
                ],
                kind: PathKind::Dep,
                span,
            })
        };
        let assert_msg_call_args = vec![assert_message_expr.clone(), condition];
        let assert_msg_call_expr = Expression::call(
            Expression { kind: assert_msg_call_path, span },
            assert_msg_call_args,
            span,
        );
        Some(self.resolve_expression(assert_msg_call_expr))

At the HIR level, if the assert message is not a literal string, a call to resolve_assert_message is introduced. This is the implementation of resolve_assert_message:
#[oracle(assert_message)]
unconstrained fn assert_message_oracle<T>(_input: T) {}
unconstrained pub fn resolve_assert_message<T>(input: T, condition: bool) {
    if !condition {
        assert_message_oracle(input);
    }
}

This explains this added function in the SSA:
...
    constrain v4 == u1 1 call f3(v2, v8)
...

brillig fn resolve_assert_message f3 {
  b0(v0: [u8; 14], v1: u1):
    inc_rc v0
    v2 = not v1
    jmpif v2 then: b1, else: b2
  b1():
    call v3(v0, [u8 123, u8 34, u8 107, u8 105, u8 110, u8 100, u8 34, u8 58, u8 34, u8 115, u8 116, u8 114, u8 105, u8 110, u8 103, u8 34, u8 44, u8 34, u8 108, u8 101, u8 110, u8 103, u8 116, u8 104, u8 34, u8 58, u8 49, u8 52, u8 125], u1 0)
    jmp b2()
  b2():
    dec_rc v0
    return
}

Modifications at monomorphization
After monomorphization, the assert_message oracle call is modified with metadata about the type it's being called with, in a similar way as the print opcode.
        if let ast::Expression::Ident(ident) = original_func.as_ref() {
            if let Definition::Oracle(name) = &ident.definition {
                if name.as_str() == "print" {
                    // Oracle calls are required to be wrapped in an unconstrained function
                    // The first argument to the `print` oracle is a bool, indicating a newline to be inserted at the end of the input
                    // The second argument is expected to always be an ident
                    self.append_printable_type_info(&hir_arguments[1], &mut arguments);
                } else if name.as_str() == "assert_message" {
                    // The first argument to the `assert_message` oracle is the expression passed as a message to an `assert` or `assert_eq` statement
                    self.append_printable_type_info(&hir_arguments[0], &mut arguments);
                }
            }
        }

The metadata is the serialized ABI of the type:
    fn append_printable_type_info_inner(typ: &Type, arguments: &mut Vec<ast::Expression>) {
        // Disallow printing slices and mutable references for consistency,
        // since they cannot be passed from ACIR into Brillig
        if matches!(typ, HirType::MutableReference(_)) {
            unreachable!("println and format strings do not support mutable references.");
        }

        let printable_type: PrintableType = typ.into();
        let abi_as_string = serde_json::to_string(&printable_type)
            .expect("ICE: expected PrintableType to serialize");

        arguments.push(ast::Expression::Literal(ast::Literal::Str(abi_as_string)));
    }

This explains the call to v3 (assert_message) in SSA:
brillig fn resolve_assert_message f3 {
  b0(v0: [u8; 14], v1: u1):
    inc_rc v0
    v2 = not v1
    jmpif v2 then: b1, else: b2
  b1():
    call v3(v0, [u8 123, u8 34, u8 107, u8 105, u8 110, u8 100, u8 34, u8 58, u8 34, u8 115, u8 116, u8 114, u8 105, u8 110, u8 103, u8 34, u8 44, u8 34, u8 108, u8 101, u8 110, u8 103, u8 116, u8 104, u8 34, u8 58, u8 49, u8 52, u8 125], u1 0)
    jmp b2()
  b2():
    dec_rc v0
    return
}

As can be seen here, the oracle call to assert_message includes not only the string v0 but also a literal byte array, that being the abi_as_string.
Compilation and runtime
This strategy will make it so when evaluating this instruction constrain v4 == u1 1 call f3(v2, v8) (the assertion on is_some()) a brillig call will be issued to a resolve_assert_message with the string and the condition. Then the brillig function will conditionally call the assert_message oracle with the string + extra metadata if the condition is false.
But how is this assert_message oracle related with the original assertion?
The runtime (acvm_js and nargo) recognizes this special oracle, parses the type using the metadata, stores the parsed result, and when the circuit fails, it has in cache the last call to assert_message.
Comparison with static assert messages
With static assert messages, no transformations happen on the frontend. The generated SSA
Initial SSA:
acir(inline) fn main f0 {
  b0(v0: u1, v1: u8):
    v3 = call f1(v0, v1)
    constrain v3 == u1 1 "option is none"
    v6 = call f2(v0, v1)
    return v6
}

gets compiled to ACIR and in compilation to ACIR, the literal string is stored in a Map<OpcodeLocation, String> inside the circuit. So when the circuit fails, the runtime just picks what String is stored there for the failing opcode.
If the function is brillig, instead of being added to the Map<OpcodeLocation, String> the error string is directly returned as revert data.

Challenges with current state
This approach, altough a bit fragile (since modifications in SSA assertions need to account for dynamic assertions, or bugs like this one can happen) works fine for circuits. But it poses some challenges for public functions in aztec:

The runtime is required to know about dynamic assertions, storing the error payload for a later error. This is quite problematic for the implementation of the VM, since it would need to emulate what acvm_js is doing right now.
Metadata about the type is embedded in the bytecode. This would increase bytecode size and runtime cost for public functions, with this data only being relevant for tooling.
There is a small extra cost of adding an extra function call for constrains, but it should be easily removed by inlining.

Alternative approach
Instead of using an entirely new approach for dynamic assertion messages, we can follow the same approach that we have for static assertion messages.
The Map<OpcodeLocation, String> that the circuit struct holds could be extended to be a Map<OpcodeLocation, AssertionData>
The type AssertionData would be:
struct AssertionData {
    typing_metadata: PrintableType, // This is what is currently serialized into the bytecode.
    payload: AssertionPayload
};

enum AssertionPayload {
    Static(Vec<Expression>),
    Witness(Vec<Expression>),
    MemoryArray(BlockId),
    BrilligOutput,  // The brillig VM returns the data directly in revert data
}

So the example code:
fn option_expect<T, Err>(opt: Option<T>, err: Err) -> T {
    assert(opt.is_some(), err);
    opt.unwrap_unchecked()
}

fn main(option: Option<u8>) -> pub u8 {
    option_expect(option, "option is none")
}

would generate the following SSA:
Initial SSA:
acir(inline) fn main f0 {
  b0(v0: u1, v1: u8):
    v12 = call f1(v0, v1, [u8 111, u8 2⁴×7, u8 116, u8 105, u8 111, u8 110, u8 2⁵, u8 105, u8 115, u8 2⁵, u8 110, u8 111, u8 110, u8 101])
    return v12
}
acir(inline) fn option_expect f1 {
  b0(v0: u1, v1: u8, v2: [u8; 14]):
    inc_rc v2
    v4 = call f2(v0, v1)
    inc_rc v2
    constrain v4 == u1 1 v2 // Notice here constrain directly holds an SSA value
    v10 = call f4(v0, v1)
    dec_rc v2
    return v10
}
acir(inline) fn is_some f2 {
  b0(v0: u1, v1: u8):
    return v0
}
acir(inline) fn unwrap_unchecked f4 {
  b0(v0: u1, v1: u8):
    return v1
}

ACIR generation and Brillig generation need access to the PrintableType metadata that is generated from the HIR for all non-string assert payloads. It could be maintained in parallel or directly embedded in the constrain instruction in SSA, altough the latter sounds worse. This is necessary since we need to build the AssertionData defined above.
When an opcode fails, the ACVM immediately has enough data to:

Extract the payload of the failing opcode
Know the types of the failing opcode payload and how they could be printed for an user.

Advantages of this approach

Supporting static non-string assertion payloads
Removes any notion of PrintableType from the bytecode, maintaining metadata outside.
No more need to cache oracle calls to figure out the payload for a failing assertion.
The constrain instruction just holds a regular SSA value. No need to special case it for everything like we have to do with the call to resolve_assert_message that has a parallel version of the constrain condition, etc.
The distinction of static constrain payload or dynamic constrain payload happens after SSA optimization. So any payload that can be determined static via optimization wouldn't end up in the bytecode, unlike now (in fact my example above should be determined constant at inlining!)
AVM support would now be easy, since no extraneous oracles have been added. Brillig gen just has to properly serialize the passed in type (possibly using the same serialization strategy as oracle calls have).

Updates to the proposal:
No metadata in the circuit struct
We can avoid storing metadata about assertion payload types in the circuit struct if we put it in the ABI.
The circuit struct will have a map of Map<OpcodeLocation, AssertionPayload> instead of Map<OpcodeLocation, AssertionData>. The metadata about the type of the error payload will live in the ABI instead. Nargo and other runners are then responsible to properly format the error data.

No more need to cache oracle calls to figure out the payload for a failing assertion.

I think that's unnecessary now anyway now we have revert data in brillig so this is a constant between all proposals. This was just a hack in the short term.

Closing this after https://github.com/AztecProtocol/aztec-packages/pull/5949, we can reopen an issue for the next iteration.


I need help, I can't properly process pictures from S3.

First of all, I'm sorry, I don't understand English, I usually use Google translate, if I didn't express my question clearly, please understand, thank you!
I'm using an object storage compatible with AWS S3 protocol, when I'm done configuring Thumbor, try to run the image fetched from S3, Thumbor got a 500 error and I need help.
This is my config file
LOADER = 'thumbor.loaders.http_loader'
STORAGE = 'thumbor_aws.storage'
RESULT_STORAGE = 'thumbor_aws.result_storage'

################################# AWS Storage ##################################
AWS_STORAGE_REGION_NAME = 'ap-beijing'
AWS_STORAGE_BUCKET_NAME = 'img'
AWS_STORAGE_S3_SECRET_ACCESS_KEY = 'XXXXXXXXX'
AWS_STORAGE_S3_ACCESS_KEY_ID = 'XXXXXXXXX'
AWS_STORAGE_S3_ENDPOINT_URL = 'https://cos.ap-beijing.myqcloud.com'
AWS_STORAGE_ROOT_PATH = '/storage'
AWS_STORAGE_S3_ACL = None
AWS_DEFAULT_LOCATION = 'https://img.cos.ap-beijing.myqcloud.com'

################################################################################

############################## AWS Result Storage ##############################
AWS_RESULT_STORAGE_REGION_NAME = 'ap-beijing'
AWS_RESULT_STORAGE_BUCKET_NAME = 'img'
AWS_RESULT_STORAGE_S3_SECRET_ACCESS_KEY = 'XXXXXXXX'
AWS_RESULT_STORAGE_S3_ACCESS_KEY_ID = 'XXXXXX'
AWS_RESULT_STORAGE_S3_ENDPOINT_URL = 'https://cos.ap-beijing.myqcloud.com'
AWS_RESULT_STORAGE_ROOT_PATH = '/result-storage'
AWS_RESULT_STORAGE_S3_ACL = None

################################################################################

My environment configuration
thumbor        7.0.6
thumbor-aws    0.4.0
S3             private read and write

Access
http://<IP_ADDRESS>:8689/unsafe/20221/a78a8a02-6c94-452e-9322-62824c458acd.jpg

Error
2022-02-18 03:14:09 thumbor:ERROR ERROR: Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/thumbor/handlers/__init__.py", line 195, in get_image
    result = await self._fetch(self.context.request.image_url)
  File "/usr/local/lib/python3.9/dist-packages/thumbor/handlers/__init__.py", line 732, in _fetch
    loader_result = await self.context.modules.loader.load(self.context, url)
  File "/usr/local/lib/python3.9/dist-packages/thumbor/loaders/http_loader.py", line 191, in load
    response = await client.fetch(req, raise_error=True)
TimeoutError: [Errno 110] Connection timed out

2022-02-18 03:14:09 thumbor:ERROR [BaseHandler] get_image failed for url `20221/a78a8a02-6c94-452e-9322-62824c458acd.jpg`. error: `[Errno 110] Connection timed out`
2022-02-18 03:14:09 tornado.access:ERROR 500 GET /unsafe/20221/a78a8a02-6c94-452e-9322-62824c458acd.jpg (<IP_ADDRESS>) 3231.83ms

Access
http://<IP_ADDRESS>:8689/unsafe/https://www.baidu.com/img/PC_880906d2a4ad95f5fafb2e540c5cdad7.png

Error
2022-02-18 03:12:18 thumbor:ERROR [BaseHander.finish_request]
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/thumbor/handlers/__init__.py", line 514, in finish_request
    result = await self.context.thread_pool.queue(
  File "/usr/local/lib/python3.9/dist-packages/thumbor/threadpool.py", line 53, in queue
    return await self._execute_in_foreground(operation, *args)
  File "/usr/local/lib/python3.9/dist-packages/thumbor/threadpool.py", line 45, in _execute_in_foreground
    return operation(*args)
  File "/usr/local/lib/python3.9/dist-packages/thumbor/handlers/__init__.py", line 428, in _load_results
    image_extension, content_type = self.define_image_type(context, None)
  File "/usr/local/lib/python3.9/dist-packages/thumbor/handlers/__init__.py", line 393, in define_image_type
    elif self.can_auto_convert_png_to_jpg():
  File "/usr/local/lib/python3.9/dist-packages/thumbor/handlers/__init__.py", line 372, in can_auto_convert_png_to_jpg
    return self.context.request.engine.can_auto_convert_png_to_jpg()
  File "/usr/local/lib/python3.9/dist-packages/thumbor/engines/__init__.py", line 398, in can_auto_convert_png_to_jpg
    can_convert = self.extension == ".png" and not self.has_transparency()
  File "/usr/local/lib/python3.9/dist-packages/thumbor_vips_engine/engine.py", line 151, in has_transparency
    raise NotImplementedError()
NotImplementedError
2022-02-18 03:12:18 thumbor:WARNING Error while trying to fetch the image:
2022-02-18 03:12:18 tornado.access:ERROR 500 GET /unsafe/https://www.baidu.com/img/PC_880906d2a4ad95f5fafb2e540c5cdad7.png (<IP_ADDRESS>) 329.15ms

I think Thumbor's configuration on AWS S3 is very simple, but I have these problems, I try to solve them and find that nothing works, I would like to get help from the developers, I would appreciate any reply.
I have been using Thumbor 6.0 for the past two years, and I have to say, Thumbor is an excellent work!

Wait you are using thumbor vips engine? It's not ready, that's why you are getting a NotImplementedError...

About the TimeoutError: [Errno 110] Connection timed out probably some network error between your computer and the actual file. Hard to help without understanding your use case better.

BTW your english is fine :)

Wait you are using thumbor vips engine? It's not ready, that's why you are getting a NotImplementedError...

Yes, I use thumbor vips engine, but I got so many bugs that I had to stop using thumbor vips engine.
I use thumbor-aws to load images from http, this problem has been solved so far, I use thumbor-aws to transfer images from the intranet, there should be no problems with the network, thank you for your reply.


How I create an index with a dot on the beginning name?

Hi,
I need to create more indexes for ElastAlert with dot on the beginning name, as example: .gsm.alert
but, if I create this index with command elastalert-create-index, I have error:
elastalert-create-index: error: unrecognized arguments: .gsm.alert

Ok, my mistake. I found a reason.


Allow to inject javascript files into the head of the html page

Allows to inject javascript resources to the head instead of the body element

Would it not make more sense to provide an inject map allowing some scripts to be injected at head and others into the bottom of the body?
This is the method for optimising above the fold loading for large SPAs that I feel would be the most common use case for head loading scripts.
Also it would be great to allow some script/link inlining in the head when very very small content as per Google's CSS optimisation guidelines.

@tomatau - that's a great idea.
Could you propose a clean API which allows placing all, or only some picked css/js files to the head/body?

Keep the same options for true/false or placing all at 'head'/'body'.
In addition to the current API, possibly something like:
inject: {
  'head': ['chunkname1', /file-name-regex1/],
  'body': ['chunkname2', /file-name-regex2/],
}

This would allow for the easy inclusion of chunks which I feel would be cleanest but also allow for specific files.
This could cause problems when a file/chunk is selected for both head and body?

This looks quite nice.
I guess all not specified js files should be injected to the body and all not specified css files should go into the head. - What do you think?
If the configuration is wrong it should just add some warnings that the file will be placed to the head and to the body element.
Do you have time to create a merge request for this?

Re merge req: I would really like to - however I'm not familiar with the internals and time isn't on my side right now so it would take me a very long time to put this together.


Bootup process hangs on waiting for '>'

Hello,
Super confused about this problem. For some reason any version of CSR1000v I've tried on this specific host just hangs forever on that waiting for '>' stage:
2023-07-15 06:27:16,024: vrnetlab   DEBUG    Starting vrnetlab CSR
2023-07-15 06:27:16,024: vrnetlab   DEBUG    VMs: [<__main__.CSR_vm object at 0x7fed33753d30>]
2023-07-15 06:27:16,027: vrnetlab   DEBUG    VM not started; starting!
2023-07-15 06:27:16,027: vrnetlab   INFO     Starting CSR_vm
2023-07-15 06:27:16,027: vrnetlab   DEBUG    number of provisioned data plane interfaces is 0
2023-07-15 06:27:16,027: vrnetlab   DEBUG    qemu cmd: qemu-system-x86_64 -enable-kvm -display none -machine pc -monitor tcp:<IP_ADDRESS>:4000,server,nowait -m 4096 -serial telnet:<IP_ADDRESS>:5000,server,nowait -drive if=ide,file=/csr1000v-universalk<IP_ADDRESS>-serial-overlay.qcow2 -device pci-bridge,chassis_nr=1,id=pci.1 -device virtio-net-pci,netdev=p00,mac=0C:00:e5:00:0e:00 -netdev user,id=p00,net=<IP_ADDRESS>/24,tftp=/tftpboot,hostfwd=tcp::2022-<IP_ADDRESS>:22,hostfwd=udp::2161-<IP_ADDRESS>:161,hostfwd=tcp::2830-<IP_ADDRESS>:830,hostfwd=tcp::2080-<IP_ADDRESS>:80,hostfwd=tcp::2443-<IP_ADDRESS>:443
2023-07-15 06:27:22,036: launch     TRACE    OUTPUT:
  Booting `CSR1000v - packages.conf'

BOOT CMD: /packages.conf rw root=/dev/ram max_loop=64 HARDWARE=virtual quiet
console= SR_BOOT=bootflash:packages.conf
Calculating SHA-1 hash...done
SHA-1 hash:
        calculated   ac8b9d82:75cfab7a:706f71ff:13c393cd:47f4877b
        expected     ac8b9d82:75cfab7a:706f71ff:13c393cd:47f4877b
package header rev 3 structure detected
IOSXE version 17.3.02 detected
Calculating SHA-1 hash...
2023-07-15 06:27:28,045: launch     TRACE    OUTPUT: done
SHA-1 hash:
        calculated   74564399:fe87a753:c692f47a:650837d7:49a219b5
        expected     74564399:fe87a753:c692f47a:650837d7:49a219b5
Package type:0x7531, flags:0x0
linux image, size=0x682dc8

2023-07-15 06:27:30,048: launch     TRACE    OUTPUT: linux isord, size=0x278dfbc

2023-07-15 06:27:32,051: launch     TRACE    OUTPUT: %IOSXEBOOT-4-PART_VERIFY: (local/local): Verifying partition table for device /dev/bootflash...
%IOSXEBOOT-4-PART_VERIFY: (local/local): Selected MBR v2 partition layout.

2023-07-15 06:27:52,080: launch     TRACE    OUTPUT:
*Jul 15 06:27:50.811: %IOSXEBOOT-4-BOOT_SRC: (rp/0): Checking for grub upgrade

*Jul 15 06:27:51.024: %IOSXEBOOT-4-BOOT_SRC: (rp/0): Checking grub versions 2.2 vs 2.2

*Jul 15 06:27:51.026: %IOSXEBOOT-4-BOOT_SRC: (rp/0): Bootloader upgrade not necessary.

2023-07-15 06:28:06,100: launch     TRACE    OUTPUT: Jul 15 06:28:05.980: %BOOT-5-OPMODE_LOG: R0/0: binos: System booted in AUTONOMOUS mode

2023-07-15 06:28:40,151: launch     TRACE    OUTPUT:
              Restricted Rights Legend

Use, duplication, or disclosure by the Government is
subject to restrictions as set forth in subparagraph
(c) of the Commercial Computer Software - Restricted
Rights clause at FAR sec. 52.227-19 and subparagraph
(c) (1) (ii) of the Rights in Technical Data and Computer
Software clause at DFARS sec.<PHONE_NUMBER>.

           Cisco Systems, Inc.
           170 West Tasman Drive
           San Jose, California 95134-1706

Cisco IOS Software [Amsterdam], Virtual XE Software (X86_64_LINUX_IOSD-UNIVERSALK9-M), Version 17.3.2, RELEASE SOFTWARE (fc3)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2020 by Cisco Systems, Inc.
Compiled Sat 31-Oct-20 13:16 by mcpre

This software version supports only Smart Licensing as the software licensing mechanism.

PLEASE READ THE FOLLOWING TERMS CAREFULLY.

Your use of the Software is subject to the Cisco End User License Agreement
(EULA) and any relevant supplemental terms (SEULA) found at
http://www.cisco.com/c/en/us/about/legal/cloud-and-software/software-terms.html.

You hereby acknowledge and agree that certain Software and/or features are
licensed for a particular term, that the license to such Software and/or
features is valid only for the applicable term and that such Software and/or
features may be shut down or otherwise terminated by Cisco after expiration
of the applicable license term (e.g., 90-day trial period). Cisco reserves
the right to terminate any such Software feature electronically or by any
other means available. While Cisco may provide alerts, it is your sole
responsibility to monitor your usage of any such term Software feature to
ensure that your systems and networks are prepared for a shutdown of the
Software feature.

2023-07-15 06:28:44,157: launch     TRACE    OUTPUT:
All TCP AO KDF Tests Pass
cisco CSR1000V (VXE) processor (revision VXE) with 2072007K/3075K bytes of memory.
2023-07-15 06:28:46,160: launch     TRACE    OUTPUT:
Processor board ID 9A6QOVZ9BCQ
Router operating mode: Autonomous
1 Gigabit Ethernet interface
32768K bytes of non-volatile configuration memory.
3978420K bytes of physical memory.
6188032K bytes of virtual hard disk at bootflash:.

2023-07-15 06:28:51,925: launch     DEBUG    matched, Press RETURN to get started.
2023-07-15 06:28:51,925: launch     INFO     applying bootstrap configuration
2023-07-15 06:28:51,925: vrnetlab   TRACE    waiting for '>' on serial console

No errors during make docker-image and I can successfully build and run images using the same qcow2 files on another host. I cal also build and run for example n9kv on this host.
Not even sure where to look for clues, would appreciate any ideas. Thanks!

Hi
Can you run this image (or its qcow2) on any other host that you have?

Hi
Can you run this image (or its qcow2) on any other host that you have?

Yes, I can build and run images using the same qcow2 files on another host. I can also build and run n9kv images on this host, so the problem seems to be only affecting csr1kv and only on this host.

Then you have to check the cpu architecture (CPU flags) on both hosts and check the differences and maybe try running the qcow on this faulty host with just qemu and see if you have anything on the terminal output.
Not much we can help here, all points out it is either a qemu or hardware incompatibility

The confusing part is that on the other node vrnetlab sends a new line several times when prompted to press RETURN (can see it in the trace), but it doesn't do the same here.
It's prompted to press RETURN as expected but doesn't write anything to the terminal as it's supposed to. Is my interpretation wrong?

then it seems after sending the first RETURN nothing comes up on the terminal, thus the code is stuck in waiting for anything to be printed back, but nothing pops up

then it seems after sending the first RETURN nothing comes up on the terminal, thus the code is stuck in waiting for anything to be printed back, but nothing pops up

I don't think it sends the first RETURN though. Here's the output from the working node:
5234688K bytes of virtual hard disk at bootflash:.

2023-07-15 15:39:11,859: launch     DEBUG    matched, Press RETURN to get started.
2023-07-15 15:39:11,859: vrnetlab   DEBUG    writing to serial console: !<<<<<<< Notice this line, it's missing in the output of the "bad" node (see above)
2023-07-15 15:39:11,859: launch     INFO     applying bootstrap configuration
2023-07-15 15:39:11,859: vrnetlab   DEBUG    writing to serial console:
2023-07-15 15:39:11,859: vrnetlab   TRACE    waiting for '>' on serial console
2023-07-15 15:39:12,024: vrnetlab   TRACE    read from serial console:

So I noticed that the image built on the working node is slightly bigger (~50MB) compared to the image built on this node so I tried using that bigger image and it worked right away, so the problem is definitely somewhere within the build process. Maybe some missing dependency?
Anyway, I will close the issue but I still doubt the problem is on the host's sit to be honest. Thanks for looking into this.

So just fyi, I think I figured out what the problem is and it's not my server. This commit broke CSR100v built completely, really surprised nobody else has opened an issue about this.
First, there's this part in the wait_write function:
if not cmd:  # skip empty commands
    return

that prevents the self.wait_write("", wait=None) commands from the CSR's launch.py script to fire thus getting stuck on that Press Enter message.
Once that is fixed, the new clean_buffer operation appears to be reading and cleaning too eagerly and sometimes it seems to be cleaning up/deleting from buffer the currently awaited pattern and have the boot process stuck in different stages of the config loading.
I deleted both parts, basically reverting code back to v0.11.0 and it fixed the problem for me.

Once that is fixed, the new clean_buffer operation appears to be reading and cleaning too eagerly and sometimes it seems to be cleaning up/deleting from buffer the currently awaited pattern and have the boot process stuck in different stages of the config loading.

Hi @achurak, regarding this second statement, does it happen with default config or when you provide startup-config on top of the default one?

Hi @achurak, regarding this second statement, does it happen with default config or when you provide startup-config on top of the default one?

I had both happen actually. One got stuck here:
*Jul 28 03:00:24.517: %LINEPROTO-5-UPDOWN: Line protocol on Interface Lsmpi0, changed state to up
*Jul 28 03:00:24.518: %LINEPROTO-5-UPDOWN: Line protocol on Interface EOBC0, changed state to up
*Jul 28 03:00:25.122: %SYS-7-NVRAM_IN
Router>'
2023-07-28 03:00:32,782: vrnetlab   DEBUG    cleaned buffer: '
Router>'
2023-07-28 03:00:32,782: vrnetlab   DEBUG    writing to serial console: 'enable'
2023-07-28 03:00:32,782: vrnetlab   DEBUG    waiting for '>' on serial console
   The highlighted entry will be executed automatically in 0s.

And the other one got all the way to the custom startup-config and stuck there:
2023-07-28 03:00:35,671: launch     TRACE    Startup config file /config/startup-config.cfg exists
2023-07-28 03:00:35,671: launch     TRACE    Parsed startup config file /config/startup-config.cfg
2023-07-28 03:00:35,671: launch     INFO     Writing lines from /config/startup-config.cfg
2023-07-28 03:00:35,671: vrnetlab   DEBUG    waiting for '#' on serial console
2023-07-28 03:00:44,792: vrnetlab   DEBUG    read from serial console: 'copy running-config startup-config
Destination filename [startup-config]?
Building configuration...
[OK]
rtr02#'
2023-07-28 03:00:44,792: vrnetlab   DEBUG    writing to serial console: 'configure terminal'
2023-07-28 03:00:44,792: vrnetlab   DEBUG    waiting for '#' on serial console
2023-07-28 03:00:44,835: vrnetlab   DEBUG    read from serial console: '
rtr02#'
2023-07-28 03:00:44,835: vrnetlab   DEBUG    cleaned buffer: 'configure terminal
Enter configuration commands, one per line.  End with CNTL/Z.
rtr02(config)#'
2023-07-28 03:00:44,835: vrnetlab   DEBUG    writing to serial console: 'no username admin'
2023-07-28 03:00:44,835: vrnetlab   DEBUG    waiting for '#' on serial console

@jbemmel looks like this 2nd issue can be something you can have a look at as you implemented it?
The log indeed shows that buffer contains legit strings which we shouldn't discard. If you need csr image, you can find one in our registry.

Let's start by reverting the 'skip empty commands' logic

@jbemmel the empty cmd shunt has been reverted. Now probably it makes sense to look into eager eater that was introduced in https://github.com/hellt/vrnetlab/pull/115

Hello all,
many kudos for the awesome project !
Apologies for piggy-backing on the issue. Just wanted to mention that i am facing the same issue but with slightly different parameters.
I built the image 17.03.02 using commit 52f1c32206cfe79df662a4d4ce1c6258378aafc3, i have a small topology of 2-3 routers.
None/some/all of these exhibit this behavior randomly, the problem is not reproduced consistently. e.g.
try 1: 2 out of 3 are getting stuck into the waiting for '>' on serial console line
try 2: all 3 have booted correctly
try 3: 1 out of 3 is stuck
the stuck lines are always the below:
2023-08-09 12:53:46,331: vrnetlab   TRACE    cleaned buffer: '
Router>'
2023-08-09 12:53:46,332: vrnetlab   DEBUG    writing to serial console: 'enable'
2023-08-09 12:53:46,333: vrnetlab   TRACE    waiting for '>' on serial console

There is no difference if i have a start-up config or not. Attaching my topology yml for reference .
name: csrv

topology:
  nodes:
    r1:
      kind: vr-csr
      image: vrnetlab/vr-csr:17.03.02
      startup-config: r1.cfg
    r2:
      kind: vr-csr
      image: vrnetlab/vr-csr:17.03.02
      startup-config: r2.cfg
    r3:
      kind: vr-csr
      image: vrnetlab/vr-csr:17.03.02
      startup-config: r3.cfg
  links:
    - endpoints: ["r1:eth1", "r2:eth1"]
    - endpoints: ["r1:eth2", "r3:eth1"]
    - endpoints: ["r2:eth2", "r3:eth2"]

Do you happen to have any idea of the problem?
Can i help with the trouble shooting ? e.g. provide some logs ?
Many thanks !
Adam

@adampav
you can checkout to a previously released version https://github.com/hellt/vrnetlab/releases/tag/v0.11.0 to build your images
in the current master branch it is broken

I'm proposing https://github.com/hellt/vrnetlab/pull/136 to revert the change for other platforms

@hellt many thanks :) v0.11.0 works fine .
Adam

Hi @adampav @achurak
can you try the tip of the master branch where fixes have been put in place by @jbemmel?
I hope csr now works with the latest code where breaking changes were rolled back

Hi Roman!
It works for me!
Thank you all, very much for your efforts
On Sun, Aug 13, 2023, 15:11 Roman Dodin @.***> wrote:

Hi @adampav https://github.com/adampav @achurak
https://github.com/achurak
can you try the tip of the master branch where fixes have been put in
place by @jbemmel https://github.com/jbemmel?
I hope csr now works with the latest code where breaking changes were
rolled back
—
Reply to this email directly, view it on GitHub
https://github.com/hellt/vrnetlab/issues/130#issuecomment-1676341033,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AB76YFYLC6AVJX7QTM3LOETXVC76TANCNFSM6AAAAAA2LDJFWE
.
You are receiving this because you were mentioned.Message ID:
@.***>


Management Functions

These are not selected anywhere:
FMT_MOF_EXT.1 Management of Security Functions Behavior
FMT_SMF_EXT.1 Specification of Management Functions

Done


[DO NOT MERGE] Add block of Google Analytics dimensions for A/B tests

The GA dimensions from 40-49 have been allocated to A/B tests. These may be reserved for any A/B test for the duration of the test.
This commit identifies which dimension to use for the A/B test based on an extra data attribute added to the meta tag. Only dimensions in the allowed range are permitted. Any others are ignored and are not reported to GA.
It would be good to have some feedback on this approach. I'm aiming to make it as simple as possible for new teams to add an A/B test. With this proposal, they would have to do the following:

Request one of the dimensions in the 40-49 block from the performance analysts for the duration of the A/B test. They will assign one which isn't currently being used.
Set a meta tag in the header like <meta name="govuk:ab-test" content="NameOfTest:#{bucket}" data-analytics-dimension="43">.

Ideally, this would be done through the AB test gem. The application code would just need to pass the experiment name and dimension to the gem.

https://trello.com/c/g9OyC7Pn/310-tell-google-analytics-about-a-b-tests
Before this can be merged:

[x] Merge #888 and change the merge base of this PR
[ ] Add the data attribute to the meta tag on the A/B example page in frontend
[ ] Add the data attribute to the meta tag in the A/B testing gem (alphagov/govuk_ab_testing#1)

Closing this temporarily to focus on the education A/B testing. We'll revisit after that's live.
Trello: https://trello.com/c/QoJsEjgw/381-better-a-b-testing

@tijmenb - I'd like to reopen this because we need to add another dimension for the Education Navigation A/B test, and I think this change is the easiest way to do that. Is that OK with you?
The alternative would be to give the Example and EducationNavigation meta tags different names, but that won't be compatible with more general A/B test analytics which will make the switchover harder when we release this change.

I've changed the required status checks for this repo. You'll need to rebase this branch against master so that this branch gets a Jenkinsfile can be built against new CI.


Subscription URI in MpnsDelegate.

Hi,
Wouldn't it make more sense to provide the subscription URI via the delegate? This would be useful for retrying failed notifications, for example.
Regards

could you give an example?

Thanks really appreciate this

Case 1:
Lets say MPNS returns [INACTIVATE_STATE] (https://github.com/aerogear/java-mpns/blob/master/src/main/java/org/jboss/aerogear/windows/mpns/MpnsResponse.java#L92). This says that the notification may be retried, but no more than once an hour. One (maybe not so great) way of taking care of this is sticking the failed notification into the database with a failure time and subscription URI. Then simply have a service which periodically tries re-sending this class of failed notifications making sure at least an hour passes based on the failure time.
Case 2:
I think this is actually a better example.
Lets say MPNS returns [EXPIRED] (https://github.com/aerogear/java-mpns/blob/master/src/main/java/org/jboss/aerogear/windows/mpns/MpnsResponse.java#L68). THis says that the server should not continue sending notifications to this subscription URI anymore. This means, the server may want to remove the subscription URI from its database, for example.


Feature request: export_all_to_string function

export_to_string already does this, but only for one single type. export_all_to_string would basically do the same as export_all except output them as a String (Could also be a Vec<String>) to manually create the files for the types.
Currently, one can achieve the same result by manually calling export_to_string() on all types, manually searching for their dependencies, and calling export_to_string() on them as well.

The reason this is not supported is because of import paths. How would they be generated? Would it even make sense to generate them? If you export_to_string there is no guarantee that the file structure we expect will be maintained, so there is no reasonable way to implement export_all_to_string as the import paths would very quickly descend into chaos

If you really need something like this, you can iterate over all dependencies of a type recursively. See ts_rs/export for how that works.


Signal handler: linux signal

Hey!
I just have added sigterm and sigint signal in the signal manager class.
I also have added a signal manager for development mode (--dev) because django autoreloader create a new process, so we can't catch sigterm and sigint.
Bye!

Coverage decreased (-1.3%) to 75.469% when pulling 2183734eadff78d4a86a747a8e27183b52882b1b on cedricfarinazzo:fix/linux-signal-handler into 12c8f328e4ba704bd7d91534bb6569e70197b949 on arteria:master.

Coverage decreased (-1.3%) to 75.469% when pulling 2183734eadff78d4a86a747a8e27183b52882b1b on cedricfarinazzo:fix/linux-signal-handler into 12c8f328e4ba704bd7d91534bb6569e70197b949 on arteria:master.


[bug]: EventListener not functioning on Android, but works perfectly on iOS.

Description
I am attempting to trigger a personal sign when the chainId is changed. This logic functions properly on iOS; however, on Android, the event change is not being executed.
  React.useEffect(() => {
    const handleChainChanged = (newChainId: string) => {
      setLoading(false)
      if (newChainId === chainId) {
        _personalSign();
      }
    }

    if (isConnected) {
      provider?.events.addListener('chainChanged', handleChainChanged);
    }

    return () => {
      if (isConnected) {
        provider?.events.removeAllListeners();
      }
    }
  }, [isConnected, provider]);

WalletConnect Modal SDK version
1.0.0-rc.3
Output of npx react-native info
System:
OS: macOS 13.4
CPU: (10) arm64 Apple M2 Pro
Memory: 61.50 MB / 16.00 GB
Shell: 5.9 - /bin/zsh
Binaries:
Node: 20.2.0 - /opt/homebrew/bin/node
Yarn: 1.22.19 - /opt/homebrew/bin/yarn
npm: 9.6.6 - /opt/homebrew/bin/npm
Watchman: 2<IP_ADDRESS> - /opt/homebrew/bin/watchman
Managers:
CocoaPods: 1.12.1 - /Users/roshan/.rvm/gems/ruby-2.7.6/bin/pod
SDKs:
iOS SDK:
Platforms: DriverKit 22.4, iOS 16.4, macOS 13.3, tvOS 16.4, watchOS 9.4
Android SDK: Not Found
IDEs:
Android Studio: 2022.2 AI-222.4459.24.2221.10121639
Xcode: 14.3.1/14E300c - /usr/bin/xcodebuild
Languages:
Java: 11.0.19 - /usr/bin/javac
npmPackages:
@react-native-community/cli: Not Found
react: 18.2.0 => 18.2.0
react-native: 0.71.8 => 0.71.8
react-native-macos: Not Found
npmGlobalPackages:
react-native: Not Found
Expo Version (if applies)
No response
Steps to reproduce

Execute switchChain.
Add chainsChanged event listener

Snack, code example, screenshot, or link to a repository
React.useEffect(() => {
    const handleChainChanged = (newChainId: string) => {
      if (newChainId === chainId) {
        console.log('execute personal_sign')
      }
    };

    provider.events.addListener('chainChanged', handleChainChanged);

    return () => {
      provider?.events.removeAllListeners();
    };
  }, []);

SwitchChain:
export const switchChain = async (provider: any) => {
  try {
    await provider?.request({
      method: "wallet_switchEthereumChain",
      params: [{ chainId: chainIdHex }],
    });
  } catch (error) {
    await provider.request({
      method: "wallet_addEthereumChain",
      params: [
        {
          chainName,
          chainId: chainIdHex,
          nativeCurrency,
          rpcUrls,
          blockExplorerUrls,
        },
      ],
    });
  }
}

Oh i think i was able to reproduce the issue. Are you using MetaMask to test? Apparently after calling wallet_addEthereumWallet on Android and coming back to the dapp, the chain event doesn't trigger. I'll dig into this and check what's the cause

Hey @ignaciosantise, the problem still exists even after replacing provider?.events.removeAllListeners();. Yes, I'm using MetaMask to test.

@RoshanNepal maybe same with my issue https://github.com/MetaMask/metamask-mobile/issues/6706

@RoshanNepal @nhatnguyenbp yep, it's the same issue. There's nothing i can do from the SDK side, as we are not receiving chainChanged event from MM.
If you open MM again after being redirected to the dapp, you'll see that the event is now sent, and the chain is updated. There's probably a race condition or something on MM side, and they redirect before sending the chainChanged event
https://github.com/WalletConnect/modal-react-native/assets/25931366/13ccf69c-e068-4376-80ab-1047c2a2af89
Im closing this as it's an issue with MM


unbound: add CNAME support

PTR was also asked for, maybe we should make the framework more flexible...
via: https://forum.opnsense.org/index.php?topic=3246.0

@fichtner we can include the unbound cname tag, but its not very likely that it will actually work. From what I can find.

The only time your entry will be returned is if the client queries for a CNAME, which in practice means it'll probably be returned quite rarely
(source https://www.bentasker.co.uk/documentation/linux/279-unbound-adding-custom-dns-records)
The information on the unbound mailing list:
http://unbound.net/pipermail/unbound-users/2009-March/000509.html
Add the tag any way? or close the issue?

Let's keep it on 17.1, wait for feedback. It should be possible to add this in the advanced settings to collect valuable information of how that works out from the actual client side and that will likely decide in the end :)

don't really need this it seems

Lack of alias/cname support in unbound is the only thing keeping me on pfsense. I'd really like this to be implemented.

I believe it should either be in services_unbound.php or services_unbound_host_edit.php.
The implementation is in services_unbound_host_edit.php in the section, "Additional Names for this Host", which allows configuration of aliases (cnames).

Nope.
https://github.com/pfsense/pfsense/search?utf8=✓&q=cname&type=
https://forum.netgate.com/topic/110680/dns-cname-records-posible-with-unbound/5

@fichtner You are right. PfSense's implementation does not create CNAMES although it appears to do just that. It create multiple A records all tied to one IP address in configuration and in the UI, creating the convenience of a CNAME without CNAME support. I will open a different feature request for this to be added as it is terribly useful.
Your search in source for "cname" returns no results because the PfSense implementation referred to these other hosts "aliases". The implementation is in the file I linked.

@MarkNBroadhead we have this as well :)services: unbound dns: overrides


fix(smoosh): merge et al are now smoosh (smooshMap, smooshAll, etc)

Due to the TC39 proposal for Array flatten/flatMap likely being changed to smoosh/smooshMap it was also decided that, to have name parity, Observables should use those names as well.
In the case of RxJS, there are a few more operators with merge in their name that this PR fixes:

merge → smoosh
mergeMap → smooshMap
mergeMapTo → smooshMapTo
mergeAll → smooshAll
mergeScan → smooshScan

Here's some background on smoosh and the issues TC39 has had with the name "flatten":
https://github.com/tc39/proposal-flatMap/pull/56
http://logs.libuv.org/tc39/2018-03-07#01:04:06.355
https://github.com/staltz/prevent-smoosh
This is unfortunately a breaking change, but one that I think everyone agrees with so it shouldn't be a big deal.
:shipit:

@rgbkrk fixed! (assuming all the build variants pass this time)

These changes should be applied to the stable branch, too.

LGTM


Add socket.io-based challenge

Maybe forging an event emission for a challenge that does not really exist but make the client pick it up nonetheless?

---
Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/38594863-add-socket-io-based-challenge?utm_campaign=plugin&utm_content=tracker%2F6283055&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F6283055&utm_medium=issues&utm_source=github).

http://stackoverflow.com/questions/36080207/new-in-socket-io-how-to-prevent-socket-io-client-script-hacking

@cschneider4711 Seems you are a subject-matter expert on Websockets! 😃 I'd happily take ideas for practical bad implementation of socket.io in Juice Shop client or server, that could be turned into a hacking challenge. For example "Fake a solved-challenge notification that contains a link to http://kimminich.de (or https://www.christian-schneider.net)" 😀

Hi @bkimminich @J12934, I read through the documentation of socket.io. I was thinking of an attack where a user connects to the server by writing a small script like the following in a html page.
<script src="socket.io.js"></script>
<script src="https://code.jquery.com/jquery-1.11.1.js"></script>
<script>
  var sock = io('http://localhost:3000');
  sock.on('server started', function(socket){
  console.log('someone connected');
  socket.on('challenge solved', function(msg){
      console.log('message: ' + msg);
    });
  });
</script>

After connecting, the user will broadcast data  with challenge solved which will make the data reach on the original juice-shop client as well which will trigger a solved-challenge notification.
I am trying to achieve this scenario. Please let me know your views on it. Thank You

Although such a scenario is possible only when the server emits the received file to all the clients connected, which I think is the case presently?

@agrawalarpit14 are you still working on this? If not, I would be glad to pick it up. @bkimminich

You can go forward with the same. Thank You


fix: bump goproxy

Description
Bump goproxy
Type of change

[X] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

Testing performed?

[X] Unit tests
[ ] Integration tests
[ ] Acceptance tests

Checklist:

[X] This PR is being made against the main branch, or relevant version branch
[ ] I have made corresponding changes to the documentation
[ ] I have added testing for my changes

If you have any questions, or want to get attention for a PR or issue please reach out on the #logging-and-metrics channel in the cloudfoundry slack

I renamed this PR title prefix to fix since there's a bug fix involved, even if it's a false positive. :)


Constant Empty Optional

Instead of allocating a new optional empty for every bean.

Nice!!  Thanks for removing those empty files ... not sure how I managed to leave them there :)


Show only content relevant to logged in user in the Grants and Submissions tables

Work in progress PR to update various views to show only content relevant to the currently logged in user.

[x] Grants table should show only grants where currently user is the PI #142
[x] Submissions table should show only those submissions where the current user is the managing user (submission.user === current_user)
Grant-picker in the submission workflow should contain only grants where the current user is the PI #322 Will be rolled into separate PR

Will include an efficiency pass for using the search service (#280)

LGRTM 👌👍🙌


Mail plugin not working?

New to Drone here. I have Drone building my tests, the build fails (that has nothing to do with Drone :wink:) but there seems a problem with the notification mail. From the Drone Docker logs:
time="2016-03-05T22:14:17Z" level=info msg="Notification container drone_build_32_notify exited with 1"
time="2016-03-05T22:14:17Z" level=info msg="[info] Pulling image plugins/drone-email:latest\nDrone Email Plugin built from 725e06f\ndial tcp <IP_ADDRESS>:587: getsockopt: connection timed out\n[info] build failed (exit code 1)\n"
time="2016-03-05T22:15:26Z" level=info msg="closed event stream"

I never received a mail from my Drone instance. Using Mandrill as SMTP server on a Google Cloud instance.
notify:
  email:
    from<EMAIL_ADDRESS>    host: smtp.mandrillapp.com
    port: 587
    username<EMAIL_ADDRESS>    password: password
    recipients:
      -<EMAIL_ADDRESS>
Is <IP_ADDRESS> where the plugins are hosted? Does Drone download the plugin when it sees a notify block in the YAML file? Or is the email plugin already present when I installed Drone?

Please use the gitter chat for this kind of questions. We try to reserve the issue tracker for real confirmed bugs.

I forgot to mention that  <IP_ADDRESS> seems to be the ip of your mail server

Sorry, will use the gitter chat.


Enabling Stream Control Transmission Protocol (SCTP) and run verification tests

Signed-off-by: Varad Ahirwadkar<EMAIL_ADDRESS>

/lgtm

/lgtm


Makes compacted topic creation a configurable property.

Codecov Report

Merging #226 into master will increase coverage by <.01%.
The diff coverage is 84.61%.

@@            Coverage Diff             @@
##           master     #226      +/-   ##
==========================================
+ Coverage   92.66%   92.66%   +<.01%
==========================================
  Files         136      136
  Lines        2426     2428       +2
  Branches      129      128       -1
==========================================
+ Hits         2248     2250       +2
  Misses        178      178

Impacted Files
Coverage Δ

...ala/hydra/kafka/services/StreamsManagerActor.scala
92.72% <84.61%> (+0.27%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d454dd5...7086401. Read the comment docs.


Two wrong weapon parts that shouldn't show up as "Unsupported value"

I have found 2 small issues in the save editor with weapon parts showing up as "Unsupported value":

All E-tech launchers use wrong "Material" in the game. Purple Rarity "Material" are used in game instead of E-tech "Material" for launchers. So for example for Vladof E-tech launchers, the save editor should show "GD_Weap_Launchers.ManufacturerMaterials.Mat_Vladof_4" and make "GD_Weap_Launchers.ManufacturerMaterials.Mat_Vladof_5" as "Unsupported value" instead.

http://s1380.photobucket.com/user/Natsu235/media/BL2_Gibbed_Issue_01_zpsuzxtidxz.png.html

On November 21, 2013 a patch fixed the "Crit" SMG to prevent it to spawn as a Purple Rarity weapon and make it Blue Rarity instead (same problem with "Bad Touch" and "Good Touch" but were never fixed for some reasons). The "Body" part was changed from "GD_Weap_SMG.Body.SMG_Body_Maliwan_VarC" to "GD_Weap_SMG.Body.SMG_Body_Maliwan_VarB". So the save editor should not show "Unsupported value" on VarB Body for the Crit.

http://s1380.photobucket.com/user/Natsu235/media/BL2_Gibbed_Issue_02_zpsljjyyiqd.png.html
Hope it helps.
And btw, are you planning to do a "Fast Travel" tab to Pre-Sequel save editor or is there any reason to not have made it ?

Should be fixed as of https://github.com/gibbed/Gibbed.Borderlands2/commit/844ce3427b877f8c8d032df1434fe9dd3edabe86.


New SyntaxError when sending multiple lines to Terminal

SendText Plus works great! Thanks for all of the work to make it. I just started to get a new error though. If I highlight multiple lines and hit Cmd+Enter to send them to Terminal, the lines run, but for some reason SendText (or SublimeText) started appending '--' to the end of the lines. The result is a SyntaxError:
For example, I sent the following five lines (including comments) to terminal by highlighting them in SublimeText and hitting Cmd+Enter. The lines run fine, but I receive:
>>> %cpaste -q
  File "<stdin>", line 1
    %cpaste -q
    ^
SyntaxError: invalid syntax

>>> ### FRUIT MASS CLEAN
... # drop subplot 8D which is absnormally low
... fruitMass2 = fruitMass[~fruitMass['Subplot'].isin(['8D'])]
>>> #average by subplot
... fruitMass_Plot = fruitMass2.groupby(['Plot', 'Subplot', 'Temperature', 'Herbivory'], as_index = False)['FruitMass'].mean()
 >>> --
  File "<stdin>", line 1
    --
     ^
 SyntaxError: invalid syntax

This error doesn't appear if I run lines one by one, or cut and paste from Sublime into Terminal. It DOES, however, appear when I run a code block (specified by # %%). So there's something weird about highlighting or multiple lines?

it is specified to ipython. Since there is no way to correctly "copy and paste" commands to python console without the %cpaste magic. Try copy and paste all of the below and you will know why
def f():
    x = 1

    # the space above would cause error
    print(x)

I quite literally just figured this out on my own, as well. This requires that I run the iPython notebook then?

Not the notebook, but the ipython console,

Perfect. Thanks!

Isn't it possible to support bpython  console? Because %cpaste blows up. Perhaps we may have a chance to set paste way in user settings?

It is definitely possible, what does bpython handle copy and paste?

I will close this thread for now.
@isezen you could open a separate thread for feature request.


Clicking on date picker icon redirecting to #{elementId}

I observed that after calling 'createDatePicker' function if I quickly click calendar icon, page is redirected to #elementId url (like http:localhost:9000/home#input-element-id). there is anchor tag created by the datepicker with href = #input-element-id. before click event is bind to this element if I am clicking on it(which is nothing but calendar icon) the above behaviour is observed
did any one face the issue?  Please help me to fix the issue

I'm having the same issue, was there a solution to this ?


How to add custom css class

I am using Nuxt 3 and Tailwind CSS. I have implemented vue-quill. I want to add custom css class in editor's elements e.g. blockquote, ul, li etc.
Reason:-
Tailwind CSS removes number and bullet from
 and
 tags so when i display Editor contents on web page All
 and
 are without number and bullet. I need to inject tailwind css class to make them work properly.
So far i have code as below

<template>

 <form ref="form" @submit.prevent="submitProductForm">
  <ClientOnly>

Content

   <QuillEditor theme="snow" toolbar="full" placeholder="write content..." v-model:content="formData.content" />
  </ClientOnly>
  <button type="submit">Submit</button>

</template>

@rahul37865 Hi! You can use style tag with scss to customize individual elements by using @apply with tailwind classes. Example:


docs(cost-insights: Add note about sidebar to README

Hey, I just made a Pull Request!

The Cost Insights README doesn't actually say to the user how to get the link visible, so thought I'd add that.  No changeset so we avoid a version update.
:heavy_check_mark: Checklist

[ ] A changeset describing the change and affected packages. (more info)
[x] Added or updated documentation
[ ] Tests for new functionality and regression tests for bug fixes
[ ] Screenshots attached (for UI changes)

Tugboat has finished building the preview for this pull request!
Link:

https://pr4608-h4taxdblwflnmlstrnpeflqzf7dyfat5.tugboat.qa

Dashboard:

https://dashboard.tugboat.qa/6030279ff285af7ef59397fe


Grade Calculating System

Closes #275

please follow contributing.md file guidance

add project details there main.html


Implement Continuous Deployment Strategy

📖 Additional Details

Follow-on to https://github.com/NASA-PDS/devops/issues/3 design

:calendar: November status: Delayed state date due to delay in #13

📆 December Status: Delayed start due to work on PLAID and PDS Deep Archive tasks. System build is no dependent upon task completion.

📆 January status: Delayed start due to work on PLAID and PDS Deep Archive tasks. beta dev task so no impact on build deliverables

@nutjob4life will start by an anlysis of that (create a ticket for it)

Continuous Deployment

Interesting cases (lanes): Continuous Deployment (CD) Trigger + Venue
Continuous Deployment Trigger

PR to main
Start to Registry + API but should flow down to other components

Venue

Developers should have access to a consist PDS API that always works and to test against

Could be publicly accessible

"Dev demo": pds-dev-?.jpl.nasa.gov
Nightly deployment: all apps onto a developer-accessible server

Or manually

Access: pds.nasa.gov/demo

Implementation

Jenkins for on-prem
Scope: strictly internal for now

No concerns with HTTP Basic challenges or other access issues
Other approaches for in-cloud (for the future)

Future Stages

pds-int @jpl-jengelke  is ahead of the curve on this front

👉 Note: true "demo" may be something we can capture as a story (for the future). Developer staging is something else.

@ramesh-maddegoda  and @nutjob4life need to meet to have the actual registry started by jenkins.
The dev deployment should not run on pds-int so not to conflict with I&T deployments. We want to deploy in on one of pds-devX machines.

@nutjob4life An extra node can be added to PDS Jenkins for any of the pds-devX machines. Please contact Rojeh in DSIO.

Request filed

Request satisfied. However, we've got a new issue:
On pds-dev, this command succeeds:
docker container run --rm busybox nslookup -type=A pds-gamma.jpl.nasa.gov

and produces:
Server:		<IP_ADDRESS>
Address:	<IP_ADDRESS>:53

Non-authoritative answer:
Name:	pds-gamma.jpl.nasa.gov
Address: <IP_ADDRESS>

But using this docker-compose.yaml:
---
version: '3.9'
services:
    demo:
        image: busybox
        command: nslookup -type=A pds-gamma.jpl.nasa.gov.
...

and running docker-compose up, it fails with:
tmp-demo-1  | ;; connection timed out; no servers could be reached
tmp-demo-1  |
tmp-demo-1 exited with code 1

The difference? The compose version also creates its own network for the services and sets up its own name resolver, but that resolver is then unreachable and cannot forward any requests.
Try it: it'll work on your desktop and on other systems like edrn-docker.jpl.nasa.gov just fine. But on pds-dev.jpl.nasa.gov and on pds-int.jpl.nasa.gov, no dice.
As a result, the Registry API cannot retrieve the initial set of data to harvest and load into Elasticsearch (the curl to pds-gamma times out since it can't resolve the name).
Rojeh says this is because the way Docker is configured on pds-dev (and pds-int), the system is not allowed to alter firewall rules which is needed to make the containerized networks needed by the applications. So Docker is just "sort of" supported on pds-dev, but anything using a Docker Composition is pretty much out. Docker is set up this way because pds-dev (and pds-int) are on "JPLNet", and modifications to firewall rules are forbidden on such systems.
One alternative is to set up a new non-JPLNet host, say pds-deploy, which can run all the continuously deployed services. It would only be accessible from within JPL, which is fine I think. And it would not be on JPLNet, so it'd have the freedom to actually run things the ways they were meant. And since it'd be off of pds-dev, it'd be better isolated from all the other crazy things that go on there.
Thoughts @jordanpadams @tloubrieu-jpl @jpl-jengelke?

@nutjob4life I don't necessarily know enough about this stuff, but per the containerized networks, there is no way to have the SAs preconfigure that in some capacity? It is automated by docker compose?
I guess I don't understand how this isn't possible since people deploy docker containers all the time in operations, with public access no less. So I think I'm just lacking knowledge of why this is so different.
That being said, we can try AWS for this, but at that point, we should probably being using terraform for the registries, and kind of misses the point of a lot of what we are trying to do here.

Request filed

Is there some way to add me as a watcher, please? I'd like to see what was reported by the sysadmin crew. ...

So something happened between the time that I declared it working and today. Although our regression tests continue to run successfully, the test job created, called docker-test-pipeline no longer functions. It looks like permissions changed on the machine in a bad way or something is simply broken. If they hardened it, then they probably broke it. ...
https://pds-jenkins.jpl.nasa.gov/job/docker-test-pipeline/19/console
I can't even pick up an image to test, but I'm not sure that's the real issue. (We can always pick up images from CAE Jenkins.) I guess I need to chat with them.

Is there some way to add me as a watcher, please? I'd like to see what was reported by the sysadmin crew. ...

@jpl-jengelke  I shared DSIO-1481 with you; but I also shared the more interesting one, DSIO-1495.

If they hardened it, then they probably broke it.

Looking at the log for build 19 of docker-test-pipeline, the failure doesn't seem to be related to Unix socket permissions, but when they added pds-dev as a job agent. Your pipeline is running on pds-dev instead of pds-int, and pds-dev isn't set up right.
If you go to docker-test-pipeline, go to "Configure", add this to the agent block:
pipeline {
    agent {
        docker { … }
        agent { label 'pds-int' }
    }
    …
}

and that might fix it! 😉

Just committing an example working configuration here for posterity:
https://pds-jenkins.jpl.nasa.gov/job/docker-test-pipeline/27/console
pipeline {
    agent {
        label 'pds-int'
    }
    stages {
        stage('Test') {
            agent {
                docker {
                    image 'node:16.13.1-alpine'
                    args '-u 0:0 -e USER=pds4 -e USERNAME=pds4 -e GROUP=pds -v /data:/data:ro'
                    reuseNode true
                }
            }
            steps {
                sh 'node --version'
                ...

@nutjob4life Thank you. The node label must go into the docker code block. I think it picked up pds-devX first due to alpha ordering. That nslookup command is now working in a container in my test job on pds-int, see ...
https://pds-jenkins.jpl.nasa.gov/job/docker-test-pipeline/28/console
pipeline {
    agent {
        docker {
            label 'pds-int'
            image 'node:16.13.1-alpine'
            args '-u 0:0 -e USER=pds4 -e USERNAME=pds4 -e GROUP=pds -v /data:/data:ro'
            reuseNode true
        }
    }
    stages {
        stage('Test') {
            steps {
                sh 'node --version'
                sh 'ls -laF /data/int/tools/'
                sh 'echo "This is a test" >> foo.tst'
                sh 'cat foo.tst'
                sh 'nslookup -type=A pds-gamma.jpl.nasa.gov'
            }
        }
        ...

Of course this doesn't say anything about using compose, but I will try to test that on pds-int next.

OK, I think I got it to work by specifying the network mode, opening ports and specifying a bridge network mode. See the working build here for the success message and the failed build for the error message.
Why did I specify the DNS servers? I vaguely recall working a similar issue with Jeff Liu before he left the Lab. External DNS access was disabled which had some unintended consequences even when running services internally. So we now have to specify DNS in certain cases. Note that I have not tested with host network_mode which might work or be needed in some certain circumstances.
If there's a mistake here or I'm way off base, please let me know. But again, this has only been tested in pds-int and should work in pds-devX.
version: '3.8'
services:
    demo:
        image: busybox
        command: nslookup -type=A pds-gamma.jpl.nasa.gov
        network_mode: "bridge"
        ports:
            - "80:80"
            - "53:53"
        dns:
            - <IP_ADDRESS>
            - <IP_ADDRESS>

Okay, well, that lets a service resolve an external name. But internal names (service names become hostnames in a Docker Composition) don't resolve. For example:
---
version: '3.9'
services:
    dependent:
        image: busybox
        command: nc -l -v -p 4000 -s <IP_ADDRESS> -i 5
    demo:
        image: busybox
        entrypoint: /bin/sh
        command: -c 'echo hello | nc -v dependent 4000'
...

The demo service calls nc passing the hostname dependent, which Docker resolves to be address for the container also named dependent. I tried adding network_mode, ports, dns and even included Docker's own internal DNS, <IP_ADDRESS> (which is the pseudo-DNS server that maps hostnames onto service names in a Docker Composition).
What you should get (after 10 seconds):
Starting tmp_dependent_1 ... done
Starting tmp_demo_1      ... done
Attaching to tmp_dependent_1, tmp_demo_1
demo_1       | dependent (<IP_ADDRESS>:4000) open
dependent_1  | listening on <IP_ADDRESS>:4000 ...
dependent_1  | connect to <IP_ADDRESS>:4000 from tmp_demo_1.tmp_default:45629 (<IP_ADDRESS>:45629)
dependent_1  | hello
tmp_demo_1 exited with code 0
tmp_dependent_1 exited with code 0

What you get on pds-dev:
tmp-demo-1       | nc: bad address 'dependent'

Can you figure out how to get internal hostname (service name) resolution working @jpl-jengelke?
@jordanpadams I still think we should have pds-deploy as a new VM not on JPLNet.

@nutjob4life @jordanpadams I second the plan to have a separate deploy server if they'll do it.

Can you figure out how to get internal hostname (service name) resolution working @jpl-jengelke?
Probably. But it might involve setting up a combination of an /etc/hosts file inside containers and/or using bridge networking to connect to each other by exposing ports (expose keyword). I suspect it can happen with some detailed setup. I'm happy to look into it if I can make some time later in the week.

Yeesh, that sounds like it could kind of brittle and would involve services figuring out what IPs got assigned to them and then sharing that (perhaps on a message bussing queue) so each one could adjust its own /etc/hosts and then communicate with the other services it needs. (And then there's bootstrapping the message queue itself!)
And it's not in the spirit of a Docker Composition 😇
Ultimately, the sys admins have given us a hobbled Docker environment that doesn't support the some of the most essential use cases advertised by Docker.
@jordanpadams I really want to push for a separate VM not on JPLNet that can be called pds-deploy or something. We use this pattern on the Early Detection Research Network: various services like edrn.jpl.nasa.gov/cancerdataexpo or edrn.jpl.nasa.gov/portal or mcl.jpl.nasa.gov/portal all reverse-proxy to a host edrn-docker which is not on JPLNet and therefore runs a fully-functional Docker environemnt. No bullcrap.

...
And it's not in the spirit of a Docker Composition 😇
Ultimately, the sys admins have given us a hobbled Docker environment that doesn't support the some of the most essential use cases advertised by Docker.
...

Not arguing against a separate VM. I've seen environments locked down similarly which is why I think Docker evolved to provide extra controls. We might not need to specify /etc/hosts files if we use host.docker.internal. This could allow us to connect container-to-container running on the same node. I don't know if it would be possible to hop between containers outside of one node, but I'm not sure that's the use case.

@nutjob4life per:

@jordanpadams I really want to push for a separate VM not on JPLNet

Copy. This will have to happen in AWS then (hopefully not too expensive). As far and I understand it, a VM outside of JPLNet requires a DMZ and all kinds of approvals and such that we can't do.

📆 February status: Research has been completed. But task will be ongoing into B13.0 tasks

Just a quick FYI, a non-JPLNet VM is trivial to set up—even easier than provisioning AWS.

@nutjob4life awesome! let's do it then.

Request filed

:calendar: March status: design and implementation underway. going to defer the rest of the implementation to B13.0. consider this closed, but going to keep it open for now and move to B13.0 because of the detailed conversation above.

moving this out of the Sprint Backlog. See sub-tasks for details

📆 May status: In progress. On schedule

:calendar: June status: Harness investigation ongoing. On schedule

📆 July status: Harness investigation ongoing. Completion delayed awaiting decision on Harness + NGAP + issues with subcontractor transitioning from Columbus to APR.

Pilots of Harness and other tools demo-ed. Going to close out this task as completed for B13.0 and will revisit in a future build when it makes sense to deploy Harness operationally in NGAP.


On a colorspace with green, selected tools disappear

Esspecially with the previous color map, the toolbox could 'disappear' when selected because it would take the same color as the background.

This would be an issue with any color we choose for the icons, so I'm going to put this in the won't-fix box.


small details pane for toggle and other options

Signed-off-by: vineethvanga18<EMAIL_ADDRESS>Description

Notes for Reviewers
Signed commits

[x] Yes, I signed my commits.

Thank you, @vineethvanga18. It may be that we will want for the background color of this panel to differ slightly from that of the global navigation menu on the left given that this navigation menu is constrained to the MeshMap view.