Adding normal tokens

Architecture
Each transaction output must have a data field. This field is the designation of the purpose of the transaction, we must highlight the following at the moment:

TokenTransferV1 - Transfer Tokens or NFT
TokenIssuanceV1 - Token Issuance
TokenBurnV1 - Burning token
NftIssuanceV1 - NFT creation

Below, a data field representation in TxOutput:
    #[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Encode, Decode)]
    pub struct TxOutput {
        value: OutputValue,
        purpose: OutputPurpose,
    }

    pub enum OutputValue {
        Coin(Amount),
        Asset(AssetData)
    }

    #[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Encode, Decode)]
    pub enum AssetData {
        // TokenTransfer data to another user. If it is a token, then the token data must also be transferred to the recipient.
        #[codec(index = 1)]
        TokenTransferV1 { token_id: TokenId, amount: Amount },
        // A new token creation
        #[codec(index = 2)]
        TokenIssuanceV1 {
            token_ticker: Vec<u8>,
            amount_to_issue: Amount,
            // Should be not more than 18 numbers
            number_of_decimals: u8,
            metadata_uri: Vec<u8>,
        },
        // Burning a token or NFT
        #[codec(index = 3)]
        TokenBurnV1 {
            token_id: TokenId,
            amount_to_burn: Amount,
        },
        // Increase amount of tokens
        #[codec(index = 4)]
        TokenReissueV1 {
            token_id: TokenId,
            amount_to_issue: Amount,
        },
        // A new NFT creation
        #[codec(index = 5)]
        NftIssuanceV1 {
            data_hash: NftDataHash,
            metadata_uri: Vec<u8>,
        },
    }

Token Issuance
For issuing a new token, we have to create a new transaction with correct filled up Some(TxOutput.data).

token_ticker - A short name of the issueing token.
amount_to_issue - Amount of new tokens.
number_of_decimals - Token might be integer or float. There is possible to use any number of digits after decimal point, if it not more than 18.
metadata_uri - A distinct Uniform Resource Identifier (URI) for a given token. There is might be additional information about token (Full name, description, image, array of custom  properties).

Inputs should have enough amount of MLT to pay creation fee.
After signing and spending the transaction, the token will be created. To operate it, you should use token_id that essentially calculate from hash of all inputs of the issue transaction.
There is a possibility to fetch information about TX that created the token by token_id.
Obviously, ownership provided by UTXO architecture. No need to add additional storage data.
[TODO] The URI MUST point to a JSON file that conforms to the "ERC-1155 Metadata URI JSON Schema"? Or we can develop our format?
[TODO] token_id is a hash of all inputs, or it's a hash of the first input? Or, it's hash of TX that created the token?
Token Mint
Sometimes, we have to increase amount of issued tokens. It is possible to do only if reissue transaction spend by token creator. Otherwise, tx must be rejected.

token_id - ID of the token (see Token Issuance).
amount_to_issue - It's amount of the tokens that will be issue on creator destination.

Token burn
For decrease amount of tokens. Token creator can execute a tx with data:

token_id - ID of the token (see Token Issuance).
amount_to_burn - It's amount of the tokens that will be burn.

If amount in output is less than in the input then should exist at least one output for returning the funds change. In this case, you can burn any desired number of tokens. After this operation, you can use UTXO for the remaining amount of tokens.
[TODO] Which is destination for output must be?
NFT creation
For minting a new NFT, we have to create a new transaction with correct filled up Some(TxOutput.data):

data_hash - The field essentially is a hash of external data, which should be taken from the digital asset.
metadata_uri - A distinct Uniform Resource Identifier (URI) for a given NFT. There is might be additional information about NFT (Full name, description, image, array of custom  properties).

Inputs should have enough amount of MLT to pay creation fee.
[TODO] What kind of hash we should use?
[TODO] Might be worth if we research JSON formats for metadata_uri that usually use NFT marketplaces, but anyway, this is area of user responsibility.
Transfer normal token or NFT
Transfer assets to another person in a given UTXO.

token_id - ID of the token or NFT (see Token Issuance and NFT creation).
amount - Desirable amount of tokens. For NFT this field must be ignored.

Versioning
Adding a new version of data is in fact adding a new field to the enum, if the names match, add the version number at the end, for example:
TokenTransferV1
TokenTransferV2
etc

What happened if someone accidentally changed in OutputData arm order or fields order? Nothing good. It's 100% entail incompatibility with other versions of software.
There is only one way to protect fields order - make unit tests and snapshot tests.

A couple of observations here:
Why are transfers of MLTs separate from other tokens? Could we just use AssetData::TokenTransfer for MLTs and treat MLTs and other stuff uniformly? The extra OutputValue enum would become redundant, using AssetData directly.
Burning a token could be just sending it to an unspendable destination (we can have a separate destination for this to be more explicit), regardless whether it's a token or a coin. That would also allow us to treat tokens and MLTs more uniformly. A destination seems more appropriate for burning tokens than
Another related idea is that we don't have to mention the big TokenIDs (hashes) directly in transactions. The UTXO set could track the full token IDs, and the transaction could just refer to some input index that identifies the token being used, which could be a u8 or a u16 rather than full hash. Would this open some attacks? It seems to me that with some sighash types, the sighash would have to include the full token IDs or some other mitigation, so the small local IDs are guaranteed to refer to the correct global IDs even if the transaction is not signed in full. Any other comments?

Why are transfers of MLTs separate from other tokens? Could we just use AssetData::TokenTransfer for MLTs and treat MLTs and other stuff uniformly? The extra OutputValue enum would become redundant, using AssetData directly.

Because they are different types of assets. One is the native token and one is just arbitrary data that we make rules for modification. I can see how they can be unified, but maybe later after we understand the outcome better.

Burning a token could be just sending it to an unspendable destination (we can have a separate destination for this to be more explicit), regardless whether it's a token or a coin. That would also allow us to treat tokens and MLTs more uniformly. A destination seems more appropriate for burning tokens than

This was a design decision that we made back then in the substrate days. The problem here is that many things have to be consolidated together, one of which is how we treat tokens that are in inputs but not outputs. We basically will prevent that at the protocol level, and the user has to explicitly specify the burn. A specific burn command is always better as it provides clarity.

Another related idea is that we don't have to mention the big TokenIDs (hashes) directly in transactions. The UTXO set could track the full token IDs, and the transaction could just refer to some input index that identifies the token being used, which could be a u8 or a u16 rather than full hash. Would this open some attacks? It seems to me that with some sighash types, the sighash would have to include the full token IDs or some other mitigation, so the small local IDs are guaranteed to refer to the correct global IDs even if the transaction is not signed in full. Any other comments?

The only problem with that is that transaction validation will not be possible without a full node. It's another design decision. In other words, if someone sends you a transaction with tokens that does tracking like you say, there will be no way to tell whether it's valid without checking the UTXO set, which is very inconvenient. Whether we can use a shorter ID is an open question. The primary issue there is that token IDs will be malleable and a reorg can change the token id, which is a big risk. Perhaps there's a design that solves that problem... not sure.

As Sam mentioned today on the daily, NFT we add in another branch later

You seem to have forgotten something. You need some storage in chainstate storage for tokens on issuance so they can be consulted when knowing about the token. Currently you're just checking utxos.

OK, I haven't added storage part, because thought it needs only for re-issuance, that we decide to do later.

I'm sorry. This implementation is not good enough to be merged. Let me suggest the way to start doing this right.
The functions calculate_coins_total_inputs() and calculate_coins_total_outputs() should return a BTreeMap<CoinOrTokenId, Amount>. The CoinOrTokenId should be an enum that's either a token id or a coin.
The check functions must be 100% immutable, and obviously shouldn't depend on history or be in tx_verifier (which you mostly have already, except for the immutability). Meaning: While connecting a transaction, we check everything in that transaction (we double-check that the token data is valid). But if we don't connect it, it just gets checked once in check_block().
You can create a separate function in transaction_verifier::connect_transaction() that will register issuance. I'm not sure how this should be done, but you shouldn't make registering issuance a part of checking. Checking is by definition immutable.
In summary, when you connect a transaction, you check the token data, THEN check the total inputs/outputs, THEN register issuance. If you can find a cleaner way to do this, I don't mind. But in its current form, it's too complex.
Finally, all token-specific operations/functions should be moved to separate submodules.

I'm wondering whether we should close this PR. I suggest we close this PR because it got really big with way too many comments. Then, once you're done and have addressed all the issues in it, create a new one. Let me know, guys, if this makes sense.

Sadly, but make sense! Thank you for a review, guys. See you during the next attempt to merge it.
@TheQuantumPhysicist @muursh @stanislav-tkach @azarovh

Sadly, but make sense! Thank you for a review, guys. See you during the next attempt to merge it.
@TheQuantumPhysicist @muursh @stanislav-tkach @azarovh

Please don't forget to address all the comments in the PR.


Write integration tests

Need to look into how to properly test code generator tools like this one
Meaningful integration tests would probably look like this:

We have a few example schemas in fixtures

For each schema, we have a few example query fixtures

fixtures that are supposed to compile and run correctly
fixtures that don't compile (e.g. TS should complain about the types of arguments or outputs)

Assertions about types can probably be made in the format let query: TypedDocumentNode<InputType, OutputType>

Assertions about the generated queries can probably be done on the .source property of typed document nodes, or on the node object itself

Edge cases should include:

realistic/complex Hasura queries with input variables
custom scalars
enums
unions
nullable and array inputs and/or outputs
optional vs required arguments

Problems to solve:

re-generate schema codegens on test run if necessary
re-run TS compilation of codegen on test run
re-run TS compilation of example files

if type errors are expected, assert that
if not, load and run queries from the example files

runtime assertions should probably also be exported from the fixture files
node-sql testing style can be used - see https://node-sql-examples.github.io/ for a lits of all the builder/SQL examples present in the node-sql test suite.

similarly the tests can be used to organize the documentation examples.

See https://github.com/spion/typed-graphql-builder/blob/main/test/basic.test.ts#L9 for a more detailed schema


Shared voicemail for callers to leave a message for an organization.

Please validate this " Shared Voicemail". It does not exist in Call Queue GUI however It does exist in Auto Attendant Queue.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: b755d9ae-13ac-019f-f5f4-a95b1eee2989
Version Independent ID: f7afd8ad-e6e0-6c79-aed1-4ee87d48ba12
Content: Create a call queue - Microsoft Teams
Content Source: Teams/create-a-phone-system-call-queue.md
Service: msteams
GitHub Login: @CarolynRowe
Microsoft Alias: crowe

@rickleye thank you for submitting feedback.
I think the best way forward is if you open a service ticket in your tenant so this can get resolved ASAP. Based on the outcome let me know if it is something that can be called out in the docs.
Please follow this link to contact support for business products:https://docs.microsoft.com/office365/admin/contact-support-for-business-products
Please keep us posted here on the resolution so we can feed whatever information you discover into the content.


RC2: Can't debug.  "Unable to attach to application"

Happens on some machines (consistently), but not others.  I'm unable to debug any of the RC2 web templates (Core or Framework).  I get the following errors:
"Unable to attach to application 'dotnet.exe'..."
http://imgur.com/3aMoDWH

/cc @BillHiebert @barrytang

Fixed by...

Reinstalling WIndows 10
Reinstalling VS2015
Reinstalling Core RC2 update

Rehosting image:

FYI, the blacked out text is the MACHINE NAME.

In case anyone else winds up running into this: this is an issue reported from webservices.dll, which is the transport used by the debugger to communicate between its two processes. Its possible that it is an issue in webservices itself, but probably the most likely explanation is a problem with the networking stack on the computer. Certainly one way of resolving this is what Simon did - reinstall Windows. Other possibilities would be to look at what networking software is installed on the computer and try removing or repairing it.

To add some context:

This was not a problem before the move to RC2.  Everything worked fine on RC1 (and RC1 projects continued to work properly after installing RC2).

I eventually noticed that the IIS Express systray icon briefly flashed (when pressing F5 to run project) but immediately disappeared.

Not sure what any of that means, just documenting it here.

I'm getting the same issue, after uninstalling RC1 and installing RC2.  I don't really want to reinstall windows and visual studio 2015 and rc2.  Is there any other fix?

@conterio I tried uninstall every Microsoft developement related program or feature (including VS) and then reinstalling.  That didn't work.  Eventually I gave up and wiped the whole machine.  According to @gitter-badger , you might try looking at networking software on your machine to see if there is a culprit there.

@SimonOrdo Success! What it was is I'm using Sophos Agent, I turned off the service and it works, not sure why I need to disable the service with RC2 but not RC1.  either way thanks

@conterio @gregg-miskelly
I too had Sophos installed on that machine before I wiped it.  I have not reinstalled it yet, so perhaps that is why my installation is working now.
I will have it reinstalled and we'll see how it does.
It was working with Sophos in RC1.

My guess is that your RC1 project was using a 32-bit process, and your RC2 project is going to be 64-bit.
If you wouldn't mind, please report the issue to Sophos. This is almost certainly their bug.

@gregg-miskelly
You got it.  Unfortunately, I still don't understand what exactly is happening, so I'm not sure what to report.  Any idea, in general terms what's happening?  If I tell them my debugger can't attach to an ASP.NET RC2 web project, I'm not sure they're going to know what I'm talking about.

You can tell them that they are interfering with localhost traffic causing Visual Studio to be unable to debug. My guess would be that all traffic on dynamic ports to ::1 is broken, but of course, maybe it is a more specific bug.
I believe you could see the same problem when debugging any 64-bit process. You could start 64-bit notepad, and try and attach (Debug->Attach To Process) to confirm or deny this.

I will check it and report back when I have Sophos reinstalled.  Closing for now.

Also getting this. I've installed VS2015 and RC2 on a clean copy of windows. I don't have Sophos.  Should I format? I came here from linux and php dev because that was such a f*** hack.

yeaaaaaaaaah! Just ran a repair on VS. Problem fixed. I'm really impressed with what MS and the community are doing with ASP.net. This is a game changer.

I've exactly the same problem with Visual studio 2017 community edition.
Tried to disable almost all services that use network. no results
I've also this problem with Visual Studio 2015. I can't reinstall windows or other applicatios because it is a company computer.  Don't know what to do....  Hope there is a solution......

@coolrix The error you are getting is "Unrecognized error occurred in the Windows Web Services framework"?
To be clear, if that is the error you are seeing (or one of the other related problems) -- using the network isn't a problem. You could have many services operating on the network and you will be fine. The problem tends to be with horrible security software that does all sorts of nasty things to the Windows networking stack and causes it to no longer work. If so, you may be able to configure it such that it no longer breaks your network, or swap it out for something less buggy.

Thank you so much @Xeevis and @gregg-miskelly, NordVPN was also causing my issue.

Just ran into the same situation.  In my case it was IPVanishVPN.  Worked just fine until today.

FYI In Visual Studio 2017 version 15.7 (preview 3 and later), the debugger team added a new mode so that local 64-bit debugging can be configured to communicate between processes using anonymous pipes instead of TCP/IP, and thus working around these problems.
To enable this today:

Install Visual Studio from the preview channel (https://www.visualstudio.com/vs/preview/). This step will go away very soon when 15.7 enters the release channel.
Go to the Windows Start Menu and launch the “Developer Command Prompt for VS 2017 Preview”
Run: "%DevEnvDir%vsregedit.exe" set "%VSINSTALLDIR:~0,-1%" HKCU Debugger UseAnonymousPipes dword 1
Restart VS if it is already running


How can I get a specific region or country's location using search option

I have solved the problem. To find specific region it requires a property called region. Which is sent through google places api and fetch places.  It requires two digit country code  for example Bangladesh code is 'bd'.
A sample code is given below.
 <GooglePlacesAutocomplete
                       {..............}
                        query={{
                        // available options: https://developers.google.com/places/web-service/autocomplete
                        key: 'YOUR_API_KEY',
                        language: 'en', // language of the results
                        types: 'geocode' ,//[ "locality", "route", "geocode" ], // default: 'geocode'
                        region: 'bd'
                    }}
                    {..............}
                  />


[bazel-test-failure]Memory leaks with several ASAN QPS benchmarks

See logs below:
All tests with leaks: https://source.cloud.google.com/results/invocations/361eb033-92ac-49e9-be47-c57b687929a3/targets
Details of the leak: https://source.cloud.google.com/results/invocations/361eb033-92ac-49e9-be47-c57b687929a3/targets/%2F%2Ftest%2Fcpp%2Fqps:qps_json_driver_test_1@poller%3Depollex/log

Assigning to dgq for initial triage.

@ncteisen do you own these?

Changing to P1 as said tests are failing with 100% rate on master.

Adding some keywords for easy searching -
//test/cpp/qps:qps_json_driver_test_cpp_generic_async_streaming_qps_1channel_1MBmsg_insecure@poller=epollex
qps_json_driver_test

Due to the heavy weight nature of treating QPS benchmarks as unit tests, we exclude running these under ASAN and TSAN.
See here.
I think we should also exclude these for Bazel builds

@billfeng327, can you take over from here and disable ASAN and TSAN for QPS test?

Seems fixed by #16568.


simplify code

change interface{} to any
move templateLen, formattedStr after early exit, as they are not needed before it
reduce in loop indent with if ... { ... continue } else { ... } to if ... { ... continue} ...
simplify switch-case in getItemAsStr func
prettify tests code, organize into stringFormatter_test package in order not to export some test symbols, organize tests into table tests
change MapToString to accept just map, instead of pointer to it, and format instead of predefined strings, preallocate buffer which reduced memory consumption a bit
updated readme with according changes and some fixes

All tests are working, benchmark still shows slight performance advance in comparison to fmt, moving to code review


MessageTracker broken for sending messages

Since pzymq version 17.0.0 the MessageTracker does not behave as descibed in the documentation anymore. When track is activated on a send, the MessageTracker done method always returns True even if no receiver is running. In version 16.0.2 this was still working correctly.
This behaviour could be reproduces on Debian 9 and Fedora 28 with Python 2.7, 3.5 and 3.6 with the following code snippet:
from __future__ import print_function

import zmq
import time

def main():
    context = zmq.Context()
    socket = context.socket(zmq.PUSH)
    socket.connect("tcp://localhost:50300")

    print("using zmq version", zmq.__version__)
    print("using libzmq version", zmq.zmq_version())

    message = b"World"
    print("Send: ", message)
    res = socket.send(message, copy=False, track=True)

    if res.done:
        print("res: done")
    else:
        print("res: waiting")
        res.wait()
        print("res: waiting...")

    socket.close(linger=0)

if __name__ == "__main__":
    main()

Output for me:
Version 16.0.2 works
using zmq version 16.0.2
using libzmq version 4.1.6
Send:  World
res: waiting
^CTraceback (most recent call last):
  File "test/coding_tests/zmq_test/track_tests/run_track.py", line 27, in <module>
main()
  File "test/coding_tests/zmq_test/track_tests/run_track.py", line 21, in main
res.wait()
  File "/home/kuhnm/projects/hidra/virtualenvs/zmq_tests/local/lib/python2.7/site-packages/zmq/sugar/tracker.py", line 105, in wait
evt.wait(timeout=remaining)
  File "/usr/lib/python2.7/threading.py", line 614, in wait
self.__cond.wait(timeout)
  File "/usr/lib/python2.7/threading.py", line 359, in wait
_sleep(delay)
KeyboardInterrupt

17.0.0 -> broken
using zmq version 17.0.0
using libzmq version 4.1.6
Send:  World
res: done

18.1.1 -> broken
using zmq version 18.1.1
using libzmq version 4.3.2
Send:  World
res: done

Short answer: an optimization is causing small messages to be sent as if with copy=True even when called with copy=False, and copied messages cannot be tracked. You can disable this optimization by setting zmq.COPY_THRESHOLD = 0.
A more "zmq way" to wait for peers would be to wait for a message from the peer, rather than trying to observe if a message is sent to the peer, which I would recommend if that's an option available to you.
Long answer
MessageTracker doesn't directly track whether a message is sent, rather it tracks when libzmq is done using the shared memory buffer created with copy=False. Since libzmq sending is async and Python needs to know when libzmq is done with it, pyzmq has a garbage collection mechanism for making sure the Python interpreter doesn't free memory libzmq is still using (segfault), or application code doesn't overwrite it if it is an editable buffer (corrupted messages). This doesn't happen when copy=True because the buffer is not shared.
pyzmq 17 introduced zmq.COPY_THRESHOLD, a threshold below which messages with copy=False will actually be sent with copy=True, based on the fact that zero-copy is actually much more expensive for small messages than copying, due to the bookkeeping overhead (a small memcpy is pretty cheap compared to thread-syncing events). The result is that for small messages sent with copy=False below zmq.COPY_THRESHOLD, zmq is done with the shared buffer before the method returns and the corresponding tracker returned is also done immediately. The default value (64kB) is based on some testing, but is tunable by setting zmq.COPY_THRESHOLD for folks who want to do their own measurements.
This was introduced in pyzmq 17 becuase zero-copy is generally used as a performance optimization for large messages, but it is tedious in send (and impossible in recv) to pass copy=False only for the large frames at the application level. Doing it inside pyzmq with a global threshold is easier and makes the default behavior much more performant, and also is the only way to make it work for recv, because the size of the message to be received is not known beforehand.

Thank you very much for the detailed description and sorry for the late response. The workaround with zmq.COPY_THRESHOLD = 0 works like a charm and for future projects this explanation helps a lot.


typo: double "security"

@nschonni I need to check with the vendor that is merging these back to our private repo - I remember this one - the double security was an error, but the fix was to update the path to "/vsts/organizations/security/get-started-stakeholder" because that file had moved. I made that change (well I need to run it down and see where I made it so I can figure out what happened), but now it is back to the "old" way and I need to figure out what happened.


Only update auto paths when the configuration is changed

Updating every second was fine, but now because of the camera pipeline switching we can only update about every 5 seconds, which makes it really annoying to use.

Fixed some time ago idk or care when. It's very nice.


Documentation: Fixes Error in Triggers

Pull Request Template
Description
Fixes error in trigger documentation.
Type of change
Please delete options that are not relevant.

[] Bug fix (non-breaking change which fixes an issue)
[] New feature (non-breaking change which adds functionality)
[] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[] This change requires a documentation update

Closing issues
To automatically close an issue: closes #2591

/azp run

@Venkat since you are a member of Azure can you please clone the repository directly instead of using a fork. The gates have to be manually triggered for forks because of security reasons. If you create a branch it will run automatically.

sign off

/azp run


Error: Could not find or load main class HLA

Follow the instruction step by step, and mnv install was done successfully.
Why the error message still show up after I type
java -jar target/Kourami.jar

Thanks for trying out Kourami.
It's strange that mvn install was successful and you are getting the error.
In order for me to resolve the error for you, could you please provide little more detail? (listed below. please provide the output from each command):

mvn version and the version of jdk your mvn is using by running:

mvn -version

java version you are using:

java -version

content of the mamnifest file in your jar:

unzip -p target/Kourami.jar META-INF/MANIFEST.MF

List of dependency jars:

ls -l target/dependency-jars/

Finally, could you also try the precompiled version by downloading from the latest release ( https://github.com/Kingsford-Group/kourami/releases/download/v0.9.4/kourami_v0.9.4_bin.tar )?

Thanks for your reply! I successfully run that with your pre-compiled version. But I still did not get the reason why I failed to install that from source code. Following is the information
1.mvn -version
Apache Maven 3.3.9
Maven home: /usr/share/maven
Java version: 1.8.0_131, vendor: Oracle Corporation
Java home: /usr/lib/jvm/java-8-openjdk-amd64/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "4.4.0-75-generic", arch: "amd64", family: "unix"
2.java -version
openjdk version "1.8.0_131"
OpenJDK Runtime Environment (build 1.8.0_131-8u131-b11-2ubuntu<IP_ADDRESS>-b11)
OpenJDK 64-Bit Server VM (build 25.131-b11, mixed mode)
3.unzip -p target/Kourami.jar META-INF/MANIFEST.MF
Built-By: wangdatou2009
Class-Path: dependency-jars/jgrapht-core-0.9.1.jar dependency-jars/hts
jdk-2.1.1.jar dependency-jars/ngs-java-1.2.2.jar dependency-jars/comm
ons-jexl-2.1.1.jar dependency-jars/commons-logging-1.1.1.jar dependen
cy-jars/snappy-java-1.0.3-rc3.jar dependency-jars/commons-compress-1.
4.1.jar dependency-jars/xz-1.5.jar dependency-jars/ant-1.8.2.jar depe
ndency-jars/ant-launcher-1.8.2.jar dependency-jars/commons-cli-1.4.ja
r dependency-jars/fastutil-7.0.13.jar dependency-jars/jgrapht-ext-0.9
.1.jar dependency-jars/jgraphx-<IP_ADDRESS>.jar dependency-jars/jgraph-5.1
3.0.0.jar dependency-jars/jgrapht-demo-0.9.1.jar
Created-By: Apache Maven 3.3.9
Build-Jdk: 1.8.0_131
Main-Class: HLA
4.ls -l target/dependency-jars/
1933743 Oct 17 17:31 ant-1.8.2.jar
12322 Oct 17 17:31 ant-launcher-1.8.2.jar
53820 Oct 17 17:31 commons-cli-1.4.jar
241367 Oct 17 17:31 commons-compress-1.4.1.jar
267634 Oct 17 17:31 commons-jexl-2.1.1.jar
60686 Oct 17 17:31 commons-logging-1.1.1.jar
17875641 Oct 17 17:31 fastutil-7.0.13.jar
1357827 Oct 17 17:31 htsjdk-2.1.1.jar
296115 Oct 17 17:31 jgraph-<IP_ADDRESS>.jar
351208 Oct 17 17:31 jgrapht-core-0.9.1.jar
12131 Oct 17 17:31 jgrapht-demo-0.9.1.jar
34223 Oct 17 17:31 jgrapht-ext-0.9.1.jar
819906 Oct 17 17:31 jgraphx-<IP_ADDRESS>.jar
60129 Oct 17 17:31 ngs-java-1.2.2.jar
990264 Oct 17 17:31 snappy-java-1.0.3-rc3.jar
99555 Oct 17 17:31 xz-1.5.jar
Again, thanks a lot for you hard work!

Glad that the precompiled jar worked for you. I am also not sure why maven-compiled jar isn't working for you. I suspect it's probably due to how CLASSPATH is set.
Could you try running 2 commands below and see if either of these works? Thank you.
1.
java -cp target/Kourami.jar HLA

java -cp . -jar target/Kourami.jar

1.java -cp target/Kourami.jar HLA
Error: Could not find or load main class HLA
2.java -cp . -jar target/Kourami.jar
Error: Could not find or load main class HLA
3. jar -tfv target/Kourami.jar
Error: Could not find or load main class HLA

Ok. That's really strange. For the third one ( jar -tfv target/Kourami.jar) you should just get contents of the jar file and that error message shouldn't be there.
Can you retry the command? OR since unzip worked previously for you to get the content of the manifest file, perhaps the following command might be better:
unzip -l target/Kourami.jar

Thank you.

Archive:  target/Kourami.jar
Length      Date    Time    Name

    0  2017-10-17 17:31   META-INF/
  776  2017-10-17 17:31   META-INF/MANIFEST.MF
 1559  2017-10-17 17:31   LICENSE
 4818  2017-10-17 17:31   README.md
    0  2017-10-17 17:31   META-INF/maven/
    0  2017-10-17 17:31   META-INF/maven/kingsford-group/
    0  2017-10-17 17:31   META-INF/maven/kingsford-group/Kourami/
 3041  2017-10-07 02:33   META-INF/maven/kingsford-group/Kourami/pom.xml
  107  2017-10-17 17:31   META-INF/maven/kingsford-group/Kourami/pom.properties

10301                     9 files

I think you must've downloaded the precompiled tarball and ran mvn install there. This will create a jar with just the manifest file, readme, license and maven related files. You jar file does not contain any of the class files.
You should either get the latest release of the source ( https://github.com/Kingsford-Group/kourami/archive/v0.9.4.tar.gz ) or you can simply run git clone git@github.com:Kingsford-Group/kourami.git
Then you can move into the "kourami" directory and run mvn install
I am pretty sure this will work but please let me know if it works for you. Thank you.

thanks! I re-downloaded all files, and everything is fine now. Please look at the another question if you have time, it is already midnight EST, right?


Inconsistent validation with complex number

Initial Checks

[X] I confirm that I'm using Pydantic V2

Description
A field expecting a list of float number is valid for a numpy array of complex number, but a field expecting a float number is not valid for a complex number.
I would expect the behavior to be the same, i.e. the complex number is a valid float number.
The output of the example code below is (forget about the ComplexWarning):
/x/python3.11/site-packages/pydantic/main.py:211: ComplexWarning: Casting complex values to real discards the imaginary part
  validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
Traceback (most recent call last):
  File "/x/scratch_10.py", line 10, in <module>
    Model(x=1j, y=array([1.]))
  File "/x/python3.11/site-packages/pydantic/main.py", line 211, in __init__
    validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
pydantic_core._pydantic_core.ValidationError: 1 validation error for Model
x
  Input should be a valid number [type=float_type, input_value=1j, input_type=complex]
    For further information visit https://errors.pydantic.dev/2.9/v/float_type

Example Code
from numpy import array
from pydantic import BaseModel

class Model(BaseModel):
    x: float
    y: list[float]

Model(x=1., y=array([1.]))
Model(x=1j, y=array([1.]))

Python, Pydantic & OS Version
pydantic version: 2.9.0
        pydantic-core version: 2.23.2
          pydantic-core build: profile=release pgo=false
                 install path: /x/python3.11/site-packages/pydantic
               python version: 3.11.10 (main, Sep  9 2024, 00:00:00) [GCC 14.2.1 20240801 (Red Hat 14.2.1-1)]
                     platform: Linux-6.10.9-200.fc40.x86_64-x86_64-with-glibc2.39
             related packages: typing_extensions-4.12.2 typing_extensions-4.12.2
                       commit: unknown

Yep, looks like a bug 👍. Can take a look later this week.

I would like to work on this issue. @sydney-runkle, can you please assign me this issue so that others don't duplicate the effort?

hi @sydney-runkle may I try tackling this issue, thanks!

So I just actually tried out the sample code and played around with that complex coercion when I was reading https://github.com/pydantic/pydantic-core/pull/1574. It seems that this is a numpy-only behaviour and thus I'm not sure if we should implement this for all complex numbers in pydantic.

A field expecting a list of float number is valid for a numpy array of complex number

For example, numpy.array([1j]) has dtype=complex128, not the builtin complex type. These numpy complex types perform the coercion and raises a numpy ComplexWarning in such cases. If you try your sample code with
Model(x=numpy.complex128(1, 2), y=array([1.]))

pydantic won't raise any error either and you'll see the coercion.

a field expecting a float number is not valid for a complex number

It is not valid for a complex number instantiated with the builtin complex type but such fields do accept numpy.complex64/complex128. This, however, is because numpy handles the coercion, not pydantic.
I don't think we should follow this numpy case because coercions can be done differently:

Simply dropping the imaginary part: this is how numpy does it
Accepting only complex numbers with a zero imaginary part: this is what I imagine coercion to be like for complex numbers
No coercion at all: this is the default behaviour of the builtin float type, e.g. float(complex(1, 2)) gives an error, even when the imaginary part is 0

and there's no point for pydantic to simply follow how it's done in numpy. Users might expect different outcomes.


Install Issues

Hi guys,
All things good up to trying to run the program (Ubuntu 16+)

Thoughts?

Hmm are you using Python 2.7.X? when we initially tried to use Python 3 we got that issue but upon using Python 2.7.x it worked. 2.7.10 is my version I used for it.

I just removed python3 and ran the script again. Same error:

Very odd I don't recall doing anything else, however, could you please try to do:
"pip install requests" or "easy_install requests" and then run the program.
It may be that I forgot that we had to install requests, however, I thought we never did.
Note: you may have do sudo the commands above for them to run properly.

Here is a link for reference: http://stackoverflow.com/questions/17309288/importerror-no-module-named-requests

I did a pip list and this showed up:
requests (2.13.0)
So I would also do a pip list pre and post install to ensure you have something similar

I'll give it a try this evening. Out right now.
On Sun, Apr 9, 2017 at 3:00 PM, mshoote<EMAIL_ADDRESS>wrote:

I did a pip list and this showed up:
requests (2.13.0)
So I would also do a pip list pre and post install to ensure you have
something similar
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/Dreizan/Repos-Health-Study/issues/13#issuecomment-292809036,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAoDUPwdzCMf1od-pJKyphohHiVy7ESaks5ruTjzgaJpZM4M1rwI
.

--
Mutual of Omaha Associate Professor
Information Systems
College of Information Science & Technology
University of Nebraska Omaha
http://faculty.ist.unomaha.edu/mgermonprez/cv.txt

This moved me forward. Still not working all the way through.

Seems to be working. Folder structure was a bit off of what you had in your py file

That’s good to hear. Sorry that this was more trouble than it was worth.

On Apr 10, 2017, at 9:20 AM, Matt Germonprez<EMAIL_ADDRESS>wrote:
Seems to be working. Folder structure was a bit off of what you had in your py file
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_Dreizan_Repos-2DHealth-2DStudy_issues_13-23issuecomment-2D292963750&d=DwMCaQ&c=kj4KSSBt5oOaHuCCpWXkZw&r=by6HuT03RCY6YV3JvlhK_0GHFcLqCK5mXPeV7qb6MpQ&m=KxP8Eob0WoEJp8hZ5W0Lii7USlReTiLztVf3mcSfejo&s=sis7syV5bRzGjbx6WRIf194LTUL3N2K8LBmM5LorhPw&e=, or mute the thread https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AKa9dT4Lkc-2DHnT9cJIieguXG6gVGt7lOks5rujqigaJpZM4M1rwI&d=DwMCaQ&c=kj4KSSBt5oOaHuCCpWXkZw&r=by6HuT03RCY6YV3JvlhK_0GHFcLqCK5mXPeV7qb6MpQ&m=KxP8Eob0WoEJp8hZ5W0Lii7USlReTiLztVf3mcSfejo&s=UbqEltmFr17tyPCCQNvvBEesYXGrWylRhxCrOZWJLF4&e=.

I don't mind at all. I think that the errors at the end are preventing the
licenses to be displayed to screen. FYI.
On Mon, Apr 10, 2017 at 9:44 AM, Dreizan<EMAIL_ADDRESS>wrote:

That’s good to hear. Sorry that this was more trouble than it was worth.

On Apr 10, 2017, at 9:20 AM, Matt Germonprez<EMAIL_ADDRESS>wrote:
Seems to be working. Folder structure was a bit off of what you had in
your py file
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub <https://urldefense.
proofpoint.com/v2/url?u=https-3A__github.com_Dreizan_Repos-
2DHealth-2DStudy_issues_13-23issuecomment-2D292963750&d=DwMCaQ&c=
kj4KSSBt5oOaHuCCpWXkZw&r=by6HuT03RCY6YV3JvlhK_0GHFcLqCK5mXPeV7qb6MpQ&m=
KxP8Eob0WoEJp8hZ5W0Lii7USlReTiLztVf3mcSfejo&s=
sis7syV5bRzGjbx6WRIf194LTUL3N2K8LBmM5LorhPw&e=>, or mute the thread <
https://urldefense.proofpoint.com/v2/url?u=https-
3A__github.com_notifications_unsubscribe-2Dauth_AKa9dT4Lkc-
2DHnT9cJIieguXG6gVGt7lOks5rujqigaJpZM4M1rwI&d=DwMCaQ&c=
kj4KSSBt5oOaHuCCpWXkZw&r=by6HuT03RCY6YV3JvlhK_0GHFcLqCK5mXPeV7qb6MpQ&m=
KxP8Eob0WoEJp8hZ5W0Lii7USlReTiLztVf3mcSfejo&s=
UbqEltmFr17tyPCCQNvvBEesYXGrWylRhxCrOZWJLF4&e=>.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/Dreizan/Repos-Health-Study/issues/13#issuecomment-292971079,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAoDUPYOW9DYRMjBXiriHDWKCrn7IKYHks5rukA-gaJpZM4M1rwI
.

--
Mutual of Omaha Associate Professor
Information Systems
College of Information Science & Technology
University of Nebraska Omaha
http://faculty.ist.unomaha.edu/mgermonprez/cv.txt

The JSON issue should be fixed now as scan code updated the way they create the JSON file and the zip files name.


Add discord round announcements

About the PR
This PR adds a system that allows the server to send notifications for rounds starting and ending via a webhook, along with the ability to set a role to ping when a new round starts. I am by no means a pro C# writer, and I did my best to make sure this follows what the rest of the code looks like, but please let me know if I made any formatting mistakes or whatever, I'm still trying to learn.
I wasn't fully sure how to properly provided credit, but I have taken a majority of the code from a Russian fork
Media

[x] I have added screenshots/videos to this PR showcasing its changes ingame, or this PR does not require an ingame showcase

I just remembered the issue with 24+ hour rounds from a while ago, and it's likely my implementation is affected by this too.


Add reference links back again?

Not sure if they're really needed still.

Closing this for now until I feel that they're helpful again.


Reported Bug

Issue reported by: @gov.ab.ca
Web component version:
Angular/React version:
Expected Behaviour
Actual Behaviour
Replication Steps

Closing as not an issue


more misc fixes

WIP - collection point for multiple flixes

fix responsive design for table page layouts
add view tomogram button
fix runs table sizings
add text overflow to runs table names
make "Browse Data" in the navbar highlight on hover

force merging prior to bug bash


Enable "Re-enroll" edge router feature in ZAC

The Ziti API support the ability to re-enroll an edge router and create a new enrollment JWT. This should also be made available via the console.

Yay! This is super useful for in-place recovery of a router that has an expired certificate. It's superior to replacing the router with a new one because policies that grant the replaced router will break even if the replacement has the same name. This way, with re-enroll, the ziti ID is the same, so policies keep working!


Hydrogen install throws deprecation error.

Prerequisites

[x] Have you checked the solutions in our troubleshooting guide for common problems ?
[x] Have you checked that your issue isn't already filed ?

Description
I clicked the install button in the Install Packages Settings pane.
Steps to Reproduce

Click install

Versions
OS version:
Windows 10

atom --version

Atom    : 1.38.2
Electron: 2.0.18
Chrome  : 61.0.3163.100
Node    : 8.9.3

apm --version

npm  6.2.0
node 8.9.3 x64
atom 1.38.2
python
git 2.21.0.windows.1
visual studio

Hydrogen version:

2.10.3
Plugins
Have you installed and activated any of the Hydrogen plugins below ?

[ ] hydrogen-python
[ ] Hydrogen Launcher
[ ] Data Explorer

Logs
><EMAIL_ADDRESS>install C:\Users\robin\AppData\Local\Temp\apm-install-dir-119612-14548-89rboi.tqmg6\node_modules\Hydrogen\node_modules\zeromq
> node scripts/prebuild-install.js || (node scripts/preinstall.js && node-gyp rebuild)

prebuild-install WARN install getaddrinfo ENOTFOUND github-production-release-asset-2e65be.s3.amazonaws.com github-production-release-asset-2e65be.s3.amazonaws.com:443

Downloading libzmq for Windows

npm WARN deprecated<EMAIL_ADDRESS>`json-parser` is deprecated. Please use `comment-json` instead
events.js:183
      throw er; // Unhandled 'error' event
      ^

Error: getaddrinfo ENOTFOUND github-production-release-asset-2e65be.s3.amazonaws.com github-production-release-asset-2e65be.s3.amazonaws.com:443
    at errnoException (dns.js:50:10)
    at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:92:26)
npm WARN enoent ENOENT: no such file or directory, open 'C:\Users\robin\AppData\Local\Temp\apm-install-dir-119612-14548-89rboi.tqmg6\package.json'
npm WARN<EMAIL_ADDRESS>requires a peer of<EMAIL_ADDRESS>|| ^15.0.0 but none is installed. You must install peer dependencies yourself.
npm WARN<EMAIL_ADDRESS>requires a peer of vega@* but none is installed. You must install peer dependencies yourself.
npm WARN<EMAIL_ADDRESS>requires a peer of<EMAIL_ADDRESS>|| ^4.0.0 but none is installed. You must install peer dependencies yourself.
npm WARN apm-install-dir-119612-14548-89rboi.tqmg6 No description
npm WARN apm-install-dir-119612-14548-89rboi.tqmg6 No repository field.
npm WARN apm-install-dir-119612-14548-89rboi.tqmg6 No README data
npm WARN apm-install-dir-119612-14548-89rboi.tqmg6 No license field.

npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR<EMAIL_ADDRESS>install: `node scripts/prebuild-install.js || (node scripts/preinstall.js && node-gyp rebuild)`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR!      C:\Users\robin\.atom\.apm\_logs\2019-07-12T09_38_46_504Z-debug.log

1,132,909
Hydrogen2.10.3Run code interactively, inspect data, and plot. All the power of Jupyter kernels, inside your favorite text editor.
nteract
Install
26,302
hydrogen-python0.0.8Python language-specific extensions for hydrogen
nikitakit
Install
1,209
hydrogen-auth0.0.2A brute-force approach to remote gateway authentication
nikitakit
Install
3,944
hydrogen-xdbg0.0.4Improved live coding for Python 3
nikitakit
Install
33,434
hydrogen-launcher1.2.2Launch terminals and jupyter consoles connected to hydrogen
lgeiger
Install
122
hydrogen-cell-separator0.4.1Atom package to improve identification of cells by separating them with a horizontal line.
jhabriel
Install
2,013
greyatom-hydrogen1.9.0Run code and get results inline using Jupyter kernels like IPython, IJulia, and iTorch
greyatom-edu-tech
Install
79
cell-navigation0.1.0Navigate, move and select code cells defined by `# %%`. Enhance your coding experience with Hydrogen and Jupytext.
hoishing
Install
2,607
nitrogen0.1.12Nitrogen: Nitrous meets Atom
nitrous-io
Install
7,062
data-explorer0.7.0Automatic data visualization with the nteract data-explorer
BenRussert
Install
313
cert-tweaks0.1.0Add extra certificates to Atom's trust store
nikitakit
Install
43
language-nitrogen0.2.0Nitrogen language support in Atom
nitrogen-lang
Install
68
language-nitrogen-vm0.1.4Nitrogen VM language support for Atom
jonahisadev
Install
1,716
atom-hydra0.2.10Live coding hydra from within atom
ojack
Install
64
doge-img0.4.0Simple way to add doge to your website
DemSquirrel
Install
3,775
doge0.1.3such atom. wow.
tylerl0706
Install
894
ipynb0.2.0Reserved package
rgbkrk
Install
1,093
drop-up0.5.0Drag and drop image files to editor, upload and insert link, GitHub style.
sikhapol
Install
4,141
drag-drop-text0.1.4Drag and drop text (and cut/copy/paste) with only the mouse.
mark-hahn
Install
12
ProgDT1.0.1Gets the Programmer's Date &&|| Time!
DocCodes
Install
75
gammboge4.2.0JavaScript autocompletion and suggestion
Gabriel-Mtz
Install
420
js-test-gen-atom0.2.0Js-test-gen-atom generates test templates from your default/named exports.
js-test-gen
Install
92
droiuby0.0.2An atom package for the Droiuby framework
jedld
Install
54
cursor-drop0.2.2Create multiple cursors via the keyboard
ahuth
Install
8,173
android0.1.0Android development package for Atom
dalinaum
Install
66
eyedrops0.1.2Blurs editor tabs to protect the guilty
GEOFBOT
Install
5,951
language-weave0.6.4Support for Weave.jl and Pweave
JunoLab
Install
157
abjadcompile0.1.18Render algorithmically generated music scores in Atom.
jefftrevino
Install
255
atom-together0.3.1Work together via IP
MonochromeSun
Install
833
dsl-progress0.28.0A package for develop with 4GL Progress
akaytatsu
Install
Featured Packages
1,132,909
Hydrogen2.10.3Run code interactively, inspect data, and plot. All the power of Jupyter kernels, inside your favorite text editor.
nteract
Install
614,629
atom-clock0.1.17Display a customizable clock in the status bar.
b3by
Install
97,435
hey-pane1.1.2Atom Plugin to enlarge the active pane, either with a shortcut or with a super handy follow mode. Inspired by Origami for Sublime Text.
timomeh
Install
57,852
scroll-through-time0.3.1Two fingers scroll moves through time instead of space
mathieudutour
Install
803,712
teletype0.13.4Share your workspace with team members and collaborate on code in real time
atom
Install
76,323
zentabs0.8.9Keep the opened tabs amount below specified limit
ArnaudRinquin
Install
1,065,660
atom-material-syntax1.0.8A dark syntax theme for Atom that uses Google's Material Design color palette
silvestreh
Install
1,700,683
atom-material-ui2.1.3A dynamic UI theme for Atom that follows Google's Material Design Guidelines
atom-material
Install
Git
GitHub
Unstaged Changes
Stage All
Staged Changes
Unstage All
See All Staged Changes

Commit message

Commit to 3.7-rework
72
<EMAIL_ADDRESS>avatar'
exclude .pyc files from the repository
Undo
41m
<EMAIL_ADDRESS>avatar'
structure file and NEST
4d
<EMAIL_ADDRESS>avatar'
first upload - first example files
4d
<EMAIL_ADDRESS>avatar'
Initial commit
4d
00Settings
3.7-reworkPublishGitHubGit (0)5 updates

Expected behavior
The installation to succeed

Error: getaddrinfo ENOTFOUND github-production-release-asset-2e65be.s3.amazonaws.com github-production-release-asset-2e65be.s3.amazonaws.com:443     at errnoException (dns.js:50:10)
Wild guess, but are you on a corporate network or someplace you might be blocked by a firewall? Do other atom packages install properly?

On a university network yes! That might be possible, we also couldn't
download Git, another amazonaws server that we couldn't reach. I'll let you
know once I can get access from another network.
Thanks
On Fri, 12 Jul 2019, 19:55 Ben Russert<EMAIL_ADDRESS>wrote:

Error: getaddrinfo ENOTFOUND
github-production-release-asset-2e65be.s3.amazonaws.com
github-production-release-asset-2e65be.s3.amazonaws.com:443 at
errnoException (dns.js:50:10)
Wild guess, but are you on a corporate network or someplace you might be
blocked by a firewall? Do other atom packages install properly?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/nteract/hydrogen/issues/1708?email_source=notifications&email_token=AG4VQS6ZRS2VZQZJNSYYGGLP7DASZA5CNFSM4ICGALG2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZ2OGRQ#issuecomment-510976838,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AG4VQS2BF655VV2YYHXGGTLP7DASZANCNFSM4ICGALGQ
.

That was the issue, thanks.

Having the same issue, weirdly on ISP network that appears to be using Google DNS
https://issuetracker.google.com/issues/144182685


Web_version/jscolor.js license

The file states "@license For open source use: GPLv3", but this package is licensed under Expat.

Thank you for the issue. the jscolors file is an external resource which i found online. Actually I don't use it anymore reverting back to the browser stock. I'll soon remove the file from the project.


Support for WLED 0.13.1?

Has this been updated to WLED 0.13.1?
I know the README says this project won't be updated but I saw a few commits so I was wondering.
I like having the flexibility of WLED animations (and for diyHue) but also use HyperHDR with AWA protocol.

Hi
You have v7 beta available (based on WLED 0.13.1) in the releases section:
https://github.com/awawa-dev/HyperSerialWLED/releases
Please follow the instruction and set desired serial communication speed in WLED panel.
But WLED WLED 0.13.1 has some serious bugs like:
https://github.com/Aircoookie/WLED/issues/2698
https://github.com/Aircoookie/WLED/issues/2618
Don't know if it affects HyperSerialWLED because I have rewritten whole part of the WLED serial communication and at least it works for me. You can also test it and let me know about the result.

WLED 0.13.3 s released but i dont knowing if its somthing new or fixed things that can being good for our systems.
https://github.com/Aircoookie/WLED/releases/tag/v0.13.3

Will there ever be an update to wled? I am using a remote hyperhdr server with a wemos d1 mini with standard wled. I realize that my wifi configuration is not the best and not as fast as the usb solution for ambilight, but this solution has cut my costs many times over without buying a separate RPI and set-top box since I have android tv built into my TV and since it is still being updated, I'm not ready to buy a separate set-top box for it.

If someone is interested it created a pull request: https://github.com/awawa-dev/HyperSerialWLED/pull/8
Please make sure to apply the fix https://github.com/Aircoookie/WLED/pull/4142/commits/2264cc5d10aae1fff21bf34782a09ff5291c9c3d before compiling

@fl0eb did you got wled 14.4 working? if so can you explain the process of how to compile it or guide me in the right direction. I would love to try it. Thank You

Yeah, works fine for me._ I got tired of HomeAssistant complaining about the old version not being compatible.
The only real "effort" was to merge the code from 0.14.4 together with the HyperSerialWLED code.
For compilation I just used VSCode together with the PlatformIO Extension nothing fancy.
There should be plenty of guides available on how to use it, but you generally just have to execute the "Build: HTML and binary" task.

HyperSerialWLED_0.14.4_fl0eb_BETA.zip

Thanks @fl0eb it works great on one D1 Mini !!!
Hope  some can  do one V20 dicker image for HyperHDR (was making on PR for updating it but is not  being merged after many mounts).

@MattWestb I create a Hyper-HDR docker image you can try it if you want to https://hub.docker.com/r/nocturno/hyper-hdr

@fl0eb Thank You. I will give it a try

@fl0eb Hi. I don't quite understand what you did? Just combined the wled 14.4 code and hyperserialwled? And what are the advantages of such integration? It would make a lot of sense to integrate the awa protocol into wled. And so my standard wled 14.4 works quite well with the backlight for the TV via wifi.

HyperSerialWLED is a fork of WLED which replaces the native serial communication with a custom „awa“-protocol.
All I did, was merging the latest changes of WLED into awawa-devs modified WLED code.
So yes, it is WLED with awa protocol.
To make the awa protocol natively supported by WLED you would need to make some adjustments and create a pull request for aircookies repository. However I don’t know if HyperHDR and HyperionNG is enough of a reason to include another protocol natively.
HyperSerialESP32 is a more stable solution with less overhead. Using WLED together with HyperSerial is not recommended and not actively maintained.
The only reason I use this, is because I like to use the ambient light even if the TV is off and home assistant requires the new version.

There is new release based on 0.14.0: https://github.com/awawa-dev/HyperSerialWLED/releases/tag/h0.14.0
Hopefully it will be enough for next few years.
From my short testing 0.14.4 is not stable on esp32 (dev) https://github.com/awawa-dev/HyperSerialWLED/pull/10
Beside versions above 0.14.0 could have problems with esp8266: https://github.com/Aircoookie/WLED/issues/3685


Use foreach for iterating over NodeList

Closes https://github.com/webonyx/graphql-php/pull/251
Resolves https://github.com/webonyx/graphql-php/issues/252
Ran into this again, as i am utilizing the NodeList for performance gains in Lighthouse.
We embed the AST into lazy type-loading, so we store the AST in an associative array, keyed by type names. The NodeList class serves this purpose well, but does not handle non-numeric keys right now.

Thanks!

@vladar thanks for the merge. Any chance you can cut a Bugfix release now?

Yeah, will publish 0.13.2 today

FYI, just published


[Feature][Zeta] Add Master and Worker split mode deployment

Purpose of this pull request

Zeta Master is separated from the Worker
Does this PR introduce any user-facing change?

Yes, Since this PR, the user can start zeta cluster node with -r <role>, the role can be master_and_worker,master,worker. More information can get from the document in the PR.
How was this patch tested?

Check list

[ ] If any new Jar binary package adding in your PR, please add License Notice according
New License Guide
[ ] If necessary, please update the documentation to describe the new feature. https://github.com/apache/seatunnel/tree/dev/docs
[ ] If you are contributing the connector code, please check that the following files are updated:

Update change log that in connector document. For more details you can refer to connector-v2
Update plugin-mapping.properties and add new connector information in it
Update the pom file of seatunnel-dist

[ ] Update the release-note.

@Hisoka-X  @dailai  @hailin0  The KubernetesIT have some error, but it seems is not cased by this pull request.

@Hisoka-X @dailai @hailin0 The KubernetesIT have some error, but it seems is not cased by this pull request.

how about disabled ClusterFaultToleranceIT::testStreamJobRestoreInAllNodeDown. Please refer https://github.com/apache/seatunnel/pull/7101


write some docs

@kenneth-reitz How can I proceed with this Issue?

Closing this since a duplicate of #4


Fix issues in TorpedoStore.java

This PR fixes the 3 issues identified by SonarLint in TorpedoStore.java

amzing brother


Consider less agressively validating unknown paramters

See http://stackoverflow.com/questions/29503984/ajax-call-error-status-of-400-bad-request
Consider dropping unknown parameters rather than returning 400s to make using the API easier.

Wont fix this one most likely. Feel free to suggest reopen if you feel strongly.


Figure out date usage

@MObrien-IDM or @mzimmermann-IDM can you clarify how the lists of dates are used? These ones:
    # Dates
    State('dobs',               [], list),  # Dates of birth of children
    State('still_dates',        [], list),  # Dates of stillbirths -- list of lists
    State('miscarriage_dates',  [], list),  # Dates of miscarriages -- list of lists
    State('abortion_dates',     [], list),  # Dates of abortions -- list of lists
    State('short_interval_dates',[], list),  # age of agents at short birth interval -- list of lists

These end up being a pretty long list, I'm wondering whether we might want to refactor these into a dataframe or other format that's easier to work with. Alternatively, if these aren't used often, we could either remove them altogether or place them in an analyzer.

Dobs are used for calculating age-specific fertility, which is important. Dates of pregnancy events are used for looking at life courses. Short-interval dates are also used for the age-specific tracking to look at when agents have short intervals (and % of those agents who are adolescents with short intervals). Could potentially toggle these off when not using the age-specific tracking (track_as)


Improve the readme page!

The readme page is very basic. You can improve the grammar, text,  markdown styling or something else to make it spark!

If you are new, try contributing to this to get an idea about github basics

May I please work on this issue?

I'm interested in this?

Can i work on this?

Can i work on this?

Yes please.


How to Nikon and Zeiss reader updates

Add a data folder to store the data in (same as the TIFF reader branch)
Update the Nikon and Zeiss notebooks to use the updated data path
Don't save the rendered output of the notebook

Looks good! I wonder if it is worth having an optional cell to remove the data once the notebook has run - the walnut especially is quite large.

Thanks Margaret, I've added a cell showing how to delete the folders


Backend::get_host() call generates Buffer length error

When calling Backend::get_host() in the local test, the following error message is generated on the console:
(CLI version: v9.0.3, Viceroy version: 0.4.5)
2023-04-30T07:14:24.548437Z ERROR request{id=0}: Hostcall yielded an error: Buffer length error: host too long to fit in host.len()

Despite the error message, the hostname is still correctly retrieved, and there seem to be no functional issues.
The following code snippet can be used to reproduce the error:
# main.rs

use fastly::http::StatusCode;
use fastly::{Error, Request, Response, Backend};

#[fastly::main]
fn main(_req: Request) -> Result<Response, Error> {
    let backend = Backend::from_name("httpbin").unwrap();
    println!("{}", backend.get_host());
    Ok(Response::from_status(StatusCode::OK))
}

# fastly.toml

# fastly.toml
[local_server]
  [local_server.backends]
    [local_server.backends.httpbin]
      url = "https://httpbin.org/"

Are you still seeing this issue? I cannot reproduce it with Viceroy 0.9.1.

This message was to be expected -- there is a negotiation happening between the SDK and the runtime to figure out how big the buffer needs to be in order to hold the hostname.  Basically the SDK is trying a too-small buffer with the hostcall once, getting that error back with the expected length, resizing the buffer to fit, and then calling the hostcall again successfully.
The message was misleading, and we changed the logging level of these messages so they now only display at DEBUG level.  That was done in https://github.com/fastly/Viceroy/pull/314, which was included in v0.9.0.


fix: modify tip for delete backup space

Signed-off-by: liuying<EMAIL_ADDRESS>
What type of PR is this?
/kind bug

What this PR does / why we need it:
Which issue(s) this PR fixes:

Fixes #
Special notes for reviewers:

Does this PR introduced a user-facing change?

Additional documentation, usage docs, etc.:

Codecov Report

Merging #327 (aebb638) into release-1.3 (a0be023) will not change coverage.
The diff coverage is n/a.

@@             Coverage Diff              @@
##           release-1.3     #327   +/-   ##
============================================
  Coverage        47.35%   47.35%
============================================
  Files                7        7
  Lines              606      606
  Branches           171      171
============================================
  Hits               287      287
  Misses             246      246
  Partials            73       73

/lgtm
/approve


Display parameters one of multiple functions when hover

Describe the bug
I have three function definitions:
node = {}

---This behaves like math.random except that it uses true random numbers derived from the ESP8266 hardware. It returns uniformly distributed numbers in the required range.
---@param l integer the lower bound of the range
---@param u integer the upper bound of the range
---@return integer --a pseudo-random integer x such that l <= x <= u.
function node.random(l, u) end

---This behaves like math.random except that it uses true random numbers derived from the ESP8266 hardware. It returns uniformly distributed numbers in the required range.
---@param n? integer the number of distinct integer values that can be returned -- in the (inclusive) range 1 .. n
---@return integer --an integer random number x such that 1 <= x <= n.
function node.random(n) end

---This behaves like math.random except that it uses true random numbers derived from the ESP8266 hardware. It returns uniformly distributed numbers in the required range.
---@return number --a random real number with uniform distribution in the interval [0,1)
function node.random() end

In input mode, it is suggested to choose one function of three, as expected.
But when hover, shows only last defined function.

I tryed alternative method using @overload, but it doesn't works also.
---This behaves like math.random except that it uses true random numbers derived from the ESP8266 hardware. It returns uniformly distributed numbers in the required range.
---@param l integer the lower bound of the range
---@param u integer the upper bound of the range
---@return integer --a pseudo-random integer x such that l <= x <= u.
---@overload fun(l:integer, u:integer):integer --a pseudo-random integer x such that l <= x <= u.
---@param n? integer the number of distinct integer values that can be returned -- in the (inclusive) range 1 .. n
---@return integer --an integer random number x such that 1 <= x <= n.
---@overload fun(n?:integer):integer --an integer random number x such that 1 <= x <= n.
---@return number --a random real number with uniform distribution in the interval [0,1)
function node.random() end

Is it possible to display parameters of multiple functions when hover?
Environment :

OS: Windows
Client: VSCode

I tried some, but it didn't work very well. The main reason is that the three functions will produce three labels and descriptions, but the hover box will put the three labels together and then put the three descriptions together.

it seems to me the solution you suggested would work very well for displaying such a function:
---Unsubscribes from one or several topics.
---@param tbl table array of `topic, anything` pairs to unsubscribe from
---@overload fun(tbl: table, foo: function): boolean
---@param topic string a topic string
---@param foo? function|' function(client) end' optional callback fired when unsubscription(s) succeeded.
---@return boolean
function m:unsubscribe(topic, foo) end

-- instead of defining two functions=>

---Unsubscribes from one or several topics.
---@param topic string a topic string
---@param foo? function|' function(client) end' optional callback fired when unsubscription(s) succeeded.
---@return boolean
function MQTT:unsubscribe(topic, foo) end

---Unsubscribes from one or several topics.
---@param tbl table array of `topic, anything` pairs to unsubscribe from
---@param foo? function|' function(client) end' optional callback fired when unsubscription(s) succeeded.
---@return boolean
function MQTT:unsubscribe(tbl, foo) end

because only the type of one parameter changes.
However, the three node.random functions (param1, param2 | no_param) could be shown separately (not @overload).
(but, if I understood you correctly - three functions node.random with different parameters has the same labels, and hover box can't let choose one of them.)
IMO, anyway it is better to display all parameters than only one function parameters.

Now, on hover, the description of another function is shown and vice versa.

sdmmc = {}

---@class sdmmc
local card = {}

---@class SDmmcCfg
---@field cd_pin integer
---@field wp_pin integer
---@field fmax integer
---@field width  integer

---SDMMC Mode. Initialize the SDMMC and probe the attached SD card.
---@param slot integer|'sdmmc.HS1'|'sdmmc.HS2' SDMMC slot
---@param cfg? SDmmcCfg optional table containing slot configuration:
-- - **cd_pin** card detect pin, none if omitted
-- - **wp_pin** write-protcet pin, none if omitted
-- - **fmax** maximum communication frequency, defaults to 20  if omitted
-- - **width** bis width, defaults to sdmmc.W1BIT if omitted, one of:
--   - sdmmc.W1BIT
--   - sdmmc.W4BIT
--   - sdmmc.W8BIT, not supported yet
---@return sdmmc cardObj Card object. Error is thrown for invalid parameters or if SDMMC hardware or card cannot be initialized.
function sdmmc.init(slot, cfg) end

---@class SDspiCfg
---@field sck_pin integer
---@field mosi_pin integer
---@field miso_pin integer
---@field cs_pin integer
---@field cd_pin integer
---@field wp_pin integer
---@field fmax integer

---SD SPI Mode. Initialize the SDMMC and probe the attached SD card.
---@param slot integer|'sdmmc.HSPI'|'sdmmc.VSPI' SD SPI slot
---@param cfg SDspiCfg mandatory table containing slot configuration:
-- - **sck_pin** SPI SCK pin, mandatory
-- - **mosi_pin**, SPI MOSI pin, mandatory
-- - **miso_pin**, SPI MISO pin, mandatory
-- - **cs_pin**, SPI CS pin, mandatory
-- - **cd_pin** card detect pin, none if omitted
-- - **wp_pin** write-protcet pin, none if omitted
-- - **fmax** maximum communication frequency, defaults to 20  if omitted
---@return sdmmc cardObj Card object. Error is thrown for invalid parameters or if SDMMC hardware or card cannot be initialized.
function sdmmc.init(slot, cfg) end

Thank you for reporting, please open a new issue, so that I can record and fix the bug separately.


Header when editing an exercise is inconsistent

Summary
When creating exercises in the exercise editor, there are two issues:
(1)  The first time you create an exercise, the header says: "Adding Exercise to [channel name]". If you save this exercise and create new exercises and go back to the original exercise, the header changes to "Editing Content." I think this should be consistent.
(2) I think "Editing Exercise" may be clearer than "Editing Content" as a header.
Category
ENHANCEMENT
Usage Details
Please give details in how were you using the Content Curation Server.

Browser: Chrome
Channel: 7ad7f9acd1a146ba946948c0e5a5f59b
Topic/Content Item: Top level of the topic tree

How to reproduce

Create a new exercise. Save exercise.
Create a second exercise. Save exercise.
Go back to the original exercise and observe.

Screenshots
Without any exercises in the channel:

With existing exercises in the channel:

Real-life consequences (anything community should be aware of, for instance how it affects end users)
None - just slightly confusing and could use clarity.

@laurenlichtman That sounds like a good UX improvement. I feel like this inconsistency might be in other places in the app, as there are many kinds of content. If you see any more, could you log them here and we can tackle them all at once?

I could see how that could work, but we do have distinct titles for adding topics (it says "  Adding Topics to [Folder name]") so I think consistency across that is detailed versus inclusive may be a bit easier for the user. Copying @lauradanforth @navakk9 for a pedagogical perspective.


update @stacks/* deps

many of the @stacks/* libraries recently got updated and there are some breaking changes. We should update our dependencies sooner rather than later to support folks who are using the new versions.
I do worry that when we upgrade, any app that is using an older version of @stacks/transactions will need to update before their users can use the extension. @zone117x do you think that is the case?

Is this related to https://github.com/blockstack/connect/issues/151?


How can two hands be recognized and posed

For example, I have one previous picture.

It is recognized as:

When it cut with two separated hands, OK:

It looks like that the numbers of the joints is set as 21, one hand will have almost 20 points. If multiple hands, e.g., 2 hands have, how can I recognize it  successfully？ Will the setting of joint-number work?

Can any update be given？

Hi,
The model should take single hand image as input.
You can use a hand detector (ex. YOLO) to detect each hand, crop hands and feed into model one by one.

This model can only predict right hand, however, I've try this model with horizontally flipped image. The result was a surprise, it actually can predict left hand too, but you need to flip the predicted result.
What you need to do are:

feed original image
feed flipped image
flip it back as left hand prediction

Hi @momo1986
I have the exact same use case, finding fingers on a piano.
I was wondering, can you share your code / can we collaborate on this? My email address is<EMAIL_ADDRESS>


MAINT: forward port 1.3.0 final release notes

forward-port changes to SciPy 1.3.0
release notes after "final" release off the
maintenance branch

merged, thanks Tyler


Annotate management operations for BGV/CKKS

I was trying to insert relinearize/modreduce operations using analysis for optimization purpose, and find that after the current secret-to-<scheme> lowering it is hard to manipulate the IR (especially we are migrating lwe types; also check #860). Instead we can place these management ops in secret dialect.
For example, the current lazy relinearization pass is implemented in bgv dialect, it can be also applied to ckks. Also, that pass needs to worry about LWE types, which should be handled by lowering passes.
I wrote a demo using the current depth analysis in my fork
module {
  func.func @func(%arg0: !secret.secret<i16>, ...) {
    %0 = secret.generic ins(...
    ^bb0(%arg8: i16, ...):
      %1 = arith.muli %arg8, %arg9 {depth = 1 : i64, lhs = 0 : i64, relin = 3 : i64, rhs = 0 : i64} : i16
      %2 = arith.muli %1, %arg10 {depth = 2 : i64, lhs = 1 : i64, relin = 3 : i64, rhs = 0 : i64} : i16
      %3 = arith.muli %2, %arg11 {depth = 3 : i64, lhs = 2 : i64, relin = 3 : i64, rhs = 0 : i64} : i16
      %4 = arith.muli %3, %arg12 {depth = 4 : i64, lhs = 3 : i64, relin = 3 : i64, rhs = 0 : i64} : i16
      %5 = arith.muli %4, %arg13 {depth = 5 : i64, lhs = 4 : i64, relin = 3 : i64, rhs = 0 : i64} : i16
      %6 = arith.muli %5, %arg14 {depth = 6 : i64, lhs = 5 : i64, relin = 3 : i64, rhs = 0 : i64} : i16
      %7 = arith.muli %6, %arg15 {depth = 7 : i64, lhs = 6 : i64, relin = 3 : i64, rhs = 0 : i64} : i16
      secret.yield %7 : i16
    } -> !secret.secret<i16>
    return %0 : !secret.secret<i16>
  }
}

If such direction is favored, the following stpes can be done
Relinearize part

Modify SecretGenericOpMulConversion to emit RelinearizeOp as needed.
Modify OptimizeRelinearization to work on secret dialect, so that it does not need to worry about LWE types.

Modswitch/Rescale part

For modreduce, emit corresponding ModReduceOp according to operand/result depth annotation (namely the simple auto mod reduce technique), so that we do not need to rely on openfhe for level management (openfhe backend now defaults to the FLEXIBLEAUTOEXT scaling technique).
Further optimization can be implemented upon this. And they dont need to worry about LWE types.
Note that there are also other management ops like upscale for CKKS and various CKKS compiler has explored scale management. Such infra could be used to implement them.

Could you say more about what you were trying to do and the difficulties you ran into?
I'm not sure I would agree that the ciphertext management ops should be in the secret dialect, primarily because that leaks abstraction across dialect boundaries (secret is scheme-agnostic, but relinearization is scheme-specific), and second because I expect the relinearization optimization to later incorporate more scheme-specific concepts like being aware of the noise impact of inserting relin ops.
My main idea to generalize the relin optimization pass is to add traits and interfaces to the relevant scheme ops, and modify the pass to operate on ops via their interfaces. This would also allow different scheme dialects to return extra information from their interface implementations that correspond to stuff like noise modeling differences between schemes.

being aware of the noise impact of inserting relin ops.

I'm actually working on this, using average-case noise analysis (based on your old PR) to select a viable (noise allowed) and possibly minimal (implemented a cost model) param set and operation graph. I will eventually upstream it, starting from common parts like cost model.
Here is some initial result of it (finally get it end to end), you may check it [here] (https://github.com/ZenithalHourlyRate/heir/compare/ddfddb4f50131b5b25e080a65e187b00c2dc6611...ZenithalHourlyRate:eval-noise-v1) with main/run.sh
Analysis selected param and noise estimation
INFO: Running command line: bazel-bin/tools/heir-opt '--mlir-to-secret-arith=entry-function=func' --operation-balancer --annotate-secret-management --validate-noise '--debug-only=ValidateNoise' '--mlir-to-openfhe-bgv=entry-function=func ciphertext-degree=8' main/func.mlir
Selected Param: Param: n = 8192, L = 2, relinDeg = 2, qi = [55, 55, 55, ], digitSize = 30, dnum = 0, alpha = 0, pi = []
<block argument> of type 'i16' at index: 0: enc bound 26.79
<block argument> of type 'i16' at index: 1: enc bound 26.79
<block argument> of type 'i16' at index: 2: enc bound 26.79
<block argument> of type 'i16' at index: 3: enc bound 26.79
<block argument> of type 'i16' at index: 4: enc bound 26.79
<block argument> of type 'i16' at index: 5: enc bound 26.79
%1 = arith.muli %arg8, %arg9 : i16: mult bound 57.67
%1 = arith.muli %arg8, %arg9 : i16: relin bound 57.75
%2 = arith.muli %1, %arg10 : i16: mult bound 88.64
%2 = arith.muli %1, %arg10 : i16: modd bound 33.64
%2 = arith.muli %1, %arg10 : i16: relin bound 55.79
%3 = arith.muli %arg11, %arg12 : i16: mult bound 57.67
%3 = arith.muli %arg11, %arg12 : i16: relin bound 57.75
%4 = arith.muli %3, %arg13 : i16: mult bound 88.64
%4 = arith.muli %3, %arg13 : i16: relin bound 88.64
%4 = arith.muli %3, %arg13 : i16: modd bound 33.64
%5 = arith.muli %2, %4 : i16: mult bound 93.52
%5 = arith.muli %2, %4 : i16: relin bound 93.52
%5 = arith.muli %2, %4 : i16: modd bound 38.52

OpenFHE lowering and running result, with actual noise.
Element Parameters: ILDCRTParams [m=16384 n=8192 q=46767924130724242010584182085469215285039268954113 ru=0 bigq=0 bigru=0]
  m_params:
    0: ILParams [m=16384 n=8192 q=36028793789300737 ru=1134974017364 bigq=0 bigru=0]
    1: ILParams [m=16384 n=8192 q=36028786272993281 ru=603296523949 bigq=0 bigru=0]
    2: ILParams [m=16384 n=8192 q=36028712183676929 ru=8581620453067 bigq=0 bigru=0]
  m_originalModulus: 0

Encoding Parameters: [p=65537 rootP =0 bigP =0 rootBigP =0 g=0 L=8192]

v28=Encrypt v24, v27    cv 2 Ql 3 logQ: 165 logqi: [ 55 55 55 ] noise: 25.7879 budget 138.212
v35=Encrypt v31, v34    cv 2 Ql 3 logQ: 165 logqi: [ 55 55 55 ] noise: 26.007 budget 137.993
v42=Encrypt v38, v41    cv 2 Ql 3 logQ: 165 logqi: [ 55 55 55 ] noise: 25.9658 budget 138.034
v49=Encrypt v45, v48    cv 2 Ql 3 logQ: 165 logqi: [ 55 55 55 ] noise: 25.9233 budget 138.077
v56=Encrypt v52, v55    cv 2 Ql 3 logQ: 165 logqi: [ 55 55 55 ] noise: 25.8424 budget 138.158
v63=Encrypt v59, v62    cv 2 Ql 3 logQ: 165 logqi: [ 55 55 55 ] noise: 25.9069 budget 138.093
v70=Encrypt v66, v69    cv 2 Ql 3 logQ: 165 logqi: [ 55 55 55 ] noise: 25.7699 budget 138.23
v77=Encrypt v73, v76    cv 2 Ql 3 logQ: 165 logqi: [ 55 55 55 ] noise: 25.9644 budget 138.036
v9=EvalMultNoRelin v1, v2       cv 3 Ql 3 logQ: 165 logqi: [ 55 55 55 ] noise: 56.5308 budget 107.469
v10=Relinearize v9      cv 2 Ql 3 logQ: 165 logqi: [ 55 55 55 ] noise: 57.056 budget 106.944
v11=EvalMultNoRelin v10, v3     cv 3 Ql 3 logQ: 165 logqi: [ 55 55 55 ] noise: 87.3554 budget 76.6446
v12=ModReduce v11       cv 3 Ql 2 logQ: 110 logqi: [ 55 55 ] noise: 32.4083 budget 76.5917
v13=Relinearize v12     cv 2 Ql 2 logQ: 110 logqi: [ 55 55 ] noise: 54.6418 budget 54.3582
v14=EvalMultNoRelin v4, v5      cv 3 Ql 3 logQ: 165 logqi: [ 55 55 55 ] noise: 56.6178 budget 107.382
v15=Relinearize v14     cv 2 Ql 3 logQ: 165 logqi: [ 55 55 55 ] noise: 56.7794 budget 107.221
v16=EvalMultNoRelin v15, v6     cv 3 Ql 3 logQ: 165 logqi: [ 55 55 55 ] noise: 87.4168 budget 76.5832
v17=Relinearize v16     cv 2 Ql 3 logQ: 165 logqi: [ 55 55 55 ] noise: 87.4168 budget 76.5832
v18=ModReduce v17       cv 2 Ql 2 logQ: 110 logqi: [ 55 55 ] noise: 32.4164 budget 76.5836
v19=EvalMultNoRelin v13, v18    cv 3 Ql 2 logQ: 110 logqi: [ 55 55 ] noise: 93.5568 budget 15.4432
v20=Relinearize v19     cv 2 Ql 2 logQ: 110 logqi: [ 55 55 ] noise: 93.5568 budget 15.4432
v21=ModReduce v20       cv 2 Ql 1 logQ: 55 logqi: [ 55 ] noise: 38.5568 budget 15.4432

Now here is the ugly part:
It was first implemented on bgv dialect, as the implemented analysis is bgv-specific. Then I found that it is hard to manipulate bgv IR, especially we are migrating LWE types. I think those type constraint (rlweparams.dimension, basis) should be cared by the lowering pass, and optimization pass should care less about them. Over-early over-specified type will add burden to optimization pass writers.
So I migrated to the title suggested approach: using attr on the operation and lowering pass will lower them accordingly. Unfortunately we do not have a bgv.bgv type (with such type we can have a clean computation graph without LWE types, and the pipeline can be --mlir-to-secret-arithmetic --secret-to-bgv-type --optimization-pass --bgv-type-to-lwe --bgv-lwe-to-openfhe) so I have to do it directly on secret dialect.
Result of --mlir-to-secret-arithmetic --operation-balancer --annotate-secret-management --validate-noise
module {
  func.func @func(%arg0: !secret.secret<i16>, ...) -> !secret.secret<i16> attributes {digitSize = 30 : i64, keySwitchTechnique = "BV", maxRelinSkDeg = 2 : i64, multiplicativeDepth = 2 : i64, numLargeDigits = 0 : i64, plaintextModulus = 65537 : i64, ringDim = 8192 : i64, scalingModSize = 55 : i64} {
    %0 = secret.generic ins(%arg0, ... : !secret.secret<i16>) attrs = {depth = 3 : i64} {
    ^bb0(%arg8: i16, ...):
      %1 = arith.muli %arg8, %arg9 {mgmt = ["relin"]} : i16
      %2 = arith.muli %1, %arg10 {mgmt = ["modd", "relin"]} : i16
      %3 = arith.muli %arg11, %arg12 {mgmt = ["relin"]} : i16
      %4 = arith.muli %3, %arg13 {mgmt = ["relin", "modd"]} : i16
      %5 = arith.muli %2, %4 {mgmt = ["relin", "modd"]} : i16
      secret.yield %5 : i16
    } -> !secret.secret<i16>
    return %0 : !secret.secret<i16>
  }
}

Now the hacky part:
The above are about lowerings. For param passing through all the way down to openfhe, now I have to tag those attr on FuncOp so --openfhe-configure-crypto-context can get the information. Do you have suggestion on how to share information between passes?

Do you have suggestion on how to share information between passes?

For cryptosystem parameter selection I think this will be a rather important topic, and I don't know exactly what we'll do long term, especially considering the possibility of scheme switching, in which we'll have multiple sets of parameters floating around. It may make sense at some point to have a region-holding op that is annotated with the parameters that apply to that region. But until we get to the point where we need it, I think a safe approach would be to annotate the moduleOp with attributes. Maybe bundle them in a sensible way under a single parent attribute (say, BGVParametersAttr) so that it's easier to move it around.
As for the rest, I think you should feel free to put attributes on ops in any dialect ad hoc, but you wouldn't be able to assume they persist across optimization passes and lowerings. They are preserved in a "best effort" manner, especially by upstream passes, and anything which rewrites ops will not be guaranteed to pass along attributes from the ops that are erased. The best approach is to organize your work such that the analysis that adds the attributes can be run within the same pass that actually does the work. If you need to keep op annotations around for a larger section of the pipeline, it may be hard to preserve them as the stages of the pipeline change.
I think there's a place for a high-level (maybe scheme agnostic) optimization pass that does ciphertext management, but I still think secret may not be the best long-term place for it.
Some other ideas that popped up in conversation:

We could add a new dialect that contains dialect-agnostic ciphertext management ops (e.g., a mgmt.relin op that has Any -> Any type semantics) and insert them at the secret.generic level before being lowered to an FHE-specific dialect.
We could create an intermediate dialect that represents the common abstractions over BGV, BFV, and CKKS (an "arithmetic FHE" dialect) that includes a ciphertext type that would fill the role of bgv.bgv as you mentioned earlier. This makes sense to me in that it could be the main place that some ciphertext parameters are selected, specifically those that are required to convert ciphertexts to LWE types.
We could add sufficient helpers and abstractions to make the LWE types easier to work with. If the migration were completed, would you still find difficulty?

We could add sufficient helpers and abstractions to make the LWE types easier to work with. If the migration were completed, would you still find difficulty?

I think there are chicken-and-egg problems here. LWE type already contains all the parameter information (e.g. plaintext modulus, modulus chain, depth, encoding/packing method). The current ToBGV(actually ToLWE) lowering just use some default values. We should use some passes to decide them prior to it, instead of placing some placeholders then change them. Also, passes like deciding encoding/packing could totally change the structure of the circuit, then other parameters like depth/modulus would then need to be recalculated inside the pass to ensure a valid LWE IR.
The natural logic should be "decide one param -> decide another -> generate/lower" instead of "generate/lowerWithDefault -> change param (bunch of other change) -> change circuit (then bunch of param change) -> change another param (then bunch of change)"
https://github.com/google/heir/blob/6d91164e43876b3b64cc952e367b188a859cfc59/lib/Dialect/Secret/Conversions/SecretToBGV/SecretToBGV.cpp#L47-L95

We could create an intermediate dialect that represents the common abstractions over BGV, BFV, and CKKS (an "arithmetic FHE" dialect) that includes a ciphertext type that would fill the role of bgv.bgv as you mentioned earlier. This makes sense to me in that it could be the main place that some ciphertext parameters are selected, specifically those that are required to convert ciphertexts to LWE types.

I agree with such reasoning.

Also, passes like deciding encoding/packing could totally change the structure of the circuit

I think we all agree this is the essential complexity here. I don't yet know what is the right order to choose all of these things. And I think there will be some decent amount of research one can do by changing the order of choices to enable a later choice to optimize differently based on assumptions of an earlier choice.
I think this bleeds into the way one writes an optimization pass for one or more of these choices (e.g., to allow ciphertext dimension to be chosen by the pass or assume it is fixed and known).
If one were forced to do this in LWE could essentially ignore the parts of the type they want to re-select when doing the analysis (effectively forming an in-memory "LWE" type with only the subset of fields they want to consider fixed), and then reconstruct all of the ops afterward with new LWE types matching the output of the pass. This would have the benefit of having all the different passes be more or less interchangeable in order.
If, by contrast, we add multiple intermediate dialects and "choose -> lower -> choose -> lower ..." then reordering the optimization passes becomes much harder, and we have the additional infrastructure of these extra dialects and lowerings to maintain. If we can manage to keep most of these kinds of passes at one granularity (either LWE or this hypothetical "abstracted arithmetic FHE dialect") then I think the system would remain flexible enough to change our minds later about ordering optimizations.
CC @asraa @AlexanderViand-Intel for any objections.

discuss in the next open meeting

I'll demostrate it in more details in the next meeting

the end goal of this work is to do parameter selection (at least a rough one at a high level).

Basically yes, but I think I am doing it in a more fine-grained manner. The current implemented method is, using average-case noise estimation (available paper gives underestimation though but I'm trying to fix them with different assumptions) to ensure for each ciphertext the noise wont blow up. And given such constraint, the extra bit the analysis has done is to try on different mgmt op placement (e.g. lazy relinearization/lazy modreduce), generating all possible schedules, and among all the schedules, select a minimal cost (with cost model) schedule, and this natually produces an "optimal" parameter for this circuit (minimize L, minimize Q, minimize N).
Also, the current searching algorithm is quite brutal; I will soon investigate how to use ILP for that.
I think in the future the optimization should also consider other schedule options like scale management (some paper, for CKKS), bootstrap op placement (e.g. DaCapo) and encoding/packing selection (lots of paper) and still many other considerations; Now I have not explored them yet, I think I'll do them in the next two or three steps.

CKKS expert colleague who said he had an idea about how we might do parameter selection

I am curious about it. My current working direction is to migrate my current design (from BGV) to CKKS, as their noise model are somewhat similar, but faced a issue that their optimization goal is different (i.e. BGV only needs to ensure noise wont blow up, but the in-my-mind-hypothetical goal for CKKS is to ensure more precision, or given a precision requirement, select a param set/circuit (things become even more complex as there are security concerns like noise flooding))

For CKKS, I just want to point out Grafting (https://eprint.iacr.org/2024/1014) which simplifies modulus management significantly because it decouples the ciphertext modulus itself from the scaling factor. This also works for BGV (disclaimer: I've not checked the math in detail so maybe I'm missing something).
Food for thought: The more I think about it, the more convinced I am that use-case specific parameters are only worth the effort in LHE mode. Once we need bootstrapping, most of the modulus is used for that anyway and there's not much left to optimize for.

Grafting (https://eprint.iacr.org/2024/1014)

Thank you! I'll look into this.

Food for thought: The more I think about it, the more convinced I am that use-case specific parameters are only worth the effort in LHE mode. Once we need bootstrapping, most of the modulus is used for that anyway and there's not much left to optimize for.

I kind of agree. I have some arguments to mitigate this but I have not exprimented it yet so I'm not sure.
One thing I observe is that in the experiment the "schedule" above can sometime produce a circuit with less MultiplicativeDepth, namely L'=L-1, the circuit produced is similar to "Three-way Multiplications" in https://eprint.iacr.org/2012/099). One key advantage we have here is that we have kind-of-accurate noise estimation so we can concisely control the noise growth. I deem too-often mod reduce (where modreduce added-noise becomes dominant) will waste some noise space, from which we can profit.
With this observation, it might be possible to optimize the circuit between bootstrapping, and as bootstrapping itself have placement problem (e.g. Dacapo), the in-between circuit can also be different subject to that placement and could be optimized, possibly reducing the total depth count. I have not exprimented so I do not know how much margin we can gain here. A even wilder try is to run the optimization on the bootstrap circuit itself. I'll definitely try it.


Remove the usage of BinaryFormatter

BinaryFormatter is deprecated
To use this library BinaryFormatter has to be explicitly reanabled by adding <EnableUnsafeBinaryFormatterSerialization>true</EnableUnsafeBinaryFormatterSerialization> to .csproj files, which is not recommended by Microsoft. From what I see, BinaryFormatter is used only to perform a deep copy, so if we can use something else, users of the library won't have to add that potentially dangerous setting.
I run the unit tests and all of them passed.

when will this be added to the latest version?

I've started moving it over to github workflows. Going to require some tinkering, and I'm a bit busy the next couple days. Hopefully soon I can have this package ready on nuget.


Change the way we deal with parameters and whitelists/blacklists of CodeValidator

We should make them public, but prohibit modification. Custom validator parameters and whitelists/blacklists should be passed as validate function arguments. The same goes for validator groups. Then we can abolish CodeValidator object altogether, putting defaults into their own module.
For example, if we wanted to modify a whitelist now, we would do the following:
code_validator = CodeValidator()
code_validator.whitelists['has_no_calls_with_constants'] = ['pow', 'exit']
code-validator.validate(repo)

What I'm offering to do is something like this:
from fiasko_bro import defaults, validate

my_whitelists = defaults.get_whitelists()  # returns a copy
my_whitelists['has_no_calls_with_constants'] = ['pow', 'exit']
validate(repo, whitelists=my_whitelists)

Current problems with the solution:

it's implicit
there can be no semantically different whitelists simultaneously (e.g. a whitelist of files and a whitelist of variable names)


Keyboard is not working, but I can paste

Platform:
KDE Plasma on Ubuntu (more below).
Steps:

Open or create a note.
Put the caret into the text box.
Start typing.
Copy a text from elsewhere, and paste it into the text box, using the mouse.

Expected:

On 3., the typed letters appear.
On 4., the text from copy/paste appears.

Actual:

(bad) On 3, typed letters do not appear in the note.
(bad) Also, other shortcuts like Ctrl-Z or backspace don't work.
(good) The text from copy/paste appears.
(good) I can move the text caret with the mouse.

More observations:

In the past it used to work. Something must have changed.
Other applications receive key strokes just fine.

Conclusions:

The note is not "write-only" (if that is even possible).
Key strokes don't make it into the note app.

 I routinely test on KDE and never seen anything like this.  Some questions please -

Your version of Ubuntu, '12' can you please check that, I doubt you mean 12.04, that  is twelve years old. I might have to build a replica.
In tomboy-ng, can you type into other places, eg, the search field from the Search Window ?
Which version of tomboy-ng are you using ?  One from Ubuntu repository or downloaded from here (.deb or .AppImage ?).

Davo

Hi.
With `lsb_release -a' I see "Ubuntu 24.04.1 LTS".
But my "About this system" dialog says Ubuntu 12.
So this dialog is really malfunctioning.
Here the output from tomboy-ng "About".
(Unfortunately I don't find how to copy this so I have to type.)

Version 0.39, QT5
Build date 2024/04/01 (Laz 3.0.0)
TargetCPU x86_64 OS Linux KDE
QT_QPA_PATFORMTHEME :
QT_QPA_PLATFORM : xcb

In synaptic package manager it says tomboy-ng 0.39-1build2
I suppose this means I did use that for installing it.

In tomboy-ng, can you type into other places, eg, the search field from the Search Window ?

Good that you ask!
I can use the keyboard in the main dialog window with the search box etc.
I can not use the keyboard in the window that shows an individual note.

OK, I am pulling down a KUbuntu 24.04-1 image as we speak. Pretty slow network here ...
Yeah, their "About this system" is broken !
tomboy-ng 0.39-1build2 sounds like an Ubuntu rebuild of a Debian package, nothing unusual about that.
And only not working in the note window ? That sounds like a KMemo issue to me and that could be quite messy. You are already using xcb so blaming Wayland is unlikely but not impossible.
So, in case I cannot replicate it here, some tests please ?
Could you please log out (not shutdown) and select your user name but before typing in your password, switch to X11 instead of Wayland. From memory, a clickable link lower left of the logon screen.  This is quite reversible, nothing permanent.
Then, password and test tomboy-ng again.  I have found some substantial differences in how KDE adapts to Wayland (compared to Gnome) so, its possible.
Davo

Yeah, I am afraid I cannot replicate that problem. And everything matches. Its even tells me its running Ubuntu 12 too !
But no sign of that problem.
Now Don, that means if we are to get to the bottom of this, it might need some help from your end. Not sure exactly what just yet ....
Davo

Alright, thanks for the quick feedback!
I can test with and without wayland, sure.
Or reinstall tomboy-ng. Or reinstall my system which seems a bit unreliable lately.
If nothing else, perhaps if others have the same experience they can post here.

Hmm, not a big fan of reinstalling, this is not Windows.  tomboy-ng is pretty deterministic, all its settings are visible to you, nothing happening below your gaze. I'd like to think your OS is too. Maybe not.
But do look in your settings.

Settings -> Input Methods : you get a list of characteristics, be good to compare to mine, might be a screen scrape ...
Settings -> : regional settings perhaps ?

Make sure all your work is saved before you start this, I managed to lock KUbuntu up by looking at the regional settings page. Thats a most un-ubuntu sort of thing. Maybe some interaction with VirtualBox ?
Now, other things to try. Get these going, try to type into a note window .... quit.
First, easy one, please start tomboy-ng from the command line, useful to see if there are any debug messages coming through.
$> tomboy-ng  <enter>
...
$> QT_QPA_PLATFORM=xcb  tomboy-ng <enter>

In both cases, please ensure that your tomboy-ng is not running before you start, if it is, it just gets brought forward and the new one you are trying to start, terminates.
Anyway, another thing I'd like you to try please.  In the download repo, here,  https://github.com/tomboy-notes/tomboy-ng/releases/tag/v0.40a there are some beta test install kits. No, I don't expect a new .deb will help, I have made no recent (or otherwise) changes to input tools. But there are two AppImage packages there, recognizable by their name. One for Qt5 and one for Qt6, they both should run without any further libraries or installing. Just download, set the file's attribute to executable   $>  chmod u+x  tomboy-ng-x86_64-Qt5.AppImage  <enter>  and make sure your existing tomboy-ng is not running, then run it. The Qt6 one is particularly interesting as its possible (just) that we are seeing a Qt problem.
Thanks for your help !
Davo

I have interesting observations.
On wayland it works perfectly.
On X11 it does not (even after restart).
But I notice that on X11 I can use German umlaut keys ö, ä, ü.
However I cannot use any of the other letters or keys like backspace etc.

OK, now, that is confronting !  It works on Wayland but not (generally) on X11 ?   Thats the very first time I heard of anything working better under Wayland !
Did you try the comman line tests ?  A Qt5 app uses Wayland directly unless the QT_QPA_PLATFORM is set to xcb.  I wonder if your issue is xcb related ?
Using the GTK2 version of tomboy-ng suddenly becomes interesting too. It ignores Wayland completely going direct to X11.
Davo