Explore Cardano implementation

Explore Cardano Node implementation, including its Data Diffusion and Networking, as well as Ouroboros Consensus.

Following is an overview of Cardano Node implementation, as of version 8.1.2 (July 2023).
General Structure
Cardano is an open source, decentralized proof-of-stake blockchain platform. It is written in Haskell, an advanced, purely functional programming language. Thanks to the functional programming paradigm providing the referential transparency property, one can reason about and conveniently test individual components in isolation, employing a whole range of powerful methods for ensuring correctness of the code, such as basing the implementation on formal and executable specifications, extensive property-based testing, and running tests in simulation.
The general architecture of Cardano Node is described at high level in its documentation. It is based on concurrent Haskell, using the Async API, and software transactional memory (STM). An introduction to the architecture was presented in the talk "Using STM for Modular Concurrency" by Duncan Coutts. Basically, in this architecture, concurrent components expose STM transactions for retrieving relevant pieces of their mutable current state; the components then interact by combining and atomically executing such STM queries from other components and atomically updating the corresponding pieces of their own mutable state.
STM is an optimistic concurrency mechanism that allows STM transactions over shared mutable variables (transactional variables or TVars) to be safely composed and atomically executed, without exposing the implementation details, which is problematic with traditional synchronization primitives, such as locks. Beyond that, an STM transaction can be rolled back and retried based on an arbitrary condition; such retried transactions are suspended until one of the TVars that it has read from has been updated. In addition to that, alternative STM transactions can be composed together using the orElse combinator, which completes the first STM transaction which does not retry. The monadic interface and Haskell type system ensure that STM transactions cannot have visible side effects and thus are safe to roll back and retry. Though, STM transaction can evaluate to an IO action, which can then be executed in the IO monad and produce any desired side effects, after the STM transaction successfully completes.
Cardano makes use of the standard STM abstractions built on top of TVar, retry , and the orElse combinator, such as the synchronizing variable TMVar or bounded FIFO queue TBQueue. To avoid issues with lazy evaluation, such as space leaks due to buildup of large chains of unevaluated expressions, there are versions of those abstractions with strict evaluation implemented in the Control.Concurrent.Class.MonadSTM.Strict module of the strict-stm package. There is also at least one place in the code base that uses an ordinary synchronizing variable MVar instead of STM: the implementation of the addTx function on Mempool. The reason to forgo composable atomicity and blocking provided by STM in that case is for ensuring fairness between concurrent competing threads of execution.
Apart form that, Cardano implements some other interesting concurrency abstractions, e.g.:

TMergeVar, which is like  TMVar but has a non-blocking write operation, writeTMergeVar combining the new value with the one already stored in TMergeVar using the Semigroup operator <>, if the TMergeVar is not empty;
RAWLock, a Read-Append-Write (RAW) lock, which allows multiple concurrent readers, at most one appender, which is allowed to run concurrently with the readers, and at most one writer, which has exclusive access to the lock.

There is a convenience utility type Watcher providing the withWatcher and forkLinkedWatcher functions, which spawn a new thread that repeatedly performs an STM transaction checking the fingerprint from the result of an STM transaction and executes the specified action each time the fingerprint changes. To facilitate this pattern, there is a special data type, WithFingerprint, which stores a value together with its fingerprint. The fingerprint is represented by the Fingerprint type, an alias to Word64. WithFingerprint comes useful for fingerprinting values that cannot be compared directly, e.g. functions.
Cardano code makes heavy use of advanced type-level programming features supported by Haskell and the language extensions implemented by GHC, such as generalised algebraic data types (GADTs), type families, datatype promotion, pattern synonyms, etc. The code is highly polymorphic, in particular over the specific choice of ledger and the main monad.
The ledger-specific types are defined through type families indexed by the block type, e.g.:

BlockProtocol: mapping to the type of consensus protocol,
BlockConfig: mapping to the type of static configuration required to work with this type of blocks,
CodecConfig: mapping to the type of static configuration required for serialization and deserialization of types pertaining to the block type,
Header: mapping to the block header type, etc.

The abstract interface required from the main monad is represented with the IOLike type class, whereas the io-classes package defines a monad class hierarchy that is used in the type constraints of IOLike. The type classes from io-classes provide a drop-in replacement for the IO monad. This allows using IOSim, a pure IO simulator monad provided by the io-sim package, in place of the  IO monad with the same code base.
To conveniently manage allocated resources, there is the ResourceRegistry type. ResourceRegistry represents a piece of state tracking which resources have been allocated so that resources can be allocated/deallocated without strict lexical scoping. Normally, the registry is only used as a fallback for resource deallocation. There are also the linkToRegistry and forkLinkedThread functions linking a thread to the resource registry and the thread that has created the registry. Linking threads to the registry works similarly to how the link function from the async package allows linking a spawned thread to the current thread to rethrow exceptions from the child to the parent thread.
Cardano Node's main entry point is the runNode function from the Cardano.Node.Run module of the cardano-node package, whereas the main components are created in the runWith function from the Ouroboros.Consensus.Node module of the ouroboros-consensus-diffusion package. The central data type of the node is NodeKernel, which provides access to node's main components. The main high-level components are the following:

Mempool for maintaining outstanding transactions that should be included in the next blocks;
ChainDB for maintaining the chain database comprised of:

ImmutableDB, storing the part of the chain that can't roll back,
VolatileDB, storing the blocks near the tip of the chain, possibly in multiple competing forks,
LedgerDB, storing snapshots of the ledger state for blocks in ImmutableDB (and in-memory snapshots for the rest);

BlockchainTime for measuring time slots;
Block Forging for producing new blocks;
Data Diffusion layer for communicating (relaying) blocks and transactions across nodes.

Communication
Communication between Cardano Nodes is based on long-running TCP connections established between nodes and providing reliable, ordered message streams represented with the Channel type. The ordered and reliable delivery is guaranteed by the TCP/IP protocol. Relying on these assumptions greatly simplifies the design of network protocols built on top of such message streams. The overall communication protocol is composed from largely independent mini-protocols so that no single protocol becomes too complex to reason about.
In order to take full advantage of reliable and ordered message streams, the mini-protocols are peer-to-peer, connection-oriented, and stateful. A node runs many instances of mini-protocols, including many instances of the same mini-protocol, where each instance communicates with the dual instance of the corresponding peer node, i.e. there is a fixed pairing of mini-protocol instances between peer nodes, each instance only communicating with its dual instance. Mini-protocol instances maintain limited local information based on previous direct interaction with the dual instance, thus each received protocol message is processed in the context of the ordered sequence of messages previously exchanged with the remote peer. The mini-protocols use consumer-driven control flow, which allows simply disconnecting from an offending peer in case of any detected protocol violation.
All mini-protocols that communicate with the same peer share a single communication channel, which is multiplexed by splitting data from individual mini-protocols into segments for transmission over a long-running TCP connection. The multiplexing is implemented in the network-mux package. The multiplexer ensures certain level of fairness among mini-protocol instances competing for the same channel by maintaining a round-robin queue of TranslocationServiceRequests from mini-protocols. The demultiplexer, on the other side, maintains a limited queue of outstanding inbound messages for each mini-protocol instance; overflowing a queue is treated as a protocol violation. The application-level protocol logic, thus, has to ensure that the amount of data in flight never exceeds the queue limit. This means that application-level mini-protocols cannot take full advantage of backpressure provided by the transport layer. It is also worth noting that there is no prioritization between different kinds of traffic, due to the assumption that bulk delivery of blocks (i.e. in recovery mode) and normal, interactive, operation (e.g. chain following) are mutually exclusive.
Mini-protocols follow pull-based strategy, namely exclusively using demand-driven control flow without polling, so a node only requests more work when it is ready, rather than having work pushed upon it. This approach suits particularly well in a trustless setting where both sides need to be protected against resource consumption attacks from the other side: for each node and each peer connected to that node, the node controls the rate of data arriving, the maximum concurrency, and the amount of outstanding data.
Cardano's architecture follows a threat-aware design approach. It emphasizes detecting protocol violations when handling inbound data as early as possible. The checks are moved to the earliest point in the operational cycle where the data is available but the least resources have been expended to process the received data. For instance, block and transaction relaying is interleaved with validation to avoid circulating invalid data between nodes. This approach works well with the connection-oriented and stateful mini-protocol implementation: Inbound messages must be well-formed syntactically and semantically valid in the context of information previously received from the peer node.
The implementation of mini-protocols relies on the typed-protocols package, a generic framework for implementing application-level protocols, with enforcement of certain properties via Haskell types. The framework was presented in the talk "Well-Typed Communication Protocols" by Duncan Coutts.
The framework implements a simple form of session typing. Within this framework, protocols are described as (not necessarily finite) state machines encoded into Haskell types. Each protocol involves only two parties where both parties must agree on the same state of the protocol state machine. The allowed transitions between states correspond to messages exchanged between the peers, so the protocol state determines which messages are allowed to be sent or must be accepted when received, at type level. In each protocol state, only one side has the agency to send a message; the other side must accept any message valid for the state. These constraints simplify application-level protocol implementation, allow early detection of protocol violations, and make the protocols themselves deadlock-free by construction.
To provide good performance through latency hiding, typed protocols can be pipelined, in many cases without any change to the protocol state machine. The framework provides a way to write a protocol implementation in a pipelined way. A pipelined client (server) can be paired up against a "normal" server (client) without any change to the latter. The pipeline depth is expressed through dependent types to be _controlled by the application-level protocol.
The state machine of a typed protocol is defined in terms of the Protocol type class, which declares the message type and agency. The protocol implementation is then expressed in terms of the Peer data type, or PeerPipelined for a pipelined version. These definitions heavily rely on advanced type-level programming techniques, such as GADTs and dependent types. Peer and PeerPipelined follow a pattern similar to stream programming, e.g. as implemented in the pipes package. Types protocols can be executed using the runPeerWithDriver or runPipelinedPeerWithDriver function against the Driver interface used as a source and sink for the typed protocol messages. This does not require buffered communication, i.e. all protocols defined in the typed-protocols framework can be run with synchronous communication.
Though, it is not particularly convenient to deal with complex type structure when expressing protocols in terms of the Peer and PeerPipelined. Therefore there is an alternative, equivalent way to represent a typed protocol implementation using protocol-specific wrappers, with simpler types. In that representation, the actual implementation in terms of Peer/PeerPipelined is derived from simpler algebraic data types, protocol wrappers, that represent message handlers and state transition actions. Each protocol state where the side has agency to send a message corresponds to a sum type with data constructors corresponding to possible messages to be sent, whereas the other side uses a product type with fields corresponding to handlers for the possible inbound messages. For a non-trivial example, the Chain Synchronization protocol is defined in the Ouroboros.Network.Protocol.ChainSync.Type module, whereas its protocol wrappers together with functions to convert the wrapped representation are defined in the Ouroboros.Network.Protocol.ChainSync.Client, Ouroboros.Network.Protocol.ChainSync.ClientPipelined, and Ouroboros.Network.Protocol.ChainSync.Server modules. The actual implementation of the protocol in terms of the protocol wrappers can be found in the Ouroboros.Consensus.MiniProtocol.ChainSync.Client and Ouroboros.Consensus.MiniProtocol.ChainSync.Server modules.
Mini-protocols in Cardano are request-response protocols describing the initiator and the responder within the communication stream; they are polymorphic in the request and response data that is being transmitted. There are the following main node-to-node mini-protocols:

Chain Synchronisation: a protocol that allows a node to reconstruct a chain of an upstream node;
Block Fetch: a protocol that allows a node to download block bodies from another node;
Transaction Submission: a protocol that allows submission of transactions;
Keep-Alive: a protocol that ensures continuous connection between nodes.

More details can be found in the documentation and the following documents:

An introduction to the design of data diffusion and networking for Cardano Shelley,
The Shelley networking protocol specification.

Consensus Protocol Implementation
Cardano runs the Ouroboros protocol, a time-slot based, proof-of-stake consensus protocol. There is no clear line between the consensus and network layers in the Cardano Node design. Consensus protocol logic seems to be distributed across different components. The chain database component (ChainDB) maintains a set of candidate blocks, performs the final stages of block validation, final chain selection and adoption of a new chain as the current chain. The data diffusion layer performs initial validation and disseminates blocks and transactions among peers. The Block Forging component creates new blocks when the node becomes the slot leader. Those components incorporate the common logic, which is parameterized for a concrete version of the consensus protocol depending on the specific choice of ledger, determined by the block type. For instance, the ConsensusProtocol type class describes the consensus-specific part, whereas the BlockSupportsProtocol provides an evidence that a block supports the protocol. BlockForging defines consensus-specific interface for block production; it comes from ProtocolInfo. There is an overview for the type trickery of the abstract protocol specification and consensus-level ledger view.
Tracing and Controlling Execution
The code uses the contra-tracer package for extensive tracing. The package implements a so-called contravariant tracing. This is an interesting approach to logging, tracing, monitoring, etc. in which domain-specific values are provided to domain-agnostic processors. The Contravariant instance for Tracer m allows domain-agnostic tracers to be adapted and stand in where a domain-specific tracer is required. This discourages using text of JSON within the program for diagnostic logging/tracing in favor of dedicated domain-specific event types. The same mechanism is used to collect metrics.
Since the code is polymorphic in the main monad, it can be interpreted in simulation for testing. The io-sim  repository implements a set of packages for that purpose. It provides a simulation environment for the IO monad, IOSim, that closely follows core Haskell packages (base, async, stm). It provides the following features:

deterministic execution of concurrent code,
tracing of interesting execution events, e.g. for debugging, analysis, and constructing complex property tests,
discrete-event simulation for faster-than-real-time simulation of time delays,
running precisely the same code in IO for production or in the IO simulator for testing.


Docs suggestions and typo fixes

Will continue to re-read the docs during the week and maybe add more commits. Thank you all!

Great stuff, thank you!


Bump plutus and fix breaking changes

Mainly updating ByteString to BuiltinByteString.
Reference:

https://github.com/input-output-hk/plutus/commit/1e287eb7eefb25b4a41f4d930ae0f8335431304f

thank you for helping out @kk-hainq ! We actually already moved to BuiltinByteString (see #2407).

thank you for helping out @kk-hainq ! We actually already moved to BuiltinByteString (see #2407).

Oh, it's in plutus-staging! Maybe I can force push the delete of the duplicate entry in cabal.project for this one?

thank you for helping out @kk-hainq ! We actually already moved to BuiltinByteString (see #2407).

Oh, it's in plutus-staging! I think we can close this then, any idea when we can merge that one?

Oh, it's in plutus-staging! I think we can close this then, any idea when we can merge that one?
Also wonder if we can still force push to keep this one, for deleting the duplicate flat entry in cabal.project.

I'm not sure, @michaelpj probably knows more, though.
I can put up a separate PR to remove the flat entry. I'd have to do that anyway, since our CI won't run on forks. (which is really unfortunate, see https://github.com/buildkite/feedback/issues/293)

Oh, it's in plutus-staging! I think we can close this then, any idea when we can merge that one?
Also wonder if we can still force push to keep this one, for deleting the duplicate flat entry in cabal.project.

I'm not sure, @michaelpj probably knows more, though.
I can put up a separate PR to remove the flat entry. I'd have to do that anyway, since our CI won't run on forks. (which is really unfortunate, see buildkite/feedback#293)

Sure, thanks. I'll just close this for now then.

The workflow is to merge plutus-staging regularly, although we haven't worked out a schedule yet (we're still working on this process...). I was planning to do it today.

The workflow is to merge plutus-staging regularly, although we haven't worked out a schedule yet (we're still working on this process...). I was planning to do it today.

Cardano has brought us so much opportunity that we've been wanting to contribute a lot back. Bumping dependency versions of key components when there are breaking changes is a good start, please tell us if we could help in any way.
We'd like to write more documentation, especially on Plutus and Smart Contracts too.


Jackson based dataformats for Avro and Protobuf

Fixes CAMEL-16615
This adds camel-jackson-avro and camel-jackson-protobuf.
The advantage of these is that they can all work with Jackson types so we can seamlessly convert data between formats.
I was not able to fix "since" version in autogenerated adocs, but should not be a big issue.

The catalog file is just a copy, so we can just regen and then it will copy the fresh file over into catalog.


[ansible] github-release option failing due to flannel timeouts

I have 4 nodes. 1 Master and 3 Workers. I am running Fedora 24 on all of them and have run dnf -y update on all of them as well. I ran the same experiment earlier today with the packageManager option and everything worked and I had a working Kube 1.2.2 cluster. However, I want a Kube 1.4 cluster, so I used the github-release option on an identical setup but new cluster and it fails with the exception below:
TASK [flannel : Start flannel] *************************************************
changed: [leader.kube]
fatal: [node-2.kube]: FAILED! => {"changed": false, "failed": true, "msg": "Job for flanneld.service failed because a timeout was exceeded. See "systemctl status flanneld.service" and "journalctl -xe" for details.\n"}
fatal: [node-1.kube]: FAILED! => {"changed": false, "failed": true, "msg": "Job for flanneld.service failed because a timeout was exceeded. See "systemctl status flanneld.service" and "journalctl -xe" for details.\n"}
fatal: [node-3.kube]: FAILED! => {"changed": false, "failed": true, "msg": "Job for flanneld.service failed because a timeout was exceeded. See "systemctl status flanneld.service" and "journalctl -xe" for details.\n"}

@wattsteve can you ssh to any of the machines, run systemctl start flannel manually and get output of systemctl status flannel (if there are some logs). Or run the flannel binary itself and get its error output?

Reproduced it with Vagrant. The problem is flanneld service calls /usr/bin/flanneld -etcd-endpoints=${FLANNEL_ETCD} -etcd-prefix=${FLANNEL_ETCD_KEY} $FLANNEL_OPTIONS where FLANNEL_ETCD is set to http://kube-master-1:2379. Looking into /etc/hosts there is no entry for kube-master-1. I am curious how much ansible 2.2 changes from 2.1 since with the 2.1 this does not happen. Investigating further.

Ok, with ansible 2.1 the etcd looks like this:
<IP_ADDRESS> kube-node-1 kube-node-1  localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6

<IP_ADDRESS> kube-master-1
<IP_ADDRESS> kube-node-1
<IP_ADDRESS> kube-node-2

With ansible 2.1, the last three lines with cluster member ips are gone.

Hmm, 2.1:
TASK [Build hosts file] ********************************************************
[DEPRECATION WARNING]: Using bare variables is deprecated. Update your
playbooks so that the environment value uses the full variable syntax
('{{groups['all']}}').
This feature will be removed in a future release.
Deprecation warnings can be disabled by setting deprecation_warnings=False in
ansible.cfg.
[DEPRECATION WARNING]: Using bare variables is deprecated. Update your
playbooks so that the environment value uses the full variable syntax
('{{groups['all']}}').
This feature will be removed in a future release.
Deprecation warnings can be disabled by setting deprecation_warnings=False in
ansible.cfg.
[DEPRECATION WARNING]: Using bare variables is deprecated. Update your
playbooks so that the environment value uses the full variable syntax
('{{groups['all']}}').
This feature will be removed in a future release.
Deprecation warnings can be disabled by setting deprecation_warnings=False in
ansible.cfg.
changed: [kube-master-1] => (item=kube-master-1)
changed: [kube-node-1] => (item=kube-master-1)
changed: [kube-node-2] => (item=kube-master-1)
changed: [kube-master-1] => (item=kube-node-1)
changed: [kube-node-2] => (item=kube-node-1)
changed: [kube-node-1] => (item=kube-node-1)
changed: [kube-master-1] => (item=kube-node-2)
changed: [kube-node-1] => (item=kube-node-2)
changed: [kube-node-2] => (item=kube-node-2)

2.2:
TASK [Build hosts file] ********************************************************
skipping: [kube-node-1] => (item=groups['all'])
skipping: [kube-master-1] => (item=groups['all'])
skipping: [kube-node-2] => (item=groups['all'])

So far to ignoring warnings. @wattsteve what is content of /etc/hosts inside of node-[1-3].kube?


Build: Enable Assertions

(for week 10 tutorial)
Now, assertions are enabled on gradle and we are able to utilize this important tool to test developer assumptions.
Lets improve build process

Great job enabling assertions!


size mismatch for model.diffusion_model.output_blocks.11.1.transformer_blocks.0.attn2.to_v.weight: copying a param with shape torch.Size([320, 768]) from checkpoint, the shape in current model is torch.Size([320, 1280]).

how to solve

Is this happening during inference (when running txt2img.py etc.)? It looks like you're trying to load a stable diffusion embedding into an LDM model. Please make sure you're using the correct inference script and configuration file.

train is: "python main.py --base configs/stable-diffusion/v1-finetune_lowmemory.yaml -t --actual_resume models/ldm/text2img-large/model.ckpt -n sanna --data_root D:\Projects\AIPaint\SDTrain\Stable-textual-inversion_win\SD\women --no-test true --gpus 1 --init-word woman"
paint is: ">python txt2img.py --ddim_eta 0.0 --n_samples 8 --n_iter 2 --scale 10.0 --ddim_steps 50 --embedding_path "D:\Projects\AIPaint\SDTrain\textual_inversion\embeddings\woman01.pt" --ckpt_path "D:\Projects\AIPaint\SDTrain\textual_inversion\models\model.ckpt" --prompt "a woman01""
model.ckpt is from stabel diffusion v1.4

--actual_resume models/ldm/text2img-large/model.ckpt implies that you're loading the LDM model, but you're using --base configs/stable-diffusion/v1-finetune_lowmemory.yaml which is a stable-diffusion and not a latent-diffusion config.
Given your inference command, I'm guessing you want to train and run stable-diffusion, so you should change the training command to use --actual_resume "D:\Projects\AIPaint\SDTrain\textual_inversion\models\model.ckpt"


Android app will restart when I back to desktop and click the app icon

Hi!
I use the react-native 0.28.0 Android  int  the Mac,the problem will be appear when I release the android app and install the phone .When I was operating app for some time, back to the desktop to click on the app icon, app will be re opened and back to the first page of app. I do not know whether the react-native or the other class libraries cause this problem. This is my pakage.json:
 {
 "name": "FasApp",
  "version": "0.0.1",'
  "private": true,
  "scripts": {
    "start": "node node_modules/react-native/local-cli/cli.js start",
    "dev": "JS_DIR=`pwd`; cd node_modules/react-native; npm run start -- --root $JS_DIR  --reset-cache",
    "release": "rm -rf $TMPDIR/react-*; rm ios/main.jsbundle; react-native bundle --platform ios --dev false --entry-file index.ios.js --bundle-output ios/main.jsbundle --assets-dest ios/",
    "test": "jest"
  },
  "dependencies": {
    "backo2": "^1.0.2",
    "co": "^4.6.0",
    "component-bind": "^1.0.0",
    "component-emitter": "^1.2.1",
    "dateformat": "^1.0.12",
    "events": "^1.1.0",
    "flux": "^2.1.1",
    "indexof": "0.0.1",
    "json-stringify-safe": "^5.0.1",
    "keymirror": "^0.1.1",
    "lodash": "^4.13.1",
    "moment": "^2.13.0",
    "mx-artifacts": "^0.2.2",
    "numeral": "^1.5.3",
    "qs": "^6.2.0",
    "react": "^15.1.0",
    "react-addons-pure-render-mixin": "^15.1.0",
    "react-native": "^0.28.0",
    "react-native-alphabetlistview": "^0.2.0",
    "react-native-device-info": "^0.9.3",
    "react-native-dismiss-keyboard": "^1.0.0",
    "react-native-extra-dimensions-android": "^0.17.0",
    "react-native-fs": "^1.5.1",
    "react-native-invertible-scroll-view": "^1.0.0",
    "react-native-qiniu": "^0.1.0",
    "react-native-share": "^1.0.11",
    "react-native-swipeout": "^2.0.12",
    "react-native-vector-icons": "^2.0.3",
    "react-native-viewpager": "^0.2.11",
    "react-timer-mixin": "^0.13.3",
    "realm": "^0.13.2"
  },
  "devDependencies": {
    "jest-cli": "^0.9.2"
  },
  "jest": {
    "rootDir": "./",
    "scriptPreprocessor": "<rootDir>/node_modules/react-native/jestSupport/preprocessor.js",
    "setupEnvScriptFile": "<rootDir>/node_modules/react-native/jestSupport/env.js",
    "testPathIgnorePatterns": [
      "/node_modules/",
      "packager/react-packager/src/Activity/"
    ],
    "testFileExtensions": [
      "js"
    ],
    "unmockedModulePathPatterns": [
      "promise",
      "source-map"
    ]
  }
}

Are you using a low memory Android phone?

There's in an options "don't keep activities" in developer-options of your phone. Did you enable that?

@satya164  i m using a sony z5 premium and i m having the same issue. i verified @fab1an  option and it disabled.

@vison123 could you please paste your index.android.js ?

@vison123 do you make use of BackAndroid component ? if not its normal you app restart everytime, cause it kill the app if you use backbutton

I have the same issues =\

@vison123  @Rpaixao  you have to remove you listener when you leave the component overwise it ll be executed twice and more ( depending on how much listener you add )
see https://github.com/facebook/react-native/issues/4736

This is happening in this app too:
https://github.com/discourse/DiscourseMobile

I found the next issue related to this thread:
Happening in my project v0.29, but i just tested init a new project v0.33 and it's behaving the same way.
When the app is running, press on the home button of the device/simulator and open the app by clicking on the desktop icon, what happens here is that the app restarts mounting the component again without have unmounted it before, resulting in several instances of the app running if you do this many times.

After this if you exit the app as you did initially pressing on the home button and open it by clicking on the desktop icon, this won't restart the app mounting it again, behaving from this moment correctly.
In the same way, if once the app first starts you exit it using the back button, afterwards the home button will behave right.
I don't know anything about android, but what is happening when the app is killed that after this if you send the app to the background using home button it works properly?
Also  in the same manner, If you implement a package like react-native-activity-android to avoid kill the app when the back button is pressed, you will end up with multiples js app instances running.

should this be a new issue?@satya164

Something like this makes the trick, but I also would need to hook in home button press in case you exit the app first pressing this button

Is this somehow related to https://github.com/facebook/react-native/issues/7079?
I have similar problem as https://github.com/facebook/react-native/issues/8570#issuecomment-249070558 when:

Starting my app from "Launch" button in play store
Then launch the app from launcher
Then launch the app from play store
???
I ended with a stack of ReactActivity

I believe (1) opens the app using Deep link via intent but then again it would be nice if someone that's well-versed in Android comment on this..

It seems like since there is a reasonable workaround we can close this issue now. Thanks folks for chipping in figuring out what to do here.


Add badges to README, restructure content, green builds

Adds a few sweet badges at the top of the README  and restructures the content.
Please only merge, when the Travis build is green! -- I may add a few more commits to make this happen.

This is now ready for being merged! :+1:


Discussion: deprecating the prop API in favor of tProp

There's been some discussion of this in a couple places so I thought it might be useful to have a separate issue about it. https://github.com/xaviergonz/mobx-keystone/issues/299 https://github.com/xaviergonz/mobx-keystone/discussions/303
Currently there are two APIs for defining model properties in mobx-keystone. prop allows defining properties that only have compile time typescript types. tProp allows defining properties that have compile time typescript types and optionally the ability to check types at runtime. tProp with runtime checking disabled is identical in behavior to prop.
Per previous discussions, there might be a number of benefits to removing the prop API since identical behavior can be achieved with tProp. So far I see the following benefits:

Remove confusion surrounding two APIs with overlapping purposes
Potentially remove the need for $modelType
Simplify snapshot and model creation behavior, automatically construct nested models and avoid silent runtime failures where objects are returned instead of model instances
Easier to add new functionality since only a single API needs to be considered and runtime type info could be used to support features (ex: Enable deep optional properties on nested objects)
Closer to the MST API, easier for people coming from MST to understand

I think there are a number of potential downsides to removing the prop API as well:

Existing users of the prop API would eventually need to transition to tProp. I think this might be very straightforward however since tProp is a superset of prop. Perhaps it could even be handled automatically by a codemod if it's a large impact?
Potentially slower typescript performance? I'm not sure if there's any significant overhead in inferring prop types from calls to tProp vs a direct type definition on prop.

I'm sure I'm missing things however. What do you think @xaviergonz @sisp ?

v0.64.0 added a feature to alleviate this problem.
Basically, now if you use a tProp to model then $modelType is not a requirement anymore in the input snapshot (it will still show in the getSnapshot result though)
eg.
@model(...)
class M1 extends Model({
  x: tProp(0)
}) {}

@model(...)
class M2 extends Model({
  m: tProp(M1)
}) {}

const sn = { m: { x: 10 } }
const m2 = fromSnapshot(M2, sn); // ok

That's awesome!


Provide a way to reset the SA login's password in cases where it has been forgotten

For anyone trying to workaround this whilst waiting for an official fix, it seems fairly easy to do so by installing sudo and making a small tweak to one of the scripts.
In your Dockerfile
FROM microsoft/mssql-server-linux:latest
[...]
RUN apt-get update && apt-get install -y \
	sudo \
	&& rm -rf /var/lib/apt/lists/*

And then in the shell adjust the script (or use a Docker RUN command):
sed -i 's/mssqlUser = "mssql"/mssqlUser = "root"/g' /opt/mssql/lib/mssql-conf/mssqlconfhelper.py

Finally, you can use the normal command to reset the password:
sa_password=`cat /run/secrets/sa_password`
MSSQL_SA_PASSWORD=$sa_password /opt/mssql/bin/mssql-conf --noprompt set-sa-password

Any movement on this?
@ph1ll I'm not seeing a docker file (by default) on macOS in ~/Library/Containers/com.docker.docker/Data/vms/0 (where the SQL docker image is).


Add Jupyter notebook for SVS Support

Update notebooks/Welcome.ipynb
Add notebooks/Supporting_Aperio_SVS_Format.ipynb to show loading Aperio SVS images

rerun tests

There's some commented code in the notebook like
#         start_loc_data = [(sx, sy)
...

Should we drop that?

There's some commented code in the notebook like
#         start_loc_data = [(sx, sy)
...

Should we drop that?
Edit: Also wonder if we should break the Experiment Code section out into a script

Good idea! Thank you for the suggestion. Moved as scripts and removed unnecessary comments.

@gpucibot merge


Validation: Duplicate alt-text error fixes.

Fixed:

image-alt-text-duplicated errors (duplicate alt text descriptions within an article)

This PR fixes 96 issues across 4 files and is part of ongoing Validation cleanup efforts. No other content has been edited or added.

Global effort to fix validation errors
@Rick-Anderson
The Content & Learning team is fixing build validation errors on docs.microsoft.com. This effort will eliminate potential accessibility, security, and usability issues. This PR includes only build validation fixes and does not change other content. I’d very much appreciate your timely review and feedback on these changes I’ve suggested. Thanks!


Something chews CPU time on uniform uploads

Even when a scene has fully loaded and the camera is stationary, uniform uploads seem to be burning a rather large amount of CPU time. What's happening?

I might be mistaken on this... it looks like the flag goes away once the group is rendered once. However, something is clearly burning cycles on uniform updating...

The biggest culprit might be context switches between different textures (ie calls to bindTexture and activeTexture). Not sure what the best way to reduce these would be. In particular, probably because of layering, terrain tends to cause a lot more of these calls than, say, a few WMOs or doodads.

It would appear that objects with transparent materials are sorted by depth, whereas opaque objects are sorted by material. We're overusing transparent materials for M2s: the no-backface-culling flag and blending mode 1 both cause transparent flagging, even though neither case ought to necessitate it.


Cannot read properties of undefined (reading 'memory') in plonk wasm file

When I run vite build in my app, I get this error:
vite v4.3.9 building SSR bundle for production...
✓ 68 modules transformed.

node:internal/event_target:916
  process.nextTick(() => { throw err; });
                           ^
TypeError [Error]: Cannot read properties of undefined (reading 'memory')
    at Object.<anonymous> (/Users/me/Projects/zkapps/mina-rng/node_modules/snarkyjs/dist/node/_node_bindings/plonk_wasm.cjs:14:27)
    at Module._compile (node:internal/modules/cjs/loader:1103:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1157:10)
    at Module.load (node:internal/modules/cjs/loader:981:32)
    at Function.Module._load (node:internal/modules/cjs/loader:822:12)
    at Function.wrappedLoad [as _load] (/Users/me/Projects/zkapps/mina-rng/node_modules/newrelic/lib/shimmer.js:313:24)
    at ModuleWrap.<anonymous> (node:internal/modules/esm/translators:168:29)
    at ModuleJob.run (node:internal/modules/esm/module_job:197:25)
    at async Promise.all (index 0)
    at async ESMLoader.import (node:internal/modules/esm/loader:337:24)
Emitted 'error' event on Worker instance at:
    at Worker.[kOnErrorMessage] (node:internal/worker:289:10)
    at Worker.[kOnMessage] (node:internal/worker:300:37)
    at MessagePort.<anonymous> (node:internal/worker:201:57)
    at MessagePort.[nodejs.internal.kHybridDispatch] (node:internal/event_target:647:20)
    at MessagePort.exports.emitMessage (node:internal/per_context/messageport:23:28)

I am using version 0.10.1.

It seems that vite tries to bundle the nodejs version of snarkyjs instead of using the prebuilt web bundle - this won't work for a couple of reasons

SnarkyJS tries to tell bundlers how to resolve the import by declaring an 'exports' field in package.json. I would assume that vite can handle that, but maybe we have to tweak for it to work

Oh.. just realized that it refers to the 'SSR bundle'. So it bundling the node version is correct after all (although I don't see the point of bundling code for nodejs, after all it works to just execute it)

@mitschabaude In the past I've had success by just disabling SSR for all of my pages.  I can try that.
This is the RNG app I built for zk-ignite cohort 0, which was working fine before.  I found that if I update just the snarkyJS version and pin @svekte/kit and @svelte/adapter-node to their previous versions, then the app builds.  But in the zkapp-cli, you are using the newest version of @svelte/kit and @svelte/adapter-auto in the example project, which builds fine.  It's annoying to have to track down build errors more often than not when making new projects and updating old ones.

just realized that it refers to the 'SSR bundle'. So it bundling the node version is correct after all

Is this actually true?  My understanding of SSR is that the server generates an html and js functions before the user requests it, but then if the user interacts with those functions, then they will be executed in a browser environment, not in node.  That's why it's always made sense to me to disable SSR for snarkyJS interactions.

Hm maybe we should pin the sveltekit version 🤔

@mitschabaude the mina-rng app does not use the zkapp-cli scaffolding.  I only mention that because that scaffold does use the most up-to-date of every package, and my app only works when I don't.
I should clarify something - the mina-rng app only uses snarkyJS on the server side.  The html in the app is just some documentation about how to use the API.  The API is where I use snarkyJS to sign data, and node is the correct execution environment to use.
let { isMainThread, workerData } = require('worker_threads');

let env = {};
if (isMainThread) {
  env.memory = new WebAssembly.Memory({
    initial: 20,
    maximum: 65536,
    shared: true,
  });
} else {
  env.memory = workerData.memory;  <-------
}

This is the the line with the error.  workerData is apparently undefined (Cannot read properties of undefined (reading 'memory')), but it is required and should be present?  My node version is v16.14.2

For reference, here's the app: https://github.com/45930/mina-rng

I only mention that because that scaffold does use the most up-to-date of every package, and my app only works when I don't.

I understood that! Pinning would solve it randomly breaking when new versions come out
Yeah the error you have is weird. Maybe the code doesn't work because it all runs in a worker so isMainThread is never true?

@mitschabaude @45930 have you found a fix for this error? we are facing the same issue running vite build
We are using:
"snarkyjs": "^0.12.1",
"@sveltejs/adapter-auto": "^2.0.1",
"@sveltejs/kit": "^1.15.10",
"@sveltejs/adapter-static": "^2.0.2"
SSR is disabled

#@leomanza are you running into this error after generating a svelte project using the zkapp-cli?

Yes, we have used this: https://github.com/mazito/zkLearn/tree/main/05-zkapp-sveltekit-ui that was based on the zkapp-cli, because the zkapp-cli had problems already described in this post.
Initially we had no problems with build, but from what I am testing now the problem seems to happens with dynamic routes (such as /credentials/[uid] and similar), because svelte-kit can not prerender them.
Please note the we are prerendering all pages.

So, this is what I have found so far.
First we set prerender on all pages, so +layout.js contains:
// for generating a static site
export const prerender = true;
export const ssr = false;
export const csr = true;

// may need it for static sites ??? NOT SURE
export const trailingSlash = 'always';

Second, we add dynamic pages to the prerender section in svelte.config.js:
//import adapter from '@sveltejs/adapter-auto';
import adapter from '@sveltejs/adapter-static';
import { vitePreprocess } from '@sveltejs/kit/vite';
import path from 'path';

/** @type {import('@sveltejs/kit').Config} */
const config = {
  // Consult https://kit.svelte.dev/docs/integrations#preprocessors
  // for more information about preprocessors
  preprocess: vitePreprocess(),

  kit: {
    // adapter-auto only supports some environments, see https://kit.svelte.dev/docs/adapter-auto for a list.
    // If your environment is not supported or you settled on a specific environment, switch out the adapter.
    // See https://kit.svelte.dev/docs/adapters for more information about adapters.
    adapter: adapter({
      // default options are shown. On some platforms
      // these options are set automatically — see below
      pages: 'build',
      assets: 'build',
      // fallback: undefined, // for static site
      fallback: 'index.html', // for SPA
      precompress: false,
      strict: true
    }),

    prerender: {
      entries: [
        '*',
        '/credential/claimed/[uid]',
        '/credential/issued/[uid]',
        '/credential/edit/[uid]',
      ]
    },

  }
};

export default config;

Then it builds, except when we import something from snarkyjs in the +pages.js of the given page.
For example, the following code in /credential/claimed/[uid]/+pages.js:
import { error } from '@sveltejs/kit';
import { getPlan, getCommunity, getClaim } from "@apis/queries";
import { getCurrentUser } from '@models/current-user';

async function loadClaim(params, user) {
  const claimUid = params.uid;

  let claim = await getClaim(claimUid);

  const plan = await getPlan(claim.planUid);

  const org = await getCommunity(claim.communityUid);

  claim.community = org.name;
  claim.type = plan.name;
  claim.description = plan.description;
  claim.image = plan.image;

  return claim;
}

/** @type {import('./$types').PageLoad} */
export async function load({ params, route, url }) {
  if (params.slug !== "") {
      let user = await getCurrentUser();

      let aClaim = await loadClaim(params, user);
      return JSON.parse(JSON.stringify(aClaim));
    }
    throw error(404, 'Not found');
}

Builds correctly:
...
.svelte-kit/output/server/index.js                                                          83.51 kB

Run npm run preview to preview your production build locally.

> Using @sveltejs/adapter-static
  Wrote site to "build"
  ✔ done
✓ built in 7.36s

BUT adding just import { Field } from 'snarkyjs'; to the top of the same code it generates the error:

node:internal/event_target:1036
  process.nextTick(() => { throw err; });
                           ^
TypeError [Error]: Cannot read properties of undefined (reading 'memory')
    at Object.<anonymous> (/home/mzito/tmp/sc-ui2/node_modules/snarkyjs/dist/node/bindings/compiled/_node_bindings/plonk_wasm.cjs:14:27)
    at Module._compile (node:internal/modules/cjs/loader:1275:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1329:10)
    at Module.load (node:internal/modules/cjs/loader:1133:32)
    at Module._load (node:internal/modules/cjs/loader:972:12)
    at ModuleWrap.<anonymous> (node:internal/modules/esm/translators:165:29)
    at ModuleJob.run (node:internal/modules/esm/module_job:193:25)
Emitted 'error' event on Worker instance at:
    at [kOnErrorMessage] (node:internal/worker:309:10)
    at [kOnMessage] (node:internal/worker:320:37)
    at MessagePort.<anonymous> (node:internal/worker:216:57)
    at [nodejs.internal.kHybridDispatch] (node:internal/event_target:761:20)
    at exports.emitMessage (node:internal/per_context/messageport:23:28)

Node.js v19.9.0

Please note that since we are using an static adapter and forcing CSR and disabling SSR for all pages (it is an SPA), it should never try to include any node code, as it seems it is happening.
Conclusions:

Including snarkyjs in any +page.js generates the build error.
Including snarkyjs (or some other code that includes snarkyjs) in .svelte files seems to work ok.

Hope this helps.

@mazito @45930 I believe we fixed this issue in the zkapp-cli scaffold after adding this to the vite config.
   optimizeDeps: {
        esbuildOptions: {
            target: 'esnext'
        }
    },

You should be able to generate and build a Svelte project using the zkapp-cli with the latest version 0.11.0 although the scaffold does not use a static adapter.

@ymekuria Yes, I have that option already settled in the project.
And it works fine for static routes, but the problem reported here is different, it happens with dynamic routes (such as /credential/[uid]).  I don't know if only with static adapter, but I suppose not.
Have you tested it with dynamic routes and including snarkyjs in the +pages.js of such routes ?
Anyway I solved it by:

cleaning up all snarkyjs imports from those pages, and also avoiding to import any code that imports snarky in any +pages.js too.

including those dynamic routes in the prerender.entries section of svelte.config.js.

Just writing it here for anyone who may have the same problem.


Feedback on first cut: methane values

There is a difference between CMIP6 and the first cut around 1950 for CH4. It is unclear whether this is a feature or a bug, to be investigated.
Data attached
Uploading 20240424_ch4-for-paul.tar.gz…

Data available here:
20240424_ch4-for-paul.tar.gz


Create test for the Capture logic

In the contract code we do not yet have logic to test that captures are working successfully. The test exists but the capture logic is not being tested. We need to write a test for this and ensure that all of it is functioning properly.
The test can be found here. https://github.com/realm-of-ra/mancala/blob/develop/contracts/src/tests/test_world.cairo#L115
The implementation of the capture logic can be found here: https://github.com/realm-of-ra/mancala/blob/develop/contracts/src/models/mancala_game.cairo#L167

please can i be assigne to this?

@web3technologies kindly assign this issue to me.

@mubarak23 do you have any updates or need any assistance with this issue?

The capture implementation is now at https://github.com/realm-of-ra/mancala/blob/7a80ad80c38762acbfd1d01ad886a68bf68b1dbc/contracts/src/models/mancala_game.cairo#L188 and test at https://github.com/realm-of-ra/mancala/blob/7a80ad80c38762acbfd1d01ad886a68bf68b1dbc/contracts/src/tests/test_world.cairo#L164

@xpanvictor yes these functions already exist and we need to test that the capture logic is properly implemented. Are you wanting to take this issue?

Yes, assign me to this, I'll work on it @web3technologies

Hi @web3technologies, I have experience implementing a starknet component and writing test for it from the previous ODHack, I would be a perfect fit for this issue.

Hello @web3technologies, can I get assigned this issue. Here is my profile https://app.onlydust.com/u/No-bodyq
I have some experience writing tests as you may see on my profile

I can do this

I've looked into this and I'm confident in delivering this if assigned. 2-3 days

Hello, Can i have some assistance with this... I'm working from a codespace. I've installed dojo and scarb but i keep getting this error code whenever i try to run scarb test  or scarb cairo-test. Am i missing something?

@No-bodyq you cannot use Scarb as a package manager for dojoengine.
Install dojo https://book.dojoengine.org/getting-started#install-using-dojoup and then use sozo test
Maybe go through this tutorial to help you get familiar with dojo
https://book.dojoengine.org/tutorial/dojo-starter
Feel free to join project telegram if you need further assistance:
https://t.me/+0UKU3P3Eh_s2NTRh


registeredSoftwareUpdatesProposals is supposed to be preserved

Background
During audit, we noticed that https://github.com/input-output-hk/cardano-ledger/blob/bccf097bc589d082600132b63e9e787071568e12/cardano-ledger/src/Cardano/Chain/Update/Validation/Interface.hs#L497 sets registeredSoftwareUpdateProposals to M.empty.  According to the spec (Ledger rule 49, Fig. 45), it should be preserved.
Proposed Solution

Giving this high priority; discrepancy needs to be at least investigated.

Addressed through https://github.com/input-output-hk/cardano-ledger-specs/pull/1279 - we bring the formal spec in line with the exec spec and the implementation.


[stable-4.2] Drop more insights-related code

(4.2: here, 4.3: eol, 4.4: #3173, 4.5: #3174, 4.6: #3175, master: #3176)
Follows #2413 & #1918
We're still depending on redux transitively and have a bunch of configs that are always set to the same value because most but not all insights-related code was removed from stable branches.
Removing unused dependencies, code and configs.

[webpack-cli] Failed to load '/home/himdel/ansible-hub-ui-4.2/config/standalone.prod.webpack.config.js' config
[webpack-cli] TypeError: Cannot read properties of undefined (reading 'appname')

TODO add #3178 and #3198


rm ./build/repalungs-wasm/.gitignore converts to rmdir .\build\repalungs-wasm\.gitignore

Describe The Bug

./build/repalungs-wasm/.gitignore is interpreted as a directory, not a file.
To Reproduce

Use rm ./build/repalungs-wasm/.gitignore.
Error Stack
Irrelevant
Code Sample
# using cargo-make
[tasks.my-task]
script_runner = "@shell"
script = '''
cargo new my_project
rm ./my_project/.gitignore

@JohnScience thanks a lot for reporting this bug.
i've fix it and released a new version.
you would require to reinstall cargo-make to get it.

Thank you!


Pretrained discriminator for BigGAN of 256 and 512?

Hi. I just wonder if there are pretrained discriminators for BigGAN under 256 and 512 size? Currently, only the corresponding pretrained  generators are avaliable. Thanks.

Hi @IceClear , we do not have pretrained discriminators for biggan 256 or 512, maybe you can just refer to https://github.com/huggingface/pytorch-pretrained-BigGAN. We would appreciate it a lot if you'd like offer converted weights.


Update vitessce-grid

Towards https://github.com/hubmapconsortium/portal-ui/issues/963, this upgrades Vitessce.  Another release will be required after this.
Related to #678.

I do see a warning in the console, but it looks like it is from a line of code unrelated to those that you updated

I think moving this line in VitessceGrid into a useEffect would solve it https://github.com/hms-dbmi/vitessce-grid/blob/2ad79a6309b90da813c9dc76cce9457abf74fba1/src/VitessceGrid.js#L64
I realize that fixing this would require releasing VitessceGrid again... every time we need to upgrade vitessce grid I ask myself whether it would be easier just to move the 4 source files (2 of which are test) to the Vitessce repo. I don't think there is enough of a benefit to keeping it in a separate repo, (though open to alternative thoughts about that )

That sounds good.  Going to merge this and then we can fix that as well before the next release, hopefully.


[MRG] Unifying preprocessors

Following the discussion in  #160, I took a shot at unifying the preprocessors now that apply_function is available for the Epochs object too (https://github.com/mne-tools/mne-python/pull/9235).
This is still work in progress, and I wanted to hear what @gemeinl @robintibor @agramfort think before going further. Notably, I made the choice to keep the following three cases possible:

fn is a string -> the corresponding method of Raw or Epochs is used
fn is a callable and apply_on_array is True -> the apply_function method of Raw or Epochs is used to apply the callable on the underlying numpy array
fn is a callable and apply_on_array is False -> the callable must modify the Raw or Epochs object directly (e.g. by modifying its _data attribute inplace or calling its methods)

I'm not sure apply_on_array is the best choice of name for this. Also, although case 3 seems like it might enable bad practice in some cases, it was required to allow the existing preprocess.filterbank function to work and so I included it.
The following are still missing:

I haven't modified the examples yet.
Version check for the latest MNE main (commit 5cc6eb8d7ebd102049843336d9d4a342f0f3e966 from 2021-04-05, which is not in the latest release v0.22.1)
If we decide to go with this new unified preprocessor object we might be able to go back to defining the preprocessing steps inside an OrderedDict as pairs (fn, kwargs) instead of a list of preprocessor objects.

The tests failed because the examples have not been updated yet, and there is no updated version check for MNE.

@sbbrandt You also had something in this direction? How do you feel about this implementation? Is it compatible with what you had?

@sbbrandt You also had something in this direction? How do you feel about this implementation? Is it compatible with what you had?

@robintibor You mean the work on the data augmentation issue? There I used a 'DataLoader' to manipulate the data during the training. That wouldn't be affected. The proposed data augmentation feature by @Simon-Free worked on the data set level and could need some updates after this PR.
Beside that I think unifying the preprocessors is good and I like the implementation.

@robintibor @sbbrandt what do you guys think about going back to the old way of defining preprocessing steps with an OrderedDict? I like it because it simplifies the API (the user doesn't have to understand another object).

Do you think you can make it backwards-compatible with deprecation notice inside constructor of MNE/NumpyPreproc @hubertjb ? Otherwise too many things will break I feel. What do others feel about this preprocessing API change @gemeinl @sliwy ?
regarding ordered dict, I am more against it, you will have to learn the structure then anyways what the different fields mean, to me object just more explicit about it... also remember last time we had the bug-generating issue that of course if you have twice the same key in the ordereddict only one will exist... but against it in any case :)

Having one unified Preprocessor sounds great.
Regarding the OrderedDict, I would prefer to use explicit Preprocessors definition - to me it's clearer and easier to understand.
Generally, regarding the API, our Preprocessors do similar job to torchvision transforms (except we apply Preprocessors to whole datasets instead of tensors). I would prefer to have similar API with something close to torchvision.transforms.Compose, e.g.:
from barindecode.preprocessors import Compose, Preprocessor
preprocessors = Compose([
        Preprocessor('pick_types', eeg=True, meg=False, stim=False),
        Preprocessor(scale, factor=factor)
])
preprocessors(ds)

It is similar to what people can already know from torchvision (when it comes to way of creating and applying) and it allows easily applying preprocessors without need to import preprocess function. Created preprocessors object can be next saved and as it contains all the needed tools/transforms to preprocess dataset, it can be used for other datasets without need to import preprocess function. I am not sure if we need this additional layer of abstraction with Compose class but in the end now we have it as well in a form of preprocess function. If you like the idea, maybe we should change it later and not in this PR just to not block the release.

@robintibor Thanks for the feedback! Very good point about OrderedDict, I had forgotten about that problem. I'll take it out of the PR. I'll see how I can make a deprecation notice for MNEPreproc and NumpyPreproc too.
@sliwy I really like the idea of a Compose object for preprocessors, it would be a lot more consistent with on-the-fly transforms. I agree with you, maybe let's keep this for a future PR?

@robintibor The tests have passed, let me know if you have any additional comments!

gret thanks merged

don't forget to add line in what's new. after lukas PR closed I will also merge that one and than we can always directly add inside the PR @hubertjb


Add Concourse pipeline for terraforming EKS

This PR creates a pipeline for terraforming EKS, consisting of two jobs terraform-eks-cluster and terraform-eks-cluster-addons, which terraforms the EKS cluster and the addons respectively. This is necessary for the reasons as detailed in commit c228db6.
We also change the way we assume the govuk-concourse-deployer role.  Instead of configuring the role once per Terraform provider per top-level module (deployments/*), we do the assume-role as part of the terraform-apply.sh script (by running aws sts assume-role). This results in less overall complexity. (Config for the k8s TF provider to assume a role is different from the equivalent config for the AWS provider.)
An alternative would be to keep all the existing assume-role configs and pass --role-arn to the aws command from the k8s TF provider config. Our approach however means we have one place and one way of assuming the role, rather than two different ways and multiple places (one per provider per top-level module).
We also add the govuk-concourse-deployer role to the aws-auth configmap so that Concourse can apply changes to the Kubernetes cluster via the Kubernetes Terraform provider. This is only strictly necessary when Concourse is modifying an existing cluster not created by Concourse (i.e. not created by the govuk-concourse-deployer role), but we're including this for robustness and to make the required permissions explicit.
We're running Terraform in automation mode per the best practices described in https://learn.hashicorp.com/tutorials/terraform/automate-terraform
Trello: https://trello.com/c/dqG4nXn0
Tested: successfully updates the existing govuk-tmp cluster (see Concourse pipeline).

Thanks for this work.
Did you consider using the flag --role-arn when setting up the k8s provider here?
Just trying to see what is the main benefit of setting the credentials using sts and having to deal with jq.

Thanks - we hadn't actually twigged that we could do this(!), but we think the solution we came up with is less complex and easier to understand than duplicating the config lots of times in two different ways for the two providers in every top-level module. We've updated the commit message and added a comment to signpost this.


[iOS] crash on open url click

i noticed that the app sometimes crashes when clicking the buttons to open a URL (twice). running in the simulator reveals that this is because the webbrowser::open should be called from the main thread.
here is the callstack:
Main Thread Checker: UI API called on a background thread: -[UIApplication openURL:options:completionHandler:]
PID: 15268, TID: 2913564, Thread name: Compute Task Pool (2), Queue name: com.apple.root.default-qos.overcommit, QoS: 0
Backtrace:
4   mobile                          0x00000001028b3264 _ZN10webbrowser2os21open_browser_internal17hb2a8817b3035f33bE + 276
5   mobile                          0x00000001028b3c54 _ZN10webbrowser25open_browser_with_options17hbb11ab7890e7699eE + 316
6   mobile                          0x00000001028b3abc _ZN10webbrowser12open_browser17ha7a37639961bfb2aE + 120
7   mobile                          0x000000010281a104 _ZN10 mobile 4menu17click_play_button17h6d942eb70d80c3f1E + 1452
8   mobile                          0x0000000102864ddc _ZN120_$LT$bevy_ecs..system..function_system..FunctionSystem$LT$Marker$C$F$GT$$u20$as$u20$bevy_ecs..system..system..System$GT$10run_unsafe17hebfc3b3728c14d1aE.llvm.4102418084858017275 + 228
9   mobile                          0x00000001042f4bdc _ZN91_$LT$futures_lite..future..CatchUnwind$LT$F$GT$$u20$as$u20$core..future..future..Future$GT$4poll17hed7ae4b87d5ebccaE + 68
10  mobile                          0x00000001042f2b74 _ZN3std9panicking3try17h026468a883549e6eE + 108
11  mobile                          0x00000001042fca44 _ZN10async_task3raw28RawTask$LT$F$C$T$C$S$C$M$GT$3run17h8c6e9ffc152f0c0fE.llvm.8900356627072360647 + 172
12  mobile                          0x00000001043312d0 _ZN12futures_lite6future8block_on17h16a86458e063e77aE + 2648
13  mobile                          0x00000001043394e0 _ZN3std10sys_common9backtrace28__rust_begin_short_backtrace17h7cc592fe5ea1097eE.llvm.16434831150002698626 + 320
14  mobile                          0x0000000104334f00 _ZN4core3ops8function6FnOnce40call_once$u7b$$u7b$vtable.shim$u7d$$u7d$17hfca70cd91ab5e418E + 200
15  mobile                          0x00000001044a1e60 _ZN3std3sys4unix6thread6Thread3new12thread_start17hcdba6f4cb854f822E + 36
16  libsystem_pthread.dylib             0x00000001075074c0 _pthread_start + 104
17  libsystem_pthread.dylib             0x00000001075026f0 thread_start + 8

I have not been able to reproduce this on any of the platforms I have access to (no iOS).
You wrote on Discord that you tried to add a non-send resource to the system and it did not fix the issue. Have you learned anything about it since?

diagnosed and fixed over in webbrowser crate: https://github.com/amodm/webbrowser-rs/issues/83

Awesome, thank you for following up on the issue. I will close the issue as soon as the patch is released from webbrowser.

its merged and released


How is this copyrightable?

Great work here but how is this list copyrightable?
https://www.copyright.gov/help/faq/faq-protect.html#:~:text=A mere listing of ingredients,a basis for copyright protection.
https://www.copyright.gov/circs/circ33.pdf
https://opensource.stackexchange.com/questions/10670/copyrightablity-of-word-lists#:~:text=Under US law%2C a list,original content%2C cannot be copyrighted.

Hi @rsterbal a list of words is not really copyrightable ...
If there were definitions attached then it could be copyrightable.


Remove deprecated configuration parameters from procedure resource

Terraform CLI and Provider Versions
Terraform CLI: 1.0.9
Snowflake-Labs/snowflake Version: 0.47.0 -> 0.80.0
Use Cases or Problem Statement
According to the official Snowflake documentation using the optional parameter “VOLATILE/IMMUTABLE” is deprecated in all languages.
https://docs.snowflake.com/en/sql-reference/sql/create-procedure#optional-parameters
Currently, this does not result in any major errors when creating/updating a stored procedure via terraform, but it might will in the future.
Proposal
Get rid of optional parameter “return_behavior”:
https://github.com/Snowflake-Labs/terraform-provider-snowflake/blob/main/pkg/resources/procedure.go
How much impact is this issue causing?
Low
Additional Information
No response

Hey @LukLight. Thanks for creating the issue.
We did not include these deprecated parameters in the SDK. We will remove it during migrating the resource to use the SDK implementation instead of the old one.

Sorry for the late answer @sfc-gh-asawicki !
I'll close the request as it is included in the Pull Request.


CrowPanel 7 Inch support.

Hi,
Can you add support for it's ESP WZ8048C070 ?
https://www.elecrow.com/esp32-display-7-inch-hmi-display-rgb-tft-lcd-touch-screen-support-lvgl.html
Specification:
Model: 7.0 inch -CrowPanel ESP32 display
Main Processor: ESP32-S3-WROOM-1-N4R8
Resolution: 800*480
Flash: 4MB
PSRAM: 8MB
I use these settings and the screen works fine with 0.7.0rc8 version. I haven't tried the latest 0.7.0rc11.
settings:
https://github.com/HASwitchPlate/openHASP/discussions/559
https://github.com/HASwitchPlate/openHASP/discussions/559#discussioncomment-7922944
openHASP
--
Version | 0.7.0-rc8 6cf4262
Build DateTime | Nov 22 2023 07:15:32 UTC
Environment | elecrow-s3-8048C070_4MB
Uptime | 25d 6h 30m 7s
Idle | long
Active Page | 1

Free Heap | 102.28 KiB
Free Block | 91.98 KiB
Fragmentation | 10%
PSRam Free | 7.16 MiB
PSRam Size | 7.99 MiB

Total | 64.00 KiB
Free | 24.79 KiB
Fragmentation | 4%

Server | 192.168.1.xx
Username | plate01
Client ID | plate01_8d2cd8
Status | Connected
Received | 220771
Published | 19713
Failed

BSSID | 2C:xxxxxxxxxx
SSID | Livebox-xxxx
Signal Strength | -69dBm (Weak)
IP Address | <IP_ADDRESS>
Gateway | <IP_ADDRESS>
DNS Server | <IP_ADDRESS>
MAC Address | EC:xxxxxxxxx

Model | ESP32-S3 rev0
Frequency | 240MHz
Core Version | <IP_ADDRESS>722
Reset Reason | CPU0: POWERON_RESET / CPU1: POWERON_RESET
Flash Size | 4.00 MiB
Program Size Used | 1.59 MiB
Program Size Free | 1.68 MiB
Filesystem Size | 560.00 KiB
Filesystem Used | 12.00 KiB
Filesystem Free | 548.00 KiB

This board has now been added to https://nightly.openhasp.com/

Hi,
I updated via ÔTA. During the update, the screen was shaking but it's good.
I'm going to do more in-depth testing, but for now it's working well.

Thank you very much for the addition, I'm so happy.

I don't see the option for Wireguard. I don't use Wireguard, it doesn't bother me.
Crowpanel 7inch 0.7.0-rc12

Sunton 7inch 0.7.0-rc11

It's a compile time option that's disabled now by default as it hasn't been well tested.
If you need WireGuard you can enable it when compiling.

It's a compile time option that's disabled now by default as it hasn't been well tested. If you need WireGuard you can enable it when compiling.

I thought it was a bug. No worries, I don't use wireguard ;-)

@fvanroie Have you tested the 7" screen yet, interested to see if its better than the sunton ?

@smcgann99,
I have both, a 7 inch sunton and 7 inch elecrow. The Elecrow only has 4MB of flash compared to 16MB on the Sunton. Afterwards they both work as well as the other for me.
The sunton I tinkered with to stabilize the wifi, I don't know if there has been a fix since.

Hello
does some have an image (bin) for the elecrow Crowpanel 7 Version3 ?@WarC0zes @fvanroie
i tried the nightbuild and made my own via gitpod, but both won't work with touch .
With ESPHOME (same Crowpanel) does the Touch work. very strange
Touchsettings in crowpanel-hmi.ini are similar to ESP Home Settings
see https://github.com/HASwitchPlate/openHASP/blob/master/user_setups/esp32s3/crowpanel-hmi.ini
But I have the exact values as in ESPHome where the same touchpanel works
; Touch Settings
-D HASP_USE_LGFX_TOUCH=1
-D TOUCH_WIDTH=800
-D TOUCH_HEIGHT=480
-D TOUCH_DRIVER=0x911
-D TOUCH_SCL=20
-D TOUCH_SDA=19
-D TOUCH_IRQ=-1
-D TOUCH_RST=-1
-D I2C_TOUCH_ADDRESS=0x5D
-D I2C_TOUCH_FREQUENCY=400000
are similar to ESPHome Settings
for Touch
i2c:
sda: 19
scl: 20
touchscreen:
platform: gt911
id: LCD_touchscreen
display: main_display
update_interval: 50ms
address: 0x5D
if i run mqtt publish to the device
hasp/crowpanel_7/state/statusupdate
the rsponse is
..
"tftDriver": "Other",
...
Any ideas ?
regard
Oliver


Add optional custom error message to assert

The user may now provide an optional 2nd argument to assert, which is the error message thrown if assert evaluates to false (e.g. assert(1 == 2, "Error")).

looks good :)


Different Governance Support

Overview
Updates the app to support a DAO having 'different' governance module than just the one option.
Major Changes

Seperates daoData Context into 3 providers, one for each current module
replaces the whole app in the DAODataProvider with FractalProvider
Setting and unsetting of a DAO is now handled by the DAOController component
Wraps the individual routes for the modules
Routing to a module and rendering a module is now done with a moduleAddress param

Minor Changes

Replaced the DAO 'Home' page with the DAODetails content.
Moved Proposal and nested routes to module route
updated breadcrumbs to work with new structure
Refactor any hooks into methods as needed.

Nitpick comment: is it possible to make a hard refresh work when on a module route? For example, when I'm on the treasury page, and refresh, I'm kicked back to the DAO's home page.
Browser metadata
Path:      /#/daos/0xcAa51de500b1b3ECA13529747F54F1f63c49C7de/modules/0x319C00f6f3a9fe6E3700D5bf23dF35B17babC5D7
Browser:   Chrome <IP_ADDRESS> on Mac OS 10.15.7
Viewport:  1440 x 796 @2x
Language:  en-US
Cookies:   Enabled

Open in BrowserStack

Open Deploy Preview · Mark as Resolved

Nitpick comment: is it possible to "save state" for specific module data, after they've been loaded initially?
(I'm guessing "no", as a result of wrapping modules themselves in their own data providers)
For example, loading the Treasury shows an "empty" treasury for a second, while the tokens load, then shows the tokens. After navigating back to the DAO Home page, and back into the Treasury, it needs to re-load the Treasury's tokens.
Browser metadata
Path:      /#/daos/0xcAa51de500b1b3ECA13529747F54F1f63c49C7de/modules/0x319C00f6f3a9fe6E3700D5bf23dF35B17babC5D7
Browser:   Chrome <IP_ADDRESS> on Mac OS 10.15.7
Viewport:  1440 x 796 @2x
Language:  en-US
Cookies:   Enabled

Open in BrowserStack

Open Deploy Preview · Mark as Resolved

Nitpick comment: is it possible to make a hard refresh work when on a module route? For example, when I'm on the treasury page, and refresh, I'm kicked back to the DAO's home page.

I'm I think I can adjust the Module Controller to 'wait' for the app to load before deciding to kick back. I'll look into this

Nitpick comment: is it possible to "save state" for specific module data, after they've been loaded initially?
(I'm guessing "no", as a result of wrapping modules themselves in their own data providers)
For example, loading the Treasury shows an "empty" treasury for a second, while the tokens load, then shows the tokens. After navigating back to the DAO Home page, and back into the Treasury, it needs to re-load the Treasury's tokens.

Right now you are right each time you visit the page it would refresh as the module state is reread. I'll think on this for a while. I think I can make this work. If I did something like I did with the DAO controller and Fractal Provider.
Moving the data back to the top (provider wraps the app) and the data is populated through dispatch lower.

Nitpick comment: is it possible to make a hard refresh work when on a module route? For example, when I'm on the treasury page, and refresh, I'm kicked back to the DAO's home page.

This has been fixed.

On my end, a hard refresh is still sending me back to the /#/daos route

Sorry I went ahead and pushed the cleanup commit here. and removed the approvals. Thought it would be easier. Last commit removes daoData folder and moves last bit of hooks and types out and updating the imports.


[FLINK-32849][runtime][JUnit5 Migration] The resourcemanager package of flink-runtime module

What is the purpose of the change
[runtime][JUnit5 Migration] The resourcemanager package of flink-runtime module
Brief change log
[runtime][JUnit5 Migration] The resourcemanager package of flink-runtime module
Verifying this change
This change is already covered by existing tests, such as (please describe tests).
Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): (yes / no)
The public API, i.e., is any changed class annotated with @Public(Evolving): (yes / no)
The serializers: (yes / no / don't know)
The runtime per-record code paths (performance sensitive): (yes / no / don't know)
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (yes / no / don't know)
The S3 file system connector: (yes / no / don't know)

Documentation

Does this pull request introduce a new feature? (yes / no)
If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

Hi， Thanks so much for @1996fanrui your review and patience.
I updated it based on your comments.
And I triggered about 4 times ci-flink-bot by modifying commit(--amend). Its succeeded final.
PTAL in your free time~  :)


Ytd and collapse legs

Year to date didn't include today
After showing legs in journal, they couldn't be collapsed again (regression from 7f16559a)

:+1:

Thanks! Works so far, but the Journal Leg toggling still has a bug: When showing all legs ("Show legs") and then clicking on a transaction, the legs will be shown twice.

Good catch, I missed that one.
It was pretty much the same issue (missing parent-hash)

Thanks, @corani, works now perfectly!


Nan loss and Some important things for training

When I run this repo，I got this issue
/home/lxt/study/face-py-faster-rcnn/tools/../lib/fast_rcnn/bbox_transform.py:50: RuntimeWarning: overflow encountered in exp
pred_w = np.exp(dw) * widths[:, np.newaxis]
/home/lxt/study/face-py-faster-rcnn/tools/../lib/fast_rcnn/bbox_transform.py:51: RuntimeWarning: overflow encountered in exp
pred_h = np.exp(dh) * heights[:, np.newaxis]
I0603 15:44:13.525365 30407 solver.cpp:218] Iteration 20 (2.86135 iter/s, 6.98971s/20 iters), loss = nan
I0603 15:44:13.525403 30407 solver.cpp:237]     Train net output #0: accuracy = 1
I0603 15:44:13.525410 30407 solver.cpp:237]     Train net output #1: loss_bbox = nan (* 1 = nan loss)
I0603 15:44:13.525416 30407 solver.cpp:237]     Train net output #2: loss_cls = 87.3365 (* 1 = 87.3365 loss)
I0603 15:44:13.525421 30407 solver.cpp:237]     Train net output #3: rpn_cls_loss = 0.683836 (* 1 = 0.683836 loss)
I0603 15:44:13.525427 30407 solver.cpp:237]     Train net output #4: rpn_loss_bbox = 0.0186136 (* 1 = 0.0186136 loss)
I0603 15:44:13.525432 30407 sgd_solver.cpp:105] Iteration 20, lr = 0.001
I0603 15:44:20.724417 30407 solver.cpp:218] Iteration 40 (2.77823 iter/s, 7.19884s/20 iters), loss = nan

Also Please change your readme.md for saving others time


Conventional commits

af9c706 Close #8
dc91f66 no link in subject
36ed635 no link in subject

@ngonzalezpazFC Please review

@ngonzalezpazFC Please review

LGTM


rendering 2 toasts

had to install 1.2.1 instead of 1.2.2, because a toast always got rendered twice. checked with console.log() if the parent function got called twice, but no, it didnt. Also there is a console.log in index.mjs file which prints undefined unnessesary often.
Not able to provide any reproducable code now, but i like this package a lot. So this are my 2 cents. Thanks for your work.

Detected the issue.
It doesnt has anything to do with the version. If i put all those buttons as type="submit", the toast gets called 2 or 3 times. depends on the amount of buttons with type submit. Even tho..! the submit handler gets called only one time and the other buttons are obviosly not rendered because of the ternary conditions.
So i will fix this by not using the type="submit" functionality, even it would be better ux

Fixed it to work with type="submit" by using only one button with type submit and handling the function call in the onSubmit callback of the form.
Below an image of the console.logs

Please upgrade to the newest version to get rid of the console logs.

I'm facing this issue as well. Two toasts are getting rendered even when I am calling toast.error only once.

did you update the newest version?


Empty Toast Shown on the App Launch

After setting toast:true and initialising the framework with the options, the home page of the app shows an empty toast at the bottom.

Can you please provide source code of how you initialized the framework? Can you check the console if there are any errors? It's hard to say w/o any clues.


Fix documentation for Agent::init

Clarify and possibly rename two usages of "level" throughout Agent

@dianarg @cmcantalupo can someone elaborate on this issue?


Add husky and pre-push scripts

We should add pre-push scripts that will check staged files for error. We should lint scss, js and check php files using phpcs.
An example of php linting script:
#!/usr/bin/env bash

# pre-commit.sh
#
# This hook is ran every time a commit is attempted. For commit to pass, the minimum requirement is that
# WPCS checks pass.

PROJECT=`php -r "echo dirname(realpath('$0'));"`
STAGED_FILES_CMD=`git diff --cached --name-only --diff-filter=ACMR HEAD | grep \\\\.php`

#Determine if a file list is passed
if [ "$#" -eq 1 ]
then
  oIFS=$IFS
  IFS='
  '
  SFILES="$1"
  IFS=$oIFS
fi
SFILES=${SFILES:-$STAGED_FILES_CMD}

echo "-----------------"
echo "Checking PHP Lint"
echo "-----------------"
for FILE in $SFILES
do
  php -l -d display_errors=0 $PROJECT/$FILE
  if [ $? != 0 ]
  then
    echo "Fix the error before commit."
    exit 1
  fi
  FILES="$FILES $PROJECT/$FILE"
done

if [ "$FILES" != "" ]
then
  echo "--------------------"
  echo "Running Code Sniffer"
  echo "--------------------"
  ./vendor/bin/phpcs --standard=Infinum --encoding=utf-8 -p --parallel=4 --colors $FILES
  if [ $? != 0 ]
  then
    echo "Possible warnings and errors found."
    exit 1
  fi
fi

exit $?

we should test this and add js and scss linting. lets schedule this for version 3.1

Husky is added in the #91 PR, I tested it in another project that is using the boilerplate and all the linting is working (on mac) :)


Protect status bar appearance

Setting an explicit border radius for all corners makes it less likely site stylesheets will override it.
For example, the site http://www.example.com has div { border-radius: 1em } in their stylesheet, which affects Minimal Status Bar because it is only explicit about the border radius for one corner.

Thanks for the PR.

released in v1.13


Unable to Create a Role in The Secondary User Store Through SCIM 2.0 API

Describe the issue:
When a request to create a role in secondary user store is sent, it sends a response with 201 status code showing that the new role is created.
Request:
curl --location --request POST 'https://localhost:9443/scim2/Roles' \ --header 'Content-Type: application/json' \ --header 'Authorization: Basic YWRtaW46YWRtaW4=' \ --data-raw '{     "displayName": "POSTGRESQL/pm",     "schemas": [         "urn:ietf:params:scim:schemas:extension:2.0:Role"     ] }'
Response:

Although the response says that it has created a new role with displayName="POSTGRESQL/pm", it doesn't create a new row in the "um_role" table in the secondary user store database. Note that my secondary US domain name is POSTGRESQL.
The newly created role is shown in the management console, but when I try to delete it through management console, it shows the following error.

How to reproduce:

Create a new role in secondary user store via SCIM 2.0 API
Check whether it's store in the "um_role" table in the secondary user store
Check whether the newly created role is shown in the management console
Delete the new role through management console

Expected behavior:
The newly created role should be appeared as a new row in the secondary user store but it's not saved in the secondary user store.
If the domain (user store) is invalid, it should prevent creating a new role via SCIM 2.0 API and it should not be listed in the management console.
Environment information:

Product Version: IS 6.0.0-m1
OS: Mac OS 12.3.1
Database: PostgreSQL1 14
Userstore: JDBC with POstgreSQL 14 DB

There is no valid use case to create Roles in user stores. It will be grouped that you can create in userstores. In this case, another internal role will be created in IS with the given name. However, we need to check the error. @ravinduramesh can you please try the same flow using scim2/Groups API? FYI @aaujayasena

@ashensw it works well with scim2/groups api endpoints.
I think this misleading role creation flow can be avoided/fixed by just returning a response with an error when someone tries to create a role in a secondary user store through scim2/roles api endpoints.

@ashensw it works well with scim2/groups api endpoints.
I think this misleading role creation flow can be avoided/fixed by just returning a response with an error when someone tries to create a role in a secondary user store through scim2/roles api endpoints.

Great! Yes but the role endpoint does not have context about userstores. For Roles endpoint, it's another role name and it will consider domain separator (/) as another character. But it cannot be broken. If we don't allow that character in the role name we need to validate it properly at API level.


cocreate, flexography

This pull request was generated by the 'mq' tool
[test]
flake rate: 0.4
logical conflict every: 100
sleep for: 300s
[pullrequest]
requests per hour: 30
deps=[c,f]

/trunk merge


Make Columns and Tables Optional

This allows for Columns to be declared optional via the
ClientDatabaseModel.
Fields may be marked as optional within the struct tag.
This allows a connection when a mapped table (and columns that reference it) aren't available at runtime to support upgrades.
To allow a user to easily see which Tables and Columns are supported two
new APIs were added.
IsTableSupported and IsColumnSupported
Fixes: #235

Note: I want to add some more tests before we merge this.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

47 of 78 (60.26%)  changed or added relevant lines in 8 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.2%) to 72.652%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

cache/cache.go
0
3
0.0%

mapper/info.go
30
34
88.24%

client/client.go
0
24
0.0%

Totals

Change from base Build<PHONE_NUMBER>:
-0.2%

Covered Lines:
4192

Relevant Lines:
5770

💛  - Coveralls

@amorenoz

How is the user expected to use this mechanism? Are you planning in adding more logic to modelgen so it can add this tag automatically? (e.g: give it two schemas and insert the tag in those fields that are not the same in both)
If not, are we expecting the user to modify it by hand before compiling?

Having modelgen do this would be the eventual goal I think, but for now I'm recommending documenting this option and asking users to do it by hand.

Preventive: Add a field to the "optional" map in the to the ClientDBModel.
I don't see this having any effect at all. It's not checked anywhere AFAICS

It should be checked when we do schema validation - we skip any fields that are omitted due to not being supported by the runtime schema. The contract with the user is - "if you use this feature, it's on you to guard against sending bad transactions to ovsdb-server" - as such we don't need to check the list of tables omitted tables again, although we could and provide a better error message I suppose :laughing: but that could happen in a follow up PR.

Note: I want to add some more tests before we merge this.

Beyond func TestValidateOptional(t *testing.T) { ?


[admin] merge to live

@TCourtneyOwen This PR removes the blank line. Should be visible on live docs momentarily! Thanks for the fix.


[Release-2.1] Fix declaration emit when using type parameters

Fix #11855

@yuit can you please port this to master as well.


Too slow to use (suspends main thread)

I simply added this view in XML in a Fragment, and even on a Pixel is causes a noticeable jerk in the UI as the view is inflated. This is especially noticeable in a Fragment in a ViewPager. The built-in CalendarView, however is seamless, it does not block the UI thread at all.

Any work around?

Hey, any updates on that issue?


[Bug] Modify dynamic binding evaluation to support escaped characters without double escapes

Escape characters (like \t, \n etc) are currently not read as such during evaluation. They need to be written as a double escaped \\n for the character to make sense while evaluation. This is non-intuitive since the editor space is not represented as a string literal.

@hetunandu @mohanarpit @trishaanand re-opening this as I reported today

Reopening this as it's still happening. We're back and forth on this issue a lot!
@hetunandu @trishaanand what's happening here?

@Nikhil-Nandagopal : I closed this because I assumed it was reopened due to #5328

After speaking to @hetunandu, turns out it was my mistake closing it the first time since this scenario was only covered in the smart substitution/prepared statement scenarios.

@hetunandu is this hard to support?

This issue is causing linting to break when escape characters are used
https://github.com/appsmithorg/appsmith/issues/6900

This would be a breaking change.
Code editor today persists "\n" as "\n" and "\n" as "\\n" which is correct.
We unescape the JS string we get for evaluations, which translates "\n" to "actual new line" and "\\n" to "\n".
The solution here is to not unescape the JS string during evaluations.
To ensure backward compatibility, we can translate all the existing usage of "\n" to "\n" but that would mean that a user will never be able to actually output the string '\n' (by typing \n), which although rare, is another bug.

@mohanarpit @arunvjn this seems risky. Can we do this in a safer way? Should we not allow auto-update for this version?

Saptami Ram is inviting you to a scheduled Zoom meeting.
Topic: [appsmithorg/appsmith] [Bug] Modify dynamic binding evaluation to
support escaped characters without double escapes (#2862)
Time: Sep 20, 2021 04:30 PM India
Join Zoom Meeting
https://zoom.us/j/9333609750?pwd=ZkZUVzV6NnAzd25XVzR2N09iUFFSQT09
Meeting ID:<PHONE_NUMBER>
Passcode: 86ZJpd

Reopening this issue since freshly cloned apps are unable to handle single backslash on new line character. The others, \b \t work as expected.

@trishaanand can we fix this issue with cloned apps? It seems to work fine for other apps but this inconsistency is bad

Let's test that this works for export/import flow before closing.
cc: @ramsaptami @abhvsn

Let's test that this works for export/import flow before closing.
cc: @ramsaptami @abhvsn

I have tested this for duplicate, fork and import-export flows and works as expected.


[Solved] Using File Open API on ImGui

I have seen your reply in an issue of "file system dialogs" from ImGui, https://github.com/ocornut/imgui/issues/88#issuecomment-431040283.
Your API's features are so nice that I am tring to use your File Open API on ImGui(currently, my project is based on glfw-opengl3-demo.)
But an error that freeze the graphic interface happened and the file open dialog could not appear if I pasted your example codes
(Example 3: asynchronous file open) https://github.com/samhocevar/portable-file-dialogs/issues/3#issue-371472393.
The console were printing messages (as you can see it below), the others were stopped though.

It is demo codes
static void ShowExampleMenuFile()
{
    ImGui::MenuItem("(dummy menu)", NULL, false, false);
    if (ImGui::MenuItem("New")) {}
    if (ImGui::MenuItem("Open", "Ctrl+O"))
    {
        // File open dialog
        auto dialog = pfd::open_file("Select file to open");

        // Do something while waiting for user input
        while (!dialog.ready(1000))
        std::cout << "Waited 1 second for user input...\n";

        // Act depending on the user choice
        std::cout << "Number of selected files: " << dialog.result().size() << "\n";
    }
    if (ImGui::BeginMenu("Open Recent"))
    {

Would you let me know how the File Open API should be applied to ImGui if you have tried?
Thank you in advance.

The problem is that your while loop waits indefinitely for user action. You need instead to return as soon as possible and check again at each iteration of the main loop.
Here is one way it could work. You need to ensure that the user cannot click on Open File if there is already a dialog opened. I do this using the Disabled item flag, which requires #include "imgui_internal.h" (making these flags available to the user are still in Dear ImGui’s TODO):
static pfd::open_file *open_file;

ImGui::PushItemFlag(ImGuiItemFlags_Disabled, open_file != nullptr);
if (ImGui::Button("Open File"))
	open_file = new pfd::open_file("Choose file", "C:\\");
if (open_file && open_file->ready())
{
	auto result = open_file->result();
	if (result.size())
		std::cout << "Opened file " << result[0] << "\n";
	delete open_file;
	open_file = nullptr;
}
ImGui::PopItemFlag();

Wow it has been solved! Thanks a lot @samhocevar!


Licensing metadata is very long

Hi platformdirs folk,
I noticed on https://pypi.org/project/platformdirs/ that the licensing metadata section of the package is including the full license. I'm not sure if that was intended; it flagged for me on a script that was expecting something more like "MIT" or "MIT License".

Broke in https://github.com/platformdirs/platformdirs/pull/92 (see https://pypi.org/project/platformdirs/2.5.2/)
Hatchling produces a less verbose license section on PyPI because until they support PEP 639 the rendering is not what one may think. It's actually "License: {license_classifier.split(' :: ')[-1]} ({license['text']})"

@ofek so how do I fix it? 😆

go back to license = "MIT"

      ValueError: invalid pyproject.toml config: `project.license`.
      configuration error: `project.license` must be valid exactly by one definition (2 matches found):

          - keys:
              'file': {type: string}
            required: ['file']
          - keys:
              'text': {type: string}
            required: ['text']

seems pypy setup triggers setuptools that runs its check automatically and fails with this 🤦 once making your suggested change. Ah setuptools what a mess!

when is setuptools triggered? pypy works fine in other ci

See https://github.com/platformdirs/platformdirs/actions/runs/3417581616/jobs/5688883308

Fixed now


Download fails on update to Luma 7.1 on New 3DS XL on 11.3

When trying to update from Luma 7.0.4-725a8257 (dev) on Luma Updater v1.4.2 to Luma 7.1 the download fails and prompts to press START to exit.

7.1 only works on boot9strap consoles. If you have a9lh, follow this to update to boot9strap. 7.0.5 is the last update for non-boot9strap consoles.

Ok, thanks!


fixes #1: get unnecessary dependencies out of package.json

I found them hanging around in here, so I killed them.

@geoffp thanks for this. https://github.com/pattern-lab/patternlab-node/issues/350 is meant for patternlab-node core too. I can remove them too, just whoever gets it first :+1:


Clone linked list with random pointer

Also @ming-tsai,
Highly appreciated if you make me a contributor to this DSA repository, it saves lots of time. Btw I love to review code also.

While I agree with @sridhar-5 , I'd also like to mention the fact that the program input is hardcoded and of size 5. Would be better if the user had an option to choose size of list and input their own data.

Hi @cyberwizard1001 ! I code will also work perfectly if the user gives its own input for any length of the linked list. But if I need to write the code for user input I can make the changes.

Sorry, please solve the merge conflict

Can you make me a contributor for the DSA repository please?

Sorry, please solve the merge conflict

Conflict resolved. @ming-tsai


Compare expenses made with lodging against official prices of rooms

Filtering quota's dataset by records with value 'Lodging, except for congressperson from Distrito Federal' in the column subquota_description will return many expenses made with hotels. We could match the value in the receipt against publicly available (through Booking.com, for instance) range of prices.

Holidays and events should be considered as they usually significantly the price of rooms.

Does anybody have an idea how to proceed with this scraping? I mean, in addition to what is already being done by @Lrcezimbra at #100.
I had a look into booking.com but couldn't find any suitable API. I also tried decolar.com (they do have a public and free API [1]), but their terms of usage doesn't seem to allow the kind of data scraping we need (I don't even know why I thought it would :smile:).
[1] http://dev.despegar.com/howto/hotels

Closed accidentally by unrelated commit from Rosie/Jarbas repos.


NSObject return from Objective-C/Swift function has undefined properties

Problem: NSObject return from Objective-C/Swift function has undefined properties until  we check it with  instanceof
wonder if I am missing something or it is a known issue with NativeScript

Here is an example

swift code
@objc public class HeartRateData: NSObject  {
    public let heartRate:UInt
    public init(heartRate:UInt) {
        self.heartRate  = heartRate
    }
}

@objc(TestClass)
public class TestClass: NSObject {
    public func getSomething() -> HeartRateData{
        return HeartRateData(heartRate: 55)
    }
}

generated Objective-C
SWIFT_CLASS("_TtC13TestClass13HeartRateData")
@interface HeartRateData
@property (nonatomic, readonly) NSUInteger heartRate;
- (nonnull instancetype)initWithHeartRate:(NSUInteger)heartRate  OBJC_DESIGNATED_INITIALIZER;
@end

SWIFT_CLASS_NAMED("TestClass")
- (nonnull instancetype)init OBJC_DESIGNATED_INITIALIZER;
- (HeartRateData * _Nonnull)getSomething;
@end

JavaScript/TypeScript
var testClass = new TestClass();
var data = testClass.getSomething()
// why undefined?
console.log(data.heartRate) // undefined
// magic happens here...
console.log((data instanceof HeartRateData)) // true
// working as expected
console.log(data.heartRate) // 55

@xmlking Help me with reproducing the issue. I guess the Swift code is part of a framework. Can you upload a pre-build version of the framework or give me a link where I can download it. If it is easier, you can upload an example app.

The Swift code base is at https://github.com/xmlking/MicrosoftBand
if I don't do instanceof workaround , I will receiving nulls for short time from my Observable.
NativeScript code that is using above  Swift Package.
https://github.com/xmlking/nativescript-ngx-microsoftband/blob/446ed7bc5e13cc3d0973f55df70143416f35a85c/src/app/services/microsoftband.service.ios.ts#L116

I just ran into this and was wondering if there was more information on how to avoid it?

Hi, Jacob. Thank you for bringing our attention to this issue. We'll investigate it and write back soon.

The reason for the undefined properties of the Swift object is that due to a bug in the runtime, the metadata for the class cannot be found correctly and it is being casted to its base (in the original report HeartRateData becomes NSObject).
If you happen to instantiate the class' constructor function before instances of it are returned to JS, this constructor function is cached and it can be correctly discovered, thus alleviating the issue. We will think for a fix to this issue but in the meantime there are two workarounds that we can suggest:

Instantiate the types you will be using early in your app/plugin's initialization logic (e.g. const HeartRateDataProto = global.HeartRateData). This should be enough to make it work.
Make the Swift classes used in JS have a human-readable assigned Objective-C name with the @objc attribute i.e. @objc public class HeartRateData becoming @objc(<NAME>) public class HeartRateData would do the trick. <NAME> can be any suitable Obj-C identifier (e.g. HeartRateData, HeartRateDataSwift, MyLib_HeartRateData, ...)

Thank you so much for the fast reply! Option 1 seems to work great, thanks so much for the reliable workaround. I'm surprised to hear that providing the Name in @objc results in different behavior, but then again I'm still very green in the ObjC/Swift area.

I'm surprised to hear that providing the Name in @objc results in different behavior, but then again I'm still very green in the ObjC/Swift area.

This is due to some of the specifics causing the issue. By default, the Swift class name becomes the mangled version of SwiftModule.ClassName and that's what causes metadata discovery failure. When you explicitly set a name in Objective-C then the JS name matches the native one and the runtime manages to discover it.


shared: Match literal emoji in typeahead.

This PR implements checking for a literal emoji match in emoji
typeaheads. In other words, if you paste or type panda face into an
emoji typeahead, panda face should be presented as an option to choose
from.
This behavior is currently present in the mobile app, adding it to
shared will enable both platforms to utilize this logic.

Great, merged, and opened #21714 for the follow-up. Thanks @Fingel!


If I longpress the select key，the onSelected event will trigger many times

if i want to  trigger the onSelected event until the select key up，because i want to has a press state to give a press style。

how to do it

Sorry, won’t fix this.