seekr_graph with cross species classification

Hi Jessime,
I'm trying to use seekr_graph to find communities of lncRNAs within our transcriptome assembly of a non-model organism and compare it across species to GENCODE mouse ncRNAs.  I extracted the lncRNAs, normalized against the total StringTie assembly and repeated the process with GENCODE mouse to perform the pearson correlation.
Running the seekr_graph module consistently errors out though. seekr_graph with self vs self correlations functions perfectly but not using cross-species correlations.
Traceback (most recent call last):   File "/seekr/virtualenv/bin/seekr_graph", line 10, in <module>     sys.exit(console_graph())   File "/seekr/virtualenv/lib/python3.7/site-packages/seekr/console_scripts.py", line 500, in console_graph     float(args.resolution), int(args.n_comms), args.seed)   File "/seekr/virtualenv/lib/python3.7/site-packages/seekr/console_scripts.py", line 475, in _run_graph     maker.make_gml_csv_files()   File "/seekr/virtualenv/lib/python3.7/site-packages/seekr/graph.py", line 175, in make_gml_csv_files     self.build()   File "/seekr/virtualenv/lib/python3.7/site-packages/seekr/graph.py", line 166, in build     self.graph = networkx.from_pandas_adjacency(self.adj)   File "/seekr/virtualenv/lib/python3.7/site-packages/networkx/convert_matrix.py", line 185, in from_pandas_adjacency     raise nx.NetworkXError("Columns must match Indices.", msg % missing)
Thank you!

Hey, this is a great question. The answer is going to depend a on what experiment you're trying to perform, exactly.
seekr_graph expects a square adjacency matrix where the names of the rows match the names of the columns. Lets say that your species has n transcripts, and mouse has m transcripts. You would need to pass the whole, (n+m)^2 matrix, which would be asking the question, "What communities form if we throw all transcripts together and run community detection?"
Is this what you want to do? There isn't a neat tool to do it from the command line at this point, but I can write up something for you in Python.
Another approach would be to do the self vs. self communities for both, and then compare the communities afterwards, using something like normalized information. We've successfully used this approach in the past.

Hi Jessime,
We performed an RNA-Seq experiment for our non-model species and and are interested in the population of lncRNAs in our transcriptome assembly that are differentially expressed in two different conditions. I'd like to use seekr to identify non-coding genes that are in both our assembly and a well-annotated assembly like GENCODE human or mouse as a potential clue for their function in our organism.
I suppose the question being asked is, given a subset of lncRNAs that are DE: n(DE), which ones are similar to the total population of mouse lncRNAs, m. Is there an easy way to compare across species? I suppose it makes sense to perform self vs. self for both transcriptomes than compare between them. Would you recommend this strategy or something else entirely?
Thanks!

I still think there are several ways you could go about this. I'll show you how I would directly answer:

given a subset of lncRNAs that are DE: n(DE), which ones are similar to the total population of mouse lncRNAs, m?

as an example. Let's assume you've just finished running:
$ seekr_pearson DE_counts.csv mouse_counts.csv -o adj.csv

You could open python and run:
import pandas as pd

df = pd.read_csv('adj.csv', index_col=0)
mean_similarity = df.mean(axis=1).sort_values(ascending=False)
mean_similarity

This will give you a sorted list of how similar each DE transcript is to the mouse transcriptome, on average.
Alternatively, if the number of DE transcripts is fairly small (hundreds?) it might also make sense to see which DE transcripts fall into which mouse communities.

I don't think the communities are directly relevant to our work. We are more interested in identifying specifically conserved lncRNA in our assembly in relation to a well annotated species like mouse i.e. the likelihood that transcript n is Xist based on pearson correlation.  Does seekr support something like a one to one comparison between  two distinct lncRNAs? Would it be feasible to find the best cross-species pair that have the best pearson correlation or am I mistaken?

It sounds like all the information you want is just in adj.csv, right? It's just a matter of querying it to answer your question. Right now, the command line tools don't have a way to do this, but it's straightforward in python.
Using the example:

the likelihood that transcript n is Xist based on pearson correlation.

import pandas as pd

df = pd.read_csv('adj.csv', index_col=0)
most_xist_like = df['Xist-201'].sort_values(ascending=False)

Assuming you've labeled one of the mouse rows as Xist-201 (you can replace Xist-201 with the whole fasta header, another lncRNA, or whatever row you need), you'll have a sorted list of how similar each of your non-model transcripts are to Xist.

Hi Jessime!
Thanks for this. It was a great help. One thing I noticed was that there was a low degree of correlation between lncRNAs between species; even those that were evolutionarily conserved. This seems counterintuitive to me: wouldn't conserved lncRNAs have a high correlation due to having similar k-mer composition? Or am I just missing something?

Great, glad it's working out!
When you use 6mers, correlations are always pretty low, because of the large number of kmers in each profile. The math works out that as the length of two arrays approaches infinity, the correlation value between the arrays approaches 0, no matter the contents of the arrays. But even with the low values, we found that we could detect a significantly higher set of correlations between known homologs than other random comparisons.
I should say though that if your sole goal is to find evolutionary homologs, there are better tools than SEEKR, even for lncRNAs. SEEKR is probably better for detecting functional similarity.

hello.
I have a similar question. I want to find cross-species homologs of lncRNA and protein-coding genes. I just put all of them in a same fasta file. Can it work?
It seem like in your manuscript, humans and mouse are separately run seekr_kmer_counts. Then using seekr_compare compare them similarity.
which one is more suitable?  Or they are the different routes to the same goal?

Hey @huanananan,
The difference between putting the species in the same file versus different files isn't going to be huge, but it might be significant. The difference between the two is the baseline normalization will change. If you have the time, I'd recommend trying both and seeing which works better for you.
If you only want to use one approach, I'd suggest keeping the files separate. This does two things:

The normalization for each species is then relative only to itself, which I think makes more sense.
The output file size is going to be much smaller this way.

Thank you. Very helpful.

Of course. I'm going to go ahead and close this issue since it's pretty old. But feel free to open new ones if you run into other questions/issues with seekr!


wrong screenshot of "Troubleshooting jobs on Elastic MapReduce" wiki page

On Diagnosing an ETL job failure wiki, the screenshot for bootstraping failure and way-through failure are the same.

Thanks for raising @luochenhuan !

@alexanderdean, this was fixed earlier, please, close. When updating the link didn't realise that there was actually a ticket opened for it.

Great, closing


Email invite links not working

I have sent some invites, but the links don't work. This is the raw generated email code:
http://foooo.meteor.com/
andrew [/users/andrew] invited you to join Foooo

Join Foooo [/register]

Change your notifications settings =
[http://site.meteor.com/account]

As you can see, the two most important links are missing the website url.
Any ideas how I can fix this, or is it user error?

+1. There's a PR for that here: https://github.com/TelescopeJS/Telescope/pull/1242.

Hey, folks! Telescope has changed a lot in the past months, and there is a new React-based version, Telescope Nova (intro video). If you know Crater, it runs on the latest version.
The Legacy version is no longer supported. I'm closing this issue (which has been fixed on legacy & nova).
I invite you to check the new version, it's awesome!
Happy hacking! 🚀
Ps: by the way, don't hesitate to join the Slack to ask any question, you are welcome! 👍


travis build being triggered continuously/in a loop with flux-sync tag

We have automated deployment of a kubernetes manifest using fluxctl. Changes to the yaml are resulting in successful deployments - but we see travis build being triggered continuously/in a loop with flux-sync tag - why is this happening, how do we avoid this

Flux uses a tag to mark the revision up to which it has synchronised the cluster. Whenever it synchronises new commits, it changes where the tag points and pushes that to the upstream (github) repo.
My guess is that TravisCI is noticing that a tag moved in the repo, and triggering a rebuild. To be honest we hadn't really anticipated that! We use CircleCI, which builds branches but doesn't appear to build tags unless you explicitly put config in to do that.
https://docs.travis-ci.com/user/customizing-the-build/#Building-Specific-Branches explains how to control which branches and tags trigger builds. I think you want, in your .travis.yml file, something like
branches:
  except:
  - flux-sync

(you may need to change the flux-sync value, depending on what fluxd has been given as arguments -- that's the default value).

I think this deserves a bit of documentation -- a section in faqs, or somewhere in site/ anyway.

You can use the --git-ci-skip (and optionally, --git-ci-skip-message) to tell flux to label all its commits with [ci skip] (or your own extra text). See #1011


Getting Error On Using Tool With HERE NAVSTREETS Data

We are using the latest version of tool (<IP_ADDRESS>) on top of ArcGIS 10.4.1 (built <IP_ADDRESS>86) to process Street data from HERE NAVSTREETS. but after spending couple of hours we are getting this error

Exception from HRESULT: 0x800422F8
at ESRI.ArcGIS.Geodatabase.IDatasetContainer2.CreateDataset(IDEDataset DataElement)
at GPProcessVendorDataFunctions.ProcessNavStreetsDataFunction.CreateAndBuildNetworkDataset(String outputFileGdbPath, Double fgdbVersion, String fdsName, String ndsName, Boolean createNetworkAttributesInMetric, Boolean createArcGISOnlineNetworkAttributes, String timeZoneIDBaseFieldName, Boolean directedTimeZoneIDFields, String commonTimeZone, Boolean usesHistoricalTraffic, ITrafficFeedLocation trafficFeedLocation, Boolean usesTransport) in C:\xxx\Master\ProcessNavStreetsDataFunction.cs:line 6997
at GPProcessVendorDataFunctions.ProcessNavStreetsDataFunction.Execute(IArray paramvalues, ITrackCancel trackcancel, IGPEnvironmentManager envMgr, IGPMessages messages) in C:\xxx\Master\ProcessNavStreetsDataFunction.cs:line 3146
Failed to execute (ProcessNavStreetsData).

Basically, I am using the same tool except of disabling two Fields "EXPR_LANE","CARPOOLRD" from the StreetsFieldNames array and also commenting out
//evalNetAttr = CreateAvoidNetworkAttribute("Avoid Express Lanes", "[EXPR_LANE] = \"Y\"",  true, fgdbVersion, edgeNetworkSource, -1.0);
//attributeArray.Add(evalNetAttr);
//evalNetAttr = CreateAvoidNetworkAttribute("Avoid Carpool Roads", "[CARPOOLRD] = \"Y\"", true, fgdbVersion, edgeNetworkSource, -1.0);
//attributeArray.Add(evalNetAttr);

since I was getting these errors when trying to run the tool

"Field named EXPR_LANE not found"
"Field named CARPOOLRD not found"

the Errors however refers to:
CreateAndBuildNetworkDataset(outputFileGdbPath, fgdbVersion, fdsName, ndsName, createNetworkAttributesInMetric,
                                             createArcGISOnlineNetworkAttributes, timeZoneIDBaseFieldName, directedTimeZoneIDFields, commonTimeZone,
                                             usesHistoricalTraffic, trafficFeedLocation, usesTransport);

and
var networkDataset = (INetworkDataset)datasetContainer2.CreateDataset(deDataset);
Can you please let me know what causing the issue and how Can I fix this?Thanks

@behClevest,
HRESULT: 0x800422F8 if you convert it to a signed 32-bit integer represents -2147212552.  If you look up this number in the list of fdoError constants at http://desktop.arcgis.com/en/arcobjects/latest/net/webframe.htm#fdoError.htm, you'll see that the error is FDO_E_NETWORK_TRAVEL_MODE_RESTRICTION_ATTRIBUTE_NOT_FOUND.
Many of the network attributes created in the tool are also used in travel modes also created in the tool.  Since you're commenting out the creation of these network attributes, you must also comment out the lines that add these network attributes you've excluded to various travel modes.

Thanks,
After commenting out all other associated fields from restrictionsArray we were able to run the tool successfully

I'm glad to hear that you're now able to get this working.


how to cross compile MXML for  STM32 MCU?

hi ,i want to use MXML in a board based on STM32 MCU without linux. Is there a way to cross comile MXML for this ?
Thanks for any help!

The configure script supports the usual GNU cross-compilation options (--host=xxx and --build=xxx). Beyond that you'd need to set the CC and AR environment variables to point to your cross-compiler and cross-archiver programs.


Invalid ancestor re-org test reports wrong invalid transaction

Describe the bug
The Invalid Ancestor Chain Re-Org, Invalid ReceiptsRoot, Invalid P8', Reveal using sync hive test currently fails due to an invalid block. However, it is not the correct invalid block.
The test sets up a valid chain (of length 15) using forkchoiceUpdated / getPayload / newPayload. It then creates an invalid chain in the background, with an invalid block. In this case the invalid block is block 8 (P8').
The simulation fails because we report SYNCING (as we download blocks one by one) until we eventually report the wrong block as INVALID:
INFO (Invalid Ancestor Chain Re-Org, Invalid ReceiptsRoot, Invalid P8', Reveal using sync): Secondary Node has correct block
>> (d2d82b78) {"jsonrpc":"2.0","id":47,"method":"engine_newPayloadV1","params":[{"parentHash":"0xfaf8c0fa94e9f2bdef86fb3774a185fe2ad53a35b144eaf6ebccc6ab7d980939","feeRecipient":"0x0000000000000000000000000000000000000000","stateRoot":"0x96dd2f9b51e6e62e5eb0a61989f84b14f9a5b4143d6e44f48a0784448bdb16f7","receiptsRoot":"0x5d721667f3f41c6e8a5e630b2c5f259855d2365513295c4fd0c5d05498ce6489","logsBloom":"0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000","prevRandao":"0xf0c12e098ed6a02bfe6a6f34b9f0fbf6a37b765b7a9d3f817341f88c25973e0a","blockNumber":"0xf","gasLimit":"0x3043ed","gasUsed":"0xa860","timestamp":"0x1243","extraData":"0x01","baseFeePerGas":"0x8545e76","blockHash":"0xb2efad206aa9c68d7f93720bf8ef8c1f1aeb2de6e95ae0a370d1cbd0e8047fe4","transactions":["0x02f86c0709843b9aca008506fc23ac00830124f89400000000000000000000000000000000000003160180c080a0bb476cdbc61448c8f3df3a60b239ece15aecef4b3e0e2ea83bb826e14d014286a056697ae094e913705310289f44afa0503f911444a5e766fc43cfafb3791eb3da"]}]}
<< (d2d82b78) {"jsonrpc":"2.0","result":{"status":"SYNCING","latestValidHash":null,"validationError":null},"id":47}
INFO (Invalid Ancestor Chain Re-Org, Invalid ReceiptsRoot, Invalid P8', Reveal using sync): Response from main client: {SYNCING <nil> <nil>}

... repeats

>> (d2d82b78) {"jsonrpc":"2.0","id":53,"method":"engine_newPayloadV1","params":[{"parentHash":"0xfaf8c0fa94e9f2bdef86fb3774a185fe2ad53a35b144eaf6ebccc6ab7d980939","feeRecipient":"0x0000000000000000000000000000000000000000","stateRoot":"0x96dd2f9b51e6e62e5eb0a61989f84b14f9a5b4143d6e44f48a0784448bdb16f7","receiptsRoot":"0x5d721667f3f41c6e8a5e630b2c5f259855d2365513295c4fd0c5d05498ce6489","logsBloom":"0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000","prevRandao":"0xf0c12e098ed6a02bfe6a6f34b9f0fbf6a37b765b7a9d3f817341f88c25973e0a","blockNumber":"0xf","gasLimit":"0x3043ed","gasUsed":"0xa860","timestamp":"0x1243","extraData":"0x01","baseFeePerGas":"0x8545e76","blockHash":"0xb2efad206aa9c68d7f93720bf8ef8c1f1aeb2de6e95ae0a370d1cbd0e8047fe4","transactions":["0x02f86c0709843b9aca008506fc23ac00830124f89400000000000000000000000000000000000003160180c080a0bb476cdbc61448c8f3df3a60b239ece15aecef4b3e0e2ea83bb826e14d014286a056697ae094e913705310289f44afa0503f911444a5e766fc43cfafb3791eb3da"]}]}
<< (d2d82b78) {"jsonrpc":"2.0","result":{"status":"SYNCING","latestValidHash":null,"validationError":null},"id":53}
INFO (Invalid Ancestor Chain Re-Org, Invalid ReceiptsRoot, Invalid P8', Reveal using sync): Response from main client: {SYNCING <nil> <nil>}
>> (d2d82b78) {"jsonrpc":"2.0","id":54,"method":"engine_forkchoiceUpdatedV1","params":[{"headBlockHash":"0xb2efad206aa9c68d7f93720bf8ef8c1f1aeb2de6e95ae0a370d1cbd0e8047fe4","safeBlockHash":"0xb2efad206aa9c68d7f93720bf8ef8c1f1aeb2de6e95ae0a370d1cbd0e8047fe4","finalizedBlockHash":"0x0000000000000000000000000000000000000000000000000000000000000000"},null]}
<< (d2d82b78) {"jsonrpc":"2.0","result":{"payloadStatus":{"status":"INVALID","latestValidHash":"0x5a949798b3aaa04604a28146a9a0ada452b4507e7d9884ad00844249395ca2d5","validationError":"links to previously rejected block"},"payloadId":null},"id":54}
INFO (Invalid Ancestor Chain Re-Org, Invalid ReceiptsRoot, Invalid P8', Reveal using sync): Response from main client fcu: {INVALID 0x5a949798b3aaa04604a28146a9a0ada452b4507e7d9884ad00844249395ca2d5 0x4005f51cd0}

... repeats as simulation waits

>> (d2d82b78) {"jsonrpc":"2.0","id":71,"method":"engine_newPayloadV1","params":[{"parentHash":"0xfaf8c0fa94e9f2bdef86fb3774a185fe2ad53a35b144eaf6ebccc6ab7d980939","feeRecipient":"0x0000000000000000000000000000000000000000","stateRoot":"0x96dd2f9b51e6e62e5eb0a61989f84b14f9a5b4143d6e44f48a0784448bdb16f7","receiptsRoot":"0x5d721667f3f41c6e8a5e630b2c5f259855d2365513295c4fd0c5d05498ce6489","logsBloom":"0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000","prevRandao":"0xf0c12e098ed6a02bfe6a6f34b9f0fbf6a37b765b7a9d3f817341f88c25973e0a","blockNumber":"0xf","gasLimit":"0x3043ed","gasUsed":"0xa860","timestamp":"0x1243","extraData":"0x01","baseFeePerGas":"0x8545e76","blockHash":"0xb2efad206aa9c68d7f93720bf8ef8c1f1aeb2de6e95ae0a370d1cbd0e8047fe4","transactions":["0x02f86c0709843b9aca008506fc23ac00830124f89400000000000000000000000000000000000003160180c080a0bb476cdbc61448c8f3df3a60b239ece15aecef4b3e0e2ea83bb826e14d014286a056697ae094e913705310289f44afa0503f911444a5e766fc43cfafb3791eb3da"]}]}
<< (d2d82b78) {"jsonrpc":"2.0","result":{"status":"SYNCING","latestValidHash":null,"validationError":null},"id":71}
INFO (Invalid Ancestor Chain Re-Org, Invalid ReceiptsRoot, Invalid P8', Reveal using sync): Response from main client: {SYNCING <nil> <nil>}
>> (d2d82b78) {"jsonrpc":"2.0","id":72,"method":"engine_forkchoiceUpdatedV1","params":[{"headBlockHash":"0xb2efad206aa9c68d7f93720bf8ef8c1f1aeb2de6e95ae0a370d1cbd0e8047fe4","safeBlockHash":"0xb2efad206aa9c68d7f93720bf8ef8c1f1aeb2de6e95ae0a370d1cbd0e8047fe4","finalizedBlockHash":"0x0000000000000000000000000000000000000000000000000000000000000000"},null]}
<< (d2d82b78) {"jsonrpc":"2.0","result":{"payloadStatus":{"status":"INVALID","latestValidHash":"0x5a949798b3aaa04604a28146a9a0ada452b4507e7d9884ad00844249395ca2d5","validationError":"links to previously rejected block"},"payloadId":null},"id":72}
INFO (Invalid Ancestor Chain Re-Org, Invalid ReceiptsRoot, Invalid P8', Reveal using sync): Response from main client fcu: {INVALID 0x5a949798b3aaa04604a28146a9a0ada452b4507e7d9884ad00844249395ca2d5 0x400680cb10}
FAIL (Invalid Ancestor Chain Re-Org, Invalid ReceiptsRoot, Invalid P8', Reveal using sync): Timeout waiting for main client to detect invalid chain

Reth reports that block 6, which the first valid block in the sidechain, and before the invalid block for that sidechain, is invalid:
2023-06-07T05:53:27.014018Z TRACE consensus::engine: start downloading full block. hash=0xce88b40d4b60053eb485e2143bbcd36e87f30d144192b5b8a617bbd1c5f0bde9
2023-06-07T05:53:27.015846Z TRACE consensus::engine: Downloaded full block hash=0xce88b40d4b60053eb485e2143bbcd36e87f30d144192b5b8a617bbd1c5f0bde9 number=8
2023-06-07T05:53:27.015921Z TRACE blockchain_tree: Inserting block hash=0xce88b40d4b60053eb485e2143bbcd36e87f30d144192b5b8a617bbd1c5f0bde9 number=8 parent_hash=0x07cecf1eb7bfee3a485f605e00bcafef25062bb74af3eb192d3296d57dfe3061
2023-06-07T05:53:27.015954Z  INFO try_insert_validated_block{block=(8, 0xce88b40d4b60053eb485e2143bbcd36e87f30d144192b5b8a617bbd1c5f0bde9)}: blockchain_tree: return=Ok(Disconnected { missing_parent: (7, 0x07cecf1eb7bfee3a485f605e00bcafef25062bb74af3eb192d3296d57dfe3061) })
2023-06-07T05:53:27.015961Z TRACE consensus::engine: start downloading full block. hash=0x07cecf1eb7bfee3a485f605e00bcafef25062bb74af3eb192d3296d57dfe3061
2023-06-07T05:53:27.025847Z TRACE consensus::engine: Downloaded full block hash=0x07cecf1eb7bfee3a485f605e00bcafef25062bb74af3eb192d3296d57dfe3061 number=7
2023-06-07T05:53:27.025915Z TRACE blockchain_tree: Inserting block hash=0x07cecf1eb7bfee3a485f605e00bcafef25062bb74af3eb192d3296d57dfe3061 number=7 parent_hash=0x60e9824348b6e06eeca12b55ed25332a9dc3a3024e64b486a7839755e796cefb
2023-06-07T05:53:27.025974Z  INFO try_insert_validated_block{block=(7, 0x07cecf1eb7bfee3a485f605e00bcafef25062bb74af3eb192d3296d57dfe3061)}: blockchain_tree: return=Ok(Disconnected { missing_parent: (6, 0x60e9824348b6e06eeca12b55ed25332a9dc3a3024e64b486a7839755e796cefb) })
2023-06-07T05:53:27.025992Z TRACE consensus::engine: start downloading full block. hash=0x60e9824348b6e06eeca12b55ed25332a9dc3a3024e64b486a7839755e796cefb
2023-06-07T05:53:27.032844Z TRACE consensus::engine: Downloaded full block hash=0x60e9824348b6e06eeca12b55ed25332a9dc3a3024e64b486a7839755e796cefb number=6
2023-06-07T05:53:27.032908Z TRACE blockchain_tree: Inserting block hash=0x60e9824348b6e06eeca12b55ed25332a9dc3a3024e64b486a7839755e796cefb number=6 parent_hash=0x5a949798b3aaa04604a28146a9a0ada452b4507e7d9884ad00844249395ca2d5
2023-06-07T05:53:27.032948Z DEBUG try_insert_validated_block{block=(6, 0x60e9824348b6e06eeca12b55ed25332a9dc3a3024e64b486a7839755e796cefb)}:try_append_canonical_chain: blockchain_tree: Appending block to canonical chain head=0x60e9824348b6e06eeca12b55ed25332a9dc3a3024e64b486a7839755e796cefb parent=(5, 0x5a949798b3aaa04604a28146a9a0ada452b4507e7d9884ad00844249395ca2d5)
2023-06-07T05:53:27.033010Z TRACE try_insert_validated_block{block=(6, 0x60e9824348b6e06eeca12b55ed25332a9dc3a3024e64b486a7839755e796cefb)}:try_append_canonical_chain: blockchain_tree: Executing sidechain block canonical_fork=(5, 0x5a949798b3aaa04604a28146a9a0ada452b4507e7d9884ad00844249395ca2d5)
2023-06-07T05:53:27.033056Z  INFO try_insert_validated_block{block=(6, 0x60e9824348b6e06eeca12b55ed25332a9dc3a3024e64b486a7839755e796cefb)}: blockchain_tree: return=Err(InsertBlockError { error: Execution(Validation(EVM { hash: 0x68578d8e615f14e5ff8babeaa4116e28263f1ddab4d67b7dc3d4502f18c8c6f9, message: "Transaction(LackOfFundForGasLimit { gas_limit: 0x0000000000000000000000000000000000000000000000000007fe5cf2bea001_U256, balance: 0x0000000000000000000000000000000000000000000000000000000000000000_U256 })" })), hash: 0x60e9824348b6e06eeca12b55ed25332a9dc3a3024e64b486a7839755e796cefb, number: 6, parent_hash: 0x5a949798b3aaa04604a28146a9a0ada452b4507e7d9884ad00844249395ca2d5, num_txs: 1, .. })

The error is LackOfFundForGasLimit:
2023-06-07T05:53:27.033069Z DEBUG consensus::engine: Failed to insert downloaded block err=InsertBlockError { error: Execution(Validation(EVM { hash: 0x68578d8e615f14e5ff8babeaa4116e28263f1ddab4d67b7dc3d4502f18c8c6f9, message: "Transaction(LackOfFundForGasLimit { gas_limit: 0x0000000000000000000000000000000000000000000000000007fe5cf2bea001_U256, balance: 0x0000000000000000000000000000000000000000000000000000000000000000_U256 })" })), hash: 0x60e9824348b6e06eeca12b55ed25332a9dc3a3024e64b486a7839755e796cefb, number: 6, parent_hash: 0x5a949798b3aaa04604a28146a9a0ada452b4507e7d9884ad00844249395ca2d5, num_txs: 1, .. }

Steps to reproduce
Run the hive test (it may help to add some traces to clients/reth/reth.sh):
./hive --client reth --sim "ethereum/engine" --sim.limit "engine-api/Invalid Ancestor Chain Re-Org, Invalid ReceiptsRoot"

Node logs
No response
Platform(s)
No response
What version/commit are you on?
No response
Code of Conduct

[X] I agree to follow the Code of Conduct

Checking this test again - it's been renamed to engine-api/Invalid Missing Ancestor ReOrg, Invalid ReceiptsRoot.
This is the current state of these tests:
INFO[09-26|19:27:19] API: test started                        suite=0 test=2 name="Invalid Missing Ancestor ReOrg, Invalid ReceiptsRoot, Invalid P8', Reveal using sync (Paris) (reth)"
INFO[09-26|19:27:19] API: client reth started                 suite=0 test=2 container=21d56582
INFO[09-26|19:27:37] API: container IP requested              network=bridge container=21d56582 ip=<IP_ADDRESS>
INFO[09-26|19:27:40] API: test ended                          suite=0 test=2 pass=false
INFO[09-26|19:27:40] API: test started                        suite=0 test=3 name="Invalid Missing Ancestor ReOrg, Invalid ReceiptsRoot, Invalid P8', Reveal using sync (ReOrg from Canonical) (Paris) (reth)"
INFO[09-26|19:27:41] API: client reth started                 suite=0 test=3 container=3e20d0e4
INFO[09-26|19:27:41] API: container IP requested              network=bridge container=3e20d0e4 ip=<IP_ADDRESS>
INFO[09-26|19:28:22] API: test ended                          suite=0 test=3 pass=true
INFO[09-26|19:28:22] API: test ended                          suite=0 test=1 pass=true

The test now fails after marking the invalid block as INVALID correctly:
INFO (Invalid Missing Ancestor ReOrg, Invalid ReceiptsRoot, Invalid P8', Reveal using sync): Response from main client: {INVALID 0x9e5524f777cb5f84f7fc954daa65a1659986a513a38c4587c9a19182c9860186 0x400087eb90}
>> (21d56582) {"jsonrpc":"2.0","id":8,"method":"engine_forkchoiceUpdatedV1","params":[{"headBlockHash":"0x61966c1fc55f9e574a72eed5b410aefaf30a87419bf788bdf568465354c5c53f","safeBlockHash":"0x61966c1fc55f9e574a72eed5b410aefaf30a87419bf788bdf568465354c5c53f","finalizedBlockHash":"0x0000000000000000000000000000000000000000000000000000000000000000"},null]}
<< (21d56582) {"jsonrpc":"2.0","result":{"payloadStatus":{"status":"INVALID","latestValidHash":"0x9e5524f777cb5f84f7fc954daa65a1659986a513a38c4587c9a19182c9860186","validationError":"links to previously rejected block"},"payloadId":null},"id":8}
INFO (Invalid Missing Ancestor ReOrg, Invalid ReceiptsRoot, Invalid P8', Reveal using sync): Response from main client fcu: {INVALID 0x9e5524f777cb5f84f7fc954daa65a1659986a513a38c4587c9a19182c9860186 0x4000968d70}

DEBUG (Invalid Missing Ancestor ReOrg, Invalid ReceiptsRoot, Invalid P8', Reveal using sync): Failed `Expect*` routine called from: file=/source/suites/engine/invalid_ancestor.go, line=483
FAIL (Invalid Missing Ancestor ReOrg, Invalid ReceiptsRoot, Invalid P8', Reveal using sync): Unexpected status response on EngineNewPayloadV1: ACCEPTED, expected=VALID, payload={

node logs:
2023-09-26T23:27:40.232674Z  WARN consensus::engine: Failed to insert downloaded block err=InsertBlockError { error: Execution(Validation(ReceiptRootDiff { got: 0x5d721667f3f41c6e8a5e630b2c5f259855d2365513295c4fd0c5d05498ce6489, expected: 0x5d721667f3f41c6e8a5e630b2c5f259855d2365513295c4fd0c5d05498ce6476 })), hash: 0x48431d3ea93b8aa441478f1a073f6e0497337343c9bb8b2cf9736222a22d2d6f, number: 13, parent_hash: 0x9e5524f777cb5f84f7fc954daa65a1659986a513a38c4587c9a19182c9860186, num_txs: 1, .. }
2023-09-26T23:27:40.232679Z  WARN consensus::engine: Marking block as invalid invalid_number=13 invalid_hash=0x48431d3ea93b8aa441478f1a073f6e0497337343c9bb8b2cf9736222a22d2d6f err=Execution(Validation(ReceiptRootDiff { got: 0x5d721667f3f41c6e8a5e630b2c5f259855d2365513295c4fd0c5d05498ce6489, expected: 0x5d721667f3f41c6e8a5e630b2c5f259855d2365513295c4fd0c5d05498ce6476 }))

The ReceiptRootDiff is the correct validation error! Now the only thing to solve is why we are returning ACCEPTED:

Is this due to the test design? If the test sets up a sidechain after the block is marked invalid, using many engine_newPayload calls, then we'll return ACCEPTED because we don't validate sidechain state roots, or return VALID on sidechains yet.

@Rjected which branch of https://github.com/paradigmxyz/hive do you use to test?
In the master branch, there is no test related to engine-api/Invalid Missing Ancestor ReOrg, Invalid ReceiptsRoot ?

@Rjected which branch of https://github.com/paradigmxyz/hive do you use to test?
In the master branch, there is no test related to engine-api/Invalid Missing Ancestor ReOrg, Invalid ReceiptsRoot ?

Ah, we actually merged the reth config into upstream hive, so that repo is no longer necessary. See the eth hive tests:
https://github.com/ethereum/hive/blob/24c412a635463ae0fc2d65ad7524696227661c05/simulators/ethereum/engine/suites/engine/invalid_ancestor.go#L27-L33


Add null session

This is just a session backend which does nothing.
It's so when we're using a wayland/xorg/whatever (anything that isn't DRM, really) backend, the user can be given this session and not worry about checking what backend is actually being used.
This fixes a bug where closing one of the examples using the wayland backend would absolutely destroy your session (massive visual glitches, unable to change VT), and forces you to restart your computer. This happened to me when using an example under weston under X.
This PR doesn't add "session/null.c" to the build system. I'll add it when/if the meson PR is accepted.
Instead of doing this sort of patchwork fix, we might want to consider whether we want wlr_session to be part of the public API. Maybe all of the session stuff could be handled in wlr_backend_autocreate, so we don't accidentally try to open a logind/direct session when we're not using DRM.

Instead of doing this sort of patchwork fix, we might want to consider whether we want wlr_session to be part of the public API. Maybe all of the session stuff could be handled in wlr_backend_autocreate, so we don't accidentally try to open a logind/direct session when we're not using DRM.

We want as much as possible to be in the public API. We want the user to have as much control over the system as we can give them.

We want as much as possible to be in the public API. We want the user to have as much control over the system as we can give them.

Yes, but wlr_session is tied so ridiculously close to the DRM backend, and isn't generally useful for other backends at all.
I think session should just be bundled into DRM and then the state about the session could be accessed through the 'backend-specific state' functions, whenever those get added.

That's fair. I wouldn't be opposed to rolling this more closely into DRM.

Ok. This PR isn't really needed then.
I'll look into the best place it fits at some point.


Got 100% CPU Load on a production server with huge parallel load

Hello,
I am getting 100% CPU Load almost instantly (after 10-15 seconds of run) on a production server with a lot of parallel usage (~500 users in parallel opens WebSocket and send some data).
Trying to figure out the bottleneck I have ended up with an absolutely empty code that causes a 100% CPU usage. Xdebug suggests that the issue is in react/event-loop.
I have tried StreamSelectLoop (the default one) and ExtEventLoop (this one selected automatically when php-pecl-event enabled. I have php-pecl-event 2.2.1 and libevent 2.0.22 installed) - both make 100% CPU Load.
CacheGrind files:
ExtEventLoop: https://www.dropbox.com/s/s5lmg9ey5lyxrf5/cachegrind_ExtEventLoop.zip
StreamSelectLoop: https://www.dropbox.com/s/q61kbzy1y7cpxg3/cachegrind_StreamSelectLoop.zip
Sample code:
<?php
  require __DIR__ . '/vendor/autoload.php';
  use Ratchet\ConnectionInterface;

  class EmptyClass implements Ratchet\MessageComponentInterface {
    public function onMessage(ConnectionInterface $from, $msg) {
    }
    public function onOpen(ConnectionInterface $conn) {
    }
    public function onClose(ConnectionInterface $conn) {
    }
    public function onError(ConnectionInterface $conn, \Exception $e) {
    }
  }

  $webServer = Ratchet\Server\IoServer::factory(
          new EmptyClass(),
    8080
  );
  $webServer->run();
?>

composer show -i:
cboden/ratchet            v0.3.5 PHP WebSocket library
evenement/evenement       v2.0.0 Événement is a very simple event dispatching library for PHP
guzzle/common             v3.9.2 Common libraries used by Guzzle
guzzle/http               v3.9.2 HTTP libraries used by Guzzle
guzzle/parser             v3.9.2 Interchangeable parsers used by Guzzle
guzzle/stream             v3.9.2 Guzzle stream wrapper component
guzzlehttp/psr7           1.3.1  PSR-7 message implementation
psr/http-message          1.0.1  Common interface for HTTP messages
ratchet/pawl              v0.2.2 Asynchronous WebSocket client
ratchet/rfc6455           v0.2.1 RFC6455 WebSocket protocol handler
react/cache               v0.4.1 Async caching.
react/dns                 v0.4.3 Async DNS resolver.
react/event-loop          v0.4.2 Event loop abstraction layer that libraries can use for evented I/O.
react/promise             v2.4.1 A lightweight implementation of CommonJS Promises/A for PHP
react/promise-timer       v1.1.0 Trivial timeout implementation for Promises
react/socket              v0.4.3 Library for building an evented socket server.
react/socket-client       v0.5.1 Async connector to open TCP/IP and SSL/TLS based connections.
react/stream              v0.4.5 Basic readable and writable stream interfaces that support piping.
react/zmq                 v0.3.0 ZeroMQ bindings for React.
symfony/event-dispatcher  v3.1.7 Symfony EventDispatcher Component
symfony/http-foundation   v3.1.7 Symfony HttpFoundation Component
symfony/polyfill-mbstring v1.3.0 Symfony polyfill for the Mbstring extension
symfony/routing           v3.1.7 Symfony Routing Component

php -v
PHP 7.0.13 (cli) (built: Nov  8 2016 20:28:51) ( NTS )
Copyright (c) 1997-2016 The PHP Group
Zend Engine v3.0.0, Copyright (c) 1998-2016 Zend Technologies
    with Zend OPcache v7.0.13, Copyright (c) 1999-2016, by Zend Technologies
    with Xdebug v2.4.1, Copyright (c) 2002-2016, by Derick Rethans

Any help is highly appreciated!

Hello again,
Digging further and using a simple strace I see that the infinite loop appears at the state "EMFILE (Too many open files)", so increasing the open fd limit ultimately solves the problem completely.
This limit was set to 1024 and was used up in a few seconds on a high load. CentOS allows us to set this limit as big as 1048576 for a single user.
On a production server having ~100 persistent opened connections in a single point of time (having a lot of connections being opened and closed before that) I see, that an example code above have ~1200 opened sockets (which indeed counts in file descriptors), which used up a limit. I suspect that some sockets not been closed correctly.
I am not sure is this a bug or not, but from the other side these limits can be tracked like this:
<?php
  $fd_opened_by_this_process = iterator_count(new FilesystemIterator("/proc/self/fd/", FilesystemIterator::SKIP_DOTS));

  $soft_fd_limit = posix_getrlimit()['soft openfiles'];
  $hard_fd_limit = posix_getrlimit()['hard openfiles'];

Using this information we can gracefully stop accepting new connections with some user-friendly error instead of going into infinite loop.
You may close this issue when you decide what to do with this ;-)
Off Topic:
I saw such behavior, i.e. in MySQL Server when you just can't increase a maximum connection value without increasing an open files limit (https://codepoets.co.uk/2015/mysql-max_connections-stuck-on-214/)

I have probably the same problem.
I have already installed the ev PHP extension (see: http://socketo.me/docs/deploy#evented-io-extensions)
Should I try the event?
I have a WebSocket Server that get some data over a ZMQ push/pull setup.
My Architecture:

WebSocket that holds the TCP Socket and listens for Data
$loop   = React\EventLoop\Factory::create();

$context = new React\ZMQ\Context($loop);

$pull = $context->getSocket(\ZMQ::SOCKET_PULL);
$pull->bind( $this->dns );

// main WebSocket app -> inject socket that will listen for data (TCP)
$mapUpdate = new Main\MapUpdate($pull);
// "onMessage" listener
$pull->on('message', [$mapUpdate, 'receiveData']);

// Set up our WebSocket server for clients wanting real-time updates
$webSock = new React\Socket\Server($loop);
// Binding to <IP_ADDRESS> means remotes can connect (Web Clients)
$webSock->listen($this->wsListenPort, $this->wsListenHost);
new IoServer(
    new HttpServer(
        new WsServer(
            $mapUpdate
        )
    ),
    $webSock
);

$loop->run();

TCP Socket that pushes data to the WebSocketServer (see above)
$context = new \ZMQContext();
$socket = $context->getSocket(\ZMQ::SOCKET_PUSH);

$send = [
    'task' => $task,
    'load' => $load
];

$socket->connect($this->socketUri);
$socket->send(json_encode($send));
$socket->disconnect($this->socketUri);

Any ideas, what could be wrong with that? Thanks :)

Closed due to inactivity.


Update dedicated-ip-addresses.md

Checklist
Required

[x] I acknowledge that all my contributions will be made under the project's license.

PR Details
Description of the change:  Updated language in the docs to accurately reflect the current user experience and limitations of the user interface and product.
Reason for the change:  The information was not accurate to the current user experience.
Link to original source:

Closes #

Checked "I acknowledge that all my contributions will be made under the project's license."


Current server selector position

The current Server selector position in the navigation panel is not centered.

#1

New navigation style


Query parameters are not seen by react-router server side

When I add the following code to a page component it will print query {} server side, but query Object {q: "test"} client side.
static willTransitionTo(transition, params, query) {
    console.log('query', query);
  }

I will investigate further

Ask here please: https://github.com/rackt/react-router/issues


List Index Out Of Range

Hello (again)!
I just had a quick question about the proper configuration for evaluating RBPN. I input the following:
python eval.py rbpn --cuda --test /usr/local/VideoSuperResolution/LR_INPUT/
This returns an error:
2019-07-06 00:52:13,227 INFO: LICENSE: RBPN is implemented by M. Haris, et. al. @alterzero
2019-07-06 00:52:13,227 WARNING: I use unsupervised flownet to estimate optical flow, rather than pyflow module.
2019-07-06 00:52:16,775 INFO: Total params: 14517449
Traceback (most recent call last):
  File "eval.py", line 162, in <module>
    main()
  File "eval.py", line 153, in main
    loader = Loader(test_data, 'test', loader_config, n_threads=opt.thread)
  File "/usr/local/VideoSuperResolution/VSR/DataLoader/Loader.py", line 322, in __init__
    augmentation, **kwargs)
  File "/usr/local/VideoSuperResolution/VSR/DataLoader/Loader.py", line 127, in __init__
    self.parser = _m.Parser(dataset, config)
  File "/usr/local/VideoSuperResolution/VSR/DataLoader/Parser/custom_pairs.py", line 46, in __init__
    vf.pad([depth // 2, depth // 2])
  File "/usr/local/VideoSuperResolution/VSR/DataLoader/VirtualFile.py", line 419, in pad
    self.file.insert(0, self.file[0])
IndexError: list index out of range

I have tried inputting both .mp4 files and frames titled 01.jpg, 02.jpg, 03.jpg ...  (in separate attempts) and the input is about 350 by 350 pixels. I've tried using a video that only has 30 or so frames, and I also tried a 700 frame video. All give the same error.
By the way, here is a link for some of the training pth files I made on the CelebA dataset for esrgan:
https://drive.google.com/open?id=1e9F7QCd-SeK0FCxQ_cJz09XIUDwxJVYv
It might be useful for someone... after epoch 310, I switched from PSNR to gan-oriented training with a weight of 5.0e-4. The results at epoch 527 look good :)
EDIT: I just put some video frames into ESRGAN, and I seem to be getting the same problem in all images for ESRGAN as well (which is weird because ESRGAN worked a few minutes ago).
EDIT 2: Huh, that is so weird, ESRGAN sometimes works, and sometimes gives this error (for the same images!!!) It might have something to do with the graphics card, because ESRGAN never works on my computer for these images, but sometimes works on google colab (when the servers arent busy ig).

Got it, will take some time to verify it.

You should arrange a video into sequential pictures and put them under a folder:
|-LR_INPUT
|----video0
|--------frame000.png
|--------frame001.png
|--------...
|----video1
|----...


Exclude demo code from library

Demo code shall no longer be part of library jar.
Change POM to exclude demo sources and demo classes.

Done with 97ad96a63941d573cd04c80f241e8acfe597dee0.


Fixing for extremely deep nested editing.

Hi! 👋
Firstly, thanks for your work on this project! 🙂
Errors occur while editing a deeply nested structure with optional array elements. Tested with mongoose.
Here is the diff that solved my problem:
diff --git a/node_modules/adminjs/lib/utils/param-converter/convert-nested-param.js b/node_modules/adminjs/lib/utils/param-converter/convert-nested-param.js
index eb46488..1af7c5a 100644
--- a/node_modules/adminjs/lib/utils/param-converter/convert-nested-param.js
+++ b/node_modules/adminjs/lib/utils/param-converter/convert-nested-param.js
@@ -20,14 +20,16 @@ const convertNestedParam = (parentValue, subProperty) => {
   const {
     type = 'string'
   } = subProperty;
-  let value = parentValue[path];
+  let value = parentValue?.[path];

   if (type === 'mixed') {
     const nestedSubProperties = subProperty.subProperties;

     for (const nestedSubProperty of nestedSubProperties) {
       if (subProperty.isArray) {
-        value = [...value].map(element => convertNestedParam(element, nestedSubProperty));
+        if (value) {
+          value = [...value].map(element => convertNestedParam(element, nestedSubProperty));
+        }
       } else {
         value = convertNestedParam(value, nestedSubProperty);
       }

This issue body was partially generated by patch-package.

Can you show a structure I can reproduce your issue with for testing?

Unfortunately, I have already changed the structure of the database and now the error does not appear. I'm sorry. When I encounter this problem again, I will describe the full case so that you can reproduce it. I think now we can close this issue


Feature Request: Send resolved immediatly if all alerts in a group are not firing anymore

Given send_resolved is active on a given alert-route. If all alerts in a group are resolved the "final resolved notification" is sent the next time the group_wait expires. It would be nice if in this case the group_wait is skipped and the "final resolved notification" is sent immediatly.
The motivation is to have faster feedback that the countermeasures that have been taken to adress the issue causing the alerts to fire have been successful.

This is in direct opposition to the purpose of group_interval, which is to
throttle the rate of notifications from from a group. This would be like
asking for an instant notification when there was a new alert. You can
always reduce group_interval if you want to know faster.
Also, that the alerts have stopped firing does not mean that the issue has
resolved.
On Sat 11 Aug 2018, 20:40 Christoph Maser<EMAIL_ADDRESS>wrote:

Given send_resolved is active on a given alert-route. If all alerts in a
group are resolved the "final resolved notification" is sent the next time
the group_wait expires. It would be nice if in this case the group_wait is
skipped and the "final resolved notification" is sent immediatly.
The motivation is to have faster feedback that the countermeasures that
have been taken to adress the issue causing the alerts to fire have been
successful.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/prometheus/alertmanager/issues/1510, or mute the
thread
https://github.com/notifications/unsubscribe-auth/AGyTdtAeYyDan49-bLNYVy2a_wbdJHgMks5uPyUDgaJpZM4V5RJK
.

The the number of notifications should not change at all. Unless some corner where case your servicies in the same alert group are flapping in such way that the would go all to non-alerting and back to alerting before group_wait expires. If you have such a situation you have done something severely wrong anyway.
Basically with this proposed feature you might shorten the time notifications for a given alert group are sent but not their number, so where is the harm? Comparing this to sending the first alert immediatly is not correct IMHO.

Basically with this proposed feature you might shorten the time notifications for a given alert group are sent but not their number,

That does affect their number, as the time between them would be reduced. If the resolution is just a flap for example.

so where is the harm?

Spamming the oncall is to be avoided.

Comparing this to sending the first alert immediatly is not correct IMHO.

I think you're confusing group_wait and group_interval. The relevant setting here is group_interval.

Like i laid out flapping with resolving all allerts on a group should beconsidered a major flaw in your setup so I in my mind raising the number of notifications is not a thing.
Also this raises the question about a flap detection feature :smile:

Yes I meant group_interval not group wait.
Yes a prometheus restart could trigger this, I did not think about this case. Shouldn't that be mitigated in part by by https://github.com/prometheus/prometheus/pull/4061
About the flap detection, isn't the current way of doing things more a flap-hiding. Potentially there could be flapping alerts and the user sees a continous problem instead of beeing notified about the flapping going on.

Shouldn't that be mitigated in part by

Yes, one of the main ways it could happen should happen a lot less often.

About the flap detection, isn't the current way of doing things more a flap-hiding.

Yes, it's more like hysteresis. By the time the alert is firing we're past the point of considering flaps (alerts thresholds are meant to be set so hitting them once is severe enough to cause a notification).

Potentially there could be flapping alerts and the user sees a continous problem instead of beeing notified about the flapping going on.

We don't care about flaps per-se. We care about there being a bad state that the oncall needs to know about. We also care about not spamming that oncall, and all the group_wait/group_interval logic we have here is to increase the signal to noise ratio.
If you want fast feedback that your mitigations/fixes are taking effect you will best get that from dashboards, not the alertmanager which will take potentially minutes to hours to pick that up depending on the nature of the system and alert.

I also opened a discussion around the same topic on the google group users.
Basically what we are trying to achieve is to remove every updates and simply forward the first firing event (after group_wait) and the "full resolved" event.
This looks impossible right now.
I would like to have on top of send_resolved, a send_updates boolean?
Also setting a group_interval very long (5y for instance) is removing every new alerts of the same group even if they were fully resolved in between. To me this is not expected...

This is in direct opposition to the purpose of group_interval, which is to throttle the rate of notifications from from a group. This would be like asking for an instant notification when there was a new alert. You can always reduce group_interval if you want to know faster. Also, that the alerts have stopped firing does not mean that the issue has resolved.
…
On Sat 11 Aug 2018, 20:40 Christoph Maser, @.***> wrote: Given send_resolved is active on a given alert-route. If all alerts in a group are resolved the "final resolved notification" is sent the next time the group_wait expires. It would be nice if in this case the group_wait is skipped and the "final resolved notification" is sent immediatly. The motivation is to have faster feedback that the countermeasures that have been taken to adress the issue causing the alerts to fire have been successful. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#1510>, or mute the thread https://github.com/notifications/unsubscribe-auth/AGyTdtAeYyDan49-bLNYVy2a_wbdJHgMks5uPyUDgaJpZM4V5RJK .

I feel both of your statements are misunderstanding and mischaracterizing the request and logic behind it.  Settings exist because of ideas/needs like the Op's and it should be more honestly considered.

send_resolved is not triggered unless prometheus deems the alert resolved.  This is completely different than the point you state "alerts have stopped firing does not mean that the issue has resolved.".  send_resolved is sent because prometheus has data that the issue condition has ended -- completely different than simply not sending more alerts about the problem.
send_resolved is a special and special case singular alert, I think the request makes a ton of sense to be allowed to bypass group wait with a send_resolved alert that supersedes all other alerts or remediations.  Group_wait may assist with overly chatty alerts, however send_resolved is unlikely to be one of those or contribute to the 'too many alerts' issue that group_wait mitigates.

.

so where is the harm?

Spamming the oncall is to be avoided.

In conditions where an error has resolved itself, the on-call's likely to get more notifications and be more awake and stay that way because alerts are showing systems being down and downstream alert systems like pager-duty will continue on their alert cycles because the send_resolved has to wait a full period to take effect.  I was woken up last night due to an error that resolved itself by the time I was at keys, in my desires to not have that happen again I researched the send_resolved feature, and then researched how to make it take effect faster and found this thread.

Would including 'status' in the group_by field help with this? That would treat "resolved" messages as being in a different group to "firing" messages, and thus avoid the grouping delay from the initial alert.

@OrlandoArcapix Thats exactly our situation. "resolved" message should get delivered faster. In our case in 5 minutes, not having to wait for the group_interval: 6h. Any update on this issue?

Hi, any update on this issue? Getting resolved messages immediately would be great.

hi, any update on this issue?


[hotfix][tests] Replace MockOperatorStateBackend with default

Addition of  MockOperatorStateBackend in #17645 was unnecessary because DefaultOperatorStateBackend would suffice.
cc: @Myasuka

If removing MockOperatorStateBackend, I think the constructor of MockStateBackend(boolean emptySnapshot) is useless.

new MockStateBackend(true) is still used by UnalignedCheckpointFailureHandlingITCase while other tests use false
(this is because UnalignedCheckpointFailureHandlingITCase actually sends the snapshot to JM which won't serialize mock state handle).

Thanks for the review!
Merged<PHONE_NUMBER>bf5989e5e3afae94b938cec1e685f3 into master.


A new interpreter and partial-evaluation-based approach for EverParse 3D

This is a draft PR, not yet ready for merging. Meant more to collect a worklist of items that must be completed before merging.
In no particular order:

[ ] Complete and merge the F* branch nik_erasable into F*. It supports erasing erasable arguments in the normalizer and nbe (nik)

[ ] Support for dependence on Enum-typed fields (by inlining them). This is not supported on master anyway, but it should be. (nik)

[ ] Support for Enum typed parameters, by adding, at least within the internal syntax, a first-class notion of refinement type (at the moment, refinements are only allowed on the fields of a struct, not in parameter types). This too is not fully supported on master, but it should be. (nik)

[ ] Remove the interpret_alt flag and retain --interpret only. (nik)

[ ] Adding back support for error handling wrappers on generated validators (nik)

[ ] Fix a bug where type aliases to defined struct or case types don't work (nik)

[ ] Add an interface file for interpreter mode so that Kremlin only generates a .h for the specified entrypoint, rather than for all functions (nik)

[ ] Propagating variable names from the source for rename_let: currently they are all harcoded to "a". In fact, in F* we should forbid using the empty string "" with rename_let. This led to strange failures in Kremlin. (nik)

[ ] Revise the syntax of actions so that they don't have to be so explicitly monadic.

[ ] Separate :on-success actions from :check actions. The former return unit; the latter return bool and can cause validation to fail. This will save us from having to write return true everywhere in actions.

[ ] Fixing F* --dep so that it does not emit .krml targets for files that are explicitly excluded from extraction using the --extract option (aseem)

[ ] Restoring batch and makefile mode to support compiling the generated C files (tahina)

[ ] Support for Action_call in Interpreter.fst and then support for output types based on it (aseem)

Fixing F* --dep so that it does not emit .krml targets for files that are explicitly excluded from extraction using the --extract option (aseem)

This will break a large number of applications (e.g. Noise*, HACL*, and soon MLS*) that rely on this behavior.
In those applications, we do --extract Spec.* -Impl.*, meaning "extract specs to OCaml and ditch low-level code" -- this allows us to then test the specs via OCaml compilation. Low-level compilation is controlled by kremlin's bundle flags.
Note that changing this behavior might be dangerous, too, since for Low* code you may inadvertently remove a definition that would be needed by another file, resulting in Warning 2 errors and potentially invalid C code if, for instance, B.foo needs A.bar but you forgot that and passed --extract -A.
Hope this helps,
Jonathan

Thank you Jonathan for sharing your concerns on skipping .krml files for F* --dep, let's continue that discussion in FStarLang/FStar#2385.


manual control on hide/show result list

More precise control on when to show the result list.
Possible fix for #417 and #413

hey @ahmedsadman I wonder how can I use it? What should I do to toggle the list?

Inside the component you add a prop like, ref={(comp) => this.googleRef = comp}, then from anywhere you type this.googleRef.showListView()

There is another way of toggling the list.
In definition file of GooglePlacesAutocomplete component there is default prop: listViewDisplayed='auto'. I also saw in definition of this component that this can accept boolean value true and false
You can create a wrapper for this component, in my case I want to hide the list after user picks specific address.
import React, {Component} from 'react';
import {GooglePlacesAutocomplete} from 'react-native-google-places-autocomplete';

class AutocompleteInput extends Component {
  constructor(props) {
    super(props);

    this.state = {
      isListViewVisible: 'auto'
    };

  }
  render() {
    return (
      <GooglePlacesAutocomplete
        listViewDisplayed={this.state.isListViewVisible}
        onPress={(data, details = null) => {
          this.setState({
            isListViewVisible: false,
          })
        }}
      />
    )
  }
}

export default AutocompleteInput

Thanks for the PR!
This seems redundant to me as there is already a listViewDisplayed prop that handles this.


[2.x] Node stuck while processing blocks (deadlock?)

We have subscription to various events in Blockchain and these events are all initiated by channel send in storeBlock. notificationDispatcher then multiplexes them to real set of subscribers. One of these subscribers is network.Server that wants to know about new blocks. It accepts them in relayBlocksLoop and then broadcasts inv messages to all peers.
Depending on network conditions one of attempts to send a message can block. If it blocks, it can affect notification dispatcher in Blockchain and subsequently stop storeBlock itself. Usually storeBlock is invoked from blockQueue handler, so it stops block processing completely.
This technically should be resolved with write timeout (#1637), so that when timeout occurs we should get back to queue processing and unblock everything, but the peer might be disconnected already by this time, so handleQueues will just exit leaving queues with no reader thread.
Therefore:

we need to always flush queues on (*TCPPeer).handleQueues exit
block broadcasts should follow the same 2/3 model as transactions (hi, #608)

This needs to be done for Neo 3, but after some testing there we'll also need to port appropriate changes to master-2.x branch because there are production Neo 2 nodes that can block this way.

In this case p.done must be closed and putPacketIntoQueue should unblock, I don't see how it could happen without p.done closing. So it's something different (but when it happens (*Server).run stops also), and I wasn't able to untangle it without trace data. Flushing can be added of course, but it probably won't affect anything, so we better wait for this problem to reproduce again and then see what actually happens there.

Actual log of this case:
Feb 27 07:22:34 neo-node neo-go[2905]: 2021-02-27T07:22:34.152Z#011INFO#011blockchain persist completed#011{"persistedBlocks": 1, "persistedKeys": 89, "headerHeight": 6975428, "blockHeight": 6975428, "took": "155.383944ms"}
Feb 27 07:22:49 neo-node neo-go[2905]: 2021-02-27T07:22:49.195Z#011INFO#011blockchain persist completed#011{"persistedBlocks": 0, "persistedKeys": 1, "headerHeight": 6975428, "blockHeight": 6975428, "took": "39.057501ms"}
Feb 27 07:22:50 neo-node neo-go[2905]: 2021-02-27T07:22:50.845Z#011WARN#011peer disconnected#011{"addr": "<IP_ADDRESS>:10333", "reason": "handling getheaders message: key not found", "peerCount": 5}
Feb 27 07:32:23 neo-node neo-go[2905]: 2021-02-27T07:32:23.011Z#011ERROR#011Error encountered with rpc request#011{"method": ""}
Feb 27 07:37:59 neo-node neo-go[2905]: 2021-02-27T07:37:59.498Z#011ERROR#011Error encountered with rpc request#011{"method": ""}

Then nothing useful happens, no block events, no peer events, nothing except RPC errors (so RPC still works, but it doesn't help a lot).

It can't be reproduced on Legacy, it can't be on N3, a lot of code has changed since, so this issue doesn't make much sense.


Raceday Operations Dashboard text needs update

The Raceday Operations Dashboard portal item (https://afmiedev.esri.com/arcgis/home/item.html?id=18d3a5d24d8c4028854a45a9ed4e10f2) text needs to be updated.
Summary:
An Operations Dashboard for Fire/Rescue to use when managing incidents on race day.
Description:
An Operations Dashboard for Fire/Rescue to use when managing incidents on race day.

Since we will recreate this content on NSEC, I am closing this issue.


How to impersonate

Hello.
There is a way to impersonate to any store? That would be great as an admin

There's no built in way to do this. Not sure it's 100% possible since clicking the app in your dev store would cause hmac issues if you spoofed the shop in auth process. You could add a hidden page in your app to switch the shop session (override the Shopify domain session to the new domain and things should work..).
That's all the advice I can offer

Instead of impersonating the accounts, I'd just us a backend to view the data. Impersonations usually means security issues.
https://laravelvoyager.com


Adding same node twice creates perpetual loop

Environment

OS / version: Fedora 29
Processor architecture: x86_64
TPM Manufacturer:  emulator
Keylime version: master

Description
Add a node twice creates a perpetual loop
Expected behavior vs. actual behavior
Keylime_verifier and keylime_tenant should report this only once, and then break / exit the loop.
Steps to reproduce problem
Run the following command:
keylime_tenant -v <IP_ADDRESS> -t <IP_ADDRESS> -f somefile --uuid D432FBB3-D2F1-4A97-9EF7-75BD81C00000
Run the same again:
keylime_tenant -v <IP_ADDRESS> -t <IP_ADDRESS> -f somefile --uuid D432FBB3-D2F1-4A97-9EF7-75BD81C00000
Result
2019-03-08 13:34:45.752 - keylime.cloudverifier - INFO - DELETE returning 200 response for instance id: D432FBB3-D2F1-4A97-9EF7-75BD81C00000
2019-03-08 13:34:45.761 - keylime.cloudverifier - WARNING - Node of uuid D432FBB3-D2F1-4A97-9EF7-75BD81C00000 already exists
2019-03-08 13:34:45.771 - keylime.cloudverifier - INFO - DELETE returning 200 response for instance id: D432FBB3-D2F1-4A97-9EF7-75BD81C00000
2019-03-08 13:34:45.780 - keylime.cloudverifier - WARNING - Node of uuid D432FBB3-D2F1-4A97-9EF7-75BD81C00000 already exists
2019-03-08 13:34:45.789 - keylime.cloudverifier - INFO - DELETE returning 200 response for instance id: D432FBB3-D2F1-4A97-9EF7-75BD81C00000
2019-03-08 13:34:45.799 - keylime.cloudverifier - WARNING - Node of uuid D432FBB3-D2F1-4A97-9EF7-75BD81C00000 already exists
2019-03-08 13:34:45.808 - keylime.cloudverifier - INFO - DELETE returning 200 response for instance id: D432FBB3-D2F1-4A97-9EF7-75BD81C00000
2019-03-08 13:34:45.818 - keylime.cloudverifier - WARNING - Node of uuid D432FBB3-D2F1-4A97-9EF7-75BD81C00000 already exists
2019-03-08 13:34:45.827 - keylime.cloudverifier - INFO - DELETE returning 200 response for instance id: D432FBB3-D2F1-4A97-9EF7-75BD81C00000
2019-03-08 13:34:45.837 - keylime.cloudverifier - WARNING - Node of uuid D432FBB3-D2F1-4A97-9EF7-75BD81C00000 already exists
2019-03-08 13:34:45.846 - keylime.cloudverifier - INFO - DELETE returning 200 response for instance id: D432FBB3-D2F1-4A97-9EF7-75BD81C00000
2019-03-08 13:34:45.856 - keylime.cloudverifier - WARNING - Node of uuid D432FBB3-D2F1-4A97-9EF7-75BD81C00000 already exists
2019-03-08 13:34:45.865 - keylime.cloudverifier - INFO - DELETE returning 200 response for instance id: D432FBB3-D2F1-4A97-9EF7-75BD81C00000
2019-03-08 13:34:45.875 - keylime.cloudverifier - WARNING - Node of uuid D432FBB3-D2F1-4A97-9EF7-75BD81C00000 already exists
2019-03-08 13:34:45.884 - keylime.cloudverifier - INFO - DELETE returning 200 response for instance id: D432FBB3-D2F1-4A97-9EF7-75BD81C00000
2019-03-08 13:34:45.894 - keylime.cloudverifier - WARNING - Node of uuid D432FBB3-D2F1-4A97-9EF7-75BD81C00000 already exists

2019-03-08 13:34:44.995 - keylime.tenant - WARNING - Node already existed at CV.  Deleting and re-adding...
2019-03-08 13:34:45.004 - keylime.tenant - INFO - Node D432FBB3-D2F1-4A97-9EF7-75BD81C00000 deleted from CV
2019-03-08 13:34:45.004 - keylime.tenant - DEBUG - b64_v:bsoXgk2ceWMVn42XlXiLhOPtzCRAMQ1fbVwCwCyqQcs=
2019-03-08 13:34:45.013 - keylime.tenant - WARNING - Node already existed at CV.  Deleting and re-adding...
2019-03-08 13:34:45.023 - keylime.tenant - INFO - Node D432FBB3-D2F1-4A97-9EF7-75BD81C00000 deleted from CV
2019-03-08 13:34:45.023 - keylime.tenant - DEBUG - b64_v:bsoXgk2ceWMVn42XlXiLhOPtzCRAMQ1fbVwCwCyqQcs=
2019-03-08 13:34:45.032 - keylime.tenant - WARNING - Node already existed at CV.  Deleting and re-adding...
2019-03-08 13:34:45.042 - keylime.tenant - INFO - Node D432FBB3-D2F1-4A97-9EF7-75BD81C00000 deleted from CV
2019-03-08 13:34:45.042 - keylime.tenant - DEBUG - b64_v:bsoXgk2ceWMVn42XlXiLhOPtzCRAMQ1fbVwCwCyqQcs=
2019-03-08 13:34:45.052 - keylime.tenant - WARNING - Node already existed at CV.  Deleting and re-adding...
2019-03-08 13:34:45.061 - keylime.tenant - INFO - Node D432FBB3-D2F1-4A97-9EF7-75BD81C00000 deleted from CV
2019-03-08 13:34:45.061 - keylime.tenant - DEBUG - b64_v:bsoXgk2ceWMVn42XlXiLhOPtzCRAMQ1fbVwCwCyqQcs=
2019-03-08 13:34:45.070 - keylime.tenant - WARNING - Node already existed at CV.  Deleting and re-adding...
2019-03-08 13:34:45.080 - keylime.tenant - INFO - Node D432FBB3-D2F1-4A97-9EF7-75BD81C00000 deleted from CV
2019-03-08 13:34:45.080 - keylime.tenant - DEBUG - b64_v:bsoXgk2ceWMVn42XlXiLhOPtzCRAMQ1fbVwCwCyqQcs=

The above are repeatably logged until the terminal session is <ctrl-c>

yes, i've seen this before.  it's a nasty problem that has to do with threading, python, and synchronization.  Tornado has multiple processes with their own event loops (see https://github.com/keylime/python-keylime/blob/master/keylime/cloud_verifier_tornado.py#L507 ), so data synchronization happens through sqlite.  when a node already exists the tenant code tries to delete and re-add it (https://github.com/keylime/python-keylime/blob/master/keylime/tenant.py#L481 ).  Because the synchronization happens lazily in the verifier, it takes it a moment to register the node deleted and the tenant keeps trying to add it again.
I see a few potential fixes for this:

properly fix the race condition (ewww, may take a large amount of re-factoring)
set multiprocessing_pool_num_workers in keylime.conf to 1.  can't have a race when there's only one
sleep a little after the call to delete to let it persist before trying to re-add
raise an error and tell the user they must delete first before adding (this really just masks the race)

I'm partial to number 4.  if you call status on the node as it's being deleted, it will tell you that it's been terminated and is starting to go away.  it just takes it once through the cloud verifier state machine to actually get removed from the database.
anyone have thoughts on this?  @jetwhiz @lukehinds

another option i just thought of: the tenant code can keep some state that it's trying to delete it and suppress the error messages until it is done.  Probably need a small sleep to keep the tenant from polling too quickly.  So, this is in essence a pretty version of number 3 above.  but instead of a random sleep, it's a polling sleep.

I initially though of 3 / 4, but the problem then is what happens if there are more then one tenant (or more than one entity making the call to add). This is why fixing this in the verifier might be a better approach. I don't know the Rest API to well, but I guess the same call is available there and if we patch up the tenant, then the same issue would still reside over an API call to add?
We could perhaps use a sort of token bucket in the verifier?

@nabilschear @jetwhiz , so to summarise, unless there is a specific need or something I have not considered, can we fix this by removing:
https://github.com/keylime/python-keylime/blob/7b0a5b4c57e0efd7add2cac88a7ad0eca95e9a90/keylime/tenant.py#L482-L483
And let the user call with the args delete and add and make the decision themselves?

the intent behind the delete/re-add pattern was to allow a user to update an existing node.  if some of the metadata changes (i.e., new pcr values), this was the mechanism to handle this.
what about this plan?
I agree that add on an existing node shouldn't update it.  i'll update the code to raise an error if you try to add to an existing node.
to more explicitly handle this, I'll add an update command that will do the delete/re-add pattern.  though we also have re-activate which actually does change the node in place.
and finally i think that delete should poll the verifier to confirm the node was deleted before returning.  this will keep the race from happening even if someone else calls keylime directly and does delete re-add

sounds good to me @nabilschear , thank you!

I grabbed all of the logs as my environment is really unstable / unpredictable and I am not sure if I still undstand how things should behave.
The commands used were:
keylime_tenant -v <IP_ADDRESS> -t <IP_ADDRESS> -u D432FBB3-D2F1-4A97-9EF7-75BD81C00000 -c delete
keylime_tenant -v <IP_ADDRESS> -t <IP_ADDRESS> -u D432FBB3-D2F1-4A97-9EF7-75BD81C00000 -f somefile -c add

Logs:
https://gist.github.com/lukehinds/c832697109b552c1d7776d96560b2766
https://gist.github.com/lukehinds/4341434641db5ec0b053e8f430b6db51
https://gist.github.com/lukehinds/4d4c22104e1984109934e598930790a8


Investigate PSYCO for performance improvement

This issue was originally created at: 2003-10-21 10:36:11.
This issue was reported by: stevenknight.
stevenknight said at 2003-10-21 10:36:11

Tracking this item, low priority:
http://sourceforge.net/projects/psyco

knuta said at 2006-01-10 04:25:40

IMHO this should be given a higher priority.  A simple
up-to-date check for some of our projects typically takes
more than 10 seconds.  In my experience, python scripts in
which string and table operations are the main bottleneck
will get around a 5 time performance boost from psyco.
BTW, I tried adding a simple psyco.full() call to the main
script, but it didn't work because of some f_locals /
f_globals issues.  Since I have no knowledge of scons
internals, I have no idea how hard it would be to get around
this.
Just my 5 cents. :)

issues@scons said at 2006-01-10 04:25:40

Converted from SourceForge tracker item 827670

gregnoel said at 2008-04-08 16:27:25

Triage to future; will revisit when resources permit.

Closing. The PSYCO project is dead. Replaced by pypy.


amplify function update failing to update dependsOn

Describe the bug
When updating a function and removing a layer, amplify-meta.json fails to remove Layer from dependsOn.
Amplify CLI Version
4.37.1
To Reproduce

create lambda and layer, attach layer to lambda.
attempt to amplify remove function, targeting the layer

Expected behavior
It should remove without error
Screenshots

Desktop (please complete the following information):

OS: [e.g. Mac/Windows/Ubuntu]
Node Version. 12

Additional context
:baby_chick:

@r0zar The error in "Red" mentions that this layer has a dependency on the Lambda:AppSyncAuthorizer function. If you remove the layer, it'll have unintended implications on your resource in the project.
Not able to get what the error you're trying to point out here is about.

You might want to remove your AppSyncAuthorizer function first and then remove the lambda layer resource.

I did this. That's why I've raised it as a bug.

I've verified by manually removing it from amplify-meta.json, it will successful remove.

Hi @r0zar
We have a PR that will fix this issue soon #6040

Closing this as a duplicate of #5845


Make sure that Telegram tests are not run against mock API by setting…

… CAMEL_QUARKUS_START_MOCK_BACKEND=false #1717
@llowinge this is an idea how we could do that. Done only for Telegram, but other tests could use the same pattern with the same env var CAMEL_QUARKUS_START_MOCK_BACKEND, so that you can set it once for all tests. WDYT?

c31201e fixed the formatting

@ppalaga I like the idea with possible switching between mock/external service. Would community accept the CAMEL_QUARKUS_START_MOCK_BACKEND  be set by default to false and in your tests would be switched to true ? In this manner we QE can be safe that the tests will really be run against external service even if property name CAMEL_QUARKUS_START_MOCK_BACKEND will change in future.

Would community accept the CAMEL_QUARKUS_START_MOCK_BACKEND be set by default to false

I discussed this with @jamesnetherton  and @oscerd and we prefer not to do that, because it would require all devs to export CAMEL_QUARKUS_START_MOCK_BACKEND=true
@jamesnetherton came with an alternative proposal that is implemented in 7042574
With this solution in place, you can do the following:

Run the tests with CAMEL_QUARKUS_START_MOCK_BACKEND=false
Count the occurrences of <artifactId>camel-quarkus-integration-test-support-mock-backend</artifactId> in integration-tests/*/pom.xml
Count the occurrences of Real backend will be used for in the output of all tests
The numbers should be non-null and equal (as long as you only run the JVM tests)

WDYT?

Superceded by https://github.com/apache/camel-quarkus/pull/1818


adds unit test config and initial spec

this is just a really basic sanity-check test. I'll add more specs as I refactor.
And, yes, it takes a ton of dependencies to setup decent tests ... ;)

Should we bump up a patch version or wait until you are done and go up a major version?

There aren't really any functional changes in the last two PRs, so it's not necessary, but it also wouldn't hurt ... There is some dead code removal and basic refactoring in the last one.


Add coverage test

Figure out how to get Istanbul or similar working with Babel.

Check my research, might be helpful https://github.com/nkbt/esnext-quickstart/issues/4
So after all coverage is working well, except there are still some minor issue with the way es6 classes are transpiled.

Thanks! I was planning to try that $ babel --auxiliary-comment "istanbul ignore next" hack. Nice job capturing that documentation. =)

I'm adding you as a contributor to this repo. Feel free to suggest changes or submit PRs for review if you like. =)

Added some scripts, basic test and istanbul.
Coverage looks like this:

To test if auxiliary-comment works, added import test from 'tape'; to index.js. Works fine. Extra text does not affect coverage.

I've also added branch coverage with bunch of extras for testing and coverage (karma, webpack, watch-mode) but then decided to keep it lean for the moment. Check it anyway.

also check this project : https://github.com/douglasduteil/isparta :  code coverage tool for ES6 using babel.

I tried it several months ago with not much luck. Need to check again in case of something good happened.
Btw coverage is working and in master now. Just does not look too pretty.

My ideal solution would be - take some super-modern-nightly-build browser with full support of modules, classes and all new shiny es6/7 features and run coverage on that. But there is no such browser yet.

Just found a perfect solution https://github.com/nkbt/esnext-quickstart/issues/4#issuecomment-112242629 with https://github.com/deepsweet/isparta-loader

Will update the code tonight

Cool.. Please use the pull request mechanism when you contribute code changes.

@ericelliott yes, will do.

:+1:
Thanks! =)

Thanks a lot for taking the time to figure out how to get the coverage report looking like it should! I really appreciate that effort.
I just have a few concerns with the PR... looking forward to your response so we can get this great work merged. =)

I will make changes accordingly. Communication might take some time due to different timezones (I am in Sydney, GMT+10)

Very cool. =)

:tada: :fire: :clap:


Func-1-2-3

My diagrams looks too easy, so, maybe, there're some problems with them

My diagrams looks too easy, so, maybe, there're some problems with them

this is not always the case

I think:

in this diagram should be exception about non-existing id
in this diagram write exception about id like above
in this diagram your exception is as result, but not as error, that will crush a program. You can write like that: "Search by parameters of a non-existent hotel"

changed

Hmm, it looks good, but somehow everything is two-block. You can just add in each case the process of sending information back to the user. In CheckForDeals case it seems to me that it doesn't work that way, every time the user opens the page with the ID of a certain hotel \ a request is sent to the database to display financial information about the hotel (also when searching by criteria) DB search for the new price or availability of special offers.

changed, but have some problems with this > a request is sent to the database to display financial information about the hotel (also when searching by criteria). Just don't know where and what exactly to put in this diagram


CHANGES.txt: most recent changes at the top of the file

Hi Pim,
I thinks it's more convenient for users if the most recent changes are displayed first, but I guess it's just a matter of personal preferences, so feel free to just close the pull request if you disagree.
Cheers,
Pascal

Agree.

Hi Pascal, I totally agree! Thanks. I'm want to release 2.2.3 today, so if you have more useful requests.... Regards, Pim

On 7 jan. 2016, at 20:07, Pascal Schumacher<EMAIL_ADDRESS>wrote:
Hi Pim,
I thinks it's more convenient for users if the most recent changes are displayed first, but I guess it's just a matter of personal preferences, so feel free to just close the pull request if you disagree.
Cheers,
Pascal
You can view, comment on, or merge this pull request online at:
https://github.com/opentelecoms-org/jsmpp/pull/21 https://github.com/opentelecoms-org/jsmpp/pull/21
Commit Summary
CHANGES.txt: most recent changes at the top of the file
File Changes
M CHANGES.txt https://github.com/opentelecoms-org/jsmpp/pull/21/files#diff-0 (100)
Patch Links:
https://github.com/opentelecoms-org/jsmpp/pull/21.patch https://github.com/opentelecoms-org/jsmpp/pull/21.patch
https://github.com/opentelecoms-org/jsmpp/pull/21.diff https://github.com/opentelecoms-org/jsmpp/pull/21.diff
—
Reply to this email directly or view it on GitHub https://github.com/opentelecoms-org/jsmpp/pull/21.

Hi Pim,
thanks for releasing 2.2.3. https://github.com/opentelecoms-org/jsmpp/pull/22 is my last pull request for today (of course you can also release 2.2.3 without it).
Cheers,
Pascal

Thanks. I just release 2.2.3, so it will show up in Maven Central shortly...

On 7 jan. 2016, at 20:34, Pascal Schumacher<EMAIL_ADDRESS>wrote:
Hi Pim,
thanks for releasing 2.2.3. #22 https://github.com/opentelecoms-org/jsmpp/pull/22 is my last pull request for today (of course you can also release 2.2.3 without it).
Cheers,
Pascal
—
Reply to this email directly or view it on GitHub https://github.com/opentelecoms-org/jsmpp/pull/21#issuecomment-169783329.


2853 fixes

Update documentation to reflect limitations on the ORACLE_SID for 11g/18c XE and 23ai Free editions, and limitations on the PDB_NAME in 23ai Free.

Thanks for your contribution, @oraclesean!


Separate scheduling API from dask implementation

xref #29
This PR moves the dask specific scheduling logic into a separate dask.py file, as a first step for adding support for alternative schedulers. (I'm particularly interested in supporting Apache Beam.)
The existing tests pass (with minor modifications), but the documentation still needs updating.
Notes:

I put staged_copy into a single function, but perhaps there are other generic methods (execute?) that would justify using a class?
Rechunked no longer inherits from dask.delayed.Delayed, and no longer has any dask specific logic at all. I think this is important for generic scheduler support, but it does means make it a little less reusable in larger pipelines. _delayed is currently a private attribute, but we should probably expose the scheduler equivalent of "delayed" objects in some way. I guess this is a use-case for class-based interface from the previous bullet.
Rechunked now always contains zarr arrays/groups rather than dask arrays. This makes the repr a little less informative, e.g., it no longer shows chunk size. This should probably be fixed before merging.
Will "two stage" copying always suffice? The interface I wrote for staged_copy supports any number of stages (in theory). That might be useful in the future, or it might be unnecessary complexity.
To verify that adding a new scheduler is not too painful, I should probably write at least a second example. I'll start with a naive "reference" scheduler in pure Python (this could go in the docs) and I'll think about adding a Beam implementation as well. Beam is perhaps a nice example because its execution models is so different from dask (based on higher level transforms like "map" rather than individual tasks).

Stephan it is really great to get this PR. I'm so happy that you have found time to help contribute to rechunker.
Overall, I really like your design. I will try to find time in the next few days (realistically, early next week) for a more thorough review. In the meantime, maybe @TomAugspurger can have a look?

Stephan it is really great to get this PR. I'm so happy that you have found time to help contribute to rechunker.

Thank you for releasing this tool in the first place! This fills an important niche for our current project, so I'm excited to be able to work with you on it.

Just a thought that occurred to me last night: it would be awesome to implement a prefect scheduler as well.

I implemented a second executor just using Python. It's only ~15 lines of code.
Looking at the two executors (Dask and Python), it felt like a class would be appropriate to codify the interface. So now we have a (very lightweight) Executor class.

This is ready for a full review.
I'm intentionally not documenting the Executor class interface in the docs for now, because I suspect it will change as we write the next few executors. For now, anyone who wants to explore it should be willing to dive into the source code, and ideally would submit their Executor upstream into Rechunker itself!


example error detecting

#include <mruby.h>
#include <mruby/compile.h>

int main(void)
{
  mrb_state *mrb = mrb_open();
  if (!mrb) { printf("Nie działa mruby\n"); }
  mrb_load_string(mrb, "100.times.each{|i| p (1 .0 / rand(10)) } ");
  mrb_close(mrb);
  return 0;
}

error is in line "100.times.each{|i| p (1 .0 / rand(10)) } " (extra space between 1.0)
How detecting error in function object and in running (divide zero)
I read a mruby code from user. I dont know what user write.

mrb->exc is set after mrb_load_string. The return value should be an exception object (or undef). See mrbgems/mruby-bin-mruby/tools/mruby/mruby.c for example:
    v = mrb_load_string(mrb, str);
    if (mrb->exc) { /* error happens */
      if (!mrb_undef_p(v)) {
        mrb_print_error(mrb); /* print exception object */
      }
      n = EXIT_FAILURE;
    }

Thanks for reply @matz but how check error in run separately?
 mrb_funcall(mrb, my_obj, "function", MRB_ARGS_NONE());
If I load good and without error but next in running?
This are intentions of autors 'How detecting error in function object and in running (divide zero)'

Are you asking how to check errors in mrb_funcal()? It's the same (if you are at the top-level).
Check mrb->exc after calling mrb_funcall().  If you call mrb_funcall() from the method, exceptions will jump to the nearest handler (rescue).


Allow to use regexp validator with arrays

Ex. this will work
params do
  requires :names, type: Array[String], regexp: /^[a-z]+$/
end

@akoltun Thanks for your pull request. This change looks like reasonable.
Please check my comment out.

This looks good, merging.


Fix #63: Cannot compile in subdirectory

This whole issue is pointlesss actually. If you want to build a specific module in Maven you do not cd into the directory you use the correct Maven -pl argument. So instead in the example, to juts build the functional tests module from the top level directory you just run:
mvn clean install -DskipTests -pl functional-tests

Current instructions at https://github.com/oracle/coherence-operator/tree/master/functional-tests/README.md don't work if this is not fixed.
Definitely convenient for developers to be able to do this in the sub-directory.


Leviton DZMX1 dimmer needs a delay before automagic refresh after being manually tapped

Describe the bug
When the physical button on the Levition DZMX1 dimmer switch is tapped, the device emits an ApplicationUpdateRequest.  zwave-js correctly interprets this as a state-change notification (logging "Node does not send unsolicited updates, refreshing actuator and sensor values...") and immediately performs a refreshValues() on the node, sending a MultilevelSwitchCCGet command.  But, the device immediately responds with an "Application Status" command indicating "busy, try again later", which is dropped as unimplemented by zwave-js.  Ten seconds later, the outstanding Get command times out.
The consequences of all this is:

The state of the device is not automatically updated after a physical button tap.
Attempts to communicate with the device after a physical button tap (e.g., to manually poll the value, or change the value via zwave) will be delayed for 10 seconds, waiting for the automatic Get command to timeout.

I figure the "busy, try again later" response from the device was a technique to avoid violating the old Lutron patent.  A refreshValues() on the node does succeed if it is delayed at least 950ms after receiving the ApplicationUpdateRequest.  So, it looks like Leviton implemented "Well, let's just require a 1 second delay, so that no one can possibly call this an 'instant update'."
To handle this, I propose inserting a device-specific delay before refreshValues() is called in this block https://github.com/zwave-js/node-zwave-js/blob/6cc6466e2ce050b95364b1740cc6d123af3cdba1/packages/zwave-js/src/lib/node/Node.ts#L1505-L1512  The delay is trivial, but what about the device-specific aspect?  Would it be reasonable to just add a new parameter to the schema for device configuration files?
(Another possibility would be to implement handling the "Application Status, busy, try again later" message, and automatically retrying the request.  But... even that would require knowing how long to wait before the retry, hence a device-specific parameter.)
Device information
Which device(s) is/are affected (make/model)?  Leviton DZMX1
What are the node IDs?  Node 2
Last Known Working Configuration

[x] New device
[ ] Previously working device (node-zwave-js)
[ ] Previously working device (other platform)

Installation information
How did you install node-zwave-js?

[ ] zwavejs2mqtt (latest) docker image
[ ] zwavejs2mqtt (dev) docker image
[ ] ioBroker.zwave2 adapter
[x] Pkg (npm install @zwave-js/server)
[ ] Manual Docker build

node-zwave-js branch:
zwavejs2mqtt branch:

[x] Manually built (to experiment with delays before refreshValues())
[ ] Other:

To Reproduce
Steps to reproduce the behavior:

Have DZMX1 in network.
Tap its button.
Within the next 10 seconds, try to do something with it over RF.
Look at logs.

Additional context
none
Logfile:
This logfile is from stock zwave-js (without any delays hacked in).  The timeline is:

Light is off, zwave-js-server started up.
Human taps physical button.
Device calls for help, zwave-js responds, device says "Naw, I am busy", zwave-js ignores that.
3-4 seconds later, human runs script asking zwave-js to poll the current level value and then set the value to zero.
10 seconds after the button tap, when zwave-js's original automatic Get request times out, the requested poll/set finally occur.

zwave-175924.log

I don't think waiting 1 second before polling is terrible here. After all - that is a workaround for legacy devices.
The 10s delay should no longer be a problem once #1518 is solved.

I don't think waiting 1 second before polling is terrible here. After all - that is a workaround for legacy devices.

I think I could come up with a 1-line PR for a fixed-duration delay.  :^)  But...
...I wouldn't want to introduce a regression.  If I have legacy devices that do not require the delay and they happily notify me of status changes within 10's of ms with the current code, and then after an update there is a weird inexplicable 1s delay, I would be confused at first and then peeved.
Looking back at #398 (the issue that led to this code in the first place), I wonder if @RoboPhred's switch requires a delayed refresh or not.  I'd guess "no", otherwise he/she would have reported that the value still wasn't getting updated.

The 10s delay should no longer be a problem once #1518 is solved.

(But with that alone, the refreshValues() would still timeout (albeit faster) and there would still be no update, right?)

Hm you got a point there. I'm not sure how frequently this will come up, but I guess we could add a compat flag manualValueRefreshDelay.

I can take a stab at adding this.
Any hints/clues as to where to start (a filename, a keyword --- some crate or barrel to form the basis of my cargo cult) would be appreciated.

There are a few parts to this.

define the compat flat:
1.1 JSON schema: maintenance\schemas\device-config.json, somewhere in the properties key, line 223
1.2 parsing: packages\config\src\CompatConfig.ts, class CompatConfig
1.3 Documentation: docs\development\config-files.md
use the compat flag (you know where), you can access it with this.deviceConfig.compat?.manualValueRefreshDelay.


TP2000-654 Create 403 error page

TP2000-654 Create 403 error page
Why
When a user attempts to access a part of the service to which they don't have the required access rights, a generic 403 error response is displayed without guidance on how to rectify the problem or raise an issue.
What
Adds custom 403 error page with a link to a form where the user can request access or raise an issue.
Checklist

Requires migrations? No
Requires dependency updates? No

Where do users currently see a 403? Could you add a test for at least one view in the "wild" to check that a user without permissions sees this page?

Codecov Report
Base: 93.11% // Head: 93.00% // Decreases project coverage by -0.10% :warning:

Coverage data is based on head (b636c18) compared to base (dcff105).
Patch coverage: 52.08% of modified lines in pull request are covered.

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #749      +/-   ##
==========================================
- Coverage   93.11%   93.00%   -0.11%
==========================================
  Files         369      370       +1
  Lines       23735    23789      +54
  Branches     1834     1836       +2
==========================================
+ Hits        22100    22125      +25
- Misses       1289     1318      +29
  Partials      346      346

Impacted Files
Coverage Δ

common/jinja2.py
84.44% <ø> (ø)

commodities/tests/test_migrations.py
28.12% <28.12%> (ø)

common/tests/test_views.py
100.00% <100.00%> (ø)

common/views.py
83.33% <100.00%> (+0.30%)
:arrow_up:

urls.py
63.63% <100.00%> (+3.63%)
:arrow_up:

measures/tests/test_migrations.py
27.77% <0.00%> (-1.32%)
:arrow_down:

conftest.py
93.46% <0.00%> (-1.02%)
:arrow_down:

reports/views.py
100.00% <0.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


the module ,apollo-client included by @vue/cli will warning.

Version
3.0.0-rc.10
Node and OS info
Node 8.11.3/ npm 6.2.0/OS 10.13.6
Steps to reproduce
after run  'npm i<EMAIL_ADDRESS>-g',
run  'npm ls -g --depth==0',
What is expected?
no warning
What is actually happening?
run 'npm i -g<EMAIL_ADDRESS>it's doesn't work

https://github.com/vuejs/vue-cli/issues/1991#issuecomment-409057682