functional-tests: hash difference in 'slow network', 'kill all members'

slow network tests immediately precedes kill all members test case, and we got different hash error in functional tests.
Logs coming...

2016-04-17 02:31:25.479332 I | etcd-tester: [round#1866 case#10] checking current storage hashes failed (map[<IP_ADDRESS>:2379:3125174186 <IP_ADDRESS>:2379:2549669347 <IP_ADDRESS>:2379:2549669347])
2016-04-17 02:31:25.479346 I | etcd-tester: [round#1866 case#10] cleaning up...
2016-04-17 02:31:27.687000 I | etcd-tester: [round#1866] compacting storage at 864874280 (current revision 864884280)
2016-04-17 02:31:28.734299 I | etcd-tester: [round#1866] compactKV error (rpc error: code = 11 desc = etcdserver: storage: required revision is a future revision)
2016-04-17 02:31:28.734348 I | etcd-tester: [round#1866 case#0] cleaning up...

Will redeploy testing cluster with more logs...

# TESTER
2016-04-19 05:22:53.712452 I | etcd-tester: [round#430 case#10] start failure slow down all members' network
2016-04-19 05:22:53.712487 I | etcd-tester: [round#430 case#10] start injecting failure...
2016-04-19 05:22:56.746705 I | etcd-tester: [round#430 case#10] injected failure
2016-04-19 05:22:56.746742 I | etcd-tester: [round#430 case#10] start recovering failure...
2016-04-19 05:23:01.927553 I | etcd-tester: [round#430 case#10] recovered failure
2016-04-19 05:23:01.927599 I | etcd-tester: [round#430 case#10] canceling the stressers...
2016-04-19 05:23:01.930186 I | etcd-tester: [round#430 case#10] canceled stressers
2016-04-19 05:23:01.930214 I | etcd-tester: [round#430 case#10] checking current revisions...
2016-04-19 05:23:03.111432 I | etcd-tester: [round#430 case#10] all members are consistent with current revisions
2016-04-19 05:23:03.111471 I | etcd-tester: [round#430 case#10] checking current storage hashes...
2016-04-19 05:23:03.111496 I | etcd-tester: [round#430 case#10] checking current storage hashes failed (map[<IP_ADDRESS>:2379:3034809049 <IP_ADDRESS>:2379:3034809049 <IP_ADDRESS>:2379:48445316])
2016-04-19 05:23:03.111546 I | etcd-tester: [round#430 case#10] cleaning up...
2016-04-19 05:23:04.382879 I | etcd-tester: [round#430] compacting storage at 185959998 (current revision 185969998)
2016-04-19 05:23:05.923550 I | etcd-tester: [round#430] compactKV error (rpc error: code = 11 desc = etcdserver: storage: required revision is a future revision)
2016-04-19 05:23:05.923598 I | etcd-tester: [round#430 case#0] cleaning up...

# AGENT1
2016-04-19 05:23:00.392273 W | raft: AGENT1 stepped down to follower since quorum is not active
2016-04-19 05:23:00.392340 I | raft: AGENT1 became follower at term 4805
2016-04-19 05:23:00.392355 I | raft: raft.node: AGENT1 lost leader AGENT1 at term 4805
2016/04/19 05:23:00 grpc: Server.processUnaryRPC failed to write status: stream error: code = 4 desc = "context deadline exceeded"
2016-04-19 05:23:01.844223 I | raft: AGENT1 [term: 4805] received a MsgVote message with higher term from AGENT3 [term: 4806]
2016-04-19 05:23:01.844281 I | raft: AGENT1 became follower at term 4806
2016-04-19 05:23:01.844303 I | raft: AGENT1 [logterm: 4805, index: 186040347, vote: 0] rejected vote from AGENT3 [logterm: 4805, index: 186040076] at term 4806
2016-04-19 05:23:01.844327 I | raft: AGENT1 [term: 4806] received a MsgVote message with higher term from AGENT3 [term: 4807]
2016-04-19 05:23:01.844341 I | raft: AGENT1 became follower at term 4807
2016-04-19 05:23:01.844351 I | raft: AGENT1 [logterm: 4805, index: 186040347, vote: 0] rejected vote from AGENT3 [logterm: 4805, index: 186040076] at term 4807
2016-04-19 05:23:01.844381 I | raft: AGENT1 [term: 4807] received a MsgVote message with higher term from AGENT3 [term: 4808]
2016-04-19 05:23:01.844392 I | raft: AGENT1 became follower at term 4808
2016-04-19 05:23:01.844402 I | raft: AGENT1 [logterm: 4805, index: 186040347, vote: 0] rejected vote from AGENT3 [logterm: 4805, index: 186040076] at term 4808
2016-04-19 05:23:01.845765 I | raft: found conflict at index 186040077 [existing term: 4805, conflicting term: 4808]
2016-04-19 05:23:01.845778 I | raft: replace the unstable entries from index 186040077
2016-04-19 05:23:01.845785 I | raft: raft.node: AGENT1 elected leader AGENT3 at term 4808

# AGENT2
2016-04-19 05:22:58.872186 I | raft: AGENT2 is starting a new election at term 4805
2016-04-19 05:22:58.872270 I | raft: AGENT2 became candidate at term 4806
2016-04-19 05:22:58.872281 I | raft: AGENT2 received vote from AGENT2 at term 4806
2016-04-19 05:22:58.872305 I | raft: AGENT2 [logterm: 4805, index: 186040065] sent vote request to AGENT1 at term 4806
2016-04-19 05:22:58.872317 I | raft: AGENT2 [logterm: 4805, index: 186040065] sent vote request to AGENT3 at term 4806
2016-04-19 05:22:58.872327 I | raft: raft.node: AGENT2 lost leader AGENT1 at term 4806
2016-04-19 05:23:00.672176 I | raft: AGENT2 is starting a new election at term 4806
2016-04-19 05:23:00.672242 I | raft: AGENT2 became candidate at term 4807
2016-04-19 05:23:00.672253 I | raft: AGENT2 received vote from AGENT2 at term 4807
2016-04-19 05:23:00.672264 I | raft: AGENT2 [logterm: 4805, index: 186040065] sent vote request to AGENT1 at term 4807
2016-04-19 05:23:00.672274 I | raft: AGENT2 [logterm: 4805, index: 186040065] sent vote request to AGENT3 at term 4807
2016-04-19 05:23:00.674823 I | raft: AGENT2 [term: 4807] ignored a MsgVote message with lower term from AGENT3 [term: 4806]
2016-04-19 05:23:00.674853 I | raft: AGENT2 [logterm: 4805, index: 186040065, vote: AGENT2] rejected vote from AGENT3 [logterm: 4805, index: 186040076] at term 4807
2016-04-19 05:23:00.674880 I | raft: AGENT2 received vote rejection from AGENT3 at term 4807
2016-04-19 05:23:00.674889 I | raft: AGENT2 [quorum:2] has received 1 votes and 1 vote rejections
2016-04-19 05:23:00.770100 I | raft: AGENT2 [term: 4807] ignored a MsgApp message with lower term from AGENT1 [term: 4805]
2016-04-19 05:23:00.770111 I | raft: AGENT2 [term: 4807] ignored a MsgApp message with lower term from AGENT1 [term: 4805]
2016-04-19 05:23:01.844943 I | raft: AGENT2 [term: 4807] received a MsgVote message with higher term from AGENT3 [term: 4808]
2016-04-19 05:23:01.844992 I | raft: AGENT2 became follower at term 4808
2016-04-19 05:23:01.845018 I | raft: AGENT2 [logterm: 4805, index: 186040065, vote: 0] voted for AGENT3 [logterm: 4805, index: 186040076] at term 4808
2016-04-19 05:23:01.846918 I | raft: raft.node: AGENT2 elected leader AGENT3 at term 4808
2016-04-19 05:23:02.613149 I | raft: AGENT2 [term: 4808] ignored a MsgHeartbeat message with lower term from AGENT1 [term: 4805]
2016-04-19 05:23:03.832310 N | osutil: received terminated signal, shutting down...

# AGENT3
2016-04-19 05:22:59.041872 I | raft: AGENT3 is starting a new election at term 4805
2016-04-19 05:22:59.041971 I | raft: AGENT3 became candidate at term 4806
2016-04-19 05:22:59.042008 I | raft: AGENT3 received vote from AGENT3 at term 4806
2016-04-19 05:22:59.042043 I | raft: AGENT3 [logterm: 4805, index: 186040076] sent vote request to AGENT1 at term 4806
2016-04-19 05:22:59.042055 I | raft: AGENT3 [logterm: 4805, index: 186040076] sent vote request to AGENT2 at term 4806
2016-04-19 05:22:59.042080 I | raft: raft.node: AGENT3 lost leader AGENT1 at term 4806
2016-04-19 05:22:59.992769 I | raft: AGENT3 [term: 4806] ignored a MsgHeartbeat message with lower term from AGENT1 [term: 4805]
2016-04-19 05:22:59.992810 I | raft: AGENT3 [term: 4806] ignored a MsgHeartbeat message with lower term from AGENT1 [term: 4805]
2016-04-19 05:23:00.292536 I | raft: AGENT3 [term: 4806] ignored a MsgHeartbeat message with lower term from AGENT1 [term: 4805]
2016-04-19 05:23:00.541819 I | raft: AGENT3 is starting a new election at term 4806
2016-04-19 05:23:00.541891 I | raft: AGENT3 became candidate at term 4807
2016-04-19 05:23:00.541909 I | raft: AGENT3 received vote from AGENT3 at term 4807
2016-04-19 05:23:00.541937 I | raft: AGENT3 [logterm: 4805, index: 186040076] sent vote request to AGENT1 at term 4807
2016-04-19 05:23:00.541956 I | raft: AGENT3 [logterm: 4805, index: 186040076] sent vote request to AGENT2 at term 4807
2016-04-19 05:23:00.673022 I | raft: AGENT3 [term: 4807] ignored a MsgVote message with lower term from AGENT2 [term: 4806]
2016-04-19 05:23:00.673077 I | raft: AGENT3 [logterm: 4805, index: 186040076, vote: AGENT3] rejected vote from AGENT2 [logterm: 4805, index: 186040065] at term 4807
2016-04-19 05:23:00.673902 I | raft: AGENT3 received vote rejection from AGENT2 at term 4807
2016-04-19 05:23:00.673917 I | raft: AGENT3 [quorum:2] has received 1 votes and 1 vote rejections
2016-04-19 05:23:01.841874 I | raft: AGENT3 is starting a new election at term 4807
2016-04-19 05:23:01.841943 I | raft: AGENT3 became candidate at term 4808
2016-04-19 05:23:01.841955 I | raft: AGENT3 received vote from AGENT3 at term 4808
2016-04-19 05:23:01.841969 I | raft: AGENT3 [logterm: 4805, index: 186040076] sent vote request to AGENT1 at term 4808
2016-04-19 05:23:01.841980 I | raft: AGENT3 [logterm: 4805, index: 186040076] sent vote request to AGENT2 at term 4808
2016-04-19 05:23:01.845284 I | raft: AGENT3 received vote from AGENT2 at term 4808
2016-04-19 05:23:01.845322 I | raft: AGENT3 [quorum:2] has received 2 votes and 0 vote rejections
2016-04-19 05:23:01.845346 I | raft: AGENT3 became leader at term 4808
2016-04-19 05:23:01.845359 I | raft: raft.node: AGENT3 elected leader AGENT3 at term 4808
2016-04-19 05:23:01.845765 I | raft: AGENT3 [term: 4808] ignored a MsgVoteResp message with lower term from AGENT1 [term: 4806]
2016-04-19 05:23:01.846498 I | raft: AGENT3 [term: 4808] ignored a MsgVoteResp message with lower term from AGENT1 [term: 4807]

Right after test failure recovery, we get a new leader and revision matches but hash doesn't.

So agent1 hash doesn't match with agent2 and agent3. I checked the data dump logs:
# AGENT1
4808	 186040344	norm	method=SYNC time="2016-04-18 22:23:02.241813971 -0700 PDT"
4808	 186040345	norm	method=SYNC time="2016-04-18 22:23:02.741829546 -0700 PDT"
...
4808	 186040363	norm	ID:13685124444814249823 put:<key:"foo82857" value:"hsmkkhuhheesapepirmcvjxecrjomaxxdpsyjymyuwzzobaltcjsbedvwiijgpxbbtxmoozxortmedmvfauqgoijsadoyjwdxwjz" >
4808	 186040364	norm	ID:13685124444814249824 put:<key:"foo3054" value:"ehanduemfqyzjuwjhywsoofovlacpnqyoeacolmhcnmqfngmqqisvpjcrverghftchzasuvecwiomzlkgiodfkgpchwmpivwcusl" >
4808	 186040365	norm	ID:13685124444814249825 put:<key:"foo175188" value:"xhizehpcgksnrwofhubpdbxncufvxpjeqbgnfzznlonpawlibbyarfkqonfbspxamrjedbfpgpajtmtifouxagwzfbajashsyezu" >

# AGENT2
4808	 186040344	norm	method=SYNC time="2016-04-18 22:23:02.241813971 -0700 PDT"
4808	 186040345	norm	method=SYNC time="2016-04-18 22:23:02.741829546 -0700 PDT"
...
4808	 186040363	norm	ID:13685124444814249823 put:<key:"foo82857" value:"hsmkkhuhheesapepirmcvjxecrjomaxxdpsyjymyuwzzobaltcjsbedvwiijgpxbbtxmoozxortmedmvfauqgoijsadoyjwdxwjz" >
4808	 186040364	norm	ID:13685124444814249824 put:<key:"foo3054" value:"ehanduemfqyzjuwjhywsoofovlacpnqyoeacolmhcnmqfngmqqisvpjcrverghftchzasuvecwiomzlkgiodfkgpchwmpivwcusl" >
4808	 186040365	norm	ID:13685124444814249825 put:<key:"foo175188" value:"xhizehpcgksnrwofhubpdbxncufvxpjeqbgnfzznlonpawlibbyarfkqonfbspxamrjedbfpgpajtmtifouxagwzfbajashsyezu" >
4808	 186040366	norm	method=SYNC time="2016-04-18 22:23:03.24179165 -0700 PDT"
4808	 186040367	norm	method=SYNC time="2016-04-18 22:23:03.741791123 -0700 PDT"

# AGENT3
4808	 186040344	norm	method=SYNC time="2016-04-18 22:23:02.241813971 -0700 PDT"
4808	 186040345	norm	method=SYNC time="2016-04-18 22:23:02.741829546 -0700 PDT"
...
4808	 186040363	norm	ID:13685124444814249823 put:<key:"foo82857" value:"hsmkkhuhheesapepirmcvjxecrjomaxxdpsyjymyuwzzobaltcjsbedvwiijgpxbbtxmoozxortmedmvfauqgoijsadoyjwdxwjz" >
4808	 186040364	norm	ID:13685124444814249824 put:<key:"foo3054" value:"ehanduemfqyzjuwjhywsoofovlacpnqyoeacolmhcnmqfngmqqisvpjcrverghftchzasuvecwiomzlkgiodfkgpchwmpivwcusl" >
4808	 186040365	norm	ID:13685124444814249825 put:<key:"foo175188" value:"xhizehpcgksnrwofhubpdbxncufvxpjeqbgnfzznlonpawlibbyarfkqonfbspxamrjedbfpgpajtmtifouxagwzfbajashsyezu" >
4808	 186040366	norm	method=SYNC time="2016-04-18 22:23:03.24179165 -0700 PDT"
4808	 186040367	norm	method=SYNC time="2016-04-18 22:23:03.741791123 -0700 PDT"

AGENT1 is short of two entries at the end.
(data log at https://console.cloud.google.com/storage/browser/etcd-issues-5121-00)

@gyuho Well... First sync does not change the hash state of the backend. Second, it might just because the log entries have not been replicated through leader to follower yet.

Can you share the link of the data dirs of all the members?

nvm. i found it in your previous comment.

2016-04-19 05:23:05.923550 I | etcd-tester: [round#430] compactKV error (rpc error: code = 11 desc = etcdserver: storage: required revision is a future revision)

I still feel this is the root cause. I can double check.

Yeah it's in cloud storage. Will take a look too. Thanks!

After some investigation, we found that even after the tester thinks it cancelled all the stressors, there are still put requests going into etcd. We need to ensure that there is no pending requests before getting hashes and compare them.

Once https://github.com/coreos/etcd/pull/5145 gets merged, I will redeploy the testing cluster to investigate more about lingering stressers. In the meantime, I will analyze db files to find the exact index of different hash value, which will explain how many commits were off from each node.

As @xiang90 said, some stress requests were still in progress, causing the hash differences. Here's another example:
2016-04-22 01:01:57.065400 I | etcd-tester: [round#131 case#10] all members are consistent with current revisions
[revisions: map[
<IP_ADDRESS>:2379:57467585
<IP_ADDRESS>:2379:57467585
<IP_ADDRESS>:2379:57467585
]]

2016-04-22 01:01:57.065438 I | etcd-tester: [round#131 case#10] checking current storage hashes...
2016-04-22 01:01:57.065449 I | etcd-tester: [round#131 case#10] checking current storage hashes failed
[hashes: map[
<IP_ADDRESS>:2379 :<PHONE_NUMBER>
<IP_ADDRESS>:2379 :<PHONE_NUMBER>
<IP_ADDRESS>:2379 :<PHONE_NUMBER>
]]

2016-04-22 01:01:57.065470 I | etcd-tester: [round#131 case#10] cleaning up...

Question:
Final hash is<PHONE_NUMBER> and different hash is<PHONE_NUMBER>.
Need to find the hash<PHONE_NUMBER>

<IP_ADDRESS>:2379 with hash value<PHONE_NUMBER> and revision 57467585 was the leader
The actual snapshot db file shows all 3 nodes at the current revision have the same hash values of<PHONE_NUMBER>
In order to confirm that<PHONE_NUMBER> was the hash value of nodes waiting to be synced with the leader, we back-track the db file by decreasing the consistent index and deleting key-value entry one by one (delete when the request method is not SYNC), starting from the the revision 57467585
And we found at revision 57467549 and consistent index 57488881, the database has the hash value of<PHONE_NUMBER>
This confirms that there's nothing wrong in etcd or database side. We just got the hash value of previous rounds
This only happens with case #10 which slows down the network of all members.
We will introduce some wait time to give enough time to recover from the slow network


Clarity on if it's safe to call next() after send()

Branching https://github.com/expressjs/serve-static/issues/136#issuecomment-661169863 since it's more general than express-static
The use case I'm trying to catch is when the client aborts. I found in my unit testing it's actually hard to get express/https.server to shutdown cleanly i.e. if the client aborts while the request is asynchronously talking to the database, https.server.close won't wait for that to finish/won't wait for any in progress async middleware to finished before the "close" callback is called.
I'm trying to add middleware at the beginning and end of each request to track what's still in progress independent of the connection being severed so I can delay fully closing if there's any middleware that might still use external resources. on-finished isn't what I'm looking for because if the client closes the connection in the middle of a request using async middleware "close" will be fired before all the middleware has finished.
I'm trying to find official expressjs documentation saying it's unsafe to process middleware after sending the response (as long as you don't try to edit anything about the request object). I just realized I got the idea from this non-official article I know saying "it works for me" is not a sign that it's a supported use case but it is working so far.
Should this be resolved by:

adding explicit wording in the documentation saying calling next() after end() is wrong and it should throw an error. I know I'm being pedantic but the docs currently say

If the current middleware function does not end the request-response cycle, it must call next() to pass control to the next middleware function.

without being specific about about what's considered the full request-response cycle.

making this a supported use case for people who don't want to block sending a response for server specific work tied to the middleware chain (and not specifically the connection) and adding tests to spot regressions.

adding a dummy middleware that just calls res.end() and acts like 2) but with a delay and using https://github.com/expressjs/serve-static/issues/136#issuecomment-661125380 for serve-static.

I'm a fan of 2. The only other option I see is adding checks after every promise in middleware to see if the request is active but that doesn't catch issues if the request is closed mid-await.

To clarify, yes, I tried  listening to "finish" and "close".  Given:
async function myDatabaseMiddleWare(req, res, next) {
   await databaseQuery1();
   await databaseQuery2();
  next();
}

if server.close is called between databaseQuery1 and databaseQuery2, the "close" even will fire for req but the middleware will still try to call databaseQuery2. Express doesn't try to stop the rest of the middleware function.
This means server.close will return before its safe to actually tear down the database connection because it will still be used for databaseQuery2.  I'm trying to catch this with the (super simplified) code below.
const requestsStillRunningMiddleware = 0;

// at the start of the middleware stack
app.use((req, res, next)=>{
  ++requestsStillRunningMiddleware;
  next();
})

// at the end of the middleware stack
app.use((req, res, next)=>{
  --requestsStillRunningMiddleware;
  if(requestsStillRunningMiddleware=== 0){
      // signal that it's safe to clean up resources used in middleware.
   }
})

But it requires all middleware to call next, including the one that actually calls send().
Yes, I am trying to implement a graceful shutdown system. There are some examples on the Express website: http://expressjs.com/en/advanced/healthcheck-graceful-shutdown.html. However, Terminus and Lightship require an external service to ping your server, which is overkill for an integration test. All of them including http-terminator keep track of connections between the client and the server. They do not keep track of connections between the server and backend services (postgres/redis/etc) except through explicit health checks. But this doesn't allow you to make sure no one is using those resources before you consider the server "closed".


incorporating Context Vectors?

How can I use Context Vectors as in https://github.com/salesforce/cove?

Check this repo https://github.com/matthew-z/R-net


fix: enabling the integration test on the correct kokoro target

When moving the  to individual profile, I accidentally moved it to a incorrect target instead of beamIntegrationTest.
This PR is to fix this and run Beam integration tests in presubmit and nightly.

I dont think this is the correct way to go about this.
java-shared-config disables IT tests by default:
https://github.com/googleapis/java-shared-config/blob/393ba5669675afb44740fdc45c8110feccf0b3b6/pom.xml#L65
This is done because we want to be able to build the project immediately after git clone and integration tests require the developer to set up cloud resources and creds.
To actually run the integration tests, java-shared-config provides this profile:
https://github.com/googleapis/java-shared-config/blob/393ba5669675afb44740fdc45c8110feccf0b3b6/pom.xml#L651
Which can be activated by -Penable-integration-tests. I believe the intention is to enable that profile during build similar to how java-bigtable does it:
https://github.com/googleapis/java-bigtable/blob/2821902b34ae04596771a9fc5e2b62d5c24a7253/.kokoro/build.sh#L64
Would you mind updating build.sh instead?


feat(config): add ZSE70 device file; Add additional templates to Zooz template file

Add Zooz ZSE70

I'm at loss in seeing where the lint test is failing.


[WIP] X.509

TODOs:

tests
github workflow
finish README

@randombit this is still WIP but it might make sense to review rust/x509/src/x509_example_rust/src/wasm_only.rs and double-check locally that the produced certificates verify and are well-formed.


Use OpenEO for fetching data?

OpenEO provides a unified API for getting data from various data providers with different clients. It is especially tailored to data providers with (batch) processing capabilities, i.e. subselect/filter/aggregate/manipulate the data before fetching/downloading it. It works, among others, with google earth engine, so a lot of data should be available in this way. Note that some of these processing operations may have costs associated with them, so that's something we have to figure out.

Interesting blog post that sets out the philosophy: https://r-spatial.org/2016/11/29/openeo.html

ESA has an openEO cloud. You need a subscription, but there are several free plans that could be interesting. https://openeo.cloud/

Also see https://eodag.readthedocs.io/en/stable/

Concluded that openEO is nice, but the data providers that are implementing it are not yet mature enough to be used reliably


[NFR]: Add EndsWith trait

Add EndsWith trait to check whether a string ends with a particular string

Resolved in bb3ebf14a86809998457f34a33f2687b002a7d1c


Add test if not yet present in dictionary.

@farlee2121 I could be wrong but this seemed to help to show the new tests in my project.
Reasoning: the newly added test won't be in the existing dictionary and the check needs to be negated.
I could be wrong, please let me know if this makes sense.
Fixes https://github.com/ionide/ionide-vscode-fsharp/issues/1997

Fixes https://github.com/ionide/ionide-vscode-fsharp/issues/1997

If I remember right, that condition is adding tests to the explorer when we've previously been able to match a non-standard location (e.g. a referenced, but not directly nested testList in Expecto). The displaced fragment cache is constructed beforehand isn't being modified in mergeCodeUpdates.
I ran some testing and something is definitely off about the explore behavior on code update. Renaming methods is causing them to disappear. I am able to add and remove test methods and see live updates though.

Similarly, when I set "FSharp.TestExplorer.AutoDiscoverTestsOnLoad" to false, my tests are no longer discovered at all. Even though I see the server sending the notifications with the correct information.
The reason I want FSharp.TestExplorer.AutoDiscoverTestsOnLoad to be false is that it runs all unit tests when I open my solution. I get why this is useful in some cases but for the Fantomas tests, all tests can be discovered via the AST quite accurately.

If I'm understanding right, you want to discover tests on load via code analysis alone and not running tests.
That works against the base assumptions of the test explorer. A lot of the work I did was to re-orient the test explorer around test results since the code analysis results were causing flakiness (i.e. incorrectly located tests which broke the explorer).

If I'm understanding right, you want to discover tests on load via code analysis alone and not running tests.

Yes, this used to work fine for me when I originally ported the test explorer code from Neptune.
Right now I'm a bit annoyed that all my tests are needlessly run when I open vscode (with setting on) or when I use the refresh action (with setting off).
The notifications work fine for me, this should also be possible to work with.

My concern is that building from code analysis results would re-introduce the issues outlined in #1874.  Test results aren't always the fastest, but they are a consistently reliable method for constructing the test explorer.
I'm open to suggestions.

I would go for some middle ground here.
Not all projects face the issue of #1874, so there should be an opt-out.
In some codebases, it really is that simple that every [<Test>] equals a runnable test entry.
My suggestion would be to always process the incoming notifications. This seems to be off when "FSharp.TestExplorer.AutoDiscoverTestsOnLoad": false I suspect. And I would assume the server sends them anyway.

Building the test explorer from code analysis isn't a matter of leaving it on when result-based updates are off, it'd require a separate mode. Currently, code analysis is not used as a primary truth source. It relies on the baseline provided by test results to prevent the unusable states described in #1874 (i.e. where run all doesn't work anymore).
Though, the explorer now should be able to run when non-existent tests are selected and be refreshed from test results if it gets off. Perhaps a "trust all code analysis" mode wouldn't get the explorer into an unrecoverable state.

I've been digging a bit to understand how Visual Studio manages test location discovery.
I'm still getting a handle on it, but they seem to manage the whole discovery process through the Adapter Extensibility model.
Maybe that's the way forward here too? Then we're capable of anything VS is capable of and we can offload discovery to the adapters and hopefully some of the orchestration to vstest.
I'll keep digging.

I've not gotten as much research into this as I'd hoped, but I did poke around VS Test and found that this is the package meant to smooth integration for editors: Microsot.TestPlatform.TranslationLayer.
They have an example and a playground for internal developers that demonstrate it's use.


[Bug]: Switching from npm_install to npm_translate_lock and adapting to js_* rules causing Module not found

What happened?
Switching from npm_install to:
npm_translate_lock(
    name = "ops_web_node_modules",
    pnpm_lock = "//eddie/services/ops_web:pnpm-lock.yaml",
    npmrc = "//eddie/services/ops_web:.npmrc",
    verify_node_modules_ignored = "//:.bazelignore",
)

I'm using pnpm, and running
bazel run -- @pnpm//:pnpm install --dir $PWD --lockfile-only

the packages.json file is
{
  "name": "my-app",
  "version": "0.1.0",
  "private": true,
  "dependencies": {
    "@babel/preset-react": "^7.24.7",
    "@material-ui/core": "^4.8.3",
    "@material-ui/icons": "^4.5.1",
    "@material-ui/lab": "^4.0.0-alpha.42",
    "@rooks/use-interval": "^3.6.0",
    "@rooks/use-outside-click": "^3.6.0",
    "@rooks/use-visibility-sensor": "^3.6.0",
    "@trivago/prettier-plugin-sort-imports": "^4.3.0",
    "ag-grid-community": "^30.2.1",
    "ag-grid-enterprise": "^30.2.1",
    "ag-grid-react": "^30.2.1",
    "ansi-to-react": "^6.1.5",
    "axios": "^1.7.2",
    "core-js-compat": "^3.37.1",
    "css-loader": "^3.2.0",
    "cytoscape": "^3.28.1",
    "cytoscape-klay": "^3.1.4",
    "date-fns": "^2.9.0",
    "eslint": "^8.57.0",
    "eslint-config-prettier": "^8.6.0",
    "eslint-plugin-prettier": "^4.2.1",
    "eslint-plugin-sort-keys-fix": "^1.1.2",
    "file-saver": "1.3.8",
    "http-proxy-middleware": "^0.21.0",
    "http-server": "^14.1.1",
    "jsonwebtoken": "^9.0.2",
    "lodash": "^4.17.21",
    "material-ui-dropzone": "^3.5.0",
    "moment": "^2.30.1",
    "notistack": "^3.0.1",
    "papaparse": "^5.4.1",
    "prettier": "^2.8.3",
    "prop-types": "^15.7.2",
    "query-string": "^6.13.1",
    "react": "^16.12.0",
    "react-cytoscapejs": "^1.2.1",
    "react-datepicker": "^4.25.0",
    "react-dom": "^16.12.0",
    "react-is": "^16.13.1",
    "react-json-editor-ajrm": "^2.5.14",
    "react-router-dom": "^5.3.4",
    "react-scripts": "5.0.1",
    "react-simple-animate": "^3.5.2",
    "react-textarea-autosize": "^7.1.2",
    "sass": "^1.75.0",
    "styled-components": "^5.3.11"
  },
  "overrides": {
    "react-refresh": "0.14.0"
  },
  "scripts": {
    "build": "react-scripts build",
  },
  "eslintConfig": {
    "extends": "react-app"
  },
  "babel": {
    "presets": [
      "react-app",
      "@babel/preset-react"
    ]
  },
  "browserslist": {
    "production": [
      ">0.2%",
      "not dead",
      "not op_mini all"
    ],
    "development": [
      "last 1 chrome version",
      "last 1 firefox version",
      "last 1 safari version"
    ]
  }
}

When running build it fails due to
Module not found: Error: Can't resolve '@material-ui/styles' in '/private/var/tmp/_bazel_eddie/475d098e755821c69bb3ea8af0f5264b/sandbox/darwin-sandbox/2/execroot/eddie/bazel-out/darwin-opt/bin/eddie/services/ops_web/src/components/App'

When I'm adding the missing packages to the file and lock it again, it works. Missing packages example:
    "@material-ui/styles": "^4.11.5",
    "styled-components": "^5.3.11",
    "postcss": "^8.4.4",
    "postcss-flexbugs-fixes": "^5.0.2",
    "postcss-loader": "^6.2.1",
    "postcss-normalize": "^10.0.1",
    "postcss-preset-env": "^7.0.1",
    "prompts": "^2.4.2",

The bigger problem is that these packages are not imported in the code at all. So I'm not sure what is happening here, and why after switching to pnpm and using js_* it requires packages it not used before.
My BUILD file look like below:
load("@ops_web_node_modules//:defs.bzl", "npm_link_all_packages")
load("@ops_web_node_modules//eddie/services/ops_web:react-scripts/package_json.bzl", react_scripts_bin = "bin")

npm_link_all_packages(name = "node_modules")

react_scripts_bin.react_scripts_binary(
    name = "react_scripts",
    chdir = package_name(),
)

js_library(
    name = "lib",
    srcs = glob([
        "src/**",
        "public/**",
    ]) + [
        ":node_modules",
        "package.json",
        ".eslintrc.json",
        ".env",
        ".prettierrc",
    ],
)

js_run_binary(
    name = name,
    srcs = [":lib"],
    args = ["build"],
    chdir = package_name(),
    mnemonic = "BuildOpsWeb",
    out_dirs = ["build"],
    tool = ":react_scripts",
    visibility = ["//visibility:public"],
)

Version
Development (host) and target OS/architectures: Linux
Output of bazel --version: bazel 6.5.0
Version of the Aspect rules, or other relevant rules from your
WORKSPACE or MODULE.bazel file:
http_archive(
    name = "aspect_rules_js",
    sha256 = "2cfb3875e1231cefd3fada6774f2c0c5a99db0070e0e48ea398acbff7c6c765b",
    strip_prefix = "rules_js-1.42.3",
    url = "https://github.com/aspect-build/rules_js/releases/download/v1.42.3/rules_js-v1.42.3.tar.gz",
)

Language(s) and/or frameworks involved:
How to reproduce
I was following this example to switch to `js_*` rules with `npm_translate_lock`
https://github.com/bazelbuild/examples/blob/main/frontend/react/BUILD.bazel

Any other information?
No response

I have created a simple react-script app following https://create-react-app.dev/docs/getting-started
And wrapped it with bazel..
package.json:
{
  "name": "my-app",
  "version": "0.1.0",
  "private": true,
  "dependencies": {
    "@testing-library/jest-dom": "^5.17.0",
    "@testing-library/react": "^13.4.0",
    "@testing-library/user-event": "^13.5.0",
    "react": "^18.3.1",
    "react-dom": "^18.3.1",
    "react-scripts": "5.0.1",
    "web-vitals": "^2.1.4"
  },
  "scripts": {
    "start": "react-scripts start",
    "build": "react-scripts build",
    "test": "react-scripts test",
    "eject": "react-scripts eject"
  },
  "eslintConfig": {
    "extends": [
      "react-app",
      "react-app/jest"
    ]
  },
  "browserslist": {
    "production": [
      ">0.2%",
      "not dead",
      "not op_mini all"
    ],
    "development": [
      "last 1 chrome version",
      "last 1 firefox version",
      "last 1 safari version"
    ]
  }
}

Same error for missing dependencies:
bazel build eddie/services/my_app
INFO: Build option --check_visibility has changed, discarding analysis cache.
INFO: Analyzed target //eddie/services/my_app:my_app (1333 packages loaded, 11983 targets configured).
INFO: Found 1 target...
ERROR: /Users/eddie/code/eddie/eddie/services/my_app/BUILD:12:17: BuildMyAppWeb eddie/services/my_app/build failed: (Exit 1): react_scripts.sh failed: error executing command (from target //eddie/services/my_app:my_app) bazel-out/darwin-opt-exec-2B5CBBC6/bin/eddie/services/my_app/react_scripts.sh build

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
Creating an optimized production build...
Failed to compile.

Loading PostCSS "postcss-flexbugs-fixes" plugin failed: Cannot find module 'postcss-flexbugs-fixes'
Require stack:

It looks like a bug.

Ok, I think I found the solution. The hint came from this example.
Adding the following to package.json and locking the file fixed the Module not found error:
  "pnpm": {
    "packageExtensions": {
      "postcss-loader": {
        "peerDependencies": {
          "postcss-flexbugs-fixes": "*",
          "postcss-preset-env": "*",
          "postcss-normalize": "*"
        }
      }
    }
  },

I will close this issue but keep it for reference, it might be useful to others as well.

@mashayev was this consistently reproducible for you ? We've facing a bunch of such module not found issues post the upgrade to rules_js 2.0 , but they seem to be flaky


[WIP] Fixed #64679 - Delete Lines now works with multiple selections on multiple lines

This PR fixes https://github.com/Microsoft/vscode/issues/64679.
It now collapses all cursors:

Also, now deletes multiple lines with multiple selections:

Thanks for considering this request.

Thanks for the feedback, I'll go back and make the appropriate changes

So made the changes you requested.
I removed DeleteLinesCommand entirely and now issue edits to remove text.
Here is a couple of gif to show the changes.
Three Cursors:

Two Lines with Multiple Cursors:

My branch needs some work done on it but I wanted to see if the cursors locations after Delete Lines are good or if they need adjustment.

:+1: Thank you! It was almost perfect, I just needed to add some accounting for the deleted lines in establishing the resulting cursor selection.


Disable markdownling check for ops portal URL protected by Check-in

Disable markdownling check for ops portal URL protected by Check-in

This should fix the issue reported in https://github.com/EGI-Federation/documentation/pull/530


Support for /game for GOG Linux native games

Thanks!
Following up from https://github.com/mtkennerly/ludusavi-manifest/issues/11:

Both Gog and Gog Linux are fine IMO, but the later one makes it clearer to me. Else one could rename Gog to "Gog Offline" since it is an offline installer I get on Linux. But it should be checked if the offline installers on Windows exhibit the same behavior.

The GOG option is already meant to be for the standalone/offline installers, so that's why I'd prefer not to add another variant. We could rename it, although for now I'd rather just clarify it in the documentation if necessary.
I just tested a couple of offline installers on Windows (Dead Cells, Tyrian 2000), and neither one includes the game subfolder, so I think we can assume Linux only for now.

Hm ... I am new to the pull request process on github, pushing the changes as requested should be sufficient, right?

Hm ... I am new to the pull request process on github, pushing the changes as requested should be sufficient, right?

Yep! Everything looks great. Thanks again for your help :D


provide removed in react-redux core

I noticed that provide was removed from react-redux core per https://github.com/reactjs/react-redux/commit/09494885cad9cebd744b8bf05b3b22e45061c214
how does that decision to change the api affect the roadmap of this project? I personally favor the provide decorator but obvs the creator has other thoughts.

I think you are in a wrong repo. It's quite old commit and they just removed decorator in favor of high order component because it made much more sense there.

@amcdnl The only thing this library uses from react-redux is the following:
import shallowEqual from 'react-redux/lib/utils/shallowEqual';
import wrapActionCreators from 'react-redux/lib/utils/wrapActionCreators';

Other than that it is completely independent from react-redux.


Windows 7 - DLLNotFoundException

I've been trying to build WebWindow on a x64 Windows 7 machine but every time it fails with:
System.TypeInitializationException: 'The type initializer for 'WebWindows.WebWindow' threw an exception.' DllNotFoundException: Unable to load DLL 'WebWindow.Native' or one of its dependencies. (0x8007007F).
I already tried a lot of things. Reinstaling Visual C++, different versions of WebView2 and Chromium Edge (beta, dev) but nothing worked. And this only happens on Windows 7.
Any ideas on what's causing this issue?

Found the problem. GetDpiForWindow(_hWnd) is Windows 10 only. Changing WebWindow::GetScreenDpi solved it for now.
unsigned int WebWindow::GetScreenDpi()
{
if (IsWindows10OrGreater)
{
return GetDpiForWindow(_hWnd);
}
else
{
return 0;
}
}


bump 0.10.1

Coverage: 96.582%. Remained the same when pulling cb08a36b6658c481ae2bb436dde8a0678eeaa13a on release/0.10.1 into c6d289c6ce162aa57cac4aafa1fc87cb05b219fe on main.


Add istio-env label when creating test namespaces.

The installer in istio-ecosystem] supports multiple control planes. To identify which control plane instance should provide configuration for the pods in a namespace, the namespace label istio-env:NAME_OF_ENV is used instead of istio-injected:true.
When running the echo test in a standard istio installation with the istio-ecosystem installer, the echosrv pods do not get a sidecar, since the istio-env-label is missing for test namespaces. To fix this and successfully execute the test, it needs to be set to istio-control.

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: jkbschmid
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approver: geeknoid
If they are not already assigned, you can assign the PR to them by writing /assign @geeknoid in a comment when ready.
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

pkg/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Hi @jkbschmid. Thanks for your PR.
I'm waiting for a istio member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.
Once the patch is verified, the new status will be reflected by the ok-to-test label.
I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

/ok-to-test

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: jkbschmid, nmittler
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

pkg/OWNERS [nmittler]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

need rebase to fix test failure

@howardjohn The cla-bot doesn't seem to be working again. Mind giving this a cla:yes?

New changes are detected. LGTM label has been removed.

@nmittler @howardjohn Just rebased and all required tests have passed now.
Is there anyone we can turn to for fixing the CLA-bot? Always setting the label manually is really annoying (for you and for us).

@jkbschmid: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-integ-k8s-tests.sh
8a66769a5d281517116634c4b45df8c761da4364
link
/test istio-integ-k8s-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@jkbschmid I briefly looked at it and apparently it is working as intended, so not sure it could be fixed, even though it is annoying. It only happens when 2 people are on the same PR

@jkbschmid: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-integ-k8s-tests.sh
8a66769a5d281517116634c4b45df8c761da4364
link
/test istio-integ-k8s-tests

prow/istio-integ-local-tests.sh
c2107066c1284f9b653352a8c8e69b0fc8abbb49
link
/test istio-integ-local-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@jkbschmid: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-integ-k8s-tests.sh
8a66769a5d281517116634c4b45df8c761da4364
link
/test istio-integ-k8s-tests

prow/istio-integ-local-tests.sh
c2107066c1284f9b653352a8c8e69b0fc8abbb49
link
/test istio-integ-local-tests

prow/istio-unit-tests.sh
c2107066c1284f9b653352a8c8e69b0fc8abbb49
link
/test istio-unit-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@jkbschmid: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-integ-local-tests.sh
c2107066c1284f9b653352a8c8e69b0fc8abbb49
link
/test istio-integ-local-tests

prow/istio-unit-tests.sh
c2107066c1284f9b653352a8c8e69b0fc8abbb49
link
/test istio-unit-tests

prow/istio-integ-k8s-tests.sh
c2107066c1284f9b653352a8c8e69b0fc8abbb49
link
/test istio-integ-k8s-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@jkbschmid: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-unit-tests.sh
c2107066c1284f9b653352a8c8e69b0fc8abbb49
link
/test istio-unit-tests

prow/istio-integ-k8s-tests.sh
c2107066c1284f9b653352a8c8e69b0fc8abbb49
link
/test istio-integ-k8s-tests

prow/istio-integ-local-tests.sh
354225f50bd67fb0b7204fc3864ec57f0d79f8ce
link
/test istio-integ-local-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@jkbschmid: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-integ-k8s-tests.sh
c2107066c1284f9b653352a8c8e69b0fc8abbb49
link
/test istio-integ-k8s-tests

prow/istio-integ-local-tests.sh
354225f50bd67fb0b7204fc3864ec57f0d79f8ce
link
/test istio-integ-local-tests

prow/istio-unit-tests.sh
354225f50bd67fb0b7204fc3864ec57f0d79f8ce
link
/test istio-unit-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

Setting cla to yes since everyone consented.
Sorry the consent thing is annoying

/lgtm

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: howardjohn, jkbschmid, nmittler
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

pkg/OWNERS [nmittler]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: howardjohn, jkbschmid, nmittler
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

pkg/OWNERS [nmittler]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@jkbschmid: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-integ-k8s-tests.sh
bf3f2b35572a2dacba6f8bf4582e77e32a1b456f
link
/test istio-integ-k8s-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.


CPP Minigo

Just a note that there may be a C++ implementation of minigo and I want to take a few minutes to discuss this in SWG this week.

See here;
https://github.com/tensorflow/minigo/tree/master/cc

This comes with no promises and just an interesting thing for your consideration.


Create CONTRIBUTING.md

Please click "Files Changed" above this box, and click "View" on the right side to see what I've done.

Just a hint, when making a pull request that directly addresses an issue, in the description write
Closes #2
This (should) automatically close the related issue when a PR is merged :)


How to use regex search

I can't find any documentation about how to use regex search, I found there is the search-regexp option, but when I try inputting any regex it just repeats the options.
How would I go about finding, for example, any message starting with a number in the #general channel of team FOOBAR?

keybase chat search-regexp --regex keybase "^\d.*"

Thanks, I understand the regex search a little better. Is the search-after and search-before format only /mm/dd/yyyy  or other formats?

We use this library and try to parse many formats: https://github.com/araddon/dateparse#extended-example


Get rid of OrderedDicts

addressing the issue: #233
Changed OrderedDict to dict as in python ^3.7 dict are ordered by default

@ritikramuka there was some issues with circle-ci so i had to roll an update. Can you rebase master so ci tests are run again?

@sr-gi can you help me figure out where the test is failing!

Have you run the test locally? Looks like you've replaced OrderedDict by dict but you have not updated the methods.

@sr-gi I am not able to run any test locally... as having an issue setting up leveldb and plyvel


yt-dlp does not respect output option

Checklist

[X] I'm reporting a bug unrelated to a specific site
[X] I've verified that I'm running yt-dlp version 2<IP_ADDRESS>. (update instructions)
[X] I've checked that all provided URLs are alive and playable in a browser
[X] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[X] I've searched the bugtracker for similar issues including closed ones. DO NOT post duplicates
[X] I've read the guidelines for opening an issue

Description
yt-dlp -o test.mp4 https://youtube.com/blabla
I expected a test.mp4 to be present on the filesystem but I get a test.mp4.webm
Verbose log
[youtube] ltZKAe12neg: Downloading webpage
[youtube] ltZKAe12neg: Downloading android player API JSON
[info] ltZKAe12neg: Downloading 1 format(s): 248+251
[download] Destination: /usr/src/app/downloads/1647629624688.mp4.f248.webm
[download] 100% of 68.19MiB in 00:02
[download] Destination: /usr/src/app/downloads/1647629624688.mp4.f251.webm
[download] 100% of 6.88MiB in 00:00
[Merger] Merging formats into "/usr/src/app/downloads/1647629624688.mp4.webm"
Deleting original file /usr/src/app/downloads/1647629624688.mp4.f251.webm (pass -k to keep)
Deleting original file /usr/src/app/downloads/1647629624688.mp4.f248.webm (pass -k to keep)

You are actually downloading a webm file despite the filename set. Use -f or -S to select an actual mp4 format.

true but the documentation says that -o, even along with a full path, should not alter the final filename.
this behavior is also different from the original youtube-dl

this behavior is also different from the original youtube-dl

No, it is the same

this behavior is also different from the original youtube-dl

No, it is the same

I just switched to yt-dlp and it's not the same, at least in my tests, with the same params I was able to keep the filename I specified in the output without any changes

maybe the behavior depends on the source? that would explain it since I've tested only youtube in yt-dlp and other sources in youtube-dl

The file extension is forced if a merge is needed. ffmpeg doesn't merge the files properly unless the extension is right.
I have considered forcing the extension always for consistent behavior (and coz I've seen too many issues due to user passing wrong extension), but then people would genuinely complain that the behavior is different from youtube-dl. So I'll need to make a compat option, document the change and stuff...


How to handle duplicate id

Getting this error.
:76: element div: validity error : ID header-print already defined

How can I get it to continue past this? And no I can not remove the duplicate tag.

If you'd like to avoid an error output, do as follows.
var err : NSError?
let option = CInt(HTML_PARSE_NOERROR.value | HTML_PARSE_RECOVER.value)
var parser     = HTMLParser(html: html, encoding: NSUTF8StringEncoding, option: option, error: &err)

similar #10


How to use UserManager in Console Application

I have to use UserManager and RoleManager to add Users and Roles in .net core console application. How to resolve them from IocManager? I can't resolve it directly or using service builder.

You have to bootstrap the app correctly, then add Abp.Zero and create the
respective derived classes
El El mar, 6 de junio de 2017 a la(s) 12:34 a.m., Chris L.(Luo Sheng) <
<EMAIL_ADDRESS>escribió:

I have to use UserManager and RoleManager to add Users and Roles in .net
core console application. How to resolve them from IocManager? I can't
resolve it directly or using service builder.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/aspnetboilerplate/module-zero/issues/376, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AIycYWJxZfL0zYwRei775BW-vGPrNJY4ks5sBOTJgaJpZM4Nw4OI
.

Castle.MicroKernel.Handlers.HandlerException:“Can't create component 'XXX.Authorization.Users.UserManager' as it has dependencies to be satisfied.
'XXX.Authorization.Users.UserManager' is waiting for the following dependencies:

Service 'XXXl.Authorization.Roles.RoleManager' which was registered but is also waiting for dependencies.

'XXX.Authorization.Roles.RoleManager' is waiting for the following dependencies:

Service 'System.Collections.Generic.IEnumerable1[[Microsoft.AspNetCore.Identity.IRoleValidator1[[XXX.Authorization.Roles.Role, XXX.Core, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=null]], Microsoft.AspNetCore.Identity, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=adb9793829ddae60]]' which was not registered.

Service 'Microsoft.AspNetCore.Identity.ILookupNormalizer' which was not registered.

Service 'Microsoft.AspNetCore.Identity.IdentityErrorDescriber' which was not registered.

Service 'Microsoft.Extensions.Logging.ILogger1[[Abp.Authorization.Roles.AbpRoleManager2[[XXX.Roles.Role, XXX.Core, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=null],[XXX.Authorization.Users.User, XXX.Core, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=null]], Abp.ZeroCore, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=null]]' which was not registered.

Service 'Microsoft.AspNetCore.Http.IHttpContextAccessor' which was not registered.

Service 'Microsoft.Extensions.Options.IOptions`1[[Microsoft.AspNetCore.Builder.IdentityOptions, Microsoft.AspNetCore.Identity, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=adb9793829ddae60]]' which was not registered.

Service 'Microsoft.AspNetCore.Identity.IPasswordHasher`1[[XXX.Authorization.Users.User, XXX.Core, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=null]]' which was not registered.

Service 'System.Collections.Generic.IEnumerable1[[Microsoft.AspNetCore.Identity.IUserValidator1[[XXX.Authorization.Users.User, XXX.Core, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=null]], Microsoft.AspNetCore.Identity, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=adb9793829ddae60]]' which was not registered.

Service 'System.Collections.Generic.IEnumerable1[[Microsoft.AspNetCore.Identity.IPasswordValidator1[[XXX.Authorization.Users.User, XXX.Core, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=null]], Microsoft.AspNetCore.Identity, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=adb9793829ddae60]]' which was not registered.

Service 'Microsoft.AspNetCore.Identity.ILookupNormalizer' which was not registered.

Service 'Microsoft.AspNetCore.Identity.IdentityErrorDescriber' which was not registered.

Service 'System.IServiceProvider' which was not registered.

Service 'Microsoft.Extensions.Logging.ILogger1[[Microsoft.AspNetCore.Identity.UserManager1[[XXX.Authorization.Users.User, XXX.Core, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=null]], Microsoft.AspNetCore.Identity, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=adb9793829ddae60]]' which was not registered.

”

Alright, I think I found the problem. I'm using 2.1.0-rc1 and all repositories are not loaded.

You probably haven't used AddAbpIdentity from the console application. module-zero-core-template includes IdentityRegistrar for it (https://github.com/aspnetboilerplate/module-zero-core-template/blob/master/aspnet-core/src/AbpCompanyName.AbpProjectName.Core/Identity/IdentityRegistrar.cs) which is called by Startup class inside the web application.
For console, it may be a bit tricky because you can not find ServiceCollection. It was similar for unit test. To make them working, you can use such a code in Initialize method of your console app module:
var services = new ServiceCollection();
IdentityRegistrar.Register(services);
WindsorRegistrationHelper.CreateServiceProvider(IocManager.IocContainer, services);

Ah, So Castle Windsor has a extend function that can inject service. I just thought that abp has one and could not find it. Thanks.

ServiceCollection is Microsoft's new DI registration way and independent from Castle and Abp. Services should register via ServiceCollection should be manually registered before initialze Abp like that. WindsorRegistrationHelper is an adapter for ServiceCollection and WindsorContainer.

There is another problem here.
The instance of entity type 'User' cannot be tracked because another instance of this type with the same key is already being tracked. When adding new entities, for most key types a unique temporary key value will be created if no key is set (i.e. if the key property is assigned the default value for its type). If you are explicitly setting key values for new entities, ensure they do not collide with existing entities or temporary values generated for other new entities. When attaching existing entities, ensure that only one entity instance with a given key value is attached to the context.
I think I've aleady added all unitofworks and detach in the code, but the problem also happens.
                const string name = "xxx";
                const string password = "aA123456";
                const string email =<EMAIL_ADDRESS>                const string phone = "111";

                User user;

                using (var unitOfWork = unitOfWorkManager.Begin())
                {
                    // Create Admin User:
                    user = userManager.FindByNameAsync(name).Result;
                    unitOfWork.Complete();
                }

                if (user == null)
                {
                    user = new User
                    {
                        UserName = name,
                        Name = "xxx",
                        Surname = "",
                        EmailAddress = email,
                        PhoneNumber = phone,
                        Type =
                            corporationRepository.FirstOrDefault(
                                    corporation => corporation.Name == CorporationDefine.Corporation)
                                .Type
                    };

                    using (var unitOfWork = unitOfWorkManager.Begin())
                    {
                        var result = userManager.CreateAsync(user, password).Result;
                        dbContext.Entry(user).State = EntityState.Detached;
                        unitOfWork.Complete();
                    }

                    using (var unitOfWork = unitOfWorkManager.Begin())
                    {
                        userCorporationRepository.Create(new ApplicationUserCorpration()
                        {
                            UserId = user.Id,
                            CorporationId =
                                corporationRepository.FirstOrDefault(
                                        corporation => corporation.Name == CorporationDefine.Corporation)
                                    .Id
                        });
                        unitOfWork.Complete();
                    }

                    using (var unitOfWork = unitOfWorkManager.Begin())
                    {
                        user = userManager.FindByNameAsync(name).Result;
                        dbContext.Entry(user).State = EntityState.Detached;
                        unitOfWork.Complete();
                    }

                    using (var unitOfWork = unitOfWorkManager.Begin())
                    {
                        //throws here
                        var result = userManager.SetLockoutEnabledAsync(user, false).Result;
                        dbContext.Entry(user).State = EntityState.Detached;
                        unitOfWork.Complete();
                    }

                    using (var unitOfWork = unitOfWorkManager.Begin())
                    {
                        // Add user admin to Role Admin if not already added
                        var rolesForUser = userManager.GetRolesAsync(user).Result;
                        if (!rolesForUser.Contains(roleName2))
                        {
                            //throws here
                            var result = userManager.AddToRoleAsync(user, roleName2).Result;
                            dbContext.Entry(user).State = EntityState.Detached;
                        }
                        unitOfWork.Complete();
                    }
                }

I also found that any time when the I want to update User in dbcontext, this bug will be thrown.
I'm using ZeroCore.IdentityServer4 rc1.

Hi,
unitOfWorkManager.Begin() does nothing if this code is already inside a UOW. If you want to start an independent, inner UOW, you can use unitOfWorkManager.Begin(TransactionScopeOption.RequiresNew).
However, your code seems a bit odd. I suppose because it's just a test code. If you need, I can create an empty console application in the module-zero-core-template when I've time.

That's not only happen in console application but happens everywhere in my project.
For example when I login in IdentityServer. It throws a similar exception.
 Abp.Domain.Uow.AbpDbConcurrencyException: Database operation expected to affect 1 row(s) but actually affected 0 row(s). Data may have been modified or deleted since entities were loaded. See http://go.microsoft.com/fwlink/?LinkId=527962 for information on understanding and handling optimistic concurrency exceptions. ---> Microsoft.EntityFrameworkCore.DbUpdateConcurrencyException: Database operation expected to affect 1 row(s) but actually affected 0 row(s). Data may have been modified or deleted since entities were loaded. See http://go.microsoft.com/fwlink/?LinkId=527962 for information on understanding and handling optimistic concurrency exceptions.    at Microsoft.EntityFrameworkCore.Update.AffectedCountModificationCommandBatch.ThrowAggregateUpdateConcurrencyException(Int32 commandIndex, Int32 expectedRowsAffected, Int32 rowsAffected)    at Microsoft.EntityFrameworkCore.Update.AffectedCountModificationCommandBatch.<ConsumeResultSetWithoutPropagationAsync>d__6.MoveNext() --- End of stack trace from previous location where exception was thrown ---    at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)    at Microsoft.EntityFrameworkCore.Update.AffectedCountModificationCommandBatch.<ConsumeAsync>d__2.MoveNext() --- End of stack trace from previous location where exception was thrown ---    at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)    at Microsoft.EntityFrameworkCore.Update.ReaderModificationCommandBatch.<ExecuteAsync>d__32.MoveNext() --- End of stack trace from previous location where exception was thrown ---    at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)    at Microsoft.EntityFrameworkCore.Update.Internal.BatchExecutor.<ExecuteAsync>d__10.MoveNext() --- End of stack trace from previous location where exception was thrown ---    at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)    at Microsoft.EntityFrameworkCore.Storage.Internal.SqlServerExecutionStrategy.<ExecuteAsync>d__62.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.d__54.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.d__52.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.EntityFrameworkCore.DbContext.d__35.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Abp.EntityFrameworkCore.AbpDbContext.d__36.MoveNext()
--- End of inner exception stack trace ---
at Abp.EntityFrameworkCore.AbpDbContext.d__36.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Abp.EntityFrameworkCore.Uow.EfCoreUnitOfWork.d__20.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Abp.EntityFrameworkCore.Uow.EfCoreUnitOfWork.d__12.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Abp.EntityFrameworkCore.Uow.EfCoreUnitOfWork.d__14.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Abp.Domain.Uow.UnitOfWorkBase.d__57.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Abp.Domain.Uow.UnitOfWorkInterceptor.<>c__DisplayClass6_0.<b__2>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Abp.Threading.InternalAsyncHelper.d__51.MoveNext() --- End of stack trace from previous location where exception was thrown ---    at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)    at Abp.Authorization.Users.AbpUserStore2.d__36.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.AspNetCore.Identity.UserManager1.<UpdateUserAsync>d__159.MoveNext() --- End of stack trace from previous location where exception was thrown ---    at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)    at Microsoft.AspNetCore.Identity.UserManager1.d__79.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.AspNetCore.Identity.SignInManager1.<CheckPasswordSignInAsync>d__34.MoveNext() --- End of stack trace from previous location where exception was thrown ---    at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)    at Abp.IdentityServer4.AbpResourceOwnerPasswordValidator1.d__10.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at IdentityServer4.Validation.TokenRequestValidator.d__16.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at IdentityServer4.Validation.TokenRequestValidator.d__13.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at IdentityServer4.Validation.TokenRequestValidator.d__12.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at IdentityServer4.Endpoints.TokenEndpoint.d__7.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at IdentityServer4.Endpoints.TokenEndpoint.d__6.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at IdentityServer4.Hosting.IdentityServerMiddleware.d__4.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at IdentityServer4.Hosting.IdentityServerMiddleware.d__4.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at IdentityServer4.Hosting.FederatedSignOutMiddleware.d__6.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at IdentityServer4.Hosting.AuthenticationMiddleware.d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.AspNetCore.Cors.Infrastructure.CorsMiddleware.d__7.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at IdentityServer4.Hosting.BaseUrlMiddleware.d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.AspNetCore.Builder.Extensions.MapMiddleware.d__3.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware1.<Invoke>d__18.MoveNext() --- End of stack trace from previous location where exception was thrown ---    at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()    at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware1.d__18.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware1.<Invoke>d__18.MoveNext() --- End of stack trace from previous location where exception was thrown ---    at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()    at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware1.d__18.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware1.<Invoke>d__18.MoveNext() --- End of stack trace from previous location where exception was thrown ---    at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()    at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware1.d__18.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware1.<Invoke>d__18.MoveNext() --- End of stack trace from previous location where exception was thrown ---    at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()    at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware1.d__18.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware1.<Invoke>d__18.MoveNext() --- End of stack trace from previous location where exception was thrown ---    at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()    at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware1.d__18.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.VisualStudio.Web.BrowserLink.BrowserLinkMiddleware.d__7.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.AspNetCore.Diagnostics.EntityFrameworkCore.MigrationsEndPointMiddleware.d__5.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.AspNetCore.Diagnostics.EntityFrameworkCore.DatabaseErrorPageMiddleware.d__6.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at Microsoft.AspNetCore.Diagnostics.EntityFrameworkCore.DatabaseErrorPageMiddleware.d__6.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.d__7.MoveNext()
`

Just released v2.1.1.
We'll document IdentityServer4 document in a short time.

Alright, finally I lock the problem. I will open another issue.


function querystate command causes the number of broker threads to increase

Describe the bug
bin/pulsar-admin function querystate --name xxx --tenant public --namespace default --key SumFunction -w
Resulting in a surge in the number of pulse broker server threads, eventually OOM
To Reproduce
Steps to reproduce the behavior:

bin/pulsar-admin function querystate --name xxx --tenant public --namespace default --key SumFunction -w
run the command on broker server jstack sudo jps|grep Pul|awk '{print $1}'|grep tid |awk '{print $1 }'|sort|unqi -c

Expected behavior
xxx-OrderedScheduler The number of threads is gradually increasing
Screenshots

Desktop (please complete the following information):

OS: 16.04.1-Ubuntu SMP Tue Feb 12 17:45:24 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
version：apache-pulsar-2.4.0

@wolfstudy to take a look

Closed as stale. Please open a new issue if it's still relevant to the maintained versions.


Matplotlib issues

I can change the line style in Matplotlib, but the style resets immediately to the default upon closing the dialog box.

I fixed the dependencies needed to allow matplotlib 3 in #30, but still have this bug with the style option. It's likely other improvements are possible, but I think I will just stick to this one issue for the time being.


Update loss method in KerasClassifier

Signed-off-by: Beat Buesser<EMAIL_ADDRESS>Description
This pull request updates the method loss in KerasClassifier to pass a Keras specific unittests for MembershipInferenceBlackBox.
Type of change
Please check all relevant options.

[ ] Improvement (non-breaking)
[x] Bug fix (non-breaking)
[ ] New feature (non-breaking)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

Checklist

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[x] I have commented my code
[x] I have made corresponding changes to the documentation
[x] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] New and existing unit tests pass locally with my changes

Codecov Report

Merging #651 into dev_1.4.2 will increase coverage by 0.88%.
The diff coverage is 0.00%.

@@              Coverage Diff              @@
##           dev_1.4.2     #651      +/-   ##
=============================================
+ Coverage      58.39%   59.27%   +0.88%
=============================================
  Files            155      155
  Lines          14091    14092       +1
  Branches        2513     2513
=============================================
+ Hits            8228     8353     +125
+ Misses          5089     4948     -141
- Partials         774      791      +17

Impacted Files
Coverage Δ

art/estimators/classification/keras.py
65.64% <0.00%> (+1.02%)
:arrow_up:

art/defences/detector/poison/activation_defence.py
85.13% <0.00%> (+0.33%)
:arrow_up:

art/estimators/tensorflow.py
37.97% <0.00%> (+2.53%)
:arrow_up:

art/data_generators.py
74.78% <0.00%> (+20.00%)
:arrow_up:

art/estimators/classification/tensorflow.py
50.34% <0.00%> (+21.64%)
:arrow_up:

Yes, this looks correct. Thanks.


ZDEMO_EXCEL2: add change_cell_style showcase

fixes #73
needed to test https://github.com/abap2xlsx/abap2xlsx/pull/1260

Could you run zdemo_excel_checker (regression checker) to regenerate the new files zdemo_excel2.w3mi.data.xlsx and zdemo_excel15_02.w3mi.data.xlsx? (Choose any folder on your laptop, run the program, the 2 programs zdemo_excel2 and zdemo_excel15_02 should be in red status, click on the Save button to upload the 2 changed Excel files in the Web repository, and push the changes to your branch)

Could you run zdemo_excel_checker (regression checker) to regenerate the new files zdemo_excel2.w3mi.data.xlsx and zdemo_excel15_02.w3mi.data.xlsx? (Choose any folder on your laptop, run the program, the 2 programs zdemo_excel2 and zdemo_excel15_02 should be in red status, click on the Save button to upload the 2 changed Excel files in the Web repository, and push the changes to your branch)

if we get stuff like https://github.com/abap2xlsx/demos/pull/69 decided/merged, then one day it should be possible to have github actions automatically generate the xlsx files, and check/show differences if any

@bigld can you help updating the branch, or setting "allow changes by maintainers"

lets go


Shell escape keychain paths in CertChecker

Addresses https://github.com/fastlane/fastlane/issues/4734

👍


Error but anything works fine

VSCode Version: Code 1.10.2 (8076a19fdcab7e1fc1707952d652f0bb6c6db331, 2017-03-08T14:02:52.799Z)
OS Version: Windows_NT ia32 10.0.14393
Extensions:

Extension
Author
Version

code-runner
formulahendry
0.6.15

Go
lukehoban
0.6.57

Hi, im Having this error but anything works fine, it's annoying and i want to know how to fix it:
'Error' message: 'can't load package: package c:/work/src/primerServer: cannot find package "c:/work/src/primerServer" in any of:
C:\Go\src\c:\work\src\primerServer (from $GOROOT)
c:\work\src\c:\work\src\primerServer (from $GOPATH)
i'm not importing that package, it's my project's and folder's name.
How can i fix it?
Steps to Reproduce:

Please file this directly against the go extension


GitHub-Flavored Markdown isn't supported

See e.g. this package documentation rendered on npmjs.com versus the same thing rendered on GitHub. Admonitions aren't rendered on npmjs.com.

To be fair, it's a brand new GFM feature.

I wouldn't call it brand new, it's coming up on 9 months since official release, but it's not technically an official part of the GFM specification and definitely easy to overlook or deprioritize in the scheme of things. 😄
Anywho, I also find the raw syntax a bit jarring, so I just threw together a userscript to fix the rendering like:

Quickly tested in ViolentMonkey against the package linked above. Does the trick for me!
@ljharb I briefly looked around, but didn't see anywhere in the NPM org where those package pages are rendered. Do you know if that's open source and where I could contribute to a proper fix, or if it's closed and we just need to wait for a staffer to pick it up? Thanks!

I believe it's closed and only staff can do something about it.


Warn if dispatch config is used in services.xml

Log a warning if config for old dispatch process removed in Vespa 7 is used. Add deprecation TODOs and remove code only used in tests

@geirst FYI


Not-tech-related Question: Why this name

By helping coders to use drak mode, why not keep useing 'dark-mode' name? That is more suitable with the purpose of this repostory, isn't it?

By the way, your work helped me a lot. There are many bugs in RN-0.62 Appearance.

And, please, can you change the link that mentioned in the dark mode repo's README.md to this github repo? It is now linked to npm page. Link to github page is better, I think.

By helping coders to use drak mode, why not keep useing 'dark-mode' name?

To use dark mode you don't need to use react-native-dark-mode anymore. It would be misleading.
A lot of people will also upgrade without reading the changelog, break half their app and then open issues again without reading it.
One more reason would be uniformity, since in the Appearance API the concept is called color scheme, and the hook is called useColorScheme, I wanted to stay away from the dark mode naming and just use color scheme, useDarkMode sounds cool though.
Initially I wanted to call it react-native-dynamic-stylesheet but that is taken and with the changes npm made a while ago I can't use react-native-dynamic-style-sheet for example. Then I figured out that I also have dynamic values, so technically you could use this without the stylesheet part. I don't really care about the name, this is just a module that we use at work and I'm sharing with the world in case someone finds it useful.

By the way, your work helped me a lot. There are many bugs in RN-0.62 Appearance.

Glad to hear that! After looking at the inner workings of useColorScheme I was confused as to why aren't they using context but rather setup a subscription every time you use the hook, even though we mentioned it in the discussion prior to the creation of the Appearance API.

And, please, can you change the link that mentioned in the dark mode repo's README.md to this github repo?

Done. 🙂


Change file paths to use an uppercase drive letter when normalized

When using the extension to lint, it would neglect to use the .sqlfluffignore file in the root of my project. According to the SQLFluff Docs, this is where the file should be placed.
I found that the issue would be resolved if I set the sqlfluff.workingDirectory using a capital C - C:\dev\MyProject. If I used a lowercase c - c:\dev\MyProject -  it would not work. While debugging, I noticed that omitting the sqlfluff.workingDirectory or using a variable (such as ${workspaceFolder}) would result in a lowercase c being used as the drive. This aligns with the VS Code API Docs for fspath. It always returns a lowercase drive letter.

@mhahn-ts Are you using Windows? I didn't think the Windows filesystem was case-sensitive, so not sure why having a lowercase drive would matter?

@mhahn-ts Are you using Windows? I didn't think the Windows filesystem was case-sensitive, so not sure why having a lowercase drive would matter?

Yes, I'm on Windows. I am able to reproduce with just SQLFluff in the command line. In fact, it seems to treat the entire filepath in a case-sensitive manner.

In my opinion, it's an issue with SQLFluff. I was planning to create an issue in that repo. This is especially true since you cannot specify absolute path in the .sqlfluffignore. You must just give relative path.
But, based on how the extension automatically gets that ${workspaceFolder}, I do think it is still appropriate to fix it here. If SQLFluff requires a case-sensitive matched filepath, we should make sure that the filepath variables are created that way.

Interesting, I just assumed it would work with lowercase drive letters. Thanks for looking into this. I don't think it is necessary to put it behind a config setting, your changes should be good. I should have time tonight to get them merged and publish a release with the changes.

@mhahn-ts This has been released in v2.4.3


Private members may not satisfy protocol requirements, ever.

Finish up the primary work on SE-0025 by forbidding private members from satisfying protocol requirements. Then, update the CHANGELOG and docs; SE-0025 is implemented!

Before merging this pull request to apple/swift repository:

[ ] Test pull request on Swift continuous integration.

Triggering Swift CI
The swift-ci is triggered by writing a comment on this PR addressed to the GitHub user @swift-ci. Different tests will run depending on the specific comment that you use. The currently available comments are:
Smoke Testing
    Platform     | Comment
    ------------ | -------------
    All supported platforms     | @swift-ci Please smoke test
    All supported platforms     | @swift-ci Please smoke test and merge
    OS X platform               | @swift-ci Please smoke test OS X platform
    Linux platform              | @swift-ci Please smoke test Linux platform

A smoke test on macOS does the following:

Builds the compiler incrementally.
Builds the standard library only for macOS. Simulator standard libraries and
device standard libraries are not built.
lldb is not built.
The test and validation-test targets are run only for macOS. The optimized
version of these tests are not run.

A smoke test on Linux does the following:

Builds the compiler incrementally.
Builds the standard library incrementally.
lldb is built incrementally.
The swift test and validation-test targets are run. The optimized version of these
tests are not run.
lldb is tested.

Validation Testing
    Platform     | Comment
    ------------ | -------------
    All supported platforms     | @swift-ci Please test
    All supported platforms     | @swift-ci Please test and merge
    OS X platform               | @swift-ci Please test OS X platform
    OS X platform               | @swift-ci Please benchmark
    Linux platform              | @swift-ci Please test Linux platform

Lint Testing
    Language     | Comment
    ------------ | -------------
    Python       | @swift-ci Please Python lint

Note: Only members of the Apple organization can trigger swift-ci.

@swift-ci Please test

Playground logger failures are my fault and were just fixed.  This will need a rebuild.

I do want to make sure this doesn't break downstream projects, so
@swift-ci Please test OS X platform

@swift-ci Please test OS X platform

Build interrupted *sigh*.

@swift-ci Please test

Over to Ted to merge. I'll resolve all the bugs when done.


Appstudio update build-suite-test-component-git-source-ntzc

Pipelines as Code configuration proposal

Pipelines as Code CI/build-suite-test-component-git-source-ntzc-on-pull-request has successfully validated your commit.

StatusDurationName

✅ Succeeded
8 seconds
appstudio-init

✅ Succeeded
9 seconds
clone-repository

✅ Succeeded
8 seconds
appstudio-configure-build

✅ Succeeded
29 seconds
build-container

✅ Succeeded
13 seconds
show-summary

Pipelines as Code CI/build-suite-test-component-git-source-ntzc-on-pull-request has successfully validated your commit.

StatusDurationName

✅ Succeeded
8 seconds
appstudio-init

✅ Succeeded
11 seconds
clone-repository

✅ Succeeded
8 seconds
appstudio-configure-build

✅ Succeeded
32 seconds
build-container

✅ Succeeded
11 seconds
show-summary


Add FL Studio <IP_ADDRESS>59 manifest

[x] Have you signed the Contributor License Agreement?
[x] Have you checked that there aren't other open pull requests for the same manifest update/change?
[x] This PR only modifies one (1) manifest
[x] Have you validated your manifest locally with winget validate --manifest <path>?

[x] Does your manifest conform to the 1.5 schema?

Note: <path> is the name of the directory containing the manifest you're submitting.

Microsoft Reviewers: Open in CodeFlow

https://github.com/microsoft/winget-pkgs/issues/107639

Closing due to merge conflicts.


MapperFunction gets corrupted upon undo/redo

The MapperFunction sometimes gets corrupted after issuing an Undo command.  This is a non-deterministic bug, and exists for both the TwoD and Barb renderers.  Corrupt MapperFunctions appear as in the screenshot below.  Data bounds seem correct, but the histogram and colorbar are not.  The console emits glErrors from the file ShaderProgram.cpp when this happens.
To reproduce:

Load any dataset, create a new TwoD or Barb renderer
Adjust an opacity control point, then undo the change.  The MapperFunction may get corrupted.  If not, restart the application and try again.

I'm re-assigning this to me. The MappingFrame does not follow the design pattern for users of ParamsBase classes


WIP: ref/client

Coverage decreased (-1.38%) to 93.26% when pulling c3a64761b651b91a743008e0e533c1471b098a95 on isocolsky:ref/client into c1e8ac02ab16e7a9034739fd9f684ca46cff5de2 on bitpay:master.

Coverage decreased (-0.32%) to 94.33% when pulling cabdb35cb122669cccaea4a64daf918b8166e3d3 on isocolsky:ref/client into c1e8ac02ab16e7a9034739fd9f684ca46cff5de2 on bitpay:master.

Coverage decreased (-0.4%) to 94.25% when pulling d2085c9b9e446179b40b137a0cd6cc445b58700a on isocolsky:ref/client into c1e8ac02ab16e7a9034739fd9f684ca46cff5de2 on bitpay:master.

Coverage increased (+0.36%) to 95.0% when pulling a0019d966c8beed5cd7790819ec90e2573129572 on isocolsky:ref/client into c1e8ac02ab16e7a9034739fd9f684ca46cff5de2 on bitpay:master.

Coverage increased (+0.31%) to 94.95% when pulling c0b7970ff6b52d5a4245242f4811e2da13ad64e4 on isocolsky:ref/client into c1e8ac02ab16e7a9034739fd9f684ca46cff5de2 on bitpay:master.

Coverage increased (+0.69%) to 95.34% when pulling f486ecacd31e461c969f22e5e2f34f7464678d29 on isocolsky:ref/client into c1e8ac02ab16e7a9034739fd9f684ca46cff5de2 on bitpay:master.

Coverage increased (+0.69%) to 95.34% when pulling f89c863419d59596bef8cadba943b2cbd6194a4c on isocolsky:ref/client into c1e8ac02ab16e7a9034739fd9f684ca46cff5de2 on bitpay:master.


Does anyone use Rust installed not via rustup?

FWIW the officially sanctioned way of installing Rust is to use rustup.

@tangmi,
Yes I do know how one can install Rust.
I am asking if anyone really does this.

Because if not, I will remove all not Rustup related code because it takes time to update it.

Although I use rustup, I don't think it is a good idea to remove non standard installation code. There are two important use cases that I have in mind:

People who use their Linux Distribution/ Mac Homebrew for installing Rust. I imagine that many beginners would go this way.
People who want to work on Rust tools themself. e.g. If you want to write code for RLS you need to compile it yourself. To test it in VS Code, you need to be able to define the path to the binary.

If I didn't understand the problem correctly, could you point me to the code in question?

@steffinger,
Good motivation. Thank you.

I used Rust via Homebrew in the beginning and it worked just fine. It was easier to get started since I didn't need any additional tooling.

Thanks for your opinion.


Rust SDK: Import archetypes and other select types into mod rerun

What

Requires https://github.com/rerun-io/rerun/pull/3542 to be merged first

Import selected types into the re_sdk and rerun modules to reduce the typing in user code.
In a lot of examples i opted for putting rerun:: in the code, to make it super-clear to the reader what types are rerun types, and which isn't.
There are a few annoying name-clashes:

archetypes::Viewport and components::Viewport (the latter is used by Pinhole::camera_xyz)
datatype::Color and components::Color (TODO: rename the former to Rgba)
datatypes::Transform3D, components::Transform3D and datatypes::Transform3D

Checklist

[x] I have read and agree to Contributor Guide and the Code of Conduct
[x] I've included a screenshot or gif (if applicable)
[x] I have tested demo.rerun.io (if applicable)

PR Build Summary
Docs preview
Examples preview
Recent benchmark results
Wasm size tracking

In a lot of examples i opted [...]

I really dislike how inconsistent it is now though. We do explicit rerun:: in all of them except cases where we need to go into the longer versions right now.
The ones that do just go rerun:: are much nicer to read now and on average shorter as well 👍

I'll see if I can switch more to the new style


Nixos test driver: Test are failing when busybox is installed

Describe the bug
Test driver is failing when busybox is installed
Steps To Reproduce
Run this test (tested against commitf292b4964cb71f9dfbbd30dc9f511d6165cd109b (unstable branch the 6th of July))  :
{
  nixpkgs ? <nixpkgs>,
}: let
  nixos-lib = import (nixpkgs + "/nixos/lib") { };
  inherit (nixos-lib) runTest;
in runTest (
    {
      name = "Test";

      hostPkgs = import nixpkgs {};

      nodes = {
        machine = ({
          pkgs,
          ...
        }: {
        environment.systemPackages = [
          pkgs.busybox
        ];

        });
      };

      testScript = ''
        start_all()
        machine.wait_for_unit("multi-user.target")
        '';
    }
  )

Relevant output:
machine: Spawning backdoor root shell...

machine: connected to guest root shell
machine: (connecting took 80.61 seconds)
(finished: waiting for the VM to finish booting, in 80.61 seconds)
machine # [   79.791019] (udev-worker)[800]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.
machine # [   79.826603] (udev-worker)[808]: Network interface NamePolicy= disabled on kernel command line.
machine # [   79.840705] (udev-worker)[800]: Network interface NamePolicy= disabled on kernel command line.
machine # base64: unrecognized option '--wrap'
machine # BusyBox v1.36.1 () multi-call binary.
machine #
machine # Usage: base64 [-d] [-w COL] [FILE]
machine #
machine # Base64 encode or decode FILE to standard output
machine #
machine #       -d      Decode data
machine #       -w COL  Wrap lines at COL (default 76, 0 disables)
cleanup
kill machine (pid 6)
machine # qemu-kvm: terminating on signal 15 from pid 4 (/nix/store/1r6n7v2wam7gkr18gxccpg7p5ywgw551-python3-3.10.12/bin/python3.10)
(finished: cleanup, in 0.04 seconds)
Traceback (most recent call last):
  File "/nix/store/nzz7xim7mixhxjqfmh0ah664ahbrmwgs-nixos-test-driver-1.1/bin/.nixos-test-driver-wrapped", line 9, in <module>
    sys.exit(main())
  File "/nix/store/nzz7xim7mixhxjqfmh0ah664ahbrmwgs-nixos-test-driver-1.1/lib/python3.10/site-packages/test_driver/__init__.py", line 112, in main
    driver.run_tests()
  File "/nix/store/nzz7xim7mixhxjqfmh0ah664ahbrmwgs-nixos-test-driver-1.1/lib/python3.10/site-packages/test_driver/driver.py", line 147, in run_tests
    self.test_script()
  File "/nix/store/nzz7xim7mixhxjqfmh0ah664ahbrmwgs-nixos-test-driver-1.1/lib/python3.10/site-packages/test_driver/driver.py", line 143, in test_script
    exec(self.tests, symbols, None)
  File "<string>", line 2, in <module>
  File "/nix/store/nzz7xim7mixhxjqfmh0ah664ahbrmwgs-nixos-test-driver-1.1/lib/python3.10/site-packages/test_driver/machine.py", line 454, in wait_for_unit
    retry(check_active, timeout)
  File "/nix/store/nzz7xim7mixhxjqfmh0ah664ahbrmwgs-nixos-test-driver-1.1/lib/python3.10/site-packages/test_driver/machine.py", line 127, in retry
    if fn(False):
  File "/nix/store/nzz7xim7mixhxjqfmh0ah664ahbrmwgs-nixos-test-driver-1.1/lib/python3.10/site-packages/test_driver/machine.py", line 435, in check_active
    state = info["ActiveState"]
KeyError: 'ActiveState'
kill vlan (pid 5)

Expected behavior
Test should pass.
Additional context
The test is failing as a consequence of  base64 invocation here : https://github.com/NixOS/nixpkgs/blob/fe258c0d3f42fbd8a85538a33c99fc6734970fdc/nixos/lib/test-driver/test_driver/machine.py#L536
This can be easily fixed by changing the --wrap flag to -w (the latest being supported by busybox) in the line above but I don't know how to correctly test my fix since it is a change in a core feature of nixpkgs (For the moment I only tested it on the above test). However I strongly believe it won't break things.
Notify maintainers
There is no maintainers https://github.com/NixOS/nixpkgs/blob/master/nixos/lib/test-driver/default.nix
@R-VdP is the last one to have modified the faulty line according to git blame
cc @RaitoBezarius also
Metadata
Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.
[user@system:~]$ nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-linux"`
 - host os: `Linux 5.15.0-76-generic, Ubuntu, 22.04.2 LTS (Jammy Jellyfish), nobuild`
 - multi-user?: `no`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.13.3`
 - channels(root): `"nixpkgs"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixpkgs`

If it's inside the VM the problem appears, shouldn't we use abspath to the correct base64 instead of hoping the one in PATH is compatible?

Hmm, yeah, but that would introduce a stronger coupling between the test-driver and the VM.
Using /run/current-system/sw/base64 is equally unreliable, because some VMs could have busybox added to the system path, in which case we could still end up with the wrong base64 there.
Using the actual store path would completely avoid these issues, but it would also mean that the test-driver needs to have a lot more information about the VM it runs.
I also have a personal use-case where I use the test-driver with non-NixOS VMs, which would not be possible anymore if we tie it more closely to NixOS. This is not an argument against such coupling in and of itself, and I can always adapt by carrying my own patches or fork, but there might be other such uses around.

Ok, making it compatible with busybox sounds fine.