CouchDB 3.x: JobAccountant fails to authenticate to localhost CouchDB

Impact of the bug
WMAgent using CouchDB 3.x
Describe the bug
While testing a private build of WMAgent 1.5.7.patch2 + CouchDB 3.1, I noticed that JobAccountant fails to update fwjrs documents in local CouchDB. This is the error message recorded in the logs [1]
Note that the agent configuration config.JobStateMachine.couchurl already has the url in the correct format.
How to reproduce it
None
Expected behavior
The component should be setting the Authorization header and passing it over through the http request, such that updates to the CouchDB database are successful.
Additional context and error message
[1]
2022-03-17 19:58:52,098:139926084232960:INFO:AccountantWorker:Job 1065 , handle successful job
2022-03-17 19:58:52,104:139926084232960:ERROR:AccountantWorker:Error occurred: associating log collect location, will try again
 CouchUnauthorisedError - reason: Unauthorized, data: {} result: b'{"error":"unauthorized","reason":"You are not a server admin."}\n'
2022-03-17 19:58:52,104:139926084232960:INFO:AccountantWorker:Handling /data/srv/wmagent/v1.5.7.patch2-127c31293503ab5f4da2da887a06775a/install/wmagentpy3/JobCreator/JobCache/amaltaro_TC_PY3_ProdPsi_Mar2022_Val_220312_025605_5711/GenSimFu
ll/GenSimFullMergeFEVTDEBUGoutput/Digi_2021noPU/Reco_2021noPU/Reco_2021noPUMergeDQMoutput/Reco_2021noPUMergeDQMoutputEndOfRunDQMHarvestMerged/Reco_2021noPUMergeDQMoutputMergedEndOfRunDQMHarvestLogCollect/JobCollection_146_0/job_1066/Repor
t.0.pkl
2022-03-17 19:58:52,106:139926084232960:INFO:AccountantWorker:Job 1066 , handle successful job

UPDATE: the same problem happens to the CleanCouchPoller, a thread of the TaskArchiver component. Logs are:
2022-03-18 03:11:24,399:140694967277312:INFO:LogDB:<LogDB(url=https://alancc7-cloud1.cern.ch/couchdb/wmstats_logdb, identifier=vocms0261.cern.ch, agent=1)>
2022-03-18 03:11:24,411:140694967277312:ERROR:BaseWorkerThread:Error in event loop (2): <WMComponent.TaskArchiver.CleanCouchPoller.CleanCouchPoller object at 0x7ff61a13bc10> CouchUnauthorisedError - reason: Unauthorized, data: {} result: b'{"error":"unauthorized","reason":"You are not a server admin."}\n'
Backtrace:
  File "/data/srv/wmagent/v1.5.7.patch2-127c31293503ab5f4da2da887a06775a/sw.amaltaro/slc7_amd64_gcc630/cms/wmagentpy3/1.5.7.patch2-127c31293503ab5f4da2da887a06775a/lib/python3.8/site-packages/WMCore/WorkerThreads/BaseWorkerThread.py", line 161, in __call__
    self.initInThread(parameters)
  File "/data/srv/wmagent/v1.5.7.patch2-127c31293503ab5f4da2da887a06775a/sw.amaltaro/slc7_amd64_gcc630/cms/wmagentpy3/1.5.7.patch2-127c31293503ab5f4da2da887a06775a/lib/python3.8/site-packages/WMCore/WorkerThreads/BaseWorkerThread.py", line 147, in initInThread
    self.setup(parameters)
  File "/data/srv/wmagent/v1.5.7.patch2-127c31293503ab5f4da2da887a06775a/sw.amaltaro/slc7_amd64_gcc630/cms/wmagentpy3/1.5.7.patch2-127c31293503ab5f4da2da887a06775a/lib/python3.8/site-packages/WMComponent/TaskArchiver/CleanCouchPoller.py", line 131, in setup
    self.jobsdatabase = self.jobCouchdb.connectDatabase("%s/jobs" % jobDBName)
  File "/data/srv/wmagent/v1.5.7.patch2-127c31293503ab5f4da2da887a06775a/sw.amaltaro/slc7_amd64_gcc630/cms/wmagentpy3/1.5.7.patch2-127c31293503ab5f4da2da887a06775a/lib/python3.8/site-packages/WMCore/Database/CMSCouch.py", line 1008, in connectDatabase
    if create and dbname not in self.listDatabases():
  File "/data/srv/wmagent/v1.5.7.patch2-127c31293503ab5f4da2da887a06775a/sw.amaltaro/slc7_amd64_gcc630/cms/wmagentpy3/1.5.7.patch2-127c31293503ab5f4da2da887a06775a/lib/python3.8/site-packages/WMCore/Database/CMSCouch.py", line 981, in listDatabases
    return self.get('/_all_dbs')
  File "/data/srv/wmagent/v1.5.7.patch2-127c31293503ab5f4da2da887a06775a/sw.amaltaro/slc7_amd64_gcc630/cms/wmagentpy3/1.5.7.patch2-127c31293503ab5f4da2da887a06775a/lib/python3.8/site-packages/WMCore/Services/Requests.py", line 122, in get
    return self.makeRequest(uri, data, 'GET', incoming_headers,
  File "/data/srv/wmagent/v1.5.7.patch2-127c31293503ab5f4da2da887a06775a/sw.amaltaro/slc7_amd64_gcc630/cms/wmagentpy3/1.5.7.patch2-127c31293503ab5f4da2da887a06775a/lib/python3.8/site-packages/WMCore/Database/CMSCouch.py", line 135, in makeRequest
    self.checkForCouchError(getattr(e, "status", None),
  File "/data/srv/wmagent/v1.5.7.patch2-127c31293503ab5f4da2da887a06775a/sw.amaltaro/slc7_amd64_gcc630/cms/wmagentpy3/1.5.7.patch2-127c31293503ab5f4da2da887a06775a/lib/python3.8/site-packages/WMCore/Database/CMSCouch.py", line 151, in checkForCouchError
    raise CouchUnauthorisedError(reason, data, result)

I'm going to add a fix for this as well in the same PR fixing the originally reported issue.

And now WorkQueueManager also complains when writing out the spec file, likely because I reverted one line change in #11001 , which didn't look the correct fix anyways. Here is the component traceback (after modifying an error to exception log level):
2022-03-18 04:41:45,467:139675677374208:INFO:WorkQueue:Got 6 elements matching the constraints
2022-03-18 04:41:45,474:139675677374208:ERROR:WorkQueueManagerWMBSFileFeeder:Error in wmbs inject loop: url=http://localhost:5984/workqueue/amaltaro_TaskChain_ProdMinBias_Mar2022_Val_220318_034116_1199/spec, code=401, reason=Unauthorized, headers={'Cache-Control': 'must-revalidate', 'Content-Length': '78', 'Content-Type': 'application/json', 'Date': 'Fri, 18 Mar 2022 03:41:45 GMT', 'Server': 'CouchDB/3.1.2 (Erlang OTP/22)', 'X-Couch-Request-ID': 'a205801ca0', 'X-CouchDB-Body-Time': '0'}, result=b'{"error":"unauthorized","reason":"You are not authorized to access this db."}\n'
Traceback (most recent call last):
  File "/data/srv/wmagent/v1.5.7.patch2-127c31293503ab5f4da2da887a06775a/sw.amaltaro/slc7_amd64_gcc630/cms/wmagentpy3/1.5.7.patch2-127c31293503ab5f4da2da887a06775a/lib/python3.8/site-packages/WMComponent/WorkQueueManager/WorkQueueManagerWMBSFileFeeder.py", line 57, in algorithm
    previousWorkList = self.queue.getWork(resources, jobCounts,
  File "/data/srv/wmagent/v1.5.7.patch2-127c31293503ab5f4da2da887a06775a/sw.amaltaro/slc7_amd64_gcc630/cms/wmagentpy3/1.5.7.patch2-127c31293503ab5f4da2da887a06775a/lib/python3.8/site-packages/WMCore/WorkQueue/WorkQueue.py", line 333, in getWork
    wmspec = self.backend.getWMSpec(match['RequestName'])
  File "/data/srv/wmagent/v1.5.7.patch2-127c31293503ab5f4da2da887a06775a/sw.amaltaro/slc7_amd64_gcc630/cms/wmagentpy3/1.5.7.patch2-127c31293503ab5f4da2da887a06775a/lib/python3.8/site-packages/WMCore/WorkQueue/WorkQueueBackend.py", line 148, in getWMSpec
    wmspec.load(self.db['host'] + "/%s/%s/spec" % (self.db.name, name))
  File "/data/srv/wmagent/v1.5.7.patch2-127c31293503ab5f4da2da887a06775a/sw.amaltaro/slc7_amd64_gcc630/cms/wmagentpy3/1.5.7.patch2-127c31293503ab5f4da2da887a06775a/lib/python3.8/site-packages/WMCore/WMSpec/Persistency.py", line 78, in load
    data = request.makeRequest('', incoming_headers={"Accept": "*/*"}, decoder=False)
  File "/data/srv/wmagent/v1.5.7.patch2-127c31293503ab5f4da2da887a06775a/sw.amaltaro/slc7_amd64_gcc630/cms/wmagentpy3/1.5.7.patch2-127c31293503ab5f4da2da887a06775a/lib/python3.8/site-packages/WMCore/Services/Requests.py", line 161, in makeRequest
    result, response = self.makeRequest_pycurl(uri, data, verb, headers)
  File "/data/srv/wmagent/v1.5.7.patch2-127c31293503ab5f4da2da887a06775a/sw.amaltaro/slc7_amd64_gcc630/cms/wmagentpy3/1.5.7.patch2-127c31293503ab5f4da2da887a06775a/lib/python3.8/site-packages/WMCore/Services/Requests.py", line 178, in makeRequest_pycurl
    response, result = self.reqmgr.request(uri, data, headers, verb=verb,
  File "/data/srv/wmagent/v1.5.7.patch2-127c31293503ab5f4da2da887a06775a/sw.amaltaro/slc7_amd64_gcc630/cms/wmagentpy3/1.5.7.patch2-127c31293503ab5f4da2da887a06775a/lib/python3.8/site-packages/Utils/PortForward.py", line 69, in portMangle
    return callFunc(callObj, url, *args, **kwargs)
  File "/data/srv/wmagent/v1.5.7.patch2-127c31293503ab5f4da2da887a06775a/sw.amaltaro/slc7_amd64_gcc630/cms/wmagentpy3/1.5.7.patch2-127c31293503ab5f4da2da887a06775a/lib/python3.8/site-packages/WMCore/Services/pycurl_manager.py", line 306, in request
    raise exc

when it's localhost, we either pass the credentials in the url, or we pass it with the Authorization header.


Apply/517 qt analytics - save History and session time

What's included?
On this ticket we are adding History and Session time to the database. This will be stored and showed on the admin side.
Who should test?
✅ Product owner
✅ Developers
✅ UTG
How to test?
There are no changes to test on the apply as this is a backend ticket - we are just storgin information from the user, without any visual interface or interaction.
This needs to be tested in conjunction with the #945 ticket (admin).
Risk - how likely is this to impact other areas?
🟠 Medium risk - this does change code that is shared with other areas

PREVIEW:DEVELOP

This is ready for re-review.

Please refer to #1317. Some final tweaks required to the tickets which run together.

@warrensearle @londonprojects Can someone take a look at this?

@lloback see https://github.com/jac-uk/admin/pull/1317 for changes to make

Fix the "Saved"
"How many times changed answer:"
The action: 'changed' only works for Situational Judgement and Critical Analysis. Whenever the user clicks on an answer that is different from the previous selection, this count gets updated.

Approved by UTG


Core not rejecting bad settings frame

Sending a settings frame with SETTINGS_MAX_FRAME_SIZE below 2^14 is protocol error according to the http2 spec.  The server should respond with a goaway frame.

Do you have a repro? I just tried to get one on our end2end tests by hacking the client to send bad setting, and the server does indeed respond with a goaway.
It comes from here: https://github.com/grpc/grpc/blob/701a4b1c32a6da0c246bfe573ef3c96a9c652cc2/src/core/ext/transport/chttp2/transport/frame_settings.cc#L203-L212


LitePullConsumer Ack

The example code shows how to use DefaultLitePullConsumer to pull data from rocketmq:
https://github.com/apache/rocketmq/blob/8ef01a6c635f6972847c40d5540b1945180d7cbd/example/src/main/java/org/apache/rocketmq/example/simple/LitePullConsumerSubscribe.java#L35
But i noticed that unlike the push mode( return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;), there is no code to reply an ack to the rocketmq for the pull consumer. So the question is : To ensure the data are correctly pulled,  what should i do to manually reply an ack to the rockemq when using DefaultLitePullConsumer?
thanks~

you can reply ack  to broker by commitSync()

Issues are generally used for bug reports and feature requests, so if you have any questions about     Apache  RocketMQ, it is recommended to discuss it on the mailing list first, the mailing list [address]    (https://rocketmq.apache.org/about/contact). I will close the Issue first, but please feel free to     reopen it if you have any other issues.


Add the ability to listen for the actual value of AsyncValue

Is your feature request related to a problem? Please describe.
It's not an issue but the new approach of listening to AsyncValue (the unreleased one) is not clear at some cases when you're just interested in the actual value of AsyncValue (the old approach). To clarify:
the old approach:
ref.listen<AsyncValue<void>>(
  signInProvider,
  (prevState, newState) {
    prevState?.whenOrNull(
      loading: () => dismissLoadingDialog(),
    );
    newState.whenOrNull(
      loading: () => showLoadingDialog(),
      error: (failure, st) {
        showFailToast(failure);
      },
    );
  },
);

It's clear that you want to show dialog when the new state is loading and hide it when you leave loading state. and you want to show a toast when new state is error state.
But, when we apply this to the new approach:
    ref.listen<AsyncValue<void>>(
      signInProvider,
      (prevState, newState) {
        if (newState.isLoading) showLoadingDialog();
        if (prevState != null && prevState.isLoading) {
           dismissLoadingDialog();
        }
        if (newState.hasError && !newState.isLoading) {
           showFailToast(failure);
        }
      },
    );

It has tricky checks between hasError and isLoading. I think this is more error-prone and not clear as first approach.
Describe the solution you'd like
I think something like this would be great:
    ref.listen<AsyncValue<void>>(
      signInProvider,
      (prevState, newState, prevActualState, newActualState) {   // prevActualState, newActualState
        prevActualState?.whenOrNull(
          loading: () => dismissLoadingDialog(),
        );
        newActualState.whenOrNull(
          loading: () => showLoadingDialog(),
          error: (failure, st) {
            showFailToast(failure);
          },
        );
      },
    );

another suggestion:
    ref.listen<AsyncValue<void>.actualValue>(     // .actualValue
      signInProvider,
      (prevState, newState) {
      prevState?.whenOrNull(
          loading: () => dismissLoadingDialog(),
        );
      newState.whenOrNull(
          loading: () => showLoadingDialog(),
          error: (failure, st) {
            showFailToast(failure);
          },
        );
      },
    );

Maybe you're looking forAsyncValue.unwrapPrevious, which converts the AsyncValue into one without the previous state.

Yes, that's exactly what I was suggesting. This works for the mentioned example and It's as clear as the old approach:
    ref.listen<AsyncValue<void>>(
      signInProvider,
      (prevState, newState) {
        prevState?.unwrapPrevious().whenOrNull(
              loading: () => dismissLoadingDialog(),
            );
        newState.unwrapPrevious().whenOrNull(
              loading: () => showLoadingDialog(),
              error: (failure, st) {
                showFailToast(failure);
              },
            );
      },
    );


Website with Github Pages

You could move your website in a repository called victorrentea.github.io and it will automatically go live at http://victorrentea.github.io, all free of charge with HTTPS included (starting this year, also if you have a custom domain). See more about Github Pages here.
Also, if you want to get rid of having to edit HTML, you could use Jekyll, essentially turning the website's sources into Yaml/Markdown files. It's also integrated with Github, the Jekyll build will start automatically once you merge some changes.

Super cool advice, thank you!
If only I could keep up with all the good advices, these days! :)
But really, I'll try this one. I'm tired of filezilla :)
Victor Rentea
Independent Technical Trainer, Lead Architect at IBM
www.victorrentea.ro
(+4)<PHONE_NUMBER>
PS: Written on a tiny device. Please excuse any tpyos and abbrev
On Mon, Sep 3, 2018, 13:11 Mihai A<EMAIL_ADDRESS>wrote:

You could move your website
https://github.com/victorrentea/training/tree/master/victorrentea.ro in
a repository called victorrentea.github.io and it will automatically go
live at http://victorrentea.github.io, all free of charge with HTTPS
included (starting this year, also if you have a custom domain). See more
about Github Pages here https://pages.github.com/.
Also, if you want to get rid of having to edit HTML, you could use Jekyll
https://jekyllrb.com/, essentially turning the website's sources into
Yaml/Markdown files. It's also integrated with Github, the Jekyll build
will start automatically once you merge some changes.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/victorrentea/training/issues/2, or mute the thread
https://github.com/notifications/unsubscribe-auth/AL5yRjB35vxwbmIXsPwBaetX0gF_dvRnks5uXQBIgaJpZM4WXVff
.


camerad: reduce cpu usage

How much does this save?

Haven't tested on device yet. maybe 4～6%(not sure),  there is no reason to do "busy" waiting on try_pop IMO.

Seems to save about 1-2% of a core


Syslog messages sometimes not prefixed correctly

Here's a sample of syslog messages from my /var/log/messages:
Mar  3 19:27:29 rockyduck.bld.corp.google.com stenographer[10202]: 2016-03-04T02:27:29.755193Z T:921fd7 [aio.cc:190] Opening packet file /tmp/stenographer551644138/PKT0/.1457058449627005: 4
2016-03-04T02:27:29.756744Z T:919fc7 [index.cc:269] Wrote all index files for /tmp/stenographer551644138/IDX0/.1457058388538817, moving to /tmp/stenographer551644138/IDX0/1457058388538817
Mar  3 19:27:50 rockyduck.bld.corp.google.com stenographer[10202]: 2016-03-04T02:27:50.795461Z T:921fd7 [stenotype.cc:499] Thread 0 stats: MB=2413 secs=15934.3 MBps=0.151434 packets=3425698 blocks=2413 polls=16248 drops=0 drop%=0
Mar  3 19:28:30 rockyduck.bld.corp.google.com stenographer[10202]: 2016-03-04T02:28:30.872843Z T:921fd7 [aio.cc:122] Closing /tmp/stenographer551644138/PKT0/.1457058449627005 (4), truncating to 7MB and moving to /tmp/stenographer551644138/PKT0/1457058449627005
Mar  3 19:28:30 rockyduck.bld.corp.google.com stenographer[10202]: 2016-03-04T02:28:30.872979Z T:921fd7 [aio.cc:190] Opening packet file /tmp/stenographer551644138/PKT0/.1457058510749942: 4
Mar  3 19:28:30 rockyduck.bld.corp.google.com stenographer[10202]: 2016-03-04T02:28:30.874880Z T:919fc7 [index.cc:269] Wrote all index files for /tmp/stenographer551644138/IDX0/.1457058449627005, moving to /tmp/stenographer551644138/IDX0/1457058449627005
Mar  3 19:27:29 rockyduck.bld.corp.google.com stenographer[10202]: 2016-03-04T02:27:29.749447Z T:921fd7 [aio.cc:122] Closing /tmp/stenographer551644138/PKT0/.1457058388538817 (4), truncating to 6MB and moving to /tmp/stenographer551644138/PKT0/1457058388538817
Mar  3 19:27:29 rockyduck.bld.corp.google.com stenographer[10202]: 2016-03-04T02:27:29.755193Z T:921fd7 [aio.cc:190] Opening packet file /tmp/stenographer551644138/PKT0/.1457058449627005: 4
2016-03-04T02:27:29.756744Z T:919fc7 [index.cc:269] Wrote all index files for /tmp/stenographer551644138/IDX0/.1457058388538817, moving to /tmp/stenographer551644138/IDX0/1457058388538817
Mar  3 19:27:50 rockyduck.bld.corp.google.com stenographer[10202]: 2016-03-04T02:27:50.795461Z T:921fd7 [stenotype.cc:499] Thread 0 stats: MB=2413 secs=15934.3 MBps=0.151434 packets=3425698 blocks=2413 polls=16248 drops=0 drop%=0
Mar  3 19:28:30 rockyduck.bld.corp.google.com stenographer[10202]: 2016-03-04T02:28:30.872843Z T:921fd7 [aio.cc:122] Closing /tmp/stenographer551644138/PKT0/.1457058449627005 (4), truncating to 7MB and moving to /tmp/stenographer551644138/PKT0/1457058449627005
Mar  3 19:28:30 rockyduck.bld.corp.google.com stenographer[10202]: 2016-03-04T02:28:30.872979Z T:921fd7 [aio.cc:190] Opening packet file /tmp/stenographer551644138/PKT0/.1457058510749942: 4
Mar  3 19:28:30 rockyduck.bld.corp.google.com stenographer[10202]: 2016-03-04T02:28:30.874880Z T:919fc7 [index.cc:269] Wrote all index files for /tmp/stenographer551644138/IDX0/.1457058449627005, moving to /tmp/stenographer551644138/IDX0/1457058449627005

Note the two lines that start with 2016, which are missing the normal "Month Day Time Host Daemon[PID]:" prefix.

These all appear to be coming from stenotype, so my guess is it's because of the fact that we pass a syslog Logger into stenotype as STDOUT/STDERR.  In stenographer.go:
logwriter, err := syslog.New(syslog.LOG_USER|syslog.LOG_INFO, "stenographer")
...
stenotypeOutput = logwriter // for stenotype
...
env.StenotypeOutput = stenotypeOutput

In env/env.go:
cmd.Stdout = d.StenotypeOutput
cmd.Stderr = d.StenotypeOutput

We've got C writing to stderr, and (correctly) flusing after each write using flush here in util.h:
~LogLine() {
  ss_ << "\n";
  std::cerr << ss_.str() << std::flush;

But that flushes into STDERR, which Go picks up and tries to pipe into the syslog logwriter.  It uses an os.Pipe for this that it created with writerDescriptor.  I expect it's this pipe that, in certain circumstances, buffers two lines together into a single Write call to the syslog writer.


Additional IN / NOT IN simplifications

          In the following on PR, I think we can move inlist related simplifier logic to `inlist_simplifier`.

Steps in inlist_simplifier

early stage simplification like expr IN () --> false, null IN/NOT IN
convert long OR chain or AND chain to InList expressions like what or_in_list_simplifier does
shorten InList expressions like what inlist_simplifier in this PR does
convert final inlist to OR chain or AND chain if len < 3. Move inlist related simplifier out from expr_simplifier

In this way, when the logic grows more complex, we can easily find out related conversion about specific Expr
Originally posted by @jayzhan211 in https://github.com/apache/arrow-datafusion/issues/8949#issuecomment-1906115890

I am interested in this and let me try to implement it

I think this issue might be done best in multiple PRs
SO for example the first PR could just move all the inlist related simplifications into one place
Note I also have a PR to change the code, for your awareness: https://github.com/apache/arrow-datafusion/pull/8971

ok, I got it 😄

tasks:

[ ] early stage simplification like expr IN () --> false, null IN/NOT IN
[ ] convert long OR chain or AND chain to InList expressions like what or_in_list_simplifier does
[ ] shorten InList expressions like what inlist_simplifier in this PR does
[ ] convert final inlist to OR chain or AND chain if len < 3. Move inlist related simplifier out from expr_simplifier


[WIP] Chunk Storage

Gotta run for now - will get to the second half of the file later today.

I'm wondering if an hour as the hash key is a good time range. A month-long query with a single matcher would require 720 DynamoDB lookups (24 * 30) to find its chunks, plus most chunks would be in multiple hour buckets, so there'd be a lot of storage redundancy and deduping. Maybe a couple of hours or even a day would be better?

Looks generally good otherwise!

I'm wondering if an hour as the hash key is a good time range. A month-long query with a single matcher would require 720 DynamoDB lookups (24 * 30) to find its chunks, plus most chunks would be in multiple hour buckets, so there'd be a lot of storage redundancy and deduping. Maybe a couple of hours or even a day would be better?

Yes we can be flexible here, we just need to balance against 2 things - rows getting too wide, and all the writes focusing on a single row.  I think the write rate is low enough that we don't need to worry about either.

Just for clarification: when you say "rows getting too wide", what exactly do you mean with a "row"? Are you concerned about too many range keys within a given hash bucket?

Just for clarification: when you say "rows getting too wide", what exactly do you mean with a "row"? Are you concerned about too many range keys within a given hash bucket?

Yes this is basically my concern.  It comes from a bunch of places in Cassandra where operation were per-row, so big rows caused problems.  This probably doesn't apply to DynamoDB.  The analogy with DynamoDB is that each hash bucket has to fit on a single machine, and their sizes would be roughly equal to achieve good load balancing.

Thanks, makes sense. I wasn't aware that each hash bucket needed to fit on a single machine with DynamoDB, good to know.

Hmm, http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Limits.html#limits-partition-sort-keys says, "In general, there is no practical limit on the number of distinct sort key values per partition key value."

Good to merge for now from my perspective. We can iron out stuff later.


Don't require boost when it is not used

Boost is not used in the code, but required to build the project. Compare https://github.com/microsoft/vcpkg/issues/39850

Good catch, the boost dep was a leftover from the early days when I used boost.hana.


functions predeploy error: Command terminated with non-zero exit code2

Hey everyone, I just started with the first tutorial "Getting Started with Cloud Functions for Firebase using TypeScript - Firecasts" on YouTube.
While trying to deploy the code, I get the error shown below.
Device: Macbook Pro
Node version: v12.16.3
Typescript version: version 3.8.3
Expected behavior
Deploy successfully
Actual behavior
vijayraja@Vijays-MBP functions % firebase deploy

=== Deploying to 'pleasework-770e8'...

i  deploying functions
Running command: npm --prefix "$RESOURCE_DIR" run lint

> functions@ lint /Users/vijayraja/newfire/functions
> tslint --project tsconfig.json

Running command: npm --prefix "$RESOURCE_DIR" run build

> functions@ build /Users/vijayraja/newfire/functions
> tsc

src/index.ts:1:1 - error TS6133: 'functions' is declared but its value is never read.

1 import * as functions from 'firebase-functions';
  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Found 1 error.

npm ERR! code ELIFECYCLE
npm ERR! errno 2
npm ERR! functions@ build: `tsc`
npm ERR! Exit status 2
npm ERR!
npm ERR! Failed at the functions@ build script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR!      /Users/vijayraja/.npm/_logs/2020-04-29T06_39_58_655Z-debug.log

Error: functions predeploy error: Command terminated with non-zero exit code2

I have attached the all the screenshots below:
Error:

My index.ts file :

Firebase.json file:

tsconfig.json file:

package.json file:

I'm aware there are other threads with the same issue, but I tried all of them, none of them worked.
What I tried doing:

Place the "typeRoots" key inside "compilerOptions" (https://github.com/firebase/firebase-tools/issues/749)
npm run build and restart VSCode (https://github.com/firebase/firebase-tools/issues/749)

Please help.

@morganchen12

@Vijay1126 did you save your index.ts file before trying to deploy? That's a linter error that's saying the functions variable defined in line 1 of index.ts isn't used. You use it in line 6, so I think it's just a matter of saving that file and trying again 😄

THANK YOU!


Add PHPStan

Shameless bump on this ;)

If we're going to add phpstan we should add it to travis as well.

Resolved via #163


Fish Shell Completions

This PR adds desk completions for the Fish shell. It is based heavily off of Docker's fish completions.
One note: I am unable to get . to show as a suggestion for the sub-command-less desk (i.e., desk<tab> does not show . as an available sub-command; it only shows go). I assume this is a limitation of Fish.
If merged, we should also add
fish_completion.install "shell_plugins/fish/desk.fish"

to the Desk Homebrew package.

Sure thing. I'll update later today
Christian Bankester
cbankester<EMAIL_ADDRESS>On Thu, Jan 21, 2016 at 9:20 AM -0800, "jamesob" <notifications<EMAIL_ADDRESS>wrote:
Hey, @cmbankesterhttps://github.com/cmbankester, sorry for the delay and thanks for the PR. Would you also mind updating the "Enabling shell extensions" of the README.md? If you don't have time, I can do so in a follow-up PR.

Reply to this email directly or view it on GitHubhttps://github.com/jamesob/desk/pull/51#issuecomment-173642756.

Updated! Let me know if you need me to do anything else related to this PR.

Rad, thanks (as always) @cmbankester. Your contributions are muy apreciado.


tlog server should use 0-stor-lib (golang)

suggestion

remove ardb backend code for tlogserver
use 0-stor-lib: see https://github.com/zero-os/0-stor-lib/blob/master/specs/principles.md

this has following benefits

1 single code path for all our storage needs
etcd is redundant store for the metadata
etcd can be store for double linked list

these components need to be changed to use 0-stor-lib:

[ ] flusher
[ ] slave sync (only the metadata)
[ ] tlog decoder

@despiegk can this be pushed to 1.1.0-alpha-7 please? The new team members are getting still up to date with the codebase. And it seems that the 0-stor-lib and 0-stor are still very far away from getting ready for usage. So I'm very skeptical that getting this ready and than also integrated in this codebase is realistic in the next 7 days.
Even having on this 1.1.0-alpha-7 is going to be rough, but at least it would buy as another week, which might still not be enough.

Once the 0-stor-lib policy (described in https://github.com/zero-os/0-stor-lib/blob/master/specs/principles.md#pseudo-language-to-use-lib-example-in-python) it is pretty straightforward to move to 0-stor-lib from tlogserver point of view. It is because the metadata of tlog is pretty simple:

tlog flusher : redis array of 5 last hashes.
nbdserver's slave syncer : last synced sequence

I added task list to the issue description

WIP can be found on zerostor branch
progress:

all test passed, except the test -race
test -race passed in travis-testrace-tlog branch. It split current testrace into three jobs:

nbd & config
tlogclient & stor package
tlogserver
It is because the testrace just failed without any error or timeout message. After testing it in VM with small memory and CPU, i found that the test just killed, don't know the reason, still investigating

todo:

merge solution in travis-testrace-tlog to zerostor branch
documentation
fix command line config (if needed)

Test race failed sometime on nbdserver's TestBackendSigtermHandler, more info and plan can be found at https://github.com/zero-os/0-Disk/issues/381#issuecomment-323253812


[Map] Dm-Abandoned2

Hash: 39111ce7e8e58726ecfedbcd6e8e16dd880c7480
Current name: Dm-Abandoned2
According to the readme, the author is "Dangerus" Dave O [as written here with the inverted commas and misspelling]
The readme says this is a second fixed version so quite possibly DM-Abandoned is also by this author

Updated both maps, found a variation with a readme confirming author.


Location Handling

This is an issue ticket to address Location handling as a whole as as there are a lot of issues to consider and they'll likely need to be addressed together:

https://github.com/JeffersonLab/jaws-web/issues/73
https://github.com/JeffersonLab/jaws-web/issues/74

Other Location filtering ideas to keep in mind:

It may be useful to remember Location filter selection between web browsing sessions or at least during a single browsing session navigating between tabs (URL param vs cookie vs browser user storage)
It may be useful to share Location filter selection between users (related: global Location filter: https://github.com/JeffersonLab/jaws-web/issues/36).  Note: that may be overly complicated.  We could just share the filtering parameters and avoid setup of a separate Kafka topic and avoid performing the actual filtering centrally.  Note: if we DO do filtering centrally it must support multiple facilities / multiple new Kafka topics, one for each facility - advantage of client side is each client can do whatever based on what control room they're in.
It may be useful to allow conditionally enabling Location inheritance.  In JAWS locations are hierarchical, but also an alarm can be assigned multiple.   An effort is made to avoid assigning alarms a location that is implied by an existing assigned location by virtue of inheritance.  In other words only the most specific, non-overlapping locations should be assigned.  The CED Sync for example automates this when examining SegMasks.  The way Location filtering works at the moment is records from all Locations are included unless a non-empty filter is provided, in which case only the selected Locations are used when selecting records, but Location inheritance is used to include any child Locations implied by parent locations. This is probably most commonly what users want, but it does mean there is currently no way to ask to see records that have exactly a specific Location and excluding child Locations.  Most notably there is no way at the moment to see Site-wide ONLY records because you can't indicate you want to disable inheritance, but also because you literally can't select the site-wide JLab root either: (related: https://github.com/JeffersonLab/jaws-web/issues/68).
Keep in mind Location filtering appears in many tabs on the web app so updates may be spread wide

Note: When editing alarm locations via Inventory page form we could apply the same SegMask / Location squashing logic to avoid alarms with implied parent locations (as is already done via internal CED sync).


reset password url is redirecting to the login page.

Description:
Password reset URL is going to the login page.
Server Setup Information:

Version of Rocket.Chat Server: 0.61.0
Operating System: CentOS 7
Deployment Method(snap/docker/tar/etc): RPM
Number of Running Instances: 1
DB Replicaset Oplog: Disabled
Node Version: v8.9.3
mongoDB Version: 2.6.12

Steps to Reproduce:

When a user clicks on reset password.
User gets the email and a link to reset their password.
User then clicks on the reset password link.
User is redirected back to the login page.

Expected behavior:
The link shows the reset password page
Actual behavior:
The link goes to the login page
Other information -- the email example below:
Hello,
To reset your password, simply click the link below.
https://[hostDNS]/#/reset-password/[SHA256 hash]
Thanks.
* I found that removing the # hash, the link goes to the correct URL.
https://[hostDNS]/reset-password/[SHA256 hash]
* Where can I change this setting?
Thank you in advance.

Just for information, I was looking at the app.js which had this line and thought this was the problem.  I removed the #/ and restarted the service but still the same error.
...
Accounts.emailTemplates.resetPassword.html = function (userModel, url) {
url = url.replace('/#/', '/');
return html.replace(/[Forgot_Password_Url]/g, url);
};
...

code block:
https://github.com/RocketChat/Rocket.Chat/blob/develop/server/methods/sendForgotPasswordEmail.js#L14

I got around the problem by adding a place holder for the reset password URL.
https://hostname/reset-password/sha256

The only issue I have with this is this sha256 part,  I attempted to use the URL with just https://hostname/reset-password/ but the page hangs.
Can this be changed somehow, without the /sha256 part?
Allan

I think I just found where the #/ is.
https://github.com/meteor/meteor/blob/devel/packages/accounts-base/url_server.js
Can the #/ be remove as this seem to be causing the url issue.

Issue moved to RocketChat/Rocket.Chat #9923 via ZenHub


Wrapping and unwrapping DataNewtype blows up script budget

https://github.com/Plutonomicon/plutarch-plutus/blob/429d1b0292d87650a7beb1351612629434a0903f/plutarch-ledger-api/src/Plutarch/LedgerApi/V1/Time.hs#L67-L96
This module is bricked. Switched from PInteger to PPosixTime and my benchmarks had a huge regression. The issue is that all the operations wrap and unwrap the data, this results in a ton of iData and unIData builtin calls, because each addition unwrapped each integer from data and then wraps the result. Instead we should have two definitions PosixTimeData and PosixTime or anything else so that this can be avoided and we can just keep it in the non-data encoded format until we want to go back to data encoding.

Yes, I'm aware of this issue. We are going to remove PDataNewtype which was a big mistake


[doc] Client request methods return value

According to documentation request methods of client return RequestBuilder<U> -> http://hyper.rs/hyper/hyper/client/struct.Client.html#method.get
But it is actually Result(Response)
I'm not sure why it is so, but it looks strange when you see examples and when you see actual documentation for these methods.
Is it wrong or am i missing something?

It is not actually doc issue, but function signature issue i suppose?

Not an issue. Most examples include calling the send() method, which is
what returns the Response.
On Sat, Jul 11, 2015, 9:58 AM Douman<EMAIL_ADDRESS>wrote:

It is not actually doc issue, but function signature issue i suppose?
—
Reply to this email directly or view it on GitHub
https://github.com/hyperium/hyper/issues/605#issuecomment-120627547.


Perform parameter check for user/pass/dbname on Db2Container

While using the Db2Container in a new project I found that if we pass in an invalid user/pass/dbname then the container will emit an error message in logs but the container won't stop itself. This results in the test waiting around for the timeout (10m).
CC @bsideup for review

Hi @aguibert,
While failing fast is a good thing in general, I am not sure whether we should add more constraints to the user input that will be validated by the database. Imagine DB2 relaxing their validators - some users will be unable to use the container at all, although the values they provide are correct.
WDYT?

In general I agree with you @bsideup, it's better to let the validation be done on the backend. However, I was recently trying to use the DB2 container myself in a new project, and it took me 20-30m to get the first DB2 container working (and I wrote it!). I had to attach a log consumer so I could inspect container logs and see what was going on, and then comb through ~300 lines of the DB2 container output in order to see that my username/dbname was invalid. As I mentioned earlier, the DB2 container wasn't doing a good job of failing on its own (it was just sitting there until the 10m timeout), so in this case I think the testcontainers-level validation is warranted.

hi @bsideup, any further thoughts on this one?

@bsideup poking again on this one for stale bot

While there are some good points by you @aguibert regarding the better DX from fail-early validation, we don't feel confident in binding a module implementation to internal implementation details of the used images and strongly violating DRY by this (and for logic that is outside of our control).
There are other examples where this is done in existing modules (e.g. in MSSQL), and we should also remove it there.
However, we also think we can provide generally better DX by referencing the official docs (that ideally mention available configuration options and constraints) from the module documentation, and it is something we would like to follow up with.


Automatically remove whitespace at the beginning and end of the url

Hey there, thanks for creating such a simple yet powerful tool.
When using your tool, I accidentally left a space in front of the url, resulting in the following service request:
https://gatos-jabra-buster.azurewebsites.net/api/RssToCsv?url=+https%3A%2F%2Fportal.mytum.de%2Fjobs%2Fwissenschaftler%2FasRss
This obviously failed with the error message Invalid 'url' parameter.. I would therefore suggest to silently trim of any whitespace at the beginning and end of the url 😎

Thanks @vatbub for reporting this! Sorry it took me 8 months to respond but this is fixed now 😄


We should add a "deprecated-commit-db"

We deleted deprecated bugs from the commit-db, but retained the other metadata. By deleting those lines from commit-db, we lose an important piece of metadata: the commit hashes.
This metadata should be available in some form, and these bugs should still be available to check out if needed (with the understanding that they do not yield triggering tests under Java 8).
We should add a second "deprecated-commit-db" with the removed bugs.

Add a "deprecated-bugs" csv.
Add column for "reason why deprecated", add column for version number where deprecated.

Retitle commit-db to "active-bugs.csv"
Modify existing framework to work with renamed file.
Add header to CSV files explaining columns.


Why so many identical jQuery files?

When traversing the examples, why are there so many copies of the jQuery.min.js file? Couldn't this be optimized by using a cdn to host the jQuery file?

Or we could just delete Jquery from ipfs.io

Closing this issue, since we're discussing reworking the IPFS website as a whole in the near future.


[$1000] Web -  Number field gives an error and at the same time the send btn's background gets changed

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Action Performed:

Open website
Go to settings > Workspaces > Any workspace > Cards > Connect Bank Account
Reach 3rd step of Connect Bank Account
Go to Passport > Get Secure Link > Get link via sms
Type in wrong number and click on send button

Expected Result:
Number field gives an error and at the same time the send btn's background shouldn't get changed
Actual Result:
Number field gives an error and at the same time the send btn's background gets changed
Workaround:
Unknown
Platforms:

Which of our officially supported platforms is this issue occurring on?

[ ] Android / native
[ ] Android / Chrome
[ ] iOS / native
[ ] iOS / Safari
[x] MacOS / Chrome / Safari
[ ] MacOS / Desktop

Version Number: 1.3.47-3
Reproducible in staging?: y
Reproducible in production?: y
If this was caught during regression testing, add the test name, ID and link from TestRail:
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Notes/Photos/Videos: Any additional supporting documentation

https://github.com/Expensify/App/assets/93399543/2b360f88-0880-4b3c-be99-ebcb815c4ebe
Expensify/Expensify Issue URL:
Issue reported by: @muxriddinmuqimov77
Slack conversation: https://expensify.slack.com/archives/C049HHMV9SM/p1690615896430559
View all open jobs on GitHub
Upwork Automation - Do Not Edit

Upwork Job URL: https://www.upwork.com/jobs/~01cbbd9908c732f466
Upwork Job ID:<PHONE_NUMBER>255556096
Last Price Increase: 2023-10-17

Dupe of https://github.com/Expensify/App/issues/23074
ideally this should be fixed from Onfido side or have a customization option for disabled button.

@lschurr Let's put this on hold for #23074

Proposal
Please re-state the problem that we are trying to solve in this issue.
Web - Number field gives an error and at the same time the send btn's background gets changed
What is the root cause of that problem?
Firstly I don't think this is a bug at all because if we clear the input button would get enabled again.
Alternatively if we think this is a bug here's the RCA.
This is an upstream bug of  onfido-skd-ui package. They're doing returning and disabling button once the number is in-valid which shouldn't be the case. They should just show the error return without disabling the button.
What changes do you think we should make in order to solve the problem?
onfido-sdk-ui's code is closed source, couldn't able to find any related code (maybe I'm dumb :P).

We can either create an issue and let onfido team fix this issue (maybe they won't cause they feel it's not a bug).

We have the build version of onfido SDK which we can patch using patch-package, but what to update is here's the question.

I observed the code thoroughly and found out that they're disabling the button in two cases
1. If the API call is loading
2. If the phone number is invalid (which only update when we click on Send Link button)
Here 2nd point is the issue we can update the disabled prop to only n instead of n||a.
Here n means loading status of the API call, a means validation state of phone number.
I know this seems pretty weird fix but this is what I came up with :)

What alternative solutions did you explore? (Optional)
Maybe just wait till onfido does this on their own, or if I could get the repo which this code is hosted I could fix it upstream(depends on the decision from onfido).

We can observe the above code snippet here<EMAIL_ADDRESS>simply do CTRL + F -> disabled:n||a.

Thanks @situchan - on hold

This is on hold.

On hold.

On hold

On hold.

On hold.

On hold.

@lschurr why are we putting this on hold?

This has been on hold since July - https://github.com/Expensify/App/issues/23889#issuecomment-1659043671
cc @situchan

The holding issue was also on hold for very long time. So I started considering another approach - https://github.com/Expensify/App/issues/23074#issuecomment-1752245401

As onfido team will be unlikely to fix this after waiting many months, I think it's fine to apply patch ourselves.
So here, we can fix button background to match our app disabled color. And in https://github.com/Expensify/App/issues/23074, fix wrong position of loading icon.

@lschurr if you agree, we can remove hold and wait for new proposals

Updated my proposal with some proofs that my RCA and solution is correct :)

Cool, let's do it. Can you review @b4s36t4's revised proposal @situchan?

@ginsuma can you submit your proposal here?

@situchan I don't these two issues are similar may I know why you asked to submit proposal here?

@b4s36t4 I don't like your proposal

I don't like your proposal.

You could've asked me to update my proposal :).
I have updated my proposal considering this issue, please check when you have time.

Ready to take other issue if you're feeling good about my proposal.

@ginsuma's proposal looks good to me.
🎀 👀 🎀 C+ reviewed

Proposal looks good, but the 2nd screenshot with the Sending message seems too close to the border on the right, no?
Can we do something to fix that too? Maybe make the button wider in general? cc @Expensify/design just to make sure that would be the correct approach

2nd screenshot with the Sending message seems too close to the border on the right, no?

Oh well, there's already separate issue for that - https://github.com/Expensify/App/issues/23074
If we should fix here, then we can

hmmmm aren't this and that issue duplicates?? Or at least very related... I think we should close one of them and unify them.
Thoughts @situchan @CortneyOfstad @lschurr?

Agree, would be nice if that content was centered within the button itself.

If we think the bounty is too high we should feel comfortable backing it down to $500.

Let's combine them and fix together here and close https://github.com/Expensify/App/issues/23074.
And old base price (before announcement) applies as this is very old issue.
If everyone agrees?

I am fine with that

LGTM cc: @shawnborton

Nice!

@iwiznia all yours!

Could you work on the checklist for this one @situchan?

Payment summary:

Bug reporter: @muxriddinmuqimov77 $50
Contributor: @ginsuma $750
Contributor+: @situchan $750

No regression. This is long existed UI issue from Onfido sdk.
Regression test also not applies here.

Hey @situchan. Who is gonna be eligible for reporter role for this isseu? Me or @MahmudjonToraqulov?

@muxriddinmuqimov77 Not sure about your question. I see that you're marked as reporter on both issues.

@situchan And one more thing. I think the payment should be 250$ as it was
created before 30 August.
2023 okt 26, Pay, 10:53 Situ Chandra Shil @.***>:

Hey @situchan https://github.com/situchan. Who is gonna be eligible for
reporter role for this isseu? Me or @MahmudjonToraqulov
https://github.com/MahmudjonToraqulov?
@muxriddinmuqimov77 https://github.com/muxriddinmuqimov77 Not sure
about your question. I see that you're marked as reporter on both issues.
—
Reply to this email directly, view it on GitHub
https://github.com/Expensify/App/issues/23889#issuecomment-1780458193,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AYFDM5OPIWAVDW4VHSPEZADYBH3EXAVCNFSM6AAAAAA25L7GHSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTOOBQGQ2TQMJZGM
.
You are receiving this because you were mentioned.Message ID:
@.***>

Oh you're right, payment should be the older structure. I'll update the payment summary.

@muxriddinmuqimov77 please accept the offer in Upwork so that we can pay and close. https://www.upwork.com/nx/wm/offer/27261515

Accepted
2023 okt 27, Jum, 22:08 Lauren Schurr @.***>:

@muxriddinmuqimov77 https://github.com/muxriddinmuqimov77 please accept
the offer in Upwork so that we can pay and close.
https://www.upwork.com/nx/wm/offer/27261515
—
Reply to this email directly, view it on GitHub
https://github.com/Expensify/App/issues/23889#issuecomment-1783242056,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AV32OQZFL5EH2PHJ4FN4V4LYBPS7FAVCNFSM6AAAAAA25L7GHSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTOOBTGI2DEMBVGY
.
You are receiving this because you were mentioned.Message ID:
@.***>

All payments made in Upwork. Closing!


Add GiftCard resource

This adds GiftCard resource which is basic CRUD and a disable method.
Review
@csaunders

Can we merge this/

Wouldn't hurt to merge it. The API is still Plus-only though.


Support timeout for combine processor

Hi there,
Please review my idea for https://github.com/Jeffail/benthos/issues/101
Thanks

Hey @andytruong, I think I'd rather add batching by message count to the batch processor rather than add timeouts to the combine processor, then I can deprecate and eventually remove combine entirely since they basically achieve the same goal. Also there's a few issues with this pull request, the main one being that your timed flush calls don't have a way of propagating messages down the pipeline.
I've made an issue for adding message count to the batch processor: https://github.com/Jeffail/benthos/issues/104


text boxes

The text boxes also behave strangely - the cursor appears wherever the user clicks, as opposed to being aligned left.

Fixed


Update for terraform 0.12.26

Can we get a fresh build for 0.12.26 support?

Could you provide some context on what's in the new version that you need support for or that terragrunt is incompatible with?

Closing, as Terragrunt has supported 0.12, 0.13, and 0.14 for a while. https://terragrunt.gruntwork.io/docs/getting-started/supported-terraform-versions/


Restructure app and add code examples

Restructures our backend B2B demo app so that it aligns more closely with our Consumer demo app format. Adds more in-depth explanatory text and example code snippets.

Could you updated the README to reflect the new changes?

Could you updated the README to reflect the new changes?

Ahh yes, very good call, thank you! Just opened a separate PR for that: https://github.com/stytchauth/stytch-b2b-node-example/pull/39


Initial Documentation

Need to create initial set of documentation for starting/running the API.
This would include updated Readme and Contributing docs, as well as setting up a Swagger based API doc.

Updated the Readme and added the initial Swagger/OpenAPI docs.
Probably need to still add more detail, but some of the information is now there.

Need to add Request/Repsonse payload examples to documentation.


Debugging doesn't work

Extension Name: vscode-gradle
Extension Version: 3.6.1
OS Version: os x newest
VSCode version: 1.55.2
Describe the bug
I managed to set up the task using gradle.javaDebug option:

I marked breakpoint in the test code:

Yet when running the task in debug mode, whole task is being run, but debugger doesn't really launch.

I enabled debug mode of gradle and here's the
Output from "Gradle Tasks"
[warn] [gradle-server] Adding modules:domain:cleanTest to ensure task output is cleared before debugging
[debug] Get daemon status
[error] Error running build: modules:domain:test: Could not execute build using Gradle distribution 'https://services.gradle.org/distributions/gradle-6.2-bin.zip'.
[error] Unable to start Java debugging: The task completed without the debugger being attached
[error] [gradle-server] Could not execute build using Gradle distribution 'https://services.gradle.org/distributions/gradle-6.2-bin.zip'.
[debug] Get daemon status

However please note: I noticed that the "error"s appear only after the whole test task ends.
If I run the task manually using CLI (./gradlew :modules:domain:test --debug-jvm), then the gradlew is waiting for IDE to connect it:

Not sure however how this extension does it though.
Any ideas how can I fix it / debug this problem more closely?
To Reproduce
Can't provide, this is a private repo ;/
Expected behavior
Debugger should launch and stop on a breakpoint
Screenshots
If applicable, add screenshots to help explain your problem.
Does the bug still exist if you disable all other extensions?
Haven't tried. (From Java-related extensions, I only have the recommended Java ones; nothing else I think that would interfere with this one.)

P.S. I wanted to run that debug task with argument --tests="io.ailo.ledger.payment.service.AutoPayLiabilityServiceTest" but it doesn't work:

Should I create separate issue for that?

Same issue here. Some details:
Extension Name: vscode-gradle
Extension Version: 3.8.4
OS Version: Mac OS X 10.15.7 x86_64 (Catalina)
VSCode version: 1.62.3
Configured gradle version (via gradlew): 7.3
Globally installed gradle version: none
Task to debug: bootRun
It's a spring boot application, not sure if this is relevant. Running it via the play button next to the task works fine, debugging has the same behaviour as described above:
> Executing task: gradle: bootRun <

> Configure project :
Using default profile for pit-service

> Task :generateMainEffectiveLombokConfig1

> Task :bootRun
Listening for transport dt_socket at address: 63469
<===========--> 85% EXECUTING [45s]
> :bootRun

I'm having this same problem, but I used to be able to just start a debug configuration to attach to the printed port (annoying, but possible.). Recently our project was upgrade to jdk 11, and now I don't get the port printed in the console.  I just see this hanging forever
`<========-----> 66% CONFIGURING [12s]

:server`

If I ctrl-c the process, it then prints out that it was waiting on a port
`> Configure project :server
<=============> 100% CONFIGURING [1m 7s]

IDLE
Picked up JAVA_TOOL_OPTIONS: -agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=localhost:56062
Listening for transport dt_socket at address: 56062

FAILURE: Build failed with an exception.

Where:
Build file '/Users/johncrowley/indeed/moody-service/server/build.gradle' line: 2

What went wrong:
An exception occurred applying plugin request [id: 'com.indeed.common', artifact: 'com.indeed:common-gradle-plugin:1.+']

Failed to apply plugin class 'com.indeed.gradleplugin.common.subplugins.tests.TestJavaNextPlugin'.
Process 'command '/Library/Java/JavaVirtualMachines/jdk11.0.19.jdk/Contents/Home/bin/java'' finished with non-zero exit value 143

Try:

Run with --stacktrace option to get the stack trace.
Run with --info or --debug option to get more log output.
Run with --scan to get full insights.

Get more help at https://help.gradle.org`

Does anyone know how to solve this?  I can't see the port until I ill the process.  Is there a way to set the port or having it print plain text somehow?


Follow file renames

Sort of like git log file vs git log --follow file. There is https://github.com/staff0rd/github-follow-extension, but I couldn't get it to work, and not having yet another extension would be awesome.
Sorry if there is an issue for this already, I was unable to find it.
Example file: https://github.com/facebook/jest/commits/master/packages/jest-cli/src/coverage.template
$ git log --oneline packages/jest-cli/src/coverage.template
0c9d2b6e packages/jest-cli
$ git log --oneline --follow packages/jest-cli/src/coverage.template
0c9d2b6e packages/jest-cli
d612ea12 Move coverage template to non-js file
ce5c2ebc Lint fixes
8efc492a Use eslint for linting
88a94d5d FIRST!

For anyone that wants this, please also send this feature request to GitHub<EMAIL_ADDRESS>

bfred-it earned $80.00 by resolving this issue!

Checkout the Issuehunt explorer to discover more funded issues.
Need some help from other developers? Add your repositories on Issuehunt to raise funds.

v3 solution (v4 doesn't support it)
Base url example: https://github.com/sindresorhus/refined-github/commits/master/source/features/copy-on-y.tsx

async function findRename(user, repo, currentCommitSha, currentFilename) {
	const response = await fetch(`https://api.github.com/repos/${user}/${repo}/commits/${currentCommitSha}`);
	const content = await response.json();

	for (const file of content.files) {
		if (file.filename === currentFilename) {
			return `https://github.com/${user}/${repo}/commits/${content.parents[0].sha}/${file.previous_filename}`
		}
	}
}

await findRename('sindresorhus', 'refined-github', '11b927189bab258a226e251c1bd87504feee51f4', 'source/features/copy-on-y.tsx')
// https://github.com/sindresorhus/refined-github/commits/f25483fa128d9dcc8e1e74fcf2d2751d604fe11f/source/features/copy-on-y.js

Someone package it up! 😄

@issuehunt has funded $80.00 to this issue.

Submit pull request via IssueHunt to receive this reward.
Want to contribute? Chip in to this issue via IssueHunt.
Checkout the IssueHunt Issue Explorer to see more funded issues.
Need help from developers? Add your repository on IssueHunt to raise funds.

@sindresorhus has rewarded $72.00 to @bfred-it. See it on IssueHunt

:moneybag: Total deposit: $80.00
:tada: Repository reward(0%): $0.00
:wrench: Service fee(10%): $8.00

This feature doesn't seem to work on GHE.

@simonkotwicz I haven't tried it yet, but is it throwing an error of any sorts?

ya I do see
❌ branch-buttons
content.js:1689 TypeError: Cannot read property 'textContent' of null
    at getTagLink (content.js:3549)
    at async Promise.all (index 1)
    at async branch_buttons_init (content.js:3593)
    at async features_run (content.js:1684)

That error is not this feature though. 😀 You can open a different issue for that error, please include GHE version, etc.

Ya that's what I figured, I don't see any other errors.

made a PR to fix: https://github.com/sindresorhus/refined-github/pull/1913

Any feedback on how to use this? Do I need to compile something into the extension?

Any feedback on how to use this? Do I need to compile something into the extension?

No this should be working. Do you have an example where this is not working?

No this should be working. Do you have an example where this is not working?

I've got the extension installed, but when I'm looking at the file through GitHub, I can't seem to find the old history (I assume, it would be found by clicking on the History link for the file?)

I've got the extension installed, but when I'm looking at the file through GitHub, I can't seem to find the old history (I assume, it would be found by clicking on the History link for the file?)

Yes it would be under history. If a file is rename the "next" button will lead you to the renamed file

Yes it would be under history. If a file is rename the "next" button will lead you to the renamed file

Strange. The button isn't showing up for me. Do you have a screenshot you could show me that shows where it would be?

Strange. The button isn't showing up for me. Do you have a screenshot you could show me that shows where it would be?

What page are you looking at?

What page are you looking at?

A page like the following:
https://github.com/onassar/repo-name/commits/branch-name/path/to/file.css

A page like the following:
https://github.com/onassar/repo-name/commits/branch-name/path/to/file.css

Do you have an actual link I can test on?

Do you have an actual link I can test on?

It's in a private repo unfort. Do you have a public link that's working for you? I can try it out and confirm

It's in a private repo unfort. Do you have a public link that's working for you? I can try it out and confirm

(make sure your have a personal GitHub token set in the extensions options)

(make sure your have a personal GitHub token set in the extensions options)

Ahh okay that's required? I tried to read through things and assumed the token wasn't required for that to work.

Ahh okay that's required? I tried to read through things and assumed the token wasn't required for that to work.

For any api interaction on a private repo you will need a token

For any api interaction on a private repo you will need a token

Okay thanks. Which specific permissions are required for that feature?

Okay thanks. Which specific permissions are required for that feature?

The repo

The repo

Sorry to be pedantic, but which settings in repo? I don't want to grand the extension more settings than it needs. Does it need all of them?

Sorry to be pedantic, but which settings in repo? I don't want to grand the extension more settings than it needs. Does it need all of them?

If you want to use it in a private repo, repo is the only scope you need. Unfortunately repo includes everything else under it.
For public repositories you probably don't need any scopes since the token already allows read-only access to all public repositories.

Okay gr8 thanks.


Done


fix(shareReplay): handle possible memory leaks

Description:
Fixes possible memory leaks caused by the shareReplay operator
Related issue (if exists):
#5931

I tested this locally, and the changes in this PR don't fix the memory leak reported on #5931, so I'm closing it for now. I wil re-open it if I find a solution.

Actually, sorry, it does fix the memory leak

I believe this PR should be merged as there is no reason to keep those subscription objects around any longer than necessary, but I think it doesn't get to the root of the problem here.
That is, we have to ask why an instance of Subscriber (which is what Observable.prototype.subscribe(observer) returns) is maintaining any reference at all to its Observer after unsubscription (including complete & error).
With that in mind, I have made this related PR, so that this kind of thing will (hopefully) never be a problem again: #5934

I'm going to close this PR because the changes don't really do anything (as I outlined above). If there's proof that this changes the memory usage, I'll be happy to take another look.

I'm sorry if I'm being annoying, but @benlesh or @cartant could you please have a look at this codesandbox :pray:. It's simply doing a of(1).pipe(shareReplay()).subscribe() while using the current version of shareReplay on v6 with 2 console.logs that illustrate why the if (isComplete) statement that I suggested in this PR is necessary in order to avoid the memory leak.
Also, thanks a lot for the amazing work that you are doing on v7. I've been using it on a project of mine and so far things are looking great! There is just one little thing that's annoying me: which is that I'm seeing code in a chunk that I really think that it should have been "tree shaken" and it wasn't. I'm still investigating, I will open an issue or a discussion in the coming days to explain this better and I will make sure that when I open the issue I can give you as much info as I can. Thanks again!

@josepot I'll have a look at this in a bit. I'll push a test to this branch to clear things up.

Derp. @josepot is correct. I overlooked that synchronously, subscription is set after it's set to undefined when the subscribe call returns. Reopening this.

Add a comment, as suggested, above the new conditional.

:heavy_check_mark:

Add a test that proves this resolves the issue.

Is that possible without testing the internals? :thinking: I mean, the only way that I can think off, involves exposing an implementation detail... which IMO wouldn't be worth it. I'm more than open to suggestions.

Is that possible without testing the internals? 🤔

Yes. I'll add one soon.

@josepot @benlesh I've added a leak test for shareReplay to effect a failure, you will have to comment out the nulling of innerSub and subscription (although, the latter is, IMO, a smaller leak that is not related to the problem demonstrated in the Angular StackBlitz repro). Note that FinalizationRegistry is only in Node 14 and later. I'll add a similar test to master soon.

IDK what these build failures are.  Thanks a lot!
PS @voliva check this out we should start using this on @react-rxjs/core ASAP.

@josepot IDK whether or not you can see the details of the CI failure, but this is what's breaking:
Error: node_modules/@types/node/index.d.ts(74,11): error TS2300: Duplicate identifier 'IteratorResult'.
Error: node_modules/typescript/lib/lib.es2015.iterable.d.ts(41,6): error TS2300: Duplicate identifier 'IteratorResult'.
npm ERR! code 2
npm ERR! path /home/runner/work/rxjs/rxjs
npm ERR! command failed
npm ERR! command sh -c tsc -p ./tsconfig/compat/tsconfig.cjs.json

🤷♂ Maybe there is was a breaking change in the node types that match whatever semver is in this branch's package.json. IDK.

@josepot IDK whether or not you can see the details of the CI failure, but this is what's breaking:
Error: node_modules/@types/node/index.d.ts(74,11): error TS2300: Duplicate identifier 'IteratorResult'.
Error: node_modules/typescript/lib/lib.es2015.iterable.d.ts(41,6): error TS2300: Duplicate identifier 'IteratorResult'.
npm ERR! code 2
npm ERR! path /home/runner/work/rxjs/rxjs
npm ERR! command failed
npm ERR! command sh -c tsc -p ./tsconfig/compat/tsconfig.cjs.json

Maybe there is was a breaking change in the node types that match whatever semver is in this branch's package.json. IDK.

Thanks a lot @cartant ! Yep, I can see that. There were 2 different checks failing, one has been fixed by my latest commit (a linting issue, aparently). And this one, I'm investigating... The first thing that I tried was rebasing the branch from 6.x, just in case, but that didn't help... And yeah, based on a quick google search that error normally indicates an issue with the node types. I will investigate a bit more later. Thanks again.

man_shrugging Maybe there is was a breaking change in the node types that match whatever semver is in this branch's package.json. IDK.

Yep, I think that's the issue... Because -according to the logs- the task that fails is running under<EMAIL_ADDRESS>/usr/bin/tar xz --warning=no-unknown-keyword -C /home/runner/work/_temp/7bd7433d-ba81-4237-ac22-1ea42ffa579b -f /home/runner/work/_temp/1e3c4cf6-32b1-405a-90d3-3bb7230d395f
/opt/hostedtoolcache/node/13.14.0/x64/bin/node --version
v13.14.0

And the types that are defined on the package-lock.json file are 9.4.5.
Question: would it make sense to add the skipLibCheck for this task  tsc -p ./tsconfig/compat/tsconfig.cjs.json?

Great work, @josepot and @cartant. Thanks for seeing this through!


Add a toggle to disable "omitempty"

Adding a toggle to not include the "omitempty" tag from generated struct fields.
This could be useful if you're migrating an already existing codebase which doesn't use "omitempty" to go-jsonschema and want to minimise the number of changes.
The codebase which I'm working on uses mapstructure and almost never uses omitempty. Instead, it has decoder hooks to manipulate the decoding in various ways. To be honest, I think having the omitempty might not change anything about the decoding process in my particular case. But I think having such a toggle could help make sure there are no unintended consequences from having an omitempty.
I don't think most people would want to have a DisableOmitempty = true though. If you use json or yaml decoding, the validation functions which go-jsonschema generates wouldn't work correctly if omitempty is missing, since the validator wouldn't know that a non-required property is missing and it wouldn't be able to set it to its default value. My change is probably only useful for certain users of mapstructure.
An alternative way to do this change could be to add it to goJSONSchema setting so that we can override it per property, instead of setting it globally. Maybe we should have both types of toggles -  a global, and a property-specific one?

@ptodev I think this is ok, can you please fix the linting and deal with the todo, I will merge it then. also take this out of draft once it's ready. thank you!


only run updater on darwin and windows

This should fix the spamming error message about updater on Linux.

can i get come core sanity checks of this
cc: @patrickxb @mmaxim @zanderz @keybase/react-hackers

@cjb

I can still repro with chat api-listen... @zapu, can you?

LGTM - couldn't add my self as official GH reviewer

(I didn't try it like @heronhaye but change looks fine)

Talked to @heronhaye on chat. Merging!


Use pure ES modules

See background at https://github.com/netlify/team-dev/issues/36
Once https://github.com/netlify/framework-info/issues/450 is done and released, we should use pure ES modules and make a major release.
This is more than just switching from CommonJS to import/export. See this list for other changes which might be involved. This should be broken in many PRs, as much as possible, to lower the risk. Also, non-breaking changes (such as adding file extensions in imports, or loading JSON files differently) should be done before the breaking changes (such as using import/export statements).
At the moment, framework-info is only used by:

build-info: so support for pure ES modules should be added first there: https://github.com/netlify/build-info/issues/181
netlify-cli: same thing for Netlify CLI: https://github.com/netlify/cli/issues/3514
netlify-react-ui: will this be a problem when importing this module from netlify-react-ui? Do we bundle code to make ES modules work in netlify-react-ui? Or maybe we now only support browsers with ES modules support? @nasivuela Do you know what the situation is?

The following items are left, in order:

[ ] 6.1.0 changed the way JSON files were imported. build-info and Netlify CLI have successfully upgraded to 6.1.0, but we should also upgrade it in netlify-react-ui.
[ ] https://github.com/netlify/framework-info/pull/577 is a WIP to add pure ES modules. I have released a release candidate on npm which I've been trying to use in netlify-react-ui (https://github.com/netlify/netlify-react-ui/pull/9862). I've discovered 8 different issues, some rather complex, related to doing so. I have filed https://github.com/netlify/netlify-react-ui/issues/9869 to organize those issues, which are currently blocking that PR.
[ ] Remove some of the tests of this repository which are now irrelevant (https://github.com/netlify/framework-info/pull/577/files#r763497854)
[ ] Try to remove the Webpack bundle altogether. I believe this should work, but we need to make a release candidate and try it with netlify-react-ui. It would be good to do it as a follow-up step, separate from the ones above. Note this is only the Webpack bundle published to npm (npm run build:browser:core and webpack.config.core.js), not the Webpack bundle used for the "Vanilla test site" (npm run build:browser:site-react and webpack.config.site.js, which should be removed by https://github.com/netlify/framework-info/pull/577/files#r763497854)

Done!


Does not work on Chrome 76

When trying to install, this banner appears and installation fails: CRX_HEADER_INVALID

nice issue number


Extension deactivate does not complete before window is closed

See https://github.com/microsoft/vscode/issues/161147 - at the end of comments I shared a small extension demonstrating the problem, but I may have replied too late. Let me know if I should copy over the text of the original issue.

@helin24 I think this might be caused by using a fs write stream. I believe this is not getting flushed to disk by nodejs. Here is a sample doing the same as your sample which uses fs.appendFileSync to ensure that we see the log messages in the file. This works fine for me:

package.json

{
    "name": "167481",
    "publisher": "alex",
    "version": "1.0.0",
    "engines": {"vscode": "^1.22.0"},
    "main": "main.js",
    "activationEvents": ["*"]
}

main.js:

//@ts-check
const vscode = require('vscode');
const child_process = require('child_process');
const fs = require('fs');
const path = require('path');
const logLocation = path.join(__dirname, 'log.txt');

/** @type {child_process.ChildProcess} */
let daemonProcess;

exports.activate = function(context) {
	maybeLogLine(`started`);
	daemonProcess = child_process.fork(path.join(__dirname, 'daemon.js'), { detached: true });
	maybeLogLine(`daemon started: ${daemonProcess.pid}`);
}

exports.deactivate = function() {
	maybeLogLine(`Entering deactivate`);
	maybeLogLine(`Killing daemon process ${daemonProcess.pid}`);
	daemonProcess.kill();
	maybeLogLine(`Leaving deactivate`);
}

function maybeLogLine(content) {
	const time = new Date().toLocaleString();
	fs.appendFileSync(logLocation, `[extension][${process.pid}]${time}: ${content}\n`);
}

daemon.js:

//@ts-check
const fs = require('fs');
const path = require('path');
const logLocation = path.join(__dirname, 'log.txt');

maybeLogLine(`started`);
setInterval(() => maybeLogLine(`running...`), 1000);

function maybeLogLine(content) {
	const time = new Date().toLocaleString();
	fs.appendFileSync(logLocation, `[daemon][${process.pid}]${time}: ${content}\n`);
}

I place these files in a folder and I use the Developer: Install Extension from Location command. I open a window and then close it. Here is the log that I observe. The daemon process is killed:
[extension][44505]12/6/2022, 6:16:13 PM: started
[extension][44505]12/6/2022, 6:16:13 PM: daemon started: 44507
[daemon][44507]12/6/2022, 6:16:13 PM: started
[daemon][44507]12/6/2022, 6:16:14 PM: running...
[daemon][44507]12/6/2022, 6:16:15 PM: running...
[daemon][44507]12/6/2022, 6:16:16 PM: running...
[extension][44505]12/6/2022, 6:16:16 PM: Entering deactivate
[extension][44505]12/6/2022, 6:16:16 PM: Killing daemon process 44507
[extension][44505]12/6/2022, 6:16:16 PM: Leaving deactivate

Thanks for investigating! This works for me too.


Update mongodb.md

In older version docker build was failing.

Proposed changes

Unreleased project version (optional)

Related issues (optional)

Thanks, @KonradKlimczak !


chore: release 1.2.103

:robot: I have created a release *beep* *boop*
1.2.103 (2022-05-18)
Bug Fixes

git: fix error "unable to clone repo: reference delta not found" (1733ccd)
helm: prevent bug with pre-upgrade helm hooks, which was used from the previous release revision (18570d3)
post-renderer: non-strict labels and annotations validation in werf's post-renderer (18dd510)

This PR was generated with Release Please. See documentation.

:robot: Release is at https://github.com/werf/werf/releases/tag/v1.2.103 :sunflower:


feat: Add support for command_rtp

I think it would be usefull to also be able to use command_rtp. Although a proper reference is missing, functions can be looked up from cryosparc_command/command_rtp/init.py. Maybe a test_connection could be added to command_rtp in next cryosparc release.

@KiSchnelle thanks for making the change, it looks good! Could you run black . and push to fix the lint errors?


Revert "MetricsCacheMgr prototype"

Reverts twitter/heron#1720
This is failing all the internal integration tests.
@huijunw usually before merging PR of such complexity, we will run internal integration test first.
To make sure internal integration test running, I will revert your PR now and you can make a new PR once you fix the problem.

cc @ashvina and @avflor who might care

@objmagic #1720 passed all the CI and integration tests. Anyway, I will check it again

@huijunwu do you also run our internal CI?


Pipeline live debugging enabled as before (tmate)

With the old workflows it was very easy to debug worse issues with one's application (only appearing on the pipelines ;-) ) using Tmate.
However it's less convenient to do with the latest versions of the workflows.
Would it be possible to re-introduce this, truly important feature?

@carlcsaposs-canonical  do not expect Azure runners the nearest time. The topic is outside of our control.
AFAIK, SSH access is possible nowadays for self-hosted runners. Can we use this approach?
What about to add optional upterm? It can be similar to the optional allure: true.
What do you think?

Enabled now internally for Canonical users. See [MM discussion] (https://chat.canonical.com/canonical/pl/q6eay419itbu8n6xga7z78hqko)


Cython Question: How to efficiently create arrays of variable sizes in cython for loop.

Say one has an array of variable sizes that stores values from a nested for-loop. How can one perform such an operation in Cython efficiently? All the Cython sources I seem to come across do not create arrays of varying dimensions. My inefficient approach that uses more NumPy is placed below using inline Jupyter notebook. How can I optimize this further using Cython?
import cython
# Load cython extension
%load_ext Cython

%%cython
import numpy as np

def my_function(list my_list, int n, int a ):
    cdef list Estimate_1_list = []
    cdef list Estimate_2_list = []
    cdef int l,i,t,j
    for l in my_list:

        # Resizable matrices (could I use memory view?)
        a_mat = np.zeros((l,n+1),float)
        b_mat = np.zeros((l,n+1),float)

        for i  in range(n):
            t = i*a

            for j in range(l):

                # Fill matrices
                a_mat[j,i+1] = a_mat[j,i+1] + np.random.random()

                b_mat[j,i+1] = a_mat[j,i+1]/(2*t+3)

        # Append values of interest to use at different values of matrix size
        Estimate_1_list.append(np.mean(a_mat[:,n]))
        Estimate_2_list.append(np.std(a_mat[:,n]))

    # Return results
    results = [Estimate_1_list,Estimate_2_list]
    return results

# Test
my_function([1,2,3], 10, 0.5 )

This looks more like a support request to me than a bug or feature request. Those are best directed to the cython-users mailing list.
When you ask there, please try to be clearer about what you want, e.g. by providing an example of something that you tried but that does not currently work (and provide the error that you get when you try it).

Ok, thanks @scoder . I will post this onto the mailing list.


[8.12 stable] Move HTTP send operations away from the main event loop

Zedagent main event loop is responsible for handling pubsub updates along with sending messages to the controller. HTTP send is synchronous call and can stall the whole main event loop for up to 6 minutes (overall send timeout). This badly affects the system responsiveness which can be observed in air gaped environment (no connectivity to the controller), when the queue is stuck waiting for send to be completed and no other pubsub updates are handled.
Three things are done in this PR:

Introduced connect() ("dial" in terms of Go) timeout, which is less than actual send() timeout (helps to fail faster).
All direct HTTP send() calls are moved away from the main event loop and executed in dedicated goroutines.
(second commit) Also make sure that VMs (such as Local Profile Server or Local Operator Console) are not delayed by NIM testing DPC. In offline mode connectivity probes will take longer (until dial timeout elapsed).

This is a backport of original patches from @milan-zededa to the 8.12 stable branch.
CC: @milan-zededa

@rouming this has DCO failure which need to be taken care of. Then we can merge it.

Difference to the previous version:

DCO fix

Indeed fixed, wow, but the complains was about Ruslan's commit. Thanks Erik for pointing me typo out.


Setup CircleCI

Feature Request

This repo does not have CircleCI setup yet.
Why it is needed
To verify that pull request does not break lint/test/types/etc.

Possible implementation
See the .circleci folder in other community repositories, such as https://github.com/react-native-community/react-native-progress-bar-android/tree/master/.circleci

Code sample

Should be set up now.


Update Idea Version to 2019.3

This will probably fix #3.

I just realized that I didn't updated master before creating this new branch. I will rebase this later.


update VM to premium storage VM

Description
Azure ACS now supports premium storage VMs with kubernetes deployment so production deployment templates need to be updated accordingly
Issues This PR Fixes
Fixes #1971
Todos

[*] Tested and working on development environment

How to QA
Follow production deployment template and create k8s cluster on azure using docs

Taken care of (here)[https://github.com/bigchaindb/bigchaindb/pull/1992]


FutureWarning: creation of DataArrays w/ coords Dataset

Previously, this would raise a:
FutureWarning:
iteration over an xarray.Dataset will change in xarray v0.11 to
only include data variables, not coordinates. Iterate over the
Dataset.variables property instead to preserve existing behavior
in a forwards compatible manner.

[ ] Closes #xxxx (remove if there is no corresponding issue, which should only be the case for minor changes)
[ ] Tests added (for all bug fixes or enhancements)
[ ] Tests passed (for all non-documentation changes)
[ ] Fully documented, including whats-new.rst for all changes and api.rst for new API (remove if this change should not be visible to users, e.g., if it is an internal clean-up, or if this is part of a larger project that will be documented later)

I came across this when serializing/deserializing my coordinates to a json file.

I'm not sure we want to add this special case.
Logically, if a iterating over a Dataset only includes data_vars (as of xarray v0.11), then it would seem unexpected to use variables instead when a Dataset is passed as coords into the DataArray constructor.
If you want to copy coordinates coords or variables, you can do that explicitly, e.g., DataArray(array, coords=dataset.coords).

It seems like this warning isn't benign though. I will take your suggestion though (coords=dataset.coords)
I feel like I'm not the only one who probably did this.
Should you raise an other warning explicitly?

Is there a better way to serialize coordinates only?

Maybe the issue that I am facing is that I want to deal with the storage of my metadata and data seperately.
I used to have my own library that was replicating much of xarray's functionality, but your code is much nicer than anything I would be able to write in a finite time. :smile:
Following the information here: http://xarray.pydata.org/en/stable/data-structures.html#coordinates-methods
Currently, my serialization pipeline is:
import xarray as xr
import numpy as np

# Setup an array with coordinates
n = np.zeros(3)
coords={'x': np.arange(3)}
m = xr.DataArray(n, dims=['x'], coords=coords)

coords_dataset_dict = m.coords.to_dataset().to_dict()
coords_dict = coords_dataset_dict['coords']

# Read/Write dictionary to JSON file

# This works, but I'm essentially creating an emtpy dataset for it
coords_set = xr.Dataset.from_dict(coords_dataset_dict)
coords2 = coords_set.coords  # so many `coords` :D
m2 = xr.DataArray(np.zeros(shape=m.shape), dims=m.dims, coords=coords2)
# I used to just pass the dataset to "coords"
m3 = xr.DataArray(np.zeros(shape=m.shape), dims=m.dims, coords=coords_set)

If you really want to save the data / metadata separately for a DataArray, you might as well roll your own version of to_dict/from_dict. The code for these methods is remarkably simple, all using public APIs:
https://github.com/pydata/xarray/blob/ded0a684136540962bcc409e6272b1cebb5af30a/xarray/core/dataarray.py#L1662-L1674

Thanks!


ADD minor fix and code review

As discussed, let's make sure the blur value is stored when choosing it, instead of making it part of the public API.

For me is ok.

Performance enhanced

It definitely feels like it would be sensible for this to be right, but I'm not so sure anymore.


PropProof is mistaking "invalid input" with "harness crash"

I tried this code:
using the following command line invocation:
 mkdir .cargo; echo "paths =[\"/Users/ytakashl/propproof\"]" > .cargo/config.toml
cargo kani  --tests -p prost-types --default-unwind 3 --harness 'tests::check_duration_roundtrip'

with Kani version: 0.30.0
I expected to see this happen: verification success.

Instead, this happened: verification fails

Can you include the failure in the description?

@zhassan-aws, done, but I will take this.


Improve revision block for long numbers

When it's thousands of revisions, the display is a bit off. We need to wrap it into block somehow.

Hello, @undera can you help me to reproduce the issue?

Hello, @undera can you help me to reproduce the issue?

The easiest way to reproduce would be to patch the frontend or backend to artificially increase revision numbers.
In backend, it can be done in this line: https://github.com/komodorio/helm-dashboard/blob/helm-dashboard-v2/pkg/dashboard/handlers/helmHandlers.go#L701 by adding +1000 to revision value.
In frontend, you can do it by adding +1000 to revision number here: https://github.com/komodorio/helm-dashboard/blob/helm-dashboard-v2/dashboard/src/components/revision/RevisionsList.tsx#L52
Please note that we currently work in v2 branch, which should be merged in main soon.

is the v2 branch private because i am unable to view it?

We've just merged V2 to main, so now all PRs are from main branch again @harshit-mehtaa


Region hardcoded in Ghostscript layer: Comment includes fix/workaround

Branch main (and others with PDF processing)
ps-serverless-app/infrastructure/lib/core/processing.ts:
layers: [
lambda.LayerVersion.fromLayerVersionAttributes(this, 'GhostscriptLayerVersion', {
layerVersionArn: 'arn:aws:lambda:us-east-2:764866452798:layer:ghostscript:8',
compatibleRuntimes: [lambda.Runtime.NODEJS_14_X],
}),
Think this ought to be made to agree with the build/deploy account region.

Using build region eg.
** https://bobbyhadz.com/blog/cdk-get-region-accountid
(OR, provide permission for access to the external region, which is more complex)

This is a fix that I came up with - not an experienced TS dev so pls forgive any non-idiomatic code!
It just substitutes the default region in effect during the CDK build:
`
    const useDefaultRegion = (resource_arn:string): string => {
      const components: ArnComponents = cdk.Arn.split(resource_arn, cdk.ArnFormat.COLON_RESOURCE_NAME);
      const new_components: ArnComponents = {
        ...components,
        region: process.env.CDK_DEFAULT_REGION
      };
      return cdk.Arn.format(new_components);
    }

// and so, where originally there was the hardcoded resource, can now use
const createThumbnail = new NodejsServiceFunction(this, 'ThumbnailLambda', {
  entry: path.join(__dirname, '../../../services/processing/thumbnail.js'),
  timeout: cdk.Duration.seconds(120),
  layers: [
    lambda.LayerVersion.fromLayerVersionAttributes(this, 'GhostscriptLayerVersion', {
      layerVersionArn: useDefaultRegion('arn:aws:lambda:eu-west-1:764866452798:layer:ghostscript:8'),
      compatibleRuntimes: [lambda.Runtime.NODEJS_14_X],
    }),
  ],
});

`

This approach certainly works if you do intend to use a different region.  I planned to have learners just use the regions I was specifying to reduce complexity, but this will work for anyone wanting to do something different.


Add wordmark-less Datadog icon

Name with link
Datadog: https://www.datadoghq.com/
Description
I'm asking for a wordmark-less version, similar to how other logos come with both a full version and an icon only version (e.g. Google Domains).
Link to the vector file, preferably from an official source
https://github.com/gilbarbara/logos/blob/main/logos/datadog.svg

Added in 5d365d6566feda9301b95dffa57825728410ce01


Adds validity controller, refactors validation

Codecov Report

Merging #63 (d63f082) into develop (799613b) will decrease coverage by 2.62%.
The diff coverage is 81.63%.

:exclamation: Current head d63f082 differs from pull request most recent head 0fdc5bd. Consider uploading reports for the commit 0fdc5bd to get more accurate results

@@              Coverage Diff              @@
##             develop      #63      +/-   ##
=============================================
- Coverage      95.58%   92.95%   -2.63%
- Complexity       152      159       +7
=============================================
  Files             28       30       +2
  Lines            362      383      +21
  Branches          34       33       -1
=============================================
+ Hits             346      356      +10
- Misses            10       20      +10
- Partials           6        7       +1

Impacted Files
Coverage Δ

...citihub/configr/exception/BadRequestException.java
0.00% <0.00%> (ø)

...a/com/citihub/configr/schema/SchemaController.java
45.45% <45.45%> (ø)

...itihub/configr/schema/SchemaValidationService.java
96.42% <96.42%> (ø)

...tihub/configr/metadata/SchemaValidationResult.java
100.00% <100.00%> (ø)

...om/citihub/configr/namespace/NamespaceService.java
92.63% <100.00%> (-0.51%)
:arrow_down:

...b/configr/namespace/NamespaceValidationResult.java
0.00% <0.00%> (-100.00%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 799613b...0fdc5bd. Read the comment docs.

Complexity up, coverage down 😒

Complexity up, coverage down 😒

The chart is not totally up to date - the branch has parity - https://app.codecov.io/gh/citihub/configr/branch/feature%2Fmetadata


firefox: place alsaLib in lib search patch

Motivation for this change
libcubeb has dlopened libraries for awhile now. In nixpkgs there was
support for the PulseAudio backend doing this, however the ALSA backend
support was missed and caused issue #79310 (no sound with ALSA). This
gives ALSA users the ability to hear sound once again.

It should be doable: I could create an audio loopback and record that to a WAV file.
Checking if audio is working should be simple as checking if the file is nonempty.

Backported to

20.09: 9bf04bc90bf
20.03: e09c7f0b1f8


Update docs with regards to local source feature

Changes
Add link to SHIP.
Bump Go version in HACK.md.
Submitter Checklist

[ ] Includes tests if functionality changed/was added
[X] Includes docs if changes are user-facing
[X] Set a kind label on this PR
[X] Release notes block has been filled in, or marked NONE

See the contributor guide
for details on coding conventions, github and prow interactions, and the code review process.
Release Notes
NONE

/approve
you added a release note with the original PR, right @HeavyWombat ?

/approve
you added a release note with the original PR, right @HeavyWombat ?

Yes, there were release notes added.


Disable grm target cache for shoots

How to categorize this PR?

/area cost robustness
/kind enhancement
/priority normal
What this PR does / why we need it:
This PR makes use --target-disable-cache added in https://github.com/gardener/gardener-resource-manager/pull/95 for the shoot grm instances.
By this, we can significantly reduce the memory footprint and increase stability of these instances, ref https://github.com/gardener/gardener-resource-manager/issues/92.
Which issue(s) this PR fixes:
Fixes #
Special notes for your reviewer:
✅ depends on grm v0.21.0 release
Release note:

The target cache of `gardener-resource-manager` instances running in the Shoot control plane is disabled now.

Docker images built by `make docker-images` are now tagged and build with the commit hash appended to the version.

gardener-resource-manager now logs its own version on startup or when executed with `--version`.

The cache of the kubernetes client for the target cluster can now be disabled via the `--target-disable-cache` flag.

gardener-resource-manager now uses a `DynamicRESTMapper`, which will reduce the amount of explicit discovery calls and faster reconciliation loops and some cases.

/ready


Change DAPR_GRPC_ENDPOINT to infer TLS based on query parameter

In the current proposal, SDKs are meant to enable TLS on gRPC endpoints based on the scheme being https. This is wrong- gRPC uses the scheme to determine which resolver to use. Not only is wrong w.r.t to the gRPC protocol, is always means users will not be able to use other resolvers (e.g. unix://) in conjunction with TLS.
This PR updates the proposal so that the DAPR_GRPC_ENDPOINT URL now includes an optional ?tls=<bool> parameter. This parameter signals to the SDK whether to enable TLS on this endpoint or not. The SDK will continue to infer whether to use TLS or not based on the endpoint port when this parameter is not present. DAPR_GRPC_ENDPOINT defaults to port 443 if not present.

Isn’t this a breaking change that will break existing apps that are running unless they update the SDK?

@ItalyPaleAle I worked on adding these env variables for Python and JS. The JS PR hasn't been merged yet, it's currently in review. The Python one was merged, but we haven't had a new release since. So, regarding these two SDKs - now is a good time to update them.

In case of "https", there should be no way to connect to an HTTPS endpoint without TLS. If all we need is to enable TLS to other schemes that can operate with or without TLS, then it would not be a breaking change. It would simply allow other schemes to work with TLS when the default is to work without.
My suggestion is to make ?tls=<bool> as optional and ignore if when trying to disable TLS on a protocol that is over SSL by definition (like HTTPS).

In case of "https", there should be no way to connect to an HTTPS endpoint without TLS. If all we need is to enable TLS to other schemes that can operate with or without TLS, then it would not be a breaking change. It would simply allow other schemes to work with TLS when the default is to work without.

https is not a valid hostname resolver which is the reason for its entire removal in this updated proposal. The proposal today is actually saying that the https resolver is using the dns resolver, but also to use TLS in the transport to the peer. This overloads what the name resolver (schema) is used for https://grpc.io/docs/guides/custom-name-resolution/

My suggestion is to make ?tls= as optional and ignore if when trying to disable TLS on a protocol that is over SSL by definition (like HTTPS).

Similarly to above, there is no name resolver which would require TLS in the transport.

+1 binding

+1 on this proposal. gRPC is non trivial and unclear at the moment and adding the https protocol makes it confusing. We should however try to leverage existing parsers in the SDKs where we can

+1 binding

+1 binding

I can vote on the condition we keep the backwards compatibility following the deprecation timeline of 2 releases for unsupported patterns in gRPC.

I can vote on the condition we keep the backwards compatibility following the deprecation timeline of 2 releases for unsupported patterns in gRPC.

+1 from me (for Python SDK) also based on this condition.
As mentioned in another comment, Python SDK is already released. Backwards compatibility should be maintained if possible.

Here's a matrix of different possible endpoints and how they would be parsed based on this proposal. If it looks good, I can add it to the PR so it serves as a reference, and possibly as test cases for implementations.

 I merged to make progress on this.


Props never get updated in WebComponent with elmish

let MyComponent (props : IStore<Props>) =
    let store, dispatch = Store.makeElmish Model.init update ignore props.Value
    use propsSubscription = props.Subscribe (PropsChanged >> dispatch)

In above code, PropsChanged message will always get delivered with initial value of props.
Events get fired, but value of props is always the same.
Normal IStore<Props> |> Store.map rendering code appears to work fine (below): Html text is changing correctly.
Just elmish (above) never sees any change.
    Bind.el(props |> Store.map (fun m -> m.text), Html.text)

Eh it was
use propsSubscription = ...

instead of
let propsSubscription = ...

so subscription got disposed at the end of code block.

I didn't even know we had use in F# - seems to work same as using (...) { ... } in C#.
I'm glad you fixed it, and thanks for logging the issue


Sharing Unit-Conv1d Implementation

Hello, scaling a Conv1D is a piece of cake for the authors, but I was working on it and thought I'd share my implementation. It seems to be properly scaled too, I will be using it for my experiments.
@inherit_docstring(
    short_description=(
        "Applies a **unit-scaled** conv1d transformation to the incoming data."
        "\nNote that this layer sets :code:`bias=False` by default."
    ),
    add_args=[binary_constraint_docstring],
)
class Conv1d(nn.Conv1d):
    def __init__(
        self,
        in_channels,
        out_channels,
        kernel_size,
        stride,
        padding,
        dilation,
        groups,
        bias=False,
        padding_mode='zeros',
        device=None,
        dtype=None,
        constraint = "to_output_scale",
        weight_mup_type = "weight",
        scale_power= (0.5, 0.5, 0.5)
    ) -> None:
        super().__init__(in_channels, out_channels, kernel_size, stride, padding, dilation, groups, bias, padding_mode, device, dtype)
        self.constraint = constraint
        self.weight = Parameter(self.weight.data, mup_type=weight_mup_type)
        if self.bias is not None:
            self.bias = Parameter(self.bias, mup_type="bias")
        self.scale_power = scale_power
        assert len(scale_power)==3

    def reset_parameters(self) -> None:
        nn.init.normal_(self.weight)
        if self.bias is not None:
            self.bias.data.zero_()

    def forward(self, input):
        fan_out, fan_in, kernel_size = self.weight.shape
        batch_size = input.numel()//fan_in

        output_scale = 1/(fan_in**self.scale_power[0] * kernel_size **self.scale_power[0])
        grad_input_scale = 1/fan_out ** self.scale_power[1]
        grad_weight_scale = grad_bias_scale = 1/batch_size ** self.scale_power[2]

        output_scale, grad_input_scale = apply_constraint(
            self.constraint, output_scale, grad_input_scale
        )

        input = scale_bwd(input, grad_input_scale)
        weight = scale_bwd(self.weight, grad_weight_scale)
        bias = scale_bwd(self.bias, grad_bias_scale) if self.bias is not None else None

        if self.padding_mode != 'zeros':
            out = nn.functional.conv1d(
                nn.functional.pad(input, self._reversed_padding_repeated_twice, mode=self.padding_mode),
                            weight, bias, self.stride,
                            _single(0), self.dilation, self.groups)

        else:
            out = nn.functional.conv1d(
                input, weight, bias, self.stride,
                self.padding, self.dilation, self.groups
            )
        return scale_fwd(out, output_scale)

Will likely be sharing a unit-mamba in the future. I believe these two unit scaling papers deserve more attention.

Thanks so much for this - I've put up an implementation of conv1d to merge into our library. Similar to yours, plus handing a few more of the optional args: https://github.com/graphcore-research/unit-scaling/pull/67. Feedback very welcome
Would be keen to help where I can with any mamba stuff too!

I've tested your implementation and it seems to be better scaled than mine for different scenarios. Thanks for your help. For mamba, would you rather have a single issue where I share different components or an issue for every components? (ex: Softplus)

Single issues are definitely easier to review (like the conv1d), but only if it makes sense for you to decompose parts. In some cases ops make sense to scale as one big chunk (e.g. scaled dot product attn), so there's probably a balance to be found


Does pronto need to have access to the entire source?

Hi
Is it possible to run Pronto without checking out the entire source locally?
I mean, when commenting on a PR diff it does not seem necessary to be in the actual repo when running Pronto.
Something like this:
GITHUB_URL=https://github.com/mmozuras/pronto GITHUB_PR_ID=44 pronto run -f github_pr

This could just pull the diff from:
https://github.com/mmozuras/pronto/pull/44.diff
and analyze that?

Thought about it today and decided not to do it.
Requires changing approach too much, makes more sense as a completely separate project 😄.