AsyncBareProgressWorker: allow reuse of  uv_async_t and AsyncWorker components

AsyncBareProgressWorker: move uv_async_t and AsyncWorker components to non-template class AsyncBareProgressWorkerBase
Now that the larger change is merged, I'd like to break AsyncBareProgressWorker down into smaller classes so that the base can be reused in yet another worker.  Unfortunately, since this new worker uses a variadic template, it requires c++11.  Without an alternative for older compilers, I don't think this additional new worker actually belongs in NAN until after we drop support for older node versions.
**update:  I was just reading over #676 and it looks like kkoopa and bnoordhuis already agreed that it's OK to use c++11 for new code.  So, after receiving some feedback, I can add the class below to this pull request if desired.
Meanwhile, if we merge this pull request, it opens the door to do awesome things like this:
#if NAN_HAS_CPLUSPLUS_11

template<class T, typename... Targs>
/* abstract */ class AsyncBareFactoryWorker : public AsyncBareProgressWorkerBase {
 public:
  explicit AsyncBareFactoryWorker(Callback *callback_)
      : AsyncBareProgressWorkerBase(callback_) {
  }

  virtual ~AsyncBareFactoryWorker() {
  }

  class ExecutionProgress {
    friend class AsyncBareFactoryWorker;
   public:
    void Construct(Targs... Fargs) const {
        that_->ConstructProgress_(Fargs...);
    }

   private:
    explicit ExecutionProgress(AsyncBareFactoryWorker *that) : that_(that) {}
    NAN_DISALLOW_ASSIGN_COPY_MOVE(ExecutionProgress)
    AsyncBareFactoryWorker* const that_;
  };

  virtual void Execute(const ExecutionProgress& progress) = 0;
  virtual void HandleProgressCallback(const T *data, size_t size) = 0;

 private:
  void Execute() /*final override*/ {
      ExecutionProgress progress(this);
      Execute(progress);
  }

  virtual void ConstructProgress_(Targs... Fargs) = 0;
};

template<class T, typename... Targs>
/* abstract */
class AsyncFactoryWorker : public AsyncBareFactoryWorker<T, Targs...> {
 public:
  explicit AsyncFactoryWorker(Callback *callback_)
      : AsyncBareFactoryWorker<T, Targs...>(callback_) {
    uv_mutex_init(&async_lock);
  }

  virtual ~AsyncFactoryWorker() {
    uv_mutex_lock(&async_lock);

    while (!asyncdata_.empty()) {
      T *data = asyncdata_.front();

      asyncdata_.pop();

      delete data;
    }

    uv_mutex_unlock(&async_lock);
    uv_mutex_destroy(&async_lock);
  }

  void WorkComplete() {
    WorkProgress();
    AsyncWorker::WorkComplete();
  }

  void WorkProgress() {
    uv_mutex_lock(&async_lock);

    while (!asyncdata_.empty()) {
      T *data = asyncdata_.front();
      asyncdata_.pop();
      uv_mutex_unlock(&async_lock);

      // Don't send progress events after we've already completed.
      if (this->callback) {
          this->HandleProgressCallback(data, 1);
      }

      delete data;

      uv_mutex_lock(&async_lock);
    }

    uv_mutex_unlock(&async_lock);
  }

 private:
  void ConstructProgress_(Targs... Fargs) {
    T *data = new T(Fargs...);

    uv_mutex_lock(&async_lock);
    asyncdata_.push(data);
    uv_mutex_unlock(&async_lock);

    uv_async_send(&this->async);
  }

  uv_mutex_t async_lock;
  std::queue<T*> asyncdata_;
};

#endif  // NAN_HAS_CPLUSPLUS_11

This works just like the new AsyncProgressQueueWorker, but instead of std::copying the data, the progress.Construct function passes along the object constructor arguments, constructing and delivering the object to the main thread.
It allows us to Async Construct objects even if they lack a public copy constructor, although it also does support copy constructors.
Sure, the constructor arguments end up getting copied, but not the entire object, which only gets constructed once.  Depending on the code being built around this worker, this could make things much more efficient.
Since AsyncFactoryWorker uses a variadic template, one can use it in many different ways.  Pass no constructor arguments to use the default constructor.  One can even use &&T as the constructor arguments and the worker will use the move constructor!!
Meanwhile, I can go on and on about describing this new class, but all this pull request does is factor out the AsyncBareProgressWorker so that the uv_async_t and AsyncWorker components can be reused.
How does this PR alone benefit users today?  If an addon module defines multiple AsyncProgress[Queue]Workers based on AsyncProgress[Queue]Worker<T> where T is actually used for more than one type, then the compiler will have to generate entire NAN base classes for each worker.  However, with this patch applied, the AsyncBareProgressWorkerBase will be a non-templated common base class to all AsyncProgress[Queue]Workers, and will only be defined once (per code segment).  The generated template classes will be smaller.
Also, if there is any interest in a pull request that adds the above AsyncFactoryWorker classes (only when c++11 support is present), please let me know.  I'll likely publish a separate npm package for it otherwise.

This PR in itself looks good. I am not sure as to what to do about C++11, but it would need a major version bump.

@kkoopa so if you're willing to merge this one as-is, then i'll create another pull request later on with the c++11 class in the OP, and I'll expect that one might hang open for a while...

Alright, that sounds good.


Input/output error

Required Info

Camera Model
D455

Firmware Version
<IP_ADDRESS>

Operating System & Version
Ubuntu 20.04

Kernel Version (Linux Only)
5.4.0-59-generic

Platform
x86_64

SDK Version
latest from release

Language
python

Issue Description
This happens randomly and typically after 30-100 minutes of running pyglet_pointcloud_viewer.py.
The pyglet viewer will freeze, and restarting it again causes this issue. The only way I can see to get out of it it is to unplug and replug the USB cable.
$ ./pyglet_pointcloud_viewer.py
/usr/lib/python3/dist-packages/apport/report.py:13: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import fnmatch, glob, traceback, errno, sys, atexit, locale, imp, stat
Traceback (most recent call last):
  File "/usr/bin/xonsh", line 4, in <module>
    main()
  File "/usr/lib/python3/dist-packages/xonsh/__amalgam__.py", line 24019, in main
    _failback_to_other_shells(args, err)
  File "/usr/lib/python3/dist-packages/xonsh/__amalgam__.py", line 23983, in _failback_to_other_shells
    raise err
  File "/usr/lib/python3/dist-packages/xonsh/__amalgam__.py", line 24017, in main
    return main_xonsh(args)
  File "/usr/lib/python3/dist-packages/xonsh/__amalgam__.py", line 24060, in main_xonsh
    run_script_with_cache(
  File "/usr/lib/python3/dist-packages/xonsh/__amalgam__.py", line 3043, in run_script_with_cache
    run_compiled_code(ccode, glb, loc, mode)
  File "/usr/lib/python3/dist-packages/xonsh/__amalgam__.py", line 2948, in run_compiled_code
    func(code, glb, loc)
  File "./pyglet_pointcloud_viewer.py", line 103, in <module>
    pipeline.start(config)
RuntimeError: xioctl(VIDIOC_S_FMT) failed Last Error: Input/output error

Hi @dheera   As a starting point in investigating your case, could you monitor memory usage in Ubuntu whilst pyglet_pointcloud_viewer.py is running to see whether the amount of available memory is reducing over time (known as a memory leak, which could lead to the program freezing).
The speed at which memory is consumed (and therefore the time before failure of the program occurs) may vary depending on factors such as the camera resolution and FPS speed being used.  Lower resolution / FPS may mean a longer period of running time before program failure occurs.
Ubuntu's htop interface could be used to perform the memory utilization monitoring.
https://vitux.com/how-to-use-htop-to-monitor-system-processes-in-ubuntu-20-04/

Hi @dheera  Do you require further assistance with this case, please?  Thanks!

I gave up on it and switched to C++. The Python still seems buggy.

Thanks very much for the update.  Do you require any assistance with the C++ version of your project, please?

Case closed due to no further comments received.


Add sessionTokenSecretRef to Route53 provider

Is your feature request related to a problem? Please describe.
The Route53 provider allows direct credentials through the fields accessKeyIDSecretRef and secretAccessKeySecretRef. This is not enough if the credentials came from a previous assume role action. In that case you also need to use the session token.
Describe the solution you'd like
Add the optional field sessionTokenSecretRef that would refer to a session token key in a secret.
Describe alternatives you've considered
I don't see any.
/kind feature

This is not needed. The alternative is to use ambiant mode for the credentials in combination with the AWS Assume Role Operator.


Pagination does not reset when updating search params

The pagination does not reset to the first page after we select a new language, star range, or sorting (e.g., Best match). Ideally, it should show the first page after the search query changes.
Steps Reproduce

Select 'C' as the language
Set the star range from 0 to 20000 and search
Now go to the next page (ie: 2nd page)
Then update the star range from 10000 to 20000 and search

Observation: Page not found error
Expected: We should show the first page for the 'C' language with the star range from 10000 to 20000 (which only has 1 page)

I'm already working on a fix for this.

Thanks for pointing it out and working on it ✌


Makefile for generating data, cluster scripts, batching create_gene_summary

Adds Makefile, see README for instructions
Updates create_gene_summary.R to fetch gene summaries in batches (significant speedup) and save output as Rdata (instead of feather)
Includes build-slurm.sh and upload-slurm.sh scripts for generating data on HPC cluster and uploading results to data service.

@hirscheylab I've run the Makefile on HARDAC and uploaded the data to DDS. I'll next be loading the data onto the web server and sharing the files with you. The Makefile should work on your machine as well as the cluster, but let me know if you have questions.


when I train the model ? The  d_loss is always 0.0 ? Is there some problems ?

Elapsed [0:05:20.807550] batch: 252  d_loss: 0.000000  g_loss: 9.195433
Elapsed [0:05:32.021947] batch: 261  d_loss: 0.000000  g_loss: 9.544848
Elapsed [0:05:43.313830] batch: 270  d_loss: 0.000000  g_loss: 8.533874
Elapsed [0:05:54.605321] batch: 279  d_loss: 0.000000  g_loss: 8.899170
Elapsed [0:06:06.190011] batch: 288  d_loss: 0.000000  g_loss: 18.478725
Elapsed [0:06:17.612306] batch: 297  d_loss: 0.000000  g_loss: 7.378197

I also have the same issue, have you solved it?

same...

Hello!
The default GAN_loss function being used in this repo is relativistic-hinge. Please refer to this paper for more about this loss function: https://arxiv.org/abs/1807.00734. The discriminator loss in for this variant of the GAN_loss remains 0.0 for most of the training, but the GAN trains properly.

Thank you for your reply!

发件人: Animesh Karnewar @.>
发送时间: 2021年4月12日 13:51
收件人: akanimax/BMSG-GAN @.>
抄送: foxy297feng @.>; Comment @.>
主题: Re: [akanimax/BMSG-GAN] when I train the model ? The d_loss is always 0.0 ? Is there some problems ? (#40)
Hello!
The default GAN_loss function being used in this repo is relativistic-hinge. Please refer to this paper for more about this loss function: https://arxiv.org/abs/1807.00734. The discriminator loss in for this variant of the GAN_loss remains 0.0 for most of the training, but the GAN trains properly.
―
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://github.com/akanimax/BMSG-GAN/issues/40#issuecomment-817503421, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ASLDPN6JMIHWQNX57VKX6RLTIKC57ANCNFSM4R4FM3JA.


Add higher taxa authority information as the default.

I would like the results to included the alignedAuthority for higher taxonomy. It would allow me to add authors to our local datasets with less work. I currently have to do it in two passes where I query the higher taxa on there own to extract the author and year.
The current results are
alignedName: Adoretus
alignedAuthority: Dejean, 1833
alignedSubfamilyName: Rutelinae
alignedTribeName: Adoretini
I would like a fields like
alignedSubfamilyNameAuthority: Smith, 1900
alignedTribeNameAuthority: Jones, 1901
this would mean I only need to run Nomer once and I can get all the information I need.

@whitfarnum thanks for  your suggestion, I much like your idea. Adding authorities for the higher order taxa should take me about 2-4 hours to implement, including preparing a release and testing etc. Any other (java) developer would be able to do this also, perhaps with a little bit of learning curve. Any idea on how to materialize your neat proposed feature?

@jhpoelen I was implementing via a hack of submitting species names to get the current higher taxonomy. I then isolated the higher taxa and just submitted those to Nomer. When I submit the higher taxa I get the authorities. I then stitched them together via dictionaries. everything I do is in Python.

Ok, neat to see that you are being creative and sharing ideas to improve nomer. I can see how adding this nomer feature would save you time.
Do you guys have a budget to support development of open source tools like Nomer? If not, I suggest you look into that, because I can't sustain working pro bono especially when working for fancy institutions like yours. If so, please let me know how you'd like to compensate for my time.

After a few hours, I was able to come up with the following (working) example:
echo -e "\tHomo sapiens"\
 | nomer append --include-header ncbi\
 | mlr --itsvlite --oxtab cat

produced the data below (note the populated resolvedPathAuthorships).
@whitfarnum is this what you had in mind?
providedExternalId
providedName            Homo sapiens
relationName            SAME_AS
resolvedExternalId      NCBI:9606
resolvedName            Homo sapiens
resolvedAuthorship      Linnaeus, 1758
resolvedRank            species
resolvedCommonNames
resolvedPath            root | cellular organisms | Eukaryota | Opisthokonta | Metazoa | Eumetazoa | Bilateria | Deuterostomia | Chordata | Craniata | Vertebrata | Gnathostomata | Teleostomi | Euteleostomi | Sarcopterygii | Dipnotetrapodomorpha | Tetrapoda | Amniota | Mammalia | Theria | Eutheria | Boreoeutheria | Euarchontoglires | Primates | Haplorrhini | Simiiformes | Catarrhini | Hominoidea | Hominidae | Homininae | Homo | Homo sapiens
resolvedPathIds         NCBI:1 | NCBI:131567 | NCBI:2759 | NCBI:33154 | NCBI:33208 | NCBI:6072 | NCBI:33213 | NCBI:33511 | NCBI:7711 | NCBI:89593 | NCBI:7742 | NCBI:7776 | NCBI:117570 | NCBI:117571 | NCBI:8287 | NCBI:1338369 | NCBI:32523 | NCBI:32524 | NCBI:40674 | NCBI:32525 | NCBI:9347 | NCBI:1437010 | NCBI:314146 | NCBI:9443 | NCBI:376913 | NCBI:314293 | NCBI:9526 | NCBI:314295 | NCBI:9604 | NCBI:207598 | NCBI:9605 | NCBI:9606
resolvedPathNames       |  | superkingdom | clade | kingdom | clade | clade | clade | phylum | subphylum | clade | clade | clade | clade | superclass | clade | clade | clade | class | clade | clade | clade | superorder | order | suborder | infraorder | parvorder | superfamily | family | subfamily | genus | species
resolvedPathAuthorships |  |  | Cavalier-Smith 1987 |  |  |  |  |  |  | Cuvier, 1812 |  |  |  |  |  |  |  |  | Parker & Haswell, 1897 |  |  |  | Linnaeus, 1758 |  |  |  |  | Gray, 1825 |  | Linnaeus, 1758 | Linnaeus, 1758
resolvedExternalUrl     https://www.ncbi.nlm.nih.gov/Taxonomy/Browser/wwwtax.cgi?id=9606

@jhpoelen yes that is the information I had in mind.

working pro bono especially when working for fancy institutions like yours. If so, please let me know how you'd like to compensate for my time.

I will bring this subject up with my supervisor.

Just curious - which Nomer "matcher" or taxonomic resource do you typically use?

Here's an example of associated ITIS results
providedExternalId
providedName            Adoretus
relationName            HAS_ACCEPTED_NAME
resolvedExternalId      ITIS:187484
resolvedName            Adoretus
resolvedAuthorship      Dejean, 1833
resolvedRank            genus
resolvedCommonNames
resolvedPath            Animalia | Bilateria | Protostomia | Ecdysozoa | Arthropoda | Hexapoda | Insecta | Pterygota | Neoptera | Holometabola | Coleoptera | Polyphaga | Scarabeiformia | Scarabaeoidea | Scarabaeidae | Rutelinae | Adoretini | Adoretus
resolvedPathIds         ITIS:202423 | ITIS:914154 | ITIS:914155 | ITIS:914158 | ITIS:82696 | ITIS:563886 | ITIS:99208 | ITIS:100500 | ITIS:563890 | ITIS:914213 | ITIS:109216 | ITIS:112747 | ITIS:678302 | ITIS:114486 | ITIS:114493 | ITIS:678509 | ITIS:926256 | ITIS:187484
resolvedPathNames       kingdom | subkingdom | infrakingdom | superphylum | phylum | subphylum | class | subclass | infraclass | superorder | order | suborder | infraorder | superfamily | family | subfamily | tribe | genus
resolvedPathAuthorships ITIS:AUTHORSHIP:0 | ITIS:AUTHORSHIP:0 | ITIS:AUTHORSHIP:0 | ITIS:AUTHORSHIP:0 | ITIS:AUTHORSHIP:0 | ITIS:AUTHORSHIP:0 | ITIS:AUTHORSHIP:0 | ITIS:AUTHORSHIP:0 | ITIS:AUTHORSHIP:0 | ITIS:AUTHORSHIP:0 | Linnaeus, 1758 | Emery, 1886 | Crowson, 1960 | Latreille, 1802 | Latreille, 1802 | MacLeay, 1819 | Burmeister, 1844 | Dejean, 1833
resolvedExternalUrl     http://www.itis.gov/servlet/SingleRpt/SingleRpt?search_topic=TSN&search_value=187484

Currently we use Catalog of Life because they have the scarab database. I am currently curating and inventorying our scarabs. Once we are done with scarabs the plan is Carabidae since COL also has the definitive Carabidae catalog at the moment. We are pretty much prioritizing our work to focus on groups that have good online resources so we can leverage that research. I am considering writing up our uses of Nomer as a curation tool if I ever have time. I will contact you about authorship if it happens. I know this is not your envisioned use case but it has been a huge time saver. It has easily taken months of this process.

It has easily taken months of this process.

That is great to hear that Nomer saved you quite some time. I have to say that Nomer is what it is today because of folks like yourself - not shy to try out a new methods and open to sharing ideas for improvement.
(am re-building Catalogue of Life index with the latest dev version of Nomer as we speak, stay tuned . . . )

@whitfarnum here's the recently built Catalogue of Life results. Please note that the Catalogue of Life version is the one whose origin and content is packaged in the "Nomer Corpus of Taxonomic Resources" [1].
echo -e "\tAdoretus"\
 | nomer append --include-header col\
 | mlr --itsvlite --oxtab cat

yielded -
providedExternalId
providedName            Adoretus
relationName            HAS_ACCEPTED_NAME
resolvedExternalId      COL:PCX
resolvedName            Adoretus
resolvedAuthorship      Dejean, 1833
resolvedRank            genus
resolvedCommonNames
resolvedPath            Biota | Animalia | Arthropoda | Insecta | Coleoptera | Scarabaeoidea | Scarabaeidae | Rutelinae | Adoretini | Adoretina | Adoretus
resolvedPathIds         COL:5T6MX | COL:N | COL:RT | COL:H6 | COL:C2L | COL:SC | COL:6278C | COL:K9Y | COL:KJT | COL:LBJ | COL:PCX
resolvedPathNames       unranked | kingdom | phylum | class | order | superfamily | family | subfamily | tribe | subtribe | genus
resolvedPathAuthorships |  |  |  |  |  | Latreille, 1802 | MacLeay, 1819 | Burmeister, 1844 | Burmeister, 1844 | Dejean, 1833
resolvedExternalUrl     https://www.catalogueoflife.org/data/taxon/PCX

References
[1] Poelen, J. H. (ed . ) . (2024). Nomer Corpus of Taxonomic Resources hash://sha256/d2903d0384a8b8193819b8061c8c4e6fec8cc2f7fe72dc0e91c90c07ba2fe15e hash://md5/70645090fdecba640b50577e2a6f2342 (0.23) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.10810821

@whitfarnum let me know when you got a chance to confirm that the newly release Nomer has the functionality you were hoping for.
Apologies for my banter on things related to funding of Nomer activities. Many folks have been generous in the past, and I am hoping to simply open the door to funding while keeping continuously improving our tools and keeping them openly accessible. Thanks for understanding.

@whitfarnum let me if you have additional notes / questions, otherwise, I'll consider this issue closed.


How can I build APISIX-OpenRestry deb or docker image

i found that the official image of apisix will thorw an error of need to build APISIX-OpenResty to support client restriction when i use client-control plugin
i use this cmd make package type=deb app=apisix-base version=2.8 checkout=v2.8 build a deb .
but after i install this package , i don't find any executeable file on my machine

i event don't know what is the apisix-base mean and which cmd arguments should i pick
please give me some suggestion and help me solve the problem of need to build APISIX-OpenResty to support client restriction
thanks

make package version=<IP_ADDRESS>.2 image_base=ubuntu image_tag=20.04 app=apisix-base type=deb


Tag bootstrap_javascript use settings include_jquery for now

Tag bootstrap_javascript has no way to configure jquery parameter via settings.
Also include_jquery does nothing.
This PR fixes this both issues.

Coverage increased (+0.02%) to 86.468% when pulling 7be2b358ad9a9f50d952566ea8b34562a2a60f80 on METAJIJI:patch-1 into 4ed40085b291c0b75e484ffc1e414b4c8d4cb152 on zostera:master.

Coverage increased (+0.02%) to 86.468% when pulling 7be2b358ad9a9f50d952566ea8b34562a2a60f80 on METAJIJI:patch-1 into 4ed40085b291c0b75e484ffc1e414b4c8d4cb152 on zostera:master.

Coverage increased (+0.02%) to 86.468% when pulling 7be2b358ad9a9f50d952566ea8b34562a2a60f80 on METAJIJI:patch-1 into 4ed40085b291c0b75e484ffc1e414b4c8d4cb152 on zostera:master.

Coverage increased (+0.02%) to 86.468% when pulling 7be2b358ad9a9f50d952566ea8b34562a2a60f80 on METAJIJI:patch-1 into 4ed40085b291c0b75e484ffc1e414b4c8d4cb152 on zostera:master.

I'm pulling this in and I'll change (and document) falsy.


Completion of Assignment 1

What changes are you trying to make? (e.g. Adding or removing code, refactoring existing code, adding reports)
I'm adding code to develop the functionality of the anagram_checker.
What did you learn from the changes you have made?
I have learned the importance of really understanding what you want to do with your code an breaking it down into digestible pieces so that coding is a less daunting task
Was there another approach you were thinking about making? If so, what approach(es) were you thinking of?
When working on Part 2 I initially debated creating another nested if/else statement to satisfy the logic for the is_case_sensitive argument. But realized that if I instead had another if/else statement that is outside of the actual comparison of the string lists, the code would be more concise as the logic for the is_case_sensitive does not need to be included in this comparison.
Were there any challenges? If so, what issue(s) did you face? How did you overcome it?
Choosing the right methods and functions can be difficult sometimes. I initially used the .sort() method and had used a for loop to break the string down into a list of characters. I believe with time I will find the rhythm in making better decision as I am exposed to more functions and method when scripting.
How were these changes tested?
These changes were tested by running the Jupyter Notebooks. Additionally, I had added more tests to ensure that the logic was correct.
A reference to a related issue in your repository (if applicable)
Checklist

[x] I can confirm that my changes are working as intended

You did a great job here!
Please go ahead and close this pull request now. Thanks!


Foggy cityscapes dataset

I'm curious about the experiment setting of the foggy cityscapes dataset.
In foggy cityscapes, there are foggy images by three levels (0.005, 0.01, 0.02 level → 2975 images * 3 levels = 8,925).
Did you use all of this data in your experiments? or did you only use foggy images of a specific level?

Hello,
Thanks for your questions. We used only 1 level of foggy images. And we follow the "official" guideline reported here:
https://github.com/yuhuayc/da-faster-rcnn
In the section Example

Hello,
We released the dataset ad this link:
https://github.com/fpv-iplab/Cityscapes-FoggyCityscapes
We hope you can find it useful


Publish Developer Update #2

https://docs.google.com/document/d/12pq4SvIt8PI8QMmVZbaTW5Oi9L1lPV-WjUBEvVgEo4o/edit?usp=sharing

hope to get this out later today

Are these biweekly?
Looks fine :)

https://medium.com/@msena/catching-the-3box-db-wave-8fa3f0e083e
@christianlundkvist can you review?

@michaelsena Yes reviewing now

@christianlundkvist great ty! Will publish on Thursday


[ISSUE #3491] Wake up the waiting pop-request in time

Make sure set the target branch to develop
What is the purpose of the change
#3491
Brief changelog
Wake up the request when scan pollingMap
Verifying this change
XXXX
Follow this checklist to help us incorporate your contribution quickly and easily. Notice, it would be helpful if you could finish the following 5 checklist(the last one is not necessary)before request the community to review your PR.

[x] Make sure there is a Github issue filed for the change (usually before you start working on it). Trivial changes like typos do not require a Github issue. Your pull request should address just this issue, without pulling in other changes - one PR resolves one issue.
[x] Format the pull request title like [ISSUE #123] Fix UnknownException when host config not exist. Each commit in the pull request should have a meaningful subject line and body.
[x] Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
[x] Write necessary unit-test(over 80% coverage) to verify your logic correction, more mock a little better when cross module dependency exist. If the new feature or significant change is committed, please remember to add integration-test in test module.
[x] Run mvn -B clean apache-rat:check findbugs:findbugs checkstyle:checkstyle to make sure basic checks pass. Run mvn clean install -DskipITs to make sure unit-test pass. Run mvn clean test-compile failsafe:integration-test  to make sure integration-test pass.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

Hi, we are researchers in software engineering, working on a novel GitHub bot for helping understand build failures.
We have run our prototype bot flacocobot on this failing PR and below are the results.
What do you think?
Thanks a lot! --Davide (@dginelli), André (@andre15silva), Matias (@martinezmatias), Benjamin (@danglotb), Martin (@monperrus)

The line (32) of the file org/apache/rocketmq/client/consumer/rebalance/AllocateMessageQueueAveragely has been identified with a suspiciousness value of 56.20%.

Failing tests that cover this line

org.apache.rocketmq.client.consumer.rebalance.AllocateMachineRoomNearByTest#testRun10RandomCase
org.apache.rocketmq.client.impl.consumer.ConsumeMessageOrderlyServiceTest#testConsumeMessageDirectly_WithException
org.apache.rocketmq.client.impl.consumer.ConsumeMessageOrderlyServiceTest#testConsumeMessageDirectly_WithNoException
org.apache.rocketmq.client.consumer.rebalance.AllocateMachineRoomNearByTest#test1
org.apache.rocketmq.client.consumer.rebalance.AllocateMachineRoomNearByTest#test3
org.apache.rocketmq.client.consumer.rebalance.AllocateMachineRoomNearByTest#test2

The line (47) of the file org/apache/rocketmq/client/consumer/rebalance/AllocateMessageQueueConsistentHash has been identified with a suspiciousness value of 45.88%.

Failing tests that cover this line

org.apache.rocketmq.client.consumer.rebalance.AllocateMessageQueueConsitentHashTest#testRun100RandomCase
org.apache.rocketmq.client.consumer.rebalance.AllocateMessageQueueConsitentHashTest#testAllocate1
org.apache.rocketmq.client.consumer.rebalance.AllocateMessageQueueConsitentHashTest#testAllocate2
org.apache.rocketmq.client.consumer.rebalance.AllocateMessageQueueConsitentHashTest#testMessageQueueIllegalArgument

The line (106) of the file org/apache/rocketmq/client/Validators has been identified with a suspiciousness value of 45.88%.

Failing tests that cover this line

org.apache.rocketmq.client.ValidatorsTest#testCheckTopic_HasIllegalCharacters
org.apache.rocketmq.client.ValidatorsTest#testCheckTopic_BlankTopic
org.apache.rocketmq.client.ValidatorsTest#testCheckTopic_Success
org.apache.rocketmq.client.ValidatorsTest#testCheckTopic_TooLongTopic

The line (41) of the file org/apache/rocketmq/client/impl/consumer/ProcessQueue has been identified with a suspiciousness value of 39.74%.

Failing tests that cover this line

org.apache.rocketmq.client.impl.consumer.ProcessQueueTest#testFillProcessQueueInfo
org.apache.rocketmq.client.impl.consumer.ProcessQueueTest#testCachedMessageSize
org.apache.rocketmq.client.impl.consumer.ProcessQueueTest#testCachedMessageCount

The line (24) of the file org/apache/rocketmq/client/producer/selector/SelectMessageQueueByHash has been identified with a suspiciousness value of 22.94%.

Failing tests that cover this line

org.apache.rocketmq.client.producer.selector.SelectMessageQueueByHashTest#testSelect

Coverage decreased (-0.003%) to 52.611% when pulling 76e99c382617bf725ea149910acc22d69fac74ea on cserwen:wake_up into de76e06d6da8e8646d2ac59ea061e8dc2653aae8 on apache:5.0.0-alpha.


Incorrect inline style for BitDropDown component

I think the BitDropDown component styles are not set correctly in mobile mode because they don't take the full width and height of the mobile screen.
It is better to let the display mode be its normal and standard mode.
Or if the decision has been made, then please use Class instead of Inline style so that it can be easily modified.

the current style in long & small screens:

@saleh-ghezelbash
We're investigating your issue.
We'll let you know if it's possible to work on this issue or not.

@saleh-ghezelbash
The issue is accepted and planned. Resolving it will start ASAP.

@saleh-ghezelbash
Resolving this issue has started. It will be announced when this issue is resolved.

@saleh-ghezelbash
This issue has been resolved and it will be available in the next pre-release version. It will be announced when the pre-release becomes ready.

@saleh-ghezelbash
The pre-release has been published. Please give it a try and give feedback.


fix: eslint use astro parser

Description
This PR:

installs and uses standard eslint plugin
installs typescript parser for .astro files. (Cannot get around this because Astro uses typescript by default)
fixes and removes unused-vars eslint rule
move eslint config to .eslintrc.cjs file

Hey there! 👋🏻
I've would love too  some changes to the .eslintrc.cjs file that I think will be really helpful. 🏄♂
I show you and explained below in case you want to add them @kevinzunigacuellar 🍻

Explanations

The env property lets us specify which JavaScript environments we're using in the project. This helps ESLint know what global variables are available and how to handle them. For example, if we're using Node.js, we can add 'node' to this property to enable Node.js globals.

I've also added the parserOptions property, which lets us configure extra options for the code analyzer. For example, if we're using JSX in our project, we can enable JSX support by adding 'jsx: true' to this property.

We can use the plugins property to specify additional plugins that we want to use with ESLint. For example, if we're using React in our project, we can add 'react' to this property to enable React support.

Finally, the settings property lets us provide extra settings to plugins or rules. For example, if we're using the ESLint import plugin, we can add 'import/resolver' to this property to configure how modules in our project are resolved.

Code Example
module.exports = {
// ...
extends: ['standard', 'plugin:astro/recommended'],
rules: {
'no-tabs': 'off',
indent: 'off',
'space-before-function-paren': 'off'
},
env: {
browser: true,
node: true,
es6: true
},
parserOptions: {
ecmaVersion: 2020,
sourceType: 'module',
ecmaFeatures: {
jsx: true
}
},
plugins: ['react'],
settings: {
'import/resolver': {
node: {
extensions: ['.js', '.jsx', '.ts', '.tsx']
}
}
},
// ...
overrides: [
{
files: ['*.astro'],
parser: 'astro-eslint-parser',
parserOptions: {
parser: '@typescript-eslint/parser',
extraFileExtensions: ['.astro']
},
rules: {}
}
]
};

Cheers😊

Whoopsie-daisy! I might've muddled my words a bit. My english is sometimes not the best 😅
No worries though, I was just makin' a suggestion for the future like you say ❤
Yes!  Let's leave this PR just as it is
Thanks a bunch!

Hi @kevinzunigacuellar and @Fasping, those settings for linting and formatting code are great, would an .editorconfig file be useful too?

Hi @kevinzunigacuellar and @Fasping, those settings for linting and formatting code are great, would an .editorconfig file be useful too?

Sure @Felix-Icaza ,
An .editorconfig file can be useful to help developers on our team maintain consistent coding styles, regardless of their personal editor settings. It can help ensure that everyone on the team follows the same conventions for things like indentation, line endings, and character encoding.
An example of what an .editorconfig file might look like for a project that uses the JavaScript Standard style:
# editorconfig.org
root = true

[*]
indent_style = space
indent_size = 2
end_of_line = lf
charset = utf-8
trim_trailing_whitespace = true
insert_final_newline = true

Here is an example of what  .editorconfig file might look like if we want to use different conventions for .astro files:
# editorconfig.org
root = true

[*.js]
indent_style = space
indent_size = 2

[*.astro]
indent_style = space
indent_size = 4

maybe in a future PR we can implement
Cheers🍻


Edit problem on L5.5

The crud is working fine until i go to hit edit. then i get this error

Property [id] does not exist on this collection instance.

https://www.useloom.com/share/012c6306060e4c30837f0779f36b43d9 working fine


tflite_model_maker-0.4.2 - Getting nightly for ever

Click to expand!
Issue Type
Build/Install
Have you reproduced the bug with TF nightly?
Yes
Source
binary
Tensorflow Version
tflite_model_maker-0.4.2
Custom Code
No
OS Platform and Distribution
Linux Ubuntu 22.04.1 LTS and Windows 10 (21H1)
Mobile device
No response
Python version
3.10
Bazel version
No response
GCC/Compiler version
No response
CUDA/cuDNN version
No response
GPU model and memory
No response
Current Behaviour?
In Ubuntu and Windows same situation:
COMMAND:
    pip install tflite-model-maker    (as https://pypi.org/project/tflite-model-maker/)

OUTPUT:
    Download, build and install several dependencies, without problem or error.
    Collecting pycocotools>=2.0.2
    Downloading pycocotools-2.0.6.tar.gz (24 kB)
    Installing build dependencies ... done
    Getting requirements to build wheel ... done
    Preparing metadata (pyproject.toml) ... done
    Collecting tflite-model-maker    (<=== DOWNLOADING OLD VERSION ?)
    Downloading tflite_model_maker-0.2.4-py3-none-any.whl (190 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 190.1/190.1 KB 41.0 MB/s eta 0:00:00
    Downloading tflite_model_maker-0.2.3-py3-none-any.whl (114 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 114.2/114.2 KB 28.1 MB/s eta 0:00:00
    Downloading tflite_model_maker-0.2.2-py3-none-any.whl (103 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 103.3/103.3 KB 36.0 MB/s eta 0:00:00
    Downloading tflite_model_maker-0.2.1-py3-none-any.whl (102 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 102.9/102.9 KB 33.6 MB/s eta 0:00:00
    Downloading tflite_model_maker-0.2.0-py3-none-any.whl (102 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 102.6/102.6 KB 33.3 MB/s eta 0:00:00
    Downloading tflite_model_maker-0.1.2-py3-none-any.whl (104 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 104.1/104.1 KB 36.7 MB/s eta 0:00:00

   Collecting tf-nightly
    Downloading tf_nightly-2.12.0.dev20230103-cp310-cp310-
    manylinux_2_17_x86_64.manylinux2014_x86_64.whl (563.8 MB)
    ...
    Still downloading tf_nightly versions, for ever. I had canceled after 2 months of tf_nightly.

I HAD TRY:
   1) Install Windows e Ubuntu.
   2) Remove all pip package e install minimum necessary
   3) Try on python virtual enviroment

Standalone code to reproduce the issue
pip install tflite-model-maker

Relevant log output
No response

Same situation on Jupyter Notebook.
Requirement already satisfied: termcolor in /home/casa/.local/lib/python3.10/site-packages (from tensorflow-datasets>=2.1.0->tflite-model-maker) (2.2.0)
Collecting etils[enp,epath]>=0.9.0
Using cached etils-0.9.0-py3-none-any.whl (140 kB)
Requirement already satisfied: psutil in /home/casa/.local/lib/python3.10/site-packages (from tensorflow-datasets>=2.1.0->tflite-model-maker) (5.9.4)
Requirement already satisfied: requests>=2.19.0 in /usr/lib/python3/dist-packages (from tensorflow-datasets>=2.1.0->tflite-model-maker) (2.25.1)
Collecting tensorflow-metadata
Using cached tensorflow_metadata-1.12.0-py3-none-any.whl (52 kB)
Collecting tqdm
Using cached tqdm-4.64.1-py2.py3-none-any.whl (78 kB)
Collecting toml
Using cached toml-0.10.2-py2.py3-none-any.whl (16 kB)
Requirement already satisfied: protobuf>=3.12.2 in /usr/lib/python3/dist-packages (from tensorflow-datasets>=2.1.0->tflite-model-maker) (3.12.4)
Collecting dill
Using cached dill-0.3.6-py3-none-any.whl (110 kB)
Collecting opencv-python-headless
Using cached opencv_python_headless-<IP_ADDRESS>-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (49.2 MB)
Collecting pycocotools
Using cached pycocotools-2.0.6.tar.gz (24 kB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Collecting gin-config
Using cached gin_config-0.5.0-py3-none-any.whl (61 kB)
Collecting immutabledict
Using cached immutabledict-2.2.3-py3-none-any.whl (4.0 kB)
Collecting oauth2client
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting google-api-python-client>=1.6.7
Using cached google_api_python_client-2.70.0-py2.py3-none-any.whl (10.7 MB)
Collecting py-cpuinfo>=3.3.0
Using cached py_cpuinfo-9.0.0-py3-none-any.whl (22 kB)
Collecting pandas>=0.22.0
Using cached pandas-1.5.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (12.1 MB)
Collecting tf-slim>=1.1.0
Using cached tf_slim-1.1.0-py2.py3-none-any.whl (352 kB)
Collecting kaggle>=1.3.9
Using cached kaggle-1.5.12.tar.gz (58 kB)
Preparing metadata (setup.py) ... done
Collecting tensorflow-text-nightly
Using cached tensorflow_text_nightly-2.12.0.dev20230103-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.9 MB)
Collecting sacrebleu
Using cached sacrebleu-2.3.1-py3-none-any.whl (118 kB)
Collecting scipy>=0.19.1
Using cached scipy-1.9.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (33.7 MB)
Collecting seqeval
Using cached seqeval-1.2.2.tar.gz (43 kB)
Preparing metadata (setup.py) ... done
Collecting tf-estimator-nightly~=2.12.0.dev
Using cached tf_estimator_nightly-2.12.0.dev2023010309-py2.py3-none-any.whl (439 kB)
Requirement already satisfied: gast<=0.4.0,>=0.2.1 in /home/casa/.local/lib/python3.10/site-packages (from tf-nightly->tflite-model-maker) (0.4.0)
Requirement already satisfied: packaging in /home/casa/.local/lib/python3.10/site-packages (from tf-nightly->tflite-model-maker) (22.0)
Collecting tf-nightly
Using cached tf_nightly-2.12.0.dev20230102-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (563.8 MB)
Using cached tf_nightly-2.12.0.dev20230101-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (563.8 MB)
Downloading tf_nightly-2.12.0.dev20221231-cp310-cp3

Hi @RonaldoSCouto !
Thanks for sharing your observations on lite-model-maker-0.4.2 .
Could you upgrade your pip version to latest version  and let us know know.
Ref .
Thank you!

Hi,
I update from pip 22.0.2 to pip 22.3.1, purge pip cache and ran again.
I had stopped installation on tf_nightly-2.12.0.dev20221229  (7 days).
Nothing changed. Same situation. Looking for tf_nightly.

@RonaldoSCouto !
Could you share your thoughts here #53550 as Re-opening this issue will be a duplicate effort.
Thank you!

@RonaldoSCouto !
Re-opening as instructed.  Normally, these issue gets resolved if the requirement in lite_model_maker are followed. we will give you an update once it is tested in windows.
Ref
Thank you!

I tried to reproduce the issue with the colab(Ubuntu OS) but I was able to install successfully without any issues.
Here is the Gist with all the installed files.
Could you check if you have required permissions to install packages in your system, also can you let us know if you have  successfully  installed any other packages in your env like tf-nightly.
sudo pip install tflite-model-maker, if you want to install the package across the system.
or
pip install tflite-model-maker --user,  if you want to install specific to user or env.

Hi, I'm facing the same issue: pip install tflite-model-maker is fetching nightly builds non-stop.
macOS Ventura 13.1.
pip 23.0 running on python 3.10
I'm using venv for the install (I don't want to use a global install)

I was able to install by downgrading to Python 3.9


protect endpoints

/user/add should check if user already exists
rate limit endpoints that send emails/sms by ip of request

@sirpy what rules should be applied for rate limit guard?
For example - 3 requests per 1 minute.

Link to the GoodServer PR - https://github.com/GoodDollar/GoodServer/pull/102


Allow commands to customize exception handling

Fix for #2122
Allows for commands to catch configuration, environment, and command exceptions in one place (eg. forcing all commands to print the stack trace is less than desirable)
Below is an example of a custom command that will print the stack trace to stderr if someone passed in --debug, else print the error message to stdout.
public static final class CustomCommand extends Command {
    @Override
    public void configure(Subparser subparser) {
        subparser.addArgument("--debug").action(Arguments.storeTrue());
    }

    @Override
    public void run(Bootstrap<?> bootstrap, Namespace namespace) throws Exception {
        throw new RuntimeException("I did not expect this!");
    }

    @Override
    public void onError(Cli cli, Namespace namespace, Throwable e) {
        if (namespace.getBoolean("debug")) {
            super.onError(cli, namespace, e);
        } else {
            cli.getStdOut().println(e.getMessage());
        }
    }
}

Reviews, suggestions welcomed. Some things to think about:

I am unsure whether to keep Cli argument in onError or replace it with stdOut and stdErr directly from Cli
Should configured commands get another argument for onError, except it has configuration (if parsed correctly?). Same thing with environment commands.
I decided to not allow customization of HelpScreenException logging

EDIT: I should clarify, this onError gets called before onFatalError so onError doesn't need to do System.exit(1)

Coverage decreased (-0.01%) to 85.199% when pulling 49462dc732a7aa918a8f6a2d66ba408110106c94 on nickbabcock:custom-app-fatal into 4380f87a2ab11b14db07f1d1352f9de90a29fcf0 on dropwizard:master.

I like it! I think it provides great control for those user who want to do more fine-agreed exception handling.
I also agree that the check command doesn't need the whole stacktrace to be dumped, because most of the lines are useless and hide the real error.


Migrate article: Aiven statement on Meltdown and Spectre vulnerabilit…

…ies #1058
https://github.com/aiven/devportal/issues/1058
What changed, and why it matters

Closing since this is not relevant anymore (issue dated back in 2017)


Fix for multipoint

Fixes #4
If this is a hack, apologies. It seems to do the job.

I don't see what the merge conflict is. Sorry.


hotkey() with macOS

I am currently running Python 3.6.0 on macOS Sierra 10.12.6.  I've tried using pyautogui.hotkey('ctrl', 'f') and it is not working.  It does work on Windows 10 and Python 3.4.4, however.
I have tried this, but it does not work either:
pyautogui.keyDown('ctrl')
pyautogui.press('f')
pyautogui.keyUp('ctrl')

Dummy me.  Been using my Windows keyboard on my Mac.  Should be sending the Command key.


HuggingFace models support (mistralrs)

Recently https://github.com/EricLBuehler/mistral.rs released, a really nice solution to, besides mistral, add support for a lot of huggingface models.
By implementing SimplePrompt, existing transformers that prompt should be able to use it right away.

Errors currently, parked for now

@jac-cbi @timonv I would be happy to help integrating mistral.rs :)

@EricLBuehler Thanks for reaching out and love your work! I believe at the time there were multiple things going on, I don't recall exactly what was going on. Back then the only hardware I had to try it out properly was running WSL, and candle was having trouble with the cuda drivers (I think it recently got fixed that they can use dynamically linked).
#70 was done on a much older version of the Swiftide architecture (and mistral.rs), although it might just work with a little moving around. I have some time next week (although EuroRust) to try again.
Have you considered publishing mistral.rs as a crate? It would help with the amount of surport I'd have to provide for different versions.


Couple of issues with new typings in @horizon/client

Hey All,
Thanks for adding typings to the horizon client! Really helps with my workflow.
I noticed a couple of issues, though. First, fetch() and watch() are incorrectly typed to return TermBase when the should return a observable. Like this
        watch (options?: { rawChanges: boolean }): Observable<any>;
        fetch (): Observable<any>;

I imagine this is a mistake, but possibly it's an API change?
Also, if I want to create a function that explicitly accepts a HorizonInstance, like (hz: HorizonInstance) => TermBase you need to add these 2 lines to your typings so I can access the types.
export type HorizonInstance = hz.HorizonInstance;
export type TermBase = hz.TermBase;
export type Collection = hz.Collection;
```

Thanks!

@mrapogee thanks, .fetch() and .watch() returning TermBase is my mistake. I threw the typings together really quick and didn't spend much time testing. I'll get another PR up with all the changes you mentioned. Let me know if you stumble across anything else that's missing/incorrect in the public API.

Hey,
Couple more things.

Some methods were missing from HorizonInstance.

hasAuthToken (): Boolean;
clearAuthTokens (): void;
authEndpoint (authProvider: string): Observable<string>;

You need to add this line to your src index.js

Horizon.default = Horizon

This is because of the way typescript resolves imports. Typescript basically compiles import Horizon from '@horizon/client' to require('@horizon/client').default. But,since you are using export = Horizon in your index, .default will be undefined.

Typescript basically compiles import Horizon from '@horizon/client' to require('@horizon/client').default.

@mrapogee is this still true in the latest versions of Typescript? In the project I was working on (using angular-cli) the import Horizon from '@horizon/client' compiled and ran fine for me.

I think Microsoft/TypeScript#5285 may be related (fixed in v1.8).

@mrapogee also, .clearAuthTokens() is only defined statically. I added the other two missing methods though. Just let me know on the TS module resolution thing and we'll be good to go.

I'm using the latest version of<EMAIL_ADDRESS>and it resolves to undefined without it. I think it works if you use the system module system with typescript, but many people can't.

@mrapogee if you're not using system module for code generation, then shouldn't you be using const Horizon = require('@horizon/client') rather than the import syntax anyway? I'm not sure you're supposed to expect that to work otherwise. I could be wrong though, can you point to any docs?

Checkout https://www.typescriptlang.org/docs/handbook/modules.html
I can do import Horizon = require('@horizon/client') for a module that exports a function directly, but that means you need to change your index.d.ts to export = Horizon. If you do this, you can't export types from the typings file, you can only export one thing.

Is it possible to include the Horizon Client with type definitions right now? I'm still pretty new to Horizon and TypeScript, but shouldn't this be close?
import * as Horizon from '@horizon/client'; // [ts] Cannot find module '@horizon/client'

Also it appears the NPM version doesn't have the definition file yet, so I copied it into my project manually. Can't get it to recognize the name of the module for the life of me.

It's working with this tsconfig.json. You probably just need moduleResolution set to node and you can add "typings": "index.d.ts" to the @horizon/client package.json.
{
  "compilerOptions": {
    "moduleResolution": "node",
    "module": "commonjs",
    "noImplicitAny": true,
    "removeComments": true,
    "experimentalDecorators": true,
    "strictNullChecks": true,
    "noImplicitThis": true,
    "typeRoots": [
      "./node_modules/@types"
    ],
    "target": "es6"
  }
}

You can import it like import Horizon from '@horizon/client', but you'll need to add Horizon.default = Horizon to the end of @horizon/client/lib/index.js. That's my current setup and it's working great.

Thanks for the help. I still can't seem to get it to recognize the import path. I added the moduleResolution parameter to my tsconfig.json, which now looks like:
{
    "compilerOptions": {
        "outDir": "./dist/",
        "sourceMap": true,
        "noImplicitAny": true,
        "moduleResolution": "node",
        "module": "commonjs",
        "target": "es5",
        "jsx": "react"
    }
}

and my Horizon.tsx file:
import Horizon from '@horizon/client'; // [ts] Cannot find module '@horizon/client'.

const horizon = new Horizon();
horizon.onReady(function () {
  document.querySelector('h1').innerHTML = 'Horizon works!'
});
horizon.connect();

I do have @horizon/client installed in node_modules, and I modified the package.json and lib/index.js accordingly.
Still getting the error Cannot find module '@horizon/client'. Any ideas?

@abecks the typings I've added have been merged into next but not released. If you manually npm install @horizon/client@next they should get picked up automatically.

Nevermind, I was missing the index.d.ts file still. I copied it over from the master here and now the import is recognized.
Thanks @marshall007 I'll switch to next now.
Do either of you have an example of how to pass an Observable to the onReady method?

Hey @marshall007, getting the following npm error when trying to install the next branch.
npm ERR! argv "/usr/local/bin/node" "/usr/local/bin/npm" "install" "@horizon/client@next"
npm ERR! node v4.6.0
npm ERR! npm  v3.10.8
npm ERR! code ETARGET

npm ERR! notarget No compatible version found: @horizon/client@next
npm ERR! notarget Valid install targets:
npm ERR! notarget 2.0.0, 2.0.0-beta-7, 2.0.0-beta-6, 2.0.0-beta-5, 2.0.0-beta-4, 2.0.0-beta-3, 2.0.0-beta-2, 1.1.3, 1.1.1, 1.1.0, 1.0.3, 1.0.2, 1.0.1, 1.0.0, 0.1.1-0, 0.1.0, 0.0.4-4, 0.0.4-3, 0.0.4-2, 0.0.4-1, 0.0.4-0, 0.0.3, 0.0.2-1
npm ERR! notarget
npm ERR! notarget This is most likely not a problem with npm itself.
npm ERR! notarget In most cases you or one of your dependencies are requesting
npm ERR! notarget a package version that doesn't exist.

npm ERR! Please include the following file with any support request:
npm ERR!     /Users/andrew/Development/horizon/npm-debug.log

@abecks yea, I was wrong. I thought you could use that shorthand, but apparently that only works for actual npm releases, not git branches. @lirbank has a workaround in #790.


Refactor SiteDB into two parts to make it easy to mock

This is a change that will be part of @jha2 's branch for mocking SiteDB, so don't merge this. However you can look at it. The point is to split SiteDB into two parts. One which actually talks to SiteDB and the other which parses the dictionaries it returns. This makes it possible to use mock to just replace the lower level and leave the upper level alone.
@ticoann and @amaltaro , please feel free to review/comment.

@ericvaandering this one needs to be rebased (sorry!) Should we push it in for tomorrow's testbed as well?

This was already part of Manoj's PR.


Added trees_to_dataframe() method for Booster class

Worked alongside @hshujuan for this PR to add nearly equivalent methodology to python package as R for parsing a dumped text model to a pandas DataFrame. Testing on model built on agaricus dataset included...

Parse 10,000 trees of max_depth 5 in  about 3 seconds
Parse gbtree and dart boosters
Output equivalent to R's xgb.model.dt.tree() function
Can handle trees that have no splits (i.e. leaf only trees)

If any thing about this PR is not adequate, please let me know.
Thanks,
Patrick

Yes. I will do that now.


More improvements to the Controlled infrastructure

Fixes https://github.com/quantumlib/Qualtran/issues/865
Fixes https://github.com/quantumlib/Qualtran/issues/686
The improvements are -

GateWithRegisters.controlled() now returns Controlled bloq
cirq_gate_to_bloq now maps a cirq.Controlled gates to Controlled bloq.
Controlled bloq is now a GateWithRegisters. This is required because we want to support cirq style simulations for circuits containing Controlled(Rx()) bloqs and BloqAsCirqGate(Controlled()) is not sufficient because we need special logic to derive the unitary matrix for controlled bloqs (eg: when subbloq is a GateWithRegisters but doesn't have only THRU registers).

After this PR, we should never need to invoke the cirq.Controlled gates in either cirq-style or bloq-style API in Qualtran and we should always use Controlled bloq in all cases. All existing uses of cirq.Controlled gates would be updated to Controlled bloq when invoking the bloq-style API using the cirq_gate_to_bloq mapping.
This is the final PR in my series of PRs to improve the controlled infrastructure. cc @mpharrigan

@mpharrigan I'm ready to merge this but feel free to take another look since I've made some minor upgrades. Using gwr.controlled().on_registers() and gwr.on_registers().controlled_by are now equivalent and both result GateOperation(Controlled(gwr))  where the wrapped gate is a the Controlled bloq.
Tl;Dr - the discrepancy in usage from a users perspective that I described in Qualtran sync today is now gone and users don't need to worry about the intricacies when using the Cirq or Bloq style APIs.

where is the logic for controlled_by?

where is the logic for controlled_by?

No override is needed because of the way Cirq and Qualtran works. GateOperation.controlled_by delegates to Gate.controlled() by default, which now uses GateWithRegisters.controlled() which returns a Controlled(GateWithRegisters) which is a cirq gate and gets wrapped in the GateOperation(controlled_bloq).
This was true today morning as well, I just realized it when making these changes and thus added tests to gate_with_registers_test.py to make this concrete.

Merging this now


Chunking causing long running process and unresponsiveness on IE10/11

Type of issue [REQUIRED]
Bug
Uploader type [REQUIRED]
S3
Bug details [DELETE EVERYTHING IN THIS SECTION IF THIS IS A FEATURE REQUEST]
Fine Uploader version [REQUIRED]
5.4.1
Browsers where the bug is reproducible [REQUIRED]
IE10 IE11
Operating systems where the bug is reproducible [REQUIRED]
Windows 7, possibly others as well
Exact steps required to reproduce the issue [REQUIRED]

Upload a large file (500mb is what we have been using) using chunked uploads to S3

All of your Fine Uploader initialization JavaScript code [REQUIRED]
var ie = (function(){
    var undef,rv = -1; // Return value assumes failure.
    var ua = window.navigator.userAgent;
    var msie = ua.indexOf('MSIE ');
    var trident = ua.indexOf('Trident/');

    if (msie > 0) {
        // IE 10 or older => return version number
        rv = parseInt(ua.substring(msie + 5, ua.indexOf('.', msie)), 10);
    } else if (trident > 0) {
        // IE 11 (or newer) => return version number
        var rvNum = ua.indexOf('rv:');
        rv = parseInt(ua.substring(rvNum + 3, ua.indexOf('.', rvNum)), 10);
    }

    return ((rv > -1) ? rv : undef);
}());

var s3Uploader;

var cloudfrontMapping = {
  "[Env name]": {
      url: "[Cloudfront distro endpoint]",
      accessKey: "[Access Key]"
    }
  }

  //This gets us an environment from the URL
  var hostEnv = window.location.hostname.split('.')[2];

  var getFineUploaderOptions = function(keyPrefix){
  return {
    element: document.getElementById('sr-fine-uploader'),
    template: 'qq-template-s3',
    request: {
      endpoint: cloudfrontMapping[hostEnv].url,
      accessKey: cloudfrontMapping[hostEnv].accessKey
    },
    cors: {
      expected:true,
      allowXdr:true,
      allowCors: true
    },
    chunking: {
      enabled:true,
      partSize: 10485760,
      concurrent: {
        enabled:ie?false:true
      }
    },
    objectProperties: {
      acl: 'bucket-owner-full-control',
      bucket: [Bucket here],
      host: [Host here],
      region: "us-west-2",
      serverSideEncryption: true,
      key: function(id){
        return keyPrefix() + s3Uploader.getName(id);
      }
    },
    signature: {
      endpoint: '/signature',
      version: 4
    },
    retry: {
      enableAuto: true,
      autoRetryNote: document.getElementById('retry-text').title+" {retryNum}/{maxAuto} ..."
    }
    validation: {
      sizeLimit:<PHONE_NUMBER>
    }
  };
}
var keyPrefix = function(){
  var srDetails = document.getElementById('srDetails').getAttribute('data-params');
  return "id-" + srId + "/";
};
s3Uploader = new qq.s3.FineUploader(getFineUploaderOptions(keyPrefix));

Detailed explanation of the problem [REQUIRED]
When trying to do a chunked upload to S3, IE browser becomes very unresponsive, even on non-concurrent, small chunks. A profile trace revealed that 80% of its time is spent in the _doProcessBlock in the cryptojs imported library.

Potentially a dupe of  #1519?

I dont think so, in this one concurrency is a non-factor, it is off, so it is not a race condition issue. This is on a chunk by chunk basis taking a hugely excessive amount of time to process the cryptojs function in IE exclusively. They look like two separate issues.

There isn't much that can be done about the CPU usage here. This is a fundamental problem with the version 4 signing algorithm created by AWS. It requires Fine Uploader to hash each file chunk sent via the multipart upload API. This requires reading each chunk into memory and then calculating a SHA256 hash given the file bytes. For underpowered machines, this will have a noticeable impact on the CPU.
I'm not seeing the level of non-responsiveness that you've described, and I suspect this is only an issue on slow machines. To prevent the UI thread from being affected by this hash generation, I can explore pushing the hash calculation into a web worker. However, this may not solve the issue you are seeing in IE10/11, since IE does not support passing an ArrayBuffer into a web worker without explicitly copying each byte over the wire.
Your best bet may involve one of the following workarounds:

Don't use version 4 signatures
Use a small chunk size (5 MiB) to avoid pegging the CPU for long continuous periods of time
Ensure your users are utilizing more performant machines.

Sorry that there isn't a better answer to this. I am quite unhappy with Amazon's v4 signature algorithm myself, but there isn't much that can be done about it.

Thank you for the explanation.

Curious: why are you disabling concurrent chunking in IE?

Without the concurrent chunking the issue did not occur, the uploads went smoothly with no spike in CPU. As our temporary workaround for a more long term solution that was the way we went. Our next steps will be to test against 5mb chunks, if that does not fix it, we will determine if v2 signing is a valid method.

As long as you aren't uploading to any v4-exclusive regions, then there is no need to use v4 signatures. There's aren't many v4-exclusive regions at the moment. Frankfurt is one though. Also, concurrent chunking is not a factor here. If you're seeing the issue with one file and concurrent chunking enabled, you'll also see it with multiple in-progress files with concurrent chunking disabled.
You can also eliminate the entire cors section of your config. It's not needed unless your delete/signature endpoints reside on a different domain (yours don't).

Our signature is proxied to another server, without the cors section the headers for the proxy do not get set appropriately in the case of IE9.

Which headers? It seems a bit odd that you must use XDomainRequest to send a request to a same-origin endpoint. The fact that it is proxied server-side shouldn't effect the request sent by Fine Uploader.

Our current routing pattern is a bit weird due to current constraints, our apache server is too old currently to correctly route https requests to AWS which require the SNI protocols that older Apache does not support. Due to this we hit a route on the server that proxies to a NodeJS proxy which proxies using https to our signing method which is fronted by AWS ApiGateway. In the NodeJS proxy, we set the host properly, and add the content-type in the cases where it was not set by the fineuploader request (ie9). However, there were still cors issue with the flow when we tried it that allowing the cors components in fineuploader to solve. Its possible our proxy could manipulate the headers as needed, but we were not able to devote the time to discovering the issue during this setup.

As a clarification to your earlier point about concurrent chunking, you are correct, it was the disabling of chunking altogether that fixed the issue, the reason concurrent was disabled was that we tried that as a first step (chunking enabled, but only one chunk at a time). That the ie ternary is still in the concurrency is simply because it was not removed when we also added the ternary for allowing chunking at all.


[mobile][dialog][select] Select dropdown not opening if inside a dialog from mobile (touch)

🐛 Bug report
Using the select component inside a dialog component works fine on desktops but it doesn't behave correctly.
On mobile (I've been able to test only on iOS for now) it seems like the dropdown never opens after pressing the select button but in reality, it opens and quickly closes (probably sending another touch event right after the opening of the dropdown). I've been able to catch the dropdown quickly opening and then closing with the browser profiler.
💥 Steps to reproduce

Put a Select component inside a Dialog
Open on a mobile device
Open the dialog then tap the select button
The select dropdown never(?) opens up

💻 Link to reproduction
CodeSandbox reproduction: https://codesandbox.io/p/sandbox/wizardly-yonath-zc683j?file=%2Fsrc%2Fcomponents%2FTooltip.vue%3A15%2C1
🧐 Expected behavior
The select dropdown opens just fine when inside a dialog
🌍 System information

Software
Version(s)

Zag Version
0.12.0

Browser
Safari iOS

Operating System
iOS 16.2

📝 Additional information
Use the profiler and record the opening action to be able to catch when the dropdown opens and when it closes back again.

After some investigation, it seems related to focus trapping and the fact the select is teleported.
If you disabled the Teleport for the select, it should work fine.
Looking to find a way to handle this internally, or else we'll document how to manage these scenarios.

The solution is avoiding Teleport when the component is within a dialog. You can design a "portalled" prop to control this use case.
This works because the focus trap needs to consider the select's trigger and content as part of the dialog so it doesn't aggressively hijack the focus back into the dialog.

I'll update the docs to include this tip.
Here's the sandbox:
https://codesandbox.io/p/sandbox/wizardly-yonath-zc683j?file=%2Fsrc%2Fcomponents%2FTooltip.vue%3A15%2C1


Add EditOrDiscardAnswersModalCreatorTest

Description
Please explain the changes you made here.
Release notes
The title of the pull request will be used as the default release notes description. If more detail is needed to communicate to partners the scope of the PR's changes, use this release notes section.
Checklist
General
Read the full guidelines for PRs here

[ ] Added the correct label: < feature | enhancement | bug | dependencies | infrastructure | ignore-for-release | database >
[ ] Assigned to a specific person, civiform/developers, or a more specific round-robin list.
[ ] Created unit and/or browser tests which fail without the change (if possible)
[ ] Performed manual testing (Chrome and Firefox if it includes front-end changes)
[ ] Extended the README / documentation, if necessary

Database evolutions
Read the guidelines here

[ ] Assigned two reviewers
[ ] Guarded against already existing resources using IF NOT EXISTS and IF EXISTS
[ ] Downs created to undo changes in Ups

User visible changes

[ ] Followed steps to internationalize new strings

[ ] Added context strings to new messages
[ ] Didn't use a message in applicant facing code that isn't translated yet (unless behind a flag)

[ ] Wrote browser tests using the validateAccessibility method
[ ] Tested on mobile view. See mobile device mode
[ ] Manually tested at 200% size
[ ] Manually evaluated tab order

New Features

[ ] Add new FeatureFlag env vars to server/conf/helper/feature-flags.conf
[ ] Conditioned new functionality on a FeatureFlag
[ ] Wrote browser tests with the feature flag off and on, etc.

Instructions for manual testing
If instructions are needed for manual testing by reviewers, include them here.
Issue(s) this completes
Fixes #<issue_number>; Fixes #<issue_number>...

Current dependencies on/for this PR:

main

PR #6653   👈

This stack of pull requests is managed by Graphite.

Adding this test still occasionally times out the DurableJobsRunnerTest. We had a discussion in https://civiform.slack.com/archives/C01QKN2UNMA/p1707861889344949 about it but no hypotheses or solutions were found. This test isn't that important (just checks some IllegalArgumentExceptions are thrown or not thrown), so I think it's better to just not add this test and spend time on other, more important things.


[WIP] [explorev2] Refactor filter into FieldSet

Done:

Deleted old Filter and Filters components
Created FilterFIeld component that matches the structure of other fields

Todo:

add js test

Nice to see this!:

LGTM!


Valid json files are being declared "not json" if it doesn't have a json extension

This is probably minor but files that are valid json don't need a json extension (or if they do that should be documented).

::sigh::
Wrong repo.


Read this 2024 TSE paper on SZZ implementations

https://ieeexplore-ieee-org.ezproxy.rit.edu/document/10541859
And this one:
@article{rosa2023szzvariants,
title     = {A comprehensive evaluation of SZZ Variants through a developer-informed oracle},
author    = {Rosa, Giovanni and Pascarella, Luca and Scalabrino, Simone and Tufano, Rosalia and Bavota, Gabriele and Lanza, Michele and Oliveto, Rocco},
journal   = {Journal of Systems and Software},
volume    = {202},
pages     = {111729},
year      = {2023},
publisher = {Elsevier}
doi       = {10.1016/j.jss.2023.111729}
}

In particular, what I'd like to know is:

Can this run on large repos?
About how long would it take to run on ~10k fix commits in larger repos?
Can we feasibly run it on all of our fix commits in mega-foss? Right now approximately 2k commits.


IPsec settings synced to the backup node are not properly committed to the system

Important notices
Before you add a new report, we ask you kindly to acknowledge the following:
[x] I have read the contributing guide lines at https://github.com/opnsense/core/blob/master/CONTRIBUTING.md
[x] I have searched the existing issues and I'm convinced that mine is new.
Describe the bug
Every config change on the IPsec tunnel settings tab is not properly committed to the backup node.
Pressing save, Apply changes and under the High availability tab the all button (or even Synchronize and restart button on the strongswan line) does not actually write the new /usr/local/etc/ipsec.conf on the backup node.
To workaround this, someone just has to logon onto the backup node, and press save on the
IPsec Tunnel Settings tab. (The Apply changes banner does not even appear, because the changed config has been already entered the config xml).
For example, changing the MOBIKE flag on the first first node does not get properly replicated, after committing the change on the first node, and syncing it to the backup node:
root@fw01:~ # grep mobike /usr/local/etc/ipsec.conf
  mobike = yes

root@fw02:~ # grep mobike /usr/local/etc/ipsec.conf
  mobike = no

After just pressing save on the backup node:
root@fw02:~ # grep mobike /usr/local/etc/ipsec.conf
  mobike = yes

To Reproduce
Steps to reproduce the behavior:

Go to 'IPsec' -> 'Tunnel Settings'
Create a new Phase 1 entry. This is already enough to demonstrate it in our case.
Save the tunnel, save all tunnel settings, apply the changes to the system
Sync the configuration to the backup node on the 'System' -> 'High availability' tab
SSH onto both machines and compare the /usr/local/etc/ipsec.conf file

Expected behavior
Without having to click save on the backup node, the new /usr/local/etc/ipsec.conf file gets written automatically after syncing the settings to the backup node.
Screenshots
N/A
Relevant log files
Nothing in the var/log/system.log file.
Additional context
N/A
Environment
Software version used and hardware type if relevant.
e.g.:
OPNsense 19.7.6-amd64
FreeBSD 11.2-RELEASE-p14-HBSD
OpenSSL 1.0.2t 10 Sep 2019
Intel(R) Xeon(R) Silver 4215 CPU @ 2.50GHz (32 cores)
2x Dual Port Intel x520 Cards

did you restart strongswan on the backup (via the master)?

Yes, as already written, i've clicked the all button which does this automatically, and even only the sync and restart button on the strongswan line.

I've reproduced this on 2 OPNsense clusters running the latest version. I might set up a fresh OPNsense cluster the next days, to see if I can reproduce this behavior on a new install.

no need, just reproduced it here, one moment

https://github.com/opnsense/core/commit/5c954133b8e8a9b4fcd6ed0b8b471b4234bf05e2 should fix it, to try, run the following on the backup:
opnsense-patch 5c954133b8
service configd restart

And sync from the master again.

Yes, that fixes it. After it restarted the strongSwan service, the new config file gets written to the disk. Thanks!

you're welcome, closing the issue. have a nice weekend.


Incompatibility with latest ethers version

The latest version of ethers includes support for London fork by extending the Provider interface with an additional method.
This breaks waffle's MockProvider class.
Argument of type 'MockProvider' is not assignable to parameter of type 'Provider'.
  Property 'getFeeData' is missing in type 'MockProvider' but required in type 'Provider'

Yes, same issue when using the fixtures with latest ethers:
| test/1.votingContract.test.ts(94,42): error TS2345: Argument of type '([alice, firstAddress, secondAddress, noTokensAddress]: any[], provider: JsonRpcProvider) => Promise<{ contract: Contract; secondContract: Contract; alice: any; firstAddress: any; secondAddress: any; provider: JsonRpcProvider; noTokensAddress: any; }>' is not assignable to parameter of type 'Fixture<{ contract: Contract; secondContract: Contract; alice: any; firstAddress: any; secondAddress: any; provider: JsonRpcProvider; noTokensAddress: any; }>'.
 |   Types of parameters 'provider' and 'provider' are incompatible.

This is now sorted out in Waffle 4.
Internally we updated deprecated ganache-core to ganache which supports London hardfork.


nuget submission?

Have you considered submitting this to Nuget so that it can be included directly from the IDE?

Not going to submit to nuget. Bower and NPM are far more suitable distribution methods in asp.net core.


Add user contributed module?

Hello, I wrote some python code that will allow for users to view their load average and current load.  I was wondering how I would go about adding this to user contributed modules? I poked around and counldn't seem to find any guidelines for who to contact or formatting specifics.  Thank you

Click on Edit ?

is everyone allowed to contribute without any qa?

The wiki is currently open for contributions by anyone, so you should be good.
As for formatting, just look at how the other user contributed modules do it, and be sure to add your section to the TOC.


Question: "Hoisting" the DestinationsNavigator to the DestinationsNavHost?

Hi. Thanks for this library. It eliminates a ton of boilerplate while using compose navigation.
I'm looking for a way to avoid having to pass in the navigator parameter to each destination composable. My motivation is to keep all the navigateTo logic in one place. Do you have any pointers on how to do this? Or do you think this ends up fighting the system?
Some pseudocode on how I would achieve it with plain compose navigation:
@Composable
fun LoginScreen(onProfileClick: () -> Unit) {
    Button(onClick = onProfileClick) {
        Text("Profile")
    }
}

@Composable
fun ProfileScreen(onAboutClick: () -> Unit) {
    Button(onClick = onAboutClick) {
        Text("About")
    }
}

@Composable
fun AboutScreen(){
    Text("About screen")
}

@Composable
fun MyApp() {
    val navController = rememberNavController()
    NavHost(navController = navController, startDestination = "login") {
        composable("login") {
            LoginScreen(onProfileClick = { navController.navigate("profile" } )
        }
        composable("profile") {
            ProfileScreen(onAboutClick = { navController.navigate("about" } )
        }
        composable("about") {
            AbouScreen()
        }
    }
}

Note that the MyApp composable is the only one that is aware of the navController and it is the only one that makes decisions on where to navigate next.

Hi @curioustechizen !
Not fighting the system at all. You can manually call your screens using the last parameter of DestinationsNavHost method.
Check how here:
https://composedestinations.rafaelcosta.xyz/destination-arguments/navhost-level-parameters#manually-call-your-screen-composable
Keep in mind that in contrast to with official library, you are not building your nav graphs here (that info comes from the navGraph param), so you don’t need to call all composables this way. If for example some of them don’t need any navigation, then you might not call them here and the library will call you annotated Composable directly.
I’ll close this but please let me know below if the solution is good to you 🙂

Cool 👍
Did you manage to do it the way you wanted to? Calling the screens from NavHost and passing lambdas to navigate?

I did achieve it with plain compose navigation. I'll experiment with it using compose-destinations and I'll report my findings here.


Tag a release

It would be useful if you'd tag a release, e.g. v0.0.1 or whatever, so that we can use version in Go dep :)

Preferrably after #4 :)


Connecting to MySQL Database is incredibly slow

Describe the bug
When I try to connect to a MySQL database, it takes upwards of 5 minutes to actually open the connection. It takes less than a second to confirm my credentials and test the connection, however opening the database so I can execute SQL commands and view the database tables takes far too long. I can open my databases with other VS code extensions Such As MySQL and it takes less than a few seconds to connect. Connecting via external tools (Such As DB Vis) work fine as well.
To Reproduce
Steps to reproduce the behavior:

Create Connection Setting
Test Connection Setting
Save Connection Setting
Attempt to open new connection

Expected behavior
It is not fair to set a time expectation on how long a connection should take, as there are numerous things that could affect it. However, it should respond in about the same time as any other connection method used.
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):

SQLTools Version 0.21.0
VSCode Version: 1.39.2
Dialect Used/Version: MySQL 5.5.5 v15.1
OS: Windows 10

This will be edited after my connection is finished. I will replace this with the current log from SQLTools

Is there anything else I can provide for this? SQLTools was just updated and this issue is not any better.

There is some on going effort on that. But it's not about mysql only, we need to keep standards against all drivers.
I'll merge this issue into #322 since that's the root cause of slowness when connecting.

This will be released on v0.22

Can't wait for v0.22 to be available!
Does this fix avoids/limits the information_schema traversing in any way? Or is it just loading it in a separate connection or something like that?


Merge Dev ke UAT 13 Sept 2017

Coverage decreased (-0.7%) to 72.111% when pulling 8fe84b8f5877d1e3e76f6e9a67dda969fb2cbedf on dev into 183912dae8c8c43eeb28b344c8abb7285fdd66e5 on uat.


extra script to create union between ground truth and prediction

When you encounter file not found error, it's usually because you have
mismatched numbers of ground-truth and predicted files.
You can use this script to create missing files as empty file. The rationale
behind is the following:

if a file is missing from the ground truth but is present among the predictions it means that we have false positives
if a file is missing from the predictions but is present in the ground truth it means that we have a false negative

Hello, this script is based on the following, right?
It would be easier if you applied the changes directly to that script.

Yes it is, but they are doing different things. one intersects the two sets of files and mine makes the union. I think it is better to keep them separated, are you of the same opinion?

Oh sorry, I only saw this now! So, by union you mean creating empty files when missing, right?
I think that is a great idea, better than deleting or moving files (what the intersection is doing right now)!

Yes, right. That's what I mean with union. If you think that it is better then the current solution probably it is the case to change also the comment written by the code when it detects that some prediction or gt files are missing. But I let you decide about this.


Implement schema for homepage

Partial implementation for #1036
Test URLs:

Before: https://main--moleculardevices--hlxsites.hlx.page/
After: https://issue-1036-home--moleculardevices--hlxsites.hlx.page/

@alexcarol did you publish the template -> theme change? I don't see the schema appearing on https://www.moleculardevices.com/

I published it afterwards, can you check again? I can see it right now just fine.


[Release] Update top level README.md file to point to 0.9.0 release.

Release 0.9.0 has been published, it's time to update the top level README.md file.

belated +1


feat: add audio file icons

Closes #404

Roadmap:

[X] .aac (Advanced Audio Coding)
[X] .aiff or .aif (Audio Interchange File Format)
[x] .ape (Monkey's Audio)
[X] .flac (Free Lossless Audio Codec)
[X] .mp3 (MPEG-1 Audio Layer)
[X] .m4a (MPEG-4 Audio)
[X] .opus (Opus Audio File)
[X] .ogg (Ogg Vorbis)
[X] .wav (Waveform Audio File Format)
[X] .wma (Windows Media Audio)

Sorry, I made those commits in a few free minutes yesterday, I didn't notice either the extensions table or the breaking changes, I'm already working on fixing it.

Looking good: :NvimWebDeviconsHiTest
However I think we might be backwards here: all the new icons are in icons_by_filename wheras they should be in icons_by_file_extension. The user will be finding music file icons via get_icon with an extension rather than matching an exact file name.
The filetypes will still correctly map to icons_by_file_extension.

Any updates @pauchiner ? This is really close...

Sorry, I was absent due to personal things, as soon as I can I will make a commit with the changes.

So if aren't vim filetypes, how are mapped?
I think im not understanding your point, Sorry.

So if aren't vim filetypes, how are mapped? I think im not understanding your point, Sorry.

No worries; this is not very clear from the names.
Files like these are simply mapped by their extension against the keys of icons_by_file_extension. There's no need for any extra mapping.
Try
:lua print(require("nvim-web-devicons").get_icon("aiff"))

I'm not really sure if PCM file extension is really used, so could be added.

I have already deleted the filetypes that I added at the time, but then by the same logic, those that were previously there should also be deleted, such as mp3, mp4, wav...

I have already deleted the filetypes that I added at the time, but then by the same logic, those that were previously there should also be deleted, such as mp3, mp4, wav...

Good catch! Yes, they should be removed.
In the interest of speed, I'll push that change to this branch.

I'm not really sure if PCM file extension is really used, so could be added.

Sounds good! Any others you think of please do raise a PR...


Allow custom commandline switches added using AddCommandLineSwitch() to appear in the HELP list

This is a wish list item.  However it would improve the user experience.  Allow for a help description to be added as well.

@phatboyg can we easily move from the text file to something generated?

+1

Happy to review a PR on this, but don't think there's enough interest for me to do it.


Split libgmp.a

Fixes the issue mentioned in #83
* utils/hpc/stage1/build/tmp/hpc-bin
* libraries/integer-gmp/stage1/build/libHSinteger-gmp-<IP_ADDRESS>.a
* getDirectoryFiles  [libraries/integer-gmp/gmp/objs/*.o]
libraries/integer-gmp/gmp/objs: getDirectoryContents: does not exist (No such file or directory)

Thanks. We might be able to reuse the standard solution for running at least the Ar builder, but I'm not sure it is worth it.
This pattern seems useful too:
builder <- pathBuilder Builder
cmd [builder]


Get Inches Info Instead of Hundredth of Inches

Amazon Returns the dimensions info in Hundredths of Inches. Is there a way to convert that to regular inches in fraction format

Why not just convert it with PHP?

Can you post a example?


Add gas costs

Opcodes currently don't have gas costs. These need to be specified.

@adlerjohn this could likely be closed, as we now have a gas schedule benchmarking framework that we are iterating on.

Don't we want them properly documented somewhere?  If I wanted to know individual op costs (which I do when writing optimisers) where should I look?

We are working on a framework to autogenerate a yaml file of the costs based on bechmarks, and also generate a rust file which contains the instantiated gas costs.
As we expect the gas costs to change over time as we add more functionality like merkle trees or optimize the vm/client, I'm not sure if statically binding them to the specs is the best move right now.


Add section to Interoperate with OpenQASM 2 file on using Qiskit's built-in gate classes

In this file, we might want to add a section on how to use more of Qiskit's built-in gate classes when importing from OpenQASM 2, or alternatively modify the "Custom instructions" section to include it. Given the timing constraints we've got at the moment, it might be something that can wait, though - it would probably need to be a significantly rewritten version of https://qiskit.org/documentation/apidoc/qasm2.html#legacy-compatibility, since that was targetted at discussing something a bit different.
Originally posted by @jakelishman in https://github.com/Qiskit/documentation/pull/151#discussion_r1362104180
We will need Jake's help to rewrite the content.

Jake is swamped with 1.0 work so I will move this to a later sprint.

Sorry, I said I'd get to this this week, then got pulled into other technical discussion about the primitives interfaces that I wasn't expecting.  I've still got it on my to-dos, I'm just not going to have the draft by EOD today.  I will try again on Monday (sorry!)


Possible bug: ConnPool.Get return "redis: connection pool timeout" with BLPop

Version I'm using
github.com/go-redis/redis/v8 v8.10.0
What happened

create a client

       runtime.NumCPU() == 8
       redis.NewClient(&redis.Options{
		Addr:         s.conf.Redis.Addr,
		Username:     s.conf.Redis.Username,
		Password:     s.conf.Redis.Password,
		MinIdleConns: 10,
	})

create a lot of goroutine each calling client.BLPop(context.Background(), ...) with no timeout
some client.BLPop calls return error: "redis: connection pool timeout", when all other connections are blocked.

Expected behavior
client.BLPop created a new temp connection to do the request
My thoughts
I looked into ConnPool.Get
1. it calls ConnPool.waitTurn, which want to put an object into p.queue
2. since all other connection are blocked in the BLPop call, no one will release the object from p.queue
3. this goroutine got timeout from timer.C

Is that intended or a bug?

BLPOP is a blocking command and is limited by the connection pool size (PoolSize). I think it is not a good idea to use a new connection every time. If you have a large number of BLPOP or similar blocking commands, you can adjust the size of PoolSize.

I agree with @monkey92t that the best solution is probably to increase PoolSize and somehow limit the number of BLPop goroutines in your program.

client.BLPop created a new temp connection to do the request

Always creating a new connection is (very) expensive. And creating a separate pool just for BLPop looks like an overkill.

Yes, sir. Thanks XD


fix(Catalog Management): re-gen after operation name changes

PR summary
This PR contains a re-gen of the Catalog Management service after changes to some operation names.
PR Checklist
Please make sure that your PR fulfills the following requirements:

[ ] The commit message follows the Angular Commit Message Guidelines.
[ ] Tests for the changes have been added (for bug fixes / features)
[ ] Docs have been added / updated (for bug fixes / features)

Current vs new behavior

Does this PR introduce a breaking change?

[ ] Yes
[ ] No

Other information

:tada: This PR is included in version 0.17.13 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


[api/change] Switched to multitenant django-filters

Now that the multitenant django-filters PR has been merged (https://github.com/openwisp/openwisp-users/pull/347), we can switch to using the multitenant version of django-filters in the API filters here.

Coverage: 99.299% (+0.01%) from 99.289% when pulling eeef644a6d04cabab2bdf00623aa26f758ca5963 on switch-to-multitenant-dj-filters into 6c2d8a46f773fec6a45b497fa9a07c1e9568d4dd on master.


Camera not open first Time  XF Android with Xamarin.Essentials

ONLY Post issues against Latest Beta!!!
Make sure you follow all of the setup directions before posting an issue: https://github.com/jamesmontemagno/MediaPlugin#important-permission-information
If you are creating an issue for a BUG please fill out this information. If you are asking a question or requesting a feature you can delete the sections below.
Failure to fill out this information will result in this issue being closed. If you post a full stack trace in a bug it will be closed, please post it to http://gist.github.com and then post the link here.
Bug Information
Version Number of Plugin: 4.4.10-beta
Device Tested On: LG Nexus 5X Android 8.1.0
Simulator Tested On:
Version of VS: VS for Mac 8.4 (build 2657)
Version of Xamarin: <IP_ADDRESS>1265
Versions of other things you are using: Plugin.CurrentActivity (<IP_ADDRESS>), Xamarin.Essentials (1.3.1)
Steps to reproduce the Behavior
G1.- Download the example that is in the plugin repository
2.- Update all nuget and Xam.Plugin.Media to version 4.4.10-beta
4.- If we launch the project on an android device the result is correct
5.- We install Xamarin.Essentials
6.- Uninstall and launch the project on an android device, the result is that the camera does not open at first
Expected Behavior

Check Permission Camera and ReadExternalStorage
await CrossMedia.Current.TakePhotoAsync(options)
showing camera.

Actual Behavior

Check Permission Camera and ReadExternalStorage
await CrossMedia.Current.TakePhotoAsync(options)
First time not showing camera.
Repeat step 2
showing camera.

Code snippet
`
private async Task TapTakePhoto()
{
if (CheckCameraPermission() && CheckFilePermission())
{
var file = await CrossMedia.Current.TakePhotoAsync(new Plugin.Media.Abstractions.StoreCameraMediaOptions
{
Directory = "Documents",
SaveToAlbum = true,
CompressionQuality = 92,
PhotoSize = PhotoSize.Medium,
DefaultCamera = Plugin.Media.Abstractions.CameraDevice.Rear,
Name = "document_img.png"
});
            if (file == null)
                return;
            byte[] b = System.IO.File.ReadAllBytes(file.Path);
            string fichero64 = GlobalElements.BuildFieldFile(b, file.Path);
        }

}`
Screenshots

I have same issue and not only just click twice each times, sometime,  I need to click 3 or 4 times  :(

Any updates on this?

This plugin uses the Xamarin.Essentials, please follow the setup guide: http://aka.ms/essentials-getstarted

Fixed this by downgrading to Xamarin.Essentials to v1.6.0 while using Plugin.Media v5.0.1


[FR] Add support for GraphQL based queries/criteria

It’d be dreamy to be able to write a GraphQL query as an option, instead of supplying an element query via criteria. Technically, it could/should remove the need for a transformer too.
ScoutIndex::create('articles')
    // gql(string $query, array $variables): self
    ->gql('
        query($section: [String]) {
          entries(section: $section) {
            id
            title
          }
        }
    ', [
        'section' => 'articles',
    ]);

We would also need a way to specify the target schema.

Potential method arguments...
// string $query
$query = '
    query($section: [String], $offset: Int, $limit: Int) {
      scoutObjects: entries(section: $section, offset: $offset, limit: $limit) {
        id
        title
      }
      scoutTotal: entryCount(section: $section)
    }
';

// array $variables
$variables = [
    'section' => 'opportunities',
];

// \craft\models\GqlSchema $schema
$schema = Craft::$app->getGql()->getPublicSchema();

// int $limit = 100
$limit = 100;

Potential logic...
In short, Scout could expect back scoutObjects and scoutTotal, or similar, to provide a uniform way to process the returned data.
if (StringHelper::containsAll($query, ['scoutObjects', 'scoutTotal']) === false) {
    throw new Exception('Query must return `scoutObjects` and `scoutTotal`');
}

$offset = 0;
$processedCount = 0;
$totalCount = null;

$variables['limit'] = $limit;

while ($totalCount === null || $processedCount < $totalCount) {
    $response = Craft::$app->getGql()->executeQuery($schema, $query, $variables);

    $scoutObjects = ArrayHelper::getValue($response, 'data.scoutObjects');
    $totalCount = ArrayHelper::getValue($response, 'data.scoutTotal');

    $variables['offset'] = $offset += $limit;
    $processedCount += $limit;

    $this->addObjects($scoutObjects);
}

I like the idea that it could potentially eliminate transformers, but in general it seems like a round-about way to ultimately end up with a ElementQuery. If Craft data was "GQL-first", it would be different but while it is backed the ElementQuery inferface, I'm not sure it would make sense, other than just being neat :)
I'll keep thinking about it, though…and of course I'm open to considering a PR, I just don't know if it is something I would want to implement right now.

Thanks for your thoughts, @timkelty. I think that’s really fair pushback.
We have a number of projects that now consume Craft data via GraphQL so it’s helpful to match the schema in Algolia for display purposes. Ultimately, there’s nothing stopping us from just executing a GraphQL call per element in the transformer (except the obvious performance issues when batching).

I see, so are you thinking you'd potentially pull in that GQL query from another file that you would be sharing with the frontend/client, and then they could magically stay in sync?
If thats what you're thinking, they yes, your idea becomes more appealing :)

That would become an option in a monorepo, yeah.
This is, of course, doable now with the approach I mentioned on a per element basis. It’d just be more flexible, cleaner and, potentially, more optimised pulling it up the chain.


Prettied up the login and register pages a tad

the login, register, forgot password buttons now look a bit weird

same thing with signup, the alignment should be right and not left

after fixing looks good to merge

on second thought, only thing is forgot password alignment

looks good as is


New docs

Finish Ultimate guide
Add new doc about implementing the data pipeline

:tada: This PR is included in version 2.2.3 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Feature/custom application.conf

Summary
Allow the user to define a custom application.conf in the metadata folder.
PR Type: Feature
Status: Ready to review
Breaking change? No

Codecov Report

Merging #101 (2c1fb3f) into master (e4cde94) will increase coverage by 0.85%.
The diff coverage is 52.38%.

:exclamation: Current head 2c1fb3f differs from pull request most recent head 0557481. Consider uploading reports for the commit 0557481 to get more accurate results

@@            Coverage Diff             @@
##           master     #101      +/-   ##
==========================================
+ Coverage   67.66%   68.52%   +0.85%
==========================================
  Files         121      121
  Lines        5066     5080      +14
  Branches      257      264       +7
==========================================
+ Hits         3428     3481      +53
+ Misses       1638     1599      -39

Flag
Coverage Δ

unittests
68.52% <52.38%> (+0.85%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

...i/starlake/job/index/bqload/BigQuerySparkJob.scala
56.25% <ø> (ø)

...e/job/index/connectionload/ConnectionLoadJob.scala
93.33% <0.00%> (ø)

...scala/ai/starlake/job/index/esload/ESLoadJob.scala
83.63% <0.00%> (-1.55%)
:arrow_down:

...c/main/scala/ai/starlake/job/ingest/AuditLog.scala
46.15% <0.00%> (ø)

...a/ai/starlake/job/ingest/XmlSimplePrivacyJob.scala
0.00% <0.00%> (ø)

...ain/scala/ai/starlake/job/metrics/MetricsJob.scala
91.66% <0.00%> (ø)

...la/ai/starlake/schema/handlers/SchemaHandler.scala
57.47% <ø> (ø)

...in/scala/ai/starlake/utils/CometObjectMapper.scala
100.00% <ø> (ø)

.../main/scala/ai/starlake/utils/YamlSerializer.scala
85.29% <0.00%> (-2.59%)
:arrow_down:

...scala/ai/starlake/workflow/IngestionWorkflow.scala
56.89% <33.33%> (ø)

... and 11 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ee5d073...0557481. Read the comment docs.


Date picker not converting local time to UTC when used for publish_start_date

Description
When picking a date and time from the date picker for a publish_start_date and publish_end_date, it does not appear to be converting local time to UTC time before saving to the database. I discovered this when trying to do some date calculations based on now() and found that the values were incorrect.
Steps to reproduce
Select a date and time from a date picker for the publish dates and save the resource.
Expected result
Should convert the local selected time to UTC, and converted back.
Actual result
It's saving the dates in the local time as-is, with no conversion.
Versions
Twill: 2.1.1
Laravel: 7.29
PHP: 7.4.16
Database: MySQL

Hello @kirkbushell,
I am checking this, currently, the front-end does not work timezone aware, so it assumes the timezone of the back-end is used.
I think a good solution here would be to add a user setting that allows the logged-in user to set their preferred timezone.
This timezone will be used on the front end and passed along to the back-end where the conversion happens.
This strategy we should somehow apply to every date field used.

[ ] Convert it on pre-save
[ ] Visualize the timezone in the front end

What do you think @ifox?

Yup, that's exactly what's happening :)
I don't think a setting is required. The frontend should always show it in the user's timezone. That's generally accepted behaviour on user interfaces, and it makes the most sense.

This should be resolved in 3.x with: https://github.com/area17/twill/pull/1724