HDF5 to Zarr

It would be useful to have a convenience function that copies datasets across from an HDF5 file or group, recursing down through the group hierarchy. Basically a recursive copy of HDF5 group into a Zarr group.

Also agree this would be cool. Personally would be interested in conversion back from Zarr to HDF5 as well.
However, how do you handle some of the more complicated features when going to Zarr. For example, HDF5 has internal links, external links, region references, named datatypes, enumerations, etc. FWICT it doesn't seem like these things are supported by Zarr. Though I suppose they could be. What would you propose to do with these things?

As I was wanting to play with Zarr some more and needed a way to get an HDF5 file into Zarr, I wrote a primitive function to do this. Happy to PR it somewhere as you see fit. Only handles Datasets and Groups.
import os
import h5py

def hdf5_to_zarr(hdf5_file, zarr_group=None):
    try:
        unicode
    except NameError:
        unicode = str

    opened = False
    if isinstance(hdf5_file, (bytes, unicode)):
        hdf5_filename = hdf5_file
        hdf5_file = h5py.File(hdf5_file, "r")
        opened = True
    else:
        hdf5_filename = hdf5_file.filename

    if zarr_group is None:
        zarr_name = os.path.splitext(data)[0] + os.extsep + "zarr"
        zarr_group = zarr.open_group(zarr_name)

    def copy(name, obj):
        if isinstance(obj, h5py.Group):
            zarr_obj = zarr_group.create_group(name)
        elif isinstance(obj, h5py.Dataset):
            zarr_obj = zarr_group.create_dataset(name, data=obj, chunks=obj.chunks)
        else:
            assert False, "Unsupport HDF5 type."

        zarr_obj.attrs.update(obj.attrs)

    hdf5_file.visititems(copy)

    if opened:
        hdf5_file.close()

    return zarr_group

To do the same thing when converting back to HDF5, it would be pretty handy to be able to use the same visitor pattern used here.
xref: https://github.com/alimanfoo/zarr/issues/92

On Wednesday, November 30, 2016, jakirkham<EMAIL_ADDRESS>wrote:

Also agree this would be cool. Personally would be interested in
conversion back from Zarr to HDF5 as well.
However, how do you handle some of the more complicated features when
going to Zarr. For example, HDF5 has internal links, external links, region
references, named datatypes, enumerations, etc. FWICT it doesn't seem like
these things are supported by Zarr. Though I suppose they could be. What
would you propose to do with these things?

I was just thinking to implement something simple that handled arrays
(datasets) and groups and skipped any of the other things you can find in
HDF5, at least for now.
The other slight complication is that, when copying data from HDF5 into
Zarr, you might not always want to use exactly the same compression
options, chunk layout, etc. E.g., I have a use case where some datasets are
better with Blosc+LZ4 and others are better with Blosc+Zstd, and some are
better with C layout while others are better with F chunk layout. It would
probably be reasonable as a first implementation to ignore all this and
just use a single configuration for all arrays that get copied over, but
thought I'd mention it for the record. The only other idea I had was to
allow the user to specify some mapping of paths to compression
configurations, with possibly allowing wildcards in the paths, and these
then get matched as you walk down the hierarchy copying stuff over, but
that might be a little complex to start with.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/alimanfoo/zarr/issues/87#issuecomment-263930538, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAq8Qo_vAAihZ8Bstwm0wMbgCv9YEK36ks5rDawMgaJpZM4KdnU2
.

--
Alistair Miles
Head of Epidemiological Informatics
Centre for Genomics and Global Health http://cggh.org
The Wellcome Trust Centre for Human Genetics
Roosevelt Drive
Oxford
OX3 7BN
United Kingdom
Email<EMAIL_ADDRESS>Web: http://purl.org/net/aliman
Twitter: https://twitter.com/alimanfoo
Tel: +44 (0)1865 287721

Yeah, switching these various format options does get a bit hairy. Would be curious for my own edification to understand a bit more about the motivations for these sorts of changes.

FWIW when I have an array with more than 1 dimension, I usually try storing
at least a test region of the data using C and F layout and see which gives
better compression ratio. Which gives better compression will depend on the
autocorrelation structure in the data. E.g., for a 2D array, are the data
more correlated across rows or down columns?
Regarding different compression options, I usually try Blosc+LZ4 and
Blosc+Zstd and look at compression ratio and also benchmark read and write
speed. Also worth trying different compression levels and shuffle options,
can make a big difference. You might have seen this already but here's a
benchmark I did for genotype data (which is the main thing I work with):
http://alimanfoo.github.io/2016/09/21/genotype-compression-benchmark.html.
Note this is only benchmarking data stored in memory, if you are putting
data on disk then optimal compression configuration will probably change.
On Wednesday, November 30, 2016, jakirkham<EMAIL_ADDRESS>wrote:

Yeah, switching these various format options does get a bit hairy. Would
be curious for my own edification to understand a bit more about the
motivations for these sorts of changes.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/alimanfoo/zarr/issues/87#issuecomment-263989983, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAq8QsNP3QrDO1c2VkpeY3OQ5V9xxdinks5rDeBmgaJpZM4KdnU2
.

--
Alistair Miles
Head of Epidemiological Informatics
Centre for Genomics and Global Health http://cggh.org
The Wellcome Trust Centre for Human Genetics
Roosevelt Drive
Oxford
OX3 7BN
United Kingdom
Email<EMAIL_ADDRESS>Web: http://purl.org/net/aliman
Twitter: https://twitter.com/alimanfoo
Tel: +44 (0)1865 287721

This might be a bit perverse, but one thing we could consider, which I believe you and others have mentioned before, is we could create a MutableMapping-based HDF5 store. Then we could leverage store conversion methods ( https://github.com/alimanfoo/zarr/issues/137 ) to move the data to HDF5. Not sure if that would yield the desired outcome or not, but that should leave things pretty general.

When I mentioned previously using HDF5 as a Zarr store, that was the idea that you could (ab)use an HDF5 file as a simple key-value store. E.g.:
In [63]: class HDF5Store(MutableMapping):
    ...:     def __init__(self, h5g):
    ...:         self.h5g = h5g
    ...:     def __getitem__(self, key):
    ...:         ds = self.h5g[key]
    ...:         if ds.shape:
    ...:             return ds[:]
    ...:         else:
    ...:             return ds[()]
    ...:     def __setitem__(self, key, value):
    ...:         try:
    ...:             del self.h5g[key]
    ...:         except KeyError:
    ...:             pass
    ...:         self.h5g[key] = np.asarray(value)
    ...:     def __delitem__(self, key):
    ...:         del self.h5g[key]
    ...:     def __iter__(self):
    ...:         return iter(self.h5g)
    ...:     def __len__(self):
    ...:         return len(self.h5g)
    ...:

In [64]: import numpy as np

In [65]: import h5py

In [66]: h = h5py.File('spike.h5', mode='a')

In [67]: store = HDF5Store(h)

In [68]: z = zarr.zeros(100, chunks=10, store=store, overwrite=True)

In [69]: list(h)
Out[69]: ['.zarray', '.zattrs']

In [70]: z[:] = 42

In [71]: list(h)
Out[71]: ['.zarray', '.zattrs', '0', '1', '2', '3', '4', '5', '6', '7', '8', '9']

In [72]: h['0']
Out[72]: <HDF5 dataset "0": shape (), type "|S96">

Above each chunk of the Zarr array gets stored in a separate dataset in the HDF5 file.
This is a different level from what this issue is asking for, I think, where you want to mirror across the groups and arrays of a Zarr hierarchy into groups and datasets in an HDF5 file (or vice versa).

I see. That is very different indeed.

Hey, I just came across this project, very cool!
There are a lot of similarities between the Zarr storage layout and what I created for the S3-backed HDF Service.  For some background see this presentation from SciPy: http://s3.amazonaws.com/hdfgroup/docs/hdf_data_services_scipy2017.pdf.
And here's documentation on the schema layout used by the service: http://s3.amazonaws.com/hdfgroup/docs/obj_store_schema.pdf.
From what I've seen of the Zarr format I suspect the differences are mainly in a more "HDF5-centric"  way of describing the data.  E.g. types like Region References that aren't part of the Numpy world.
Also the HDF5 object schema assigns UUIDs for each object and then prepends a 5 character hash in front of the object key.  This helps out in certain cases where you have many clients trying to read the same S3 object (S3 hits a limit at about 100 req/sec for requests where the first few characters of the key match).
As part of the h5pyd project (https://github.com/HDFGroup/h5pyd) I've created apps for listing content, download from S3 to local HDF5 files, uploading to S3, etc.  The apps using the service API though, not directly reading/writing to S3.
It would be great to find a way to collaborate between these projects.  There will certainly be situations where clients would prefer to be able to read/write to object storage directly rather than going through a service interface.

Hi John, thanks a lot for getting in touch, I wasn't aware of the HDF service work, very cool also!
FWIW, one thing that @mrocklin pushed for in Zarr was to encapsulate object storage behind the MutableMapping interface. This has played out very nicely so far, as most of Zarr's logic can sit above this interface, and all issues to do with communicating with object storage (whatever type of storage it happens to be) can be hidden behind, making adding support for other types of object storage pretty straightforward.
I wasn't aware of the S3 rate limitation issue, that's good to know. Zarr uses a very simple string for chunk IDs built from indices of the chunk within the chunk grid (e.g., '0.0' for top left chunk in a 2D array), so this could definitely be a problem if many clients were trying to read from the same region.
Very happy to explore opportunities for collaboration at any level.


TEST-#7064: Explicitly check for exceptions in test_groupby.py

What do these changes do?

[x] first commit message and PR title follow format outlined here

NOTE:  If you edit the PR title to match this format, you need to add another commit (even if it's empty) or amend your last commit for the CI job that checks the PR title to pick up the new PR title.


⚠ Bofrost.de has degraded performance

In 7f549c9, Bofrost.de (https://www.bofrost.de) experienced degraded performance:

HTTP code: 200
Response time: 1774 ms

Resolved: Bofrost.de performance has improved in a282655 after 9 minutes.


Home Assistant 0.99.3

This is rather a huge pull request which probably could be split, but still.
This PR adds support for the latest (ATM) version of HomeAssistant 0.99.3:

Remove unused recipes
Update those which are still needed
Add new required ones

This basically works, but please expect more changes when I touch components/integrations/sensors of HA.
Known issues:

chromecast is not integrated
You name it

Thank you

I forgot to mention that HA now wants python >= 3.6, so there is a requirement for warrior now which provides python 3.7.2

@andr2000 Thanks for the PR.
For me it is fine to make warrior a requirement.
I don't have the means to test this, so I will just look trough your changes and then merge them.

Thank you!
Well, I will definitely add more recipes and fixes as I progress with HA. Unfortunately there is no easy way to check if all the requirements are met unless you step into some new previously untested component ;)

First of all, thank you for the upgrade. It's been on my todo list for a while.
It appears you have been quite aggressive with your deletions, for instance the yahoo weather component requires yahooweather:
https://github.com/home-assistant/home-assistant/blob/dev/requirements_all.txt#L2007
Can you revisit your deletions and make sure they aren't required by sub-components?

I've been intending to add support for the technologies I use: HomeKit, ecobee and Z-wave. Currently running on OpenHab2...

@moto-timo As I said I am not able to test all the functionality, so I decided to leave only those components which are needed for HA to start. For example, I need ebusd component which I didn't put into the PR. So, I was just not sure when to stop, e.g. we cannot have all the components integrated (unless someone is brave enough to solve all the dependencies and those of the additional components itself ;). That was the reason for aggressive deletion.
We can probably get to some acceptable HA configuration in a step-by-step manner, e.g. have basic functionality running and then adding new components. But I am not quite sure we can have something which will fit all the use-cases. Or we have to install pip on the target...

How about
├── meta-homeassistant
│   ├── recipes-devtools
│   │   └── python
----------------> **HA base components to make it running - this is what I have now**
│   ├── recipes-homeassistant
│   │   ├── homeassistant
│   │   │   └── python3-home-assistant-frontend_20190919.1.bb
│   │   └── images
│   │       └── core-image-homeassistant.bb
│   └── recipes-integrations
│           └── python
----------------> **put all additional components and their dependencies here**
│               └── python3-ebusdpy_0.0.16.bb

so we put all additional components in recipes-integrations, e.g. ebusd in my case and yahoo in yours?

@andr2000 The original idea to not have pip in the target was to be able to use a read only rootfs and the posibility to use it offline. My preferred way would be to have an automated test for every integration that is supported by this layer so we know they are working. Not sure how feasible this is tough.
Another approach would be to have some kind of heuristic that tries to find missing packages by parsing requirements_all.txt. In the ideal case this layer would provide everything that is needed for all integrations, but this might be to big.
And I like your proposal to have the essential dependencies from the integration ones.

I also tend to have NO pip installed, but it can always be added depending on other people use-cases and in other people's bbappends ;)
So, I would go with "base" HA and then "recipes-integrations". Base can be way easier validated by itself and then every time we add a new integration it needs to be validated separately.

I'm currently traying to add a CI job that builds the home assistant image https://github.com/bachp/meta-homeassistant/tree/ci but it seems the current repo doesn't work with master and warrior.

cool, I indeed had daemonize recipe, but in a different meta layer ;)

@bachp  the warrior build probably failed because the kas-warrior.yml file is missing the layers for meta-openembedded (sync it with kas-master.yml). It looks like the master build halted because warrior had already failed. The .gitlab-ci.yml is expecting kas.yml, but now we have kas-warrior.yml and kas-master.yml

@moto-timo kas-warrior.yml derives from kas-master.yml it just overrides the branch so this should work. I think the issue is that python3-daemonize is missing. @andr2000 Can you add it?
The .gitlab-ci.yml is currently not used as I was focusing on Github Actions for the moment. I should probably remove it in oder to avoid confusion.

@bachp I did, see bf332359558612f70a17b9b9a01f37855373a16d


pipenv .venv folder inside project root

When I use command pipenv --three to create virtualenv, it init env to folder $USER_HOME/.venvs/<name>. Is it possible to set virtual env folder path to $PROJECT_ROOT/.venv
In docs I see that there is some env variable PIPENV_VENV_IN_PROJECT but i can't find example how to activate it.
Thanks

In bash, export PIPENV_VENV_IN_PROJECT=1


Rosetta: Ensure duplicate error fires

Explain your changes:
The daemon can't disambiguate between duplicates and bad nonces on
transactions. So we'll use a small ring buffer as a cache of
successfully sent transactions. When we receive a bad nonce error, we'll
check the cache to see if we've already sent this transaction this
session.
Explain how you tested your changes:

Tested by running a build connected to one of the nodes on devnet and
confirmed that the duplicate error can be seen.

@bkase If there's a bad nonce, could you instead query the database for a transaction with the same hash and nonce?

ah smart -- I'll add that as another mechanism when I get back tonight

Closed as these changes are already present in the current main branches.


modify aarch64 hv exception part in axhal

And, rebase to the latest vmm branch pls.

fix conflicts manually pls


feat(Google): Added the passport-google-oauth20 support

Maybe it should be a new repository. I found that there is no example for google strategies, so I finished the part of passport-google-oauth20

A Google example is now available in a separate repository: https://github.com/passport/express-4.x-google-oauth2-example


Moving zip_iterator and transform_iterator to iterator_support module

So far these were left out of the iterator_support module because they cause a circular dependency (https://140433-4455628-gh.circle-artifacts.com/0/hpx/artifacts/circular_deps.svg). I'm not sure if creating a separate module or trying to resolve the dependency will be better. Ideas?

So far these were left out of the iterator_support module because they cause a circular dependency (https://140433-4455628-gh.circle-artifacts.com/0/hpx/artifacts/circular_deps.svg). I'm not sure if creating a separate module or trying to resolve the dependency will be better. Ideas?

The file hpx/util/detail/reserve.hpp has two functions: reserve_if_reservable and reserve_if_random_access_by_range. Only the former is needed by the serialization module and only the latter requires to include the range-related things from util. Thus duplicating (and possibly renaming) the implementation of reserve_if_reservable (which is trivial) in the serialization module should do the trick. Thanks for noticing!


HashCode

HashCode
Some of the structures like Map, Set, MapEntry, ... require used types to have HashCode() method or to be primary types to compute a hash in their own HashCode() method as shown in code below.
We can not force type in interface to implement a Hashable interface like :
export interface MapEntry<K implements Hashable, V implements Hashable>

as it would exclude primary types. We need some better way to handle the need of a hashCode method.
Maybe by throwing a MissingHashCodeException when object.HashCode is not defined and not a primary type.
Code
// MapEntry.ts

hashCode(): number {
        let hash: number = 1;

        if ((<any>this.getKey()).hashCode && (<any>this.getValue()).hashCode)
            hash = hash * 17 + (this.getKey() === null ? 0 : (<any>this.getKey()).hashCode()) +
                (this.getValue() === null ? 0 : (<any>this.getValue()).hashCode());
        else if ((<any>this.getKey()).hashCode && !(<any>this.getValue()).hashCode)
            hash = hash * 17 + (this.getKey() === null ? 0 : (<any>this.getKey()).hashCode()) +
                (this.getValue() === null ? 0 : (<any>this.getValue()) as number);
        else if (!(<any>this.getKey()).hashCode && (<any>this.getValue()).hashCode)
            hash = hash * 17 + (this.getKey() === null ? 0 : (<any>this.getKey()) as number) +
                (this.getValue() === null ? 0 : (<any>this.getValue()).hashCode());
        else
            hash = hash * 17 + (this.getKey() === null ? 0 : (<any>this.getKey()) as number) +
                (this.getValue() === null ? 0 : (<any>this.getValue()) as number);

        return hash;
    }

Something like that should help.

Find a way to correct that 0b48b4452723a5e5e0a35dc22d828d5f6eadfa91


Add sample genesis for the competition

Moved to https://github.com/b-harvest/gravity-dex-testnets/issues/1


Prometheus Federate API endpoint

Is your proposal related to a problem?

I'd like my federated prometheuses instances to use thanos as datasource instead of the underlying prometheus. That would offer deduplicated metrics without the need of adding a querier sidecar to all of them.
Describe the solution you'd like

Expose a feature complete /federate endpoint to be used with Prometheus Federation feature.
Describe alternatives you've considered

Build or use an existing proxy that translate the Query API.
Found https://github.com/snapp-incubator/thanos-federate-proxy and https://github.com/G-Research/thanos-remote-read that does not look well maintained.
Additional context

Previous issues:

https://github.com/thanos-io/thanos/issues/1360

Previous implementations:

https://github.com/thanos-io/thanos/pull/676
https://github.com/thanos-io/thanos/pull/1546

Slack discussions:

https://cloud-native.slack.com/archives/CK5RSSC10/p1626430798295800
https://cloud-native.slack.com/archives/CK5RSSC10/p1602664848261700
https://cloud-native.slack.com/archives/CK5RSSC10/p1632385059369700
https://cloud-native.slack.com/archives/CK5RSSC10/p1592820597116900
https://cloud-native.slack.com/archives/CK5RSSC10/p1570470842380100
https://cloud-native.slack.com/archives/CK5RSSC10/p1617800172179800
https://cloud-native.slack.com/archives/CK5RSSC10/p1630825274122200
https://cloud-native.slack.com/archives/CK5RSSC10/p1630925221128800
https://cloud-native.slack.com/archives/CK5RSSC10/p1576860756049900

That would offer deduplicated metrics without the need of adding a querier sidecar to all of them.

If using the existing proxy API way like exemplars, rules, the logic is a bit different. There is no deduplication for two Prometheus and we still reserve the external labels. If we want to do deduplication then it is doable but logic is more complicated.

This is still something I'd love to see in Thanos.

I have seen some use cases that require the federation API. Specifically having federation API for Ruler to expose the record metrics in TSDB.
Although this issue mainly targets Thanos Query, we can revisit it to see if we want to support it or not.

We currently use prometheus federation to pull metrics from remote prometheus instances for centralized alerting on some managed environments, however this causes a lot of duplicate metric warnings because we run multiple instances of prometheus on the remote side that collect the same metrics. We use thanos on the remote side to dedup the data, so we would love to pull the federated data from thanos instead of from prometheus.

We currently use prometheus federation to pull metrics from remote prometheus instances for centralized alerting on some managed environments, however this causes a lot of duplicate metric warnings because we run multiple instances of prometheus on the remote side that collect the same metrics. We use thanos on the remote side to dedup the data, so we would love to pull the federated data from thanos instead of from prometheus.

Same use case

Has anyone tried this? https://github.com/snapp-incubator/thanos-federate-proxy

I would also like this.  (I think I commended on some other bug about efficiency of /api/v1/query some time ago, but I could not find it today).
We have some k8s clusters with Thanos (mostly OpenShift), but we prefer to build some data sets, alert rules and dashboards outside of OpenShift, because of how easier we can make it using our own tools, compared to what OpenShift or in general k8s manifests can achieve. In fact we would not use Thanos at all, if it was not because OpenShift uses it by default, and there is no way to disable it, and there is no easy way to expose Prometheus / Thanos normal query interfaces to outside of the cluster (OpenShift disables them, and trying to enable them, they will reconverge back to blocked).
Currently we use a simple python program between prometheus and thanos, called thanos_converter that simply does /api/v1/query?query={namespace="foo"}, then on the fly converts json to text metrics exposition format. It works okish, but is really slow, causes Thanos to load entire result to memory first, serialize to json, then start outputing. That can easily take 30 seconds, before the first byte of response. Then my converter uses Python ijson library (incremental json parser) to convert response on the fly to exposition format (300MB uncompressed, as it is on localhost anyway), in about 10 seconds.
But now this uses a lot of CPU and memory on Thanos side, a lot of CPU in the converter (~10 seconds of full CPU usage on 1 core in the converter), and is close (~40 seconds) to our scrape interval of 60 seconds.
We have been running this thanos_converter to about 3 years, and I never liked it too much (because of Thanos poor performance), but it looks like https://github.com/snapp-incubator/thanos-federate-proxy is doing very similar thing (with exception that our code uses way less memory, due to response streaming).
I am attaching the code, how to use should be trivial (just run it with --help, and set few params), as how to scrape it. MIT license, Witold Baryluk, 2021-2024.
thanos_converter.tar.gz

Has there been additional work on this?  I have a use case where it would be very handy to be able to scrape metrics from a Prometheus-like federate endpoint to feed specific metrics into a SAAS observability system rather than the full Prometheus firehose.  We are using Thanos to aggregate metrics from Prometheus in several Kubernetes clusters.


Metrics Revision: Types of Contributions

Name of the Metric Being Revised
Types of Contributions
https://github.com/chaoss/wg-common/blob/main/focus-areas/contributions/types-of-contributions.md
Link to Original Metric Release Issue
https://github.com/chaoss/wg-common/issues/56
Specific Metric Details that Require Attention

Description needs to clearly identify what is being measured. Move the first sentence into the objective section.
Add explicit objectives and DEI objective (there is some content here but I think we need a little more)
Link to relevant CHAOSS metrics in the "Collect Trace Data:" section
Update template and add content where appropriate
Add revision date

Checklist

Process

[x] Create the “revising metric” labeled issue in the authoring WG’s repo for comments during review period
[ ] Create pull request of revised metric to WG’s repo (after checking Content Quality and Technical Requirements below)
[ ] Add the metric revision to release notes issue in working group repo
[ ] Update the Metrics Spreadsheet to indicate that the revised metric is under review
[ ] Create issue in CHAOSS/Translations repository to kick-off translation to other languages (please use the the translation issue template)
[ ] "Metric Candidate Release" label added to the metric release candidate issue when fully ready fo release

When above steps are completed:

[ ] Announce new/updated metric on mailing list, newsletter, community Zoom call, and Twitter. This can be coordinated with the community manager and can be done as a collective of all revised metrics from the CHAOSS community.

Content Quality (check all that apply)

[ ] Date of last review has been added to the revised metric at the bottom of the markdown file (month/year)
[ ] Formatting changes have been made
[ ] Minor editorial changes have been made
[ ] Major editorial changes have been made
[ ] Metric adheres to the current metrics template
[ ] Metric name has changed and necessary updates are made to the WG README table and metrics tracking spreadsheet

Technical Requirements for any Revisions

[ ] Metric file name is the full metric name and only contains lower case letters and hyphens (“-”) for spaces
[ ] Images are included using markdown and absolute path links (as described in the metrics template)
[ ] Images have at least one empty line above and below them
[ ] Images are placed in image folder and followed naming convention
[ ] If new a focus area is created, ensure focus area is added to wg repo readme and focus area folder readme
[ ] There is no HTML code in the metrics markdown file

Doing some repo cleanup today and because we are currently updating all metrics to a new template with a refresh, this is ok to close for now.


Circular dependency in let/subject results in preliminary exit

RSpec version 3.7
The following spec:
RSpec.describe 'circular dependency in `let`' do
  subject { a }
  let(:a) { b }
  let(:b) { 1 }

  it { is_expected.to eq 1 }

  context 'circular dependency' do
    let(:b) { a }

    it { is_expected.to eq 2 }
  end
end

results in:
circular dependency in `let`
  should eq 1
  circular dependency

as opposed to:
circular dependency in `let`
  should eq 1
  circular dependency
    should eq 2

Finished in 0.48681 seconds (files took 7.6 seconds to load)
2 examples, 0 failures

if you remove the circular dependency. Notice the missing footer in the first run.
Command return code is 0.
It would be great to add some warning message, and return status should be non-zero.

A self-referencing let is even easier to reproduce:
RSpec.describe 'circular dependency in `let`' do
  let(:a) { a }

  it { expect(a).to eq 1 }
end

Maybe it's too early for me but I can't figure out how  it { is_expected.to eq 2 } is a reasonable expected behaviour. Feels like 3.7 fixed a bug? Self-referencing let should be a stack overflow, right?

I get the same thing with your initial spec as well. I'm closing this since I can't repro, but please do re-open if you can provide more information!

Historically, self-referential let declarations have triggered a segfault due to bugs in MRI.  If @xaviershay is getting a SystemStackError that's a huge improvement and makes me think it got fixed in ruby.  @pirj if you're not on Ruby 2.5, give that a shot to see if that improves things for you.
Further reading:

#1673
#1787
https://bugs.ruby-lang.org/issues/9505

Works as expected with ruby 2.6.0 (2018-03-24 patchlevel -1), rspec 3.7.0, rspec-core 3.7.1:
$ ruby --version
ruby 2.6.0dev (2018-03-24) [x86_64-darwin17]
$ gem list rspec

*** LOCAL GEMS ***

rspec (3.7.0)
rspec-core (3.7.1)
rspec-expectations (3.7.0)
rspec-mocks (3.7.0)
rspec-support (3.7.1)

$ rspec 1_spec.rb
F
Failures:

  1) circular dependency in `let`
     Failure/Error: let(:a) { a }

     SystemStackError:
       stack level too deep
     # ./1_spec.rb:2:in `block (2 levels) in <top (required)>'
     # ./1_spec.rb:2:in `block (2 levels) in <top (required)>'
     # ./1_spec.rb:2:in `block (2 levels) in <top (required)>'
...
     # ./1_spec.rb:4:in `block (2 levels) in <top (required)>'

Top 1 slowest examples (0.00643 seconds, 63.1% of total time):
  circular dependency in `let`
    0.00643 seconds ./1_spec.rb:4

Finished in 0.01018 seconds (files took 0.16021 seconds to load)
1 example, 1 failure

Failed examples:

rspec ./1_spec.rb:4 # circular dependency in `let`

And actually re-tested with my current setup, but without spring and additional configuration, also worked fine.
So this seems to be something in the project code that is inducing this behavior. Sorry for bother, and thanks for the help! ❤


feat: supported multi-modal embeddings from Azure AI Vision service

/deploy-review

/deploy-review

/deploy-review


"alternative spelling" in Hebrew > bug or feature? how to turn off?

When I write  חת
i receive  results for either חט or הת. It is evident that QuickDic searches for similar sound:
ט sounds like ת
ה sounds like ח
This deature could be great IF the first spelling is not found.
But instead, the first results shown are the /alternative/ spellings, where in practice the original spelling should be shown first.
Consequently, I can't find the word at all !
type: חתול  --> not shown (it's a super popular word).
search in English:
type cat --> found: חתול, first result, no problems.

This would indicate that the ICU collation considers these two spellings equal.
Working around what ICU considers equal is a bit of a problem.
Though as a quick fix it might be possible to simply ensure that actually equal matches have priority over collation-equal matches, I'll have to check how easy that is to add.

If it doesn't, is it possible to compare with most basic, simple string match ?
(even as an option / settings)
Thank you!

Things are a bit more complicated than that, since there is the whole searching involved, which needs more than just equality check to be efficient.
However after some digging I am not sure that even is the problem.
It looks actually more like these words might not even exist properly in the dictionary.
I have the suspicion that somehow the dictionary only added the transliterations like khatúl to the index, not the word itself.
However this is rather hard to confirm without knowing the language. If you can check if you can find the words you are looking for by scrolling through the dictionary (or quicker, search for a word that should appear close to the dictionary and then scroll to find the word you actually look for) that might help quite a bit.
(and please quote the entry if you find it)

Well, it turned out to be the same issue as lots of similar reports with other issues.
Released 5.5.1 with new dictionaries which seems to fix this.


Any plans to provide tokenizers for llama3 or codellama ?

Can some share how to use and integrate code2prompt with LLM like llama3


[REQUEST] Add ability to search/filter

It would be great to add the ability to search stories based on keywords or filter based on topics. As the number of stories grow, it will be more challenging to reach chronologically older content, so having a way to surface those stories would be helpful.

Great suggestion! Search or a list of most commonly used tags would be fantastic. Thanks @TeresaP! I probably won't take this on immediately, in case someone wants to make this their first contribution to the project.

@all-contributors please add @TeresaP for ideas!

@juliaferraioli can i work on this issue?

@ApurvaBasule absolutely! There are at least two potential approaches:

add a link to https://www.opensourcestories.org/tags/ to the menu (good first step, probably not ideal)
customize a new list.html in the layouts/stories directory that displays the top n tags alongside the list of stories

Search would be a more advanced effort, but you're welcome to research that if you like!
Let me know if you have any questions.

Hey, I have worked on this issue and created a new PR as I thought it is pending
@juliaferraioli is it fine or against contribution guideline?

Hi @LuvvAggarwal, please see my previous comment as to the approach we're looking for.


Enable vtprotobuf

Blockers:

https://github.com/planetscale/vtprotobuf/pull/133
https://github.com/planetscale/vtprotobuf/pull/130

@howardjohn why do we need stable marshalling? is it for tests? I think Envoy currently supports dealing with reodered maps since https://github.com/envoyproxy/envoy/pull/30761

Ooh nice. Didn't know about that maybe we don't need it. maybe we don't
even need "strict" either then?
In the past (like 5 years ago) we had an issue around this and it was
pretty subtle and didn't break tests but had issues in real world so I
figured it still mattered.
On Mon, Mar 25, 2024, 12:41 PM sschepens @.***> wrote:

@howardjohn https://github.com/howardjohn why do we need stable
marshalling? is it for tests? I think Envoy currently supports dealing with
reodered maps since envoyproxy/envoy#30761
https://github.com/envoyproxy/envoy/pull/30761
—
Reply to this email directly, view it on GitHub
https://github.com/istio/istio/issues/49790#issuecomment-2018774001, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAEYGXKVMA54EQTZRWC4LWTY2B4WNAVCNFSM6AAAAABELGYLACVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMJYG43TIMBQGE
.
You are receiving this because you were mentioned.Message ID:
@.***>

FWIW java-control-plane does non-deterministic serialization (the default) and we've been using that internally for years without issues.

might be interesting to check if we really need Deterministic behaviour, maybe write some tests to see if envoy does what we think it does.

seems like since 2019 https://github.com/envoyproxy/envoy/pull/8231 envoy's been hashing deterministically even Any fields

One more blocker I think: https://github.com/planetscale/vtprotobuf/pull/137


Create groupecerco.txt

add my school
the official website:
https://groupecerco.com

@sverano First stage of verification passed, official domain is enseignement.gouv.ci as far as I verified. This is just an informational note. Pull request is still in the review. I appreciate your patience while waiting for the final decision. Proof of domain ownership: https://www.enseignement.gouv.ci/files/CERCO FORCES FINAL.pdf

@sverano I am sorry, this domain seems to represent an institution that is not eligible for adding to this repository, not a specific single university or a college. Please apply for free educational license with an official document which proves that you are a student. You can do it using "OFFICIAL DOCUMENT" tab at https://www.jetbrains.com/shop/eform/students.
I have to close the request without merging.
Thank you for your understanding!


dilemma on HTTPS_Proxy setting

After installed cf dev , try to "CF push MyApp -b java_buildpack" my java app which used to work for PCFDEV. I got error "Unable to find cached file for https://java-buildpack.cloudfoundry.org/jvmkill/trusty/x86_64/index.yml". This typically means I need to setup HTTPS_PROXY due to my company's network restrain. It was confirmed by running "curl https://java-buildpack.cloudfoundry.org/jvmkill/trusty/x86_64/index.yml". It timed out.
After setting up the HTTPS_PROXY env variable, I ran curl again, the file can be downloaded. But when I ran "cf push" again, it does not even start.
Attached both before and after logs for CF PUSH below.
Any idea?
Before Setting up HTTPS_PROXY
emailtowalter@C02WD2MEHV2H:~/mycode/cf/class-artifacts$ cf push myapp2 -p rest-data-service.jar
Pushing app myapp2 to org cfdev-org / space cfdev-space as admin...
Getting app info...
Creating app with these attributes...
+ name:       myapp2
  path:       /Users/emailtowalter/mycode/cf/class-artifacts/rest-data-service.jar
  routes:
+   myapp2.dev.cfdev.sh

Creating app myapp2...
Mapping routes...
Comparing local files to remote cache...
Packaging files to upload...
Uploading files...
 594.99 KiB / 594.99 KiB [==============================================================] 100.00% 1s

Waiting for API to complete processing files...

Staging app and tracing logs...
   Downloading binary_buildpack...
   Downloading python_buildpack...
   Downloading staticfile_buildpack...
   Downloading php_buildpack...
   Downloading nodejs_buildpack...
   Downloaded php_buildpack (592.8K)
   Downloading java_buildpack...
   Downloaded java_buildpack
   Downloading ruby_buildpack...
   Downloaded binary_buildpack (3.5M)
   Downloading dotnet_core_buildpack...
   Downloaded staticfile_buildpack (4.1M)
   Downloading go_buildpack...
   Downloaded nodejs_buildpack (3.8M)
   Downloaded python_buildpack (3.7M)
   Downloaded ruby_buildpack (4.5M)
   Downloaded go_buildpack (4.5M)
   Downloaded dotnet_core_buildpack (3.9M)
   Cell 43a13163-012b-4298-a1cd-8d6488967dfe creating container for instance 9255b0df-52a2-402a-a1b6-95698a6d1078
   Cell 43a13163-012b-4298-a1cd-8d6488967dfe successfully created container for instance 9255b0df-52a2-402a-a1b6-95698a6d1078
   Downloading app package...
   Downloaded app package (28M)
   [DownloadCache]                  WARN  Unable to download https://java-buildpack.cloudfoundry.org/jvmkill/trusty/x86_64/index.yml into cache /tmp: execution expired
   [Buildpack]                      ERROR Detect failed with exception #<RuntimeError: Jvmkill Agent error: Unable to find cached file for https://java-buildpack.cloudfoundry.org/jvmkill/trusty/x86_64/index.yml>
   Jvmkill Agent error: Unable to find cached file for https://java-buildpack.cloudfoundry.org/jvmkill/trusty/x86_64/index.yml
   None of the buildpacks detected a compatible application
   Exit status 222
   Cell 43a13163-012b-4298-a1cd-8d6488967dfe stopping instance 9255b0df-52a2-402a-a1b6-95698a6d1078
   Cell 43a13163-012b-4298-a1cd-8d6488967dfe destroying container for instance 9255b0df-52a2-402a-a1b6-95698a6d1078
   Cell 43a13163-012b-4298-a1cd-8d6488967dfe successfully destroyed container for instance 9255b0df-52a2-402a-a1b6-95698a6d1078
Error staging application: An app was not successfully detected by any available buildpack

TIP: Use 'cf buildpacks' to see a list of supported buildpacks.
FAILED

After HTTPS_PROXY setup:
Unexpected Response
Response code: 503
CC code:       0
CC error code:
Description:
<TITLE>Network Error</TITLE>

<big></big>

<big>Network Error (tcp_error)</big>

A communication error occurred: "Operation timed out"

The Web Server may be down, too busy, or experiencing other problems preventing it from responding to requests. You may wish to try again at a later time.

For assistance, contact your network support team.

FAILED

Hi @emailtowalter, simply setting HTTPS_PROXY isn't enough to get CF Dev to respect it for all of its internal network interactions. You need to start the VM with those settings. For example:
$ cf dev stop
# Set HTTPS_PROXY
$ cf dev start
...
$ cf push myApp -b java_buildpack

Please let us know if this addresses your problem.

Thanks for getting back to me. I added HTTPS_PROXY and HTTP_PROXY setting in the bash_profile, and did stop-then-start. Then all sorts of weird thing happens.
First, CF dev start seems to work, although the output log is different from without these settings. But then, I cannot login. Tried to "curl https://api.dev.cfdev.sh/v2/info", it timed out. In order to login , I have to unset these settings. Then the login seems to be fine.
But now, cf apps told me I am not logged in.
Here is the log:
"
emailtowalter@C02WD2MEHV2H:~$ cf dev start
Downloading Network Helper...
Installing cfdevd network helper...00.0%
Installing networking components (requires root privileges)
Password:
Setting up IP aliases for the BOSH Director & CF Router (requires administrator privileges)
Downloading Resources...
WARNING: CF Dev requires 8192 MB of RAM to run. This machine may not have enough free RAM.
Creating the VM...
Starting VPNKit...
Starting the VM...
Waiting for Garden...
Deploying the BOSH Director...
Deploying CF...
Uploading ReleasesDeploying Mysql...
Done (7m3s)
  ██████╗███████╗██████╗ ███████╗██╗   ██╗
 ██╔════╝██╔════╝██╔══██╗██╔════╝██║   ██║
 ██║     █████╗  ██║  ██║█████╗  ██║   ██║
 ██║     ██╔══╝  ██║  ██║██╔══╝  ╚██╗ ██╔╝
 ╚██████╗██║     ██████╔╝███████╗ ╚████╔╝
  ╚═════╝╚═╝     ╚═════╝ ╚══════╝  ╚═══╝
             is now running!

To begin using CF Dev, please run:
    cf login -a https://api.dev.cfdev.sh --skip-ssl-validation

Admin user => Email: admin / Password: admin
Regular user => Email: user / Password: pass

------------------------------------------------------------------------------------------- 08:30:23
emailtowalter@C02WD2MEHV2H:~$ cf login -a https://api.dev.cfdev.sh --skip-ssl-validation
API endpoint: https://api.dev.cfdev.sh
FAILED
Server error, status code: 503, error code: 0, message:
------------------------------------------------------------------------------------------- 08:33:26
emailtowalter@C02WD2MEHV2H:~$ curl https://api.dev.cfdev.sh/v2/info
^C
------------------------------------------------------------------------------------------- 08:33:50
emailtowalter@C02WD2MEHV2H:~$ ping https://api.dev.cfdev.sh
PING https://api.dev.cfdev.sh (<IP_ADDRESS>): 56 data bytes
64 bytes from <IP_ADDRESS>: icmp_seq=0 ttl=64 time=0.498 ms
64 bytes from <IP_ADDRESS>: icmp_seq=1 ttl=64 time=0.506 ms
64 bytes from <IP_ADDRESS>: icmp_seq=2 ttl=64 time=0.554 ms
64 bytes from <IP_ADDRESS>: icmp_seq=3 ttl=64 time=0.372 ms
^C
--- https://api.dev.cfdev.sh ping statistics ---
4 packets transmitted, 4 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.372/0.483/0.554/0.067 ms
"
......
emailtowalter@C02WD2MEHV2H:~$ cf login -a https://api.dev.cfdev.sh --skip-ssl-validation
API endpoint: https://api.dev.cfdev.sh
API endpoint:   https://api.dev.cfdev.sh (API version: 2.124.0)
Not logged in. Use 'cf login' to log in.
FAILED
Server error, status code: 503, error code: , message:
------------------------------------------------------------------------------------------- 08:39:01
emailtowalter@C02WD2MEHV2H:~$ unset NO_PROXY
------------------------------------------------------------------------------------------- 08:39:40
emailtowalter@C02WD2MEHV2H:~$ unset HTTPS_PROXY
------------------------------------------------------------------------------------------- 08:39:54
emailtowalter@C02WD2MEHV2H:~$ unset HTTP_PROXY
------------------------------------------------------------------------------------------- 08:40:00
emailtowalter@C02WD2MEHV2H:~$ env
TERM_PROGRAM=Apple_Terminal
SHELL=/bin/bash
TERM=xterm-256color
TMPDIR=/var/folders/bb/gmfzfx7n1rvgqvl22hgjzjjh0000gn/T/
Apple_PubSub_Socket_Render=/private/tmp/com.apple.launchd.tqWolvzacn/Render
TERM_PROGRAM_VERSION=404
TERM_SESSION_ID=F07AF9A0-2583-467A-A1C4-7932D6161EB5
USER=emailtowalter
SSH_AUTH_SOCK=/private/tmp/com.apple.launchd.FvAKztXrZt/Listeners
PATH=/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin
PWD=/Users/emailtowalter
LANG=en_CA.UTF-8
XPC_FLAGS=0x0
XPC_SERVICE_NAME=0
SHLVL=1
HOME=/Users/emailtowalter
LOGNAME=emailtowalter
SECURITYSESSIONID=186a7
_=/usr/bin/env
------------------------------------------------------------------------------------------- 08:40:01
emailtowalter@C02WD2MEHV2H:~$ cf login -a https://api.dev.cfdev.sh --skip-ssl-validation
API endpoint: https://api.dev.cfdev.sh
Email> admin
Password>
Authenticating...
OK
Select an org (or press enter to skip):

cfdev-org
system

Org> 1
Targeted org cfdev-org
Targeted space cfdev-space
API endpoint:   https://api.dev.cfdev.sh (API version: 2.124.0)
User:           admin
Org:            cfdev-org
Space:          cfdev-space
------------------------------------------------------------------------------------------- 08:40:16
emailtowalter@C02WD2MEHV2H:~$ cf apps
FAILED
No API endpoint set. Use 'cf login' or 'cf api' to target an endpoint.
------------------------------------------------------------------------------------------- 08:40:20
emailtowalter@C02WD2MEHV2H:~$ cf apps
FAILED
No API endpoint set. Use 'cf login' or 'cf api' to target an endpoint.
------------------------------------------------------------------------------------------- 08:40:33
emailtowalter@C02WD2MEHV2H:~$ cf domains
FAILED
No API endpoint set. Use 'cf login' or 'cf api' to target an endpoint.
------------------------------------------------------------------------------------------- 08:40:51
emailtowalter@C02WD2MEHV2H:~$

Firstly, just wanted to give a major thanks for giving our product a thorough shakedown! Though we're sad that you're running into issues, we will always be thrilled to help.

Addressing some of the major points you've mentioned:

CF dev start seems to work, although the output log is different from without these settings

Yep, that's unfortunately true. This is due to a bug in the reporting logic, and should be resolved in our next release. However, it only affects the output log and not the underlying processes.

But then, I cannot login. Tried to "curl https://api.dev.cfdev.sh/v2/info", it timed out. In order to login , I have to unset these settings.

This is perhaps a lapse in documentation.  api.dev.cfdev.sh resolves to a private IP that exists on your machine. Thus trying to interact with the domain through a remote proxy will ultimately fail, since the proxy server will not have access. Since the _PROXY env vars were added to your bash_profile, one elegant solution is to add the following to your bash_profile as well: export NO_PROXY='.dev.cfdev.sh', which will exclude http requests to your local CF Dev from going through the remote proxy.

but then my "cf push" to fail with certificate issue

Certainly peculiar. Looks like the CF Dev VM was stood up correctly since you can now access that url and get this far. I just tested this proxy scenario with a java application and things worked well, on my machine. It leads me believe that there might be something off about the proxy itself and how it is handling these certificates.
Could you please enter the trouble url (https://java-buildpack.cloudfoundry.org/jvmkill/trusty/x86_64/index.yml) in a web browser, like I've done below, and report whether you see similar certificate errors? If so, you might have to talk to your system administrator about a resolution here.

Appreciate your response!  I can access the build pack URL in browser. But the root cert and intermediate cert are different with a revocation check cert after. I believe this is a security thing that setup in my company’s network to check the revocation of the cert. I recall I had to add my company’s root cert in java cert store for ssl handshaking work.
Thinking it might be a similar case here by adding the cert to stem cell?
Mind to let me know how to do so?
Walter

On Dec 9, 2018, at 2:37 PM, Anthony Emengo<EMAIL_ADDRESS>wrote:
Firstly, just wanted to give a major thanks for giving our product a thorough shakedown! Though we're sad that you're running into issues, we will always be thrilled to help.
Addressing some of the major points you've mentioned:
CF dev start seems to work, although the output log is different from without these settings
Yep, that's unfortunately true. This is due to a bug in the reporting logic, and should be resolved in our next release. However, it only affects the output log and not the underlying processes.
But then, I cannot login. Tried to "curl https://api.dev.cfdev.sh/v2/info", it timed out. In order to login , I have to unset these settings.
This is perhaps a lapse in documentation. api.dev.cfdev.sh resolves to a private IP that exists on your machine. Thus trying to interact with the domain through a remote proxy will ultimately fail, since the proxy server will not have access. Since the _PROXY env vars were added to your bash_profile, one elegant solution is to add the following to your bash_profile as well: export NO_PROXY='.dev.cfdev.sh', which will exclude http requests to your local CF Dev from going through the remote proxy.
but then my "cf push" to fail with certificate issue
Certainly peculiar. Looks like the CF Dev VM was stood up correctly since you can now access that url and get this far. I just tested this proxy scenario with a java application and things worked well, on my machine. It leads me believe that there might be something off about the proxy itself and how it is handling these certificates.
Could you please enter the trouble url (https://java-buildpack.cloudfoundry.org/jvmkill/trusty/x86_64/index.yml) in a web browser, like I've done below, and report whether you see similar certificate errors? If so, you might have to talk to your system administrator about a resolution here.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

@emailtowalter If you'd like to append a certificate to the certificate store inside of the application container, the canonical way to modify the cflinuxfs2-rootfs.trusted_certs property of the cflinuxfs2-rootfs-setup job in the Cloud Foundry manifest. You could perform the operation in the following way:
$ eval "$(cf dev bosh env)"
$ bosh -d cf manifest > cf.yml
# edit cf.yml with the appropriate configuratoin
$ bosh -d cf deploy cf.yml

For more information you can look here: https://docs.cloudfoundry.org/running/trusted-system-certificates.html. If you'd like something a little more seamless, then perhaps the folks at the java buildpack might have alternative solutions.

Just to give a update on this issue:  I created a java offline buildpack, and cf create-builldpack, then use it for CF push, it worked. But this is just a workaround.
Next step is to try to add the CA into bosh director mentioned on top to resolve it once for all.
Note: I still have to disable the HTTPS_PROXY after CF DEV START to make CLI to work. Hope someone to track it.

When I try "bosh -d cf deploy cf.yml", I got a error. Any suggestion?
Using deployment 'cf'
Creating and uploading releases:

Uploading release 'cf-mysql/36.15.0':
stat /var/vcap/cache/releases/cf-mysql-36.15.0-97.18.tgz: no such file or directory
Uploading release 'cf-networking/2.17.0':
stat /var/vcap/cache/releases/cf-networking-2.17.0-97.18.tgz: no such file or directory
Uploading release 'bpm/0.13.0':
stat /var/vcap/cache/releases/bpm-0.13.0-97.18.tgz: no such file or directory
Uploading release 'capi/1.70.0':
stat /var/vcap/cache/releases/capi-1.70.0-97.18.tgz: no such file or directory
Uploading release 'binary-buildpack/1.0.27':
stat /var/vcap/cache/releases/binary-buildpack-1.0.27-97.18.tgz: no such file or directory

Exit code 1

@emailtowalter Apologies, this is due to a lapse in instruction. Since you are redeploying from outside the VM, the manifest must be edited further. Please open cf.yml in a text editor, navigated to the releases: block and delete the url: key-value from every release object. Then try again.

Closing due to inactivity, please respond if you're still running into issues!


⚠ Bofrost.fr has degraded performance

In 7024568, Bofrost.fr (https://www.bofrost.fr) experienced degraded performance:

HTTP code: 200
Response time: 1264 ms

Resolved: Bofrost.fr performance has improved in 85b021c after 8 minutes.


how to generate data from an audio file .

When we convert .png to .json it give us some samples array like the following link..
https://w1.sndcdn.com/PP3Eb34ToNki_m.json
My question is that how i can achieve same data from an audio file.

When we convert .png to .json it give us some samples array like the following link.. https://w1.sndcdn.com/PP3Eb34ToNki_m.json My question is that how i can achieve same data from an audio file.

bro, did u find the solution for generating data from an audio file Uri?


Rewind: Efficient rewinding

Summary
We should implement a more efficient way to rewind accounts. Currently to do it we need to query the latest account state and then fetch preceding transactions to undo the changes. This is slow and it doesn't support rewinding some information such as participation keys.
Here are two possible ways to approach this problem.

Snapshot account state every x rounds. Rewinding would pick the nearest snapshot below the given round and apply the transactions between the snapshot round and the given round.
Maintain an array (db table) with the following format: [(round, new balance)] with elements sorted by the round. For every round a balance (or other info) changes, we add an entry to the array. Querying a balance before some round then requires one binary search.

Scope/Requirements
Need to change db schema.
Urgency/Relative Priority
Not urgent but nice to have.

Rewinding is currently the only way I am aware of to have a snapshot of the balances of a set of accounts at a given time.
If you did not have rewinding, you could not do it.
Concrete example: you own accounts A and B.
You want to know your total amount of Algos.
The only way I currently know how to do it is first query the balance for A and then query the balance for B with the same round as the response for A.
If you do not do that and naively query the balance for both accounts, you may end up arbitrary increase your total amount of Algos if a transaction from A to B happened between the two calls.
Another use case is being able to understand what happened in the past.

Hi, I would like to upvote efficient rewind..
Use case:
in Vote Coin we calculate each account voting power according to the block in which is defined end of the voting session. There is currently no efficient way of getting simple algo/asset balance at specific round at specific account (or list of accounts)
We do it at the moment by using https://algoindexer.algoexplorerapi.io/v2/assets/452399768/balances?round=18988231 which is not very efficient especially if there is many owners of the assets.
https://forum.algorand.org/t/rewinding-past-txn-type-appl-is-not-currently-supported/5930
see vote coin specifications https://www.vote-coin.com/specifications/introduction

i am trying to extend the Vote Coin solution to be usable also for Algo
is there ANY way how can i get the Algo balance of the account BITSCVQHPCD7XAIYYEA5B3F3PWC57IRSIUIWPZV6JXFWUVMO7WQCXSP7NE at round 16024568 ?

Is there any update about this issue?


[feature-request] allow custom transients/commands

(I'm sure it's not being called transients in gitu but I hope you understand it nevertheless)
It would be great to be able to add commands as new transient window or add subcommands to existing transients.
For example I have this snippet configured in my emacs config for magit. It adds an s key to the l (log) transient and executes dired. Essentially by using this I can just enter ls in the magit main buffer and get a directory listing - while staying in magit.
Another thing would be to not add new commands but modify existing ones, e.g. the look of the git log output.

Related: https://github.com/altsem/gitu/issues/61


Fine-tuning on heuristics related with the issue#3568 (C/C++ header files)

Hi there,
I have done some improvements on disambiguation for ".h" files trying to solve this issue.
I also moved one file from "samples/C++/Field.h" to "samples/C/Field.h", if you analyse this file isolated, it is more like an C source file or at least it is an inconclusive sample.
I think that cases like the old "sample/C++/Field.h" should be solved with a mechanism like a white list but I do not know the Linguist source code well enough to know if it would be really possible.

@rafael-santiago Thanks for your work on this!
I'm a bit worried about the impact of these changes on performance. Could we evaluate that on a corpus of .h C files?

@pchaigno I have just added 5 smaller regexes. I am afraid of doing them all (even combined with non-capturing groups) because it could stress the ruby regex system. I have no idea how internally it is implemented in ruby, however in other Languages the suggestion is execute smaller regexes to do the whole search. Considering the short circuit used on if-branches (and operator) it could avoid this overhead because it would process only the essential for that specific data. Well, who knows...

@rafael-santiago It's generally faster to allocate a single, fixed-size unit of memory than several. That being said, it's easier for a single, large expression to be optimised in one compilation unit.
Ruby's regex engine is already tightly-wired for performance. You shouldn't have to worry about what goes on under the hood - this isn't C or assembly. Dynamic programming languages exist to take care of heavy lifting.

@pchaigno, @Alhadis
Okay, I used your suggested code snippet. Running it twice (time rake):
1st:
real 1m25.195s
user 1m20.960s
sys 0m1.836s
2nd:
real 1m21.810s
user 1m17.852s
sys 0m1.772s
The original PR code (C, Assembly old approach):
1st:
real 1m21.718s
user 1m17.632s
sys 0m1.796s
2nd:
real 1m20.875s
user 1m16.572s
sys 0m2.108s
Of course that we should run it several times to get a good average but I really do not have time for this.
@Alhadis  I have pushed your snippet (now this is breaking with "pqiv.h"), if you want to solve and use it feel free.

@Alhadis Does your script to retrieve sample files on github.com still works with the new page layout? I'd like to check the performance impact on a "representative" corpus of files.

If it did, I'd be damn surprised. This was exactly what I expected to see when I tried:

I'll need to rewrite it, I'm afraid. This time it might be better to run it within a headless browser, which will make running the script much easier.
GitHub's still limiting our filetype-searches to logged-in users only, so it's gonna be interesting dealing with 2FA...

Hm, thanks. I'll see if I can fix @larsbrinkhoff's.

@pchaigno I fixed the script. I have this aggravating tendency to assume problems are 10 times more complicated than they really are, because I'm too distracted by something else to actually bother investigating. I drive myself nuts like that.

I also moved one file from "samples/C++/Field.h" to "samples/C/Field.h", if you analyse this file isolated, it is more like an C source file or at least it is an inconclusive sample.

Could you revert this? If it is inconclusive, then I'd rather we don't change its classification for now.

I've evaluated the performance impact on a corpus of 11,523 .h files downloaded from the following search queries using Harvester:

extension:h
extension:h+-language:C
extension:h+-language:Objective-C
extension:h+-language:C++

I ran Linguist 10 times on a repository consisting of all these files (including duplicates), both with and without the present changes, as currently implemented.
In average, without this changes, Linguist takes 24s to run on the repository, whereas it takes 30s to run with them. The standard deviation over the 10 runs is low (< 1s). Removing the fallback case on C is likely to impact performance, so I'll have to run the test again.
So that's 25% longer. But. This is a huge number of .h files for a single repository (Linux has only 4 times as much). The overhead is likely to be much smaller for a normal size repository (especially since we update statistics incrementally) as the time to run heuristic rules would be a smaller portion of the total time and only that time is affected.
I'm flagging this as stale. @rafael-santiago Please tell us if you still intent to work on this.


[Launcher]: Best effort to resolve external IP and Host for local machine

Coverage decreased (-11.4%) to 50.344% when pulling b96f08b49237e08647c6e528ba75f8c3d1098fe0 on yqwang/launcher-dev into 26117df72b536a15df9a0ff49e3ee4870b3634cb on master.

Coverage decreased (-11.4%) to 50.344% when pulling b96f08b49237e08647c6e528ba75f8c3d1098fe0 on yqwang/launcher-dev into 26117df72b536a15df9a0ff49e3ee4870b3634cb on master.

Coverage decreased (-11.4%) to 50.344% when pulling b96f08b49237e08647c6e528ba75f8c3d1098fe0 on yqwang/launcher-dev into 26117df72b536a15df9a0ff49e3ee4870b3634cb on master.


System Compatibility: WFRP4 Fantasy roleplay

Hello again :D
I wonder if it would be possible to create system compatibility with Warhammer fantasy roleplay.
The system works with success levels instead of rolling a die. I spoke to the creator and he says skills rolls in the systems are items and not data properties ( if that says anything to you)
For example, if I have a thief I can use his picklock skills and I have to roll under the target number to succeed

Based on the success level (the difference between the roll and the target I will be able to open a lock or not)

There is also the expanded test, but this may be too complicated to implement. I was wondering if you could at least implement it for a success or fail option.

I will look into it and see what i can do

I just had a look at the rules for WFRP4 checks and would like to confirm with you, that i understood them correctly:

A lock could have difficulty classes with an associated value (e.g. Very easy +60 or Very hard -30)
When a player tries to pick this lock they would add their skill/attribute to this value (dex?) to get the target value and roll a d100
If the player has rolled below or equal to this target they succeed, otherwise they fail
If they player rolls a 1 they auto succeed, if they roll a 100 they auto fail
If these are they rules i can easily implement them, though locks would require the difficulty in numerical form

Yeah, that is exactly the mechanics, awesome and thx you for taking the time to check it.
It can be a bit more complicated but that you wrote it would be more than enough. If you want to get the full rules there are other things into consideration.
When you pass the role you get Success Levels SL. so for example you rolled a 20 in a target you get an SL of 3
There is also crit or fumble on doubles.
Let's say that you have a role to pass at 45, then 11, 22, 33, 44 would be crit, and the 55, 66, 77, 88, and 99 would be a fumble. I guess fumbles would be like the lock breaking or getting jammed
How the locksmith works here is the rulebook on that skill

But hey just with just that you wrote it would be super amazing like that that would be more than amazing. If you want to check. I have a thief in our game so any implementation it would be super awesome to get more for my players, so I super appreciate any effort or any way you could implement it

I have added the crit system with v1.10.7. The formula for lock picking to work is 1d100 - @actor.characteristics.dex.value. I have split it into two crit systems, one with doubles as crits, one without. I have not yet implemented anything regarding Success Levels since i was not sure how exactly these would interact with crits and how exactly they differ in regards to the success (i could however add something where they influence the successes added to the door locking and crits double the SL). The doors dc is  equivalent to the systems numerical lock difficulty (remember to never use -1 since that gets translated into infinity). I will leave this issue open for now, incase there are any questions, bugs or further feature requests.

I am a bit confused, what do you mean "The doors dc is equivalent to the systems numerical lock difficulty"

If you want the door to have a difficulty of e.g. Hard you set the DC to -20

oh ok, that makes sense. I give it a try and I let you know. Thank you so much for doing this!

Can I bother you with another question? I am trying to use lockpicks as an item to try to open doors but is not working, I am probably doing something wrong

This is the item and I have this as the settings but is not working

and this is the door

I know I am doing something wrong but I don't know what is it

btw, the rolling seems to be working. Is rolling dex instead of the skill in itself which is the lockpick skill which is based on dex but it has more points. for example
my thief dex is 32

but he allocates some points to his picklock skills which is 32 + 5 since he has added points to it

so it should be running the skill instead of the raw dex.
How did you find out the formula for the roll?
I hope I am not bothering you too much :)

A nother question regarding your problem with lock picking, were you logged in as a player or as GM when you tested lock picking? In case you were logged in as a player, was a  GM account logged in at the same time?

Yes, I was logged in as the GM and in another browser and another account I was using the player.

Were there any text popups above the door/do you have the text popups enabled?

this is when I use alt + right-click, to try to break the lock, now that I see the lock pick is working as intended :(, sorry for the confusion. It actually works when I use the lock picks using shift

No problem. I have not yet included a formula for lock breaking since i was not sure what the appropiate skill/attribute is in WFRP4, assuming it is a strength check the roll formula 1d100 - @actor.characteristics.str.value should probably work

ah ok, yeah, skills seems to be different, but I can use the current dex. If you have any possibility to get the skills working that would be amazing. Thx you for the hard work

how do you know which value is?
like how do you know is @actor.charactersitcs

when you have a token selected a you can open the console(F12) and enter canvas.tokens.controlled[0].actor, that should give you an overview over all possible attribute paths available in the roll formulas

ok, I will try that. Seems it was the s and is working but I don't understand the roll. Could you explain it for me
I have the formula and I added the modifer of -30 to break
For example in this one I got a -26

and in this one I got a 40

but I still don't understand how it comes to this numbers

In this specific casees Lock and Key will do the following:

A d100 is rolled resulting in a 4(case 1) or a 70(case 2)
It will subtract the 30 from the results
It will compare these results to the Locks DC (where result <= Lock DC is a success)

OK, do you think it would be possible to see the roll result of the d100?

You can already see the specific dice result when you click on the result itself:

Oh, that was useful! Thx you so much!

I have just released v1.11.5 wich adds the option d10 crit limit to the token settings


Content: Orthographic correction of spanish documentation.

Adding some words and correcting some Spanish grammar.

[x ] I have read freeCodeCamp's contribution guidelines.
[ x] My pull request has a descriptive title (not a vague title like Update index.md)
[x ] My pull request targets the master branch of freeCodeCamp.
[x ] None of my changes are plagiarized from another source without proper attribution.
[x ] My article does not contain shortened URLs or affiliate links.

If your pull request closes a GitHub issue, replace the XXXXX below with the issue number.
Closes #XXXXX

Hi @edgamen
As there has been no response from your end for a while, we are closing this PR.
Feel free to request this PR to be reopened or create a new PR, if you still wish to make the changes you originally added plus any additional change requests asked of you.  Also, remember to update your fork, so you have the latest version of master in case it has changed significantly.
Happy Coding!


Add .copier-answers.yml to generated .gitignore

I just made a new project with Copier. prettier immediately failed in CI because the value of the project_short_description field in .copier-answers.yml did not start on a new line.
I think this file should just be ignored by git, right?

I think it should be ignored by linters. I think you might want it in git if you use it to update the project via copier's update feature.

Does pre-commit respect generated files attributes in .gitattributes?

thanks for the fix!


Forgiving Tit For Tat

I've added a Forgiving Tit For Tat strategy and I've written unittests for it.

Nice! Keep them coming :) :+1:


ruRU locale

Hi. I translated most of it.
ruRU.txt
Rename extension to .lua

Added, THANK YOU! <3


Complete Share::FetchFolder

FetchFolder
All data is stored within Folders. Where a Folder may contain an arbitrary number of sub Folders or Objects. With this request, it is possible to read out the meta information stored for a Circle. If no Circle is provided, then it is considered an error and a Warning will be returned.
If a Circle is provided, and the Member requesting the information has a relation to this Circle, then the information is returned from the requested Folder. If no Folder information is given, the root Folder is read.
Note, that the request only returns meta information, i.e. simple Id's & names and ObjectTypes of data stored. To properly map over the ObjectTypes, please invoke the fetchObjectTypes request.

Changes in the specifications have removed the need for this feature.


DTS-HD Master Audio streams are detected as their lossy DTS core

When checking a new remux from a BluRay Audio with the newest version, I noticed that DTS XLL streams are detected as the core lossy DTS 5.1.
On v21.09 the correct information is shown.
On v23.04, the DTS-HD MA stream is the size it should be but the format, bitrate mode, bitrate, sampling rate, frame rate and compression are the core's information.
v21.09

v23.04

We added some checks for avoiding some false-positive, looks like that we have now some false-negative.
Please share a sample file (small, need 1 frame, can be shared by email if the file can not be publicly shared).


IMenu functions returning incorrect flags

The flag values returned by these functions don't match the respective function names, but are shuffled around. Probably not a big deal since those don't sound like the most commonly used functions, but that doesn't look intentional.

Thanks. Feel free to submit a PR. It may make sense to do it in an upstream fork line po3s and we'll get the fix when we sync.

Done
https://github.com/powerof3/CommonLibSSE/pull/109

Awesome thanks. I'll see about syncing again with po3.

Actually, let's leave it open until I've done the sync else I'll forget.

This should be all in for VR/NG. Thanks!


Expression indentation syntax error

I ran into this syntax error a lot today trying to do the Advent of Code
Mostly when adding commenting out the next line

@Anton-4 Could you (or someone else) please add such a test? This looks like a great enhancement.

As an update from my end. I attempted to add a test, but got stuck and could not figure it out.

@TimWhiting Would you like assistance adding such a test yourself?

@TimWhiting Would you like assistance adding such a test yourself?

Sure, if you want, I can make some time next week to pair program a test for this.

Nice! Let me know when you're available.

I was able to figure out the test & rebase on the latest main. Moved these changes to my fork, and opened a new PR #4131


Annealed Temperature in Gumbel Softmax

It doesn’t look like the temperature is annealed in your gumbel softmax. Is there a reason for this as it is not standard? @tkipf

Mostly for simplicity and because it worked well without. Adding annealing would add the extra annealing hyperparameters to tune. If you add and see any noticeable improvement please let us know!


Doesn't work fine on mobile browsers

If I zoom to the input field and click then, the position of the picker is too low, so that I can't see it on screen.
And after I zoomed one time and zoomed out again and I click on the input field, the picker hides, straight after it appeared after my click.

Additional info:
I tested this with Chrome on Android M.

+1

+1

+1


TraCI connection issues, netedit segfault on macOS Catalina

If running smarts gives you,
core/smarts.py", line 284, in setup
    provider_state = self._setup_providers(self._scenario)
  File "/Users/julian/src/SMARTS/smarts/core/smarts.py", line 550, in _setup_providers
    provider_state.merge(provider.setup(scenario))
  File "/Users/julian/src/SMARTS/smarts/core/sumo_traffic_simulation.py", line 199, in setup
    self._initialize_traci_conn()
  File "/Users/julian/src/SMARTS/smarts/core/sumo_traffic_simulation.py", line 123, in _initialize_traci_conn
    waitBetweenRetries=0.05,
  File "/usr/local/opt/sumo/share/sumo/tools/traci/main.py", line 121, in connect
    raise TraCIException("TraCI server already finished")
traci.exceptions.TraCIException: TraCI server already finished

or running netedit gives you something like,
[1]    1757 segmentation fault  netedit

and you're on macOS Catalina or newer then follow these steps to fix it https://github.com/eclipse/sumo/issues/6242#issuecomment-553458710.

Just posting this here in case others run into this issue.


Insert Default - Inserting ActiveModel with all attributes NotSet

PR Info

Resolve #420
Depends on

SeaQL/sea-query#223
#430

Oh I forgot to merge the sea-query PR

I don't think this feature is ready. I reverted the commit for now. Please revisit later.


Bump lock file

:tada: This PR is included in version 3.3.0 :tada:
The release is available on:

GitHub release
npm package (@latest dist-tag)

Your semantic-release bot :package::rocket:


Lazyloaded Picture tag doesn't work on IE.

Hi,
Because the picture tag isn't supported on IE 11, lazyloaded images always fallback to the default img.
This demo isn't working for me on IE11.
I am trying to make it work with picturefill.js but without success at the moment.
Is there a solution?

Hi @SamuelBourdon,
what you're trying to do is correct, since LazyLoad is not a polyfill, Picturefill must be used to have picture supported on IE.
If you post some code you're using maybe I can help.

Here is an example on codePen http://codepen.io/samuelbourdon/pen/rywLQW
It works for pictures loaded on page load but not for the lazyloaded ones.
I think I need to callback picturefill() before the image is loaded, callback_set seems to be the right solution but I do not know how to use it.

I have the same problem and I would like to find a solution.
I use picture tags and I'm not sure how I can use polyfill and lazyload. Polyfill requires srcset but if I set lazyload with srcset the lazy loading doesn't work.
Thank you in advance.

@dxit-gg
I updated my pen with a nearly working example using callback_set method. It seems to be ok on IE on page load but image swap failed sometimes when resizing
            (function() {                 var lazyload = new LazyLoad({                     callback_set: function () {                         picturefill({ reevaluate: true });                     }                 });             }());

Sorry guys I've been very busy with my actual job and beyond that preparing the talk for CSS Day 2017. I look forward to be back starting next Monday.

Maybe we just need another event on source change

The event on source change is the one that you're using, callback_set
I'm closing this, if you need further help please reopen.

Psst: have you already starred this repo? All you need to do is to press the Star ★ button on the top right corner of the website. ;)


Updated various notebooks to use opts

User guides

[x] 04 - Style Mapping
[x] 05 - Building Composite objects
[x] 06 - Live Data
[x] 07 - Tabular Data
[x] 08 - Gridded Data
[x] 09 - Indexing and Selecting
[x] 10 - Transforming Elements
[x] 12 - Custom Interactivity
[x] 13 - Data Pipelines
[x] 14 - Large Data
[x] 15 - Streaming Data
[x] 16 - Dashboards
[x] Network Graphs
[x] Plotting with Bokeh
[x] Plotting with Matplotlib
[x] Colormaps
[x] Continuous Coordinates
[x] Geometry data
[x] Styling Plots
[x] Linking Plots

@philippjfr Thanks for updating all these!
I'm happy to see all the magics removed. Most of my comments are about following the guidelines we are recommending people use when using options. In the customization user guide we state that we are following these guidelines so my comments are trying to make sure that is true...

Thanks for updating these guides and implementing the suggestions. There may always be some minor tweaks in subsequent PRs but the main thing is to get rid of the magics. Merging.


There is no matching data in this policy in this file

When I upload a file and then go to the 'Process Group' menu from the 'Files' drop-down menu, I get to see all my groups. When I select any one policy for that particular group say 'TOUT' and click on the 'Process' button, right next to it, I get a list of all the completed requests. When I then click on 'Show Graph' button for any policy in my list, I get the summary as "There is no matching data in this policy in this file". I have also attached a screenshot of the above-mentioned error in the file below.

This seems to happen for all the policies. So, is it my mistake in entering the rules for the policies or is it due to some other error?

Do your group actually have files? This happens when you have a group but the file doesn't exist in the server.

Yeah, I do have files in my group on the server. Is it possible for you to let me know the steps of uploading the files in the group as I think, I might be missing some step while uploading the files and I feel, it's just that error which is preventing me from moving forward.

following for docs purposes

Currently, there are three different ways of making group and it is very likely that some of them are not working very well.
So these are different ways to make a group.

Upload bunch of file and not selecting any group and select "new group" from drop down list. It will create a group when you upload multiple files at the same time.
Create a group on the file list page (top right side)
Create a group after go to file group page and do a file search and create a group based on the group that you searched.

I always go #2. Then, I will go to file edit page. (go to group process page and click the group name) and add files one by one.
Please let me know if you try this way and still doesn't work for you. I will also check the different ways to make a group and make sure it works or I will fix them.

Okay. Thanks for the above solution.
I followed your steps, but when I navigate to "Files->List Files" and when I upload a new file, by clicking on "choose files" option, I see the uploading progress at the bottom left side of the screen but I do not see anything on the "file upload progress bar".

The file after it gets uploaded 100%, I am not able to see where that file is. When I head over to "Files->Files Process Status" or to "Files->Process File", I do not see any of my files there. I have attached the screenshots of both the cases below.

Any idea on what the error might be or where those files are being saved?

It is very strange. Sometimes javascript doesn't load completely some reasons and file upload uses javascript files. Can you run ./frontend.sh and quickstartdev.sh and see it fixes it? Thank you!

You can go to developer tool on chrome. (usually 3 horizontal dots on the right side of the panel. and if you clicks that you will see "more tools" and you can select, developer tools. After you open the developer tools, you can select "console", and you can navigate through page and if you get any javascript error, especially some javascript file is not found, then, it is javascript loading error. If it is the case, please do above (run all ./frontend.sh, and quickstartdev.sh). Please let me know if it helps. Thank you!

It seems to be working now. I guess there was this JavaScript error because after running the ./frontend.sh, and quickstartdev.sh, it's working. So I guess the JavaScript wasn't able to load completely. Thanks for all the help!!

Closing this ticket, thanks for your help Yayoi!


Can't get image layer to work

Hi Folks,
Thanks for the great module.
I've tried a bunch of settings but cannot get the image-as-fog thing happening.   I've got two of the same images as background and image, all my players see is black fog.
Are there any limitations/tricks that apply to this feature that are not mentioned? Any specific other settings in the module needed to make it work - I've tried a bunch of combinations but no luck.

I have been able to make this work, friend me on Discord thagrasshoppa#9562 if you still want to troubleshoot it!


./configure option to select whether the "usb" backend should run as root

Probably most distributions make the USB raw devices for printers accessible for the "lp" user by group ownerships and permissions or by using ACLs. Probably therefore CUPS installs the "usb" backend with 755 permissions so that it gets executed as the unprivileged "lp" user.
Some distributions (at lease Debian, Ubuntu, and probably everything derived from that) run "usb" as root, simply by removing the executable bits for group and others (744) after running make install during the package build.
The same I do in the CUPS Snap. As in the Snap I cannot create an "lp" user and group and I also cannot change ownerships and permissions of the raw USB devices from within the Snap, I need to run the "usb" backend as root. Currently I am, also changing the permissions of the backend after "make install", actually as a "prime-override" in snapcraft.yaml.
Now I want to ask if one could add a new option to ./configure, like --enable-usb-backend-run-as-root to let make install to install the "usb" backend with 744 permissions right away.
I habe looked into the backend/Makefile and I was not sure how to best implement this without breaking anything.

[master bf5f5047e] Install the USB backend to run as root.


NPE at AbstractKafkaConnector.stopTask

NPE at AbstractKafkaConnector.stopTask
After stopping a kafka mirroring task I noticed this NPE in the log.
 [2019-11-27 14:12:00,896] INFO Stopping the DurableScheduledService for mirror-_test_replicator_1574862570_fc2f511c-af4f-43d1-8a35-28c3fe9627b2 (com.linkedin.datastream.common.DurableScheduledService)
 [2019-11-27 14:12:00,907] WARN Task mirror-_test_replicator_1574862570_d3b0ffd9-6d41-4217-93e7-574186923cc8(kafkaMirroringC), partitions=[0] took longer than 30000 ms to stop. Interrupting the thread. (com.linkedin.datastream.connectors.kafka.mirrormaker.KafkaMirrorMakerConnector)
 [2019-11-27 14:12:00,907] WARN Failed to check status of kafka connector tasks. (com.linkedin.datastream.connectors.kafka.mirrormaker.KafkaMirrorMakerConnector)
 java.lang.NullPointerException
 	at com.linkedin.datastream.connectors.kafka.AbstractKafkaConnector.stopTask(AbstractKafkaConnector.java:216)
 	at com.linkedin.datastream.connectors.kafka.AbstractKafkaConnector.restartIfNotRunning(AbstractKafkaConnector.java:196)
 	at java.util.concurrent.ConcurrentHashMap$KeySetView.forEach(ConcurrentHashMap.java:4649)
 	at com.linkedin.datastream.connectors.kafka.AbstractKafkaConnector.lambda$start$1(AbstractKafkaConnector.java:176)
 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
 	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
 	at java.lang.Thread.run(Thread.java:748)

Your environment
I am using this docker image dongjinleekr/brooklin:1.0.0-2, which internally uses brooklin 1.0.0

Operating System: 4.9.0-11-amd64 #1 SMP Debian 4.9.189-3+deb9u1 (2019-09-20) x86_64 Linux
Brooklin version: 1.0.0
Java version: openjdk version "1.8.0_212"
Kafka version: 2.1.0
ZooKeeper version: 3.4.10

Steps to reproduce
It doesn't always repro.
But in my case I started replicating a kafka topic and then after several minutes stopped.
maxTasks was set to 50
Expected behaviour
There should not be an NPE.
Actual behaviour
There's an NPE in the log. It didn't seem to affect the worker though, the only seen effect was in the log.

Hello @rantav ,
Thanks for reporting this issue. This issue was fixed in #682.


How Can I set Google Client options with this package?

I am having a problem with my server, I am running different laravel installs on one machine, but it seems they all want to use /tmp/Google_Client/ and it has now become an issue since the different sites run under different users.
Anyway the fix to what I am encountering is this:
http://stackoverflow.com/questions/17282894/a-client-error-occurred-could-not-create-storage-directory-tmp-google-client
Which is really just a matter of doing this:
$config = new Google_Config();
$config->setClassConfig('Google_Cache_File', array('directory' => '../tmp/cache'));
$client = new Google_Client($config);
Maybe add an option to set it via the config file or the .env?

Yep. Agreed. I think the app-local tmp path should suffice - would you care to contribute a pull request?

Reviewing the Google_Client we may be able to simply use
$client = new Google_Client();
$client->setCache(new Google_Cache_File('/path/to/shared/cache'));

On the long run it might be useful to implement some of the caching logic and set it dynamically to the most useful available cache backend. What do you think?

Forgive me I've never contributed on github before but I will make an attempt :)
As far as your other comment, I think it should be an option on the driver config, perhaps 'cache_path' and default it to storage_path('framework/cache') or override via .env GCS_CACHE_PATH ?

Exactly what I meant - maybe with the addition of GCS_CACHE_TYPE defaulting to 'file', resulting in Google_Cache_File instantiated.
I'll happily review your contribution - no worries about that :)

I'm not a fan of file caching. It won't solve wammy21's permission problems either since Google_Client will continue setting very restrictive permissions.
Would it not be better to make everything about the Google_Client configurable so we can choose our own cache type?

What about simply wrapping it into the native flysystem cache decorator?
https://flysystem.thephpleague.com/caching/

I think that's another kind of caching. The CachedAdapter seems to cache calls between the Flysystem adapter and the user; it is not a generic cache implementation that you can inject into the Google_Client itself. This issue is about configuring the Google_Client which requires a cache implementation adhering to their own interface so they can store auth tokens.

@cedricziel I have finally been able to get around to doing this - I am ready to submit this but either I am doing it wrong or you might need to give me permissions to submit a pull request. I can also submit a patch, just need to know which way is preferred. (this is for the 1.0 branch)


Separate saving intents from updating NLU model.

Issues

If you're having trouble with Articulate we definitely want to help you. If you can provide a little bit of information it will make our troubleshooting job easier.

Please describe the issue you are having here and answer the question below.

Did you download a release or pull from Master? Which release?
What operating system are you running?
Are you running locally or in the cloud?
Which browser are you using?

Feature Request

If you've got a feature request we want to hear it

Please desribe the feature you would like us to add.
If possible provide the use case for the feature and how you would use it.
Each time I save an intent the RASA NLU model is recomputed. I would like to have the option to create several intents then have the model refreshed. This would require two separate buttons.
Thanks,
Hadil G. Sabbagh, Ph. D.

Hi @hadils if you move to our latest release this is already done


Rest API GET limit issue

Hi  Team
I was trying out the browsermob's REST API for 2.1.0 release. I followed the instructions to start and create a new proxy:
$ sh browsermob-proxy -port 9090
...
$ curl -X POST -d 'port=9099' http://localhost:9090/proxy
Then i tried setting the limits by the request:
PUT localhost:9090/proxy/9099/limit?downstreamKbps=512&upstreamKbps=512&downstreamMaxKB=1024&upstreamMaxKB=1024
But when i try to get the limits, the returned values are 0.
GET localhost:9090/proxy/9099/limit    --> returns
{
"maxUpstreamKB": 0,
"remainingUpstreamKB": 0,
"maxDownstreamKB": 0,
"remainingDownstreamKB": 0
}
When i follow the same steps for browsermob 2.1.0-beta2 and beta6, i get the correct values from the GET request. Can anyone explain the difference in behavior? Which is the expected behavior?

Thanks for reporting this, @nehat09. This should now be resolved by #467. I do want to point out that the "remainingUpstream" and "remainingDownstream" fields are no longer informative with the default LittleProxy implementation. Let us know if you still have issues with this. Thanks!


Fail explicitly when initializer does not have default export

This is not the real fix for #277 as @rwjblue points out in that issue, but it will help trace the issue for Embroider-built apps more easily. Since the blueprint in Ember also has this default export, it seems reasonable to hard-error when it's not there (e.g. if someone created the file manually and forgot to add it)

Released in https://github.com/ember-cli/ember-load-initializers/releases/tag/v2.1.2


fdsfd

fsd

asdasdsadsadasd

many thanks for your deep thoughts on this issue, @marcandregirard ;) the problem was now fsddhgf'd so that the sfdfdsgf could dfsfdsffds. Issue closed!


scx.full: 1.0.7 -> 1.0.8

https://github.com/sched-ext/scx/releases/tag/v1.0.8
https://github.com/sched-ext/scx/compare/v1.0.7...v1.0.8
Tested.

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
25.05 Release Notes (or backporting 24.11 and 25.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

nixpkgs-review result
Generated using nixpkgs-review.
Command: nixpkgs-review pr 365884

x86_64-linux

:white_check_mark: 7 packages built:

scx.cscheds
scx.cscheds.bin
scx.cscheds.dev
scx.full
scx.full.bin
scx.full.dev
scx.rustscheds

Looks good, I will merge after I test the update.


iphone8+: PLP grid view has spacing issues

Environment Details
Spartacus: beta release
Browser: Chrome
Device: Iphone8+
Happens on: https://spartacus.c39j2-walkersde1-d4-public.model-t.cc.commerce.ondemand.com
Steps to Reproduce
Go to "Digital Cameras -> Shop all"
On PLP, click on the grid view icon
Actual Result
There doesn't seem to have a difference between grid and list view on an iphone.
Except that the grid view has some spacing issues.

This issue got fixed in GH-2835 but pls re-test after aforementioned ticket has been merged

This has been merged and seems to work fine in local with latest develop.
Please note this was tested in ModelT, so please wait until changes are deployed there to retest on a real device.

ccv2 was updated with beta release which was done on friday.
i tried there and i still see this issue.

This look good on my end both Chrome and Safari on an Iphone7 (same screen size as 8).
Please make sure to clear cache or open in an incognito window.

@developpeurweb I still see this issue (even after clearing cache & incognito). Did you go on the grid view on PLP page?
Because the list view is fine. It's the grid view that has the issue.

I don't see the issue on modelT anymore (but the data has changed too).
But can close the ticket for now.


Fix RHEL_RDO_REPO_RPM for current version

The https://repos.fedorapeople.org/repos/openstack/openstack-kilo/rdo-release-kilo-1.noarch.rpm was replaced by https://repos.fedorapeople.org/repos/openstack/openstack-kilo/rdo-release-kilo-2.noarch.rpm

Thank you for contributing to openstack-dev/devstack!
openstack-dev/devstack uses Gerrit for code review.
If you have never contributed to OpenStack before make sure you have read the
getting started documentation:
http://docs.openstack.org/infra/manual/developers.html#getting-started
Otherwise please visit
http://docs.openstack.org/infra/manual/developers.html#development-workflow
and follow the instructions there to upload your change to Gerrit.