ES backport: fix test

Summary of the changes / Why this improves CrateDB
See commit messages
Follows:  https://github.com/crate/crate/commit/208fc2bdc79def94f58d17fd16d84e6a514e651c
Checklist

[x] Added an entry in CHANGES.txt for user facing changes
[x] Updated documentation & sql_features table for user facing changes
[x] Touched code is covered by tests
[x] CLA is signed
[x] This does not contain breaking changes, or if it does:

It is released within a major release
It is recorded in CHANGES.txt
It was marked as deprecated in an earlier release if possible
You've thought about the consequences and other components are adapted
(E.g. AdminUI)

retest this please


ci: remove skipped job & combine (faster)

Pulling out of #762. Updates CI. Mostly a followup to #673.

I think rebasing on #778 will be good. This saves at least a minute per previously pair of runs.


Enhance web-site documentation: TreePPL language

The cheat sheet is a good start but we need more things.

Used to be this issue https://github.com/treeppl/treeppl/issues/43


EXIT command offers insufficient control

Reported by hankin on 23 Jul 2001 20:27 UTC
more controls related
to the EXIT command:
EXIT/script -- exit from current script
EXIT/loop -- break from current repeat loop (or future FOREACH)
EXIT/prompt -- return to command prompt, regardless of current activity
EXIT/program -- terminate Ferret (the current "EXIT/command_file" to be
denegrated as misleading)
===========================
Lynn DeWitt - July 18:
EXIT is placed
within a REPEAT loop.  Here's an example:  The command files test1
and test2 exit back to test.jnl, but test3.jnl exits all the way back
to the command prompt without executing test1 the second time.
===========================
Andrew Wittenberg - July 18
Is there a way to terminate all Ferret scripts and return control to the
user?  This is would be nice for dealing with errors.  EXIT terminates
only the top script on the stack, and EXIT/COMMAND_FILE terminates Ferret
itself.  For now I'm just using the (nonexistent) command "stop" so that
Ferret will give a syntax error and stop all running scripts.
Migrated-From: http://dunkel.pmel.noaa.gov/trac/ferret/ticket/292

Comment by<EMAIL_ADDRESS>on 24 Jul 2001 15:35 UTC
Improved qualifier names suggested by Andrew WhittenbergL
EXIT/ferret -- quit all scripts and Ferret
EXIT/batch  -- quit all scripts
EXIT/script -- quit the highest script on the stack
EXIT/repeat -- quit the highest REPEAT loop on the stack

Comment by ansley.b.manke on 24 Jul 2010 01:03 UTC
These have been implemented
yes? sh command exit
 EXIT/COMMAND/LOOP/SCRIPT/PROMPT/PROGRAM/CYCLE


Add support for number_of_shards

Would be a nice change! This would lower our memory pressure since we are now using loads of shards with the default settings.
Looking forward to this to be merged

Nice addition. Thanks!

Thanks. Any idea when you'll be releasing the new package to nuget?

Should be available now as 5.1.0

Thanks


Support UK TV channel S4C, relative of UK's Channel 4.

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2017.09.02. If it's not, read this FAQ entry and update. Issues with outdated version will be rejected.

[x] I've verified and I assure that I'm running youtube-dl 2017.09.02

Before submitting an issue make sure you have:

[x] At least skimmed through the README, most notably the FAQ and BUGS sections
[x] Searched the bugtracker for similar issues including closed ones

What is the purpose of your issue?

[ ] Bug report (encountered problems with youtube-dl)
[x] Site support request (request for adding support for a new site)
[ ] Feature request (request for a new functionality)
[ ] Question
[ ] Other

If the purpose of this issue is a bug report, site support request or you are not completely sure provide the full verbose output as follows:
Add the -v flag to your command line you run youtube-dl with (youtube-dl -v <your command line>), copy the whole output and insert it here. It should look similar to one below (replace it with your log inserted between triple ```):
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-v', u'http://www.s4c.cymru/clic/e_level2.shtml?programme_id=540283584']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2017.09.02
[debug] Python version 2.7.9 - Linux-3.19.0-84-generic-x86_64-with-Ubuntu-15.04-vivid
[debug] exe versions: ffmpeg 2.5.10-0ubuntu<IP_ADDRESS>, ffprobe 2.5.10-0ubuntu<IP_ADDRESS>, rtmpdump 2.4
[debug] Proxy map: {}
[generic] e_level2: Requesting header
WARNING: Falling back on generic information extractor.
[generic] e_level2: Downloading webpage
[generic] e_level2: Extracting information
[generic] e_level2: Downloading m3u8 information
WARNING: Failed to download m3u8 information: <urlopen error [Errno 110] Connection timed out>
ERROR: No video formats found; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "./youtube-dl/youtube_dl/extractor/generic.py", line 2116, in _real_extract
    doc = compat_etree_fromstring(webpage.encode('utf-8'))
  File "./youtube-dl/youtube_dl/compat.py", line 2539, in compat_etree_fromstring
    doc = _XML(text, parser=etree.XMLParser(target=_TreeBuilder(element_factory=_element_factory)))
  File "./youtube-dl/youtube_dl/compat.py", line 2528, in _XML
    parser.feed(text)
  File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1642, in feed
    self._raiseerror(v)
  File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1506, in _raiseerror
    raise err
ParseError: syntax error: line 1, column 0
Traceback (most recent call last):
  File "./youtube-dl/youtube_dl/YoutubeDL.py", line 776, in extract_info
    ie_result = ie.extract(url)
  File "./youtube-dl/youtube_dl/extractor/common.py", line 434, in extract
    ie_result = self._real_extract(url)
  File "./youtube-dl/youtube_dl/extractor/generic.py", line 2882, in _real_extract
    self._sort_formats(entry['formats'])
  File "./youtube-dl/youtube_dl/extractor/common.py", line 1072, in _sort_formats
    raise ExtractorError('No video formats found')
ExtractorError: No video formats found; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

If the purpose of this issue is a site support request please provide all kinds of example URLs support for which should be included (replace following example URLs by yours):

Single video: http://www.s4c.cymru/clic/e_level2.shtml?programme_id=540283584

Description of your issue, suggested solution and other information
S4C is a UK TV channel from the same company as Channel 4, but with Welsh-language programmes. I suspect they use the same method of providing programmes to watch online.

S4C is completely separate to Channel 4. The services are not related in any way.

At least some S4C content is available via the BBC's iPlayer, and so get_iplayer can be used.


Duplication of effort?

Hi,
First let me apologize for using the github issue system for what is not really a bug report. I am simply too lazy to sign into another forum :)
I just wanted to let you know that the of the major features you quote for Transcrypt: "Multiple inheritance, recursive tuple assignment, multi-loop nested list comprehensions, LHS and RHS extended slices, assignment of bound functions, lambdas, named, default, _args and *_kwargs parameters, properties and a hierarchical module system are just a few of its characteristics that make this clear."
only multiple inheritance is not currently supported by RapydScript. Specifically, by the work I have been doing on rapydscript https://github.com/kovidgoyal/rapydscript-ng
And adding support for multiple inheritance should not take more than a day or two of work -- it just isn't a priority for me.
RapydScript has many features it would take a fair bit of effort to duplicate: a REPL, support for gettext, a linter, a minifier, set,dict,generator comprehensions, yield and generators, ESS5 and ES6 profiles, and so on.
This is only an FYI -- it would be nice to be able to combine efforts on a single python-like transpiler, but, I totally understand if you would rather go it on your own.

P.S. If you would rather discuss this in private, you can find my email address at the top of https://github.com/kovidgoyal/rapydscript-ng/blob/master/src/lib/elementmaker.pyj

Hi Kovid, you should have received my reply. If not, let me know.
Kind regards, Jacques

I got it and replied, feel free to close this bug report, if you want to.

Do you guys mind sharing the outcome?

JdeH decided he wanted to continue to work on Transcrypt, and not merge efforts. But he is open to collaborating on implementing libraries from the python ecosystem for python-to-javascript transpilers in the future.

Correct

I've been checking out both transpilers (+ pyscript, which I'm not sure where to fit), awesome to see a collaboration effort.
If it's ok to ask - for a minimal overhead over regular JavaScript, which transpiler would you say is more suitable currently?  aka which to use if performance is a high priority?
Thanks!

See https://github.com/kovidgoyal/rapydscript-ng/issues/46 for some benchmarks

Hi!
I went over the project and while I was looking at python side I was delighted, but JS-side upset me.
My 2 cents of thoughts:
If I  want to write a more or less large app using the existing JS-framework I should understand how it works including JS itself! So, If I know JS (but love Python, I really love it)  which transpiler will I use? - I think, it will be one which is closer to JS as possible with no magic and JS-basics modification (like Array.prototype.append = ...  - are you sure that some JS-libs doesn't do the same?).
If I will want to write some awesome JS-lib/framework, then  it should be suitable for  pure JS contributors, what shall I use?
Just for example:  Autobinding makes the classes unsuitable to use with Vue (at least).  I mean the following will not work:
 Class Foo:
   def __init__(self):
         self.methods =  { 'bar': bar}

    def bar(self):
         ...

v_foo = new Vue( Foo() )

So, I can't imagine how I could use this project(
P.S. JS has the fatal drawback - it's not Python.


Enable mapping subsets and fields

PR that enables remapping the column names of a user's dataset to match the names of a resuable/custom component spec.

Mapping one source subset to different target subsets or vice-versa is not allowed since it might lead to unexpected behavior and edge cases.

Mapping one source subset to a different target subset with a different schema is also not allowed and is checked during the static evaluation of the pipeline.

Remapping the column names is transient and occurs only within the mapped component

mapped subsets_fields will be written with their original name prior to mapping
new subsets_fields introduced in the component will be written with their new name without the need to specify an explicit mapping

In order to achieve the different mapping between the components, the mapping happens at the component spec level since the component spec defines which datasets to load and how to evolve the manifest. This ensure that we don't have to remap in many places (static checking, manifest evolution), facilitates static pipeline evaluation and helps keep the lineage more consistent.

We still need to remap the column names of the dataframe after loading (to be able to execute the transform function of the component) and before writing (revert back to original column names). This is better illustrated in the following figure

Thanks @PhilippeMoussalli. I think we should add a description and maybe a small example on how to use it, to the documentation. Can imagine that this will be used a lot.

I have retested your modifications end-to-end using the constructed pipeline:
# Component 1
load_file = ComponentOp(
    component_dir="load_from_files",
     ...
)

# Component 2
language_filter = ComponentOp(
    component_dir="...",
    spec_mapping={
        # produced by component 1 -> consumed by component 2
        "file_filename": "text_data"
    }
)

# Component 3
filter_comp = ComponentOp(
    component_dir="...",
    spec_mapping={
        # produced by component 1 -> consumed by component 3
        "file_filename": "text_data"
    }
)

The subset and field mappings are working as expected. Personally, I found it a bit unintuitive to determine the correct spec_mappings. Maybe we could utilise a dataclass for defining the spec_mappings. This approach would allow us to name the properties appropriately, we would get type hints and documentation within the code.
E.g. something like this:
spec_mapping = [
   ColumnMapping(global_column="file_filename", component_column="text_data")
]

I have retested your modifications end-to-end using the constructed pipeline:
# Component 1
load_file = ComponentOp(
    component_dir="load_from_files",
     ...
)

# Component 2
language_filter = ComponentOp(
    component_dir="...",
    spec_mapping={
        # produced by component 1 -> consumed by component 2
        "file_filename": "text_data"
    }
)

# Component 3
filter_comp = ComponentOp(
    component_dir="...",
    spec_mapping={
        # produced by component 1 -> consumed by component 3
        "file_filename": "text_data"
    }
)

The subset and field mappings are working as expected. Personally, I found it a bit unintuitive to determine the correct spec_mappings. Maybe we could utilise a dataclass for defining the spec_mappings. This approach would allow us to name the properties appropriately, we would get type hints and documentation within the code.
E.g. something like this:
spec_mapping = [
   ColumnMapping(global_column="file_filename", component_column="text_data")
]

Thanks for the feedback @mrchtr! I'm glad it works as expected.
Haven't thought about adding is as a dataclass but it does makes sense, I think we will still have to translate it internally as a dictionary to pass it to kubeflow but that should not affect the interface you proposed

@mrchtr added a PR to change the user interface as you suggested, let me know if you have any additional feedback to make it more intuitive.
https://github.com/ml6team/fondant/pull/352/commits/e891f3f16ab4bba5b17efd9d5f5ad21b2063eaa3
Once we align on the interface, i'll add documentation and examples on how to use it

@mrchtr added a commit to change the user interface as you suggested. e891f3f
let me know if you have any additional feedback to make it more intuitive., once we align on the interface, i'll add documentation and examples on how to use it

I like it. It is quite smooth and intuitive now.

Remapping the column names is transient and occurs only within the mapped component

mapped subsets_fields will be written with their original name prior to mapping
new subsets_fields introduced in the component will be written with their new name without the need to specify an explicit mapping

We still need to remap the column names of the dataframe after loading (to be able to execute the transform function of the component) and before writing (revert back to original column names). This is better illustrated in the following figure

Thanks @PhilippeMoussalli! Haven't been able to look at the code in detail yet, but I don't agree with the points above. I believe we should only map the input of the component and follow the original component specification for the output.
I think this is a lot more intuitive since you can still infer which data will be produced by the component by just looking at its component specification, while still providing the same flexibility.

Remapping the column names is transient and occurs only within the mapped component

mapped subsets_fields will be written with their original name prior to mapping
new subsets_fields introduced in the component will be written with their new name without the need to specify an explicit mapping

We still need to remap the column names of the dataframe after loading (to be able to execute the transform function of the component) and before writing (revert back to original column names). This is better illustrated in the following figure

Thanks @PhilippeMoussalli! Haven't been able to look at the code in detail yet, but I don't agree with the points above. I believe we should only map the input of the component and follow the original component specification for the output.
I think this is a lot more intuitive since you can still infer which data will be produced by the component by just looking at its component specification, while still providing the same flexibility.

Thanks for the feedback, the main reasoning behind the transient renaming is mainly to avoid writing duplicate subsets, if we only rename the outputs then we will have duplicate subsets

If a subset is defined in the produces section of the component spec, it will be rewritten anyway. If a subset is consumed but not produced, we leave it in its original state. There naming should not impact if data is written or not.

Ok that makes sense! I updated the PR accordingly

Sorry for the delay on this @PhilippeMoussalli, but I'm a bit hesitant to merge until we have a better view on #244, as I believe the component spec manipulation in this PR might lead to issues when implementing that functionality.

I agree with most of your points, indeed there is some redundancy and some things can be simplified.

Thanks @PhilippeMoussalli!

Since we map the columns of the loaded dataframe to the ones in the original consumes spec before passing it to the transform method, the spec passed to the component will actually not match the data it receives.

Yes indeed, that is if you look at the spec mainly as specifying the fields required for the transformation method. I was considering more as an indicator on which fields to load.

Components accessing the data dynamically based on the spec will fail because of this.
I think this only applies to generic components but in any case the mapping there is not relevant since most of them have a distinct mapping argument (we should think about combining the two maybe at some point for less confusion)

This is logical, so maybe we need to make this explicit: let the user define a subset mapping, and a mapping per subset. I'm not sure yet what this should exactly look like, but I think it would be a lot more robust.

I defined a new schema for how this could look like based on your feedback. It also contains how the new implementation would look like. Let me know what you think

One other thing, I think by not mapping the produces section we might introduce some more work at the user's side since they will have to introduce remapping for all subsequent components in their pipelines. Not sure how big of an issue this can be.

Yes indeed, that is if you look at the spec mainly as specifying the fields required for the transformation method. I was considering more as an indicator on which fields to load.

That's a good point, I think we currently use it for both. Maybe we should make this explicit. Eg. by letting the ComponentSpec have two different representations: ComponentSpec.io_spec and ComponentSpec.transform_spec. That way we can still encapsulate the related behavior in a single class, and use the relevant represenation in each place.

I think this only applies to generic components but in any case the mapping there is not relevant since most of them have a distinct mapping argument (we should think about combining the two maybe at some point for less confusion)

True, but we still pass the spec to each component, which can lead to confusion and might still be used in some cases we currently don't plan for.

I defined a new schema for how this could look like based on your feedback. It also contains how the new implementation would look like. Let me know what you think

Looks good!

One other thing, I think by not mapping the produces section we might introduce some more work at the user's side since they will have to introduce remapping for all subsequent components in their pipelines. Not sure how big of an issue this can be.

If we can split the different functionalities of the ComponentSpec as mentioned above, I indeed see more benefits than downsides to adding produces mapping.

Yes indeed, that is if you look at the spec mainly as specifying the fields required for the transformation method. I was considering more as an indicator on which fields to load.

That's a good point, I think we currently use it for both. Maybe we should make this explicit. Eg. by letting the ComponentSpec have two different representations: ComponentSpec.io_spec and ComponentSpec.transform_spec. That way we can still encapsulate the related behavior in a single class, and use the relevant represenation in each place.

so then the default componentSpec would be equivalent to the transform_spec and the io_spec can be a class method that takes in the original spec and modifies it's specification based on the mapping spec.
The io_spec is used for loading/writing + evolving the manifest. However, the transform_spec function is to map the names of the dataframes before/after the transformation but that is not achievable by itself since by definition the spec does not have a mapping to the io_spec, so we can represent the mapping between the two specs as an additional property and use that for the mapping. However, something similar can be achieved by just using the mapper and it's inverse in which case the only purpose of having both transform and io spec + the mapping between them is to have a better representation of the mapping (without really enabling additional functionalities that cannot be achieved by the suggestion above). Is that correct?

One other thing, I think by not mapping the produces section we might introduce some more work at the user's side since they will have to introduce remapping for all subsequent components in their pipelines. Not sure how big of an issue this can be.

If we can split the different functionalities of the ComponentSpec as mentioned above, I indeed see more benefits than downsides to adding produces mapping.

Alright then I will also a separate mapping for the produces section.


Updated sql file and added security settings for LQA

Critical Changes
Changes
Issues Closed
New Metadata
Unpackaged Metadata
Deleted Metadata
Testing Notes

my understanding is that settings file is a temporary file we want to generate lqa orgs? I assume we don't want to add it to our package?

also, the branch has a typo and thus not running the builds ;)


Integrate docker compose with project so that we can start and debug in visual studio.

Update docker compose to start DTM with MySQL data storage.


Fix a couple of failing tests

Hi.
I noticed some of the tests are failing, at lest on my setup. I fixed a couple of them, there are still a few ones left to fix, though.
Per.

Very good find. Thank you.


Format telephone numbers

Telephone numbers entered should be displayed as ###-###-####.

Cypress test is also updated https://github.com/LiveChurchSolutions/ChumsTests/commit/50e7c5dd9c56d20d06f15717e24d5acaa18976fd


Added Norwegian and update support

moved language files into subfolder

Some more work done on the same branch:
Rearanged and cleaned up language files, removed deprecated strings, added missing and new ones.
Added missing 'Unique order ID' setting for VM3, and 'Distribution method' for VM2 ...
Cleaned up indentation mix (fixed to spaces), generic formating cleanup and removal of whitespaces.


Image resolution problem on mobile

On Samsung Internet on mobile, if I select a local image with the file input, it loads in a clearly degraded resolution.
E.g. Original file (Well, Github may have processed the image when uploaded here, but it gives the idea):

Screenshot of how it looks once loaded in Snapwat:

I'm taking a look at the code in src/pages/home.js that uses JavaScriptLoadImage...

See: https://www.html5rocks.com/en/tutorials/canvas/hidpi/

Fixed! All it took was defining the pixelRatio option for JavaScriptLoadImage.


Experiment: Checking ability to respond to a locked issues if @-mentioned but only previously

Body here.

@peter-kehl See if you can respond even though this issue is now locked.

Wow: Cross-referencing to a locked issue (this one, cross-referenced from #5) does NOT add/show a link here!

Cross referencing to a closed issue #7.


Have an example of Fractal running on a public dataset

Let's have at least one example in the main Fractal repo that runs on a public dataset (see https://github.com/fractal-analytics-platform/fractal/issues/163) and produces relevant outputs that would be fitting for a Fractal demo.

I uploaded the 2x2, 10Z test case as a Zenodo dataset here: https://zenodo.org/record/7057076
@tcompa Let me know if you have a smaller test set already ready or in mind. I can annotate it with the relevant metadata information about the staining etc and upload it to Zenodo :)

@tcompa Let me know if you have a smaller test set already ready or in mind. I can annotate it with the relevant metadata information about the staining etc and upload it to Zenodo :)

I'd like to have one which is as small as possible.
What about a single channel (dapi?), two Z planes (somewhere in the middle of the Z stack), and two FOVs instead of four? That would be 4 images, which seems reasonably small to me.

Note that the goals of having some data on zenodo are (from high to low priority):

Building a demo with publicly-available (and reliable) data, instead of using local copies of the same (which could get out of sync, if anything needs to be fixed e.g. in the metadata files).
Using a tiny dataset within CI. Depending on the download time (*), we could either always include these tests in the CI or keep them as optional.
Preparing a heavy integration test on a reasonably large dataset (to be run on the cluster, and not as part of the standard CI).

Given these goals, zenodo-get seems to be good enough in downloading all data for a zenodo record (where "good enough" stands for "only download files if they are not already present, and check their md5 checksum"). I'm adding it as an optional dependency, by now.
(*) From a couple of tests, I'm expecting the download time for the four-images dataset to be ~10 seconds, which eventually would be fine to be part of the CI..

@tcompa Thanks for the list of priorities, I agree with the order. Let's aim for 1 & 2 for the moment, 3 is more a long-term goal.
The tiny dataset is now also available on Zenodo: https://zenodo.org/record/7059515
Let's include it in our tests, it's just 27 MB to download and should give the confirmation that the tests run on some real data.
I also put it on our share for the moment (/data/active/fractal/3D/PelkmansLab/CardiacMultiplexing/Cycle1_tinyTestSet), but the data on Zenodo should be the relevant reference.

With https://github.com/fractal-analytics-platform/fractal-tasks-core/commit/1627e117238772f4bd036df4732615f4ebe2a96a, we have a first example running on real-data (which is a mirror of the client/server one, in the fractal repository).
With the tiny dataset, this first example runs on a local machine in a few seconds (but it will be a few minutes when adding labeling, depending on labeling_level).
We should include a couple more tasks, and then this issue can be closed.

Examples are running both on the tiny dataset and on the 2x2-FOVs one, see https://github.com/fractal-analytics-platform/fractal-server/issues/4#issuecomment-1243407277.
I'm closing this. Let's open more specific issues if needed.


Update introduction.md

Getting started
Getting started with balenaSound is as simple as deploying it to a balenaCloud application; no additional configuration is required (unless you're using a DAC HAT).
We've outlined the installation steps below. If you want a step-by-step tutorial on how to get balenaSound up and running, feel free to check these blog posts:

Turn your old speakers or Hi-Fi into Bluetooth, Airplay and Spotify receivers with a Raspberry Pi and this step-by-step guide
Build your own multi-room audio system with Bluetooth, Airplay, and Spotify using Raspberry Pis

Hardware required

Any device from our supported devices list. For the best experience, we recommend using a Raspberry Pi 3B+ or 4B.
An SD Card! We recommend the Sandisk Extreme Pro series. 8GB should be plenty enough for this project.
Power supply
3.5mm audio cable to the input on your speakers/Hi-Fi (usually 3.5mm or RCA). Alternatively you can use the HDMI port to get digital audio out.

Software required

Software to flash an SD card (balenaEtcher)
A free balenaCloud account
(optional) A download of this project
(optional) The balena CLI tools

One-click deploy
One-click deploy is the easiest way to get started with balenaSound as it allows you to deploy and configure the application with a single click and without the need of installing additional tools. Check out the CLI deploy instructions below if you are interested in an advanced alternative that enables you to achieve more complex deployments.
Deploy with balena
Click this button to go straight to application creation, where balenaSound will be pre-loaded to your application:

Provision your device

Once your application has been created you'll need to add a device to it:

Add a device to the application by clicking the add device button
Download the OS and flash it to your SD card with balenaEtcher
Power up your device and check it's online in the dashboard!

The balenaSound application will start downloading as soon as your device appears in the dashboard.
CLI deploy
This is the traditional and more advanced approach for deploying applications to balena powered devices. Installing and setting up the balena CLI is definitely more involved than using the Deploy with balena button, but it allows for more flexibility and customization when choosing what and when to deploy.
For example, if you don't plan on using the Spotify integration, you can edit the docker-compose.yml file and remove the Spotify service before deploying the application.
Provision your device

Sign up for or login to the balenaCloud dashboard
Create an application, selecting the correct device type. If you are using a Raspberry Pi (any model) we recommend setting the type as Raspberry Pi (v1 / Zero / Zero W) as your application will then be compatible with all Raspberry Pi versions.
Add a device to the application, enabling you to download the OS
Flash the downloaded OS to your SD card with balenaEtcher
Power up your device and check it's online in the dashboard

Deploy the application

Install the balena CLI tools
Login with balena login
Download this project and from the project directory run balena push <appName> where <appName> is the name you gave your balenaCloud application in the first step.

Having trouble?
If you are running into issues getting your balenaSound application running, please try the following:

Check the support and troubleshooting guide for common issues and how to resolve them.
Post in the balenaSound forum for help from our growing community.
Create an issue on the balenaSound GitHub project if you find your issue may be a problem with balenaSound.

Please make sure to read the CONTRIBUTING document before opening the PR for relevant information on contributing to the documentation. Thanks!

Hey @Conanza00 what is the purpose of the change you are making? In the description of the PR, I see balenaSound's getting started guide?


Out of memory (Coom) repeatedly

Hey guys, wondering if anyone else is having this issue. Get up to about 43% and then get an coom error. Im using a 1650 with 4gbVram and 32gb Ram on a Linux laptop. Simswap works fine, but having an issue with this.

You need at least 8GB GPU, 11GB is recommended.

You need at least 8GB GPU, 11GB is recommended.

Thank you for the answer. Wonder if there's a way to run it on a lower spec gpu. Lol thank God for simswap!

Im running sberswap on gtx1060 6GB.

Im running sberswap on gtx1060 6GB.

Thanks for the info bro.

I also have the gtx1060 6GB and i solved it by adding --batch_size 20 when i run inference.py


Failure building streaming transcription

Confirm by changing [ ] to [x] below to ensure that it's a bug:

[ x] I've gone though Developer Guide and API reference
[x ] I've searched for previous similar issues and didn't find any solution

Describe the bug
Following the instructions here to build the Amazon Transcribe Streaming C++ SDK, the build step:
cmake .. -DBUILD_ONLY=”transcribestreaming” -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=OFF

Fails with the following error:
-- Http client: Curl
-- Found CURL: /usr/lib/x86_64-linux-gnu/libcurl.so (found version "7.64.0")
--   Curl include directory: /usr/include/x86_64-linux-gnu
--   Curl library: /usr/lib/x86_64-linux-gnu/libcurl.so
-- Performing Test HAVE_ATOMICS_WITHOUT_LIBATOMIC
-- Performing Test HAVE_ATOMICS_WITHOUT_LIBATOMIC - Success
-- Considering ”transcribestreaming”
CMake Error at cmake/sdks.cmake:147 (message):
  ”transcribestreaming” is required for build, but
  aws-cpp-sdk-”transcribestreaming” directory is missing!
Call Stack (most recent call first):
  CMakeLists.txt:254 (include)

SDK version number
I checked out the current master branch, per the link cited above
Platform/OS/Hardware/Device
Debian 10
To Reproduce (observed behavior)
Follow the steps cited in the article, viz.
$ git clone https://github.com/aws/aws-sdk-cpp.git
$ cd aws-sdk-cpp
$ mkdir build
$ cd build
$ cmake .. -DBUILD_ONLY=”transcribestreaming” -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=OFF

Expected behavior
sdk gets built

Seems to be a bug in the article's code listing.  Removing the quotes fixed the issue, i.e.
cmake .. -DBUILD_ONLY=transcribestreaming -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=OFF


DataArray.plot raises exception if contents are all NaN

Code Sample, a copy-pastable example if possible
xr.DataArray(np.full((2, 2), 0)).plot.imshow()  # works
xr.DataArray(np.full((2, 2), np.nan)).plot.imshow()  # doesn't

Problem description
If you try to plot a DataArray which is entirely filled with NaN, you get an exception.  This is really, really annoying for people doing satellite image analysis, especially with timeseries where just one step might be very cloudy.
Expected Output
Plot of the array extent, entirely in the missing-value colour as for partially-missing data.
Output of xr.show_versions()
Confirmed on Windows/Linux/OSX, Xarray 0.9.6 and 0.10.0; sample show_versions below.  It's a pretty obvious error though in failing to handle the no-non-missing-data case when determining data range.

INSTALLED VERSIONS
------------------
commit: None
python: 3.6.3.final.0
python-bits: 64
OS: Linux
OS-release: 2.6.32-696.10.3.el6.x86_64
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_AU.UTF-8
LOCALE: en_AU.UTF-8
xarray: 0.10.0
pandas: 0.20.3
numpy: 1.13.3
scipy: 0.19.1
netCDF4: 1.3.1
h5netcdf: None
Nio: None
bottleneck: 1.2.1
cyordereddict: None
dask: 0.15.3
matplotlib: 2.1.0
cartopy: None
seaborn: 0.8.0
setuptools: 36.5.0.post20170921
pip: 9.0.1
conda: 4.3.30
pytest: 3.2.1
IPython: 6.1.0
sphinx: 1.6.3

Yes, I agree. We inherited the logic from seaborn, which also has this problem:
In [2]: import seaborn as sns
In [3]: import numpy as np
In [4]: a = np.zeros((2, 2)) * np.NaN
In [6]: sns.heatmap(a)
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-6-9ef15135ade2> in <module>()
----> 1 sns.heatmap(a)

~/.pyvirtualenvs/py3/lib/python3.5/site-packages/seaborn/matrix.py in heatmap(data, vmin, vmax, cmap, center, robust, annot, fmt, annot_kws, linewidths, linecolor, cbar, cbar_kws, cbar_ax, square, xticklabels, yticklabels, mask, ax, **kwargs)
    515     plotter = _HeatMapper(data, vmin, vmax, cmap, center, robust, annot, fmt,
    516                           annot_kws, cbar, cbar_kws, xticklabels,
--> 517                           yticklabels, mask)
    518
    519     # Add the pcolormesh kwargs here

~/.pyvirtualenvs/py3/lib/python3.5/site-packages/seaborn/matrix.py in __init__(self, data, vmin, vmax, cmap, center, robust, annot, fmt, annot_kws, cbar, cbar_kws, xticklabels, yticklabels, mask)
    166         # Determine good default values for the colormapping
    167         self._determine_cmap_params(plot_data, vmin, vmax,
--> 168                                     cmap, center, robust)
    169
    170         # Sort out the annotations

~/.pyvirtualenvs/py3/lib/python3.5/site-packages/seaborn/matrix.py in _determine_cmap_params(self, plot_data, vmin, vmax, cmap, center, robust)
    205         calc_data = plot_data.data[~np.isnan(plot_data.data)]
    206         if vmin is None:
--> 207             vmin = np.percentile(calc_data, 2) if robust else calc_data.min()
    208         if vmax is None:
    209             vmax = np.percentile(calc_data, 98) if robust else calc_data.max()

~/.pyvirtualenvs/py3/lib/python3.5/site-packages/numpy/core/_methods.py in _amin(a, axis, out, keepdims)
     27
     28 def _amin(a, axis=None, out=None, keepdims=False):
---> 29     return umr_minimum(a, axis, None, out, keepdims)
     30
     31 def _sum(a, axis=None, dtype=None, out=None, keepdims=False):

ValueError: zero-size array to reduction operation minimum which has no identity

Will try to submit a fix later today

Actually I changed my mind a little bit ;-). Matplotlib won't plot all NaNs images either:
import numpy as np
import matplotlib.pyplot as plt

a = np.full((2, 2), np.NaN)
plt.imshow(a)
plt.show()  # raises an error

So we have to decide if it's worth to add a bunch of logic in xarray to handle these cases or if it's the user's responsibility to check the data before plotting.

I'd argue that Xarray should handle these cases - it already does for lower dimensions, eg xr.DataArray(np.full(1, np.nan)).plot() works, and this would be really nice for exploratory analysis as well as consistency.
This is sufficiently annoying to my research group that I'm willing to write the patch, if that helps!  I also have some ideas for tests using Hypothesis to ferret out some other problems - for example there's a similar failure if plotting an array with size one in some dimension.

I'm willing to write the patch, if that helps!

Yes, thanks! The best place to start would be to add a regression test to Common2dMixin (https://github.com/pydata/xarray/blob/master/xarray/tests/test_plot.py#L560).
I investigated a bit and it seems that in the case of all nan data you would probably have to force add_colorbar to False and do other things for matplotlib to accept your all nan data.

I investigated a bit and it seems that in the case of all nan data you would probably have to force add_colorbar to False and do other things for matplotlib to accept your all nan data.

I don't need to do this, but I'd like to for another feature (imshow RGB images) - the question is how!  Passing add_colorbar=False to plot.imshow just gives me errors, and I can't find anywhere in the internals where this would be valid either - it always gets passed through to the matplotlib Artist and then an exception.

@fmaussion what version of matplotlib are you using? imshow with all NaNs works for me on matplotlib 1.5.2 (yes, we're stuck in the dark ages). This might be a  regression in matplotlib.

imshow with all NaNs works for me on matplotlib 1.5.2

ups yes, I might have been confused by the warning, but it does work with latest mpl yes. The warning says:
In [1]: import numpy as np

In [2]: import matplotlib.pyplot as plt

In [3]: a = np.full((2, 2), np.NaN)

In [4]: plt.imshow(a)
Out[4]: <matplotlib.image.AxesImage at 0x7f6e2a8ba710>

In [5]: plt.show()
/home/mowglie/.pyvirtualenvs/py3/lib/python3.5/site-packages/matplotlib/colors.py:897: UserWarning: Warning: converting a masked element to nan.
  dtype = np.min_scalar_type(value)
/home/mowglie/.pyvirtualenvs/py3/lib/python3.5/site-packages/numpy/ma/core.py:748: UserWarning: Warning: converting a masked element to nan.
  data = np.array(a, copy=False, subok=subok)


[Distributed] Guard ID synthesis from happening multiple times

Resolves rdar://92162535
In a project like
Sources
├── Lib
│   ├── OtherModule.swift
│   └── system.swift
└── ReproTry
├── SameModule.swift
└── main.swift
If the actor is declared in SameModule, and used from main.swift rather the synthesis for its ID might trigger twice and we don't protect from that happening.
The system has some logic for preventing it already but let's handle both the same way.
This results in a hard crash, of not being able to find the ID in the main.swift app - because there are TWO id properties.

A note on tests: I wasn't able to reproduce this in a test... what am I doing wrong? 🤔
I did reproduce and confirm this patch fixes it in a real swiftpm project though hmmm

@swift-ci please smoke test

@swift-ci please smoke test Linux platform

Hmmm, the synthesis is always done from within a request already: GetDistributedActorIDPropertyRequest::evaluate so I don't think it's a race; it must be something about multiple invocations per module, though I don't fully get how that happens 🤔


Update auto_debugging task description to specify Python 3

This task only makes sense if we specify the language for this task - e.g. these aren't Java or C++ programs, so if you try to run these programs with a compiler/interpreter for one of those languages you'll get very different behavior.
I'd also argue that it's worth specifying the python version (3, maybe even worth specifying e.g. 3.7), since different python versions will have different syntax and runtime behavior. The trial at line 145 is one example of this - this gives a different error when executed in Python 2 than the one listed here: RuntimeError: maximum recursion depth exceeded.

I completely agree with this change -- thank you for identifying the problem and making the PR.


Readme default for AUTOUNIT_EXCLUDED_HEADERS

Is the default [] or does it include Authorization + co?  I was wondering if it's possible to include Authorization in your tests if you override AUTOUNIT_EXCLUDED_HEADERS as empty.

No, you can't include Authorization headers in fixtures so far.
Authorization headers are excluded always regardless of AUTOUNIT_EXCLUDED_HEADERS setting. If you set this setting to [], Authorization headers will still be excluded.
I guess we could set the default setting to [...auth headers...] and if the developer wants to include them just sets the setting as [].
Anyway, should we allow the user include Authorization headers as an option?

No, you can't include Authorization headers in fixtures so far.
Authorization headers are excluded always regardless of AUTOUNIT_EXCLUDED_HEADERS setting. If you set this setting to [], Authorization headers will still be excluded.
I guess we could set the default setting to [...auth headers...] and if the developer wants to include them just sets the setting as [].
Anyway, should we allow the user include Authorization headers as an option?

I would IMO, just in case someone does want to test it.

Okay, I'll change this 👍

Solved in https://github.com/fcanobrash/scrapy-autounit/commit/13053f9f6fe0adf85a514b71905d07c7d216f92f - Closing the issue.


Validation on query/get?

I'm using the .get and .query functions to find data in my db.
If I pass a value that fails the validation for that Model/Schema the get throws an error (uncaught by the catch in the Promise) and crashes my server.
Why is it that the error is thrown in such a way the Promise doesn't catch it? Or is there a better way for me to be handling this?
My current solution is to wrap those lookup functions in try/catch blocks but that's ugly and seems to go against the idea of Promises.
Thanks for any help

Sounds like a bug, I'll try to duplicate it.  Can you provide the code that causes the crash and the attribute schema validation?

I'm pretty sure I know where the issue is.  I'll try to get a fix in this week.

Sure here's a summary of the spots in my code that seem to be causing issues and awesome! Glad you have an idea of where to look.
I've got a user schema that contains a v4 UUID that uses a regex to validate it's a proper UUID
    id: {
      type: String,
      hashKey: true,
      required: true,
      trim: true,
      default: uuid.v4,
      validate: /^[0-9a-f]{8}-[0-9a-f]{4}-[1-5][0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/i,
     }

I set the model named User and whenever I did User.get({ id: userID }); where userID is not a v4 UUID it would throw an error so I had to put it in a try/catch block.

Hey @brandongoode just wondering if you'd had a chance to work on this at all?

Thanks for the reminder -- I'll work on the fix now.

Awesome thanks! Glad it was a quick fix :)


standoff2conll

Hi,
Where is this standoff2conll script and how can we use it?

Hi @Mahmedturk,
The script can be found at https://github.com/spyysalo/standoff2conll

Thanks

Hi,
Have you used the same script for pre-processing the LINNAEUS corpus? I need to add the IDs along with the labels.  For example,
yeast    B-Species     ncbi:4932
Can you suggest a way to do that? Or it needs to be done manually?


Bump for jessie-backports.

Hello Jeremy!
As part of backporting python-sphinx to jessie, I needed to backport alabaster as well.  It's a trivial backport; just bumping the version and rebuilding got a clean lintian build.
Thanks!
-Harlan

It doesn't let me do so in a PR, but you probably want this commit on your own jessie-backports branch, not master.

Honestly I don't have a backports branch as I haven't done anything to facilitate any bpo builds.


[mdns] fix potential issue in service registration callback

The issue is fixed by https://github.com/openthread/ot-br-posix/pull/1248


Ability to initialise a line chart with a preselected marker.

I would like to be able to control which markers are marked as selected.
I think it should be added to the markers options alongside the discrete setting.
At the moment there is no way for an external component or script to mark or unmark selected markers.

chart.toggleDataPointSelection(0, 1)

where 0 is the series-index and 1 is the data-point-index
https://apexcharts.com/docs/methods/#toggleDataPointSelection


Fix rally test text

@IJW-Wartrader I think I've fixed the main problem as discussed, but I've also added some new TODO notes on awkward things that occurred to me.
I also made a few other small tweaks while I'm at it — hopefully these are improvements as I intended and don't cause new problems. 😅
See what you make of it.

Completed final checks and tweaks — ready to publish.


BLE 5 ESP32 C3

Hello how are you?
I'm making a software to read temperature sensors.
These sensors send an advertisement every 5 to 30 seconds. I can set this up. The longer the time the better for me as the battery will last longer.
I'm having trouble reading these advertising packages.
I need to read at least 1 packet per minute, worst case scenario.
But I find that sometimes I can't read any advertising for more than 3 minutes. I don `t understand why.
I don't know if it's better for me to put a higher or lower scan time...
I don't understand why you don't capture the advertising packets every time.
I'm using ESP32-C3.
I used the example for bluetooth 5
I also put it to read only the MAC of my sensor.
Can anyone help me understand this?
`#include <WiFi.h>
#include <WiFiMulti.h>
#include <NimBLEDevice.h>
#if !CONFIG_BT_NIMBLE_EXT_ADV
error Must enable extended advertising, see nimconfig.h file.
#endif
WiFiClient espClient;
WiFiMulti wifiMulti;
void scanEndedCB(NimBLEScanResults results);
#define SERVICE_UUID        "ABCD"
#define CHARACTERISTIC_UUID "1234"
static NimBLEAdvertisedDevice* advDevice;
static bool doConnect = false;
static uint32_t scanTime = 40; /* 0 = scan forever */
NimBLEScan *pBLEScan;
/* Define the PHY's to use when connecting to peer devices, can be 1, 2, or all 3 (default)./
static uint8_t connectPhys = BLE_GAP_LE_PHY_CODED_MASK | BLE_GAP_LE_PHY_1M_MASK /| BLE_GAP_LE_PHY_2M_MASK */ ;
/* Define a class to handle the callbacks for client connection events /
class ClientCallbacks : public NimBLEClientCallbacks {
void onConnect(NimBLEClient pClient) {
Serial.printf("Connected\n");
};
void onDisconnect(NimBLEClient* pClient) {
    Serial.printf("%s Disconnected - Starting scan\n", pClient->getPeerAddress().toString().c_str());
    NimBLEDevice::getScan()->start(scanTime, scanEndedCB);
};

};
/* Define a class to handle the callbacks when advertisements are received */
class AdvertisedDeviceCallbacks: public NimBLEAdvertisedDeviceCallbacks {
void onResult(NimBLEAdvertisedDevice* advertisedDevice) {
    Serial.printf("Advertised Device found: %s\n", advertisedDevice->toString().c_str());
    if(advertisedDevice->isAdvertisingService(NimBLEUUID("ABCD")))
    {
        Serial.printf("Found Our Service\n");
        /* Ready to connect now */
        doConnect = true;
        /* Save the device reference in a global for the client to use*/
        advDevice = advertisedDevice;
        /* stop scan before connecting */
        NimBLEDevice::getScan()->stop();
    }
};

};
/* Callback to process the results of the last scan or restart it /
void scanEndedCB(NimBLEScanResults results){
Serial.printf("Scan Ended\n");
if (!doConnect) { / Don't start the scan while connecting */
NimBLEDevice::getScan()->start(scanTime, scanEndedCB);
}
}
/* Handles the provisioning of clients and connects / interfaces with the server /
bool connectToServer() {
NimBLEClient pClient = nullptr;
pClient = NimBLEDevice::createClient();
pClient->setClientCallbacks(new ClientCallbacks, false);

/* Set the PHY's to use for this connection. This is a bitmask that represents the PHY's:
 * * 0x01 BLE_GAP_LE_PHY_1M_MASK
 * * 0x02 BLE_GAP_LE_PHY_2M_MASK
 * * 0x04 BLE_GAP_LE_PHY_CODED_MASK
 * Combine these with OR ("|"), eg BLE_GAP_LE_PHY_1M_MASK | BLE_GAP_LE_PHY_2M_MASK | BLE_GAP_LE_PHY_CODED_MASK;
 */
pClient->setConnectPhy(connectPhys);

/** Set how long we are willing to wait for the connection to complete (seconds), default is 30. */
pClient->setConnectTimeout(10);

if (!pClient->connect(advDevice)) {
    /* Created a client but failed to connect, don't need to keep it as it has no data */
    NimBLEDevice::deleteClient(pClient);
    Serial.printf("Failed to connect, deleted client\n");
    return false;
}

Serial.printf("Connected to: %s RSSI: %d\n",
       pClient->getPeerAddress().toString().c_str(),
       pClient->getRssi());

/* Now we can read/write/subscribe the charateristics of the services we are interested in */
NimBLERemoteService* pSvc = nullptr;
NimBLERemoteCharacteristic* pChr = nullptr;

pSvc = pClient->getService(SERVICE_UUID);

if (pSvc) {
    pChr = pSvc->getCharacteristic(CHARACTERISTIC_UUID);

    if (pChr) {
        // Read the value of the characteristic.
        if (pChr->canRead()) {
            std::string value = pChr->readValue();
            Serial.printf("Characteristic value: %s\n", value.c_str());
        }
    }

} else {
    Serial.printf("ABCD service not found.\n");
}

NimBLEDevice::deleteClient(pClient);
Serial.printf("Done with this device!\n");
return true;

}
void setup () {
Serial.begin(115200);
Serial.printf("Starting NimBLE Client\n");
WiFi.mode(WIFI_STA);
wifiMulti.addAP("xxx", "xxxx");
wifiMulti.addAP("ssid_from_AP_2", "your_password_for_AP_2");
wifiMulti.addAP("ssid_from_AP_3", "your_password_for_AP_3");
if(wifiMulti.run() == WL_CONNECTED) {
Serial.println("");
Serial.println("WiFi connected");
Serial.println("IP address: ");
Serial.println(WiFi.localIP());
}
/* Initialize NimBLE, no device name specified as we are not advertising */
NimBLEAddress a("00:3C:84:1D:CA:8D");
//NimBLEAddress b("00:3C:84:1B:42:33");
NimBLEDevice::init("");
NimBLEDevice::whiteListAdd(a);
//NimBLEDevice::whiteListAdd(b);

pBLEScan = NimBLEDevice::getScan();

/* create a callback that gets called when advertisers are found */
pBLEScan->setAdvertisedDeviceCallbacks(new AdvertisedDeviceCallbacks());
pBLEScan->setFilterPolicy(BLE_HCI_SCAN_FILT_USE_WL);
pBLEScan->setMaxResults(0);

/* Set scan interval (how often) and window (how long) in milliseconds */
pBLEScan->setInterval(97);
pBLEScan->setWindow(67);

/*  Active scan will gather scan response data from advertisers
 *  but will use more energy from both devices
 */
pBLEScan->setActiveScan(true);

/*  Start scanning for advertisers for the scan time specified (in seconds) 0 = forever
 *  Optional callback for when scanning stops.
 */
pBLEScan->start(scanTime, scanEndedCB);

}
void loop () {
/* Loop here until we find a device we want to connect to /
if (doConnect) {
/ Found a device we want to connect to, do it now */
if (connectToServer()) {
Serial.printf("Success!, scanning for more!\n");
} else {
Serial.printf("Failed to connect, starting scan\n");
}
    doConnect = false;
    NimBLEDevice::getScan()->start(scanTime, scanEndedCB);
}

delay(1000);
Serial.printf(".");
wifiMulti.run();

}`

How long do these devices advertise for? It could be that the scan has ended while the device is advertising.

How long do these devices advertise for? It could be that the scan has ended while the device is advertising.

Hi @h2zero .
The sensor sends an advertisement every 4 seconds without interruption.

Hello, anyone can help with this?

How long do these devices advertise for? It could be that the scan has ended while the device is advertising.

hi, can u help?

I know this is quite old but I have had some challenges with getting BLE5 working and here are some of my findings.
Enabling BLE5 gives you access to Extended Advertising that allows you to send up to 252 characters in the advertisement. However this requires that the client is running in active scanning mode. I also found out that most of the BLE scanners for Windows and IOS does not detect these extended packages but an ESP32 will so you should implement both the server and client. So the compatibility from what I have seen is not that good.
If you don't need this much then stick with compatible and use for instance the iBeacon format or the other standards that exist.
I also found out that once the advertisement has been scanned by the client the server will stop advertising to you will need to activate that again in the code if you want it permanent.  This can be done in the advertisement callback.

Closing as I believe @mp-se has answered this.


use sys executable instead of python

closes #318
will need #324 to go in first

The error if the command isn't found, should mention that the tool needs to be installed in the same env as nbqa.
I.e. if you have black installed globally:
black <some_path>

Will work, but
nbqa black <some_path>

would still fail, if nbqa is installed in a virtual env.

Maybe adding the convenience command nbqa install <tool> in conjunction with, #321 would be nice for users.

@s-weigand you're right, thanks! I've updated the error message

Codecov Report

Merging #325 into master will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##            master      #325   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files           10        10
  Lines          471       471
=========================================
  Hits           471       471

Impacted Files
Coverage Δ

nbqa/__main__.py
100.00% <ø> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 02d3b37...48a3743. Read the comment docs.


Unable to add torrentio to streamio with RD

Trying to add the Torrentio addon via web or the macos app, I get "failed to get addon manifest from torrentio.strem.fun/... manifest.json" "failed to fetch: network error when attempting to fetch resource"
In the app when using the install link it says "ERROR OPENING MEDIA - Failed to Fetch"

Seems like your network issues, as it fails to access Torrentio. Check with a different network maybe, or try changing yur DNS.

Thanks you were correct, it was a dns issue on my end.


Update TF Provider to 1.33.0

What problem are you facing?
Equinix TF Provider is on v1.33.0 with support for additional resources, datasources, and several attribute changes.

How could Equinix Provider help solve your problem?
Update to support 1.33.0

scraper: error: Failed to scrape Terraform provider metadata: cannot scrape Terraform registry: failed to scrape resource metadata from path: ../.work/equinix/equinix/docs/resources/equinix_fabric_routing_protocol.md: failed to convert example manifest to JSON: failed to format as JSON: marshal json: json: error calling MarshalJSON for type json.SimpleJSONValue: value is not known

This Markdown file contains HCL blocks with invalid documentation hints: property = <property value goes here>.. The unquoted <> hint is invalid. This file must be ignored.
diff --git a/Makefile b/Makefile
index fb74e5e..08502b1 100644
--- a/Makefile
+++ b/Makefile
@@ -9,7 +9,7 @@ GOLANGCILINT_VERSION ?= 1.50.0
 export TERRAFORM_VERSION := 1.3.1

 export TERRAFORM_PROVIDER_SOURCE := equinix/equinix
-export TERRAFORM_PROVIDER_VERSION := 1.20.0
+export TERRAFORM_PROVIDER_VERSION := 1.33.0
 export TERRAFORM_PROVIDER_DOWNLOAD_NAME := terraform-provider-equinix
 export TERRAFORM_PROVIDER_DOWNLOAD_URL_PREFIX := https://releases.hashicorp.com/${TERRAFORM_PROVIDER_DOWNLOAD_NAME}/${TERRAFORM_PROVIDER_VERSION}
 export TERRAFORM_NATIVE_PROVIDER_BINARY := ${TERRAFORM_PROVIDER_DOWNLOAD_NAME}_v${TERRAFORM_PROVIDER_VERSION}
@@ -121,7 +121,8 @@ pull-docs:
        fi
        @git -C "$(WORK_DIR)/$(TERRAFORM_PROVIDER_SOURCE)" sparse-checkout set "$(TERRAFORM_DOCS_PATH)"
        @echo "Removing examples known to cause parser errors for examples-generated"
-       @rm -f "$(WORK_DIR)/$(TERRAFORM_PROVIDER_SOURCE)/$(TERRAFORM_DOCS_PATH)/equinix_metal_port_vlan_attachment.md"
+       @rm -f "$(WORK_DIR)/$(TERRAFORM_PROVIDER_SOURCE)/$(TERRAFORM_DOCS_PATH)/equinix_metal_port_vlan_attachment.md" \
+               "$(WORK_DIR)/$(TERRAFORM_PROVIDER_SOURCE)/$(TERRAFORM_DOCS_PATH)/equinix_fabric_routing_protocol.md"

  generate.init: $(TERRAFORM_PROVIDER_SCHEMA) pull-docs


Delete API for Column level lineage

Feature
Currently, we only have delete API for Entity to Entity, we also need Delete API for column to column

PUT API will do the work for column-level lineage


branch change on disk not synced in VSC

Issue Type: Bug

Open a project with Code.
Switch to a branch with Code.
Switch to another branch in iTerm
Code does not show the right branch in the left-bottom bar.

VS Code version: Code 1.28.2 (7f3ce96ff4729c91352ae6def877e59c561f4850, 2018-10-17T00:18:43.347Z)
OS version: Darwin x64 18.0.0

Does the branch update as soon as you focus the VS Code window?
Do other events work? If you create a file (touch file) does the file explorer show it?

does not help if the focus is on:

VS Code window
Exporer
Source control

Nor by opening a file, editing the new file, etc.
touch a file works.

That's odd. If you run Git: Show Git Output, do you see any commands being run when you switch the branch and focus VS Code?

To be clear, the files in the Explorer are updated when I checkout another branch, but the current branch name is not updated in the lower-left part of the status bar.
Output of what you asked for:
Looking for git in: /usr/local/bin/git
Using git 2.19.1 from /usr/local/bin/git
> git rev-parse --show-toplevel
> git config --get commit.template
Open repository: /Users/bas/code/helloworld
> git status -z -u
> git symbolic-ref --short HEAD
> git rev-parse develop
> git rev-parse --symbolic-full-name develop@{u}
fatal: no upstream configured for branch 'develop'
> git for-each-ref --format %(refname) %(objectname) --sort -committerdate
> git remote --verbose
> git check-ignore -z --stdin

Is develop the wrong branch, in that example?

The Git output only shows what happens using VSC. If I switch branches from a shell there is no new output. (don't worry about the line about no upstream)

If I switch branches from a shell there is no new output.

Even if you focus VS Code? Even after 10 seconds? Hm...

vsc.mov.zip

Awesome, thanks for the screen recording, much easier to understand!
Anything special about the workspace path? What does cwd return? What about git rev-parse --show-toplevel?
PS: Gotta love that Cmd Shift 5, right?!

demo:helloworld bas$ pwd
/Users/bas/code/helloworld
demo:helloworld bas$ git status
On branch perl
Your branch is up to date with 'origin/perl'.

nothing to commit, working tree clean
demo:helloworld bas$ git rev-parse --show-toplevel
/Users/bas/code/helloworld
demo:helloworld bas$

Maybe... does it repro with --disable-extensions?

Makes no difference.

Does it repro on the insider release?

No it does not reproduce with the [insiders release](url
https://code.visualstudio.com/insiders).

Alright, how about if you run stable with --user-data-dir FOLDERNAME in order to run a fresh copy of Code?

No with a fresh --user-data-dir it does not reproduce. I figure it is not a bug then...

How about on another workspace?

What can I delete from ~/Library/Application Support/Code/ to get back on track?

Yes it does.

@joaomoreno to be clear, we are seeing the file event coming in (because the explorer updates), but your call to get the branch name returns the wrong result?

the label of the ui below the navigator, it does not update to the checked out branch

@bpasero Exactly. And a new user data dir fixes it. :thinking:

I do not see how the user data dir could have an impact on that since I do not know what code path this call takes.

None at all, related to the user data dir. :thinking:
@bbaassssiiee Is there anything else in the status bar which fails to update, as far as you can tell?
Can you also show us you full user settings? Maybe reset them to nothing and see if the issue still occurs?

OK to close.


Mark /mnt/disks/{sessions,uploads} as VOLUME

This should create them as directories even if they don't exist so #71 works.

Before, running apache-php's make docker would trigger:
...
[Wed May 06 07:48:40.606789 2020] [cgi:error] [pid 35:tid<PHONE_NUMBER>41504] [client <IP_ADDRESS>:40498] AH01215: [D][2020-05-06T07:48:40+0200][1] bool mnt::mountPt(mount_t*, const char*, const char*)():126 Mounting ''/mnt/disks/sessions' -> '/mnt/disks/sessions' flags:MS_BIND|MS_REC|MS_PRIVATE type:'' options:'' dir:false': /usr/lib/cgi-bin/nsjail-php-cgi
[Wed May 06 07:48:40.607295 2020] [cgi:error] [pid 35:tid<PHONE_NUMBER>41504] [client <IP_ADDRESS>:40498] AH01215: [W][2020-05-06T07:48:40+0200][1] bool mnt::mountPt(mount_t*, const char*, const char*)():166 open('/tmp/nsjail.1000.root//mnt/disks/sessions', O_CREAT|O_RDONLY|O_CLOEXEC, 0644): Is a directory: /usr/lib/cgi-bin/nsjail-php-cgi
[Wed May 06 07:48:40.608166 2020] [cgi:error] [pid 35:tid<PHONE_NUMBER>41504] [client <IP_ADDRESS>:40498] AH01215: [W][2020-05-06T07:48:40+0200][1] bool mnt::mountPt(mount_t*, const char*, const char*)():202 mount(''/mnt/disks/sessions' -> '/mnt/disks/sessions' flags:MS_BIND|MS_REC|MS_PRIVATE type:'' options:'' dir:false') src:'/mnt/disks/sessions' dstpath:'/tmp/nsjail.1000.root//mnt/disks/sessions' failed: No such file or directory: /usr/lib/cgi-bin/nsjail-php-cgi
[Wed May 06 07:48:40.608644 2020] [cgi:error] [pid 35:tid<PHONE_NUMBER>41504] [client <IP_ADDRESS>:40498] AH01215: [W][2020-05-06T07:48:40+0200][111] bool subproc::runChild(nsjconf_t*, int, int, int, int)():478 Received error message from the child process before it has been executed: /usr/lib/cgi-bin/nsjail-php-cgi
[Wed May 06 07:48:40.608956 2020] [cgi:error] [pid 35:tid<PHONE_NUMBER>41504] [client <IP_ADDRESS>:40498] AH01215: [E][2020-05-06T07:48:40+0200][111] int nsjail::standaloneMode(nsjconf_t*)():256 Couldn't launch the child process: /usr/lib/cgi-bin/nsjail-php-cgi
[Wed May 06 07:48:40.609194 2020] [cgi:error] [pid 35:tid<PHONE_NUMBER>41504] [client <IP_ADDRESS>:40498] AH01215: [D][2020-05-06T07:48:40+0200][111] int main(int, char**)():343 Returning with 255: /usr/lib/cgi-bin/nsjail-php-cgi
[Wed May 06 07:48:40.609572 2020] [cgi:error] [pid 35:tid<PHONE_NUMBER>41504] [client <IP_ADDRESS>:40498] AH01215: [F][2020-05-06T07:48:40+0200][1] bool subproc::runChild(nsjconf_t*, int, int, int, int)():448 Launching child process failed: /usr/lib/cgi-bin/nsjail-php-cgi
[Wed May 06 07:48:40.609649 2020] [cgi:error] [pid 35:tid<PHONE_NUMBER>41504] [client <IP_ADDRESS>:40498] End of script output before headers: nsjail-php-cgi

With the change it now mostly works (without the GCS/persistent storage setup). In other words, writing to these directories will still not persist any changes between requests,


Update README

Talks about Atom and Chrome debugger"?

Can VS Code use the Atom syntax coloring for HCLang?
Do we even need it, or does it basically "just work


Form ABS-EE

I'm making a request for form ABS-EE, in particular Exhibit 102. Thanks!

@statzhero Can you provide a company or two I can test this on?

Yes CIK =<PHONE_NUMBER> -- but just FYI I have implemented this otherwise in the meantime.

@statzhero I was looking through the filing types and it looks like we already have FilingType.FILING_ABSEE. Did you try using this and it didn't work as expected? If so, how did you fix it?

@statzhero I tried this using the following code and was unable to find any bugs. Please let me know if this issue persists. Closing for now.
from secedgar.filings import Filing, FilingType

filing = Filing(cik_lookup=["0001763799"], filing_type=FilingType.FILING_ABSEE)
filing.save("temp")

@statzhero I tried this using the following code and was unable to find any bugs. Please let me know if this issue persists. Closing for now.
from secedgar.filings import Filing, FilingType

filing = Filing(cik_lookup=["0001763799"], filing_type=FilingType.FILING_ABSEE)
filing.save("temp")


[DEVOPS-107] Re-enable Haddock

7G is too much for two jobs, which would be 14G worst case.

@int-index can you rebase on top of master? To see if things work now :)


feature request: transpose inputs (third format for input data - rows but without hierarchy for next columns) it's very popular and intuitive format for lists

Please add this data format also (as third option):
	- Row 1
		- Row 1 Col 2
		- Row 1 Col 3
		- Row 1 Col 4
	- Row 2
		- Row 2 Col 2
		- Row 2 Col 3
		- Row 2 Col 4
	- Row 3
		- Row 3 Col 2
		- Row 3 Col 3
		- Row 3 Col 4

( please fix me if I'm wrong in understanding that currently there only two supported formats:
I) data
	- Col 1
		- Col 1 Row 2
		- Col 1 Row 3
		- Col 1 Row 4
	- Col 2
		- Col 2 Row 2
		- Col 2 Row 3
		- Col 2 Row 4
	- Col 3
		- Col 3 Row 2
		- Col 3 Row 3
		- Col 3 Row 4

II) data rows
	- Row 1
		- Row 1 Col 2
			- Row 1 Col 3
				- Row 1 Col 4
	- Row 2
		- Row 2 Col 2
			- Row 2 Col 3
				- Row 2 Col 4
	- Row 3
		- Row 3 Col 2
			- Row 3 Col 3
				- Row 3 Col 4

Available in https://github.com/hkgnp/logseq-tablerender-plugin/releases/tag/v2.1.0


actors with the same name in different packages

What we do if e..g. two packages defined an actor with the same name?

In our packages, we have multiple user.py actors.

@despiegk what do you think?

this will be solved by actors namespaces feature on the packages (see actorsimprovement branch)


RFC: Allow to replace goal to arbitrary type when goal is represented by an axiom

Agda has one wonderful feature named rewrite rules
It allows to rewrite postulates (axioms in lean) by arbitrary terms
Example
open import Agda.Builtin.Equality
open import Agda.Builtin.Equality.Rewrite

postulate equiv : {A B : Set} -> A -> B -> Set

postulate fext : {A B : Set}(f : A -> B)(h : A -> B) -> (equiv f h) ≡ (∀ (a : A) -> f a ≡ h a)
{-# REWRITE fext #-}

id1 : {T : Set} -> T -> T
id1 v = v
id2 : {T : Set} -> T -> T
id2 v = v

test16 : {T : Set} -> equiv (id1 {T}) (id2 {T})
test16 = λ i -> refl

The code above is accepted by agda.
If I try to do similar thing in lean it sorta works
axiom equiv {A B:Type} : A -> B -> Type

def fext_impl : (A : Type) -> (B : Type) -> (A -> B) -> (A -> B) -> Prop :=
  fun A _ f h => (i:A) -> f i = h i

elab "fext" : tactic => do
  Elab.Tactic.withMainContext do
    let goal_ix <- Elab.Tactic.getMainGoal
    let goal <-  goal_ix.getType'
    let (lt, rt, l, r) <- match goal with
      | .app (.app (.app (.app (.const `equiv []) lt) rt) l) r => pure (lt,rt,l, r)
      | _ => throwTacticEx `fext goal_ix "Invalid goal type"
    let s <- saveState
    let same <- isDefEq lt rt
    if same then do
      let (lt, rt) <- match Expr.arrow? lt with
      | .some ts => pure ts
      | .none => throwTacticEx `fext goal_ix "given item is not a function"
      let eq := .app (.app (.app (.app (.const `fext_impl []) lt) rt) l) r
      let eq <- reduceAll eq
      let goal <- MVarId.change goal_ix eq false
      Elab.Tactic.setGoals [goal]
    else
      restoreState s
      throwTacticEx `fext goal_ix "Given functions differ in types"

def id1 : A -> A := fun i => i
def id2 : A -> A := fun i => i

example : (T:Type) -> equiv (@id1 T) (@id2 T) := fun T => by
  fext
  intro i
    /- tactic state
    T : Type
    i : T
    ⊢ i = i
    -/
  -- rfl -- (HERE)

The state after applying fext is what is expected to be, but if I uncomment HERE, lean reports an error
application type mismatch
  id ⋯
argument has type
  ∀ (i : T), i = i
but function has type
  equiv id1 id2 → equiv id1 id2

It seems to me that this code is to blame.
It has to be changed to match behaviour of agda, that is
Allow to replace goal to arbitrary type when goal is represented by an axiom

Thanks for the report. There is a lot of meta code, and it's not clear to me if the issue you would like to report cannot be minimized further, e.g. to rule out a simple bug in the meta code producing ill-typed expressions.
Can you replace your tactic invocation with refine? Maybe show_term can help with that? If not, then why?

Also I see two postulates in the agda code, and an confused that your lean equivalent doesn't have two. What happened to postulate fext?


[octavia-ingress-controller]: update label from master to excludeLB

What this PR does / why we need it:
Which issue this PR fixes(if applicable):
fixes #1724
Special notes for reviewers:

Release note:

NONE

/test pull-cloud-provider-openstack-check

/lgtm

/approve


Adding section regarding commit message format to CONTRIBUTING.md.

Checklist:

[x] Did you sign the Contributor License Agreement?
[x] Is the linter reporting 0 errors?
[ ] Does the CI pipeline pass?
[x] Did you update the documentation?

Codecov Report

Merging #106 (0230b6f) into main (d1857b9) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main     #106   +/-   ##
=======================================
  Coverage   94.82%   94.82%
=======================================
  Files          65       65
  Lines        1680     1680
=======================================
  Hits         1593     1593
  Misses         87       87

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5a5b010...0230b6f. Read the comment docs.


not compatible with eslint 9+?

I think this is because the plugin does not appear to be compatible with eslint 9?
Error: Could not warm up worker. Formatting a file for the first time may take longer than usual.
Message: Invalid Options:
- Unknown options: ignorePath, resolvePluginsRelativeTo, rulePaths, useEslintrc
- 'resolvePluginsRelativeTo' has been removed.
- 'ignorePath' has been removed.
- 'rulePaths' has been removed. Please define your rules using plugins.
Stacktrace: Error: Invalid Options:
- Unknown options: ignorePath, resolvePluginsRelativeTo, rulePaths, useEslintrc
- 'resolvePluginsRelativeTo' has been removed.
- 'ignorePath' has been removed.
- 'rulePaths' has been removed. Please define your rules using plugins.
    at processOptions (/node_modules/eslint/lib/eslint/eslint-helpers.js:839:15)
    at new ESLint (/node_modules/eslint/lib/eslint/eslint.js:593:34)
    at getESLint (/node_modules/prettier-eslint/dist/utils.js:337:12)
    at getESLintConfig (/node_modules/prettier-eslint/dist/index.js:268:39)
    at analyze (/node_modules/prettier-eslint/dist/index.js:80:77)
    at format (/node_modules/prettier-eslint/dist/index.js:50:13)
    at file:///home/node/.vscode-server/extensions/rvest.vs-code-prettier-eslint-6.0.0/dist/worker.mjs:1:1516

Here's a github issue having the same errors due to version incompatibility, hence why I assume this is the reason:
https://github.com/vercel/next.js/issues/64409

Confirmed here too

ESLint version 9 deprecated .eslintrc files in favor of what they call "flat configs" which essentially are just javascript objects with a similar but slightly different structure. https://eslint.org/docs/latest/use/configure/configuration-files
If you're trying to use the new config format, it won't work until the extension is updated.
You should be able to do one of the following to get things working:

Set ESLINT_USE_FLAT_CONFIG=false in a .env file
Downgrade ESLint to version 8

Wow! I hope it can be supported.

FWIW I use this setting now: "editor.defaultFormatter": "esbenp.prettier-vscode", no need for the extra vscode extension. Just configure eslint correctly with the prettier rules and use prettier-vscode as your formatter.

FWIW I use this setting now: "editor.defaultFormatter": "esbenp.prettier-vscode", no need for the extra vscode extension. Just configure eslint correctly with the prettier rules and use prettier-vscode as your formatter.

But by doing that you're not using eslint anymore, only prettier

FWIW I use this setting now: "editor.defaultFormatter": "esbenp.prettier-vscode", no need for the extra vscode extension. Just configure eslint correctly with the prettier rules and use prettier-vscode as your formatter.

But by doing that you're not using eslint anymore, only prettier

I use eslint to tell me things are wrong, but not formatting. That's right. There's a couple of rules I have to get vscode to fix for me.


Missing pypi license

Tools like liccheck  now report this library has an UNKNOWN license, also pypi doesn't show a license.

@matthewdeanmartin thank you for the report. It was indeed missing license keyword.


Error on line number which doesn't exists

Hey,
it is giving an error at line number which doesn't even exists
Files below has/have JSON syntax error (of 2 files)
- /github/workspace/myfile.json: 451: unexpected token at '{' ...

But the file has only 314 lines.
this is my action file
    steps:
      - uses: actions/checkout@v2
      - name: Validate JSON
        uses: limitusus/json-syntax-check@v2
        with:
          patterns: "\\.json$"

Do we need to add paths also..?
    paths:
      - '**.json'

Please help me . Thanks

I can't help, since the output is from Ruby's JSON.#parse implementation.


Initial wallectconnect protocol

@LeeAlephium @mvaivre @nop33 (alphabetic order)

@LeeAlephium I am merging this now. If you have any feedback, we could create another PR.


Upgrade rack-protection gem

There is a security vulnerability for rack-protection: CVE-2018-7212 - Path traversal on Windows. See https://gemnasium.com/psu-stewardship/scholarsphere/alerts#advisory_1318

@awead was this resolved?  I do not see any errors on Gemanisum.

@cam156 I see no commit that resolved it, but Gemnasium isn't complaining; neither is Github, which is the new reporting means for this now. I'm going to close this.


Host StorageProvider lookup includes port if not using :80 or :443

@bep
When running a local server with Listen set to :8080 and "Hosts": {  "localhost": ... }, the StorageProvider lookup fails. Changing to "Hosts": {  "localhost:8080": ... } makes it work.
Since all Hosts share the same port, this is unnecessary.
I changed this in commit https://github.com/alexandres/poormanscdn/commit/cc5851e3857df138fce8690c896a694e589368cf

Didn't test that. On a side note, I really like how easy https/virtual hosts looks in the updated README.
Re. the config examples, I assume that you know that this:
"Hosts": {
	"myblog.com": { "Bucket": "blogs", "Path": "my", "AccessKey": "", "SecretKey": "" },
	"familyblog.com": { "Bucket": "blogs", "Path": "family", "AccessKey": "", "SecretKey": "" },
	"carrentals.com": { "Bucket": "carrentals", "Path": "", "AccessKey": "", "SecretKey": "" }
}

Is the same as:
"Hosts": {
	"myblog.com": { "Bucket": "blogs", "Path": "my" },
	"familyblog.com": { "Bucket": "blogs", "Path": "family" },
	"carrentals.com": { "Bucket": "carrentals" }
}

I know it us useful to show and explain all options in one section, but the above does look better in this concrete example.


run zero-device test only on bots

When running tests locally a developer almost always has some device connected.

LGTM


The folder is not deleted

I am trying the steps from TL;DR, I wait until pod and pvc are both terminated, the folder with file is left on disk. I don't know if it worked in original example, I could not compile it, but yours does not work as expected.

I expect my cluster to not automatically delete data, so it works as I expected it.
The Google VCE Disk Provisioner for example does also retain the actual Data on disk.
On the other hand I have not used kubernetes or this plugin for quite a long time and so might very well be wrong.


Inform user and update documentation with installation folder

Hi,
Could be that I'm just blind today, but after installation, I had to spent 5 minutes googline to find out where the GUI was installed. When the GUI is installed, there was no feedback, not even a succes or failure message, let alone any information about any shortcuts being created or where to find the installation or how to start it.
Would be nice to provide some more feedback to users. It can help new users to get started easily with a smooth experience.
Valentijn

Thanks for reaching out!
Is C:\Program Files (x86)\ChocolateyGUI not a reasonably "normal" installation path?
Also, the ChocolateyGUI installer adds a Start Menu shortcut on installation, so you should be able to hit the Start Button, type ChocolateyGUI and it will be immediately available in the list.

Hmm totally missed that. I guess I saw some earlier reports that there was no start menu entry, so didn't look for it. Oops.

I take your point about adding some documentation about this though.

And maybe some kind of message indicating the installation was completed succesfully or something, or did I miss that one as well?

Did you use Chocolatey to install ChocolateyGUI?

Yes, so for the GUI it's OK.

Closing this for now.


Added screenshots for clarity

Based on product documentation feedback, added Desktop App server management screenshots to supplement the existing server management procedural steps and provide additional UI context.

@devinbinnie - This PR includes updates to address https://github.com/mattermost/desktop/pull/1926, so I've included you as a dev reviewer. I've made a number of small updates regarding keyboard shortcuts that will benefit from your review. Thanks!


run_once argument when registering callbacks

We want to do things like send out an RTU new_delta_request event and be able to await an asyncio Event/Future that is resolved by a callback that sees new_delta_event with the same transaction id as the request. After getting that a single time though, we can drop the callback

Codecov Report

Merging #107 (637633f) into main (5158dc6) will increase coverage by 0.08%.
The diff coverage is 91.66%.

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

@@            Coverage Diff             @@
##             main     #107      +/-   ##
==========================================
+ Coverage   93.52%   93.61%   +0.08%
==========================================
  Files           8        8
  Lines         510      517       +7
==========================================
+ Hits          477      484       +7
  Misses         33       33

Impacted Files
Coverage Δ

sending/base.py
92.72% <91.66%> (+0.20%)
:arrow_up:

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5158dc6...637633f. Read the comment docs.


Add support for Brazilian

KeyboardKit should have support for Brazilian.

This can be tested in the master branch once the demo uses the new versions.


update: handle processes sending open file descriptors via SCM_RIGHTS

What type of PR is this?

Uncomment one (or more) /kind <> lines:

/kind bug

/kind cleanup

/kind design

/kind documentation

/kind failing-test

/kind feature

Any specific area of the project related to this PR?

Uncomment one (or more) /area <> lines:

/area API-version

/area build

/area CI

/area driver-kmod
/area driver-bpf
/area driver-modern-bpf

/area libscap-engine-bpf

/area libscap-engine-gvisor

/area libscap-engine-kmod

/area libscap-engine-modern-bpf

/area libscap-engine-nodriver

/area libscap-engine-noop

/area libscap-engine-source-plugin

/area libscap-engine-savefile

/area libscap-engine-udig

/area libscap

/area libpman

/area libsinsp

/area tests

/area proposals

Does this PR require a change in the driver versions?

/version driver-API-version-major

/version driver-API-version-minor

/version driver-API-version-patch

/version driver-SCHEMA-version-major

/version driver-SCHEMA-version-minor

/version driver-SCHEMA-version-patch

What this PR does / why we need it:
When using unix sockets, processes can pass open file descriptors to one another by using sendmsg and recmsg syscalls. This PR attempts to reflect this in the sinsp state. Whenever the recvmsg contains ancillary data, it checks for SCM_RIGHTS. If so, we proceed with a proc scan of the file descriptors of the receiving process, since it will have new open file descriptors that we missed.
Since we only want to update the file descriptors, this PR introduces a new platform API to scan only that and avoid parsing information about the receiving process that we already have in the sinsp state. This, plus the fact that we can consider the frequency of SCM_RIGHTS messages pretty low, should mean that we are not introducing a big performance penalty.
Which issue(s) this PR fixes:
https://github.com/falcosecurity/libs/issues/1364

Fixes #
Special notes for your reviewer:
Does this PR introduce a user-facing change?:

NONE

I triggered a run of kernel testing framework against HEAD of the branch: https://github.com/falcosecurity/libs/actions/runs/6493764036

Pushed upstream branch to test: https://github.com/falcosecurity/libs/commits/PR1400; new run: https://github.com/falcosecurity/libs/actions/runs/6493859950

Thanks Fede!

 Sometimes the recvmsg event can't be found, which is really weird

LGTM label has been added.  Git tree hash: 867e8603ae65d8adbb184f842155c5f79276ee57

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: Andreagit97, FedeDP, loresuso
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [Andreagit97,FedeDP]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

/hold cancel


Documentation

All dependencies are already ready for writing Swagger documentation for the API as well as wiki

~10k LoC and still no documentation
when documentation

Stay in tune. Documentation is pointless without the base implementation. Although, it's next in the project backlog.

@i-spin Documentation is still WIP, but you can review the getting started guide I made. Suggest your improvements right here.
https://d1snin.gitbook.io/linda/guides/getting-started-with-linda


Fix messages are not mark as read if list is short. Make getStartAndEndNodes more reliable. Fix #2988

Screencast
https://cl.ly/579af5465bb9
https://cl.ly/8b85cfee1fe7
Fix #2988

Thanks @jainkuniya for taking a shot at this!
This logic is pretty convoluted, unfortunately. I think it shouldn't have to be appreciably more complicated than the existing logic it replaces.
I have a feeling that at this point the quickest way to communicate what I have in mind will be for me to just write it up myself, and you to read that; so I'm going to close this PR, and plan to do that soon.
A couple of specific things that jump out at me:

I don't know what a "doc-level node" is. If anything it sounds like the whole document, whereas this function usually returns a child of the body.
The recursion should be a loop; we discussed this on another PR recently.
The use of node[property] to get either node.previousSibling or node.nextSibling is overly magical.

I was going to comment on the original PR.
Since you are working on this code, you might want to do that change:
getMessageNode called directly is not needed in most (or all) cases now that we have getMessageIdFromNode (which of course uses that function)
So rework an callers of getMessageNode that then call getMessageIdFromNode to call the second function directly.


Implement asset conversion in transactions

Context
Currently, Midaz supports transactions involving a single asset only. However, since the Ledger is a multi-asset system, we are implementing the ability to perform asset conversions during a transaction.
To facilitate this, Midaz will offer two methods for specifying conversion rates:
1. Providing conversion rates during transaction creation
This method is recommended as it allows Midaz to utilize the most up-to-date conversion rates available. When creating a transaction, the user must include the conversion rates for operations between the assets of the accounts involved. These rates will be used directly by Midaz for transaction processing.
2. Pre-registering conversion rates in the asset-rates table
This option is better suited for cases where the conversion rate between two assets is fixed. When creating a transaction, the user does not need to provide conversion rates. Instead, if a conversion is required, Midaz will query the asset-rates table to determine the appropriate conversion rates.
This method can also act as a fallback for the first option. For instance, if the conversion rate provider is unavailable at the time of transaction creation, Midaz can refer to the asset-rates table to ensure continuity.
Responsibilities and Error Handling

The user is responsible for maintaining and updating the asset-rates table with accurate rates.
If the required asset pair is not found in the asset-rates table, the transaction will be rejected, and an error will be returned to the user.

Requirements
1. Automatic reconciliation with external accounts
By default, conversions will be processed in external world. Consequently, Midaz must automatically create operations to reconcile the values of external accounts. This ensures consistency between the values received from external world and those stored in the Ledger’s accounts.
2. Dual option for the asset in the send field
For transactions involving multiple assets, the send field must support two scenarios:
1.	Sending an amount in the asset of the source account.
2.	Sending an amount in the asset of the destination account.
Midaz must determine whether to calculate the debit amount in the source account or the credit amount in the destination account based on the specified conversion rates.
3. Determination of Conversion Rate Lookup
Midaz will always use the asset specified in the send field as the origin (from) for conversion rate lookups. For example:

When sending BRL to a USD account, Midaz will look up the conversion rate from BRL to USD.
If the rate in this order (from: BRL, to: USD) is not found in the database, the transaction will fail.

4. Conversion Rate Processing Workflow
Midaz will follow the workflow below to process conversion rates:
1. Provided Rates in Transaction:

If conversion rates are included in the transaction object, they will be used directly, ignoring the asset-rates table.
All required conversion rates must be provided in the transaction. Partial rates will not be accepted.

2. Fallback to asset-rates Table:

If conversion rates are not provided in the transaction object, Midaz will query the asset-rates table for the necessary rates.

3. Error Handling:

If any required rate is not found (either in the transaction object or in the asset-rates table), the transaction will fail, and an error will be returned to the user.

The ASSETS of From or To must always match those of SEND.
Only REMAINING and SHARE/SHARE OF will use RATE.
Is the account's ASSET always the truth?
If the RATE field is provided, should it be saved/updated in the database?
When the RATE field is not provided and the account's ASSET differs from the ASSET of SEND, retrieve it from the database.
If the ASSET of From or To differs from the ASSET of SEND, AMOUNT is not allowed.
When the ASSET of SEND is USD, for example, and To has an ASSET of BRL, validate if the distribution equals 100% of the value only and not the total in conversion.


Make it easier to use for macOS users

MR:thread-315383
Also, improve instructions, maybe add a .desktop for Linux users.

Provided the exec bit is set, a shell script with a .script extension will be double-clickable, run in Terminal, won't close by itself.
The only quirk you have to account for is that PWD is enforced to HOME ;).

That still leaves GateKeeper to deal with, so, not actually double click able in practice... Right click > open > gatekeeper pop-up

Hmm. I'll try this in the next version and see if I can do anything about GateKeeper.

@NiLuJe on a slightly unrelated note, what do you think of merging the kobopatch binaries into a fat hybrid sh-batch script so there's only one binary to run (this might be a really, really bad idea...)?

In practice, most people will only ever run the batch/shell script, so I'm not sure what the interest would be, unless I'm missing something ;).
Ironically enough, on macOS, I don't think (I'll check tomorrow) GateKeeper shouts at you for running an untrusted binary from a shell script once you've authorized the shell-script itself, so that wouldn't even help with stuff like that.

@NiLuJe It was just a curiosity, but it'd be nice to have less files for the sake of minimalism. I think the cost of it would be too high in this case, though. It would also allow me to simplify the instructions, though.
I just checked the GateKeeper thing in my Mojave VM, and it seems like this wouldn't change anything. I did notice that running the binary directly needs an admin password, unlike running the shell script by double-clicking a *.command file. For both methods, you still have to appease GateKeeper in the settings panel.

Fun fact: making the .command a symlink doesn't work ;). Clicking that would open the default app for shell-scripts (f.g., XCode).

Also, on a slightly related note about the threads themselves:

There's no mention of macOS at all in the bit that points the user to the wrapper script ;).
The link to the matching kobopatch zip for the version the thread targets could be automatically put in there, the GH Release page might be a bit scary/confusing for some...

There's no mention of macOS at all in the bit that points the user to the wrapper script ;).

That was a bit of an oversight on my part while I was writing the new instructions. I assumed everyone knew macOS was unix-like ;).

The link to the matching kobopatch zip for the version the thread targets could be automatically put in there, as the GH Release page might be a bit scary/confusing for some...

I could do that. The main reasons why I link to the releases page is so people can see the release notes without me having to convert it to bbcode (and hopefully the savvy users can also catch on to the fact they can pull in-progress stuff from GitHub), and so I don't need to edit every post on each release (MR started limiting the number of edits you can do per day, and how long after a post you can edit it). It also makes the post quote-friendly. I guess one way I could solve this is with a simple proxy to redirect to a specified file in the latest GitHub release for a repo.

Yeah, I was just thinking that getting people to see the release notes was the reasoning behind it as I wrote that ;p.

@NiLuJe, oh no... https://developer.apple.com/news/?id=09032019a.

@geek1011: That only applies to signed binaries.

Whee, realized there was a nasty thinko in my original comment. I meant .command. .script is for AppleScript -_-".

@geek1011: Re: the thread linking to the Release without a direct link to the zip: someone was just bit the asset list not being sorted naturally ;). (This was one of the many things I'd expected people to get wrong with the current approach, kinda surprised it even took this long ^^).

I'm surprised it took that long too, TBH, although you'd expect people with e-readers to be literate :P
I could link to the new kobopatch webpage, which has it sorted properly, but it doesn't work on older browsers (it uses newer JS APIs like fetch).
I'll probably put together another download page (either using more compatible JS or rendered server-side) which pulls the data through the GH API and puts it in a more user friendly form.
Do you have any other suggestions for that page when I put it together?

None that I think of right now ;).
On a slihgtly related subject, I should get around to actually do/test the macOS .command thingy because it's extremely trivial ;p.

@NiLuJe, does kobopatch still work for you after https://developer.apple.com/news/?id=12232019a?

That's for App Store submissions only ;).

What's this about, then:

In June, we announced that all Mac software distributed outside the Mac App Store must be notarized by Apple in order to run by default on macOS Catalina. In September, we temporarily adjusted the notarization prerequisites to make this transition easier and to protect users on macOS Catalina who continue to use older versions of software. Starting February 3, 2020, all submitted software must meet the original notarization prerequisites

I take that "by default" to mean without a Gate Keeper warning.

Yup, business as usual ;).

Since nothing's completely broken with the recent macOS updates AFAIK, I'm going to close this issue for now.


[BUG] Unable to stop VM which in starting state

Describe the bug
Stop button been disabled when the VM in a long starting state.

To Reproduce
Steps to reproduce the behavior:

Install Harvester with any nodes
Create an Windows iso image for VM creation
Create the Windows VM by using the iso image
When the VM in Starting state, Stop button should able to click and work as expected

Expected behavior
VM should be able to stop in Starting state
Environment:

Harvester ISO version: v1.0.2-rc1
Underlying Infrastructure (e.g. Baremetal with Dell PowerEdge R630): **Baremetal DL360 3nodes **

Verified Stop button available and work as expected.
Test Information

Environment: qemu/KVM 3 nodes
Harvester Version: master-a242948b-head
ui-source Option: Auto

Verify Steps:

Follow Steps to reproduce in https://github.com/harvester/harvester/issues/2263#issue-1232274843


Support for EPiServer CMS 11

Please add support for EPiServer CMS 11.
Thanks for a great addon!
/Kristoffer

New version is out: http://nuget.episerver.com/en/OtherPages/Package/?packageId=BVN.404Handler&packageVersion=11.0.0


Item views

Task # 84(task number on taiga board)
Proposed Changes

Support Item views for lend/borrow items
Unit testing for the new functionality

New endpoints:
POST - /lendItem/:id/incrementViewCount
POST - /borrowItem/:id/incrementViewCount

👍


Re-enable GitHub CI checks on macOS.

Re-enabling checks disabled in #997.
See, https://github.community/t5/GitHub-Actions/macOS-VM-failing-immediately/td-p/36969/page/2
Related-to: HARP-7766

Codecov Report

Merging #1013 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #1013   +/-   ##
=======================================
  Coverage   45.19%   45.19%
=======================================
  Files         249      249
  Lines       22321    22321
  Branches     5055     5055
=======================================
  Hits        10087    10087
  Misses      12234    12234

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5cc9739...e2a900e. Read the comment docs.