Trouble using nixos-containers programmatically as non-wheel user

Issue description
Maybe I'm looking at this wrong, but it seems like there are some security issues with certain use cases of containers. You've gotta be in a wheel group to use nixos-container run, for instance, and it would be quite ideal for our particular use case if non-admin users could make use of nixos-containers from a programmatic standpoint.
nixos-container login currently seems to be the only way to do this, which is fairly interactive in nature.
I might be able to work on a solution for this, but not sure it will be very simple. Best thing I can think of at the moment, which isn't perfect:

Install ssh in container.
Create wrapper/utility to inject ssh authorized keys into container user that host user has the private key for.
Create a wrapper command to automatically call ssh as desired to get into the container user from the host.

From what I could tell (which could be wrong) systemd doesn't seem to provide a more direct way to achieve non-password based authentication.
tech  details:

system: "x86_64-linux"
host os: Linux 4.14.83, NixOS, 18.03.133379.9c96d120aa7 (Impala)
multi-user?: yes
sandbox: no
version: nix-env (Nix) 2.0.4
channels(nixos): ""
channels(root): "nixos-18.03.133379.9c96d120aa7"
nixpkgs: /nix/var/nix/profiles/per-user/root/channels/nixos/nixpkgs

See upstream issue https://github.com/systemd/systemd/issues/10997


Make lovrViewID signed on Oculus Mobile (fix "mask" example)

The "mask" example was failing on Oculus Mobile on the line of glsl:
if (lovrViewID == 1) {
because lovrViewID was unsigned and 1 was signed. One way to fix this would be to replace 1 with 1u as that is unsigned, but this would be the wrong fix because lovrViewID is in fact signed on all platforms other than Oculus Mobile. lovrViewID was depending on platform defined to either gl_ViewportIndex (signed), a signed uniform, or gl_ViewID_OVR (unsigned). The solution is to place an explicit cast in the multiview definition of lovrViewID so that it is signed on all platforms.

Only thing more fun than putting dots and us after all the numbers is explicit constructors...


[Hub Generated] Review request for Microsoft.CostManagement to add version stable/2019-11-01

If you are a MSFT employee you can view your work branch via this link.
Contribution checklist:

[ ] I have reviewed the documentation for the workflow.
[ ] Validation tools were run on swagger spec(s) and have all been fixed in this PR.

/azp run automation - sdk

Can one of the admins verify this patch?

/azp run automation - sdk

/azp run automation - sdk

Hi @elabicha, unfortunately there're conflicts. please resolve it.

/azp run automation - sdk


Support parallel writes to zarr store

MCVE Code Sample
import multiprocessing
import xarray as xr
import numpy as np
from s3fs import S3FileSystem, S3Map
from time import sleep

from main import set_aws_credentials
set_aws_credentials()

def download_file(lead_time):
    return 'path_to_your_file'

def make_xarray_dataset(file_path, lead_time):
    var1 = np.random.rand(1, 721, 1440, 22)
    var2 = np.random.rand(1, 721, 1440, 22)
    lat = np.linspace(-90, 90, 721)
    lon = np.linspace(0, 360, 1440)
    height = range(22)
    ds = xr.Dataset({'var1': (['lead_time', 'lat', 'lon', 'height'], var1),
                     'var2': (['lead_time', 'lat', 'lon', 'height'], var2)},
                    coords={'lat': lat,
                            'lon': lon,
                            'height': height,
                            'lead_time': [lead_time]})
    return ds

def upload_to_s3(dataset, append):
    s3 = S3FileSystem()
    s3map = S3Map('S3_path_to_your_zarr', s3=s3)
    # If we are appending to an already existing dataset
    if append:
        dataset.to_zarr(store=s3map, mode='a', append_dim='lead_time')
    else:
        dataset.to_zarr(store=s3map, mode='w')

def lead_time_worker(lead_time, append=True):
    file_path = download_file(lead_time)
    dataset = make_xarray_dataset(file_path, lead_time)
    upload_to_s3(dataset, append=True)
    return 0

if __name__ == '__main__':
    lead_times = range(10)
    first_lead_time = True
    processes = []
    for lead_time in lead_times:
        if first_lead_time:
            process = multiprocessing.Process(target=lead_time_worker,
                                              args=(lead_time, False))
            process.start()
            process.join()
            first_lead_time = False
        else:
            process = multiprocessing.Process(target=lead_time_worker,
                                              args=(lead_time,))
            process.start()
            processes.append(process)
            sleep(5) # Sleep in order to shift the different processes so that they don't begin at the same time
    for p in processes:
        p.join()

will raise

ValueError: conflicting sizes for dimension 'lead_time': length X on 'Var1' and length Y on 'Var2'

Traceback (most recent call last):
  File "main.py", line 200, in lead_time_worker
    upload_to_s3(dataset, cloud_zarr_path, append=True)
  File "main.py", line 167, in upload_to_gcloud
    ds.to_zarr(store=s3map, mode='a', append_dim='lead_time')
  File "/home/ubuntu/anaconda3/envs/GFS-data-retrieval/lib/python3.7/site-packages/xarray/core/dataset.py", line 1414, in to_zarr
    consolidated=consolidated, append_dim=append_dim)
  File "/home/ubuntu/anaconda3/envs/GFS-data-retrieval/lib/python3.7/site-packages/xarray/backends/api.py", line 1101, in to_zarr
    dump_to_store(dataset, zstore, writer, encoding=encoding)
  File "/home/ubuntu/anaconda3/envs/GFS-data-retrieval/lib/python3.7/site-packages/xarray/backends/api.py", line 929, in dump_to_store
    unlimited_dims=unlimited_dims)
  File "/home/ubuntu/anaconda3/envs/GFS-data-retrieval/lib/python3.7/site-packages/xarray/backends/zarr.py", line 354, in store
    ds = open_zarr(self.ds.store, chunks=None)
  File "/home/ubuntu/anaconda3/envs/GFS-data-retrieval/lib/python3.7/site-packages/xarray/backends/zarr.py", line 557, in open_zarr
    ds = maybe_decode_store(zarr_store)
  File "/home/ubuntu/anaconda3/envs/GFS-data-retrieval/lib/python3.7/site-packages/xarray/backends/zarr.py", line 545, in maybe_decode_store
    drop_variables=drop_variables)
  File "/home/ubuntu/anaconda3/envs/GFS-data-retrieval/lib/python3.7/site-packages/xarray/conventions.py", line 527, in decode_cf
    ds = Dataset(vars, attrs=attrs)
  File "/home/ubuntu/anaconda3/envs/GFS-data-retrieval/lib/python3.7/site-packages/xarray/core/dataset.py", line 423, in __init__
    self._set_init_vars_and_dims(data_vars, coords, compat)
  File "/home/ubuntu/anaconda3/envs/GFS-data-retrieval/lib/python3.7/site-packages/xarray/core/dataset.py", line 445, in _set_init_vars_and_dims
    data_vars, coords, compat=compat)
  File "/home/ubuntu/anaconda3/envs/GFS-data-retrieval/lib/python3.7/site-packages/xarray/core/merge.py", line 379, in merge_data_and_coords
    indexes=indexes)
  File "/home/ubuntu/anaconda3/envs/GFS-data-retrieval/lib/python3.7/site-packages/xarray/core/merge.py", line 460, in merge_core
    dims = calculate_dimensions(variables)
  File "/home/ubuntu/anaconda3/envs/GFS-data-retrieval/lib/python3.7/site-packages/xarray/core/dataset.py", line 125, in calculate_dimensions
    (dim, size, k, dims[dim], last_used[dim]))
ValueError: conflicting sizes for dimension 'lead_time': length X on 'var1' and length Y on 'var2'

Problem Description
First of all, thanks a lot to the community for the PR #2706, I was really looking forward to it. I already experienced using the new append parameter, and got some problems trying to do it in a parallel way.
I want to upload a very big zarr (global numerical weather prediction, output of GFS model that you can check out here) on a S3 bucket. In order to speed this up, as each single file of the source contains the data for one lead time(length of time between the issuance of a forecast and the occurrence of the phenomena that were predicted) and I want to concatenate them all, I tried to have one process per lead time and all of them to append to the same data store using Dataset.to_zarr() with append=True.
However, when doing that, I get the error described above. Indeed, the processes are appending simultaneously, so the data is not necessarily consistent when a new process tries to append, some variables will already have the values of one lead time and some will not because the process is not finished, which will lead to calculate_dimensions() raising this error.
I wonder if there is a way I haven't found to work around this using simply a synchronizer? If not, do you think it would be possible (and reasonable) to implement a parameter allowing to bypass this check on the append dimension in an 'eventually consistent' approach?
 Thanks for being an early adopter! We really appreciate your feedback. I'm sorry it didn't work as expected. We are in really new territory with this feature.
I'm a bit confused about why you are using the multiprocessing module here. The recommended way of parallelizing xarray operations is via the built-in dask support. There are no guarantees that multiprocessing like you're doing will work right. When we talk about parallel append, we are always talking about dask.
Your MCVE is not especially helpful for debugging because the two key functions (make_xarray_dataset and upload_to_s3) are not shown. Could you try simplifying your example a bit? I know it is hard when cloud is involved. But try to let us see more of what is happening under the hood.
If you are creating a dataset for the first time, you probably don't want append. You want to do
ds = xr.open_mfdataset(all_the_source_files)
ds.to_zarr(s3fs_target)

If you are using a dask cluster, this will automatically parallelize everything.

Hi @VincentDehaye. Thanks for being an early adopter! We really appreciate your feedback. I'm sorry it didn't work as expected. We are in really new territory with this feature.
I'm a bit confused about why you are using the multiprocessing module here. The recommended way of parallelizing xarray operations is via the built-in dask support. There are no guarantees that multiprocessing like you're doing will work right. When we talk about parallel append, we are always talking about dask.
Your MCVE is not especially helpful for debugging because the two key functions (make_xarray_dataset and upload_to_s3) are not shown. Could you try simplifying your example a bit? I know it is hard when cloud is involved. But try to let us see more of what is happening under the hood.
If you are creating a dataset for the first time, you probably don't want append. You want to do
ds = xr.open_mfdataset(all_the_source_files)
ds.to_zarr(s3fs_target)

If you are using a dask cluster, this will automatically parallelize everything.

Hi @rabernat, thank you for your quick answer. I edited my MCVE so that you can reproduce the error(as long as you have access to a S3 bucket). I actually forgot about open_mfdataset, that's why I was doing it this way. However in the future I would still like to be able to have standalone workers, because the bandwidth quickly becomes a bottleneck for me (both on downloading the files and uploading to the cloud) so I would like to split the tasks on different machines.
With regards to open_mfdataset(), I checked the code and realized under the hood it's only calling multiple open_dataset(). I was worried it would load the values (and not only metadata) in memory, but I checked it on one file and it apparently does not. Can you confirm this ? In this case I could probably open my whole dataset at once, which would be very convenient. After reading your issue #1385, I also need to check that my case works fine with decode_cf=False. I experienced some troubles with the append on a time dimension but found a workaround, I will probably open another issue for documenting this.

With regards to open_mfdataset(), I checked the code and realized under the hood it's only calling multiple open_dataset(). I was worried it would load the values (and not only metadata) in memory, but I checked it on one file and it apparently does not. Can you confirm this ? In this case I could probably open my whole dataset at once, which would be very convenient.

Yes, this is the suggested workflow! open_mfdataset opens a collection of files lazily (with dask) into a single xarray dataset, suitable for converting into zarr all at once with to_zarr().
It is definitely possible to create a zarr dataset and then write to it in parallel with a bunch of files, but not via xarray's to_zarr() method. You would have to create the dataset and write to it with the zarr Python API directly.

Coming back on this issue (still haven't had time to try the open_mfdataset approach), I have another use case where I would like to store different variables being indexed by the same dimension, but not all available at the same moment.
For example, I would have variables V1 and V2 indexed on dimension D1. V1 would be available at time T, and I would like to store it in my S3 bucket at this moment, but V2 would only be available at time T+1. In this case, I would like to be able to save the values of V2 at time T+1, leaving the missing V2 values filled with the fill_value specified in the metadata.
What actually happens is that you can append such data, but then if you want to open the resulting zarr the open_zarr function needs to be given V2 as value for its drop_variables argument, otherwise you get the error shown in my original post. However, as the open_zarr function is called when appending as well (cf. original post's error trace), and in this case you can not provide this argument, you will fail the next append attempts, thus preventing you from appending the values of V2. Your dataset is now frozen.
Am I misusing the functionality, or do you know any workaround using xarray and not coding everything myself (for optimization reasons)?

@VincentDehaye - we are eager to help you. But it is difficult to hit a moving target.
I would like to politely suggest that we keep this issue on topic: making sure that parallel append to zarr store works as expected. Your latest post revealed that you did not try our suggested resolution (use open_mfdataset + dask parallelization) but instead introduced a new, possibly unrelated issue.
I recommend you open a new, separate issue related to "storing different variables being indexed by the same dimension".

Coming back on this issue in order not to leave it inactive and to provide some feedback to the community.
The problem with the open_mfdataset solution was that the lazy open of a single lead time dataset was still taking 150MB in memory, leading to 150*209 = 31,35GB minimum memory requirement. When I tried with a bigger (64GB memory) machine, I was then blocked with the rechunking which was exceeding the machine's resources and making the script crash. So we ended up using a dask cluster which solved the concurrency and resources limitations.
My second use-case (https://github.com/pydata/xarray/issues/3096#issuecomment-516043946) still remains though, I am wondering if it matches the intended use of zarr and if we want to do something about it, in this case I can open a separate issue documenting it.
All in all I would say my original problem is not relevant anymore, either one can do it with open_mfdataset on a single machine as proposed by @rabernat, you just need some amount of memory (and probably much more if you need to rechunk), or you do it with a dask cluster, which is the solution we chose.

Hi All,
Thanks for all of your great work, support, and discussion on these and other pages. I very much appreciate it as I am working with Xarray and Zarr quite a lot for large geospatial data storage and manipulation.
I wanted to add a note to this discussion that I have had success using Zarr's built-in ProcessSynchornizer (which relies on the fasteners package). This provides a pretty easy and clean implementation of file locks as long as you can provide a file system that is shared across any and all process that might try to access the Zarr file. For me, that means using an AWS EFS mount, which gives me the flexibility to deploy this in a serverless context or on a more standard cloud cluster.
It does seem that providing explicit chunking rules as you have mentioned above (or using the Zarr encoding argument, which I haven't tried but I think is another option) is a great way to handle this and likely outperforms the locking approach (just a guess- would love to hear from others about this). But the locks are pretty easily implemented and seem to have helped me avoid the problems related to race conditions with Zarr.
For the sake of completeness, here is a simple example of how you might do this:
synchronizer = zarr.ProcessSynchronizer(f"/mnt/efs_mnt/tmp/mur_regional_raw_sst/zarr_locks/{bounding_box['grid_loc']}_locker.sync")
compressor = zarr.Blosc(cname='zstd', clevel=3)
encoding = {vname: {'compressor': compressor} for vname in current_region.data_vars}
current_region.to_zarr(store=store, mode='w',encoding=encoding, consolidated=True, synchronizer = synchronizer)

I would be happy to discuss further and am very much open to critique, instruction, etc.

Just a note that #4035 provides a new way to do parallel writing to zarr stores.
@VincentDehaye & @cdibble, would you be willing to test this out and see if it meets your needs?

I think this was closed by https://github.com/pydata/xarray/pull/4035 (which I'm going to start using shortly!), so I'll close this, but feel free to reopen if I missed something.


Qualname should be used in function's TypeErrors

Feature

Running the two code snippets below in CPython and in RustPython give different results
Example 1
def f(a, /, b=0): pass
f.__qualname__ = "g"
f(a=1, b=0)

In CPython:
TypeError: g() got some positional-only arguments passed as keyword arguments: 'a'

In RustPython:
TypeError: f() got some positional-only arguments passed as keyword arguments: 'a'

Example 2
class A:
    def f(self, a, /, b=0): pass

A().f(a=1, b=0)

In CPython:
TypeError: A.f() got some positional-only arguments passed as keyword arguments: 'a'

In RustPython:
TypeError: f() got some positional-only arguments passed as keyword arguments: 'a'

Python Documentation

Thank you!


Add canvas grid lines

It is visually helpful to see lines between the pixels on the editor

added in #70


docs: describe where to find backend-id, in APIM

The docs neatly mention:

When you use the set-backend-service policy to redirect requests to a custom backend, refer to the backend by its identifier (backend-id), not by its URL.

But where to find the backend-id in the Azure Portal? This does not seem to be listed in the APIM > Backend blade anywhere. I have defined a Custom URL backend, not an Azure resource.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 92c18150-be00-5c8f-ad72-02101cd65f52
Version Independent ID: 0d8e1937-93df-e34c-f072-4559461e752f
Content: Azure API Management backends
Content Source: articles/api-management/backends.md
Service: api-management
GitHub Login: @dlepow
Microsoft Alias: danlep

Hi @hanssens Thanks for your feedback, when you see the set backend service policy https://docs.microsoft.com/en-us/azure/api-management/api-management-transformation-policies#SetBackendService Backend entities are managed via Azure portal, API, and PowerShell.)
you need to create a backend with your custom URL in azure portal reference: https://docs.microsoft.com/en-us/azure/api-management/how-to-configure-service-fabric-backend
please let me know incase of further queries, I would be happy to assist you.

Thanks @JananiRamesh-MSFT , that part is all clear. And I have setup a backend service in the APIM > Backends section. But the questio is where to find the ID of the backend service? Can you point to where this can be found in the portal + add a small detail to the docs on that?
Q: Where to get the backend-id?
<set-backend-service backend-id="identifier of the backend entity specifying base URL of the backend service" />

Unlike other Azure services, this isn't displayed. Could this only be available, in case of a Custom URL backend, through the Azure API?
(Also, second link points to ServiceFabric backend which isn't applicable).

@hanssens thanks for getting back, the name of the backend that you created under backend blade is your backend id

we will update the document appropriately.

Ugh, of course - that's so obvious now. 😮💨 I was expecting a guid of some sort.
Thanks a bunch @JananiRamesh-MSFT! Hope adding the comments will help others as well.


Multiple dropdowns can be open at the same time

The "show" class is not being removed from other open dropdowns when a different one is clicked

after investigation this is an conflict with the latest version of bootstrap 5.0.1, I do not get the same issue with 5.0.0-beta3 which is included in your fiddle

It is a mistake to hack for a version that has not yet been officially released. The Dropdown implementation logic has been modified for each version. I will try to solve this problem a little later. Hopefully this will be the last time.

Here for the same reason.  Example being having three levels deep open.  If you click on first level to close it the next time you open it all three are open again.  The same behavior would happen with closing of second level.  Closing of any level should close all below it.
I noticed your comment above... https://getbootstrap.com/ shows BS sitting at 5.0.2 with no mention of alpha/beta so I imagine the current behavior should stick.

I had some time to play around with this and the latest BS 5.0.2 version and get things working how I previously had them with BS4.  I modified the code to fit my needs, but maybe it will be useful to others.
A few changes of the changes I made :

Dropdown links can have an actual link rather than only #.  If a link other than # is in the html then it can be clicked to go to that link when the navbar is not collapsed, a touchscreen device is not being used, and hovering is enabled for that dropdown.  At the same time if a dropdown link with # is given and the navbar is not collapsed, a touchscreen device is not being used, and hovering is enabled for the dropdown it won't close the dropdown on click.  This is useful for main menu navbars with dropdowns.
I commented out the clickEvent portion of the original code as it didn't do anything.
Hovering is only enabled when the navbar is not collapsed.  When on smaller devices the dropdowns act normally with click only.
Added two additional functions to check if navbar is collapsed or not and whether touch device is being used or not.

This code is specific to my use case of course, but might be useful to someone.  The main reason for my changes was to have dropdowns in my main menu at the top of the page, but at the same time make sure the usability is there on all screen sizes, devices, and with/without hovering.  If you plan on using this, at the minimum, you will need to adjust the selector for the isNav() function so it can properly detect if your main menu is collapsed or not (small screen or large screen detection).
(function($bs) {

	//class to check
    const CLASS_NAME = 'has-child-dropdown-show';

    $bs.Dropdown.prototype.toggle = function(_orginal) {
        return function() {
            document.querySelectorAll('.' + CLASS_NAME).forEach(function(e) {
                e.classList.remove(CLASS_NAME);
            });
            let dd = this._element.closest('.dropdown').parentNode.closest('.dropdown');
            for (; dd && dd !== document; dd = dd.parentNode.closest('.dropdown')) {
                dd.classList.add(CLASS_NAME);
            }
            return _orginal.call(this);
        }
    }($bs.Dropdown.prototype.toggle);

	document.querySelectorAll('.dropdown').forEach(function(dd) {
        dd.addEventListener('hide.bs.dropdown', function(e) {

            if(this.classList.contains(CLASS_NAME)) {
                this.classList.remove(CLASS_NAME);
                e.preventDefault();
            }

            //REMOVED from original code
            //if(e.clickEvent && e.clickEvent.composedPath().some(el=>el.classList && el.classList.contains('dropdown-toggle'))){
            //    e.preventDefault();
            //}

            e.stopPropagation(); // do not need pop in multi level mode
        });
    });

	//ADDED handle top level dropdown clicks (actual link or #)
    document.querySelectorAll('.dropdown [data-bs-toggle="dropdown"]').forEach(function(dd) {
		dd.addEventListener('click', function(e) {

        	//if full nav view AND not touch AND (one of the hovers is enabled)
        	if( isNav() && !isTouch() && (this.classList.contains('dropdown-hover') || Boolean(this.closest('.dropdown-hover-all')) ) ) {

				e.stopImmediatePropagation();

		    	//get href of clicked toggle
		    	var href = e.currentTarget.getAttribute('href');

	   			//goto link if not #
	   			if(href != '#') {
		    		location.href = href;
	   			}
	        }
        });
    });

    //ADDED isNav - is the desktop header shown or not
    function isNav() {
        return $('#header .navbar-toggler[data-bs-target="#navbar"]').is(':hidden'); //new way that doesn't use extra markup in pages or css
    }

    //ADDED isTouch - is this a touch device or not
    function isTouch() {
		return ('ontouchstart' in window || navigator.maxTouchPoints > 0 || navigator.msMaxTouchPoints > 0);
    }

	//for hover
    function getDropdown(element) {
    	//return $bs.Dropdown.getInstance(element) || new $bs.Dropdown(element);
        return $bs.Dropdown.getOrCreateInstance(element); //changed as it's the same thing
    }

	//dropdown-hover can be used on specific dropdown
	//dropdown-hover-all can be used on a container of elements
	//MODIFIED so hover only functions when nav is full view
    document.querySelectorAll('.dropdown-hover.dropdown, .dropdown-hover-all .dropdown').forEach(function(dd) {
        dd.addEventListener('mouseenter', function(e) {
            let toggle = e.target.querySelector(':scope>[data-bs-toggle="dropdown"]');
            if( !toggle.classList.contains('show') && isNav() ) {
                getDropdown(toggle).toggle();
            }
        });
        dd.addEventListener('mouseleave', function(e) {
            let toggle = e.target.querySelector(':scope>[data-bs-toggle="dropdown"]');
            if( toggle.classList.contains('show') && isNav() ) {
                getDropdown(toggle).toggle();
            }
        });
    });
})(bootstrap);


Fix window titles during authentication

The window titles do not seem to follow the context of the page content, and it does not always update, such that:

If I click Settings then use Alt + Left Arrow to go back a page, the title remains "Secret Key".

Also, is "Secret Key" the correct window title? Should it be "Settings" to match the link?

If I close the window with either the "Secret Key / Settings" or "View your Secret Key" page open, then the next time I open connect, that window title is restored over the top of the initial "Blockstack" title on the choose an account screen.

Refreshing the page fixes this and the title reverts to "Blockstack"

Is this still relevant given the switch to extension-only?

Nope, I think we can close this one out!


"String:" in output pipe ResultSetIteratingPipe

Describe the bug
ResultSetIteratingPipe seems to preprend "String:" in output
Reporter
Vinh Ban Pham
Category

Frank Framework

To Reproduce
Steps to reproduce the behavior:
Configure adapter with ResultSetIteratingPipe and result will contain "String:"
Expected behavior
no "String:" in output result
Screenshots
na
Environment:

IAF Version 7.6-20200624.152104

Additional context
2020-06-25 15:31:26,195 DEBUG [Thread-11] mid [Test Tool correlation id] processors.MonitoringPipeProcessor - Pipeline of adapter [HandleScannedDocumentsOut_ProcessXmlFile_Child] messageId [Test Tool correlation id] is about to call pipe [addDocumentFilesToZipStream] current result (Message) [String:
in the configuration addDocumentFilesToZipStream is a ResultSetIteratingPipe. But now that i look at the logging again i am not sure if its the ResultSetIteratingPipe. Pipe that gives that result is before addDocumentFilesToZipStream is an XsltPipe which seems unlikely aswell. Please let me know if those pipes are not it, I will check at a later time. Probably somewhere a toString instead of asString according to Niels

In the given context, i.e. a log line, the use of toString() and therefor the prefix 'String:' is expected.
Please check the case using the Ladybug debugger.


Duplicate module

I have the following layout:
$ tree
myproject
├── installs
│   ├── neovim.just
│   ├── ripgrep.just
│   ├── traceroute.just
│   └── xclip.just
├── JUSTFILE
├── modules
│   ├── apt.just
│   └── curl.just
Where my intention was for each of the just files under myproject/installs/  to be runnable either standalone or from the root level myproject/JUSTFILE.
The myproject/modules/*.just files have common behaviour that was to be called from anywhere.
If we look at the various files:
The root level JUSTFILE

$ cat JUSTFILE
import "installs/ripgrep.just"
import "installs/xclip.just"
import "installs/traceroute.just"

apt-installs:
    # Update apt cache
    @apt update
    @apt autoremove

    # Installing xclip
    @just install-xclip
    @just install-traceroute

curl-installs:
    # Installing ripgrep
    @just install-ripgrep

deploy: apt-installs curl-installs

One of the ../install files (the xclip just file is identical to the below save for the name)

$ cat installs/traceroute.just
mod apt "../modules/apt.just"

# install traceroute with apt
install-traceroute:
    @just apt::get "traceroute"

And lastly one of the modules
$ cat modules/apt.just
# module for installing packages with apt package manager
get package:
    @sudo apt-get --assume-yes install {{ package }}

This arrangement yields this error
error: Module `apt` first defined on line 1 is redefined on line 1
 ——▶ installs/xclip.just:1:5
  │
1 │ mod apt "../modules/apt.just"
  │     ^^^

Which I'm guessing is a circular import.
Is this expected behaviour?
I can get it to work by removing the
mod apt "modules/apt.just"
import from the installs/*.just files and placing it within the root level JUSTFILE but this kills the modularity I was going for.
So am I doing it wrong and I have to rethink my approach or can I get around the circular import?

A circular import is when, for example, a imports b which imports a again. This is a duplicate definition.
You could work around it by importing apt.just instead of making it a module. Identical duplicate imports are allowed. Or you could make your existing imports modules, since then the apt submodules wouldn't clash.
We already have allow-duplicate-recipes and allow-duplicate-variables settings, and I guess we could add allow-duplicate modules. Those settings are kind of hacks though, and I'm not crazy about adding more.


ssp: Use sampling rate from config

Use sampling rate passed in config instead of a default one when using
config blob.
Depends on https://github.com/thesofproject/sof/pull/6045
Signed-off-by: Krzysztof Frydryk<EMAIL_ADDRESS>

@kfrydryx, the Zephyr sof module is still at 2efc3ea41c0074c6dab5f376fafaa26f52c25c75 (in west.yaml). Does the module need to be uprev'd before we merge this?

@kfrydryx, the Zephyr sof module is still at 2efc3ea41c0074c6dab5f376fafaa26f52c25c75 (in west.yaml). Does the module need to be uprev'd before we merge this?

ops, sorry I missed this before I merged. If the module's SHA needs updating please send a PR to do so in west.yml.


Generate models

New models organisation,
Easy generation of documentation,
Moving dependencies.

It will be necessary to review the standard of Node JS ...


helm chart assumes a more recent version of the controller-manager than the default image

The helm chart tries to pass the --broker-relist-interval parameter to the catalog, but this functionality was added after v0.0.3, which is the default image specified by values.yaml, so the controller-manager crash loops.

Not really sure whether to call it a bug or not, but strange behavior regardless.

Duplicate of https://github.com/kubernetes-incubator/service-catalog/issues/713

Ah, didn't notice that one was the same, thanks. Closing!


USDE modifications

Summary
We need to explore changing the USDE method.
Issues:

Currently, we have the notion of fields that are related to the issue of data loss. Do we need to have a separate list related to fields pertaining to down time, or does it overcomplicate things?
We have this issue, which shows that the current method of using "excluded fields" has limitations due to the fact that we don't store the Numaflow specs.
We are adding labels and annotations to our PipelineRollout, ISBServiceRollout, and MonoVertexRollout which get passed to the child. We probably should have the capability to consider some of them as Progressive/PPND

[ ] USDE for Progressive, Delete/Recreate
[x] USDE: specify "data loss" fields
[ ] USDE: Enable certain Labels and Annotations to be marked for requiring Progressive/PPND

Message from the maintainers:
If you wish to see this enhancement implemented please add a 👍 reaction to this issue! We often sort issues this way to know what to prioritize.

Can do this in Q2

Derek has added all of the fields he can think of that could be risks for data loss here. I think there is a complexity that some fields implicitly include their child fields, and others do not. (Of course, we can always simplify things if necessary and be conservative.)

We don't need the parent issue anymore. I am closing it.


Fix typos

Found via codespell -q 3 -L allright,ba,bu,childs,equil,esy,fo,fom,hda,localy,lod,mone,nd,ned,nnumber,numer,ot,pres,ro,seh,shat,solfes,strat,tbe,te,tthe,warmup

@luzpaz, we are about to release so I hope you don't mind that I cherry-picked this in https://github.com/mfem/mfem/pull/3199.
If that's okay with you, we can close this PR


@mention+ posts to forum walls disappear after editing

When posting a wall-to-wall post from a normal channel to a forum channel with @mention+, it is received normal and displayed on the forum's channel page/wall. But when edited by the sender* the post eventually disappears from the forum's channel page/wall. The post is not removed from the senders channel page/wall.
When checking that particular post in the item table in the DB,  item_wall is before editing 1 and after editing 0.
I tried hard to find anomalies in the debug log, but could not find something.
*Note:  From the senders channel or grid, not as the sender on the forum channel > it than gives a top-right-notification that the item can't be edited.

I think this should be sorted now.

Perfect! Also the issue after *note. Thanks again Master Mike!


Data races in extract and export for short rac-files

Two modules; extractors and exports have data races in their test cases, as well as the application, when using small enough rac-files. It is probably the setup of the tests that makes the data race happen, since it is not visible in the application.
$ go test -race ./...
ok      github.com/innosat-mats/rac-extract-payload/cmd/rac     0.033s
ok      github.com/innosat-mats/rac-extract-payload/internal/aez        0.248s
ok      github.com/innosat-mats/rac-extract-payload/internal/awstools   0.041s
ok      github.com/innosat-mats/rac-extract-payload/internal/ccsds      0.026s
ok      github.com/innosat-mats/rac-extract-payload/internal/common     0.034s
ok      github.com/innosat-mats/rac-extract-payload/internal/decodejpeg 0.438s
2020/05/25 10:32:42 here be dragons
==================
WARNING: DATA RACE
Read at 0x00c0002a53f0 by goroutine 19:
  github.com/innosat-mats/rac-extract-payload/internal/exports.TestAWSS3CallbackFactory.func1.1()
      /home/joakim/go/src/github.com/innosat-mats/rac-extract-payload/internal/exports/aws_s3_writer_test.go:179 +0x11d
  github.com/innosat-mats/rac-extract-payload/internal/exports.AWSS3CallbackFactory.func1()
      /home/joakim/go/src/github.com/innosat-mats/rac-extract-payload/internal/exports/aws_s3_writer.go:61 +0x11b

Previous write at 0x00c0002a53f0 by goroutine 18:
  github.com/innosat-mats/rac-extract-payload/internal/exports.TestAWSS3CallbackFactory.func1.1()
      /home/joakim/go/src/github.com/innosat-mats/rac-extract-payload/internal/exports/aws_s3_writer_test.go:195 +0x28d
  github.com/innosat-mats/rac-extract-payload/internal/awstools.(*Timeseries).Close()
      /home/joakim/go/src/github.com/innosat-mats/rac-extract-payload/internal/awstools/timeseries.go:30 +0x1b1
  github.com/innosat-mats/rac-extract-payload/internal/timeseries.(*CSV).Close()
      /home/joakim/go/src/github.com/innosat-mats/rac-extract-payload/internal/timeseries/csv.go:47 +0xe2
  github.com/innosat-mats/rac-extract-payload/internal/timeseries.(*CSVCollection).CloseAll()
      /home/joakim/go/src/github.com/innosat-mats/rac-extract-payload/internal/timeseries/csv_collection.go:60 +0x152
  github.com/innosat-mats/rac-extract-payload/internal/exports.AWSS3CallbackFactory.func3()
      /home/joakim/go/src/github.com/innosat-mats/rac-extract-payload/internal/exports/aws_s3_writer.go:107 +0x4a
  github.com/innosat-mats/rac-extract-payload/internal/exports.TestAWSS3CallbackFactory.func1()
      /home/joakim/go/src/github.com/innosat-mats/rac-extract-payload/internal/exports/aws_s3_writer_test.go:241 +0x3db
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:991 +0x1eb

Goroutine 19 (running) created at:
  github.com/innosat-mats/rac-extract-payload/internal/exports.AWSS3CallbackFactory()
      /home/joakim/go/src/github.com/innosat-mats/rac-extract-payload/internal/exports/aws_s3_writer.go:56 +0x642
  github.com/innosat-mats/rac-extract-payload/internal/exports.TestAWSS3CallbackFactory.func1()
      /home/joakim/go/src/github.com/innosat-mats/rac-extract-payload/internal/exports/aws_s3_writer_test.go:229 +0x2bf
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:991 +0x1eb

Goroutine 18 (running) created at:
  testing.(*T).Run()
      /usr/local/go/src/testing/testing.go:1042 +0x660
  github.com/innosat-mats/rac-extract-payload/internal/exports.TestAWSS3CallbackFactory()
      /home/joakim/go/src/github.com/innosat-mats/rac-extract-payload/internal/exports/aws_s3_writer_test.go:173 +0x1130
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:991 +0x1eb
==================
--- FAIL: TestAWSS3CallbackFactory (0.05s)
    --- FAIL: TestAWSS3CallbackFactory/Uploads_everything (0.03s)
        testing.go:906: race detected during execution of test
    testing.go:906: race detected during execution of test
FAIL
FAIL    github.com/innosat-mats/rac-extract-payload/internal/exports    0.171s
==================
WARNING: DATA RACE
Write at 0x00c0000a0528 by main goroutine:
  internal/race.Write()
      /usr/local/go/src/internal/race/race.go:41 +0x114
  sync.(*WaitGroup).Wait()
      /usr/local/go/src/sync/waitgroup.go:128 +0x115
  github.com/innosat-mats/rac-extract-payload/internal/extractors.ExtractData()
      /home/joakim/go/src/github.com/innosat-mats/rac-extract-payload/internal/extractors/extract_data.go:49 +0x447
  github.com/innosat-mats/rac-extract-payload/internal/extractors.Example()
      /home/joakim/go/src/github.com/innosat-mats/rac-extract-payload/internal/extractors/extract_data_test.go:100 +0x3a4
  testing.runExample()
      /usr/local/go/src/testing/run_example.go:62 +0x274
  testing.runExamples()
      /usr/local/go/src/testing/example.go:44 +0x211
  testing.(*M).Run()
      /usr/local/go/src/testing/testing.go:1200 +0x3b3
  main.main()
      _testmain.go:64 +0x223

Previous read at 0x00c0000a0528 by goroutine 28:
  internal/race.Read()
      /usr/local/go/src/internal/race/race.go:37 +0x1e8
  sync.(*WaitGroup).Add()
      /usr/local/go/src/sync/waitgroup.go:71 +0x1fb
  github.com/innosat-mats/rac-extract-payload/internal/extractors.ExtractData.func1()
      /home/joakim/go/src/github.com/innosat-mats/rac-extract-payload/internal/extractors/extract_data.go:27 +0x5c

Goroutine 28 (running) created at:
  github.com/innosat-mats/rac-extract-payload/internal/extractors.ExtractData()
      /home/joakim/go/src/github.com/innosat-mats/rac-extract-payload/internal/extractors/extract_data.go:26 +0x1e8
  github.com/innosat-mats/rac-extract-payload/internal/extractors.Example()
      /home/joakim/go/src/github.com/innosat-mats/rac-extract-payload/internal/extractors/extract_data_test.go:100 +0x3a4
  testing.runExample()
      /usr/local/go/src/testing/run_example.go:62 +0x274
  testing.runExamples()
      /usr/local/go/src/testing/example.go:44 +0x211
  testing.(*M).Run()
      /usr/local/go/src/testing/testing.go:1200 +0x3b3
  main.main()
      _testmain.go:64 +0x223
==================
FAIL
FAIL    github.com/innosat-mats/rac-extract-payload/internal/extractors 0.048s
ok      github.com/innosat-mats/rac-extract-payload/internal/innosat    0.036s
ok      github.com/innosat-mats/rac-extract-payload/internal/ramses     0.033s
ok      github.com/innosat-mats/rac-extract-payload/internal/timeseries 0.085s
FAIL

Original bug description:
It seems to be able to happen, but not always and I have no idea why.
Could it be some hidden timing flaw within the core logic of our control flow?
 Test
5s
##[error]Process completed with exit code 2.
ok  	github.com/innosat-mats/rac-extract-payload/internal/awstools	0.007s
ok  	github.com/innosat-mats/rac-extract-payload/internal/ccsds	0.004s
ok  	github.com/innosat-mats/rac-extract-payload/internal/common	0.007s
ok  	github.com/innosat-mats/rac-extract-payload/internal/decodejpeg	0.062s
ok  	github.com/innosat-mats/rac-extract-payload/internal/exports	0.011s
--- FAIL: Example (0.00s)
got:

want:
got stop packet without a start packet (Set1)
 STAT <nil>
  checksum bad 62691
 STAT <nil>
  checksum bad 32488
  got stop packet without a start packet (Set2)
 STAT <nil>
  checksum bad 62691
 STAT <nil>
  checksum bad 32488
FAIL
FAIL	github.com/innosat-mats/rac-extract-payload/internal/extractors	0.005s
ok  	github.com/innosat-mats/rac-extract-payload/internal/innosat	0.003s
ok  	github.com/innosat-mats/rac-extract-payload/internal/ramses	0.010s
ok  	github.com/innosat-mats/rac-extract-payload/internal/timeseries	0.007s
FAIL
##[error]Makefile:20: recipe for target 'test' failed
make: *** [test] Error 1
##[error]Process completed with exit code 2.

I got this problem (once) when running locally as well. Output also shows we're having race conditions on master.

After some (admittedly shallow) exploring, I think the issue is not primarily with how the tests are written, but that the rac "files" we use in the tests are so much shorter than what we use when testing manually, that the -race flag doesn't find them (as I understand it from the docs, the -race flag isn't guaranteed to find race conditions on execution). One indication of this is, that adding a wg.Wait() before the teardown() in aws_s3_writer_test.TestAWSS3CallbackFactory() moves the race condition to entirely within our code. Another indication is that the extract_data_test.Example() test is so minimal, that it is hard to conceive where in the test it could be going wrong.
All that being said, I find the whole wait group concept and code a bit confusing, so I can't say I'm sure at all...

I have now tried the binary compiled and installed with -race and a minimal rac-file (attached) and can confirm that we have these race conditions in our production code, not just in the tests.
Steps to reproduce:
$ go build -race ./...

$ go install -race -i ./...

$ ~/go/bin/rac -project test -aws mini.rac
==================
WARNING: DATA RACE
Write at 0x00c00012abb8 by main goroutine:
  internal/race.Write()
      /usr/local/go/src/internal/race/race.go:41 +0x114
  sync.(*WaitGroup).Wait()
      /usr/local/go/src/sync/waitgroup.go:128 +0x115
  github.com/innosat-mats/rac-extract-payload/internal/extractors.ExtractData()
      /home/skymandr/Work/MATS/rac-extract-payload/internal/extractors/extract_data.go:49 +0x425
  main.processFiles()
      /home/skymandr/Work/MATS/rac-extract-payload/cmd/rac/cli.go:116 +0x2d7
  main.main()
      /home/skymandr/Work/MATS/rac-extract-payload/cmd/rac/cli.go:185 +0x42b

Previous read at 0x00c00012abb8 by goroutine 10:
  internal/race.Read()
      /usr/local/go/src/internal/race/race.go:37 +0x1e8
  sync.(*WaitGroup).Add()
      /usr/local/go/src/sync/waitgroup.go:71 +0x1fb
  github.com/innosat-mats/rac-extract-payload/internal/extractors.ExtractData.func1()
      /home/skymandr/Work/MATS/rac-extract-payload/internal/extractors/extract_data.go:27 +0x5c

Goroutine 10 (running) created at:
  github.com/innosat-mats/rac-extract-payload/internal/extractors.ExtractData()
      /home/skymandr/Work/MATS/rac-extract-payload/internal/extractors/extract_data.go:26 +0x1e8
  main.processFiles()
      /home/skymandr/Work/MATS/rac-extract-payload/cmd/rac/cli.go:116 +0x2d7
  main.main()
      /home/skymandr/Work/MATS/rac-extract-payload/cmd/rac/cli.go:185 +0x42b
==================
2020/05/27 16:11:46 got stop packet without a start packet (mini.rac)
2020/05/27 16:11:46 checksum bad 62691
2020/05/27 16:11:46 checksum bad 32488
Found 1 data race(s)

$ ~/go/bin/rac -project skytest mini.rac
==================
WARNING: DATA RACE
Write at 0x00c00002cc48 by main goroutine:
  internal/race.Write()
      /usr/local/go/src/internal/race/race.go:41 +0x114
  sync.(*WaitGroup).Wait()
      /usr/local/go/src/sync/waitgroup.go:128 +0x115
  github.com/innosat-mats/rac-extract-payload/internal/extractors.ExtractData()
      /home/skymandr/Work/MATS/rac-extract-payload/internal/extractors/extract_data.go:49 +0x425
  main.processFiles()
      /home/skymandr/Work/MATS/rac-extract-payload/cmd/rac/cli.go:116 +0x2d7
  main.main()
      /home/skymandr/Work/MATS/rac-extract-payload/cmd/rac/cli.go:185 +0x42b

Previous read at 0x00c00002cc48 by goroutine 10:
  internal/race.Read()
      /usr/local/go/src/internal/race/race.go:37 +0x1e8
  sync.(*WaitGroup).Add()
      /usr/local/go/src/sync/waitgroup.go:71 +0x1fb
  github.com/innosat-mats/rac-extract-payload/internal/extractors.ExtractData.func1()
      /home/skymandr/Work/MATS/rac-extract-payload/internal/extractors/extract_data.go:27 +0x5c

Goroutine 10 (running) created at:
  github.com/innosat-mats/rac-extract-payload/internal/extractors.ExtractData()
      /home/skymandr/Work/MATS/rac-extract-payload/internal/extractors/extract_data.go:26 +0x1e8
  main.processFiles()
      /home/skymandr/Work/MATS/rac-extract-payload/cmd/rac/cli.go:116 +0x2d7
  main.main()
      /home/skymandr/Work/MATS/rac-extract-payload/cmd/rac/cli.go:185 +0x42b
==================
2020/05/27 16:12:27 got stop packet without a start packet (mini.rac)
2020/05/27 16:12:27 checksum bad 62691
2020/05/27 16:12:27 checksum bad 32488
Found 1 data race(s)

minirac.tar.gz

is it only with the aws flag? Or is it also when writing to disk?

Both.


floating point arithmetic error

I have a VM on GCP using Tesla t4. I'm trying to run extract_color_mesh.py.
xmim, max = -0.40, 0.30
ymin, ymax = -0.40, 0.30
zmin, zmax = -1.30, -0.60

I keep getting the error 'the ranges must have the same length!' so I printed the ranges and
xrange = 0.7
yrange = 0.7
zrange = 0.7000000000000001

??!!
What do you think is the problem?

I changed the values of ranges. Now the execution gets killed.

I removed the assertion already, please pull and check again

Thanks. I had commented the assertion and checked. The execution gets killed. I will pull and check again. The colab was working but now fails to install torchsearchsorted.


Throttling and cancelling

Given a throttling task:
diff --git i/test/dummy/app/tasks/maintenance/update_posts_throttled_task.rb w/test/dummy/app/tasks/maintenance/update_posts_throttled_task.rb
index 6bf5ba6..a70c855 100644
--- i/test/dummy/app/tasks/maintenance/update_posts_throttled_task.rb
+++ w/test/dummy/app/tasks/maintenance/update_posts_throttled_task.rb
@@ -5,6 +5,7 @@ class UpdatePostsThrottledTask < MaintenanceTasks::Task
     class << self
       attr_accessor :throttle, :throttle_proc
     end
+    self.throttle_proc = true

     throttle_on { throttle }
     throttle_on(backoff: -> { 10.seconds }) { throttle_proc }

If you run it, cancel it after it gets interrupted, it moves to cancelled properly, but the job is enqueued anyway, and after the backoff period, there's an exception, trying to move the run from cancelled to running (which causes an exception trying to move from cancelled to errored).

Fixed in #583


Python 3 support fix on _send method

Running a Django v1.10 app with Python v3.4.3, the _send method breaks.
Stack trace:
  File "/home/pablo/.virtualenvs/test/local/lib/python3.4/site-packages/graphitesend/graphitesend.py", line 291, in _send
    self.socket.sendall(message.encode("ascii"))
AttributeError: 'bytes' object has no attribute 'encode'

...
graphitesend.graphitesend.GraphiteSendException: Unknown error while trying to send data down socket to ('<IP_ADDRESS>', 8000), error: 'bytes' object has no attribute 'encode'

@daniellawrence travis errors are not from the commit i made, can you give me a hand? Thanks in advance!

Coverage decreased (-0.3%) to 93.08% when pulling 634ba684f012e56e974e12fc4106f290cccbdda5 on PabloLefort:master into 6f84c56ad4e7594cf36b0c3d4aeaa86b4dd3ef06 on daniellawrence:master.

@PabloLefort I fixed the issue you were facing with the tests. However, the coverage decrease with your commit so could you please add a test to make it at least the same as now.
Also, please note that an other ongoing PR (#70) is trying to fix a Python 3 issue and I think it's the same issue. Can you please explain what was the issue you were facing ? Our Python 3 test suite is passing so I don't understand what could be the error.
I like the fact that the PR #70 is regardless of the python version. But I also like the simplicity of your PR. You could still make it simpler by using an and clause rather than two if statements.

@Shir0kamii It could be something like this?
    def test_send_values_as_bytes(self):
        graphite_instance = graphitesend.init()
        metric = 'metric'
        if sys.version_info >= (3, 0) and type(metric) is bytes:
            metric = str(metric, 'utf-8')
            response = graphite_instance.send(metric, "1", "1")
            response = str(response)
            self.assertEqual('metric 1.000000 1' in response, True)

Thank you!

@PabloLefort I'm not sure it would increase the coverage.
Coverage is just a tool verifying that every line of code is executed. So to increase coverage you will need to make sure the test you write execute the lines of code you added. It means that you will have to send a byte object.
Also because your code will only ever be executed on Python 3, you won't be able to increase Python 2's coverage. But that's okay, I'll merge it if Python 3 pass coverage test.

@Shir0kamii Thanks! I'll update the test tonight.


auth: Inform the user that the app origin is incorrect

Expected Behavior
The error dialog in some way informs the user that the app.baseUrl is incorrect
Current Behavior
The error dialog says that auth failed be cause the "popup was closed".
Possible Solution
Detect incorrect origins in the script returned from /api/auth/<provider>/handler/frame, and pass on a more descriptive error from the popup with postMessage
Steps to Reproduce
Set up any auth provider correctly, but use an app.baseUrl that does not match the origin that the app is hosted at.

I'd like to take a look at this, could you assign me @Rugvip ?🙂

@erictnilsson sure, thanks! :grin:

Hey! I've unassigned myself from this issue @Rugvip, I'm swamped with school work, can't really find time to work on this :/

@erictnilsson alright, no problem, thank you for the update!

Hi @Rugvip , I had a look at this. I am looking at https://github.com/backstage/backstage/blob/master/plugins/auth-backend/src/lib/flow/authFlowHelpers.ts#L41-L46 but need a bit of help on the following:

under what conditions would "app.baseUrl" actually differ here? I see that it is taken from https://github.com/backstage/backstage/blob/master/app-config.yaml#L3 or any such env specific config
If I reproduce the problem by modifying baseUrl here https://github.com/backstage/backstage/blob/master/plugins/auth-backend/src/service/router.ts#L46, it will fail the postMessage (differing origin) mentioned above.
How to change baseUrl but still do postMessage successfully with an error message?

postMessage fails silently, so can't handle errors there. any hints?

@jot-hub Thanks!
The things that can differ are the origin in app.baseUrl and the actual origin of the app. It's an error that will only happen when configuration is incorrect, but that'll happen sometimes and we want it to be as easy to detect as possible :grin:
I'm thinking the simplest way to handle this might be to post two messages from the auth frame. First the auth response message that already exists, and then a second message that is posted with target origin *. If the second message is received before the first, we know that something's wrong. The second message would contain no sensitive information, as posting the auth response to a specific origin is important for security. The second messages would just be something that has a similar format, but that can easily be identified and probably contains some additional information like the expected origin.


⚠ Fortnite has degraded performance

In 341f03d, Fortnite (https://fortnite.com) experienced degraded performance:

HTTP code: 200
Response time: 1653 ms

Resolved: Fortnite performance has improved in f58d0ec.


tg_command Fails to Recognize Multi-Level Folder Paths with Wildcards

*Describe the Bug
The tg_command function does not interpret multi-level folder paths with wildcards correctly. For example, running the following command in the terminal works as expected:
terragrunt run-all plan --terragrunt-working-dir my-folder/*/*-test

In this case, terragrunt successfully plans all content within folders ending in "-test".
However, attempting to run the same command with tg_command results in an error indicating the folder path cannot be found:
tg_command: 'run-all plan --terragrunt-working-dir ${{ vars.PROJECT_ID }}/*/*-${{ env.git_environment }}'

Error received:
ERROR  lstat my-folder/*/*-test: no such file or directory

Interestingly, removing the wildcard syntax resolves the issue, as shown below:
tg_command: 'run-all plan --terragrunt-working-dir ${{ vars.PROJECT_ID }}/folder-name/folder-prefix-${{ env.git_environment }}'

This simplified version also works without error:
tg_command: 'run-all plan --terragrunt-working-dir ${{ vars.PROJECT_ID }}'

However, using a partial wildcard pattern like the example below still triggers a "folder not found" error:
tg_command: 'run-all plan --terragrunt-working-dir ${{ vars.PROJECT_ID }}/folder-name/*-${{ env.git_environment }}'

Suggestion:
The issue might stem from tg_command interpreting * as a literal folder name, rather than as a wildcard.
Expected Behavior
The tg_command should recognize and correctly handle folder paths with multi-level wildcard patterns, similar to the behavior of the terragrunt command in the terminal.
Terragrunt Action Version: V2
Environment Details:

tg_version: "0.67.16"
tf_version: "1.9.7"

Hey @guicrocetti ,
The difference between the two is what your shell is doing, not what Terragrunt is doing.
When you run this:
terragrunt run-all plan --terragrunt-working-dir my-folder/*/*-test

It's not actually Terragrunt that's evaluating those wildcards, it's your shell. You can probably see the same error if you do this:
terragrunt run-all plan --terragrunt-working-dir 'my-folder/*/*-test'

Some (basically all, I think) shells expand globs when they are used like that.
The terragrunt-working-dir CLI command is not meant to be used with multiple directories, so I don't think we're interested in adding logic to handle this on the go side if a user explicitly passes in a * string.
Why not use terragrunt-include-dir with terragrunt-strict-include instead?


spy operator added with >=, <= and == operators for use in script condition

That dropdown is added. Operators added to $scope.watcher.$$condition_operators, which could be extended to support more operators. I have not done anything about the style, so it's up to you guys to determine how it should look.

Issue/feature request: #456

@sergibondarenko

Looks good in my opinion. Whether to include all the operators or not it's up to you. Maybe it's unnecessary to have that many.


[bug] quickhud draws all types of wrong in splitscreen

a) it draws over top of cutscenes

b) It draws in the incorrect position (most visible on ss_pipsplit 1)

c) Blue doesn't draw when you have sar_hud_orange_only 1 (imo it should be identical to buillt-in crosshair)

b) has been resolved by ed3fd1b0ca6b49b95b5981750caaab27169a741e

(c) isn't an issue - that cvar is meant to disable all SAR HUD drawing for blue. It's pretttty much made for the specific case of fullscreen PIP, for which not drawing blue's quickhud is the correct thing to do. (a) sounds faiiirly easy to fix (we just need to detect that a movie is playing), although I've not looked into it


Recent merge req. with pytest and pylint.

The p-ytest and pylint still shows errors though I fixed many of the suggestions but still there are some errors. I will keep trying and If I fixed, I will let you know.

Title changed

Hi Arif,
No worries. I'm pretty sure you don't have to be 100% with the linter either so don't stress too much if you can't get it perfect


Add inter_batch_wait

Can we go ahead and enter this now to give sync users a solution that works with gam batch and does paced adds/remove. Nothing prevents the users from doing: gam csv 1000users.csv gam update group oldgam add member ~email

I don't see the point of adding further complexity when batch API calls is slower than parallel API calls. Can we look at re-implementing with parallel API calls?

Parallel add/removes have not been eliminated, you can still do that if you want.
What batch does is speeds up: gam update group<EMAIL_ADDRESS><EMAIL_ADDRESS>add/remove/sync member <ou /Path/To/OU, file FileName, ...>
--
<EMAIL_ADDRESS>
On Sep 14, 2017, at 6:09 AM, Jay Lee<EMAIL_ADDRESS>wrote:
I don't see the point of adding further complexity when batch API calls is slower than parallel API calls. Can we look at re-implementing with parallel API calls?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/jay0lee/GAM/pull/570#issuecomment-329476566, or mute the thread https://github.com/notifications/unsubscribe-auth/AIU8L1PfCPHBYtUjEUPEER56S3M9U4ohks5siSWRgaJpZM4PXgzD.

but it's not actually faster than parallel. I'm looking at re-implementing add/remove/sync to use parallel.

You have to update your school groups every night to allow for new students/departing students.
On the first day of school, batch may be somewhat slow that parallel, but on every other night you pay a big price.
Batch:
gam update group<EMAIL_ADDRESS><EMAIL_ADDRESS>sync members ou /Students/ClassOf2018
or
Parallel:
gam update group<EMAIL_ADDRESS><EMAIL_ADDRESS>clear member
echo "primaryEmail" > users.csv
gam ou /Students/ClassOf2018 print >> users.csv
gam csv users.csv gam update group<EMAIL_ADDRESS><EMAIL_ADDRESS>add member ~primaryEmail
<EMAIL_ADDRESS>
On Sep 14, 2017, at 6:20 AM, Jay Lee<EMAIL_ADDRESS>wrote:
but it's not actually faster than parallel. I'm looking at re-implementing add/remove/sync to use parallel.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/jay0lee/GAM/pull/570#issuecomment-329479443, or mute the thread https://github.com/notifications/unsubscribe-auth/AIU8L_BoNkBlGKjpTudVLJNO3Ungptxcks5siSgegaJpZM4PXgzD.

See my commit just now to switch to using parallel instead of batch for add/remove/sync. It still needs some cleanup but I'm going to stick with parallel unless you can convince me that batch is either faster or simpler.


[WIP] nixos: make several modules not run as root

Motivation for this change
A number of services are currently run as root because they need to create various directories. We can have systemd do that for us so they can start as regular users.
This isn't tested yet.
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[x] Fits CONTRIBUTING.md.

I don't think all these should be merged in one go, so I'm closing this, creating a tracking issue and doing individual PRs for each instead.


Translate AST to and from json4s

BEWARE json4s AST is vulnerable under DoS attacks: https://github.com/json4s/json4s/issues/553

did you... automate that comment?


Drop GitHub action testing for Python 3.8

Pull requests:

https://github.com/senzing-garage/template-python/pull/180


Ordered UUID for Laravel MySQL grammar query compilation via custom builder

Once pending PRs are reviewed/attended to, it will be pertinent to consider ordered uuids for setting up partial sequential ids in database tables making it easy to sort/index at DB level. The custom eloquent builder and grammar PHP classes in this package will be updated to have this new behaviour. This could be the basis of a version 2 of the package focusing on performance with respect to UUIDs.
For context, see:
https://www.percona.com/blog/2014/12/19/store-uuid-optimized-way

This package has been abandoned. Alternatives: https://github.com/michaeldyrynda/laravel-efficient-uuid & https://github.com/michaeldyrynda/laravel-model-uuid


Self test result is different

I'm evaluating the sensor using your self-test program. I have been getting the same result with different arduinos on different sensors, but they are different to your suggested IDs and acceleration values.

That's strange. I did tested it again (I only have Arduino Due and EVAL-ADXL355-PMDZ. This is what I get:
Family ID (should be 0x1D): 0x1D
Device ID (should be 0xED): 0xED
Revision ID: 0x1

Self test (should be 0.1...0.6 g, 0.1...0.6 g, 0.5...3.0 g): 0.34, 0.34, 1.41

What is your output?

Thanks, I have now been able to get the same output as well. I was using the 3.3V pin and the wrong chip select pin.
However, I would like to note that I only get the same output as you suggest when I unplug the arduino and plug it back in. If I simply press the reset button, there seems to be some interference with the communication.

I think it can depend on the specific Arduino board and chip. For my Arduino Due programming port reset works fine. It is weird that upon power up it works fine for you because it seems that you should expect more power/communication signal issues upon power up than upon reset.


Update winget.ps1

I just updated the script to version 3.0.0 and am pushing this out to your project too. 😊
https://github.com/asheroto/winget-install
You can use the signed version from releases if preferred.

Thanks for the update @asheroto


#23 Added PR Template to Github Workflow

This commit consists of a basic required PR Template so that for a Pull Request to be merge some checklists are ensured to maintain a healthy repository.

Summary by CodeRabbit

Chores

Introduced a pull request template to standardize submissions, ensuring key information is provided for every change.

@FrancescoXX Could you Verify this PR?

Vercel — Authorization required to deploy.

Vercel — Authorization required to deploy.

How Should I do this? @FrancescoXX

Vercel — Authorization required to deploy.

@FrancescoXX Ig you need to give the authorization to deploy, we actually can't give authorization to vercel as it's hosten on your vercel account.

Vercel — Authorization required to deploy.

@FrancescoXX Ig you need to give the authorization to deploy, we actually can't give authorization to vercel as it's hosten on your vercel account.

Or It would be great if you just add us to your @vercel team.

deployment failed.
try to run npm run build locally
remember to update the project

deployment failed.
try to run npm run build locally remember to update the project

But this doesn’t have any impact to the project source, as this is just a PR template.

have you tried running npm run build on your local machine before pushing?


TestTCPStackdriverMonitoring test flakes

https://prow.istio.io/view/gs/istio-prow/pr-logs/pull/istio_istio/29380/integ-telemetry-mc-k8s-tests_istio/1334598675631968256
I have seen others as well, will update as I find links

Presumed fixed somehow -  I haven't seen it


[cxx-interop] Mark C++ functions with unavailable return type as unav…

…ailable
This prevents users from calling functions with unsupported or unavailable return types. This ensures that users don't for example call a function that returns a non-copyable and non-movable type
Fixes https://github.com/apple/swift/issues/64401

@swift-ci please test

@swift-ci please test source compatibility


Feat/banner

:tada: This PR is included in version 1.31.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


display_name prompt

if the display name field is blank have the command line prompt the user to provide a substitute name for the display name

Or better yet add custom display name as an optional input parameter

yah sorry my message was unclear i think the default value should be "display_name"

can we close this?

yep
On Wed, Sep 2, 2015 at 6:28 PM, Usman Ehtesham Gul<EMAIL_ADDRESS>
wrote:

can we close this?
—
Reply to this email directly or view it on GitHub
https://github.com/McGillX/edx_data_research/issues/18#issuecomment-137263086
.

--
Alexander Steeves-Fuentes | Courseware Developer | McGillX
Teaching and Learning Services | McGill University | McLennan Library
Building, MS-12 | 3459 McTavish Street | Montreal, Quebec | H3A 0C8 |
http://www.mcgill.ca/tls


[Test] Managed Prometheus addon image latest image: 1.2.3-test

This is an automation test

This was a test


Fix addon initialization order

This addon should be initialized before the core addons so that the core addons can overwrite the blueprints from this project.
see https://github.com/ember-cli/ember-cli/issues/5868
/cc @trabus @rwjblue @stefanpenner

seems good, @rwjblue r?

this should probably be released as v0.1.1 and bumped in ember-cli


NIAD-261: Set the deregistration delay to remove the old container from LB quicker

moved from:
https://github.com/nhsconnect/integration-adaptors/pull/254


add pcregrep to main bucket

command-line perl compatible grep
see pcre.org
BSD license; meets all criteria in issue #451

Good stuff :+1:


Dialog for pinning tag in home page

Tasks

[ ] Create a new dialog for pinning tag in home page
[ ] user can store and read selection

Design

figma

Blocked by

issue

@ElkLeung please add draft PR. Thanks! 🙏🏻

Issue requires review and rearrangement. Close it.


CT check for not existing site column

On SP Prem it is possible to rollout a wsp with a ct definition with one ore more not existing linked fields. This bug comes only on light if you use this ct..
So if we export the ct to our xml we need a check for really existent fields. On the CSOM import side we get big trouble..
Tomorrow if I have access to my production environment I will upload a patch for this..

Hi Torsten, Is this issue still open, even after the changes we made in the latest months? Thanks.

if i remember this correct, there was also a pr for this.. anyway i have changed the known wsp on our side, where i detected this behavior. On the other side we should have such a check in the engine, and the engine should only write an error to the log for the not existing objects and never stop there work.

Thanks Torsten, we usually prefer to block the provisioning in case of any error, instead of proceeding making the user making wrong assumptions. I mean: if we can't properly provision the template, we prefer to stop the provisioning and make the user aware of it, rather than logging a line an proceeding with a potentially inconsistent result. I will close this issue, because there should be a merged PR for this. If you will experience it in the future, I will be more than happy to re-open it. Thanks again.

not so hurry italian man;) i have make a new pr (#401), and this will throw an error with field id and text, so the enduser using powershell will be able to find the bug inside the template xml.

Thanks for your contribution. I will double-check the PR and let you know.


website Rename

Website Renamed as AirMovie to match the functionality of the Project. Please check the README.md for further explanation

Its approved 👏👏👏

AirMovie perfect name


[Feature Request] Show connections on map

This might increase people's motivation to get in touch with each other: is it possible to extend the map by lines between nodes that can directly hear each other?
I could also think of arrows showing the diretion of the communication:
Node_A --> Node_B: only Node B can hear Node_A
Node_A <-- Node_B: only Node A can hear Node_B
Node_A <--> Node_B: Node_B and Node_A can hear each other
Line thickness or color (grey tones) could represent the quality of the connection either by signal strength and/or last time heard.

I can't get any "Nodes i heard" or 'Nodes that heard me" lines for my node on the map.
Can someone layout EXACTLY what the requirements or settings are to do get the links to show up?
I have:
-MQTT setup to liamcottle per the instructions on the map website, green and online on map
-Uplink enabled in primary Long_Fast channel
-Sent and received successful Long_Fast messages and tracerts

This is already shown on the map. You need to have uplink on a public channel, and turn on the neighbors module on all nodes. Set it to something like 1000 seconds. Then on the map, turn on the neighbors layer. You will see who heard what node, and if they can both be heard, the line is thicker.

Okay, tbh, I didn't know that.
But.. I think that the way connections are shown at the moment is pretty useless as it doesn't only show the nodes that directly hear each other via LoRa, but also the ones being connected over the internet.
What I would like to have is a view on the structure of the mesh.

I can't get any "Nodes i heard" or 'Nodes that heard me" lines for my node on the map. Can someone layout EXACTLY what the requirements or settings are to do get the links to show up? I have: -Tried firmwares 2.3, 2.4 and am currently on the latest 2.5 -MQTT setup to liamcottle per the instructions on the map website, green and online on map -Uplink enabled in primary Long_Fast channel -Sent and received successful Long_Fast messages and tracerts Note: I only have one node close by that I can hop thru to reach other nodes (all my traffic must go thu it to get to other nodes), and it is NOT connected to MQTT. But other farther nodes that should hear that in between repeater-client are Green and Online on the map

Same here. Even when it shows blue lines between the nodes, I won't get any entries in the heard lists..

This is already shown on the map. You need to have uplink on a public channel, and turn on the neighbors module on all nodes. Set it to something like 1000 seconds. Then on the map, turn on the neighbors layer. You will see who heard what node, and if they can both be heard, the line is thicker.

Okay, tbh, I didn't know that.
But.. I think that the way connections are shown at the moment is pretty useless as it doesn't only show the nodes that directly hear each other via LoRa, but also the ones being connected over the internet.
What I would like to have is a view on the structure of the mesh.


Can't run with Linux/Mono/dotnet-cli

Please see: https://github.com/dotnet/cli/issues/3835
I believe there was an error in the last version (1.0.0) and even though Kestrel runs on Linux it still tries to call WS2_32.dll
Rolling back to RC2, everything works.
Thanks!

It's related to this https://github.com/dotnet/corefx/issues/9012.
PS: We don't support mono. I'd recommend following up on this issue there.

Hello David, thanks for replying!
Do you mean that because OS detection is broken at the moment Kestrel tries to load a Windows DLL? That explains it, I guess.
I tried the workaround in https://github.com/aspnet/KestrelHttpServer/issues/942 however I am still getting the DLL not found exception for WS2_32.dll. :(
I know Mono isn't supported, but the application I am working on has some libraries which are not yet compatible with .NET Core runtime and that's why I'm still targeting net451. Everything works super fast and great on Windows and Azure, by the way, it's just that I would like to have the ability to test the application on Linux too. Also, whilst still on the RC2 bits, there were some issues with Mono and EF Core date casting giving exceptions. Do you, by any chance know if that is Mono related or ASP.NET Core related?
Thanks for your help.

Do you mean that because OS detection is broken at the moment Kestrel tries to load a Windows DLL? That explains it, I guess.

Yep! The workaround won't work I'm assuming it's picking the windows dll with the fedora.24-x64 RID for some reason. You can try rebuilding after doing something like DOTNET_RUNTIME_ID=linux and see if that makes a difference

it's just that I would like to have the ability to test the application on Linux too

On unsupported mono? Test for what purpose?

Also, whilst still on the RC2 bits, there were some issues with Mono and EF Core date casting giving exceptions. Do you, by any chance know if that is Mono related or ASP.NET Core related?

No idea, we no longer test on mono.

@davidfowl, when did mono stop being apart of your test matrix? I don't recall seeing an announcement that mono was longer even tested on. There are use cases that .net coreclr doesn't yet support that mono does. Why remove it from the test matrix? Especially given that it was part of the original project announcement that mono was going to be a tested and supported platform.

We're currently working on the message. Will update things soon.

Yep! The workaround won't work I'm assuming it's picking the windows dll with the fedora.24-x64 RID for some reason. You can try rebuilding after doing something like DOTNET_RUNTIME_ID=linux and see if that makes a difference

Tried it, still not working but I am getting a different error: https://gist.github.com/nmilosev/30da19909b83570f8e4e4a59af401cfe

On unsupported mono? Test for what purpose?

I'm developing on Linux with Visual Studio Code, so it would be nice to be able to run the application without a virtual machine. I also tried Docker, but since it is also Linux (Ubuntu) inside, it does not work.
I'll stick to Windows VM for now, but I hope we will be able to use Mono to test/run full CLR applications on Linux, at least until more libraries are compatible with .NET Core.
Thanks again for your time David!

I made it work.
You have to use latest mono:
Mono JIT compiler version 4.4.1 (Nightly <IP_ADDRESS>/4747417 Thu Jun 23 18:55:25 UTC 2016)
Copyright (C) 2002-2014 Novell, Inc, Xamarin Inc and Contributors. www.mono-project.com
	TLS:           __thread
	SIGSEGV:       altstack
	Notifications: epoll
	Architecture:  amd64
	Disabled:      none
	Misc:          softdebug
	LLVM:          supported, not enabled.
	GC:            sgen

I installed it from Fedora Rawhide repository.
Afterwards you have to do the following:
# Fix for not finding System.Native
sudo ln -s /opt/dotnet/shared/Microsoft.NETCore.App/1.0.0/System.Native.so /usr/lib/libSystem.Native.so

# this is where libraries should be
cd /usr/lib/mono/xbuild-frameworks

# get all libraries here
cp -rf ../4.5/* .

# some of them are here
cp -rf Facades/* .

Application runs now, and everything is the same as with RC2, which means that the only thing not working is the date cast I mentioned. :( I think that's Mono related so I will try to see there or with the EF Core team.
Gonna try to fix that.

I was also having this problem on OSX, and when I looked at the debug output of Mono, I noticed that the reason System.Native wasn't being loaded was because Mono claimed it was the wrong architecture.  After digging a bit, this is what I found.
As you can see here: https://github.com/dotnet/cli/blob/04f40f906dce2678d80fb9787e68de76ee6bf57e/src/dotnet/commands/dotnet-run/RunCommand.cs#L194
CLI is just exec'ing 'mono'.  On my system (El Capitan on a new MacBook Pro), I installed mono via homebrew, and the mono binary in the path is located at /usr/local/bin/mono.  This file is just a symlink to /Library/Frameworks/Mono.framework/Commands/mono, which is the 32-bit Mono runtime.  The file command reveals that most of the framework libraries packaged with Mono are universal Mach O-Files (containing both 64-bit and 32-bit images), while the System.Native.dylib shipped with CLI preview2 is 64-bit only.
Setting MONO_ENV_OPTIONS='--arch=64'  (not MONO_OPTIONS) will cause Mono to re-exec itself using the 64-bit runtime, and the System.Native library should load just fine at that point.
P.S. - if mono isn't finding the System.Native library at all, then you can create symlinks (I did), or you can use DYLD_LIBRARY_PATH to tell dlopen where to find it.  In the end, this should work:
$ DYLD_LIBRARY_PATH=/usr/local/share/dotnet/shared/Microsoft.NETCore.App/1.0.0 MONO_ENV_OPTIONS='--arch=64' dotnet run <kestrel app>

Simply symlinking the library was enough to get it to work for me:
sudo ln -s /usr/share/dotnet/shared/Microsoft.NETCore.App/1.0.0/System.Native.so /usr/lib/libSystem.Native.so
sudo ldconfig

After that, running mono MySite.exe worked fine.


Do not indent string template starting at first position of line

Description
Do not indent string template starting at first position of line
Also, the indent rule should not change unexpected indentation characters inside a string template but leave this up to the string-template-indent rule. As a result the indent rule could be a bit simplified.
Closes #2350
Checklist
Before submitting the PR, please check following (checks which are not relevant may be ignored):

[X] Commit message are well written. In addition to a short title, the commit message also explain why a change is made.
[X] At least one commit message contains a reference Closes #<xxx> or Fixes #<xxx> (replace<xxx> with issue number)
[X] Tests are added
[X] KtLint format has been applied on source code itself and violations are fixed
[X] PR title is short and clear (it is used as description in the release changelog)
[X] PR description added (background information)

Documentation is updated. See difference between snapshot and release documentation

[ ] Snapshot documentation in case documentation is to be released together with a code change
[ ] Release documentation in case documentation is related to a released version of ktlint and has to be published as soon as the change is merged to master

Closes #2530


Make genesis file more resilient to bad NetworkMagic

@edsko has just spent day trying to debug a validation failure in the new node which resulted from the following chain of events:

The RequiresMagic was set incorrectly for mainnet.
When interpreting the AVVM addresses, makeRedeemAddress (Which lives in cardano-ledger) uses the network magic to construct the address. Thus it was incorporating the incorrect network magic.
When we got to the first transactions, they failed to lookup inputs correctly because of the differently encoded address. This happened before the network magic in their own outputs was checked and found not to conform.

There are a couple of ways we could improve this situation:

Rather than having the RequiresNetworkMagic as a separate configuration item, encode it directly as a flag in the config file. To preserve the integrity of the mainnet hash, the absence of this should be interpreted as RequiresNoMagic. Any other genesis file would be required to have this field present and set to a value.

AVVM addresses in the genesis file could be considered to have the network magic pre-encoded. This approach will work if the format in the genesis file is equal to the on-chain format with no network magic. All new files would need to have the network magic pre-encoded (and we would need to check that it was the same for each address)

Rather than having the RequiresNetworkMagic as a separate configuration item, encode it directly as a flag in the config file. To preserve the integrity of the mainnet hash, the absence of this should be interpreted as RequiresNoMagic. Any other genesis file would be required to have this field present and set to a value.

To clarify, is the suggested change here allowing the RequiresNetworkMagic: key in the .yaml config file to be absent? Currently I see, for example:
...
NumCoreNodes: 1
RequiresNetworkMagic: RequiresNoMagic
PBftSignatureThreshold: 0.7
...

are we proposing that we allow
...
NumCoreNodes: 1
PBftSignatureThreshold: 0.7
...

and have it be behaviorally equivalent (RequiresNoMagic)?

AVVM addresses in the genesis file could be considered to have the network magic pre-encoded. This approach will work if the format in the genesis file is equal to the on-chain format with no network magic. All new files would need to have the network magic pre-encoded (and we would need to check that it was the same for each address)

Does this equate to decoding the addresses as though they include a NetworkMagic? For mainnet addresses, if we do that, we will get out a NetworkMainOrStage. It's then suggested (I think) that we check all NetworkMagic values of these AVVM addresses for equality, which seems like a good idea.

I assumed the value was already optional, since it's presumably not present in the original mainnet genesis file?

Posting this from Slack discussion, now that I better understand the proposal:

Me:
proposal (1) is to drop one of these two non-independent things - the RequiresNetworkMagic key in the .yaml config. we then just look for either (a) the absence of a key which indicates mainnet/staging or (b) the presence of a key which indicates a testnet.
proposal (2) is to do the same thing, in effect, but have the AVVM addresses either (a) all contain NetworkMainOrStage (which encodes down to nothing in binary) or (b) all contain the same NetworkTestnet n where all identifiers n are equal

Doing both of these, as proposed above, should make it difficult/impossible(?) to trigger the above nasty failure mode.

Doing both of these, as proposed above, should make it difficult/impossible(?) to trigger the above nasty failure mode.

Yup. Indeed, would be good to test that :) Could do (2) first, run with the wrong value, and make sure that the check fails.

Is this still an issue?

@kevinhammond Yes this is still an issue.
AVVM addresses in the genesis file could be considered to have the network magic pre-encoded. This approach will work if the format in the genesis file is equal to the on-chain format with no network magic. All new files would need to have the network magic pre-encoded (and we would need to check that it was the same for each address)

The above still needs to be implemented in cardano-ledger which I will work on next.

OK. Do we need this for the Shelley testnet?

@Jimbo4350 Did we switch to this format? Regardless, it's now too late to change if we didn't, so closing this issue!


Fix markdown formatting

Apparently everyone has made these pull requests...
These are all the same: #99, #98, #97, #96, #95, #94

Whoops, totally missed this change from Github. Thanks!


[MXNET-137]fix parameters name inconsistent for Proposal OP and Multi Proposal OP

Description
Hi, there.
There is parameters name inconsistent in Proposal OP and Multi Proposal OP.
Proposal OP:
cls_prob vs cls_score
Multi Proposal OP:
cls_prob vs cls_score
It seems that the parameter name should be cls_prob rather than cls_score.
The parameter prompt
Faster R-CNN example
So I replace cls_score with cls_prob, and I change the parameter name in the unittest for Proposal OP and Multi Proposal OP.
Checklist
Essentials
Please feel free to remove inapplicable items for your PR.

[x] The PR title starts with [MXNET-$JIRA_ID], where $JIRA_ID refers to the relevant JIRA issue created (except PRs with tiny changes)
[x] Changes are complete (i.e. I finished coding on this PR)
[ ] All changes have test coverage:
Unit tests are added for small changes to verify correctness (e.g. adding a new operator)
Nightly tests are added for complicated/long-running ones (e.g. changing distributed kvstore)
Build tests will be added for build configuration changes (e.g. adding a new build option with NCCL)
[x] Code is well-documented:
For user-facing API changes, API doc string has been updated.
For new C++ functions in header files, their functionalities and arguments are documented.
For new examples, README.md is added to explain the what the example does, the source of the dataset, expected performance on test set and reference to the original paper if applicable
Check the API doc at http://mxnet-ci-doc.s3-accelerate.dualstack.amazonaws.com/PR-$PR_ID/$BUILD_ID/index.html
[x] To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change

Changes

[x] Replace the parameter name cls_score with cls_prob in Proposal OP and Multi Proposal OP
[x] Change the parameter name cls_score in the unit-test for Proposal OP and Multi Proposal OP

This breaks API compatibility. We can only change this if and when we decide to move these ops out of contrib.
Please put a comment in there as reminder

@piiswrong
Thank you!
But I found a problem.
When using mx.nd.Proposal or mx.nd.MultiProposal, the parameter name is cls_score.
When using mx.sym.Proposal or mx.sym.MultiProposal, the parameter name is cls_prob.

that's strange. Are you sure?

@piiswrong
Yes. I use the test case, code
If I replace cls_score with cls_prob in the code, it will raise the error:
Traceback (most recent call last):
  File "/home/wkcn/proj/testpy/testmxp.py", line 95, in <module>
    test(rpn_pre_nms_top_n, rpn_post_nms_top_n)
  File "/home/wkcn/proj/testpy/testmxp.py", line 58, in test
    rpn_min_size = rpn_min_size, output_score = True)
  File "<string>", line 82, in Proposal
  File "/home/wkcn/proj/mxnet/python/mxnet/_ctypes/ndarray.py", line 92, in _imperative_invoke
    ctypes.byref(out_stypes)))
  File "/home/wkcn/proj/mxnet/python/mxnet/base.py", line 149, in check_call
    raise MXNetError(py_str(_LIB.MXGetLastError()))
mxnet.base.MXNetError: Cannot find argument 'cls_prob', Possible Arguments:

But it's right to use cls_prob in mx.sym.contrib.Proposal in the code
The reason is that the function ListArguments() in proposal-inl.h returns {"cls_prob", "bbox_pred", "im_info"}, it's used in symbol.
However, .add_argument("cls_score", "NDArray-or-Symbol", "Score of how likely proposal is object.") in proposal.cc is used in ndarray.

ok...
I think we should make them consistent by using the sym version

@wkcn ping

@piiswrong
Hi!
The document shows that the sym version of Proposal OP uses cls_score.
However, the Faster R-CNN example uses cls_prob.
If making them consistent by using the sym version(cls_score), the old example and other detection projects(Deformable-ConvNets, mx-rcnn) will be an error of parameter.
I think using cls_prob is better. mxnet.ndarray.contrib.MultiBoxDetection uses cls_prob too.

Same strange issue: symbol and ndarray proposal operator api are inconsistent.
@wkcn Do you try change symbol parameter to cls_score that would not break API compatibility?

@WalterMa
Yes.
If you replace cls_prob with cls_score in this line, the symbol parameter will change to cls_score.

@wkcn
Thanks a lot!
Just personal opinion:
Would it be better to replace cls_prob with cls_score, since in both sym and nd API docs its parameter name is cls_score.
https://mxnet.incubator.apache.org/api/python/ndarray/contrib.html#mxnet.ndarray.contrib.Proposal
https://mxnet.incubator.apache.org/api/python/symbol/contrib.html#mxnet.symbol.contrib.Proposal
But In fact this will also break API compatibility, such as these code in example/rcnn.
Don't know what to do...
This issue has a direct impact on hybrid_forward in gluon that cannot hybridize network used Proposal operator.
Hope it be addressed soon.


Steel doesn't follow Windows audio output device changes

Example would be turning on steel while my speakers are set as the audio output device, then changing to my DAC. Steel will still output notification sounds to the speakers rather than the DAC.
Restart resolves this

https://github.com/RustAudio/rodio/issues/463
https://github.com/RustAudio/rodio/issues/327

this can probably be worked around if I take a look at https://github.com/ProjectAnni/anni/pull/39 and copy their homework (specifically, 00c54984b77863a451a15c88d823c919599dcac4 -- sorry pals)


Fix proto2 defaults

Example 1
There's breaking change in defaults for proto2 (1.2.2 .. master)
Probably related to #63
defmodule Proto2 do
  use Protobuf, """
    message M {
      optional string text = 1;
    }
  """
end

#1.2.1
iex() > Proto2.M.decode("")
%Proto2.M2{text: nil}
#1.22
iex() > Proto2.M.decode("")
%Proto2.M2{text: ""}
#master
iex() > Proto2.M.decode("")
%Proto2.M2{text: ""}
#this PR
iex() > Proto2.M.decode("")
%Proto2.M2{text: nil}

I don't think that implicit default values for basic types are worth losing the ability to see whether a field is present or not.
It could be configurable same way as in gpb (see Unset optionals and the default option in gpb readme), but I would prefer to fix breaking changes first and leave this improvement for separate PR
Example 2
Constructor functions in current version generate default values for required field. It shouldn't. In fact I would expect encode function to raise when there's no explicit value set for required field.
This is because function which generates default values is based on occurancy instead of syntax.
defmodule Proto2 do
  use Protobuf, """
    message M {
      required string x = 1;
      optional string y = 2;
    }
  """
end

iex> Proto2.M.new
%Proto2.M{x: nil, y: nil} #1.2.4
%Proto2.M{x: "",  y: nil} #master
%Proto2.M{x: nil, y: nil} #this PR

This looks great! Thanks for diving in and I apologize for the breaking change :(. I try hard not to introduce breakages in minor releases (or patch releases for that matter), but sometimes it's difficult for me to catch these things if I don't have the test coverage or encounter it myself.
Thanks again for the hard work!


Feature/string contains in any order

I kindly request to add stringContainsInAnyOrder as a complement to stringContainsInOrder. It matches if all of the substrings are contained within the subject, regardless of the order.
I'm aware that you are able to achieve the same with
assertThat("this is a string", allOf(
  contains("string"),
  contains("this"),
  contains("a")));

But there are cases when you need to work with an unknown list of strings, like String[] or List.
For these cases, it would be nice to have
String[] tokens = classUnderTest.getTokens();
String subject = getTemplate();
assertThat(subject, stringContainsInAnyOrder(tokens))

Just as another workaround for now, if you are on Java8:
assertThat(
    subject,
    allOf(Stream.of(tokens).map(Matchers::containsString).collect(Collectors.toList()))
);

@tpraxl looks a good idea, happy to look at getting merged if rebased from master as hamcrest-core and hamcrest-library has been merged into hamcrest so can't currently merge in at the moment.

Going to try and kick start hamcrest, so if you want to get it merged, please rebase from the branch v2.3-candidates.
Still trying to understand how has permissions to perform a release.


pull request for e2e test at 12/11/2016 21:26:02

:white_check_mark: Validation status: passed
For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.