Enabling kubeflow behind a proxy fails

I ran into a few issues when trying to run microk8s.enable kubeflow.
I'm using a Debian 9 machine behind a proxy.
First problem I ran into was that microk8s.enable couldn't download Juju:
$ microk8s.enable kubeflow
Enabling dns...
Enabling storage...
Enabling dashboard...
Enabling ingress...
Enabling rbac...
Enabling juju...
Kubeflow could not be enabled:
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (7) Failed to connect to launchpad.net port 443: Connection refused

Command '('microk8s-enable.wrapper', 'juju')' returned non-zero exit status 1
Failed to enable kubeflow

This was happening even after I told microk8s about the proxy.
I had to add the following to /etc/sudoers (using sudo visudo):
Defaults        env_keep="HTTP_PROXY HTTPS_PROXY http_proxy https_proxy NO_PROXY no_proxy"

This allowed the Juju install to complete successfully.
Suggestion 1: It doesn't look like the run_with_sudo function passes the proxy environment variables down to its sudo commands. It also doesn't seem like the commands that are run by run_with_sudo respect the proxy settings file. Could either of these options be implemented?

Once Juju installed correctly, I got the following error:
ERROR ensuring k8s credential "microk8s" with RBAC setup: ensuring cluster role "juju-credential-microk8s" in namespace "kube-system": Get https://<IP_ADDRESS>:16443/apis/rbac.authorization.k8s.io/v1/clusterroles/juju-credential-microk8s: Service Unavailable

I had to add <IP_ADDRESS>/16 to my NO_PROXY environment variable. Once I did that, I was able to contact the Kubernetes server correctly.
Suggestion 2: Would it be possible to have your install script override the NO_PROXY variable so that you add whatever subnet you're configuring for Kubernetes?
Suggestion 3: This is only semi-related and probably doesn't belong in this issue, but it would be great to have a "verbose" mode for microk8s.enable so that we can get more information about where things are falling over.

Looks like the install ran into another problem. After making the above configuration changes, microk8s.enable kubeflow sat for a long time in the "Deploying Kubeflow..." step then crashed with the following message:
$ microk8s.enable kubeflow
Enabling dns...
Enabling storage...
Enabling dashboard...
Enabling ingress...
Enabling rbac...
Enabling juju...
Deploying Kubeflow...
Kubeflow could not be enabled:
Creating Juju controller "uk8s" on microk8s/localhost
Creating k8s resources for controller "controller-uk8s"
Downloading images
Starting controller pod
Bootstrap agent now started
Contacting Juju controller at <IP_ADDRESS> to verify accessibility...
ERROR unable to contact api server after 1 attempts: unable to connect to API: Service Unavailable

Command '('microk8s-juju.wrapper', 'bootstrap', 'microk8s', 'uk8s')' returned non-zero exit status 1
Failed to enable kubeflow

I added <IP_ADDRESS>/8 to NO_PROXY.
After doing that, things got a bit further, but crashed here:
$ microk8s.enable kubeflow
Enabling dns...
Enabling storage...
Enabling dashboard...
Enabling ingress...
Enabling rbac...
Enabling juju...
Deploying Kubeflow...
Kubeflow could not be enabled:
Located bundle "cs:bundle/kubeflow-170"
Resolving charm: cs:~kubeflow-charmers/ambassador-54
Resolving charm: cs:~kubeflow-charmers/argo-controller-53
Resolving charm: cs:~kubeflow-charmers/argo-ui-54
Resolving charm: cs:~kubeflow-charmers/jupyter-controller-54
Resolving charm: cs:~kubeflow-charmers/jupyter-web-56
Resolving charm: cs:~kubeflow-charmers/katib-controller-52
Resolving charm: cs:~charmed-osm/mariadb-k8s
Resolving charm: cs:~kubeflow-charmers/katib-manager-50
Resolving charm: cs:~kubeflow-charmers/katib-ui-46
Resolving charm: cs:~kubeflow-charmers/kubeflow-dashboard-11
Resolving charm: cs:~kubeflow-charmers/kubeflow-gatekeeper-14
Resolving charm: cs:~kubeflow-charmers/kubeflow-login-13
Resolving charm: cs:~kubeflow-charmers/kubeflow-profiles-16
Resolving charm: cs:~kubeflow-charmers/metacontroller-43
Resolving charm: cs:~kubeflow-charmers/metadata-api-6
Resolving charm: cs:~charmed-osm/mariadb-k8s
Resolving charm: cs:~kubeflow-charmers/metadata-ui-10
Resolving charm: cs:~kubeflow-charmers/minio-54
Resolving charm: cs:~kubeflow-charmers/modeldb-backend-49
Resolving charm: cs:~charmed-osm/mariadb-k8s
Resolving charm: cs:~kubeflow-charmers/modeldb-store-45
Resolving charm: cs:~kubeflow-charmers/modeldb-ui-44
Resolving charm: cs:~kubeflow-charmers/pipelines-api-58
Resolving charm: cs:~charmed-osm/mariadb-k8s
Resolving charm: cs:~kubeflow-charmers/pipelines-persistence-53
Resolving charm: cs:~kubeflow-charmers/pipelines-scheduledworkflow-55
Resolving charm: cs:~kubeflow-charmers/pipelines-ui-53
Resolving charm: cs:~kubeflow-charmers/pipelines-viewer-55
Resolving charm: cs:~kubeflow-charmers/pytorch-operator-55
Resolving charm: cs:~kubeflow-charmers/tf-job-dashboard-55
Resolving charm: cs:~kubeflow-charmers/tf-job-operator-53
ERROR cannot deploy bundle: cannot add charm "cs:~kubeflow-charmers/ambassador-54": cannot retrieve charm "cs:~kubeflow-charmers/ambassador-54": cannot get archive: Get https://api.jujucharms.com/charmstore/v5/~kubeflow-charmers/ambassador-54/archive?channel=stable: dial tcp: lookup api.jujucharms.com on <IP_ADDRESS>:53: server misbehaving

Command '('microk8s-juju.wrapper', 'deploy', 'cs:kubeflow', '--channel', 'stable', '--overlay', '/tmp/tmpo8d2nud_')' returned non-zero exit status 1
Failed to enable kubeflow

My theory at this point is that the proxy environment variables aren't being moved into whatever container is at <IP_ADDRESS>, which means it can't access the outside world.
Any thoughts on how to fix this?

In regards to 2, you can set the KUBEFLOW_NO_PROXY environment variable to have Juju ignore a proxy for that IP range while bootstrapping, which seems to be what you're looking for.
For 3, you can set the environment variable KUBEFLOW_DEBUG=true to get verbose logging.
Additionally, #989 includes the Juju binary in the snap instead of downloading it, so you won't have that issue going forward as soon as that's merged.

Thanks, @knkski! Good to know there's options for this.
I don't suppose you have any suggestions on how to deal with <IP_ADDRESS> not being able to reach api.jujucharms.com?

@Helumpago: Is it able to reach api.jujucharms.com now? I would guess that you were just encountering a network issue.

Hi,
I am also behind a proxy and juju download is working fine with channel edge now.
However, I am a CONNECTnotallowed error while juju tries to contact the api server.
18:59:31 DEBUG juju.kubernetes.provider events.go:51 getting the latest event for "involvedObject.name=controller-0,involvedObject.kind=Pod"
18:59:31 INFO  cmd bootstrap.go:729 Starting controller pod
18:59:31 INFO  cmd bootstrap.go:613 Bootstrap agent now started
18:59:31 DEBUG juju.kubernetes.provider events.go:51 getting the latest event for "involvedObject.name=controller,involvedObject.kind=StatefulSet"
18:59:31 INFO  juju.juju api.go:302 API endpoints changed from [] to [<IP_ADDRESS>:17070]
18:59:31 INFO  cmd controller.go:89 Contacting Juju controller at <IP_ADDRESS> to verify accessibility...
18:59:31 INFO  juju.juju api.go:67 connecting to API addresses: [<IP_ADDRESS>:17070]
19:09:29 ERROR juju.cmd.juju.commands bootstrap.go:778 unable to contact api server after 1 attempts: unable to connect to API: CONNECTnotallowed
19:09:29 DEBUG juju.cmd.juju.commands bootstrap.go:779 (error details: [{/workspace/_build/src/github.com/juju/juju/cmd/juju/common/controller.go:128: unable to co     ntact api server after 1 attempts} {/workspace/_build/src/github.com/juju/juju/cmd/juju/common/controller.go:44: } {/workspace/_build/src/github.com/juju/juju/cmd/     modelcmd/modelcommand.go:405: } {/workspace/_build/src/github.com/juju/juju/cmd/modelcmd/modelcommand.go:424: } {/workspace/_build/src/github.com/juju/juju/cmd/mod     elcmd/base.go:214: } {/workspace/_build/src/github.com/juju/juju/juju/api.go:72: } {/workspace/_build/src/github.com/juju/juju/api/apiclient.go:207: } {/workspace/     _build/src/github.com/juju/juju/api/apiclient.go:622: } {/workspace/_build/src/github.com/juju/juju/api/apiclient.go:967: } {/workspace/_build/src/github.com/juju/     juju/api/apiclient.go:1071: unable to connect to API} {/workspace/_build/src/github.com/juju/juju/api/apiclient.go:1096: } {CONNECTnotallowed}])
19:09:29 DEBUG juju.cmd.juju.commands bootstrap.go:1424 cleaning up after failed bootstrap

I have already tried setting the KUBEFLOW_NO_PROXY but I got the same error.
Steps to reproduce:
snap install microk8s --classic --channel=edge
# no proxy setting:
# printf -v kube_no_proxy '%s,' 10.152.183.{1..255};
# export KUBEFLOW_NO_PROXY=${kube_no_proxy%,},localhost,<IP_ADDRESS>,::1
KUBEFLOW_DEBUG=true microk8s.enable kubeflow

I've met same issue. I've configured coredns to resolve address (as explained here ) but can't download through proxy
How to configure containerd? containerd-env file is not hepling

@blacksailer have you tried the instructions in https://microk8s.io/docs/install-proxy ? What is the state of your cluster? Can you attach an microk8s inspect tarball? Thank you. Apologies for the late reply.

Well, I removed it already and installed through kubeadm.
Yes, I've tried instructions in site, but juju was complaining on network, will check it later on week. Also tried to configure env variables for proxy (juju_no_proxy, juju_http_proxy) for juju, didn't help

Hello,
I am experiencing similar issues running behind a proxy. I have already followed the proxy configuration in https://microk8s.io/docs/install-proxy without any improvement.
When enabling kubeflow, it stuck for a while on "deploying kubeflow" and eventually returns
ERROR unable to contact api server after 1 attempts: Gateway Timeout
when contacting juju controller.
It is not clear to me what should I add under the "NO_PROXY" variable.
Anyway, the problem seems bigger than kubeflow itself. After I deploy any pod, if I try to exec any command, it seems kubectl can't communicate with the pod at all:
Error from server: error dialing backend: dial tcp <IP_ADDRESS>:10250: i/o timeout
Is there any way of making this work today?
Thanks

@jameinel: do you know what values should be put in the no-proxy configuration?


Spec for keeping Upload-Expires header current

For massive uploads, we don't want the expiration taking out a file that is being actively uploaded. So instead, add into the spec a way for the Upload-Expires header to be updated with each PATCH request, thus allowing us to prune abandoned uploads faster without causing problems to those still uploading.

I agree with @nigoroll here. This seems like a implementation detail. tusdotnet already supports this by letting the developer specify an AbsoluteExpiration or a SlidingExpiration. Sliding will be updated on each PATCH request and reflected in each HEAD request.


LYTBoolValues() always renders as 1 in html report

LayoutTest 1.3.0
Both true and false are being passed to viewForData, but in the html report it is always printing as 1.

Reproduced. Will take a look.

https://github.com/linkedin/LayoutTest-iOS/pull/23

Fixed. Will get into the next release. Hopefully with the other bug fix, but can't work out whats wrong there.


Make 'lr' optional parameter

While probably 90+ % of the PyTorch optimizers have lr rate as field some exceptions exist.
I for example am often using the "Learned Optimizer List" optimizer (https://github.com/google-research/google-research/tree/master/opt_list), which does not have lras field.
(I'm already preparing my code base for an upgrade to v2.0 and this is one of the things that work in v1.x but not in v2)

Thanks!


Azure VM template

Summary
Provide an Azure VM template for OSCM
Details

[ ] Investigate VM options to setup OSCM with cloud init.
[ ] Test the installation in Azure with respective user

Acceptance criteria

[ ] Possible error cases are handled with clear messages and error return code
[ ] Secure and GBDR conform system logging
[ ] OSCM required runtime is available after VM creation (docker, docker-compose....)

Additional context
Ask me

This task is closed, due to the creation of another task in which the "Azure VM" and "AD tenant" scripts will be transferred and tested to/in the dockerbuild repository Add scripts for setup with OIDC

This task is closed, due to the creation of another task in which the "Azure VM" and "AD tenant" scripts will be transferred and tested to/in the dockerbuild repository Add scripts for setup with OIDC


added new fab_size property for custom size requirements

new fab_size xml attribute
allows for setting custom size (sometimes it's an requirement)
if omitted, default values will be used

This is how it looks with a custom FAB size set to 42dp on the pre-Lollipop device:

The white outline exists around the button because shadow is used as an image asset of a constant size. I had to adapt this shadow image to fully match two standard sizes but it's not stretched to custom size.
So until we use a shadow as an image, it's not possible to set a custom FAB size (with shadow). But I assume we can somehow generate a shadow in the runtime. I will invesigate this possibility.

Aha, yes you are right, I think I tested it but may have missed to test it with sizes smaller than default.
What do you think about constraining the size not to be smaller than the smallest default size? (48dp, I guess?) I think there may be the requirement for larger buttons, but not so much for smaller ones.
Regarding generating runtime shadows, I did some experiments for custom dropshadows in another projects, basically black-painting the object, blur it, and draw it underneath the object. So this could work as well, although potential affections on performance should be considered.
If you don't mind I'll have a look into it as soon as I have time.
Thanks for the hint!

@sfunke Did you have time to look into this again?

@MGaetan89 Yes, I just added a fix on my fork, hopefully it gets pulled in. It's true that it's a bit whacky on pre-Lollipop devices to get the shadow straight. The current workaround is that the fixed size shadow drawable gets stretched accordingly, plus taking the shadow margin into account.
Also I decided, that when fab_type="mini" is explicitly set, I ignore the fab_size property (so that FAB is shown in "mini" appearance, and it just gets it's "mini"-shadow applied.)
It's been tested on Lollipop & pre-Lollipop, as well with larger & smaller FAB-sizes, and on different resolutions.
@makovkastar What do you think?


Removed all occurrences of Open Science Grid and Open Science Grid (OS…

…G) and replaced with OSG.

Also changed out the image
I did not attempt to add OSG to the image as I [don't]() think I could effectively do it and maintain the same dimensions

@bbockelm This merge is in response to your issue of changing Open Science Grid to OSG. I did a search and replace on all occurrences, and changed the image to the blank OSG logo as shown in the screenshot.


ignore URLs

Hello and thanks for the lint rule!
Unfortunately, it triggers many false positives with URLs, like the following:
/**
  * This function is based on the following code:
  * https://github.com/typescript-eslint/typescript-eslint/blob/master/packages/eslint-plugin/docs/rules/member-ordering.md
  */
function foo() {}

Would it be possible to add a ignoreUrls option that defaults to true, like ESLint's max-len rule does?
Thank you!

For future readers, I decided to fork this rule and implement the functionality myself: https://github.com/IsaacScript/eslint-plugin-isaacscript/

Thanks for the pull request (#2) @Zamiell which fixes this.


Add mimetype renderer

Overview
This PR introduces a new Altair data transformer and renderer for displaying pre-transformed Vega specifications using standard Jupyter renderers (vega, html, svg, and png).
This renderer provides most of the benefits of VegaFusion without requiring the custom VegaFusionWidget renderer.
Usage
The new data transformer and renderer are activated using the vegafusion.enable_mime() function.
import altair as alt
from vega_datasets import data

source = data.movies()

chart = alt.Chart(source).mark_bar().encode(
    alt.X("IMDB_Rating:Q", bin=True),
    y='count()',
)

# Display chart with default Altair renderer
chart

# Display chart with vegafusion mimetype renderer
import vegafusion as vf
vf.enable_mime(mimetype="html")
chart

Other supported mimetype values are vega, svg, and png
How it works
The mimetype renderer uses an approach that is very different than the existing Jupyter Widget-based renderer.
Here is an outline of the process
1. vegafusion-inline data transformer
Inline DataFrames are extracted into a global dict by the new vegafusion-inline data transformer. The DataFrames are replaced by VegaFusion inline dataset URLs (e.g. vegafusion+dataset://table-{uuid}). This prevents DataFrames from being serialized directly to JSON.
2. compile Vega-Lite to Vega
The vegafusion-mime renderer inputs a Vega-Lite spec that has already gone through step 1.  In compiles the Vega-Lite to a Vega spec using the active plugin in the new vf.vegalite_compilers plugin registry. The default plugin uses vl-convert to perform the conversion, but it's possible to register custom compilation functions as well.
3. Pre-transform Vega spec
The Vega spec that results from step 2 is processed using the vegafusion.runtime.pre_transform_spec function.  This function inputs a Vega spec, and DataFrames extracted in step 1, extracts and evaluates as many transforms as possible, and then produces a new Vega spec with the transformed data inlined.
Unlike the VegaFusionWidget renderer, the resulting Vega spec does not require communication with the server to maintain interactivity. All of the data required for interactivity is inlined in the pre-transformed Vega spec. The benefit is that this pre-transformed spec can be renderered just like a normal Vega spec.
4. Produce mime bundle
The renderer uses this pre-transformed Vega spec to generate mimetype bundles for the various mime types. The default mime type is "text/html", which works in the classic Jupyter Notebook and JupyterLab. The "application/vnd.vega.v5+json" mime type works in JupyterLab and other environments that provide a Vega mimetype renderer. The "image/svg+xml" mimetype uses vl-convert to convert the pre-transformed Vega spec to an SVG image and similarly the "image/png" mimetype uses vl-convert to create a PNG image.
What's the benefit?
Here are three examples that demonstrate the benefit of using the VegaFusion mime renderer compared to the default Altair renderer.  The specs themselves are below, but I thought it would be easier to walk through the benefits in a screencast.
https://www.loom.com/share/094b3958ffad49bdaa3a7af19f26370b
Example 1: Aggregated chart
import altair as alt
from vega_datasets import data

source = data.movies()

agg_chart = alt.Chart(source).mark_bar().encode(
    alt.X('IMDB_Rating:Q', bin=True),
    alt.Y('count()')
)
agg_chart

Example 2: Non-aggregated scatter chart
import altair as alt
from vega_datasets import data

source = data.movies()

scatter_chart = alt.Chart(source).mark_circle().encode(
    alt.X('IMDB_Rating:Q'),
    alt.Y('Rotten_Tomatoes_Rating:Q'),
    alt.Tooltip(["IMDB_Rating", "Rotten_Tomatoes_Rating", "MPAA_Rating", "Worldwide_Gross"])
)
scatter_chart

Example 3: Interactive Chart
import altair as alt
from vega_datasets import data

source = data.seattle_weather()
brush = alt.selection(type='interval', encodings=['x'])

bars = alt.Chart().mark_bar().encode(
    x='month(date):O',
    y='mean(precipitation):Q',
    opacity=alt.condition(brush, alt.OpacityValue(1), alt.OpacityValue(0.7)),
).add_selection(
    brush
)

line = alt.Chart().mark_rule(color='firebrick').encode(
    y='mean(precipitation):Q',
    size=alt.SizeValue(3)
).transform_filter(
    brush
)

interactive_chart = alt.layer(bars, line, data=source)
interactive_chart

Comparison to the widget renderer.
The existing VegaFusion renderer can be enabled using vegfusion.jupyter.enable() or (now in this PR) vegafusion.enable_widget().  Here are some details on how the two renderers compar.

The widget renderer writes DataFrames to arrow files on disk in a _vegafusion_data directory while the mime renderer handles everything in memory without temporary files
The widget renderer requires the VegaFusionWidget be installed in the front end (e.g. the Jupyter Notebook), and interactive charts require communication with a running Python kernel.  The mime renderer requires no custom client logic as charts are displayed using the regular Vega embed renderer.
The widget renderer transmits transformed data from the server to the client in Arrow IPC format while the mime renderer serializes transformed data as JSON to inline in the resulting Vega specification.  Neither renderer serializes the input DataFrame to JSON as is required by the default Altair renderer.

Looks great! Question, will the following also work?:
with vf.enable_mime(mimetype="html"):
    chart

No, but that's a good idea! https://github.com/vegafusion/vegafusion/issues/200


Integrate with ink-wrapper

We can provide drink as an alternative backend to ink-wrapper (https://github.com/Cardinal-Cryptography/ink-wrapper).

Currently being done in https://github.com/Cardinal-Cryptography/ink-wrapper/pull/67


Change hbase.regionserver.thrift.framed and hbase.regionserver.thrift.compact to True?

Thanks for providing the container! After pulling the image we notice that hbase.regionserver.thrift.framed is set to False in the configuration file.
However, the HBase official document and Cloudera troubleshooting page recommends to set hbase.regionserver.thrift.framed and hbase.regionserver.thrift.compact to True :

This is the recommended transport for thrift servers and requires a similar setting on the client side. Changing this to false will select the default transport, vulnerable to DoS when malformed requests are issued due to THRIFT-601.

To prevent the possibility of crashes due to buffer overruns, use the framed and compact transport protocols by setting hbase.regionserver.thrift.framed and hbase.regionserver.thrift.compact to true in hbase-site.xml.

Maybe it's better to enable these two parameters? Thank you!

Thanks for pointing out this. I have added that to the default configuration files and rebuilt the images.
BTW, you can mount your own configurations files to /config/hbase and they will be replaced automically at startup.
docker service create \
	--name hbase-master \
	--hostname hbase-master \
	--network swarm-net \
	--replicas 1 \
	--detach=true \
	--endpoint-mode dnsrr \
	--mount type=bind,source=/etc/localtime,target=/etc/localtime,readonly \
	--mount type=bind,source=/data/hbase/config,target=/config/hbase \
	--mount type=bind,source=/data/hbase/logs/master,target=/usr/local/hbase/logs \
	newnius/hbase:1.2.6

Thank you for changing the configuration and the response! It can be helpful.


Add docs tooltip

To support changes in https://github.com/murcia-cs4215/ocontract/pull/24.
Now shows the documentation tooltip:

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

1 of 20 (5.0%)  changed or added relevant lines in 3 files are covered.
1 unchanged line in 1 file lost coverage.
Overall coverage decreased (-0.008%) to 6.563%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

src/commons/sagas/WorkspaceSaga.ts
0
5
0.0%

src/commons/documentation/Documentation.ts
0
14
0.0%

Files with Coverage Reduction
New Missed Lines
%

src/commons/documentation/Documentation.ts
1
0%

Totals

Change from base Build<PHONE_NUMBER>:
-0.008%

Covered Lines:
138

Relevant Lines:
2059

💛  - Coveralls


Get the docs running

We're in the process of migrating the docs away from Chiller to use Starlight by Astro instead, which should make it easier to support multiple languages (our main priority with the docs).

Can I assist with this ?

Appreciate the offer, @stefaanv. All that's left to do is a bit of documentation and a couple small bugs. The site will be up this week. 😄

The docs are now live at: https://radashi-org.github.io
If you have any ideas about improving Radashi's messaging (e.g. how we pitch Radashi to potential new users), please share your thoughts in this discussion.


make  controllers. minterManager private and create getter functions

Consider restricting the visibility of the controllers and minterManager variables to internal or private and providing the required getter functions as a good encapsulation practice.

https://github.com/centrehq/centre-tokens/pull/277


Custom format enabled, TR-DOS 5.25 disk can read only one side

Testing new beta feature- custom format. Can not force to read some hacked ZX Spectrum TR-DOS disk. It is 16 sectors, 256 bytes/sector format. Somehow I can read only one side. Looks like there is data on other side too.
disk zx.trdos.640
cyls = 80
heads = 2
tracks *:ibm.mfm
secs = 16
bps = 256
end
end
(other params are unknown for me)
Please check attached flux image.
TRDOS_D35.zip

Looks like the head (H) field of sectors is 0 on both sides?
I assume this is two single-sided images. Perhaps someone used a side-select switch to manually select side 1?
You should be able to dump as follows:

Change to heads = 1 in your config.
Dump Side A
Dump Side B by adding --tracks=hswap to your command line.

I created TR-DOS disk in emulator. Both "sides" have id=0...
TRDOS.zip

If it is actually a two-sided disk image with both sides having head fields with H=0, then add h=0 inside your tracks definition block.
You can define a disk with multiple different track formats by declaring multiple tracks blocks within a disk. You can specify exactly which tracks each block applies to by changing the * to an explicit track list. For example 1-5.0 would be cylinders 1-5, head 0. If you get stuck send me a disk and I will make a disk definition for you as an example.

Then you want
disk zx.trdos.640
cyls = 80
heads = 2
tracks *:ibm.mfm
secs = 16
bps = 256
h = 0
end
end

Yes! It works. At least it is reading. As TR-DOS is "normal" OS for ZX spectrum, I hope it will work with fuse emulator (this emulator has more options compared to others). Will report the result.

It is interesting if TR-DOS always marks both sides as 0. For example, FlashFloppy TRD image handler does not do this. I wonder how well tested FlashFloppy is with TRD images? Do you know of anyone using FF with TRD images?

TR-DOS reading confirmed using FUSE emulator. It can boot/run games from disk image if BETA disk controller is enabled in 80 track mode.
GW can not write directly to .trd format, so the only way is to use HxC to convert from .hfe to .trd or .dsk


update paths

it seems like the path to the examples changed, but the docs were not updated, so currently links are 404 💗

Thanks for spotting this!


 Such a shame, used to be so easy. Anyway, do you think you could implement this as an action? See https://github.com/Schamper/nodebb-plugin-shoutbox/tree/master/public/js/lib/actions

Let me know if you ever rework this.


check an index of histo

I got a segmentation fault in production.
segfault at 2284c808 ip<PHONE_NUMBER>412301 sp 00007f82c65aae80 error 6 in memcached[400000+20000]

The version we are using is 1.4.28. The segmentation fault occurs here
I used memcached over than 6 months and 130 servers. But it occurred only once.
Probably current_time will be greather than search->exptime at items.c#L1300.
But I couldn't reproduce its phenomenon at 1.4.28 and 1.4.36.
However waiting 1 second before current_time subtracted from search->exptime did easily reproduced.
So check the bucket before using.

Good catch! Will review this soon.

sorry for the long delay. this should go out in the next release.
thanks again!


"ITMS-90426: Invalid Swift Support"

Description
as use only c++ and cmake, including firebase ios sdk requires to also include Swift language support into cmake.
which I did, and I can build and deploy the app just fine.
However when uploading generaed app to TestFlight / AppStore it gives me following error:

when I open generated *.app to check, it realy does not contain SwiftSupport folder.
So I guess I need to tell Cmake to require generating this folder upon build?
cmakes generate xproject which I open in xcode to build...
In the documentations neither examples I found anything how can I influence this from Cmake, neither directly set in xcode.
same/similar question unaswered here https://github.com/firebase/firebase-ios-sdk/issues/10006
Reproducing the issue
No response
Firebase SDK Version
11.3
Xcode Version
16.0.1
Installation Method
N/A
Firebase Product(s)
Analytics, Messaging
Targeted Platforms
iOS
Relevant Log Output
No response
If using Swift Package Manager, the project's Package.resolved
No response
If using CocoaPods, the project's Podfile.lock
No response

Transferred to the firebase-cpp-sdk repo to investigate how to apply the guidance from https://github.com/firebase/firebase-ios-sdk/issues/10006 to C++ and cmake.

@paulb777 have you created issue for firebase-cpp-sdk? can i please have a link?

https://github.com/firebase/firebase-cpp-sdk/issues/1666

thank you, however i still think its pure ios sdk issue, as this happens only when to c++ project load swift stuff... which happens only in firebase ios sdk :)

The solution for the pure iOS issue is described in https://github.com/firebase/firebase-ios-sdk/issues/10006. it sounds like you need additional help for a C++ context.

I dont see any solution there... theoretical solution was to include swift file which I already did with no success, and this was also mentioned by @ndahlquist that he did and didnt help. "contains embedded Swift" also can say didnt help me neither for that topic. No other solution was proposed there yet


⚠ Bofrost.de has degraded performance

In 1681224, Bofrost.de (https://www.bofrost.de) experienced degraded performance:

HTTP code: 200
Response time: 2208 ms

Resolved: Bofrost.de performance has improved in 9f6b07a after 6 minutes.


Revert "fix(v2): remove auto wrap for code blocks (#2048)"

Motivation
As mentioned in https://github.com/facebook/docusaurus/pull/2056
and https://github.com/facebook/docusaurus/pull/2048#issuecomment-558557240
I think it might be best to revert #2048 first. If we want to attempt on removing the codeblock autowrap, we can do it in new PR. Lot of changes has been added since alpha.36 and I think it'd be best to release soon so we can dogfood versioning & lot of other critical bugfix can land.
Because I'm seeing some unpleasant layout that is kinda zoomed and cut off. Notice the scrollbar
https://v2.docusaurus.io/docs/next/lifecycle-apis/

https://v2.docusaurus.io/docs/next/markdown-features

Have you read the Contributing Guidelines on pull requests?
yes
Test Plan
No longer zoomed

We can close this if we find a solution. This is quite a temporary solution, because I was hoping to release alpha.37 soon since there has been too many commits already 😆
I do have another concern though. CodeBlock is one of the most swizzled component in Docusaurus 2. Because to add new language, you need to swizzle CodeBlock. Need to remind user to update their CodeBlock css when I release new alpha version if we're not reverting.

Let's revert this first.


Extend API to support AWS AppSync with custom auth provider

Is your feature request related to a problem? Please describe.
AWS AppSync SDK is now in Maintenance and users are recommended to switch to Amplify library for Swift.
However amplify-swift API lacks functionality (configuration) of AWS AppSync SDK.
Consider an existing AppSync endpoint (NOT managed by an Amplify project).
In AWS AppSync SDK the configuration requires providing of the endpoint URL and an implementation of auth provider, example:
let client = try AWSAppSyncClient(url: url, authProvider: authProvider)

amplify-swift does not provide type safe interface to configure the endpoint;
the only way is via a JSON file which is:

not type safe and
inconvenient when using different configurations (for example for testing or QA).

/// API Category Configuration
public struct APICategoryConfiguration: CategoryConfiguration {

    /// Plugin keys to plugin configuration
    public let plugins: [String: JSONValue]

    /// Initializer for API configuration
    ///
    /// - Parameter plugins: plugin configuration map
    public init(plugins: [String: JSONValue] = [:]) {
        self.plugins = plugins
    }
}

The JSON configuration is limited to a few predefined authorization modes and does not support explicitly setting of the access token;
Note that is possible in AWS AppSync SDK by providing an implementation of AWSCognitoUserPoolsAuthProviderAsync.
public protocol AWSCognitoUserPoolsAuthProviderAsync: AWSCognitoUserPoolsAuthProvider {
    func getLatestAuthToken(_ callback: @escaping (String?, Error?) -> Void)
}

(Its a critical functionality in my projects which use custom authentication endpoint.)
Describe the solution you'd like
Please:

extend amplify-swift API client to allow custom implementation AppSync auth provider
extend amplify-swift API client to support type safe explicit configuration of AppSync endpoints
allow explicit configuration of subscription endpoints (currently its hardcoded), see https://github.com/awslabs/aws-mobile-appsync-sdk-ios/issues/506

Describe alternatives you've considered
Looking into 3rd party GraphQL clients for Swift.
However I would prefer to keep using a library backed up by AWS.
Is the feature request related to any of the existing Amplify categories?
No response
Additional context
No response

@pokryfka Thanks for submitting the issue.
Have you look at using the Amplify Auth category to support your authentication use cases? https://docs.amplify.aws/lib/auth/getting-started/q/platform/ios/
Please also refer to the Amplify API category documentation for additional details on setting up and using API: https://docs.amplify.aws/lib/graphqlapi/getting-started/q/platform/ios/#initialize-amplify-api
There is also several authorization modes supported in the API category.  Documentation can be found here: https://docs.amplify.aws/lib/graphqlapi/authz/q/platform/ios/#oidc
The Amplify API category can be configured using the Amplify CLI.  This would manage and generate the amplifyconfiguration.json file which automatically used to configure the API plugin at Amplify initialization and configuration time.

@phantumcode Thank you for the references.
I have not tested it but https://docs.amplify.aws/lib/graphqlapi/authz/q/platform/ios/#oidc seems to allow implementation of custom auth provider for AppSync.
However as of now it is inconvenient and not type safe to configure that.
Please correct me if I am wrong but it does not seem possible at all to use Amplify CLI without a Amplify project in AWS?
I just want to use existing AppSync endpoints - which I deploy via serverless - and I dont need AWS Amplify project for that.
Is it possible to extend API of the library so that Amplify CLI and the JSON configuration file are optional?
Otherwise amplify-swift is not really a replacement for AWS AppSync SDK

Hi @pokryfka, thank you for the details on describing the solutions you'd like. I agree a type-safe way to provide the configuration will make it less coupled with the Amplify CLI's amplifyconfiguration.json file. In the current architecture, the entry point in configuring the plugins is two steps:

Adding the plugin to Amplify, ie. Amplify.add(plugin: AWSAPIPlugin())
Calling Amplify.configure() or Amplify.configure(config)

If calling Amplify.configure(), this will use amplifyconfiguration.json. If calling Amplify.configure(config), config is of type AmplifyConfiguration
I can see that if we go the route of using an instance of AmplifyConfiguration, then we would have to create the configuration for every plugin in-memory. We can't have, for example, the auth configuration in amplifyconfiguration.json while having the AWSAPIPlugin's configuration in-memory. It is either the entire configuration from amplifyconfiguration.json or the entire config from the AmplifyConfiguration object. I assume this may not be reasonable for you since you will most likely have the AWSCognitoAuthPlugin and that configuration is quite large, generated in amplifyconfiguration.json, but feel free to see if using AmplifyConfiguration gets your code in a place that works for you.
What if we exposed a public configuration struct from AWSAPIPlugin and extend its initializer to take it in, and implement that it overrides conflicting sections in the the amplifyconfiguration.json file/AmplifyConfiguration object?
The DX could look something like this
// `amplifyconfiguration.json` is added to the target, and may contain other plugin configurations
// ie. storage, auth, etc.

// Create an in-memory AWSAPIPlugin configuration that will override any config
// from `amplifyconfiguration.json`
let configuration = AWSAPIPluginConfiguration(
    apis: ["apiName": .init(endpointType: .graphQL,
                            endpoint: URL(string: urlString),
                            region: "us-east-1",
                            authorizationType: .userPool)])
Amplify.add(plugin: AWSAPIPlugin(configuration: configuration))
Amplify.configure()

Would this developer experience solve the problem for you?

Thank you following up on that.
Personally I think having a type safe well defined configuration type, like in your example, will be very convenient.
Especially for project using AppSync without Amplify (mine scenario and all other projects which used to use AppSync SDK).
Please note, or correct me if I am wrong here, that the JSON file cannot be generated without an Amplify project. Hence I need to reverse engineer the JSON file syntax.
I think either JSON file or in memory is very reasonable.
If I understand documentation correctly, In my case, I'd need to use OPENID_CONNECT authorizationType and then add  auth provider factory:
try Amplify.add(plugin: AWSAPIPlugin(apiAuthProviderFactory: MyAPIAuthProviderFactory()))

I am assuming this would still work with in memory configuration.

Yes that's right, if you use AppSync without an amplify project, then you won't be able to get the amplifyconfiguration.json file and have to manually create this file (or create the in-memory AmplifyConfiguration object). This fields are documented here: https://docs.amplify.aws/lib/graphqlapi/existing-resources/q/platform/ios/
There are a few options to defining your own interceptor, the one you have above is the ODIC use case. You can also set it to NONE and create a URLRequestInterceptor (https://docs.amplify.aws/lib/graphqlapi/authz/q/platform/ios/#none).
How are you defining the type to decode the response from AppSync? Do you hand write these? I'm aware that Amplify CLI provides a few tools to create types to use in the app

amplify codegen types produces the  API.swift
amplify codegen models produces the swift model files from the source schema (the source schema is the one that's annotated using amplify directives like @model)

I tried to configure swift-amplify but the more I get familiar with ir the more it seems that its by design VERY different than AWS AppSync SDK and cannot be used as its replacement.
I used to generate Swift code using GQL schema and GQL query/mutations/subscriptions with
amplify codegen types.
But it does not seem possible to use GQL in amplify, unless GQL types are decorated with proprietary decorators like @model or @searchable in which case amplify codegen models a lot of CRUD queries and mutations in an opinionated way.
--
TL;DR
@lawmicha I appreciate your comments but I dont think its possible to use amplify-swift as GraphQL (AppSync) client the way AWS AppSync SDK works; they both have been designed to work very differently (even though amplify can use AppSync under the hood)

amplify codegen types produces the  API.swift

^ this is the way I generate code

In Amplify CLI 12.3.0, we introduced a compatibility layer so you can re-use API.swift with Amplify.API, please take a look at https://docs.amplify.aws/lib/graphqlapi/upgrade-guide/q/platform/ios/

In Amplify CLI 12.3.0, we introduced a compatibility layer so you can re-use API.swift with Amplify.API, please take a look at https://docs.amplify.aws/lib/graphqlapi/upgrade-guide/q/platform/ios/

I appreciate your prompt response.
I think while AWS AppSync SDK was an implementation of GraphQL client (based on Apollo and tested by AWS team to work with AppSync),
amplify-swift tries to do much much more than just being a GraphQL client.
This may have some benefits - for example, it generates cloud formation templates for AWS resources (adding @searchable sets up ElasticSearch stack!), or Amplify Admin out of a box.
At the same time it adds a lot of complexity and imports many dependencies (like all AWS SDK) if only AppSync integration is needed.
For my needs I just need a replacement of GraphQL client; I would prefer to use a library supported by AWS/AppSync teams but I think Amplify is just way too heavy.
Currently testing swift-graphql and it seems to cover all my needs.

Thank you @pokryfka for your feedback and sharing your experience with us!


Dep Update

I came across this crate and was wondering why pin-project was a dependency. Also it seems openssl-sys is also not needed as it is not referenced anywhere. I also went ahead and updated rust-openssl to the newest version, maybe it would make sense to specify this version as "0.10". I hope im not overlooking something but tests seem to pass.

openssl-sys is needed for the build script to detect the openssl version.

I have added openssl-sys back


[capnproto] Update to 0.9.1

Describe the pull request

What does your PR fix?
Updates capnproto to 0.9.1

Which triplets are supported/not supported? Have you updated the CI baseline?
As before, Yes

Does your PR follow the maintainer guide?

Yes

Hi @chausner
Thanks for your PR.
Have you tested the feature openssl?

Current version told me this error
error LNK2038: mismatch detected for 'RuntimeLibrary': value 'MT_StaticRelease' doesn't match value 'MD_DynamicRelease'

please check it again.

Have you tested the feature openssl?

I have tested that it compiles without issues on x64-windows and that it links to OpenSSL as expected. I have not tested whether the library functions correctly.

Current version told me this error
error LNK2038: mismatch detected for 'RuntimeLibrary': value 'MT_StaticRelease' doesn't match value 'MD_DynamicRelease'

please check it again in your new version.

Hi @ssrlive
Could you please help confirm how did you encounter this problem? If possible, please help provide the steps to reproduce.
Thanks.

Hi @ssrlive
Could you please help confirm how did you encounter this problem? If possible, please help provide the steps to reproduce.

I think @ssrlive refers to an issue with capnproto 0.8.0, correct?

@ssrlive Please help confirm on which version you encountered the issue. Thanks.


Expand md lint tests

Add root of docs folder to markdown validation and fix issues

cc @PowerShell/area-documentation


Consolidate Data Prepper build to a single job

Is your feature request related to a problem? Please describe
The Data Prepper Jenkins promotion job has a step which copies from the Data Prepper staging ECS repository into the promotion DockerHub and ECS repositories. That stage is shown below.
https://github.com/opensearch-project/opensearch-build/blob/2300e055106a23ac98801d209647bdfa590b893f/jenkins/data-prepper/release-data-prepper-all-artifacts.jenkinsfile#L80-L109
After running this Jenkins job, the build process requires that the Jenkins team run the "Copy Docker Image Jenkins job" up to four more times. These jobs will apply either the latest version label (e.g. 2) or the latest label. Please note that not all releases need these. For example running a patch for 1.5.2 after releasing 2 should not update latest.
We should be able to supply all the parameters needed into the one Data Prepper Jenkins job.
Describe the solution you'd like
The Data Prepper promotion Jenkins job could take in two new boolean flags:

"Label latest major" - If set to true, the job will parse the version string for the major version (e.g. 2.0.0 -> 2). Then it will run the "Copy Docker Image Jenkins job" to apply the major version label (e.g. 2).
"Label latest" - If set to true, then run the "Copy Docker Image Jenkins job" to apply the latest label.

Describe alternatives you've considered
Instead of the boolean flags, we could have a comma-delimited list of extra labels to apply. However, these specific flags are more explicit to the users releasing Data Prepper.
Additional context
No response

@asifsmohammed , @gaiksaya

[Triage]
@dlvenable we are ok with the approach you can either do the label major/latest or just code other 3 copies after the initial copy, either way works.


Hello I want a new feature plz to implement tun in any language without opening new terminal windows

When I run invisibleman-tun.exe -port=2223 into cmd, a new terminal is opened. Is it possible to continue working in the background without opening a new terminal?
I already tried to do this in C++ with CreateProcess but the socket is always off
here is my cpp code:
``` STARTUPINFO startupInfo;
PROCESS_INFORMATION processInfo;
ZeroMemory(&startupInfo, sizeof(startupInfo));
startupInfo.cb = sizeof(startupInfo);
startupInfo.dwFlags = STARTF_USESHOWWINDOW;
startupInfo.wShowWindow = SW_HIDE;
ZeroMemory(&processInfo, sizeof(processInfo));
LPSTR cmdLine = const_cast<LPSTR>("-port=2223");
if (!CreateProcess("./tun.exe", cmdLine, NULL, NULL, TRUE, CREATE_NO_WINDOW, NULL, NULL, &startupInfo, &processInfo))
{
    std::cerr << "Error creating process: " << GetLastError() << std::endl;
    return 1;
}
WaitForSingleObject(processInfo.hProcess, INFINITE);
CloseHandle(processInfo.hProcess);
CloseHandle(processInfo.hThread); ```

this code done without error but socket not working

Hi @hosseinkhojany!
Invisible Man TUN needs administrator privilege to work properly. So, to run the service without opening a new terminal, you need to run your program as administrator.
Also, can you show me how you make a socket connection between your app and tun service?

Honestly, my program in Flutter(dart) connects to TUN socket
await Socket.connect("<IP_ADDRESS>", 2223, sourcePort: 2223);
String command =
"-command=enable -device=zVPN -proxy=<IP_ADDRESS>:10801 -address=<IP_ADDRESS> -server=${selectedVpnService
.config?.vlessConfig != null ? selectedVpnService.config!
.vlessConfig!.add! : selectedVpnService.config!.vmessConfig!
.add} -dns=<IP_ADDRESS>";
print(selectedVpnService
.config?.vlessConfig != null ? selectedVpnService.config!
.vlessConfig!.add! : selectedVpnService.config!.vmessConfig!
.add!);
String delimiter = '';
String message = command + delimiter;
List bytes = utf8.encode(message);
socket?.add(bytes);

Also I have customized the socket address to <IP_ADDRESS>

How to find the correct -address parameter ?
-command=enable -device=zVPN -proxy=<IP_ADDRESS>:10801 -address=how to find this-server=example.com -dns=<IP_ADDRESS>

@hosseinkhojany You need to find the default gateway based on the address that you set on your wintun adapter. There are many ways to find the network's default gateway. You can write a code to find it automatically or enter the gateway manually.
In Windows, you can find the default gateway automatically by using iphlpapi.dll. In Invisible Man TUN, I wrote a code to find the default gateway. See these files:

NetworkUtility -> GetDefaultGateway(string address)
IPHelperWrapper.cs


feat: Incidence algebras

Define incidence algebras
From LeanCamCombi
Co-authored-by: Alex J. Best<EMAIL_ADDRESS>

bors merge


MdAutocomplete not filtering when using displayWith

Bug, feature request, or proposal:
Bug
What is the expected behavior?
Items displayed on the panel should be filtered based on the search string
What is the current behavior?
Full list of items displayed
What are the steps to reproduce?
Plucker example https://plnkr.co/edit/l7Vb9lmu9oUcJO3XLqXf?p=preview
Which versions of Angular, Material, OS, browsers are affected?
@angular/cli: 1.0.2
node: 7.10.0
os: darwin x64
@angular/animations: 4.1.0
@angular/common: 4.1.0
@angular/compiler: 4.1.0
@angular/core: 4.1.0
@angular/forms: 4.1.0
@angular/http: 4.1.0
@angular/material: 2.0.0-beta.3
@angular/platform-browser: 4.1.0
@angular/platform-browser-dynamic: 4.1.0
@angular/router: 4.1.0
@angular/cli: 1.0.2
@angular/compiler-cli: 4.1.0

This is a bug with your code, rather than material. Your filter is not testing against option.name

@willshowell this is not my code this is the code on the docs, here at the bottom https://material.angular.io/components/component/autocomplete

Ohh I see! Yeah it should be this
filter(name: string): User[] {
  return this.options.filter(option => new RegExp(`^${name}`, 'gi').test(option.name));
}

@willshowell That works, thanks.

Cool, just opened a PR to fix the docs


Enable RMINC tests

Upstreamed here.

Done, except that a test failure doesn't fail the build.

Also test_mincTFCE.R sporadically fails: test_mincTFCE.R:70: failure: lm randomization worksbetween(thresh[2], 0.5, 5) isn't true.

Resolved by #6 and #7 (except the upstream test_mincTFCE.R issue).

see https://github.com/Mouse-Imaging-Centre/RMINC/#290


[Bug]: CI/CD Extensions Deployment doesnt work

Please confirm that you have:

[X] Searched existing issues to see if your issue is a duplicate. (If you’ve found a duplicate issue, feel free to add additional information in a comment on it.)
[X] Reproduced the issue in the latest CLI version.

In which of these areas are you experiencing a problem?
App, Extension
Expected behavior
npm run shopify app deploy -f should deploy the extensions
Actual behavior
Github CI/CD exit with Error: Process completed with exit code 1.
Verbose output
╭─ error ──────────────────────────────────────────────────────────────────────╮
│                                                                              │
│                                                                              │
│  To investigate the issue, examine this stack trace:                         │
│    at tokenRequestErrorHandler                                               │
│    (@shopify/cli-kit/src/private/node/session/exchange.ts:175)               │
│      return new InvalidRequestError()                                        │
│    at mapError (@shopify/cli-kit/src/public/node/result.ts:142)              │
│      return err(mapper(this.error))                                          │
│    at requestAppToken                                                        │
│    (@shopify/cli-kit/src/private/node/session/exchange.ts:154)               │
│      const value =                                                           │
│      tokenResult.mapError(tokenRequestErrorHandler).valueOrBug()             │
│    at processTicksAndRejections (node:internal/process/task_queues:95)       │
│    at async exchangeCustomPartnerToken                                       │
│    (@shopify/cli-kit/src/private/node/session/exchange.ts:93)                │
│      const newToken = await requestAppToken('partners', token,               │
│      ['https://api.shopify.com/auth/partners.app.cli.access'])               │
│    at async ensureAuthenticatedPartners                                      │
│    (@shopify/cli-kit/src/public/node/session.ts:32)                          │
│      return (await exchangeCustomPartnerToken(envToken)).accessToken         │
│    at async ensureDeployContext                                              │
│    (@shopify/app/src/cli/services/context.ts:289)                            │
│      const token = await ensureAuthenticatedPartners()                       │
│    at async deploy (@shopify/app/src/cli/services/deploy.ts:51)              │
│      let {app, identifiers, partnersApp, partnersOrganizationId, token} =    │
│      await ensureDeployContext(options)                                      │
│    at run (@shopify/app/src/cli/commands/app/deploy.ts:48)                   │
│      await deploy({app, apiKey: flags['api-key'], reset: flags.reset,        │
│      force: flags.force})                                                    │
│                                                                              │
╰──────────────────────────────────────────────────────────────────────────────╯

Reproduction steps

Go to https://shopify.dev/docs/apps/tools/cli/ci-cd#github-actions
Throw the main.yml for github in your project
push to main

Operating System
Mac OS
Shopify CLI version (check your project's package.json if you're not sure)
3.44.1
Shell
zsh
Node version (run node -v if you're not sure)
v20
What language and version are you using in your application?
Ruby 3.0.1

So, i could sort it kinda out. This error mentioned above was caused by my API Key, now i am a step further and get this error
╭─ error ──────────────────────────────────────────────────────────────────────╮
│                                                                              │
│  Failed to prompt:                                                           │
│                                                                              │
│  Match mix-matchify-form (local name) with Mix & Matchify form block (name   │
│  on Shopify Partners, ID                                     │
│                                                                              │
│  This usually happens when running a command non-interactively, for example  │
│   in a CI environment, or when piping input from another process.            │
│                                                                              │
│  To resolve this, specify the option in the command, or run the command in   │
│  an interactive environment such as your local terminal.                     │
│                                                                              │
╰──────────────────────────────────────────────────────────────────────────────╯

So, i could sort it kinda out. This error mentioned above was caused by my API Key, now i am a step further and get this error
╭─ error ──────────────────────────────────────────────────────────────────────╮
│                                                                              │
│  Failed to prompt:                                                           │
│                                                                              │
│  Match mix-matchify-form (local name) with Mix & Matchify form block (name   │
│  on Shopify Partners, ID                                     │
│                                                                              │
│  This usually happens when running a command non-interactively, for example  │
│   in a CI environment, or when piping input from another process.            │
│                                                                              │
│  To resolve this, specify the option in the command, or run the command in   │
│  an interactive environment such as your local terminal.                     │
│                                                                              │
╰──────────────────────────────────────────────────────────────────────────────╯

Sorry for the false report, i sorted it out. It works now, i had a different name in the backend of the App for the Extension.


Strange .Validator behavior

Hi there! I'm wondering if I'm the only one who finds the behavior of this package a bit strange:
package main

import (
  "context"
  "errors"
  "os"
  "strings"

  "github.com/urfave/cli/v3"
)

func main() {
  if err := (&cli.Command{
    Action: func(ctx context.Context, command *cli.Command) error {
      return nil
    },
    Flags: []cli.Flag{
      &cli.StringFlag{
        Name: "api-key",
        Validator: func(s string) error {
          if !strings.HasPrefix(s, "API-") {
            return errors.New("wrong API key format")
          }

          return nil
        },
      },
    },
  }).Run(context.Background(), os.Args); err != nil {
    println(err.Error())

    os.Exit(1)
  }
}

It seems that the default value validation is failing before the actual flag value is parsed. Since the default value will always be invalid, this validation function renders the app unusable, even when the correct flag value is provided:
❯ go run . -h

Incorrect Usage: wrong API key format

...

wrong API key format
exit status 1

❯ go run . --api-key API-123 -h

Incorrect Usage: wrong API key format

...

wrong API key format
exit status 1

Currently, the only option I have is to set a default value that appears to be "valid-like":
&cli.StringFlag{
	Name: "api-key",
+	Value: "API-<set-your-key>",

Could you please confirm if this is intentional or if it might be a bug?

[!NOTE]
github.com/urfave/cli/v3 v3.0.0-alpha9
go version go1.22.5 linux/amd64

It's looks like it the same as #1948

Yes thats the current design. Since the flag is not required it will use the default value and then try to set it. We can look and see whether we can optimize this.

I would be extremely grateful if you could decide that the flag value should be validated only after parsing, once the default value has been replaced by the one provided by the user, but before allowing the developer to use it. Additionally, when the --help flag is used, no validation should be performed. I believe this is critically important


Add ECLYPSIUM_TOKEN environment variable | ENG-5336

To enable eclypsium scanning/registration during deprovision but not expose the
token in git we allow the token to be set during run time via an environment
variable and then pass the token into osie during boot.

add ECLYPSIUM_TOKEN environment variable
supply eclypsium_token kernel param during osie if defined

lgtm, please squash/rebase.


Removing Payment Allocations settings from Allocations settings page

Critical Changes
Changes
Issues Closed
New Metadata
Deleted Metadata

I just reverted to the commit before the payment allocations PR was merged. It looks the same after this PR as it does in managed NPSP today.
A small concern is related to packaging: do we need to add references to the labels that are removed from this page for packaging reasons? @davisagli

**lurch:attach W-030157

Tracking W-030157

Something is not right with this page. The lookup icon on the Default Allocation field is missing again.

Nice catch @force2b !

@njjc I'm not quite sure how the packaging org will behave with regard to the removed labels but I suspect that they will still be included in the package due to being referenced explicitly in CustomLabels.labels. Does that sound right @cdcarter @force2b?

All looks good- merging PR. @force2b @dvenor @davisagli  fyi.


To stop recording

To stop liverecording,  Is it correct to add # before the address of the URL_config.ini file??
ex)  #https://www.pandalive.co.kr/live/play/bara0109, # https://www.pandalive.co.kr/live/play/bara0109
I've tried it, but the recording doesn't stop
I need your help

@ebtemesk5 This feature will be available in version 3.0.8, it will be released soon. :)

@ebtemesk5 This feature will be available in version 3.0.8, it will be released soon. :)

thx
I'm always using DouyinLiveRecorder well


Are there plans to support flutter_web?

as title

I haven't tried it. Does it not work like this? The event bus needs very little to work: Dart Streams and some generic types. See https://github.com/marcojakob/dart-event-bus/blob/master/lib/event_bus.dart

Yes, it doesn't work in flutter_web project, I have n’t even started using it, I just import it at the top of the file, and I got the following error message:
Unable to find modules for some sources, this is usually the result of either a
bad import, a missing dependency in a package (or possibly a dev_dependency
needs to move to a real dependency), or a build failure (if importing a
generated file).

Please check the following imports:

`import 'package:event_bus/event_bus.dart';` from ...
PageWorks.dart at 8:1

Failed after 193ms

How does your pubspec.yaml look like? Here is how to install it: https://pub.dev/packages/event_bus#-installing-tab-

Ok, closing this as we didn't receive any further info.

Sorry, I have forgotten to reply due to the progress of the project in the past few days.
My project uses a lot of other dependent libraries, it seems that the configuration method is the same. I do n’t think there will be any problems in installation, otherwise my "import" statement should be a problem, rather than reporting an error at runtime
However, if other users do not encounter problems, I think it may be related to my computer or environment. I will try again when I have time.
thank you for your reply!


Allow for PNI to be enabled for GKE clusters

What kind of request is this (question/bug/enhancement/feature request):
Enhancement
Result:
When provisioning a GKE cluster or registering an existing GKE cluster, users should be able to select an option that will enable PNI
Other details that may be helpful:
GKE cluster provisioning has an option that allows for specifying "Enable Network Policies". This will cause the cluster to provisioned with the Calico CNI, which supports NetworkPolicy resources. GKE cluster provisioning should be enhanced to toggle this during provisioning if PNI is selected.

UI validation and toggle for Network Policy for Nodes has been done as part of: https://github.com/rancher/rancher/issues/33026
Tested on Rancher master-head version 5b0da3d.
Create a GKEv2 cluster with PNI enabled.
After cluster is available, the following network policies are seen:
> kubectl get networkpolicies -A
NAMESPACE             NAME                   POD-SELECTOR   AGE
cattle-fleet-system   default-allow-all      <none>         31m
cattle-system         np-default-allow-all   <none>         30m
default               hn-nodes               <none>         30m
default               np-default             <none>         30m
kube-node-lease       np-default-allow-all   <none>         30m
kube-public           np-default-allow-all   <none>         30m
kube-system           np-default-allow-all   <none>         30m

Network policies are created in new namespaces in default/new projects:
kubectl get networkpolicies -n testns
NAME         POD-SELECTOR   AGE
hn-nodes     <none>         4s
np-default   <none>         4s

The same behavior is seen on an imported GKE cluster with PNI enabled.


⚠ API - Lens has degraded performance

In 59b2a44, API - Lens ($LENS_API_URL) experienced degraded performance:

HTTP code: 503
Response time: 354 ms

Resolved: API - Lens performance has improved in 489ed7c after 8 minutes.


Various ZFSFilesystem operations are not atomic

If we crash half way through some of the volume operations we can be left in inconsistent state. How do we deal with this?
Currently this only includes:

#679 results in renames, but that'll be fixed in that issue.
Marking things as read-only after changing owner or cloned.

We may be able to get away with just ad-hoc solution for the latter.

We are moving our development planning to JIRA. This is issue is now being tracked at https://clusterhq.atlassian.net/browse/FLOC-924. You are welcome to file additional issues in GitHub if that's easier for you.


Completed reformatting and expansion of docs.

Also fixes some overly verbose names identified during the process.  See revscoring/scorer_models/svc

SVCModel --> SVC
LinearSVCModel --> LinearSVC
etc.

I created aliases to maintain backward compatibility.

See live docs here: http://pythonhosted.org/revscoring/index.html


[CCXDEV-12256] Making new consumer for internal data pipeline

Description
Creating new internal data pipeline consumer in way to pipeline unification
Fixes # CCXDEV-12256
Type of change
Please delete options that are not relevant.

New feature (non-breaking change which adds functionality)
This change requires a documentation update

Testing steps
tested localy
Checklist

[ ] make before_commit passes
[ ] updated documentation wherever necessary
[ ] added or modified tests if necessary
[ ] updated schemas and validators in insights-data-schemas in case of input/output change

Codecov Report
Attention: 25 lines in your changes are missing coverage. Please review.

Comparison is base (3855a8e) 85.82% compared to head (4aaf818) 83.14%.

Files
Patch %
Lines

ccx_messaging/consumers/internal_kafka_consumer.py
0.00%
25 Missing :warning:

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #150      +/-   ##
==========================================
- Coverage   85.82%   83.14%   -2.68%
==========================================
  Files          25       26       +1
  Lines         776      801      +25
==========================================
  Hits          666      666
- Misses        110      135      +25

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Fix build problem for ceres, upgrade to support ceres 2.0.0

Good improve to support new ceres version, but this pull wont work on old ceres version since summary.IsSolutionUsable() is not usable for ceres version below 1.12

I have fixed the problem, please pull and check it. Thanks for the advice.


Dispatch

Added multiple dispatch into the mix.  Also reorganized the file structure.  The idea being that we define assets as classes in the asset folder, then for each asset define the relevant calculations in the calculators folder where the file name is equal to the asset name.
Also moved the time to utils.
Changed the Bond classes a little by making a base bond class.  Since all bond calculations are simply based on the cash flows of the bond, child bond classes just define how those cash flows are created.
Try it out:
from pennies.assets.bonds import Bond, ZeroCouponBond
from pennies import present_value
b = Bond(100, '2015-01-01', '2015-12-31', 'Q', 0.05)
present_value(b, 0.01)
z = ZeroCouponBond(100, '2015-01-01', '2015-12-31', 0.05)
present_value(z, 0.01)

Notice that pennies.calculators.bonds.present_value is only defined once for Bond, but also happily handles ZeroCouponBond.
The calculation may be a little wrong, if I remember correctly the discount_factor should be to the power of the fractional point in time.

Also works for dataframes:
import pandas as pd
from pennies import present_value
from pennies.assets.bonds import Bond, ZeroCouponBond
b = Bond(100, '2015-01-01', '2015-12-31', 'Q', 0.05)
z = ZeroCouponBond(100, '2015-01-01', '2015-12-31', 0.05)
bonds = pd.DataFrame([b, z])
present_value(bonds, 0.01)

Changes Unknown when pulling 9edd086a2e8f3e0b050aa092f6047f9ded4209d5 on dispatch into ** on master**.

PR also includes cash flows and fixes to travis configuration.

Changes Unknown when pulling 16d9ba1b3ab29abe60c202e01ea4bc152b2902f3 on dispatch into ** on master**.


chore: Remove unused imports & variables

I came across a lot of stray unused imports & unused variables. This PR removes them.

Closing in favour of #87


Remove workbench.action.terminal.focus in favor of workbench.action.focusPanel

See #7540

@Tyriar I pushed https://github.com/Microsoft/vscode/commit/bf1db232cacf17e909a355456443ca68ab6180dd


nwg-launchers: 0.4.2 -> 0.4.3

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://repology.org/metapackage/nwg-launchers/versions.
meta.description for nwg-launchers is: "GTK-based launchers: application grid, button bar, dmenu for sway and other window managers"
meta.homepage for nwg-launchers is: "https://github.com/nwg-piotr/nwg-launchers"
meta.changelog for nwg-launchers is: ""
Updates performed

Version update

To inspect upstream changes

Release on GitHub

Compare changes on GitHub

Impact

Checks done (click to expand)

built on NixOS

The tests defined in passthru.tests, if any, passed

0 of 0 passed binary check by having a zero exit code.

0 of 0 passed binary check by having the new version present in output.

found 0.4.3 with grep in /nix/store/avflmj8gv3qnsz6hsiily0lny5vr9nd9-nwg-launchers-0.4.3

directory tree listing: https://gist.github.com/224ab3894242477e2b61162e886e161f

du listing: https://gist.github.com/8cb2b072229ce1d0335cefa2bec8436e

Rebuild report (if merged into master) (click to expand)

3 total rebuild path(s)

1 package rebuild(s)

1 x86_64-linux rebuild(s)
1 i686-linux rebuild(s)
0 x86_64-darwin rebuild(s)
1 aarch64-linux rebuild(s)

First fifty rebuilds by attrpath
nwg-launchers

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/avflmj8gv3qnsz6hsiily0lny5vr9nd9-nwg-launchers-0.4.3 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A nwg-launchers https://github.com/r-ryantm/nixpkgs/archive/1c89c00ffcfc2724d76cbb09ad8868a4645818a2.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/avflmj8gv3qnsz6hsiily0lny5vr9nd9-nwg-launchers-0.4.3
ls -la /nix/store/avflmj8gv3qnsz6hsiily0lny5vr9nd9-nwg-launchers-0.4.3/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

1 package built:

nwg-launchers

Maintainer pings
cc @bbigras for testing.

Result of nixpkgs-review pr 110139 run on x86_64-linux 1

1 package built:

nwg-launchers

Result of nixpkgs-review pr 110139 run on x86_64-linux 1

1 package built:

nwg-launchers

Builds and runs fine.

Builds and runs fine.


[staging-23.05] libwebp: 1.3.1 -> 1.3.2

Description of changes
fixes CVE-2023-4863, which allows us to drop the patch
(cherry picked from commit 6cb2474327e3941c31f170de34d8a97c9871ea4f)
backport of #255102
Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.11 Release Notes (or backporting 23.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

@ofborg build libwebp

Let's see if this build attempt fares any better...
@ofborg build libwebp


[AUTHZ] Refactor Authz reflection with kyuubi-util's DynMethods

Why are the changes needed?

add reflection utils in kyuubi-util-scala, using kyuubi-util's DynMethods
continue to provided simplified reflection calling in scala

How was this patch tested?

[ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

[ ] Add screenshots for manual tests if appropriate

[x] Run test locally before make a pull request

Codecov Report

Merging #4873 (916b7af) into master (13f6aff) will not change coverage.
The diff coverage is 0.00%.

@@          Coverage Diff           @@
##           master   #4873   +/-   ##
======================================
  Coverage    0.00%   0.00%
======================================
  Files         556     556
  Lines       30697   30677   -20
  Branches     3998    3994    -4
======================================
+ Misses      30697   30677   -20

Impacted Files
Coverage Δ

.../kyuubi/plugin/spark/authz/PrivilegesBuilder.scala
0.00% <ø> (ø)

...uubi/plugin/spark/authz/ranger/AccessRequest.scala
0.00% <ø> (ø)

...k/authz/ranger/RuleReplaceShowObjectCommands.scala
0.00% <0.00%> (ø)

...e/kyuubi/plugin/spark/authz/serde/Descriptor.scala
0.00% <ø> (ø)

...i/plugin/spark/authz/serde/catalogExtractors.scala
0.00% <ø> (ø)

.../plugin/spark/authz/serde/databaseExtractors.scala
0.00% <ø> (ø)

...ubi/plugin/spark/authz/serde/tableExtractors.scala
0.00% <ø> (ø)

...he/kyuubi/plugin/spark/authz/util/AuthZUtils.scala
0.00% <ø> (ø)

...plugin/spark/authz/util/RangerConfigProvider.scala
0.00% <0.00%> (ø)

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

The lineage plugin can also benefit from this in followups, since it has similar utils and method as Authz's.

Pending this PR. Local builds failed unexpectedly.
Running build/mvn clean package -pl :kyuubi-spark-authz_2.12 -Dranger.version=2.1.0
object ReflectUtils is not a member of package org.apache.kyuubi.util.reflect
...
package java.lang.invoke is not a value
...
not found: value getFieldVal
...
not found: value invokeAs

please add -am in the command

OK. Built and tested successfully by running with  -am.


shep logs has nasty errors on never run functions

If a function has never been run then the user will see Error: No log groups found for specified function this could probably be prettier.
Also as @reconbot pointed out, if a specific version of a function has never been ran then the user gets a cryptic
InvalidParameterException: 1 validation error detected: Value '[]' at 'logStreamNames' failed to satisfy constraint: Member must have length greater than or equal to 1
    at Request.extractError (/Users/wizard/src/gradius/node_modules/aws-sdk/lib/protocol/json.js:48:27)
    at Request.callListeners (/Users/wizard/src/gradius/node_modules/aws-sdk/lib/sequential_executor.js:105:20)
    at Request.emit (/Users/wizard/src/gradius/node_modules/aws-sdk/lib/sequential_executor.js:77:10)
    at Request.emit (/Users/wizard/src/gradius/node_modules/aws-sdk/lib/request.js:678:14)

I think we should catch these and display something along the lines of "Logs for the version of the function you requested don't exist yet"

This was resolved somewhere in time


Out of date

The latest version of TOML is now 4.0, not 3.1 as the README states.

To adding to this - the README contains the following line:
let r = parseTomlDoc "" txt
Which leads to the following error:
    Couldn't match type ‘[Char]’ with ‘Data.Text.Internal.Text’
    Expected type: Data.Text.Internal.Text
      Actual type: String
    In the second argument of ‘parseTomlDoc’, namely ‘txt’
    In the expression: parseTomlDoc "" txt

@voidzero It seems your REPL is not picking up the .ghci file. I've stackified the instructions in the README, maybe you can have another look.

@bsima I'll pull in some PRs that will fix this, in the meanwhile I'll close this issue.


Admin configuration for posts in the past

Allow for admin configuration for "posts in the past"
-enable/disable
-allow "posts in the past" within one day/week/month
If new user joins, find a post they like, boost an old post, other new users can discover and also interact
As timeline for new instance become busy, this can then be disabled

I don't quite understand what the feature you want here is - you want the ability to turn off the restrictor we have in place that stops old posts from people you follow appearing at the top of the timeline, or something else?

Sorry read your 0.7 release blog post about "Posts From The Past" and should've referenced it
Here's the section this issue is referring to:
"We did make a change to our timeline builder so it doesn't add things to the Home timeline if they were received more than a day after they were published"
Does Takahe currently have admin setting to configure this?

reference

No, and I don't quite see why you'd want to, as it's very confusing for users to see a normal post from 30 days ago at the top of their timeline. Note this doesn't affect boosts of old posts - those appear just fine - it's the actual plain posts themselves.

Got it. Thought it meant the home timeline would not receive anything old. Closing.


MIUI12 opacity problem while casting with screen OFF/ON

[x] I have read the FAQ.
[x] I have searched in existing issues.

Environment

OS: Macos 10.15
scrcpy version: new
installation method: brew
device model: k20 pp MIUI12
Android version: 10

Describe the bug
I have a similar problem, but the difference is that I can't get around it with #1301 and #1306.
Entering the desktop with "Turn screen off" enabled doesn't show any icons. The difference is that all the icons are completely gone, and clicking on any area or sliding the desktop will have no effect. Closing the screen and opening it again won't fix it either. But other functions (notification bar, etc.) are still working.
Disable "Turn screen off" and it will be back to normal.

I have a similar problem, but the difference is that I can't get around it with #1301 and #1306.

Yes, that's the exact same problem, and there's nothing scrcpy can do about this (it's the device behavior): https://github.com/Genymobile/scrcpy/issues/1301#issuecomment-616318495


add boxch

close

Please read https://github.com/solana-labs/wallet-adapter/blob/master/WALLET.md
We are not accepting new adapters for this type of wallet. Implementing the Wallet Standard will be your best path forward.


servicebus release

https://github.com/Azure/sdk-release-request/issues/3733

API change check
APIView has identified API level changes in this PR and created following API reviews.
azure-arm-servicebus


feat: rename methods

Related Issues

hotfix (comments from silvano)

Proposed Changes?

merge upload files and upload folder
add nesting folder
renamed dc_files to files and let the typing take care of documenting

How did you test it?

unit tests

Notes for the reviewer
Checklist

[ ] I have updated the referenced issue with new insights and changes
[ ] If this is a code change, I have added unit tests
[ ] I've used the conventional commit specification for my PR title
[ ] I updated the docstrings
[ ] If this is a code change, I added meaningful logs and prepared Datadog visualizations and alerts

cc @silvanocerza


⚠ Bofrost.lu has degraded performance

In 080cb4a, Bofrost.lu (https://www.bofrost.lu) experienced degraded performance:

HTTP code: 200
Response time: 1047 ms

Resolved: Bofrost.lu performance has improved in 5a0262e after 6 minutes.


Payment

What's included?

Integrate with Stripe.
Set up env files.

Stripe:

Dashboard: https://dashboard.stripe.com/
Test payment link: https://buy.stripe.com/test_9AQ4hLagT2t71ywfYY
Test credit cards: https://stripe.com/docs/testing (e.g. 4242 4242 4242 4242)

Please remove "Stripe" from the payment button. Thank you!

Not sure why it says "PayPal" here?


Integration with ScalaJSON

Finally after a long amount of time, ScalaJSON has been released on the scala-platform, follow on from SLIP-28 (its currently as a milestone release, see https://github.com/mdedetrich/scalajson). A proper release is expected to happen within a few days, as long as no major changes are expected. Its also a response to

You generally shouldn't need or want to work with JSON ASTs directly. This is part of the reason I couldn't personally care less about SLIP-28: I think JSON is a horrible serialization format, and I want to help people not have to think about JSON. If you absolutely have to work with JSON values directly, you at least shouldn't have to worry about keeping track of traversal history or manually handling modification of deeply nested structures (i.e. you should be using a relatively nice API like ACursor or the facilities supported by Monocle in circe-optics).

What this means concretely: If something like SLIP-28 ever actually lands in the standard library, I'll probably adopt it (assuming it doesn't violate other design principles here—which is a huge assumption), but I hope most users won't even notice that change.

In the design readme (https://github.com/circe/circe/blob/master/DESIGN.md)
This issue is to describe the likelyhood (and how) to integrate the AST into Circe (or if there is just going to be a circe-scalajson module which means no integration and just a conversion between the 2 modules). Ideally a proper integration would be great, however its understood if this is going to cause problems as the ScalaJSON AST is fairly different in design to the circe JSON AST (although most circe users shouldn't be working directly on the Json AST). ScalaJSON AST is implemented as a proper Scala ADT where as the Json type in Circe has no ADT exposed
Here are the the different ways I envisage to integrate the ScalaJSON AST into Circe

Just integrate scalajson.ast.unsafe.JValue in a low level way, and expose functions that allow you to retrieve this JValue (you can always get a scalajson.ast.JValue from a scalajson.unsafe.JValue using ScalaJSON)
Integrate both scalajson.ast.unsafe.JValue and scalajson.ast.JValue into Circe, exposing scalajson.ast.JValue when it makes sense to do so. Note that doing this is a slight contradiction of Circe's original design (i.e. users should not have to deal with the Json AST type directly) however it can provide extra benefit for users. Note that scalajson.ast.JValue doesn't implement the whole possible JSON spectrum (see  https://github.com/mdedetrich/scalajson#goals for more info)
Don't integrate at all, and instead have a module as part of Circe which converts from a Circe Json value to a ScalaJSON JValue and vice versa. Not ideal in terms of performance/speed since we are not using the AST directly (and also end users needing another module) however the easiest to work with from Circe's POV.
Least ideal situation, ScalaJSON provides a module to integrate with Circe which ScalaJSON will manage. This is least ideal because Circe contributors have a much better idea of how their release cycles work.

If there are any questions let me know. I am also happy to help with the integration, and there is room to do some extra changes providing they aren't completely major (@travisbrown the change you requested in regards to number handling with overflow due to BigDecimal has already been implemented)
Also note there is an issue in jawn for integration the JSON into their parser, you can see it here https://github.com/non/jawn/issues/89

Thanks for opening this, @mdedetrich, and congrats on the release being almost ready!
Low-level adoption of the ScalaJSON AST
I still think it's unlikely that circe will switch to using the ScalaJSON AST at a low level—I don't intend to criticize the ScalaJSON design decisions, but they're fairly different from circe's, and I'm not convinced the performance benefits over a conversion module would be worth trying to make the two fit together. Specifically:

I'm still not sold on the safe / unsafe split. My goal for the AST in circe is to have something that can usefully represent any valid JSON value, supports a few popular distinctions that the JSON spec doesn't make (negative zero, order preservation for object fields), and will never, ever throw exceptions, while still performing sufficiently well for most reasonable use cases. I think the current AST is more or less successful at doing this (and is still improving), and I haven't seen any concrete evidence that the performance gains of an unsafe representation would offset the extra complexity.
I just don't think having exceptions thrown on simple cases like JNumber("1.0").to[Int] or JNumber("1e2147483648").to[BigDecimal] (the safe versions of both of these throw) is a good user experience—I don't really want to expose types that do this in the core circe API, and internally I'd have to do a lot of wrapping just to maintain the safe behavior I already have with io.circe.Json.

I'd consider revising this decision if there turns out to be overwhelming demand for optimally fast ScalaJSON integration, or if someone produces evidence that the performance supported by the unsafe representation is just vastly better than io.circe.Json for use cases people care about.
Conversions
I'm happy to add a ScalaJSON conversion module in circe right now, and I think we'd consider including ScalaJSON conversions in circe-core if there proves to be a lot of demand from circe adopters over the next few months.
Future integration
I'm also working on a major change to circe that would make this whole question a little less relevant. The key idea is that circe's Decoder (and Encoder) will no longer use any particular AST directly: you'll write your Decoder in an AST-agnostic way (using an API that's like 95% source compatible with the current circe Decoder) and then use an AST-specific interpreter to decode your actual JSON representation (or your BSON or YAML representation, etc.).
This change (which I now think has maybe an 80% chance of actually happening) goes a step further in emphasizing encoders and decoders as the key components in circe, and de-emphasizing the AST and cursors. Once it's done we can provide a circe encoder and decoder interpreters for ScalaJSON and users who want ScalaJSON AST values will never have to instantiate io.circe.Json values at all.

I'm still not sold on the safe / unsafe split. My goal for the AST in circe is to have something that can usefully represent any valid JSON value, supports a few popular distinctions that the JSON spec doesn't make (negative zero, order preservation for object fields), and will never, ever throw exceptions, while still performing sufficiently well for most reasonable use cases. I think the current AST is more or less successful at doing this (and is still improving), and I haven't seen any concrete evidence that the performance gains of an unsafe representation would offset the extra complexity.

The safe/unsafe split is due to the unfortunate redefining of JSON by mr Crockford in his specification. The differences are largely due to ordering/duplicate keys in JObject (in Javascript which is where JSON came from, you can't have duplicate keys/ordering in a Map, this was just added by Crockford when making the spec with weasel words such as "should" for justifying them). The reason behind the split is that there is a "sane" version of JSON which is what everyone should be using (this is the ScalaJSON standard AST), if they don't use this "sane" version then they are probably going to break something, somewhere for someone. In reality there are reasons for these corner cases (i.e. ordering keys when it comes to serialization), hence the reason for unsafe.JValue.

I just don't think having exceptions thrown on simple cases like JNumber("1.0").to[Int] or JNumber("1e2147483648").to[BigDecimal] (the "safe" versions of both of these throw) is a good user experience—I don't really want to expose types that do this in the core circe API, and internally I'd have to do a lot of wrapping just to maintain the safe behavior I already have with io.circe.Json.

I am actually not a fan on these and want to remove them, comment on https://github.com/mdedetrich/scalajson/issues/17 if you agree

I'm happy to add a ScalaJSON conversion module in circe right now, and I think we'd consider including ScalaJSON conversions in circe-core if there proves to be a lot of demand from circe adopters over the next few months.

Awesome thanks, I was expecting this response anyways ;)

I'm also working on a major change to circe that would make this whole question a little less relevant. The key idea is that circe's Decoder (and Encoder) will no longer use any particular AST directly: you'll write your Decoder in an AST-agnostic way (using an API that's like 95% source compatible with the current circe Decoder) and then use an AST-specific interpreter to decode your actual JSON representation (or your BSON or YAML representation, etc.).

I actually like this change and I think the decoupling makes sense

Following up here on a conversation that's partly happening in this thread as well.
I think the duplicate keys issue is kind of a distraction and a special case—I'm not sure anybody thinks it's a good idea, and it's only allowed by ECMA 404 (not the RFCs). Even circe doesn't follow ECMA 404 here, in that currently only the last value for a key is preserved (although I could imagine that changing to be more literally in line with ECMA 404 if enough users asked for it).
Apart from that one detail, I don't really see how ECMA 404 is more corner-casey than the ECMAScript behavior. It's a remarkably simple spec (like a tiny fraction of a footnote compared to e.g. IEEE 754), and it's not that hard to provide pretty complete support for it (up to the limits of memory, etc.). This is what circe is aiming for.

This is why there are two AST's in ScalaJSON, the same [safe] one is what @alexandru is describing and the unsafe one is the one with all of these corner cases.

As a side note, I think it would be helpful if this distinction were clearer in the ScalaJSON docs and maybe even the names. When I hear "unsafe" I think "this is going to give up some correctness or purity or whatever for the sake of performance", not "this provides a more literal interpretation of ECMA 404".
One other thing: in the other thread @alexandru argues that circe's behavior is cases like this is actually less safe:
scala> io.circe.jawn.parse("9007199254740993")
res0: Either[io.circe.ParsingFailure,io.circe.Json] = Right(9007199254740993)

…because JavaScript's Json.parse would truncate it to<PHONE_NUMBER>740992 and if your Scala JSON library does the right thing you might forget that JavaScript clients won't, so you might as well use Double to represent numbers even on the JVM side and leave it at that.
I don't think this is an unreasonable argument, but it's definitely not at all the approach circe is taking. Nothing in circe is specifically designed to take the behavior of JavaScript into account (although we do our best to document places where the boundary might be confusing).

@travisbrown See https://github.com/mdedetrich/scalajson/issues/16#issuecomment-312086085 for a more detailed answer

@travisbrown A lot of your last issues regarding scalaJSON (total function for JNumber) and a lot of other fixes have been addressed in the new 1.0.0-M4 release. Not sure if this will change your final opinion, just notifying you about the changes!

@mdedetrich Thanks for the update! Will take a look asap.

The main thing to note is that there is a seperate version of scalaJSON specific for scala 2.10.x which simulates a case class using a normal class, where as for future versions of scala we use a case class for JNumber. In all cases the String constructor is private, and there is a JNumber.fromString method available on all platforms that returns a Option[JNumber] (on Scala 2.11 and 2.12, the standard JNumber.apply(value: String) method returns an Option[JNumber])

Closing in cleanup run, If anyone cares about this, please comment and I'll reopen.


Styling

It looks like a really bad bootstrap page

Its a slightly less-shitty looking bootstrap page now.


EventHub: Added New API to 2018-01-01-preview

Latest improvements:
MSFT employees can try out our new experience at OpenAPI Hub  - one location for using our validation tools and finding your workflow.

Contribution checklist:

[X] I have reviewed the documentation for the workflow.
[X] Validation tools were run on swagger spec(s) and have all been fixed in this PR.
[X] The OpenAPI Hub was used for checking validation status and next steps.

ARM API Review Checklist

[ ] Service team MUST add the "WaitForARMFeedback" label if the management plane API changes fall into one of the below categories.
adding/removing APIs.
adding/removing properties.
adding/removing API-version.
adding a new service in Azure.

Failure to comply may result in delays for manifest application. Note this does not apply to data plane APIs.

[X] If you are blocked on ARM review and want to get the PR merged urgently, please get the ARM oncall for reviews (RP Manifest Approvers team under Azure Resource Manager service) from IcM and reach out to them.
Please follow the link to find more details on API review process.

/azp run automation - sdk

Can one of the admins verify this patch?

/azp run automation - sdk

/azp run automation - sdk

/azp run automation - sdk

/azp run automation - sdk

/azp run automation - sdk

@ryansbenson updated with changes and comments. One quick questions, is it okay to remove the changesrequested label to show up as review requested.

Signing off from ARM side as Ryan's comments have been addressed.

/azp run automation - sdk

Hi @v-Ajnava, is this PR still valid, as it is idle for a long time, looks this version(2018-01-01-preview) is already merged.

closing the PR as its not need know.


style(app): App use textTransform from typography

Overview
Use  textTransform, textDecoration prop from TYPOGRAPHY.
Update the components that are updated within 12 months.

Changelog

change to TYPOGRAPHY.textTransformCapitalize  from  TEXT_TRANSFORM_CAPITALIZE
change to TYPOGRAPHY.textTransformUppercase from TEXT_TRANSFORM_UPPERCASE
change to TYPOGRAPHY.textDecorationUnderline from TEXT_DECORATION_UNDERLINE

Review requests
The updated props are the same as the previous
TEXT_TRANSFORM_CAPITALIZE --> TYPOGRAPHY.textTransformCapitalize
TEXT_TRANSFORM_UPPERCASE --> TYPOGRAPHY.textTransformUppercase
TEXT_DECORATION_UNDERLINE --> TYPOGRAPHY.textDecorationUnderline

Risk assessment
low

Changed the target to edge.
The timing will after merging release_6.0.0. to avoid any unexpected issues


Filter out duplicate POIs (one from landuse AOI, one from building & etc)

Cloning this to a new issue to track down further edge cases...
I'm still seeing duplicate POIs, I think from different layers where one has an area and the other doesn't, but they have the same OSM ids. The proposed implementation is only within a single layer, do we need it to span across layers, or pack less stuff into multiple layers? (Seems like less double packing is the way to go.)
#16/37.7653/-122.4413

From Matt's comment on that original issue:
I think the tile with the museum must have just not updated yet - what I see is below:

Regarding the duplicate schools; looks like they're coming from an amenity=school polygon which we put into the landuse layer and gets a label from there, and a building=school polygon which we put into the buildings layer and gets a label from there.
We have two options here:

Noting that this is very similar to the situation around the SFU campus, where buildings are appearing like POIs because their kind ends up being the same as the landuse polygon containing them, you had previously suggested suppressing the kind when it's the same as the landuse_kind and turn it into kind=building instead. We could do this as a separate post-process filter. (Alternatively, we could suppress the landuse polygon, but I think we'd prefer to keep the "larger" object, which is usually the landuse not the building polygon.)
Suppress duplicate POIs (across layers) by extending the post-processing step. This will also run into more corner cases, as we'd effectively need to define an ordering between layers so that we could choose the more important duplicate to keep.

In either case, I think it's worth tracking the follow-up as a separate issue so as not to hold up getting this, or anything merged after this, into prod.

Another example, the Lombard Motor Inn in SF as building label and as POI.
#19/37.80054/-122.42523

Suppress duplicate POIs (across layers) by extending the post-processing step. This will also run into more corner cases, as we'd effectively need to define an ordering between layers so that we could choose the more important duplicate to keep.

This approach smells better to me. There are going to be corner cases either way, and this option seems to allow us more determinism.

Another example, the Lombard Motor Inn in SF as building label and as POI.

It is mapped as two separate objects, so we'll have to use a post-process de-dup filter since it's duplicated in the original data.
http://www.openstreetmap.org/#map=19/37.80074/-122.42582

Mapped as a building with name: http://www.openstreetmap.org/way/243512088
Mapped as a motel: http://www.openstreetmap.org/node/992609593

Arguably, these should be merged. But there's probably many, may other examples so it might be better just to leave it as-is so that we can use it as an example.

Tracking the Lombard Motor Inn as an example of the kind of thing we want to filter out in #266.

This may be a duplicate of #266. Reevaluate once that is in dev.

Pretty sure this was a dupe.


Responsiveness issue on showcase page

Actual Behaviour
On the showcase page, Nominate a project button breaks down when the device width is less than 375px.
Expected Behaviour
The button should be responsive.
Steps to reproduce it
By simply adding a @media query in the CSS file will work
.
Screenshots of the issue

Would you like to work on the issue?
Yes, I would like to work on this.

@sraghuvanshi Go ahead with the changes. Thanks!


Add Kdenlive

Closes #919

Nice! 👍
Try to run .\bin\checkver.ps1 <app> -f before submitting a PR, to check if everything works 😄

@r15ch13 ok.


Issue 2179 - Get serialised records returned from query in QueryWebSocketClient

Make sure you have checked all steps below.
Issue

[x] My PR addresses the following issues and references them in the PR title. For example, "Issue 1234 - My Sleeper
PR"

Resolves https://github.com/gchq/sleeper/issues/2179

Tests

[x] My PR adds the following tests OR does not need testing for this extremely good reason:

Updated WebSocketResultsOutputIT
Updated QueryWebSocketClientTest

Documentation

[x] In case of new functionality, my PR adds documentation that describes how to use it, or I have linked to a
separate issue for that below.
[x] If I have added, removed, or updated any external dependencies used in the project, I have updated the
NOTICES file to reflect this.

This doesn't do what the issue says any more, please can that be updated?
Actually, it looks like this just fixes this bug now?

WebSocket query script fails to print records to the console #2213

Thanks, I've updated the issue title and description. It still adds a method to retrieve the results for an executed query, which will be used by the system test driver.

Tested with AWS and this resolves the linked bug