x/mobile: performance degradation by 2 orders of magnitude when running inside a Swift app

What version of Go are you using (go version)?

go version go1.20.5 darwin/amd64

Does this issue reproduce with the latest release?
Yes.
What operating system and processor architecture are you using (go env)?
go env Output
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/user/Library/Caches/go-build"
GOENV="/Users/user/Library/Application Support/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/user/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/user/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GOVCS=""
GOVERSION="go1.20.5"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/dev/null"
GOWORK=""
CGO_CFLAGS="-O2 -g"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-O2 -g"
CGO_FFLAGS="-O2 -g"
CGO_LDFLAGS="-O2 -g"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -arch x86_64 -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/0s/b1y_wjhc8xl6nc008k8dhz1h0000gq/T/go-build2158411908=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?
I'm trying to integrate some Go code with a Swift app. I noticed that code runs much slower when integrated on the Swift app than standalone on a pure Go environment. I did some experiments and came up with an interesting use case. Take this function:
func Something() {
	start := time.Now()
	var numbers []int
	for i := 0; i < 1000000; i++ {
		numbers = append(numbers, i)
	}
	end := time.Now()
	fmt.Printf("Duration: %vms\n", end.UnixMilli()-start.UnixMilli())
}

I wrote a simple benchmark for this:
func BenchmarkSomething(b *testing.B) {
	for i := 0; i < b.N; i++ {
		Something()
	}
}

When running this from Visual Studio or directly from the command line using go test -bench=. , this function takes about 7ms to run. However, after building a framework for integrating with the Swift app (using gomobile bind -target macos -ldflags="-s -w" -o hello.xcframework), it takes, at best, ~700ms to run.
If I remove the code that creates and appends to the slice, the execution time is similar on both environments, so I suspect this is something related with memory management.
I wondered if the gomobile command was building without optimizations or with any kind of debug code, but as far as I can understand (new Go user here), there's nothing like -O here (Go builds are always as optimized as they can be?). So I'm a bit stuck on why this is so much slower when running from Swift. I'm ruling out any bridging code since the slowness happens entirely inside Go code, not pushing data back and forth between both worlds.
What did you expect to see?
A similar execution time in both cases.
What did you see instead?
Running from Swift is ~100x slower.

Interesting: if I run the app directly from Finder, or if I launch it from Xcode after turning off the "Debug Executable" option, it runs at the expected speed. So this seems some kind of instrumentation Xcode is adding to the runtime (even if I build in Release mode) that slows down things considerably on the Go side (I don't see this kind of performance degradation on Swift code).

@arroz Indeed, it's possible that what you're seeing here is an effect of XCode's instrumented environment.


docs: Update README.md with new Discord Channel information

@mason-at-pieces just wanted to get an update on this PR - still in draft but just wanted to check in

@jordan-pieces Yep, should be good to go unless you had any opinions on it


Add a script to calculate some simple stats about web-features

Example output:
{
  "features": 742,
  "compatKeys": 9478,
  "compatCoverage": 0.6361073825503356,
  "compatKeysPerFeatureMean": 12.773584905660377,
  "compatKeysPerFeatureMedian": 4,
  "compatKeysPerFeatureMode": 1
}

Thanks, James!
(I'm going to merge this without an owner review—under the owner delegate rule—since this is support script that doesn't actually affect any feature data.)


app/common/ModalDialog

Something in ModalDialog prevents it from rendering a
dijit/form/Select

Test:
to app/common/SignIn.js add
    ,"dijit/form/Select"

To app/common/templates/SiginIn.html insert in the form element:

      Alabama
      Alaska
      Arizona
      Arkansas
      California
      New
  Mexico

This is happening due to one peculiar design flow in one of the CSS files. Unfortunately, Geoportal team doesn't have a control over that file, so we cannot fix it permanently.
We can provide some guidance how to mitigate the issue, though. It is all based upon manually setting a 'zIndex' value of a particular elements to the value higher than 1050. Which elements it depends on the Dojo control used on the Dialog box.
Here is an example how it's been done on the TemporalFilterCallendar dialog:
https://github.com/Esri/geoportal-server-catalog/pull/245/files#diff-c1f7d887ad20bf176f06bfff4b6ffba9
This dialog has two Calendar widgets. Each calendar widget exposes DropDown-like behavior and needs its zIndex to be adjusted to 1100. Moreover, each such DropDown has another DropDown (months) and that needs to have its zIndex adjusted as well to 1101.


[3.13 beta 1] tileSprite.setFrame() doesn't work properly

In v3.13  beta, calling setFrame() on a Tile Sprite doesn't have the intended effect, the frame doesn't change but the size is distorted. Reproduced in this example: adding images[0].setFrame('bunny'); at the end of create() should change the frame of the first scrolling column, but it doesn't. Both frames (the old and the new) belong to the same atlas.
Interestingly, adding images[0].setFrame('bunny'); in update() instead, so that it's called every frame, does work! I guess this suggests that at some point the change of frame is overridden by internal code.
I reproduce it here in v13.3 beta1 but I initially encountered the issue in my project which uses v3.12.

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Thank you for submitting this issue. We have fixed this and the fix has been pushed to the master branch. It will be part of the next release. If you get time to build and test it for yourself we would appreciate that.


add generic metadata traversal

Provides a generic resolver interface that can traverse arbitrary structs, messages, maps, and lists, and return serialized values.
/assign @jplevyak
/assign @mandarjog

Looking for initial feedback on the approach. What is missing is some form of testing and creating a more usable C++ API inside the sandbox.

@PiotrSikora from my limited knowledge, C-style arrays are not friendly to Go without doing something like cgo (which is a pain) since we don't have access to raw pointers. That's probably true for most managed languages.

Too many conflicts, closing and restarting the PR.


[Build Analysis] Add area tags information to known issues report

As part of the known issue report it would be useful to display the area tags (runtime labels) to facilitate the classification of the issues.
This is the report https://helix.dot.net/BuildAnalysis/KnownIssues , also the information doesn't need to be necessarily on the report, it can be just a kusto query, but right now we don't have that information stored anywhere, so the TODO list for this issues are:

[ ] Store tag labels of issues classified as known issues, this value is constantly updated so we need to have this stored value also updated
[ ] Provide the report or kusto query for this information

This need was mentioned in a mail thread, so I will add the sentiment feedback tag.

Release Note Category

[x] Feature changes/additions
[ ] Bug fixes
[ ] Internal Infrastructure Improvements

Release Note Description

Added runtime area tags information to known issues report in landing page

PR: https://dev.azure.com/dnceng/internal/_git/dotnet-helix-service/pullrequest/35848

rolled out and validated


Don’t set the notification # to 0 if loading fails

If you have a certain amount of notifications and then open the app without connection, it will reset the counter to 0.

Bug Report Dump (Auto-generated)

Version 1.21.0<PHONE_NUMBER>)
Device: iPhone X (iOS 11.4)
TestFlight: true

you mean to say caching to do here??

@BasThomas  @TheCodeTalker The classes in charge of this issue are being updated in #1870 so prob re-check this issue after that PR is merged.

Any update on this??

Yep, still up for grabs.
Re caching: no, that should not be needed.

My understanding for this issue if open the app without a connection it should be zero only right because not able fetch the count. @BasThomas

When you open the app without a connection, it shouldn't update the badge count (also not to 0).

ok make sense.  i will make a PR

is this one still a problem/ pr worth if no one is working at it?

Yep, go for it @romankl!


No active campaigns to mine drops for. Waiting for an active campaign...

No matter what I try it doesnt seem to start. I see the message "No active campaigns to mine drops for. Waiting for an active campaign..." even when I have no priority set...

Do you have games linked? Can you add a screenshot of the inventory with all options checked?

@penisgesicht231 Have you added any games to the priority list first? It won't do anything until you do.

TBF I do have: No available channels to watch. Waiting for an ONLINE channel...

I set it to prio only and World of Tanks. My fork is farming on their official channel.

Do you have games linked? Can you add a screenshot of the inventory with all options checked?

I didnt link any accounts yet is that required to start farming ? Would you need to link every single game if I wanted to farm them all ?

@penisgesicht231 Have you added any games to the priority list first? It won't do anything until you do.

I tried it with priority games too yes I currently have no game account linked is that required to start watchgin ?=

Do you have games linked? Can you add a screenshot of the inventory with all options checked?

I didnt link any accounts yet is that required to start farming ? Would you need to link every single game if I wanted to farm them all ?

Yes. You need that anyway to get the rewards.
From README.md:

TDM is not intended for/as:

Mining channel points - again, it's about the drops: only. The current points you're getting are a byproduct of getting the drops, not the main goal of it.
Mining anything else besides Twitch drops - no, I won't be adding support for a random 3rd party site that also happens to rely on watching Twitch streams.
Unattended operation: worst case scenario, it'll stop working and you'll hopefully notice that at some point. Hopefully.
100% uptime application, due to the underlying nature of it, expect fatal errors to happen every so often.
Being hosted on a remote server as a 24/7 miner.
Being used with more than one managed account.
Mining campaigns the managed account isn't linked to.

Do you have games linked? Can you add a screenshot of the inventory with all options checked?

I didnt link any accounts yet is that required to start farming ? Would you need to link every single game if I wanted to farm them all ?

Yes. You need that anyway to get the rewards.
From README.md:

TDM is not intended for/as:

Mining channel points - again, it's about the drops: only. The current points you're getting are a byproduct of getting the drops, not the main goal of it.
Mining anything else besides Twitch drops - no, I won't be adding support for a random 3rd party site that also happens to rely on watching Twitch streams.
Unattended operation: worst case scenario, it'll stop working and you'll hopefully notice that at some point. Hopefully.
100% uptime application, due to the underlying nature of it, expect fatal errors to happen every so often.
Being hosted on a remote server as a 24/7 miner.
Being used with more than one managed account.
Mining campaigns the managed account isn't linked to.

hmm ok will try later with added accounts I usually claim on twitch and add account later always worked guess not here

It seems this has something to do with Exclude behavior. Even if my Exclude list is empty, in the Inventory all games only show up after I check "Excluded".

I feel like this is the same "issue" as #537. Due to semi-recent changes, all ACL channels start as OFFLINE now, which may cause the miner to switch to IDLE mode right after a reload, displaying that message. The channels will gradually come ONLINE over the next 2-3 minutes though, and the mining will continue as normal. This can also be seen on this picture: https://github.com/DevilXD/TwitchDropsMiner/issues/543#issuecomment-2295286209  Most channels have the OFFLINE ⏳ status, indicating they're about to switch to ONLINE within the next 2 minutes or so.
@penisgesicht231 You have to link your Twitch account before you attempt mining drops, as unlinked accounts have a 24 hours period to link after claiming a drops, so you'd have to do it soon anyway. Linking is a single-time process, and persists between campaigns for a given game, so that shouldn't be an issue.

Hey, I'm still getting the error ‘07:36:35: No active campaigns to mine drops for. Waiting for an active campaign...’ Although I have set a game in the settings:

In addition, the inventory also works I do not understand what it is I hope someone has a solution ready :):

The new bulk channel status refresh logic has been implemented, so this shouldn't be happening anymore. Otherwise, please open another issue.


Logging out doesn't clear the session

else if (isset($_GET['slo'])) {
    $returnTo = null;
    $paramters = array();
    $nameId = null;
    $sessionIndex = null;
    $nameIdFormat = null;

    if (isset($_SESSION['samlNameId'])) {
        $nameId = $_SESSION['samlNameId'];
    }
    if (isset($_SESSION['samlSessionIndex'])) {
        $sessionIndex = $_SESSION['samlSessionIndex'];
    }
    if (isset($_SESSION['samlNameIdFormat'])) {
        $nameIdFormat = $_SESSION['samlNameIdFormat'];
    }

    $auth->logout($returnTo, $paramters, $nameId, $sessionIndex, false, $nameIdFormat);

Note what happens when someone logs out - the session never gets cleared!
So when you head back to index.php you get the same page with the same user info - confusing! Worse, there is no link to log in, just one to log out!

On SAML, at a SP-initiaited Single Logout, the session is not cleared before sending the LogoutRequest, instead, when receiving a LogoutResponse from the IdP.
So the sls code should validate the LogoutResponse and close the session


Reasonable use of _hidden

The context is a mvc application which uses unobtrousive validation.
Using a custom format submit the value (and the name of the input) is moved to the hidden input.
This behaviour breaks the validations, so 2 changes to the hidden field are needed:

copy all the data-val information to the hidden input
attach a class to the hidden input that could be used as selector to remove the hidden field from the ignore list of jquery validator

+1


Hotfix/cu nkd4t0 resolution settings

Resolution displayer isn't correctly set. Fixed
Increase / Decrease Resolution buttons are not set correctly.
Doesn't set resolution correctly.

Task linked: CU-nkd4t0 Resolution Settings script isn't correct

Setting fullscreen to true and then to false leaves the resolution not properly set, this doesn't break gameplay or anything, it just doesn't work 100% as it should


Add support for C++

One nit, add .cpp support, and this is approved.

Done

Thank you.  Merged into master.


Move the sizing to method recommended by chart.js

Add a chart-container wrapper div and move the sizing styling to be
applied to this as recommended by the chart.js docs: http://www.chartjs.org/docs/latest/general/responsive.html#important-note
This means you can also set a height for the charts by adding css rules
that apply to the chart container combined by setting
maintainAspectRatio to False
For example:
Adding options = {'maintainAspectRatio': False} to your Chart subclass
and adding the below css rule.
.chart-container {
    height: 300px;  /* this can also be in other units such as viewport units*/
}

Regarding the code itself:

I can’t see where the async templates are used - It looks to me like they’re not but I changed the async template to be in line with the other template anyway.
It is hard to test that it has the expected effect (essentially none for existing users). Let me know if you can think of a way to test it @matthisk
This potentially has implications for the check that asserts that responsive=False before you can set a width or height as it is also fine if you have maintainAspectRatio=False.

Coverage remained the same at 95.354% when pulling fa5d8055b6edde9488cbcf8f4c8795f0f2f9a04e on intiocean:chart-container into abbc6feed5fa3e2d3681ceab8a6473b25ec7d455 on matthisk:master.

Hi, @matthisk - any objections to merging this?

@intiocean sorry for the late reply. I am going to have a look at it now.

Alright I merged this and will create a new release.

Btw since this is already your second contribution to this project I invited you to become a collaborator. I do not have a lot of time to work on this project atm and am not using it in any production codebases currently. I think there are still some feature requests in the issues section, maybe we could implement one of those requests. Let me know.

Fab @matthisk - thanks for taking a look and merging. 👌


Add learning_methods and confidence to FilterInput

Modified since the database changed

Codecov Report
Patch coverage: 41.86% and project coverage change: -0.08 :warning:

Comparison is base (cdd9fc4) 54.51% compared to head (1d6ef5a) 54.44%.

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
Additional details and impacted files
@@            Coverage Diff             @@
##             main       #5      +/-   ##
==========================================
- Coverage   54.51%   54.44%   -0.08%
==========================================
  Files          45       45
  Lines        8087     8129      +42
==========================================
+ Hits         4409     4426      +17
- Misses       3678     3703      +25

Impacted Files
Coverage Δ

src/graphql/node.rs
86.02% <ø> (ø)

src/graphql/event.rs
68.36% <30.76%> (-1.20%)
:arrow_down:

src/graphql/filter.rs
69.74% <56.25%> (-0.66%)
:arrow_down:

src/lib.rs
0.49% <100.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


Improve changelog processing to prevent conflicts

Is there an existing issue for this?

[X] I have searched the existing issues

Does this enhancement require public documentation?

[X] I have added an Acceptance Criteria item for adding and/or adjusting public documentation (if applicable)

Problem Statement
Currently we edit changelog by directly editing the CHANGELOG.md file. This method has several inconveniences:

Very easy to fall into conflicts. When 2 PRs are both editing the changelog, It is likely to conflict on the CHANGELOG.md file. It rarely succeed to create a backport PR automatically when the PR has changelog updates because of conflicts.
Need to manually add items and diff links in each release. The two parts are on the top and bottom of the whole changelog separately. It is possible to forget one of them.

Proposed Solution

Develop a tool to extract changelog from PR description or title
Let the tool to generate the changelogs for each release

Additional information
No response
Acceptance Criteria

[ ] PR template includes the "Changelog" section to set the  wanted changelog item related to this PR
[ ] A tool automatically fetches the changelogs from the PR and generates the summarized changelog for each release
[ ] Include guide in CONTRIBUTING.md to help contributors to set changlog in PRs

FWIW here's the tool that mesh uses: https://github.com/kumahq/ci-tools/blob/main/cmd/release-tool/changelog.go
It's published and I think it's can easily be extended to support KIC's use cases

As internal discussion, we would keep using the current way of managing changelogs. Close this.


dm doesn't work at all

simply doesnt return a message object on dm.

this is a discordjs bug, likely resolved in the detritus switch

u suck
( i love you )

ily2 doge :3

😆♥♥♥♥♥💘♥♥❤❤❤❤❤🤎😍😍💖


Multi-arch containers

It's cumbersome to know how many versions make up a single build of a multi-arch container. It seems that each distro needs it's own version, but then there are some magic ones that are also around. It could be that I'm misconfiguring my build. Here's my jobs:
jobs:
  docker:
    name: Docker
    runs-on: ubuntu-latest
    timeout-minutes: 10
    steps:
      - uses: actions/checkout@v3
      - uses: docker/setup-qemu-action@v2
      - uses: docker/setup-buildx-action@v2
        with:
          platforms: 'linux/amd64,linux/arm64'
      - id: version
        shell: bash
        run: echo "version=$(date +%Y%m%d%H%M%S)" >> $GITHUB_OUTPUT
      - uses: docker/login-action@v2
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ secrets.GH_TOKEN }}
      - uses: docker/build-push-action@v3
        with:
          context: '.'
          push: true
          platforms: 'linux/amd64,linux/arm64'
          tags: ${{ env.BUILD_DOCKER_IMAGE }}:${{ steps.version.outputs.version }},${{ env.BUILD_DOCKER_IMAGE }}:latest
          cache-from: type=registry,ref=${{ env.BUILD_DOCKER_IMAGE }}:latest
          cache-to: type=inline
      - uses: actions/delete-package-versions@v4
        with:
          package-name: ${{ env.APP_NAME }}
          package-type: container
          min-versions-to-keep: 1
          token: ${{ secrets.GH_TOKEN }}

When this runs and I try to pull afterward, I get manifest unknown. When I bump it to 2, same thing. 3, sometimes works but sometimes not. 4 seemed to consistently work until this morning. My goal is to only store the latest docker image but I'm having trouble figuring out how to selectively keep all the "latest" images for the various architectures I need to support.

I've been having the same problem, the thing here is that from my understanding multi-arch images are not an actual image but a pointer to images for different architectures.
This means not all the images are tagged, only the pointer image is tagged as only one image can have the same tag, the rest are left untagged.
So when delete-only-untagged-versions: 'true' or min-versions-to-keep: 1  is used all the images that are pointed to by the multi arch image are deleted leaving only the multi-arch "image" that points to them. Because they have been deleted the manifest error occurs on pull.
If you only have images being built one way, e.g. on merge or on release, it is relatively simple to delete old images, you can just keep the last x untagged images where x is the number of architectures being build.
If you are building on schedule, on merge, on release, manually with different tags being used for each this becomes a nightmare as all of them produce untagged images at different intervals and you couldn't delete the last x images because if you would risk deleting an image referenced by another tag that may not be updated so frequently.
For example if schedule was run daily and release was weekly, and then you have some adhoc builds thrown in also by the time your on day 6 you may have 30 odd images and if you delete x amount you could risk deleting the images referenced by main because a lot of pr's happened.
Really to safely delete old multi-arch containers a list of tags would need to be used, that would be needed to be used to pull the multi-arch manifests and find the images referenced in them, then everything else could be deleted. I'm not sure whether or not that would be in the scope of this package but believe it is a valid want for developers.

Also interested in this. Took me a while to figure out why I'm getting "manifest unkown" after switching to multi-arch / buildx.
I agree it should be possible that only untagged packages get deleted that are not referenced by other multi-arch packages.

Specifically for ghcr.io containers packages I figured out how todo this in a new action I wrote: https://github.com/dataaxiom/ghcr-cleanup-action

Specifically for ghcr.io containers packages I figured out how todo this in a new action I wrote: https://github.com/dataaxiom/ghcr-cleanup-action

This worked for me.


"systemctl status harvest" not accurate

Hi
my harvest environment seems to be working, however the systemctl status harvest command does not show my poller as running, even though it works...
some details about the environment:
 systemctl status harvest
● harvest.service - Harvest
   Loaded: loaded (/etc/systemd/system/harvest.service; enabled; vendor preset: disabled)
   Active: active (running) since Fri 2021-05-21 08:06:19 UTC; 5min ago
  Process: 8875 ExecStart=/opt/harvest/bin/harvest restart --config /opt/harvest/harvest.yml (code=exited, status=0/SUCCESS)
 Main PID: 8882 (poller)
    Tasks: 8
   Memory: 31.2M
   CGroup: /system.slice/harvest.service
           └─8882 bin/poller --poller cluster1 --loglevel 2 --config /opt/harvest/harvest.yml --daemon

May 21 08:06:19 rhel6 systemd[1]: Starting Harvest...
May 21 08:06:19 rhel6 harvest[8875]: Datacenter            Poller                PID        PromPort        Status
May 21 08:06:19 rhel6 harvest[8875]: +++++++++++++++++++++ +++++++++++++++++++++ ++++++++++ +++++++++++++++ ++++++++++++++++++++
May 21 08:06:19 rhel6 harvest[8875]: lod                   cluster1                                         not running
May 21 08:06:19 rhel6 systemd[1]: Started Harvest.

Polling Harvest:
curl localhost:31000/metrics | grep volume_write_data
volume_write_data{datacenter="lod",cluster="cluster1",node="cluster1-01",svm="nfs_svm",aggr="aggr1",type="flexvol"} 0
volume_write_data{datacenter="lod",cluster="cluster1",node="cluster1-01",svm="nfs_svm",aggr="aggr1",type="flexvol"}<PHONE_NUMBER>9938857
volume_write_data{datacenter="lod",cluster="cluster1",node="cluster1-01",svm="cluster1-01",aggr="aggr0",type="flexvol"} 11622.309120154781

my Harvest configuration
$ more harvest.yml
Exporters:
  Harvest:
    exporter: prometheus
    port: 31000
    addr: <IP_ADDRESS>

Defaults:
  collectors:
    - Zapi
    - ZapiPerf
  exporters:
    - Harvest
  use_insecure_tls: true

Pollers:
  cluster1:
    datacenter: lod
    addr: <IP_ADDRESS>
    auth_style: basic_auth
    username: admin
    password: Netapp1!

Environment configuration:

NetApp Lab on Demand "Using Trident with Kubernetes and ONTAP v4.0"
host: RHEL 7.5
installation method: yum install harvest-21.05.1-1.x86_64.rpm

Could you check output of below command.
cd /opt/harvest
bin/harvest status
I suspect, systemd is just showing cmd output and got the poller status "not running" when it was executed. it should be showing old print log of harvest. systemctl doesn't run bin/harvest status internally. It just shows service status.

yeps, looks like it !
[root@rhel6 harvest]# pwd
/opt/harvest
[root@rhel6 harvest]# bin/harvest status
Datacenter            Poller                PID        PromPort        Status
+++++++++++++++++++++ +++++++++++++++++++++ ++++++++++ +++++++++++++++ ++++++++++++++++++++
lod                   cluster1              9006                       running
+++++++++++++++++++++ +++++++++++++++++++++ ++++++++++ +++++++++++++++ ++++++++++++++++++++

I guess I just need to use the right command to get the accurate status.
What is the PromPort supposed to display?

PromPort is Prometheus Port used for poller. I see , its empty in your case but you are able to curl.
Could you share the output of
ps -ef | grep poller

$ ps -ef | grep poller
root        26     2  0 May20 ?        00:00:00 [edac-poller]
root      9489     1  0 09:00 ?        00:00:02 bin/poller --poller cluster1 --loglevel 2 --config ./harvest.yml --daemon
root      9533  8196  0 09:11 pts/0    00:00:00 grep --color=auto poller

Can you change your exporter value to Prometheus. (capital P)  Current check in code is case sensitive for it to show in status.
exporter: **P**rometheus
port: 31000
addr: <IP_ADDRESS>

that did the trick !
thanks @rahulguptajss !!


Unit tests and integration test as per standard Adobe practice

Unit tests and Integeration tests should be added that can be triggered by maven as per the usual practice in Adobe AEM development.

We've created some unit and integration tests. We have a doubt related to the workflow is in charged of generating content using the corresponding config file. Should we force the workflow execution or  on the contrary configure a launcher?

I guess the integration test should do everything as close to real-world behavior as possible. So this means configuring a WF launcher that launches when the the config file is placed in the repo. The test then tests by putting a test config in the repo location, triggering the WF. Doe this make sense or do I misunderstand the issue?

We want to ask you about the transformer it should be used in test cases. We are thinking it's better to use a transformer that doesn't execute and create anything in JCR. If you agree, we can reuse an empty implementation it was developed by us for the test approach.

For the unit tests, we want them to run locally on each maven build (unless, of course, your specify skip tests). These tests must run directly without AEM. For example, on of these tests should just run net.sf.saxon.Transform with the dita-to-content.xsl against the example mcloud.ditamap and check if the output matches a pre-defined xml document. Other units test should also be written, basically to cover anything that can be tested without a running AEM.
The integration tests are those that require a running AEM instance and test the actual creation of content in the repo, etc. For these we should use the maven surefire plugin. These should be runnable (optionally, not automatically) from maven on a local machine with a running AEM present. Ideally we would want to set up a continuous integration for this. Alex and I will look into this.

I've finished a first set of tests following the requirements you described. I only have to add it how to invoke integration test using the suitable profile in maven.

Lydia gave a nice preso at CQ Con. please take a look at it [1]. The key for integration testing is to use quickstart-maven-plugin which let you instrument as many AEM instances as you need.
If you're not familiar with this maven plugin yet, just ping me.
[1] http://www.slideshare.net/CQCON/prsentation-lydia-balazs

Could you please shed some light about this plugin? Thank you in advance.

I haven't received any information about the adobe maven plugin in order to be much simpler integration test. Could you please send me it?

Interestingly enough, I just sent you the info :D.

I tried to test quickstart plugin, but there isn't anything to download in adobe maven repository. May I ask you for sending me all necessary files to add to my local maven repository or where am I able to get them?
Thank you.

Hi @jralonso84, I'm working with our guys to publish the plugin. Stay tuned!

The tests to be written should cover the following (from Gaetano and Javier's own proposal):

Test all expected error conditions. For example, what if certain parameters are missing from the config file or form?

A unit test per example transformation. There will be a number of example DITA docsets and Docbook docset to test against. Cover each of these

Similarly, each doc set transformation should be covered by a full beginning-to-end integration test.

Test situations where a small amount of content has changed. For example, a correction in the written content of a single page. This requires that issue #17 be resolved

Test situations where zip file has required files in root vs. within another subfolder inside the zip.

One more thing. If it is not clear, please write tests for both DITA and Docbook, even though the Docbook XSLT is not quite ready yet, we want to be ready for it.

Sorry, accidentally closed....open again

A small recap for today:

We've created some tests checking the final content comparing a text result in a page.
We've replicated all situations from DITA transform test cases to DocBook
We are preparing some tests in order to check expected errors to be returned by the importer.


Remove resizeInput interface to TFLite Interpreter

As per the issue linked below, it appears that resizeInput is no longer supported due to performance issues so should be removed as an interface from the TFLite Interpreter.

@Burton2000 So we essentially dropped support of dynamic input shapes generally speaking due to performance considerations and the difficulty of supporting it while maintaining good performance. But of course since this issue is old, maybe this wasn't true at the time, and I think resizeInput is a legacy interface that we should probably remove. In order for us to properly prioritize, can you perhaps create a new issue to remove that interface so that we can remain consistent in our design philosophy? Thanks for your help.

Originally posted by @pkgoogle in https://github.com/tensorflow/tensorflow/issues/44016#issuecomment-1787818211

Hi @Burton2000, apologies for all the whiplash but TF and TFL are a living codebases so everything is quite dynamic still. With 2.15 we are currently supporting dynamic input shapes: https://www.tensorflow.org/lite/guide/inference#run_inference_with_dynamic_shape_model That being said, please try your previous test to see if the original issue, is still an issue, and then we can take it from there.

Hey @pkgoogle not a problem. There is no .aar of 2.15.0 of TFLite available on maven to try, I updated to 2.14.0 which is the latest available there but no difference to previous issue reported.

Hi @LakshmiKalaKadali, can you please take a look? Thanks.

As this issue is no longer valid and there is a workaround for the original issue which does not affect performance we will close this issue as invalid.


Protocol relative urls support

Support for protocol relative urls, for ex. '//domain.com/styles.css'. See issue #205

Does this really solve your issue?
Extname does not care about the path:
> var path = require('path');
> path.extname('##¿¿¿¿\\\//a/b.css')
'.css'

My case is solved, as chunk name is //domain.com/styles.css
Do you think there is a better way to extract an extension? '''str.split()''' should also work

Which node version do you use?
path.extname does not care about the protocol.

C:\Windows\System32>node -v
v4.2.1

C:\Windows\System32>node
> require('path').extname('//domain.com/styles.css')
require('path').extname('//domain.com/styles.css')
''
>

Returns empty string in this case

By the way, I found a workaround to remove publicPath from webpack config when not in development. Then it just generates relative urls (without any protocols and domains) and seems to work.
Can close  the issue, if you think that chunk names with protocol relative urls is not a valid use case for html-webpack-plugin

Ah - windows ;)
Could you please add a unit test for it so it won't be broken again by future changes?

Changed it a little bit and merged it in 2.8.2!
Thamks for your help


move from redis master to redis cluster

Current REDIS based ISB implementation does not support REDIS in cluster mode. To have a scalable ISB, we need to support REDIS cluster.

closed via #195


[Backport] Retrying with a backward compatible task type on unknown task type error in parallel indexing (#8905)

Backport of #8905 to 0.16.1-incubating

Team city error is unrelated, merging


InsecureRequestWarning on secure request

I have my ES cluster behind HAProxy with SSL from Let's Encrypt.
Still, I get those warnings at every request.
Nov 18 21:03:10 elc-1 prometheus-es-exporter[4275]: [2020-11-18 21:03:10,155] elasticsearch.INFO Thread-62 GET https://xxxxxx:443/_nodes/stats [status:200 request:0.119s]
Nov 18 21:03:22 elc-1 prometheus-es-exporter[4275]: [2020-11-18 21:03:22,938] py.warnings.WARNING MainThread /usr/local/lib/python3.5/dist-packages/urllib3/connectionpool.py:1020: InsecureRequestWarning: Unverified HTTPS request is being made to host xxxxxxx Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
Nov 18 21:03:22 elc-1 prometheus-es-exporter[4275]:   InsecureRequestWarning,

Is there a possibility to actually check the certificate if it's not given in configuration?

Hi @askz, you'll need to provide the --ca-certs option, pointing to some set of CA certs.
The exporter doesn't (explicitly) ship with CA certs, as I'd then need to release a new docker image whenever the bundle is updated (and a lot of orgs need their own internal certs anyway).
As it happens, the docker image actually contains (at least) two CA cert bundles, which you can use to quickly get cert checking running (assuming you're using docker). However, these aren't explicit dependencies, so are certain to be out of date (or soon will be), and may disappear in the future.
OS certs:
--ca-certs='/etc/ssl/certs/ca-certificates.crt'

Python certifi package (based off Mozilla's bundle):
--ca-certs='/usr/local/lib/python3.8/site-packages/certifi/cacert.pem'

I strongly recommend using some other method of provisioning certs so you can ensure they're available and manage their update cycle. For example, you could extend the docker image to explicitly install the latest version of certifi.

Thanks.
Closing as it's resolved !


Animation from GLTFLoader

Hello,
Thanks for the great package and work you do.
I am having some issues getting my model to animate after being imported via GLTFLoader ( probably through user error ) is there something I should be using instead of primitive or if not where can I pass my animations.
const path = Config.modelURL;
const resource = createResource(file => new Promise(async res => ( new GLTFLoader().load(path + file, res))));

function Model({ file }) {

  const { scene, animations } = resource.read(file);
  console.log('Scene: ', scene);
  console.log('Animations: ', animations);

  // let mixer;
  // mixer = new AnimationMixer(scene);
  // scene.animations.forEach((clip) => {
  //   mixer.clipAction(clip).play();
  // });

  return <primitive object={scene}  />;
}```

once you have the model:
                animator = new THREE.AnimationMixer(obj.scene)
                obj.animations.forEach((clip)=>{
                    animator.clipAction(clip).play();
                });

and then on the render loop:
           animator.update(delta)
i.e:
        var clock = new THREE.Clock();
        let delta = 0;
        useRender(i=>{
            if(!animator)return;
            if(!this.state.animate)return; //this is what requires us to keep the scene in the viewer class
            delta = clock.getDelta();
            animator.update(delta)
        })

I haven't used threejs animations before but useRender also gives you a timestamp for free
useRender((state, time) => ...)

Other than that, you can use everything in the react eco system to animate these objects.
As for instanciating the mixer, this shouldn't be smack in the middle of the render function. Something like this perhaps:
const { scene, animations } = resource.read(file);
const mixer = useState(() => new AnimationMixer(scene))

// Play animations after mount
useEffect(() => {
  scene.animations.forEach(clip => mixer.clipAction(clip).play())
}, [])

return <primitive object={scene}  />

@drcmda I have done a quick test, that time arg is not a literal delta, but instead the running time of that specific render, the altered code to handle that and keep regular behavior is something like:
        let lastime=0;
        useRender((state, time)=>{
            if(animator && this.state.animate){
                animator.update((time-lastime)/1000)
            }
            lastime=time;
        })

Fantastic this works perfectly, thank you for the help guys!

@UltimateForm shouldn't it pass on the delta then? i'm not super knowledgable with this stuff, but the mere timestamp is useless, right? If i make your take official and you could get
useRender((state, delta)=> {

would this have any downsides? And what dows the /1000 do, btw?

@drcmda I would prefer if it passed the literal delta yes, but I don't know if it's doing that for a reason so I'm not so sure. the delta/1000 is because the time passed is on milliseconds units, and the the mixer expects a delta value in seconds units


WinAppDriver Integration

Is there any plan to support WinAppDriver?

Yes, Windows testing will be supported by leveraging the WinAppDriver project. We've done some POCs and created the scaffolding for this integration, but I wouldn't be able to commit to a date at this time.


View | Toggle Minimap only works when an editor is open

Testing https://github.com/Microsoft/vscode/issues/33301
Open an editor
Note the visibility state of minimap, either on or off
Close all editors
View | Toggle Minimap
Open an editor
Note that the visibility of the minimap has not changed
If you examine settings.json, you see that the value is not changed there either
This is the same behavior as the other "Toggle" commands in the menu group.
It makes me wonder, why are these all "Toggle"? Why aren't they check marks of on or off, like File | AutoSave.

:+1: Nice find regarding the toggle not working when all editors are closed.
:+1: on using checkmarks next to the toggle actions. I will, however, extract this to a separate feature request. The problematic thing here is that those Toggles are possibly editor instance specific. i.e. focused editor specific.
One could configure to see minimap only in TypeScript files and not in JavaScript files via our language selectors. Users would expect then, when focus is in a TypeScript file, and they invoke the menu, that the checkmark would be checked since minimap is enabled in TypeScript files. Also, when focus is in a JavaScript file, and they invoke the menu, the checkmark should be unchecked.
Our menus do not update when a specific editor is focused, so our only chance for these potentially editor specific actions is to have a generic "Toggle X" label, until we can make that the menus update whenever a specific editor gets focused.

Will postpone for September. Multiple Toggle actions share the same problem.

I have migrated ToggleMinimapAction, ToggleRenderControlCharacterAction and ToggleRenderWhitespaceAction to be workbench actions such that they can run also when an editor is not in opened.


Not setting CountryCode of OriginAddress in UPSProvider.BuildRatesRequestMessage Method

While Create the XML, CountryCode of Shipment.OriginAddress.CountryCode is not wriiten. Due to this shipping rates for shippings which have origin country Canada not comes.
When I insert 'writer.WriteElementString("CountryCode", Shipment.OriginAddress.CountryCode);' at  UPSProvider 176  It stared to giving shipping rates for Canada.

thanks for the report. I have fixed this and added a test to ensure it doesn't pop up again. v<IP_ADDRESS> has been deployed to NuGet.


Incompatibility with tailwind eslint

I'm receiving errors on my formatter:
./src/components/sidebar/account-dropdown.tsx
36:33  Warning: Classname 'radix-side-top:animate-slide-up' is not a Tailwind CSS class!  tailwindcss/no-custom-classname
36:33  Warning: Classname 'radix-side-bottom:animate-slide-down' is not a Tailwind CSS class!  tailwindcss/no-custom-classname

Config
{
  "extends": [
    "next/core-web-vitals",
    "prettier",
    "plugin:storybook/recommended",
    "plugin:tailwindcss/recommended"
  ]
}

I’d suggest you report this to the eslint plug-in repository that it doesn’t seem to resolve plug-in generated classes correctly. Nothing I can do from this side.

@yagiz-noonlight Did you find a solution? If not, I created a ticket: https://github.com/francoismassart/eslint-plugin-tailwindcss/issues/210


Add competition support

This PR refactors the action and adds support for competitions.

[x] Add basic implementation of competitions
[x] Refactor animations - move to Python
[x] Fix usage of WEBOTS_HOME
[x] Add mandatory type parameter (can be demo, competition and competitor)
[x] Add support for C/C++ controllers
[x] Add local deployment mode for development (check DEVELOPMENT.md)

Finalization points

[x] change title to "Rat's Life Leader Board"
[x] the ranking should start at 1, not 0.
[x] we should probably remove the dummy controller from the competition.
[x] in the heading of the ranking table, I would use "participant" or "competitor" instead of "controller".
[x] build animation for competitors
[x] improve descriptions
[x] make the links more visible
[x] upgrade to R2021a
[x] compete with a better controller
[x] auto-accept invitations in CI
[x] improve results format
[x] add timestamp of the last round
[x] add interval

It seems there is a conflict preventing merge...

It seems there is a conflict preventing merge...


When user is not sign in, click on navigation item should open login modal

Now we can see a cursor but click do nothing

Prefered to make links not clickable.
Fixed in this commit be64e6534abcffc382372e1354e0e9657aec6cab


Small refactoring in class.go

Just a small refactoring to prepare updating api.doc

Rename builtin class methods for consistency in naming
Remove unnecessary func call
Swap declarations in class.go for consistency

@hachi8833 Can you comment on the places you change below?

Remove unnecessary func call
Swap declarations in class.go for consistency

The diff is quite large it's hard to find them.

Sorry, I shouldn't have squashed them.

Just swapped the order of func builtinCommonClassMethods and builtinCommonInstanceMethods for consistency. No other changes have been made.

Found that the branch was invalid. Sorry.
Closing PR


How to feed transformed data into a trainer?

Issue:
When I'm feeding transformed time series data into the trainer, processing appears to start, but actually doesn't make any progress.
Description:
With gluonts.transform.Chain I am trying to transform my ListDataset to a richer dataset which will hopefully improve my predictions. However, when I am trying to train an estimator on this transformed dataset processing appears to get stuck. Debugging I found out that the object returned by the transformation is a generator and not a ListDataset. The structure of data inside this generator is similar to structure of the data inside the original ListDataset, although the generator does not contain target key.
How do I successfully train estimators on a transformed dataset?
Replicate
To replicate this issue I converted my problem into a notebook using some tutorial data. Please find it at the following link:
https://colab.research.google.com/drive/1xvVRY8wrWVuT8VydJoUAOgWfFvawXqb1?usp=sharing

If you want to change the transformation chain used by a pre-defined model, the simplest way is to subclass the model and override the create_transformation method, e.g. in your example
from gluonts.model.simple_feedforward import SimpleFeedForwardEstimator

class MySimpleFeedForward(SimpleFeedForwardEstimator):

  def create_transformation(self):
    return Chain(
        [
            AddObservedValuesIndicator(
                target_field=FieldName.TARGET,
                output_field=FieldName.OBSERVED_VALUES,
            ),
            AddAgeFeature(
                target_field=FieldName.TARGET,
                output_field=FieldName.FEAT_AGE,
                pred_length=self.prediction_length,
                log_scale=True,
            ),
            InstanceSplitter(
                target_field=FieldName.TARGET,
                is_pad_field=FieldName.IS_PAD,
                start_field=FieldName.START,
                forecast_start_field=FieldName.FORECAST_START,
                train_sampler=ExpectedNumInstanceSampler(num_instances=1),
                past_length=self.context_length,
                future_length=self.prediction_length,
                time_series_fields=[
                    FieldName.FEAT_AGE,
                    FieldName.FEAT_DYNAMIC_REAL,
                    FieldName.OBSERVED_VALUES,
                ],
            ),
        ]
    )

The alternative is to use a transformation.TransformedDataset to apply a transformation to a dataset and obtain a new dataset. However, in that case the transformation declared in the model will still be applied, and you have to make sure that whatever you are applying additionally can be combined (e.g. in your example, it doesn't make sense to apply InstanceSplitter twice).

Cheers, that helps! I'm closing this issue.


[Parse] Reject trailing closures on literals

1 { } was parsed as a call expression with a trailing closure. This made the diagnostics for var x = 1 { get { ... } } extremely bad. I believe this should never be allowed for literals, and refusing to parse it as a trailing closure allows much better diagnostics to kick in:
Before
error: use of unresolved identifier 'get'
var x: Int? = nil { get { return 2 } }
                    ^~~

After
error: variable with getter/setter cannot have an initial value
var x: Int? = nil { get { return 2 } }
    ^         ^~~

Resolves SR-3671, at least for cases where the initial value is a literal.

@swift-ci Please smoke test

@swift-ci please smoke test

@shahmishal the OS X build seems very unhealthy..possibly just temporary network issues. https://ci.swift.org/job/swift-PR-osx-smoke-test/4813/console

@swift-ci please smoke test macOS platform

This looks good to me.
"Only for literals" looks like a good idea.
We can never have the perfect  (source compatible) solution for this problem, anyway.
func get(_: () -> Void) {}
func foo(_: () -> Void) -> Int { return 1 }

let x: Int = foo {
    get {}
}

I suppose we could delay all of these diagnostics to Sema, where it can know for certain if the braced block is a valid closure or more like a getter/setter block. But it's still kind of a guess anyway.

This is all on a recovery path, and this still seems like an incremental improvement. I say go for it.

Shall we wait for @DougGregor (who is the CODE_OWNER of Parse and Sema)? I'm not sure to what extent the code owners desire to comment on every little change that goes in their component.

I think it's okay. The process is supposed to be that the code owner makes sure everything in their component gets reviewed by someone they trust, not that it's been reviewed by them specifically. And they can always comment post-merge, and as code owner have the power to revert if they disagree. Between me and @rintaro I think you've gotten sufficient review.

Thanks!

Code owners don't have to weigh in on or review everything... we just try to make sure nothing goes unreviewed.


Unable to select fields for drillthrough in view mode

Steps to reproduce
In viewing mode, I tried to do a drillthrough. However, it displays the last selected fields of drillthrough which was done in Editing mode and I have no ways of changing these fields.

In this case, there can only be 1 predefined view of drillthrough for all the dashboards in viewing mode.
Environment

atoti: 0.4.2
Operating System: win32

Hi HuiFang, the view mode is made for when your dashboards are fully built and ready to be shared with less technical users.

Hi @tibdex,
even if I predefined, it meant one configuration for all dashboards isn't it?
Or can I define a set of drillthrough columns per dashboard?

I think the idea of the view mode is to stop modifying the structure of the dashboard and just browse what's already there with minimal distractions, potentially changing the filters a bit but not going much further in the analysis. That's why I'm saying that you could go back to edit mode to create all your drillthroughs with the columns you want, save your dashboard and then switch back to view mode and it will keep showing the columns you picked before.


Feature :: Popover Component

Codecov Report

Merging #136 into master will decrease coverage by 16.54%.
The diff coverage is 50.45%.

@@             Coverage Diff             @@
##           master     #136       +/-   ##
===========================================
- Coverage   93.83%   77.29%   -16.55%
===========================================
  Files          17       15        -2
  Lines         373      458       +85
  Branches       55       54        -1
===========================================
+ Hits          350      354        +4
- Misses         14       95       +81
  Partials        9        9

Impacted Files
Coverage Δ

src/lib/popover.ts
100% <100%> (ø)

src/components/Popover.vue
49.53% <49.53%> (ø)

src/components/Step.vue
71.42% <0%> (-28.58%)
:arrow_down:

src/components/Pipeline.vue
69.44% <0%> (-25%)
:arrow_down:

src/lib/translators/base.ts
97.87% <0%> (-2.13%)
:arrow_down:

src/components/DomainSelector.vue
81.81% <0%> (-1.52%)
:arrow_down:

src/lib/pipeline.ts
94.59% <0%> (-0.76%)
:arrow_down:

src/lib/translators/mongo.ts
96.55% <0%> (-0.7%)
:arrow_down:

src/components/DataViewer.vue
100% <0%> (ø)
:arrow_up:

src/main.ts
0% <0%> (ø)
:arrow_up:

... and 11 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1eb9b69...68178b1. Read the comment docs.


build: remove LLBUILD_FRAMEWORK, simplify

This removes the LLBUILD_FRAMEWORK option.  Use a proper
FindLLBuild.cmake module that searches for the framework if desired,
allows the user to specify the llbuild to use, or can use the export
targets from an llbuild instance.

Please test with following PRs:
apple/swift-llbuild#615
@swift-ci please smoke test

None of this builds on the ci so merging


fix syntax error in log-note-css-darkmode

Fixed  a syntax error in chapter 2, 3  and updated Readme file

Thanks for the edits! Great job catching my errors!


bump analyzer 2.2.0

Hi @smkhalsa I'm pingin you cause you was the last who contributed to gql_code_builder
I'm helping @comigor to maintain artemis
We would like to update our deps to latest version but we cannot due to analyzer: 1.2.0 dep in gql_code_builder
I made this PR to update ananlyzer to latest version.
I also added some fixes for failing actions - mostly dependency reordering.
I would appreciate if you merge and release my PR.
Thanks

@klavs may be you could review this PR?

@vasilich6107 can you run a pubspec clean with multipack to fix the CI errors?

thanks @smkhalsa for your support.
I ran the multipack command to resolve issues.
Some of them where resolved and some not.
I'm completely lost why github action demands me to sort dependancies in gql_example_build while they are sorted...

@vasilich6107 an alpha release should be published automatically with these changes.

Hi @smkhalsa
I would like to request more help from you.
It is true that gql_code_builder where released 28 Sep.
The issue is it depends on gql: 0.13.1-alpha+1632808907128 but there is no corresponding version of gql on pub.dev
In the meantime the gql_exec: ^0.3.1-alpha+1632808907137 dependacy where successfuly released yesterday.
Could you help with missing gql: 0.13.1-alpha+1632808907128?

Hmm, that's odd. I didn't set up the alpha release workflow, so I'm not sure why this could be happening.
I'm traveling at the moment, so I likely won't be able to look into this until next week, but maybe one of the other maintainers has some insight.

@smkhalsa do you know some who setted up the alpha release flow?

@klavs I saw you were adjusting the flow
https://github.com/gql-dart/gql/commit/092db3e39178959093bab866351d49e62123a4e0
Could you help us?

As you can see from the screenshot - all packages were updated on 28 Sep except gql

@agent3bood may be you will be able to help with issue of non-relased gql?

@vasilich6107 The build/release script need some reworking, at the moment can you just use direct git url for the version you need?
https://flutter.dev/docs/development/packages-and-plugins/using-packages#dependencies-on-unpublished-packages

@agent3bood thanks for quick response.
Unfortunately the direct git dependency is not an option - pub.dev does not allow to publish packages with overridden dependencies. I was doing this PR to unblock atremis package from json_serializable update.

Ok. Let's try one more time)
May be @micimize or @TarekkMA could help me with this release issue?

I don't know what is cats and why it's located in source files rather than test files.
We can extract it to another separate package. It doesn't seem to have a dependency on gql files.
Also, we can add it to .pubignore file, if it's not intended to be exported.

@vasilich6107 gql: ^0.13.1-alpha+1633799193898 is now publish on pub. 🥳


FPS is capped at 15

I have a Raspberry Pi 3B+ that can only record up to 1080p at 20 fps before losing frames. I really wanted to hit 30 fps, so I got a Pi 4. When I setup the RPI Cam Web Interface, the live preview was showing at 1080p at 15 fps. I increased the frame rates for recording and boxing to 30, but it is still streaming and recording only at 15 fps. If I play around with different numbers by gradually increase the frame rate, I can maybe hit 20 fps sometimes. I've tried decreasing the quality to even 5 or 1 to see if there was a performance limit, but that doesn't seem to be it. I don't know why the frame rate seems to be locked. I am still able to change other camera settings; it's just the frame rate that I can't affect.

There is no specific cap on the frame rate in this software.
Are you saying that when you change the frame rate on the web interface then it comes back with a different number?
Note that the live stream (preview) is not the same as a video recording. It is a jpeg stream either of separate images or formed into a mjpeg stream. The latter is more efficient and is selected under system settings. Normally most users have the preview at less than full resolution (preview settings) and rely on the actual video recordings which take place at the full specified resolution. However one can up the preview width to match the recording width to get the higher resolution, This does increase the bandwidth needed for the preview stream and this can become an issue for some networks particularly if over wifi. One can see the effect of the preview by turning on developer tools in the browser to see the network activity. Here it is quite good to turn off the mjpeg stream temporarily as you will then see each frame being delivered over the network and can see the packet size.

Thank you for the response.
Here's what I'm experiencing. On the web interface, I will enter say '25' or '30' for the recording and boxing fps. When I click 'ok' to apply the setting, the stream reloads, but I do not see the stream play at a higher fps than around 15. I'm certain it's not a network bandwidth issue since recorded videos also have a similarly low fps of 15.
I've already used the same camera on my Pi 3B+, and it was definitely capable of streaming and recording at around 24 fps. I was expecting the Pi 4 to achieve a higher frame rate, given that it's a newer hardware and all. It's almost as if there's a configuration file or something that overrides my settings on the web interface.


"'Default' tag requires a nonempty 'Extension' attribute" on Win10

I'm trying to build an installer, but when I try to run it the installation fails with 'Default' tag requires a nonempty 'Extension' attribute:
2016-02-25 08:03:31> IEnableLogger: Failed to create uninstaller registry entry: System.Xml.XmlException: 'Default' tag requires a nonempty 'Extension' attribute. Line 10, position 4.

I've done some googling around and it appears to be an issue with files without extensions.
The template.nuspec tries to create a LICENSE file. Removing the relevant line from <files> seems to resolve the problem.
 In the meantime, building installers on Windows (use AppVeyor if you don't have access to it) will solve this

@paulcbetts to clarify, your plan is to wait until this is fixed in Squirrel.Windows? (not complaining, just need to know so I can plan around :) )

@fcurella So, this bug only happens when you build a Windows installer with OS X or Linux because of a bug in NuGet (the C# version of RubyGems / npm / etc). If you build an installer with Windows as the host OS, it'll Just Work

@paulcbetts that means you'd rather wait for the bug to be properly fixed in NuGet, correct?

@fcurella No, you just need to use Windows to build your installer. We're probably going to work around this in Squirrel

Thanks for clariying @paulcbetts :+1:
FWIW for now I got it working just by editing my local copy of template.nuspec. I'm not familiar with developing on Windows, so that was much quicker for me.

FWIW for now I got it working just by editing my local copy of template.nuspec. I'm not familiar with developing on Windows, so that was much quicker for me.

@fcurella can you elaborate on what you changed in template.nuspec? Thanks.

@jprichardson I've simply removed the line mentioning LICENSE: https://github.com/electron/windows-installer/blob/master/template.nuspec#L22

Is there any update on if someone is working on this issue or if it will be fixed any time soon?

@Mercieral Try electron-builder ;)


503 ERROR The request could not be satisfied.

Before opening, please confirm:

[X] I have checked the custom domains troubleshooting guide.
[X] I have read the guide for submitting bug reports.
[X] I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.

App Id
d3e4u1u0b8ejac
Region
us-east-1
Step
Visit https://master.d3e4u1u0b8ejac.amplifyapp.com
Status
Failed
Issue or question
Since yesterday my app could be deployed, but just couldn't be visited. The error message is listed below.
Error message
503 ERROR
The request could not be satisfied.
The Lambda function associated with the CloudFront distribution is invalid or doesn't have the required permissions. We can't connect to the server for this app or website at this time. There might be too much traffic or a configuration error. Try again later, or contact the app or website owner.
If you provide content to customers through CloudFront, you can find steps to troubleshoot and help prevent this error by reviewing the CloudFront documentation.
Generated by cloudfront (CloudFront)
Request ID: F0SkirWMPKKGZZVhIRDRzSYDrNLs_qLw2TT9SQdZzSPQdHFZC0coBw==
Additional information
I have no idea what I've done with the lambda function, which I haven't put my hands on it.

Downgrading Next to 10.0.5 and setting webpack5: false in the next.config.js did the trick.

source: https://github.com/aws-amplify/amplify-console/issues/412#issuecomment-851704285
This also worked for me.

Downgrading Next to 10.0.5 and setting webpack5: false in the next.config.js did the trick.

source: #412 (comment)
This also worked for me.

Oh, man !! You're my Super Hero!! It works. Danke!

I downgraded to v11.1.3 and set webpack5 to false and then it worked for me.


Is it possible to send the message from simulink to carla?

Hi everyone,
I want to evaluate my control algorithm in Carla based on this interface, if it is possible.
Thanks a lot.

You could look into this blockset provided by Mathworks. It basically allows you to send and receive data to/from Unreal Engine - https://www.mathworks.com/matlabcentral/fileexchange/65966-vehicle-dynamics-blockset-interface-for-unreal-engine-4-projects

@lzhhh93 You could look into this blockset provided by Mathworks. It basically allows you to send and receive data to/from Unreal Engine - https://www.mathworks.com/matlabcentral/fileexchange/65966-vehicle-dynamics-blockset-interface-for-unreal-engine-4-projects

Thanks a lot. i am looking into it.
I have also thought about sending and receiving data through a Python API: https://de.mathworks.com/matlabcentral/fileexchange/77221-how-to-interact-with-simulink-using-python

To connect MATLAB with Carla:
Use matlab engine ( https://de.mathworks.com/help/matlab/matlab-engine-for-python.html ) to send and get data from Python API of Carla.
small hint: it is not neccesary to start MATLAB Engine for Python, if we only want to connect shared MATLAB session to MATLAB Engine for Python. Starting MATLAB Engine for Python takes a lot computation (somehow).
exemple:
in a python script:
    `## make sure the following command is executed in matlab command window, before running python scripts:
    ## matlab.engine.shareEngine
    import matlab.engine
    eng = matlab.engine.connect_matlab()
    throttle = eng.LongPID(self._k_p,error,self._k_d,_de,self._k_i,_ie)`

While in MATLAB:
function [throttle] = LongPID(k_p,error,k_d,de,k_i,ie)                throttle =  (k_p * error) + (k_d * de) + (k_i * ie);        end


suggestion: kalimon minimal image

the interesting idea is create a minimal version of kalimon (for 4gb memories) without GUI. It is useful to create a minimal image for small size memories

Great suggestion.  Should be easy to implement.

I would appreciate you uploading an ISO to download

https://github.com/nethunteros/rpi3-kalimon/issues/9#issuecomment-280896020


Get firmware info from Shelly update server

Official URLs verified with Shelly

I am also against having this functionality in aioshelly. Albeit rare, devices may require specific upgrade paths for some older versions. If the wrong firmware is flashed on a device it may brick. Our update server gets identifying bits from the devices when they ask for new versions and takes the necessary steps for flashing intermediate versions.
We are discussing adding a public repository with stable and previous stable releases, but what you have now is not it.

Albeit rare, devices may require specific upgrade paths for some older versions. If the wrong firmware is flashed on a device it may brick. Our update server gets identifying bits from the devices when they ask for new versions and takes the necessary steps for flashing intermediate versions.

@kzyapkov , this PR is not about flashing the firmware, but only retrieve the information of the latest available (thus the title).
Is in charge to the user to flash it.
On the other hand, take care that we have a update firmware button in HA that will flash latest.
This functionality is there since ages. So far, AFAIK, we didn't had a single report of a issue.

We are discussing adding a public repository with stable and previous stable releases, but what you have now is not it.

Nice! Thx.

On the other hand, take care that we have a update firmware button in HA that will flash latest.
This functionality is there since ages. So far, AFAIK, we didn't had a single report of a issue.

As I already tried to explain on discord (which you try to ignore) when we ask the device to trigger an update we are not providing the firmware URL. We just trigger update to latest which is  the same as the update button on the device web, the firmware URL is reported to the device based on multiple factors as @kzyapkov explained above.
I still insist we shouldn't provide any URL to the user, a simple example is Shelly RGBW2 which if uses the URL provided by this design will flash an incorrect firmware to the device (I can explain you the specific logic for this devices, but I don't know what is the logic in all devices), this is just one example and I am not sure how this will work for the entire install base and if there are other devices that can break.

You could let devices fetch update info -- call Shelly.CheckForUpdate

@thecode, please check the code.
It DOESN'T fetch any url at all.
It just retrieve the firmware build string, nothing else.

You could let devices fetch update info -- call Shelly.CheckForUpdate

The issue is linked to misconfigured devices that don't have internet access and don't notify the user of a new available firmware.
The idea is just to make them aware a new version exist.
But no action at all will ever be performed.

@thecode, @kzyapkov, sorry for the imprecision: I just noticed that I wrote "button" instead of "service".
What I wanted to focus is that we have a service in HA that can be used to flash device providing a URL to the OTA endpoint.
If this can brick the device, we should that avoid the possibility to use custom URLs.

What I wanted to focus is that we have a service in HA that can be used to flash device providing a URL to the OTA endpoint.

There is url parameter in aioshelly for gen1 devices but we don't use it in the integration. So we only run the API methods to start an OTA update.
https://github.com/home-assistant/core/blob/83e0a7528d8052005c338530ceb430e52e3ce08a/homeassistant/components/shelly/update.py#L212
https://github.com/home-assistant/core/blob/83e0a7528d8052005c338530ceb430e52e3ce08a/homeassistant/components/shelly/update.py#L297

What I wanted to focus is that we have a service in HA that can be used to flash device providing a URL to the OTA endpoint.

There is url parameter in aioshelly for gen1 devices but we don't use it in the integration. So we only run the API methods to start an OTA update.

Yes, if you look at the trigger_ota_update(), but the service itself allows the url parameter in its call.
This means that a user can specify a custom url for update.
As this can in rare cases be a issue as @kzyapkov wrote, I just wanted to point it out.

but the service itself allows the url parameter in its call

Which service?

I still think this simple notification (no action at all) could be very useful for all users that are not notified of a new firmware for whatever reason.
But Team decided not to go this way, so I close this PR.


Need to document (add tooltip) about the data directory structure

When I put the data directory as:
ebola/fastq_pass
ebola/fast5_pass
It got confused:

When it is as below, it is fine:
ebola/runid/fastq_pass
ebola/runid/fast5_pass

fixed in 5974abc9b46fd725f0c8a731f02191b89663a585 and 91da7094575a411de3025ff5ca424cfa3497e135

I think the proper names:
sample_id and run_uuid


Forgot password flow

I'm trying to set up the "forgot password flow". In my Rails 5 backend I'm using devise_token_auth (0.1.40).
I set the frontend up as in the documentation, meaning when an user requests a new password, I call:
this._tokenService.resetPassword({
    email:     this.resetPasswordForm.value.email,
});

This works as intended, and the backend sends an e-mail to the user with a token.
However, when the user clicks the link, and the password reset form is I call this:
this._tokenService.updatePassword({
    password:                this.resetPasswordForm.value.password,
    passwordConfirmation:    this.resetPasswordForm.value.password_confirmation,
    passwordCurrent:         null,
    resetPasswordToken:      this.resetPasswordForm.value.password_token
});

I now see a call happening to /auth with the provided data which fails with error "User not found". In my Rails log, I see this:

| Started PUT "//auth" for ::1 at 2017-04-02 14:23:03 +0200
| Processing by DeviseTokenAuth::RegistrationsController#update as JSON
|   Parameters: {"password"=>"[FILTERED]", "password_confirmation"=>"[FILTERED]", "reset_password_token"=>"[FILTERED]", "registration"=>{"password"=>"[FILTERED]", "password_confirmation"=>"[FILTERED]", "reset_password_token"=>"[FILTERED]"}}
| Unpermitted parameters: reset_password_token, registration

It seems that the password, password_confirmation and reset_password_token are being sent, and somewhere being re-routed/copied into registration: { password, password_confirmation, reset_password_token}.
I've looked into the source code of devise_token_auth, and noticed that the flow of reset password might be different than you expect. It seems you need an extra call, to allow a user-object to change the password once without a "current password".
The 'def edit' in the PasswordController of devise_token_auth states:

this is where users arrive after visiting the password reset confirmation link

So, it seems like when you open the form, you have to make a call to tell the backend an user has visited the form with the reset-token.
https://github.com/lynndylanhurley/devise_token_auth/blob/master/app/controllers/devise_token_auth/passwords_controller.rb#L94
There it seems like the user-permissions is set to change the password once, basically setting up the User model to expect an incoming password change.
Any ideas?
If you require additional information, I'd be happy to help.

Thanks for the explanation @cybey, were you ever able to get this working?
I'm trying to generate a custom link to allow users to 'set' their passwords initially after our admins create them from the backend.  I've got angular2-token set up to call the backend using:
this._tokenService.updatePassword({
    password:                this.resetPasswordForm.value.password,
    passwordConfirmation:    this.resetPasswordForm.value.password_confirmation,
    passwordCurrent:         null,
    resetPasswordToken:      this.resetPasswordForm.value.password_token
});

Then I get the same output on the server that you're getting:
Started PUT "/auth" for ::1 at 2017-04-30 22:46:24 -0400
Processing by DeviseTokenAuth::RegistrationsController#update as JSON
  Parameters: {"current_password"=>"[FILTERED]", "password"=>"[FILTERED]", "password_confirmation"=>"[FILTERED]", "reset_password_token"=>"[FILTERED]", "registration"=>{"current_password"=>"[FILTERED]", "password"=>"[FILTERED]", "password_confirmation"=>"[FILTERED]", "reset_password_token"=>"[FILTERED]"}}
Unpermitted parameters: reset_password_token, registration
Completed 404 Not Found in 1ms (Views: 0.3ms | ActiveRecord: 0.0ms)

What was the configuration change you made to devise_token_auth to get this working?
Thanks!

I'm also using the correct passwordCurrent value in the above example, although ideally I wouldn't need one at all

I got it working through some hacks.
In devise_token_auth I made an override for PasswordsController:
module Overrides
  class PasswordsController < DeviseTokenAuth::PasswordsController

    def edit
      @resource = resource_class.reset_password_by_token({
        reset_password_token: resource_params[:reset_password_token]
      })

      if @resource && @resource.id
        client_id  = SecureRandom.urlsafe_base64(nil, false)
        token      = SecureRandom.urlsafe_base64(nil, false)
        token_hash = BCrypt::Password.create(token)
        expiry     = (Time.now + DeviseTokenAuth.token_lifespan).to_i

        @resource.tokens[client_id] = {
          token:  token_hash,
          expiry: expiry
        }

        # ensure that user is confirmed
        @resource.skip_confirmation! if @resource.devise_modules.include?(:confirmable) &&<EMAIL_ADDRESS>
        # allow user to change password once without current_password
        @resource.allow_password_change = true;

        @resource.save!
        yield @resource if block_given?

        render json: {
            token:          token,
            client_id:      client_id,
            expiry:         expiry,
            uid:            @resource.uid,
          }
      else
        render_edit_error
      end
    end

  end
end

In my frontend, I have a component where users can reset their password and added this in the ngOnInit:
ngOnInit() {
  this._route.queryParams.subscribe((params: Params) => {
    let token = params['reset_password_token'];
    let options = { search: { reset_password_token: token, redirect_url: 'auth/password' } };
    this._tokenService.get('auth/password/edit', options)
                      .subscribe(
                        res => this.headerInfo = res.json(),
                        err => {
                          this._alertService.error('Password could not be reset with this token');
                          this._router.navigate(['/users/signin']);
                        }
                      );
  });
}

Then I use the "headerInfo" to submit the new password-reset when submitting the form:
    public resetPassword = (headerInfo: any, params: any): Observable<Response> => {
      let headers = new Headers();
      headers.append('Content-Type', 'application/json');
      headers.append('Accept', 'application/json;q=0.9,*/*;q=0.8'); // Fix for Firefox
      headers.append('access-token', headerInfo['token']);
      headers.append('client', headerInfo['client_id']);
      headers.append('expiry', headerInfo['expiry']);
      headers.append('uid', headerInfo['uid']);
      headers.append('token-type', 'Bearer');

      return this._http.put(this._baseUrl + '/auth/password', params, { headers: headers }).map(res => res.json());
    }

    public submitForm() {
          let params = {
              password:                 this.resetPasswordForm.value.password,
              password_confirmation:    this.resetPasswordForm.value.password_confirmation
          };

          this.resetPassword(this.headerInfo, params);
    }

So what's happening is that before the user can enter anything, I do a call to the backend to log the user in based on the reset token. Then I store that information and use that header-info to submit the real password change.
(these are really rough copy/pastes, so please excuse any errors)

Thanks @cybey, I ended up using another somewhat hacky approach and just sidestepped devise altogether.  Basically, I generate a link with a password reset token and just search my users by that token when the user comes back, update the user with the password/password_confirmation provided by the client, return a 200 then login from the client using the newly set password and _tokenService.signIn().  Not the best but it seems to be the simplest for my purposes.
Thanks for the help!

I'm getting this same issue and I'm wondering if it's a problem with a devise version possibly? https://github.com/plataformatec/devise/issues/4446#issuecomment-284602642

Ah, no i figured this out just now. Here's the deal. Check out the line of code here: https://github.com/neroniaky/angular2-token/blob/fa6aa092dc1b54b1d7940e2cc84042575f94acad/src/angular2-token.service.ts#L270
So it's actually checking for null. Here's the problem with the reset password flow (something I may submit an issue for): the interface for updating a password has current password as a required attribute.
If you create a hidden attribute of "passwordCurrent" and set it to empty in your form:
<input ngModel type="hidden" name="passwordCurrent" value="" />

And then in your service reset passwordCurrent to null:
updatePassword(resetData: ResetForm) {
    resetData.passwordCurrent = null;
    return this.tokenService.updatePassword(resetData);
  }

Then in the payload that gets sent, the current password is not set/sent with the new password and the token. Success!

Thanks for that Ben. The problem is I'm using UI Rotuer 2 and it doesn't easily have a mechanism for accessing a query before the hash. I ended up using url-parser (https://github.com/unshiftio/url-parse) like this:
import { Component, OnInit } from '@angular/core';
import {Angular2TokenService} from "angular2-token";
import {MatSnackBar} from "@angular/material";
import {StateService, Transition} from '@uirouter/core';
import * as urlParse from 'url-parse';

@Component({
  selector: 'app-reset-password',
  templateUrl: './reset-password.component.html',
  styleUrls: ['./reset-password.component.css']})
export class ResetPasswordComponent implements OnInit {
  credentials = {
    password: '',
    passwordConfirmation: '',
    resetPasswordToken: ''
  };
  constructor(
    private tokenService: Angular2TokenService,
    private snackBar: MatSnackBar,
    private stateService: StateService,
  ) {
    console.log("In Reset Password constructor")

  }

  ngOnInit() {
  }
  resetPassword() {
    const url = new urlParse(window.location.href, true);
    console.log("In Reset Password resetPassword, url etc", url.query, url.query.token);
    if(url.query && url.query.reset_password && url.query.token){
      this.credentials.resetPasswordToken = url.query.token
    }
    console.log("requestPasswordReset, Credentials  : ", this.credentials);

    this.tokenService.updatePassword(this.credentials).subscribe(
      res => {
        console.log(res);
        let snackBarRef = this.snackBar.open(
          'Instructions for resetting your password have been emailed to you. Please follow the link in the email message to change your password.',
          'OK')

        snackBarRef.onAction().subscribe(() => {
          this.stateService.go('signIn');
          return;
        });
      },
      error => {
        console.log("In Request Password, error: ", error);
        let snackBarRef = this.snackBar.open(
          'There was a problem requesting a password reset - are you sure your email address is correct?.',
          'OK')

        snackBarRef.onAction().subscribe(() => {
          this.snackBar.dismiss();
          return;
        });
      }
    );
  }

}

Note that the true in  new urlParse(window.location.href, true) is vital as it says to parse the query string into an object, so you can do url.query.token. Without that flag it just has a big query string with all query elements together.

OK, turns out that isn't enough. as cbey discovered earlier in the thread, update password needs the auth headers in the request, which is why they are included in the redirected url from devise token auth. I am using my own fork of this lib that uses HTTPClient and interceptors, so I'll probably just modify the code there to extract the headers from the query strings and send the password update with the new headers.
As it stands, this library has no mechanism for dealing with a reset password where the user isn't currently logged in - aka forgot password flow. Including the reset token as a property in the update password call is pointless as that call needs the auth headers if it is to work and currently the only time it has the auth headers is when the user is already logged and it has them from the normal back and forth. What is interesting is that in the only working scenario, user logged in, the reset password token isn't actually used at all, you can delete it from the url and the update still works, as it is using the actual auth header token from the live user session. It isn't totally clear to me that the reset token property would ever be actually used as the real way to do a forgot password reset to use the reset token to set the token header property, not to include it in the payload. I checked this with a working project that uses ng token auth (aka the client lib for AngularJS) and it doesn't include the reset token in the payload, instead it copies the token into the usual token header property and that approach works for a forgot password flow. Note also that ng token auth lib figures all this out itself - it inspects the url, extracts the data from the queries and sets up the header for the update password request. This is the approach this lib should take.  As it stands, the only way to get this working within the confines of the lib itself is to use the .request() method to set up a manually created call to update password with manually created headers that you extract yourself from the queries in the redirected url. This is similar to what cbey did although he went the whole hog and just did a direct http api call.
I'll post a doco PR to at least try to clarify some of this in the readme.

I've made the changes in my fork and the forgot password flow is now working. Briefly, I changed getAuthDataFromParams in tryLoadAuthData to use url-parse instead of relying on activatedRoute to parse the url. I then called tryLoadAuthData inside the update password call.
Based on that, it looks like just calling tryLoadAuthData from within update password might work for the master branch.
For the record, my getAuthDataFromParams now looks like this:
    // Try to get auth data from url parameters.
    private getAuthDataFromParams(): void {
        const url = new urlParse(window.location.href, true);
        if(url && url.query) {
            let authData: AuthData = {
                accessToken: url.query['token'] || url.query['auth_token'],
                client: url.query['client_id'],
                expiry: url.query['expiry'],
                tokenType: 'Bearer',
                uid: url.query['uid']
            };

            if (this.checkAuthData(authData)){
                this.atCurrentAuthData = authData;

            }
        }
    }

I also obviously call this directly, I am not checking for activatedRoute :
    // Try to load auth data
    private tryLoadAuthData(): void {

        let userType = this.getUserTypeByName(localStorage.getItem('userType'));

        if (userType)
            this.atCurrentUserType = userType;

        this.getAuthDataFromStorage();

        this.getAuthDataFromParams();

        if (this.atCurrentAuthData)
            this.validateToken();
    }

Finally, my update password :
    // Update password request
    updatePassword(updatePasswordData: UpdatePasswordData): Observable<any> {

        if (updatePasswordData.userType != null)
            this.atCurrentUserType = this.getUserTypeByName(updatePasswordData.userType);

        let args: any;

        if (updatePasswordData.passwordCurrent == null) {
            args = {
                password: updatePasswordData.password,
                password_confirmation: updatePasswordData.passwordConfirmation
            }
        } else {
            args = {
                current_password: updatePasswordData.passwordCurrent,
                password: updatePasswordData.password,
                password_confirmation: updatePasswordData.passwordConfirmation
            };
        }
        // Redo the header load in case this is a forgot password scenario and we need to get the headers from the
        // redirected URL
        this.tryLoadAuthData();

        let body = JSON.stringify(args);

        return this.request('PUT', this.getUserPath() + this.atOptions.updatePasswordPath, body);
    }

will try your approach !
https://github.com/neroniaky/angular-token/issues/469

@colmben Can you please explain me how do you "intercept" on the frontend the client_id, expiry, token and uid??
This is the big and only mystery for me because I can only intercept the "token param" that is sent from there, nothing else

I think I found a reasonably elegant solution to work with rails, working on some of the code in this thread.Effectively, this works around the issue that you cannot use the 'update password' method in angular-token library since there is no token set when you are not logged in and a token is required by devise_token_auth to access the reset password link . What I have done is effectively created a 'magic link' to auto sign in the user upon opening the email redirect url by overriding the passwords controller edit function. This fetches the user based on the url, , generates new tokens and returns the tokens generated to the front end so that it can be used in the ui:

Update the passwords_controller in rails:
`  class OverridePasswordsController < DeviseTokenAuth::PasswordsController

def edit
  @resource = resource_class.reset_password_by_token({
    reset_password_token: resource_params[:reset_password_token]
  })

  if @resource && @resource.id
    client_id  = SecureRandom.urlsafe_base64(nil, false)
    token      = SecureRandom.urlsafe_base64(nil, false)
    token_hash = BCrypt::Password.create(token)
    expiry     = (Time.now + DeviseTokenAuth.token_lifespan).to_i

    @resource.tokens[client_id] = {
      token:  token_hash,
      expiry: expiry
    }

    # ensure that user is confirmed
    @resource.skip_confirmation! if @resource.devise_modules.include?(:confirmable) &&<EMAIL_ADDRESS>
    @resource.save!
    puts @resource.tokens[client_id]
    yield @resource if block_given?

    render json: {
        token:          token,
        client_id:      client_id,
        expiry:         expiry,
        uid:            @resource.uid,
      }
  else
    render_edit_error
  end
end

end`
Now, once the tokens are in the front end, you can now set them to the localstorage in your browser, which effectively logs the user in and then, use the updatePassword method within the angular token library to update the password. Ta -daaa.

Update the component where I am setting the password (post reset password link)

`ngOnInit() {
this.activatedRoute.queryParams.map((params: Params) => {
this.reset_password_token = params['reset_password_token'];
return this.reset_password_token
).switchMap(token => {
let httpHeaders = new HttpHeaders({
'Content-Type' : 'application/json'
});
return this.http.get(this._tokenService.tokenOptions.apiBase + '/auth/password/edit', {headers:httpHeaders, params: {reset_password_token: this.reset_password_token, redirect_url: '/auth/password'}})
}).subscribe(
res => {
console.log(res);
localStorage.setItem('accessToken', res['token']);
localStorage.setItem('client', res['client_id']);
localStorage.setItem('expiry', res['expiry']);
localStorage.setItem('uid', res['uid']);
localStorage.setItem('tokenType', 'Bearer');
}, error => {
console.log(error);
}
)
}

public submitSetPasswordForm($ev, value: any) {
      this._tokenService.updatePassword({
			'password': value.passwordGroup.password,
            'passwordConfirmation': value.passwordGroup.confirmPassword
      }).subscribe(
      	res => {
      		console.log(res);
            this.router.navigate(['/home']);
           },
      	err=> console.log(err));
}`

I like this.
Maybe these snippets should be added to the wiki

I'm in the process of looking at this very issue. I don't think it needs that magic on the server side. The current process effectively logs in the user when they click on the email link (you can test this by pasting in a protected URL at that point). The password reset page response from the server after clicking the email link includes a new valid access code. The problem is that the actual call to edit the password doesn't have this code in it (instead it repeats the reset token, which will actually work if you set your "burst" time setting on the server to something long enough to allow you to get the new password in before it expires). So it's a one line fix I think. I'm currently fighting with the structure of this lib trying to set up a test of that one line, but I think I have figured that out, so should have a pr by tomorrow (big sports game in Dublin today! )

Sooo, I have an answer, at least for the standard Angular default way of doing local client routing. First off - the library as it stands does actually work for Forgot Password flow! There are a couple of things to consider:
Angular can do local URL routing 2 ways - either the old way it did by putting a hash symbol in the URL that indicated anything after that was for Angular or the new (now default) way of using some modern browser magic to prevent local routes getting sent out to the server. These are known as HashLocationStrategy and PathLocationStrategy respectively. PathLocationStrategy is the default for new Angular apps. It is great not to have the strange hashes in the URL any more but it has one rather significant side effect - on a refresh or a URL paste the full URL will pass through the server. Therefore you need to set up the server to automatically pass any unknown server routes straight through to index.html, which is the start of the Angular app (which then sends into Angular router etc.). For most web server this can be done using a very low level setting to catch what would normally be a 404 response (via .htaccess for example on an Apache web server). For a rails server, the situation can be a little trickier depending on your set up. My initial way of doing was to just put a catch all route that rendered to public/index.html (which is where my Heroku hosted app builds the angular index.html, yours might be in dist/index.html etc.). I did this by putting a match everything in routes.rb:
match "*path", to: 'boot#angular' , via: :all
and setting that action to render the file :
class BootController < ApplicationController
  def angular
    puts "in Angular render action"
    render file: 'public/index.html', :layout => false
  end
end

and that actually solves all the refresh issues with a PathLocationStrategy Angular app. But it introduced a subtle devise auth token problem. The match all -> index.html response was getting a token added to it! This didn't matter for refreshes (for reasons I haven't delved into) but it is fatal for the password redirect from the forgotten password email link. That redirect is issued by the server, and it then then flows through the server into the match all and on to index.html to get to the resetPassword Angular controller, which sets up the new password screen. But now the server has moved on one token and because the redirect request wasn't issued by Angular, it can't get at the response token (or at least I couldn't figure out how it can access it). After trying a number of different ways around this, I realised that all we have to do is stop the server putting a token on that redirect (and indeed on any non-matched route). My problem was that the controller I was using to do the render to index.html was inheriting from ApplicationController, which has the devise token auth Concerns line in it, which the bit that does all the token adding. So all I had to do was step back a level and instead inherit from ActionController.
class BootController < ActionController
  def angular
    puts "in Angular render action"
    render file: 'public/index.html', :layout => false
  end
end

And that worked! No token added to the redirect, so the resetPassword updatePassword call gets given the reset token as the access token by the lib and the server recognises that as the last valid token for the user and the password change works.
So actually no need for a lib change if you are using PathLocationStrategy.
For HashLocationStrategy, I figured it would actually work straight off as we don't have the issue of the redirect going through the server, but for some reason the redirect is eating the query string and so your resetPassword controller can't get at the resetToken to manually change the access_token. I didn't look to hard at this process as I want to use PathLocationStrategy.
I'll submit a PR tomorrow to take out the old doco saying none of this works (which I actually wrote, but for the old pre-interceptor version) and add in some notes about using PathLocationStrategy.
 Confusing when you are searching.

Just a quick update, I figured out why HashLocationStrategy apps wouldn't work, the Devise Token Auth lib made a change a few years ago to put the query params before the hash fragment instead of afterwards. This causes current version of Angular to eat those query params so they don't show in the final redirected url nor are they catchable via activatedRoute.queryParams. I raised a PR on that lib to just dupe the query params when we have a fragment so they are sent both before and after the fragment. This fixes the password reset flow for HashLocationStrategy apps and leaves PathLocationStrategy apps working ok.


Replaced some duplicated strings with constant strings

Also used introspection to simplify the population of the STRATEGIES dict.
(This change depends on changes to goldilocks.py & util.py)

Coverage decreased (-0.2%) to 69.279% when pulling dc148b875a111c86753cd47df68362e6ddfd6d83 on mark-summerfield:patch-1 into ea333a739baf71e6d2e3a9a2b5afde9e14cf9eb1 on SamStudio8:master.

Current coverage is 61.75% (diff: 0.00%)

Merging #14 into master will decrease coverage by 0.19%

@@             master        #14   diff @@
==========================================
  Files             5          5
  Lines           636        638     +2
  Methods           0          0
  Messages          0          0
  Branches        177        178     +1
==========================================
  Hits            394        394
- Misses          194        196     +2
  Partials         48         48

Powered by Codecov. Last update ea333a7...dc148b8


[health priority] Replace Potto Loris & Calisphere beanstalks with Fargate

See also older Trello card "Cloudfront Origin Encryption and Beanstalk Managed Updates" : https://trello.com/c/62Fi4LqE

Should we break this down into two separate tickets?


admin pages filter for sites in a multisite settings

Rationale / Begründung
An admin wants to filter multiple admin pages for elements relating to a single site in a multisite setting.
Affected
Admins
Minimal functionality
Admin filter on User accounts for sites
Nice-to-have functionality
apply this filter in one click to the other pages (such as Roles) as well
References / Verweise

https://github.com/rdmorganiser/rdmo/issues/422

It is implemented for Roles and User and Roles have a OneToOneField relation so this is resolved.


Reproducible Deterministic Builds

#96 is probably a prerequisite.
We'd like people is to know that the code in google-play is the same as in the code in the repo. See #74 .

Sadly docker isn't meant for deterministic builds. So my PR is more of a "let the user compile for themselves" and less into proving that the code in google-play is the same as in the repo.
for that you'll need a more robust system probably, (I know how to do it in C/C++ not in Java/Android especially when react-native(JS) is involved)