(feat) bitfinex listen for subscriptions

Fixes #6590 and maybe #5919.
Before submitting this PR, please make sure:

[x] Your code builds clean without any errors or warnings
[x] You are using an approved title ("feat/", "fix/", "docs/", "refactor/")

A description of the changes proposed in the pull request:
To make listen_for_subscriptions work for Bitfinex, this pull request implements the following methods:

[x] _connected_websocket_assistant
[x] _subscribe_channels

Additionally:

[x] new tests for these new methods

Tests performed by the developer:

[x] Successful local compilation.
[ ] Successful local test runs.

Tips for QA testing:
None.
Request for help:
Please verify _connected_websocket_assistant to help me out. It's only four lines of code, but it will clear my doubts.

@nikspz Please verify _connected_websocket_assitant to help me out. It's only four lines of code, but it will clear my doubts.

@0xferit Could you check failing tests here?
https://github.com/hummingbot/hummingbot/actions/runs/7332771296/job/19973633881?pr=6738

I checked the test already and spent two hours on it but I couldn't find the problem. Also, I wrote that test myself, so it does not really help to check against my own test.
Can you please review a piece of code and let me know what I am doing wrong? Again, I don't need a full review but only a review of the implementation of _connected_websocket_assistant, which is only four lines of code and should take five minutes.

@0xferit Could you please resolve branch conflicts?

@0xferit Could you please resolve branch conflicts?

Sure, doing now.

Done @nikspz

Happy New Year, @nikspz.
Here is a friendly reminder to make sure you remember this pull.

Happy New Year, @nikspz.
Here is a friendly reminder to make sure you remember this pull.

please run this command to format the code, and then commit.
“isort hummingbot/hummingbot/connector/exchange/bitfinex/bitfinex_api_order_book_data_source.py”

"ws_connection: websockets.WebSocketClientProtocol = await websockets.connect(ws_url=BITFINEX_WS_URI,"
There is one less await in line 499.

@yancong001 @0xferit I think that yancong is right. Tag me when it's done ;) I thought that we were using aiohttp instead of websockets library

@yancong001 @0xferit I think that yancong is right. Tag me when it's done ;) I thought that we were using aiohttp instead of websockets library

Added missing await.
About aiohttp, I don't know really, you should tell me what's wrong here.

@nikspz

@yancong001 @nikspz

hi, @0xferit .
I think the easiest defense to make bitfinex work is to add the following two lines inside "bitfinex_api_order_book_data_source.py", preferably in line 86,87. Nothing else needs to be changed
    async def listen_for_subscriptions(self):         pass

hi, @0xferit . I think the easiest defense to make bitfinex work is to add the following two lines inside "bitfinex_api_order_book_data_source.py", preferably in line 86,87. Nothing else needs to be changed
   async def listen_for_subscriptions(self):
        pass

This will only make error messages about subscriptions disappear, but it will not make listening for subscriptions functionality work. Am I wrong? And if I'm right, are you sure that is the objective here?

My approach makes the connector work properly

My approach makes the connector work properly

Care to elaborate?

Run the pmm strategy smoothly with bitfinex connector

@yancong001 I am testing a fork having that change. Will share my results.

I think you might be right @yancong001 .
By eliminating listen_for_subscriptions, you essentially eliminate error log spam. I tried it, and it didn't work. But then, rate oracle errors were spamming as well. Then, I eliminated rate oracle errors, and it worked (Running for 40 hours so far with no problems, usually breaking within the first 20 hours). I strongly suspect that 100% of CPU problems are caused by error log spam, and we should investigate logs instead.
@nikspz FYI.

Closing this in prior to https://github.com/hummingbot/hummingbot/pull/6841


[Android] Crash on requestPermission

Description of the problem
Hello, i am experiencing a new crash inside my application on android when requesting permissions through notifee
The sentry stacktrace shows a crash related to the notifee requestPermission callback:

i noticed that the activity itself has a null check here:

but my guess is that the this context inside the code snippet below may be null too. I don't know why exacly this could be possible but that is my only possible explanation why there could be a null pointer/ref crash. It may be related to shifting the app from background state to foreground state so the this context may get destroyed.
activity.requestPermissions(
        new String[] {Manifest.permission.POST_NOTIFICATIONS},
        Notifee.REQUEST_CODE_NOTIFICATION_PERMISSION,
        this); //this may be null

Reproduction steps
It was not possible to reproduce the crash it is just showing up on several devices from time to time in sentry.
Version
Notifee 7.3.0 (related requestPermission code didn't change from this version to latest notifee release)
Android 13 + 14

may be related to https://github.com/zoontek/react-native-permissions/issues/820 , they implemented a silent fail to fix it cause it seems to be a bug inside android itself

I am experiencing this on a react-native app. notifee/react-native version 7.8.2. This started happening when the compileSdkVersion was set to 34 (with Google's recent requirements)


Add ability to preview an order

This adds a new endpoint v3/order/<order-id>/quote/preview which acts exactly like the create but without actually saving the changes.
The endpoint is used to preview the content before creating a quote.

Codecov Report

Merging #438 into develop will increase coverage by 0.01%.
The diff coverage is 100%.

@@             Coverage Diff             @@
##           develop     #438      +/-   ##
===========================================
+ Coverage     95.9%   95.91%   +0.01%
===========================================
  Files           94       94
  Lines         3074     3084      +10
  Branches       272      272
===========================================
+ Hits          2948     2958      +10
- Misses          85       86       +1
+ Partials        41       40       -1

Impacted Files
Coverage Δ

datahub/omis/quote/urls.py
100% <ø> (ø)
:arrow_up:

datahub/omis/quote/serializers.py
100% <100%> (ø)
:arrow_up:

datahub/omis/quote/views.py
100% <100%> (ø)
:arrow_up:

datahub/company/admin.py
89.47% <0%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e44d6e6...57d011f. Read the comment docs.


fcm-notifications: tokens

In the fcm-notifications sample, line 49, are tokens unique device ids or user ids? How can we obtain them in the first place? Thanks
@nicolasgarnier

@ShuaiW tokens are unique for device ids, which, for browsers means specific to a certain browser instance and a specific domain.
To get the token on a web app you simply call  firebase.messaging().getToken() (after getting authorization from the browser to display notifications). This is shown on the sample here: https://github.com/firebase/functions-samples/blob/master/fcm-notifications/public/main.js#L186

Thanks @nicolasgarnier for your reply. I am working on an Ionic app, which is JavaScript but targeted at mobile (both iOS and Android). I was trying firebase.messaging().getToken() (with authorization) but in vain. Any chance you know how to handle it in my situation or do I miss something? Thanks in advance.

I @ShuaiW, I recommend you to use @ionic-native/firebase
Here's some code of how I get the token and getting notifications:
import { HttpClient } from '@angular/common/http';
import { Injectable } from '@angular/core';
import { Firebase } from '@ionic-native/firebase';
import { Platform } from 'ionic-angular';
@Injectable()
export class FcmServiceProvider {
constructor(public platform: Platform, public http: HttpClient, public firebaseNative: Firebase) {
console.log('Hello FcmServiceProvider Provider');
}
async getToken() {
let token = null;
if (this.platform.is('android')) {
token = await this.firebaseNative.getToken()
}
if (this.platform.is('ios')) {
token = await this.firebaseNative.getToken();
await this.firebaseNative.grantPermission();
}
return token;
}
listenToNotifications() {
return this.firebaseNative.onNotificationOpen()
}
}
In this piece of code you have two methods, one for getting the token and the another to say what to do if a notification is comming.
Tokens as @nicolasgarnier said are unique, but if you force your device to get token every time you "logIn" you'll get a different token every time, and you have to store it by yourself if you want to use in cases like a chat with firebase functions. Hope to help you.


Parallel segmentation

Parallelizes image segmentation, uses image mask (to prevent wasted cycles, available in scikit-image 0.17.+), adds documentation, and safeguards segmentation (for line chunks where there may be no data).  Also add typing safeguards.
Caution: for very large images (many columns), default use through apply_oe.py may cause memory errors.  Future adjustments could query the available memory and adjust, or just use a reduced number of cores.
@davidraythompson - ready for your review.

Excellent, thank you.


Provide instructions for the .env file in the readme

** I struggled a bit to get my bot working because of the .env file requirements, so I'd like to help noobs like me**

Instruction on where to create the .env file
Example of what should be in the .env file (faking the token obv.)

Waarom heb ik dit in het Engels geschreven ? Nobody knows...
Anyway, ik zou het ook graag zelf doen om de cyclus van branching, pull request, enz.. es door te lopen.

We prefer english while developing, we agree with your original language choice :D

It seems that you fixed this, so I think it's safe to say to close this? If not, you can re-open.


Special rendering of Silver Line Waterfront trips & route page

Probably best reviewed one commit at a time.
Summary of changes
Asana Ticket: 🐞Schedule Finder | Silver Line variants are displayed incorrectly
Per the ticket:

Silver Line buses like 746 (Silver Line Waterfront) can be seen inside other SL schedules, and
found via google (they are not usually directly linked on MBTA.com).
When viewing schedule pages and schedule finder results for these vehicles we see poorly styled > results.

Here we improve the rendering of SL Waterfront results in the places specified in the ticket, as well as a few other places that seemed logical (e.g. the variant selector in the direction picker).

@phildarnowsky Should we close this? It seems like most of the relevant work has been offloaded to the API and #394.


[CosmosDB] Improved help for failover-priority-change command

Precisely described the validation rules applied by Cosmos DB Resource Provider when it processes this request.

CosmosDB


Add nolock

Adds with (nolock) to list_relations_without_caching macro. Fixes deadlock issues when running dbt tests in issue #195. Also see Slack thread here: https://getdbt.slack.com/archives/CMRMDDQ9W/p1640064902122500
Outstanding questions:
Are we sure adding nolock is acceptable here? My understanding is that each dbt test is independent and does not use the same database objects, which should make this fine. There could be something I'm missing though.

I am torn on this one.      I am a user of dbt and SQL Server, not a dbt core or adapter developer.
The PR fixes the issue.  No deadlocks when doing dbt test with 4 treads and ~150 tests, which all run in 7 seconds.   Without this fix it would cause multiple deadlocks.
Like you, I'm concerned about introducing nolock to the project. It's usually better to understand if/how the application (in this case dbt) is causing the deadlocks. and try to prevent them there.   It could be harder in this case because SQL Server might have its own quirks about concurrency of DDL operations and reads on information_schema.
A agree that use of nolock in this particular macro (sqlserver__list_relations_without_caching) is probably safe to use in dbt test, because the tests are independent of each other.   It is probably also safe in  dbt run  and dbt build (which also call that macro) because they are constrained by dbt to run in DAG order.  They are unlikely to be sensitive to uncommitted rows/deletes from another transaction being returned by READ_UNCOMMITTED, which is the behaviour invoked by nolock.
Are their other callers of that macro?
Do we need to consider MS db platform compatibility? nolock is supported  on SQL Server and Azure SQL Database.  Does dbt-sqlserver target any other MS data platforms?

Maybe this should be optional? Perhaps set in profiles.yml.

Good  idea.
And/or only use nowait when threads > 1
Either way gives users/developers a heads up that 'nowait' is used as a
concession for multithreading.  As profile/cli doco, or simply the presence
of 'if' condition in the macro.
On Fri, 28 Jan 2022, 6:32 pm Mikael Ene, @.***> wrote:

Maybe this should be optional? Perhaps set in profiles.yml.
—
Reply to this email directly, view it on GitHub
https://github.com/dbt-msft/dbt-sqlserver/pull/204#issuecomment-1023956442,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ANX27CR3FTXRJCRQXOWTWMTUYJA73ANCNFSM5M6VYAZQ
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you commented.Message ID:
@.***>

Needs rebase on latest master

Hi folks,

Like you, I'm concerned about introducing nolock to the project. It's usually better to understand if/how the application (in this case dbt) is causing the deadlocks. and try to prevent them there. It could be harder in this case because SQL Server might have its own quirks about concurrency of DDL operations and reads on information_schema.

Not sure, I am of much help, but the locks seem to always be on
database_name.sys.sysschobjs
and involves more or less
select
   table_catalog as [database],
   table_name as [name],
   table_schema as [schema],
   case when table_type = 'BASE TABLE' then 'table'
when table_type = 'VIEW' then 'view'
else table_type
   end as table_type
  from [database_name].information_schema.tables
 where table_schema like 'dbt_test__audit'

and things like
USE [database_name];   if object_id ('"dbt_test__audit"."unique_table_name1_key_temp_view"','V') is not null
   begin
   drop view "dbt_test__audit"."unique_table_name1_key_temp_view"
   end

... long story short, I also cannot come up with a better solution than nolock on the list_relations_without_caching - (even though I don't know why the upper statement can not simply what that the lower one is finished. But it's getting late).
Anyways, maybe this PR could proceed?
Cheers
Hannes

@mikaelene @sdebruyn @dataders - any updates/suggestions on this PR?

I've not been super involved in the development of this adapter lately. But when running tests, you usually don't update any data or definitions so for that reason I think it's perfectly fine to use "with no lock". I don't know how people use this adapter these days but my aim when creating it was to implement cloudDW-like behaviour on SQL server and the other clouds doesn't use lock what I know of. In an OLTP-setting it would be a bad idea, but as for DW and querying data for tests I guess it would be ok. Looks like the CI build is failing. If someone fix that I think we can merge.

I managed to fix some other deadlocks as well in #368 but I definitely think we need this one as well


unique user ids

I appreciate the effort to help solve f capping. I'm unsure on a few pts though. In the goals section you mention:
We would like to ensure that critical frequency capping use cases can be supported via RTB without relying on the presence of user identifiers in bid requests
And in the assumptions section you mention:
the exchange should be able to identify when an additional impression would violate the limits specified in the caps for a given user
then again in the assumptions section
The experiment assumes that exchanges have the means of identifying the same user across different sites or mobile apps in respective environments – for instance, by using a pseudonymous identifier based on a third-party cookie or a mobile device advertising identifier.
This mention of a cross domain user id, is that something that could be shared from the publishers to the exchange but not to DSPs?  Is that the idea?
It seems like cross domain user ids in the ad request are ok, but not in the bid request?

Thank you for your interest in the frequency capping proposal.
The proposal explores whether and how frequency capping functionality could be supported in RTB without the reliance on identifiers shared in bid requests. The suggested experimental implementation is only intended to validate the approach of specifying frequency limits in the bid response; the early experiment would indeed rely on some form of a stable identifier, but that doesn’t have to be the case in a practical, real-world implementation.
For example, the same API principle with frequency limits specified in the bid response could also be backed by an on-device frequency capping implementation; TURTLEDOVE proposal briefly mentions that as a possibility. Alternatively, exchange-assisted frequency capping could rely on identifiers scoped to a single site or app, with modelling cross-domain limits based on aggregated reporting information.
It’s not the intent of this proposal to suggest how an exchange could implement  frequency capping support, or to suggest what types of identifiers should be available to which parties in programmatic advertising. Rather, we suggest a new way of supporting frequency management in real-time bidding API that does not rely on user identifier sharing, and thus may improve user privacy protections. Our goal is to get feedback from the RTB community.


HDDS-7829. SCM to reject adding container to closed pipeline

What changes were proposed in this pull request?
This is a correction of the fix for "SCM terminates when adding containers to a closed pipeline" in HDDS-7738.
In general, due to the lack of atomicity and the fact that SCM status can change between transactions, SCM should just reject transactions gracefully if some logical condition is not met.
For example, when processing a transaction adding a container, SCM expects the associated pipeline to be in an OPEN state and rejects the container creation if the pipeline is CLOSED.
What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-7829
How was this patch tested?
Unit test.
I could not find a definite way to create an integration test for this situation, because it involves race conditions, i.e. pipeline is closed right after being allocated to a container. Yet, I manually injected errors and verified that SCM works normally after rejecting the failed transaction.

@swamirishi can you please take a look?

Following up on the testing part of this patch, the unit test is good, but it does not test the changes to SCMStateMachine#applyTransaction. It looks like this could be done by modifying TestPipelineClose#testPipelineCloseWithClosedContainer to use 1 node Ratis for SCM, then calling pipelineManager#addContainerToPipeline on the closed pipeline and check that an exception is thrown.
I don't see a way to test that generic IOExceptions or SCMExceptions with fatal result codes terminate the SCM without failing the test due to JVM exit.

Following up on the testing part of this patch, the unit test is good, but it does not test the changes to SCMStateMachine#applyTransaction. It looks like this could be done by modifying TestPipelineClose#testPipelineCloseWithClosedContainer to use 1 node Ratis for SCM, then calling pipelineManager#addContainerToPipeline on the closed pipeline and check that an exception is thrown.

@errose28, thanks to your suggestion, I was able to create a proper integration test for the failure scenario.

I don't see a way to test that generic IOExceptions or SCMExceptions with fatal result codes terminate the SCM without failing the test due to JVM exit.

I believe that can be resorted by ExitUtils.disableSystemExit, which prevents actual JVM exit for testing purposes.
Yet, I scanned the code and could not find a good way to produce internal errors, maybe we can somehow generate TimeoutException and SCM always translates it to internal errors.


🌿 introduce realtime transcriber

This PR introduces a Realtime Transriber.

Not crucial right now, but we should support auth via token too.


fix(api): updated return value code with value type

PR Checklist

[x] Unit Tests have been added for new changes
[x] API tests have been updated if applicable
[x] All commented code has been removed
[ ] If you've added a dependency, you've ensured license is compatible with Apache 2.0 and clearly outlined the added dependency.

What are you changing?

Anything the reviewer should know when reviewing this PR?
If the there are associated PRs in other repositories, please link them here (i.e. open-amt-cloud-toolkit/repo#365 )

Codecov Report

Merging #476 (3aa8cd9) into main (019895e) will increase coverage by 0.21%.
The diff coverage is 88.57%.

@@            Coverage Diff             @@
##             main     #476      +/-   ##
==========================================
+ Coverage   68.90%   69.12%   +0.21%
==========================================
  Files          49       49
  Lines        2206     2241      +35
  Branches      346      354       +8
==========================================
+ Hits         1520     1549      +29
- Misses        686      692       +6

Impacted Files
Coverage Δ

src/routes/amt/eventLog.ts
97.29% <50.00%> (-1.32%)
:arrow_down:

src/routes/amt/userConsent/request.ts
87.50% <70.00%> (-12.50%)
:arrow_down:

src/routes/amt/userConsent/send.ts
88.46% <72.72%> (-11.54%)
:arrow_down:

src/amt/ConnectedDevice.ts
61.17% <93.75%> (-0.94%)
:arrow_down:

src/routes/amt/getVersion.ts
100.00% <100.00%> (ø)

src/routes/amt/powerCapabilities.ts
100.00% <100.00%> (ø)

src/routes/amt/userConsent/cancel.ts
100.00% <100.00%> (ø)

src/test/helper/wsmanResponses.ts
100.00% <100.00%> (ø)

src/amt/CIRAHandler.ts
20.18% <0.00%> (+1.83%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 019895e...3aa8cd9. Read the comment docs.


S3 Example: Retrieving the response data.

Anyway to get the s3 data response from the s3 example? Thanks.

mistake


[Incorrect fix?] tpl/data: Prevent getJSON and getCSV fetch failure from aborting build

This PR supposedly fixes #5643, but how that this PR is pushed, I become convinced that this is not the right fix because the context of the error is lost:
Before (0.53):

Error: Error building site: failed to render pages: render of "home" failed: "/home/foka/web/example.ca/layouts/index.html:24:13": execute of template failed: template: index.html:24:13: executing "main" at <getJSON "https://api...>: error calling getJSON: failed to get getJSON resource "https://api.example.com/test?q=abc": Failed to retrieve remote file: Not Found

After (with this PR):

Building sites … ERROR 2019/01/28 17:05:11 failed to get getJSON resource "https://api.example.com/cards/test?q=abc": Failed to retrieve remote file: Not Found

So, perhaps this PR is not the right fix?  More discussion at the issue #5643.

Merged.

I suspect this issue was introduced as a side-effect of me trying to improve error messages. I guess this PR reverts that in this particular case, but we can revisit that later. This fix is fine, but you need to adjust the tests to make them pass.
Note that I will probably do a patch release on Monday, so it would be good to get this in before then.

Thank you so much for adjusting the tests for me in ddc6d4e30f282f23b703a3b42da552886062c8c8.  After reading the changes in that commit, I realize that I wouldn't have known how to fix these tests properly (due to me being still a very amateurish Go programmer and my lack of familiarity with much of Hugo's code), so it was wonderful that you just went ahead and fixed it.  And thank you for the new 0.54.0 release!  It is a very pleasant surprise!

@anthonyfok does this allow us to react to the error from our template? If so, how?
I'd like to output a proper result when the GetJSON doesn't return anything.
https://discourse.gohugo.io/t/handling-getjson-errors-in-shortcodes/18597/3
Thanks for any tips


winget source add doesn't warn you when adding an unsupported source.

Brief description of your issue

winget source add doesn't warn you when adding an unsupported source.
Steps to reproduce

Stand up a REST Source that only advertises support for version 1.1.0 of the schema, not 1.0.0.
As an administrator, run winget source add -n <name of repo> <address of repo> --type "Microsoft.Rest"
Notice that it returns zero and acts like everything is hunky-dory.
Try to use winget show/install/list/upgrade/search and realize they don't work because even though your source added successfully, it isn't supported by this version of winget.

Expected behavior
An error when you try to add the unsupported source. (So winget should reach out to the information endpoint and check the ServerSupportedVersions then. Or, better yet, have the winget client be able to handle minor schema changes, as right now it can't talk to 1.1.0 REST sources even if I set the ServerSupportedVersions to 1.0.0 because it doesn't know to ignore the new fields. but that's probably a separate bug).
Actual behavior
PS C:\Users\easton> sudo winget source reset --force
Resetting all sources...Done
PS C:\Users\easton> sudo winget source add -n eastonsrepo https://<IP_ADDRESS>:8000 --type "Microsoft.Rest"
Adding source:
  eastonsrepo -> https://<IP_ADDRESS>:8000
Done
PS C:\Users\easton> winget show --source eastonsrepo --id Microsoft.EdgeBeta --exact
Failed to open the source; try the 'source reset' command if the problem persists.
An unexpected error occurred while executing the command:
0x8a150038 : The configured rest source is not supported
PS C:\Users\easton> winget install Google.Chrome
Failed to open the source; try the 'source reset' command if the problem persists.
An unexpected error occurred while executing the command:
0x8a150038 : The configured rest source is not supported
PS C:\Users\easton> ((iwr https://<IP_ADDRESS>:8000/information/).Content | ConvertFrom-JSON).Data

SourceIdentifier              : A WinGet source!
SourceAgreements              :
ServerSupportedVersions       : {1.1.0}
UnsupportedPackageMatchFields :
RequiredPackageMatchFields    :
UnsupportedQueryParameters    :
RequiredQueryParameters       :

PS C:\Users\easton> # But you said 'adding source.... done' :(

Environment
[winget --info]
Windows Package Manager version 1.0.11694
Windows: Windows.Desktop version 10.0.22000.168
Package: Microsoft.DesktopAppInstaller version 1.15.11694.0

Any other software? I believe Microsoft Solitaire Collection is installed, if that changes anything :D

This should be fixed in PR #1396


Suggestions for the Windows installer

During gVim 8.1 installation on Windows 7 i get command prompt dialogs popping up quickly and closing so its like a flash when the install starts and dialogs that stays in the back of the install/uninstall window.
I would prefer if these dialogs could be avoiding.
Maybe use an argument like --quiet i think some programs use something like that to run commands without having dialogs popping up.
Also during uninstall i get 3 questions about do i want to delete this and that.
I suggest to have a check box in the uninstall window like "keep/save/delete user settings" as seen in other software so the user can make the choce before clicking the uninstall button.
And then when the user clicks the uninstall button the program just gets uninstalled without all kinds dialogs popping up and having to answer questions.

Related: #279, #751, etc.
See also the todo.txt:
https://github.com/vim/vim/blob/f0447e89a52885630947510f2d1b55f665a1a20e/runtime/doc/todo.txt#L203-L214

closing as duplicate of #751


Validate non-existing params

Hi!
First of all, great library! Many thanks! 🙏🏻
I was wondering if there is (or will be) support for validating fields that do not occur in the defparams?
For example, suppose we have the following module:
defmodule Params do
  use Goal

  defparams :body do
    optional :field, :string
  end
end

When I run the following, I get:
iex> Params.validate(:body, %{non_existing_field: "..."})
{:ok, %{}}

I would like to have the option to return an error tuple saying something like :non_existing_field does not exist.
Thanks in advance!

Hi! Thanks for your message. I think it is possible to add such a feature, but my question would be what you want to use it for?
If your servers are exposed to the internet (untrusted traffic) then that feature could pose a security risk, because any external actor can send an unlimited list of parameters; which would accumulate memory (and maybe atoms, depending on the implementation) in the system. It could expose the servers to Denial of Service attacks through OOM and atom exhaustion https://erlef.github.io/security-wg/secure_coding_and_deployment_hardening/atom_exhaustion.html
For internal traffic it could be less harmful, if you know the client will receive only a few unknown parameters

Thank you for your reply! I only needed it for internal use, but your response makes perfect sense. It's better not to include it then.

I too am interested in this feature and I don't really get the issue, you can just choose to not dynamically create atoms for these errors. I understand goal does make atoms for the 'good' params, but it could just return error strings for any unallowed ones


Create personal profile view

Check instagram personal profile view for reference.
Components

Activity Bar Section

user_name with drop down icon.
Plus box with context menu.
Menu icon with sheet on tap.

Dashboard Section

Account Category Name "Dashboard".
"Tools and resources for creators".
Right arrow.

Main Section

Tappable profile image.
No. posts with bottom label.
No. followers with bottom label.
No. following with bottom label.

Name.

Account Category if applicable.

Bio.

Helpers Section

Edit profile button.
Ad tools button -no link.
Insights button.

Highlights Section

Highlights as tappable circles with name underneath.
Tappable plus circle creates a new highlight with "New" label underneath.

Personal Posts and tagged posts TabView section

Personal posts tab with grid icon.
Tagged posts tab with grid icon.
Personal posts grid.
Tagged posts grid.

Personal profile view partially created


Do checksum check after fetching platform zip file

Recently we modified code so that platform code is directly fetched inside the container: https://github.com/hashgraph/solo/pull/84
As it is not using PackageDownloader to download first,  it doesn't do the checksum check as before: https://github.com/hashgraph/solo/blob/main/src/core/package_downloader.mjs#L208

:tada: This issue has been resolved in version 0.22.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Fix Provisioning

There is a bug in provisioning where the data was
not being translated correctly from the Javascript to the Python backend. This commit fixes that.

Navigate to the nodes/leases panel, click the actions dropdown button and click the provision nodes button. Try to provision with all inputs (image, network, key), there will be a toast pop up saying Error: Unable to provision a node. Provisioner.provision_node() got an unexpected keyword argument 'network'. There is a more detailed message in the console logs.


Question: why PGPainess uses getKeyEncryptionAlgorithm() to determine if it is encrypted instead of analyzing presence of the S2K?

Found this piece of code:
    public boolean isFullyDecrypted() {
        if (isSecretKey()) {
            for (PGPSecretKey secretKey : getSecretKeys()) {
                if (secretKey.getKeyEncryptionAlgorithm() != SymmetricKeyAlgorithm.NULL.getAlgorithmId()) {
                    return false;
                }
            }
        }
        return true;
    }

Is that a valid way to determine if a key is encrypted? Some other implementations in other programming languages (for example openpgp.js) do analyze presence/absence of the S2K.

This is a way I came up with during testing.
I will look into it to see if checking for S2K is the better way to do this :+1:
Thanks for bringing this to my attention :)

Here's what openpgp.js does:
https://github.com/openpgpjs/openpgpjs/blob/2e19f1401ca12eec425d6e758393c8c949d60915/src/packet/secret_key.js#L149

I just pushed a commit that adopts openpgp.js' behavior (which also seems to be more standards compliant, looking at https://tools.ietf.org/html/rfc4880#section-<IP_ADDRESS>).

Excellent!


Revise implicit tagging

I think my logic was flawed here: https://github.com/a-sit-plus/kmp-crypto/blob/82dccc97817fed4cc1bc79e194bd86942cadf61b/datatypes/src/commonMain/kotlin/at/asitplus/crypto/datatypes/asn1/BERTags.kt#L65
Can't we implicitly tag CONSTRUCTED ASN.1 Objects too?
Do we want to have the the tag override for all Asn1Decodables? and merge this extended functionality in the base interface? As we have it now, this implicit tagging only makes sense for primitives, but this is obviously related to the first bullet.

Thoughts, anyone?!

Turns out we can tag CONSTRUCTED objects too, but we need to take care that primitives cannot be tagged such that the CUNSTRUCTED bit is set too. So there need to be separate functions for CONSTRCUTED and primitive ASN.1 elements (and the tag computation works differently)
I imagine creating implicitly tagged objects to work as follows:

someAsn1Element.tagImplicitly(7u) in contrast to explicitly tagging (Asn1.Tagged(7) {+someAsn1Element})

Analogously, I imagine decoding implicitly tagged elements as follows:

someAsn1Element.verifyImplicitTag<Asn1Sequence>(7) -- as inline function with reified generic types.

In theory someAsn1Element.stripImplicitTag<Asn1Primitive>() is a thing, but where would that make sense?.
Please, @iaik-jheher and @nodh,  let me know what you think about this

#117 solves many issues. while not too efficient in creating implicitly tagged elements after they've been constructed, someAsn1Element.tagImplicitly() has the advantage, that a single function on the base Asn1Structure and Asn1Primitive suffices to achieve all goals and would let you even override the constructed bit, if desired. This would then return either an Asn1Primitive or an Asn1CustomStructure with the same content as the original element.
verifyImplicitTag(forceParseToStructure=false) would also work out nicely. and for the base case where the constructed bit has not been overridden, no magic whatsoever is required.

Please, @iaik-jheher and @nodh, let me know what you think about this

Suggested functions above sound fine to me


Decouple from sassc

It would be nice to import code from sassc so we don't have to use a git branch everywhere.
I understand this was made likely to make easier to migrate without breaking everyone's dependency, but we could cut a major release that does not require to add:
gem 'sassc', github: 'sass/sassc-ruby', ref: 'refs/pull/233/head'

This can make it easier for gems to target this new approach.

@ntkme if you agree I would love to help and perhaps provide a MR with the proposal above. Ideally this would be done and released as part of sass/sassc-ruby but until we have them take action this is the second best option, please consider it.

@brodock I'm working on it now. Going to release as a new gem.

@brodock raised PR to SassC Ruby actually: https://github.com/sass/sassc-ruby/pull/240

The main problem is that many apps have sassc as dependency and require 'sassc' in dependency code that they don't have any control over. While it is possible to also have lib/sassc.rb in this project, the $LOAD_PATH order is not grauanteed, which is why this currently relies on a fork of sassc so that the shim code can be loaded whenever original sassc is required.
With that said, the only way to further improve current user expenrience is what @johnnyshields is attempting to do, which is to make the shim the actual upstream implementation. However, I don't really think upstream would support it.
Even the upstream is willing to take this as sassc 3.x, there are quite a few issues. Dart Sass made the decision to revamp the importer behavior and which is very different from sassc. In Dart Sass, an @import or @use directive will strictly map to only one file, whereas in sassc, it used to be possible use glob like @import 'modules/*' to import multiplle files. In order to support this, sassc-embedded uses a custom importer that generates an intermediate stylesheet to import all the files resolved from a glob pattern. This is not only discouraged as it defeats the purpose of the new module system (@use), and will be broken once Sass drops support of @import in the future. There are a few more things made more strict in Dart Sass that sassc-embedded  has they loosen to be compatible with sassc, for example, to allow importing files without extension name, which can be a security issue depends on system configuration. Other than importers, there's the issue with custom functions in sassc that functions are registered as globals rather than locals, leading to thread safety problems like https://github.com/rails/sprockets/pull/759, and eventually affect performance even after the fix. Not to mention no way to access keyword arguments and other new features not being supported (not possible without breaking existing code).
My take is that sassc-embedded is nothing but a stopgap solution to buy some time and let users in the ruby ecosystem migrate their Sass/Scss code to the latest Sass implementation with minimum effort.
The shim is not meant to be the long term solution, as the sass-embedded gem with the new Ruby API closely mirroring the new JS API from upstream is the future.

TL;DR: If you care about the experience, please directly integrate with sass-embedded or ask the dependency you use to do so. So far I have had success with jekyll-sass-converter and tilt now that they natively support sass-embedded without shim. I would encourage the community to contribute such integration in other projects.


Made sparkline component responsive by default for borisyankov/react-sparklines#49

Updated documentation and changed default options so that height and width of the SVG element have to be explicitly defined.

Hey there, a fellow react-sparklines user here.

Would you mind elaborating the rationale behind making height and width required? Isn't there a reasonable default (or implicit value) we could set?
I worry that changing the meaning of these existing props may break some code out there more than necessary.

I'm all for separating the viewbox size from the element's size - I just think the defaults should be carefully considered.

I think I was not clear.  The height and width currently have defaults, and I did not change those.  I added a new set of attributes for the width and height of the SVG element itself.  Those have no defaults, and as such it causes the element to fill the parent element and be responsive.
I was trying to say that if you want the SVG element to not fill you can set the svgHeight or svgWidth attributes which I just added.
Hope that helps!

@codecounselor Picking up from https://github.com/borisyankov/react-sparklines/issues/49#issuecomment-210463289:
I understand your PR better now and I'm fine with it, personally. I've actually merged this into my personal-use master branch and am happy with how it works.
And yes, you'll find that if you merge #48 (or equivalently my branch fix-travis) into your master it will pass CI.

👍


record last compacted position

Record last compacted position for last raft compact task, so the later compact task no need to seek to first raft entry.
Maybe we need more discussion, @siddontang what's your suggestion?
@ngaut @siddontang @BusyJay @disksing  PTAL

LGTM

Could you use atomic variable instead of lock ?
There is no atomicU64.

LGTM

PTAL

Can we have a test to compact the same region at lease two times?


(1041) Update year in CourseParticipationOutcomeFactory

Context
A helper method was included in this factory to generate a random non-future year. We've since added validations to the year fields, but not updated this method. We were passing in a minimum year of 1980, but the minimum valid year is now 1990
This was producing flaky results when asserting against success messages in integration tests, since the factory would occasionally produce invalid course participation outcomes that would block success conditions
Changes in this PR

Update the minimum year in the course participation outcome factory

Release checklist
Release process documentation
As part of our continuous deployment strategy we must ensure that this work is
ready to be released once merged.
Pre-merge

[ ] There are changes required to the Accredited Programmes API for this change to work...

[ ] ... and they have been released to production already

Post-merge

[ ] This adds, extends or meaningfully modifies a feature...

[ ] ... and I have written or updated an end-to-end test for the happy path in the Accredited Programmes E2E repo

[ ] This makes new expectations of the API...

[ ] ... and I have notified the API developer(s) of changes to the contract tests (Pact), or the API is already compliant

[ ] Manually approve release to preprod
[ ] Manually approve release to prod

 Make flaky integration tests more robust (S)


[NuGet Symbol Server] Add download symbols support to Package details page

This PR adds support to download the snupkg from the package details page.
Addresses #5978  and #5970, also added tests.
The idea is to allow downloading a snupkg if the latest uploaded snupkg is "Available". If there is a newer uploaded snupkg that failed validations or still validating, no download allowed. This is taken care of in the download API and in the package details page view.

Probably should get a different icon. Looks great otherwise!


Corrects vars in bail()

I went against the knowledge of people smarter than me and changed where client_name was looking in the event dict when porting from Ruby, as I thought it was incorrect. It turns out that I was actually testing the handler with incorrect data,which lead to my bad conclusion and @cwjohnston has now put me on the straight and narrow. I've re-ran some checks of my handler and so far nothing else has been affected by the bad test data I used initially, and using the right event data for a check result has lead to me closing another PR as it's no longer unnecessary (Free features!).
This highlights the fact that we need more testing and CI, as per Slack discussion. Hopefully we can get the ball rolling with that in another PR shortly :smile:
This will might need rebasing if the other bugfix PR is merged?

Branch name doesn't mean much now because it just fixes bail(), not event_settings as this will be in feat/tests :man_shrugging:


Run tests for multiple python versions

I decided on 3.6 as it is the oldest we support and 3.8 as it is the newest we support.


How to create our own data model in CDM

Looking for proper guidance and training on this CDM for my developer. How to get trained end-2-end on creating required files for CDM.

Hi @sing-ideas, there are quite a lot of documentation on CDM. Please start here if you would like to know more and learn about CDM and its scope. If you already know about CDM and the SDK, then please refer here for creating schema documents for the entities and generate manifests.

Yes, we saw that documentations and exploring for some time. Is there any way to get trained or is there any forum specific to developing CDM for an ISV (Independent Software Vendor), we understood the importance of CDM, it is wonderful and started using Power Apps Data Verse, but looking to create an independent application that runs independently and integrates with Power Apps. Other than the document links is there a way to get trained on this.


Allow for hiding of root node

Should this be handled in trrack-vis as well as trrackjs?
If the visualization hides the root node, should undo using trrack API go to the hidden root node?
How would trrack-vis handle this state?

We were able to hide the root node in our application (sightline). As @dvzacharycutler suggested we removed the root node from  trrack.graph.backend.nodes and set the root to the first child node. Of course we had to ensure that the node navigation stays within our customized graph. Even though this solution basically works, it would still be nice to support a custom root node by default in TrrackVis.


Warnings: LF will be replaced by CRLF in Dockerfiles

@marcusbooyah: I'm seeing a bunch of these build warnings and remembered you having trouble building Dockerfiles.
Is this still the case?  It would be nice to get rid of these warnings.


feat: return array shape from model only method

[x] Added or updated tests

Closes #1645
Changes
Returns the array shape provided to $model->only(), credit goes to @simon-tma for writing the extension.
Thanks!

Thank you!


curve task

claim curve task

Will you cover convex finance?

claim curve task
Will you cover convex finance?

please read the link https://github.com/rebase-network/Dapp-Learning/blob/main/docs/dao.md
add WeChat to join our developer group


Update configuration to use the new AppEngine Flex config

Addresses #31

👍   Is this also a user-facing issue?  i.e. any user that maybe affected by this, and needs to be notified?  If so, do we have a mechanism to do that?

For users who already installed policy scanner and deployed it (e.g. myself), they shouldn't have any issues whether or not they merge this change in, for now. I wasn't able to reproduce the error about requiring an app.yaml with my already-deployed project. However, when I created a new project and new appengine app and tried to deploy policy scanner, that's when I saw the error.
Hopefully if someone is running into issues, they will let us know... I will also update the README to encourage them to file issues in Github.


Shouldn't AddEntityFrameworkStores use services.TryAdd instead of just services.Add ?

I created my own implementation of the IUserStore and added it to the services, but calling AddEntityFrameworkStores registers a new implementation for the interface (the default one) and makes UserManager to use that one instead of mine.
I believe the problem can be easily solved by changing the implementation of the extension method from:
public static IdentityBuilder AddEntityFrameworkStores<TContext>(this IdentityBuilder builder) where TContext : DbContext
{
    builder.Services.Add(IdentityEntityFrameworkServices.GetDefaultServices(builder.UserType, builder.RoleType, typeof(TContext)));
    return builder;
}

to:
public static IdentityBuilder AddEntityFrameworkStores<TContext>(this IdentityBuilder builder) where TContext : DbContext
{
    builder.Services.TryAdd(IdentityEntityFrameworkServices.GetDefaultServices(builder.UserType, builder.RoleType, typeof(TContext)));
    return builder;
}

So that it only adds the services if I haven´t added my own implementation before.

Yep, good catch, these should be TryAdds...


Gives phantoms the same nubbin as build errors

This is just a very simple copy-paste from Default's exec.py to sync the styling of LSP phantoms with build errors.
before

after

The docs suggest giving the phantoms a package specific id to help targeting style customisation. However, I think the idea here is to match the built-in phantoms so it's good as is at least for now. It's a shame these styles aren't easily shared. It's also not guaranteed to get out of sync.. ¯_(ツ)_/¯

Look great to me so far, thanks for the screenshots.
Let me know if you were planning on doing more work in this PR, or if it should be merged!

@tomv564 No, I intend to break it up, the other stuff will probably take more time and will not touch the same code at all. So go ahead and merge!


Add UBX-MON-COMMS

I think UBX-MON-COMMS is working and I'm getting a message like this:
header:
  stamp:
    sec:<PHONE_NUMBER>
    nanosec: 268752991
  frame_id: gnss
version: 0
n_ports: 5
tx_errors: 16
reserved0: 0
prot_ids:
- 0
- 1
- 5
- 6
ports:
- port_id: 256
  tx_pending: 0
  tx_bytes: 793237
  tx_usage: 0
  tx_peak_usage: 10
  rx_pending: 0
  rx_bytes: 0
  rx_usage: 0
  rx_peak_usage: 0
  overrun_errs: 0
  msgs:
  - 0
  - 0
  - 0
  - 0
  skipped: 0
- port_id: 0
  tx_pending: 0
  tx_bytes: 0
  tx_usage: 0
  tx_peak_usage: 2
  rx_pending: 0
  rx_bytes: 3231614
  rx_usage: 0
  rx_peak_usage: 0
  overrun_errs: 0
  msgs:
  - 0
  - 0
  - 0
  - 0
  skipped: 19775
- port_id: 0
  tx_pending: 0
  tx_bytes: 0
  tx_usage: 0
  tx_peak_usage: 0
  rx_pending: 0
  rx_bytes: 65
  rx_usage: 0
  rx_peak_usage: 3
  overrun_errs: 0
  msgs:
  - 0
  - 0
  - 0
  - 0
  skipped: 1296
- port_id: 1024
  tx_pending: 0
  tx_bytes: 36
  tx_usage: 0
  tx_peak_usage: 0
  rx_pending: 0
  rx_bytes: 0
  rx_usage: 0
  rx_peak_usage: 0
  overrun_errs: 0
  msgs:
  - 0
  - 0
  - 0
  - 0
  skipped: 4585706
- port_id: 257
  tx_pending: 0
  tx_bytes: 2628010
  tx_usage: 1
  tx_peak_usage: 7
  rx_pending: 0
  rx_bytes: 9398
  rx_usage: 0
  rx_peak_usage: 0
  overrun_errs: 0
  msgs:
  - 0
  - 0
  - 0
  - 0
  skipped: 0

but honestly, I can't figure out if the values make any sense since I don't know exactly what each field means

@ARK3r its one of those messages that could be used to debug a configuration.
Some of the field descriptions are a bit poor in the manual.
I noticed you didn't add the out frame? That's used to potential poll for the data as opposed to having it sent based on a frequency update.
One issue is the cpu usage on the device. If everything is turned on, it can't cope and limits output anyway.
Which firmware are you using? ... I upgraded to the latest this week.
Is it possible for you to confirm the changes I made for the spartn messages?

@hortovanyi Thanks for your work on the the rxm-spartn and rxm-spartnkey.
I'm just confused about how I would provide the keys. This is a good example of how a json file gets parsed, and this shows which data bits are used and here they are put together as a UBX msg to be sent.
Regarding spartn message this is what I'm getting:
header:
  stamp:
    sec:<PHONE_NUMBER>
    nanosec: 634434780
  frame_id: gnss
version: 1
msg_used: 1
sub_type: 0
msg_type: 1
---
header:
  stamp:
    sec:<PHONE_NUMBER>
    nanosec: 641821699
  frame_id: gnss
version: 1
msg_used: 1
sub_type: 0
msg_type: 0
---
header:
  stamp:
    sec:<PHONE_NUMBER>
    nanosec: 651982608
  frame_id: gnss
version: 1
msg_used: 1
sub_type: 1
msg_type: 0
---
header:
  stamp:
    sec:<PHONE_NUMBER>
    nanosec: 661819118
  frame_id: gnss
version: 1
msg_used: 1
sub_type: 2
msg_type: 0
---
header:
  stamp:
    sec:<PHONE_NUMBER>
    nanosec: 671816052
  frame_id: gnss
version: 1
msg_used: 1
sub_type: 0
msg_type: 1

which I assume means spartn messages are being recieved but not integrated, as I have not yet provided the key.

I'll check the out frame right now and get back to you.
I upgraded the firmware last week as well to: ZED-F9P L1/L2 HPG 1.50 firmware 25-Jul-2024

@ARK3r thanks 😊 ... ohh I see, so we need a separate node to get the keys and then need to publish a msg for ublox_dgnss_node to subscribe to, to then send the spartnkeys to the device??

I pushed the out frame debug info to my branch, not sure how I can reopen this pr or if I should make another one.
I did make a super hacky script recently, let me check if spartn messages are being decrypted properly using this method, and then we can integrate it in somehow
super hacky script:
from sys import argv
from time import sleep
import os

from serial import Serial
from pyubx2 import UBX_PROTOCOL, UBXMessage, UBXReader, POLL, val2bytes, SET, U1, U2, U4

from datetime import datetime, timedelta

from spartn_json_config import SpartnJsonConfig

GPSEPOCH0 = datetime(1980, 1, 6)  # for Wno and Tow calculations
RESERVED0 = b"\x00\x00"
RESERVED1 = b"\x00"

def date2wnotow(dat: datetime) -> tuple:
    """
    Get GPS Week number (Wno) and Time of Week (Tow)
    for given datetime.

    GPS Epoch 0 = 6th Jan 1980

    :param datetime dat: calendar date
    :return: tuple of (Wno, Tow)
    :rtype: tuple
    """

    wno = int((dat - GPSEPOCH0).days / 7)
    tow = ((dat.weekday() + 1) % 7) * 86400
    return wno, tow

def send_keys():
    json_file = os.path.join(os.path.expanduser("~"), "keys.json")

    spartn_json_config = SpartnJsonConfig(json_file)

    spartn_key1 = spartn_json_config.current_key
    spartn_key2 = spartn_json_config.next_key

    print(spartn_key1)

    version = val2bytes(1, U1)
    numKeys = val2bytes(2, U1)
    key1 = bytes.fromhex(spartn_key1[0])
    keylen1 = val2bytes(len(key1), U1)
    wno1, tow1 = date2wnotow(spartn_key1[1])
    wno1b = val2bytes(wno1, U2)
    tow1b = val2bytes(tow1, U4)
    key2 = bytes.fromhex(spartn_key2[0])
    keylen2 = val2bytes(len(key2), U1)
    wno2, tow2 = date2wnotow(spartn_key2[1])
    wno2b = val2bytes(wno2, U2)
    tow2b = val2bytes(tow2, U4)

    payload = (
        version
        + numKeys
        + RESERVED0
        + RESERVED1
        + keylen1
        + wno1b
        + tow1b
        + RESERVED1
        + keylen2
        + wno2b
        + tow2b
        + key1
        + key2
    )
    msg = UBXMessage("RXM", "RXM-SPARTN-KEY", SET, payload=payload)

    return msg

def main(**kwargs):
    """
    Main routine.
    """

    port = kwargs.get("port", "/dev/ttyACM0")
    baudrate = int(kwargs.get("baudrate", 38400))
    timeout = float(kwargs.get("timeout", 0.1))

    with Serial(port, baudrate, timeout=timeout) as stream:
        ubr = UBXReader(stream, protfilter=UBX_PROTOCOL)

        print("\nStarting. Press Ctrl-C to terminate...")

        try:
            while True:
                # Polling RXM-SPARTN-KEY message type
                print("Polling RXM-SPARTN-KEY message type...")
                msg = UBXMessage("RXM", "RXM-SPARTN-KEY", POLL)
                stream.write(msg.serialize())

                # Check if data is available and read it
                if stream.in_waiting:
                    try:
                        raw_data, parsed_data = ubr.read()
                        if parsed_data and parsed_data.identity == "RXM-SPARTN-KEY":
                            print("has keys" if parsed_data.numKeys > 0 else "no keys found")
                            if parsed_data.numKeys > 0:
                                break
                            else:
                                keys_msg = send_keys()
                                stream.write(keys_msg.serialize())
                            # print(parsed_data.numKeys)  # Directly print parsed data
                            # print(parsed_data.identity)
                    except Exception as err:
                        print(f"\n\nSomething went wrong {err}\n\n")

                # Sleep briefly to avoid busy-waiting and control polling rate
                sleep(1)

            print(parsed_data)

        except KeyboardInterrupt:  # capture Ctrl-C
            print("\n\nTerminated by user.")

        print("\nProcessing complete")

if __name__ == "__main__":

    main(**dict(arg.split("=") for arg in argv[1:]))

The only thing I wanted to point out is that this is params I ended up using:
  params = [
        {'FRAME_ID': "gnss"},
        # {'DEVICE_SERIAL_STRING': '/dev/ttyUbloxGPS'},
        {'CFG_USBOUTPROT_NMEA': False},
        {'CFG_RATE_MEAS': 10},
        {'CFG_RATE_NAV': 100},
        {'CFG_MSGOUT_UBX_NAV_HPPOSLLH_USB': 1},
        {'CFG_MSGOUT_UBX_NAV_STATUS_USB': 1},
        {'CFG_MSGOUT_UBX_NAV_COV_USB': 1},
        {'CFG_MSGOUT_UBX_NAV_SAT_USB': 1},
        {'CFG_MSGOUT_UBX_NAV_RELPOSNED_USB': 1},
        {'CFG_SPARTN_USE_SOURCE': 1},
        {'CFG_UART2INPROT_SPARTN': True},
        {'CFG_MSGOUT_UBX_RXM_RTCM_USB': 1},
        {'CFG_MSGOUT_UBX_RXM_SPARTN_USB': 1},
        {'CFG_MSGOUT_UBX_RXM_COR_USB': 1},
        {'CFG_MSGOUT_UBX_MON_COMMS_USB': 1},
        # {'CFG_MSGOUT_UBX_MON_COMMS_USB': 1},

    ]

because there was some sort of an error saying CFG_UART2INPROT_SPARTN expects boolean and not integer

i've tried to keep all the code besides the launch files as c++ ... we could set up a ros2 service to receive the keys?? (and then publish them) or otherwise are you suggesting to pass the json file name as a parameter (it not specified dont do anything  or if there at a later state after initialisation send to the device once off)??
Am not sure if some of the other implementations are reading the keys straight from the internet? and then sending to the device?
I'll leave this decision up to you at present. Just need to make sure that if people are using NTRIP instead, that the node still works the same

This is what the structure of the json file that I got from thingstream looks like (with the values redacted):
- MQTT:
        - Connectivity:
                - Name: **********
                - ClientID: **********
                - Protocol: **********
                - ServerURI: **********
                - ClientCredentials:
                        - Type: **********
                        - Key: **********
                        - Cert: **********
                        - RootCA: **********
                - CleanSession: True
                - ConnectionTimeout: **********
                - KeepAliveInterval: **********
                - SSL:
                        - Mode: **********
                        - Protocol: **********
        - Subscriptions:
                - Key:
                        - QoS: **********
                        - KeyTopics: **********
                - AssistNow:
                        - QoS: **********
                        - AssistNowTopics: **********
                - Data:
                        - QoS: **********
                        - DataTopics: **********
        - dynamickeys:
                - current:
                        - start: **********
                        - duration: **********
                        - value: **********
                - next:
                        - start: **********
                        - duration: **********
                        - value: **********

This structure suggests there must be more ways other than just providing the dynamic keys and using those to decrypt the spartn messages, however, at the moment I have only experimented with providing the keys and the f9p has achieved "fixed" state by connection to the d9s.
I think a service that accepts these keys with a custom message actually sounds good, and it offloads the work of processing the json file, or if someone creates their own way of passing the keys to the device connected to the module, I think a service would be the most versatile solution.
I spent some time looking at the UBX-ESF-MEAS message and I'm afraid it looks way too complicated for me to figure out at the moment. Is it possible that you implement this service? If not, I can work on it sometime next week.

@ARK3r need to create a new Full Payload to send messages to the device ... it basically is taking the ros2 msg and creating a UBX binary payload to send (ie poll) out. look at load_from_msg method
https://github.com/aussierobots/ublox_dgnss/blob/0c37a4884c4914ff36d8947f77137d51f0ad0675/ublox_dgnss_node/include/ublox_dgnss_node/ubx/esf/ubx_esf_meas.hpp#L136
I suspect from what I'm seeing in this chat, is that a new node will be needed as a MQTT client as there are subscription and dynamic keys ...  the ubx message allows for dynamic key sizes. I had to think twice about what I coded up for all of that. Extrapolating from the keys binary to the ubx poll payload, while tricky is not impossible - its a bit different from the other messages so takes some time.
I dont know the frequency of change in any of the those keys ?? at first I had thought they were relatively static but looking at your last comment makes me think not.
Have you created a MQTT client in c++ before for a ros2 node? I haven't and am not sure if the std ros2 build farm libraries include the client??? Normally this stuff when I try it, works and just compiles. I suspect the mqtt clients are standard libraries in linux distributions.
There doesnt seem to be anything here that suggests its not worthwhile doing, however I cant test it all so probably best for you to code it when you can. Can help you if you get stuck

@ARK3r any progress? if stuck let me know and I'll help best I can

I have been pulled to other things at the moment, once I have to get back on this I'll reach back out.

@ARK3r ok. I might set up an Issue for this and move the conversation there


When to use split vs gain for plot_importance?

Description
The default plot_importance function uses split, the number of times a feature is used in a model.  Gain is an alternative choice.
I am wondering whether there is any guidance or recommendation when to use split and when to use gain for feature importance understanding. Thanks!

@annaymj Thanks for using LightGBM! In decision tree literature, the gain-based feature importance is the standard metric, because it measures directly how much a feature contributes to the loss reduction. However, I think since the split features are selected through split gain, number of times being used should result in similar feature importance ordering.
I'm not sure whether you observed a significant difference between the two in your use cases. Please feel free to post further questions here and I'll try my best to help.


Workflow when deployed to different location from VSCode will run into new connection creation issue

Describe the Bug
Getting invalid connection error for Office_365 connection after deploying the workflow from VS code. There is automatically change in location as per Workflow Json . Selected location in Vs code to create a logic app is East US. But in portal automatically Central US location is getting added as per Workflow JSON code.
Added Workflow JSON code od VS code and portal in Workflow JSON tab.
Tried to add a new connection for deployed logic app in portal but it is not allowing to add the new connection as the location is different. Attached the screenshot for the same .
Plan Type
Standard
Steps to Reproduce the Bug or Issue

Create a new Logic App Project
Add a workflow to your local project
Open the workflow.json file in designer and author a new workflow
a. Add http request Trigger
b. Add a compose action here with custom text
c. Add a ServiceBus action. From the add new paramter dropdown add content and add random text in JSON format like {"a":"b"}
d. Add a send email action with the body being the output of the compose action (step b)
e. Add a new connection to your email action
f. Add http response action
Press F5 to build and run your project
Run the workflow
a. Open overview page. Right + Click on workflow.json file and "Overview"
b. Copy call back URL from the overview page
c. Send the http request POST via Postman and wait for the response
Refresh the overview page and your should see your local run
Open the monitoring view for your run from the overview page
Verify the run details and action inputs and output
Create a logic app and Deploy to Azure
10.After deployment is success, Open the deployed logic app in portal.
Open the workflow in designer .
Workflow should not show any error.

**Expected:**Workflow should not show any error.
Actual: Getting invalid connection error for Office_365 connection after deploying the workflow from VS code
Workflow JSON
Portal Workflow JSON:

{
  "definition": {
    "$schema": "https://schema.management.azure.com/providers/Microsoft.Logic/schemas/2016-06-01/workflowdefinition.json#",
    "actions": {
      "Response": {
        "type": "Response",
        "kind": "Http",
        "inputs": {
          "statusCode": 200
        },
        "runAfter": {
          "Send_an_email_(V2)": [
            "SUCCEEDED"
          ]
        }
      },
      "Compose": {
        "type": "Compose",
        "inputs": "vs code 0606",
        "runAfter": {}
      },
      "Send_message": {
        "type": "ServiceProvider",
        "inputs": {
          "parameters": {
            "entityName": "test",
            "message": {
              "contentData": {
                "a": "b"
              }
            }
          },
          "serviceProviderConfiguration": {
            "connectionName": "serviceBus",
            "operationId": "sendMessage",
            "serviceProviderId": "/serviceProviders/serviceBus"
          }
        },
        "runAfter": {
          "Compose": [
            "SUCCEEDED"
          ]
        },
        "trackedProperties": {}
      },
      "Send_an_email_(V2)": {
        "type": "ApiConnection",
        "inputs": {
          "host": {
            "connection": {
              "referenceName": "office365-2"
            }
          },
          "method": "post",
          "body": {
            "To"<EMAIL_ADDRESS>            "Subject": "Vs code 06/06/2023",
            "Body": "
@{outputs('Compose')}
",
            "Importance": "Normal"
          },
          "path": "/v2/Mail"
        },
        "runAfter": {
          "Send_message": [
            "SUCCEEDED"
          ]
        }
      }
    },
    "contentVersion": "<IP_ADDRESS>",
    "outputs": {},
    "triggers": {
      "When_a_HTTP_request_is_received": {
        "type": "Request",
        "kind": "Http"
      }
    }
  },
  "connectionReferences": {
    "serviceBus": {
      "connection": {
        "id": "/serviceProviders/serviceBus/connections/serviceBus"
      },
      "connectionName": "abc",
      "api": {
        "id": "/serviceProviders/serviceBus"
      }
    },
    "office365-2": {
      "connection": {
        "id": "/subscriptions/53ce23c6-ff79-490b-a7fb-9590fe7a414a/resourceGroups/VSCode-v-govubilish/providers/Microsoft.Web/connections/office365-11"
      },
      "connectionName": "office365-11",
      "api": {
        "id": "/subscriptions/53ce23c6-ff79-490b-a7fb-9590fe7a414a/providers/Microsoft.Web/locations/centralus/managedApis/office365"
      },
      "authentication": {
        "type": "ManagedServiceIdentity"
      }
    }
  },
  "parameters": {}
}

VS code Workflow JSON:

{
    "definition": {
        "$schema": "https://schema.management.azure.com/providers/Microsoft.Logic/schemas/2016-06-01/workflowdefinition.json#",
        "actions": {
            "Response": {
                "type": "Response",
                "kind": "Http",
                "inputs": {
                    "statusCode": 200
                },
                "runAfter": {
                    "Send_an_email_(V2)": [
                        "SUCCEEDED"
                    ]
                }
            },
            "Compose": {
                "type": "Compose",
                "inputs": "vs code 0606",
                "runAfter": {}
            },
            "Send_message": {
                "type": "ServiceProvider",
                "inputs": {
                    "parameters": {
                        "entityName": "test",
                        "message": {
                            "contentData": {
                                "a": "b"
                            }
                        }
                    },
                    "serviceProviderConfiguration": {
                        "connectionName": "serviceBus",
                        "operationId": "sendMessage",
                        "serviceProviderId": "/serviceProviders/serviceBus"
                    }
                },
                "runAfter": {
                    "Compose": [
                        "SUCCEEDED"
                    ]
                },
                "trackedProperties": {}
            },
            "Send_an_email_(V2)": {
                "type": "ApiConnection",
                "inputs": {
                    "host": {
                        "connection": {
                            "referenceName": "office365-2"
                        }
                    },
                    "method": "post",
                    "body": {
                        "To"<EMAIL_ADDRESS>                        "Subject": "Vs code 06/06/2023",
                        "Body": "
@{outputs('Compose')}
",
                        "Importance": "Normal"
                    },
                    "path": "/v2/Mail"
                },
                "runAfter": {
                    "Send_message": [
                        "SUCCEEDED"
                    ]
                }
            }
        },
        "contentVersion": "<IP_ADDRESS>",
        "outputs": {},
        "triggers": {
            "When_a_HTTP_request_is_received": {
                "type": "Request",
                "kind": "Http"
            }
        }
    },
    "kind": "Stateful"
}

Screenshots or Videos

Additional context
Artifacts details:

VS Code extension:  vscode-azurelogicapps-2.24.3.vsix
NuGet package: https://artprodeussu2.artifacts.visualstudio.com/A41bf5486-7392-4b7a-a7e3-a735c767e3b3/b32aa71e-8ed2-41b2-9d77-5bc261222004/_apis/artifact/cGlwZWxpbmVhcnRpZmFjdDovL21zYXp1cmUvcHJvamVjdElkL2IzMmFhNzFlLThlZDItNDFiMi05ZDc3LTViYzI2MTIyMjAwNC9idWlsZElkLzc0MTk5ODA4L2FydGlmYWN0TmFtZS9kcm9wX2J1aWxkX0J1aWxkQXJ0aWZhY3RzX1dpbmRvd3M1/content?format=file&subPath=%2FMicrosoft.Azure.Workflows.WebJobs.Extension.1.25.1.nupkg
Extension Bundle (CDN URL): https://cdnforlogicappsv2.blob.core.windows.net/logicapps-vendortesting
Extension Bundle (Direct Download): https://artprodeussu2.artifacts.visualstudio.com/A41bf5486-7392-4b7a-a7e3-a735c767e3b3/b32aa71e-8ed2-41b2-9d77-5bc261222004/_apis/artifact/cGlwZWxpbmVhcnRpZmFjdDovL21zYXp1cmUvcHJvamVjdElkL2IzMmFhNzFlLThlZDItNDFiMi05ZDc3LTViYzI2MTIyMjAwNC9idWlsZElkLzc0MjAxMjgxL2FydGlmYWN0TmFtZS9kcm9wX2J1aWxkX0J1aWxkQXJ0aWZhY3RzX1dpbmRvd3M1/content?format=file&subPath=%2FMicrosoft.Azure.Functions.ExtensionBundle.Workflows.1.25.1_any-any.zip

AB#24188832
AB#24192914

The invalid connection issue is now fixed in latest deployment, but the remaining and active issue from this is -
If LA is deployed to different region from what was selected in vscode local project then while creating new connection we should take location from apiId instead of location from workflow app, this would make sure we have successful connection creation.

Intermittent Issue repro in the latest request in VS code run history with status code as 200.
Artifacts details:
1.VS Code extension: vscode-azurelogicapps-2.24.3.vsix
2.NuGet package: https://artprodeussu2.artifacts.visualstudio.com/A41bf5486-7392-4b7a-a7e3-a735c767e3b3/b32aa71e-8ed2-41b2-9d77-5bc261222004/_apis/artifact/cGlwZWxpbmVhcnRpZmFjdDovL21zYXp1cmUvcHJvamVjdElkL2IzMmFhNzFlLThlZDItNDFiMi05ZDc3LTViYzI2MTIyMjAwNC9idWlsZElkLzc0NTYyNjAxL2FydGlmYWN0TmFtZS9kcm9wX2J1aWxkX0J1aWxkQXJ0aWZhY3RzX1dpbmRvd3M1/content?format=file&subPath=%2FMicrosoft.Azure.Workflows.WebJobs.Extension.1.26.1.nupkg
3.Extension Bundle (CDN URL): https://cdnforlogicappsv2.blob.core.windows.net/logicapps-vendortesting
4.Extension Bundle (Direct Download): https://cdnforlogicappsv2.blob.core.windows.net/logicapps-vendortesting/ExtensionBundles/Microsoft.Azure.Functions.ExtensionBundle.Workflows/1.26.1/Microsoft.Azure.Functions.ExtensionBundle.Workflows.1.26.1_any-any.zip

Issue Repro in the new VSIX
With new VSIX (2.24.6), Getting the same error message popup with 200 as status code only in VS code for office-365 after deploying the workflow and running it. In portal when we open the same run history we are not getting the error message popup
Artifacts details:
1.VS Code extension: vscode-azurelogicapps-2.24.6.vsix

2.NuGet package: https://artprodeussu2.artifacts.visualstudio.com/A41bf5486-7392-4b7a-a7e3-a735c767e3b3/b32aa71e-8ed2-41b2-9d77-5bc261222004/_apis/artifact/cGlwZWxpbmVhcnRpZmFjdDovL21zYXp1cmUvcHJvamVjdElkL2IzMmFhNzFlLThlZDItNDFiMi05ZDc3LTViYzI2MTIyMjAwNC9idWlsZElkLzc0NTYyNjAxL2FydGlmYWN0TmFtZS9kcm9wX2J1aWxkX0J1aWxkQXJ0aWZhY3RzX1dpbmRvd3M1/content?format=file&subPath=%2FMicrosoft.Azure.Workflows.WebJobs.Extension.1.26.1.nupkg
3.Extension Bundle (CDN URL): https://cdnforlogicappsv2.blob.core.windows.net/logicapps-vendortesting
4.Extension Bundle (Direct Download): https://cdnforlogicappsv2.blob.core.windows.net/logicapps-vendortesting/ExtensionBundles/Microsoft.Azure.Functions.ExtensionBundle.Workflows/1.26.1/Microsoft.Azure.Functions.ExtensionBundle.Workflows.1.26.1_any-any.zip

Issue repro in latest request
With new VSIX (2.28.3), Getting the same error message popup with 200 as status code only in VS code for office-365 after deploying the workflow and running it. In portal when we open the same run history we are not getting the error message popup
Artifacts details:
1.VS Code extension: vscode-azurelogicapps-2.28.3.vsix
2.Nuget package:
https://artprodeussu2.artifacts.visualstudio.com/A41bf5486-7392-4b7a-a7e3-a735c767e3b3/b32aa71e-8ed2-41b2-9d77-5bc261222004/_apis/artifact/cGlwZWxpbmVhcnRpZmFjdDovL21zYXp1cmUvcHJvamVjdElkL2IzMmFhNzFlLThlZDItNDFiMi05ZDc3LTViYzI2MTIyMjAwNC9idWlsZElkLzc1MzA5MDMwL2FydGlmYWN0TmFtZS9kcm9wX2J1aWxkX0J1aWxkQXJ0aWZhY3RzX1dpbmRvd3M1/content?format=file&subPath=%2FMicrosoft.Azure.Workflows.WebJobs.Extension.1.28.2.nupkg
3.Extension Bundle (CDN URL): https://cdnforlogicappsv2.blob.core.windows.net/logicapps-vendortesting
4.Extension Bundle (Direct Download): https://cdnforlogicappsv2.blob.core.windows.net/logicapps-vendortesting/ExtensionBundles/Microsoft.Azure.Functions.ExtensionBundle.Workflows/1.28.2/Microsoft.Azure.Functions.ExtensionBundle.Workflows.1.28.2_any-any.zip

I was able to repro this, will make a fix and will update thread.


Simply doesn't work

calling cmcClient.getTicker() results in an error "Cannot destructure property 'start'"

try cmcClient.getTicker({})

@abletaz should be resolved with v1.0.2. Please reopen if issue still exists.


Summary validation enhancements

Description

There are two separate changes in this PR:

When a navigationbutton shows Back to summary, it now respects the trigger that was on it.
A new trigger has been added: validateCurrentAndPreviousPages (quite a mouthful). This is to solve the case where you have an intermediate summary, and you want all of the previous pages to be validated before you can proceed with filling out the rest of the form.

Note: This means that navigating to the next page is blocked if there are validation errors on any previous pages. This is not the case if the trigger has validatePage. I was not 100% sure how validateAllPages should be handeled. Previously it would only block navigation if there was an error on the current page. It does not make sense to block navigation if there are only errors in "future" pages, as that would seem a little weird to not be able to proceed when the error are forwards. So for now, I made validateAllPages act the same way as currentAndPrevious in that it will block if there are any errors on the current and previous pages, instead of just the current page. This is technically a breaking change, and I could add a special case for it so that it acts the same way as before; however, I am not sure that anyone has used validateAllPages in this way before, since you would get bombarded by validation errors on fields you have not seen yet. I am open to suggestions on how this trigger should be handled.

Related Issue(s)

closes #1112

Verification/QA

Manual functionality testing

[x] I have tested these changes manually
[ ] Creator of the original issue (or service owner) has been contacted for manual testing (or will be contacted when released in alpha)
[ ] No testing done/necessary

Automated tests

[x] Unit test(s) have been added/updated
[ ] Cypress E2E test(s) have been added/updated
[ ] No automatic tests are needed here (no functional changes/additions)
[ ] I want someone to help me make some tests

UU/WCAG (follow these guidelines until we have our own)

[ ] I have tested with a screen reader/keyboard navigation/automated wcag validator
[x] No testing done/necessary (no DOM/visual changes)
[ ] I want someone to help me perform accessibility testing

User documentation @ altinn-studio-docs

[ ] Has been added/updated

[ ] No functionality has been changed/added, so no documentation is needed
[ ] I will do that later/have created an issue

Changes/additions to component properties

[x] Changes are reflected in both src/layout/layout.d.ts and layout.schema.v1.json, and these are all backwards-compatible
[ ] No changes made

Support in Altinn Studio

[ ] Issue(s) created for support in Studio

[ ] This change/feature does not require any changes to Altinn Studio

Sprint board

[x] The original issue (or this PR itself) has been added to the Team Apps project and to the current sprint board
[ ] I don't have permissions to do that, please help me out

Labels

[x] I have added a kind/* label to this PR for proper release notes grouping
[ ] I don't have permissions to add labels, please help me out

Sounds good!

When a navigationbutton shows Back to summary, it now respects the trigger that was on it.

I'm note entirely sure about this. I remember looking into this myself, a while back, and I vaguely remember getting scared off from making that change. I would have to do some digging to find out why, but in principle I don't see why we need it to respect the trigger? 🤔 You've been to the summary page already, and the button just warps you back there. If the page you're on contains lots of validation errors, has a validatePage trigger, why would we not allow you to go back to the summary page and continue working from there?


Nice new colors unusable on white background

The white text color of pipenv graph (for instance) isn't really easily readable on white backgrounds; it would be wonderful if it were.
Thanks.

Pipenv has a no color mode set PIPENV_COLORBLIND=1,
99% of people use black terminals. and, you can configure those colors, normally.

I am not sure about those 99%
https://opensource.com/life/14/9/what-color-terminal-background

those people shouldn't have WHITE configured to be the same color as your background.
 This is a terminal configuration issue, not a pipenv issue.

And, as mentioned, there is a setting to disable colors, if you prefer.

https://github.com/kennethreitz/pipenv/commit/1f1e9423688ec6c26c7214c21c8eda52f2c81d36

Fixed in v7.9.3

Dark blue foreground on a blue background is equally invisible. As  in pipenv --man:

Do you see the run? (4NT shell on Win-10).
Since I'm a Python neophyte, I had to go into pipenv/vendor/colorama/ansitowin32.py to fix this.

@gvanem, I'm not sure we can do much here. No matter which colors we choose, you will always be able to set your terminal background to a similar color. Black and white backgrounds account for the majority of terminals and supporting those is probably all we'll be doing at this time.

However you can change what 'blue' means for your terminal rendering

@techalchemy What's the easiest way? (I've just started with pipenv today. Not so promising so far).

@gvanem This really depends on your terminal.  On my mac, I use iTerm which gives me the ability to define ansi colors.  This is essentially what themes do.  You can define foreground blue to be different than background blue.

@nateprewitt I know we don't want to go down a rabbit hole of supporting everything, but on posix the majority are black and white backgrounds, however for windows, powershell is just straight up blue.  It is also not as straightforward to change ansi colors in powershell.  Given that it's the default shell on windows (not including cmd), and not many people use Cmder, we should perhaps look into changing that color so it works on posix and windows out of the box.

change what 'blue' means ..

My response to this really meant, "what change in pipenv can easly do". I know how to change the colours in Windows console. Duh!
But compared to many other consoles, WinCon is a old dog. I think displaying those cool emojis would require something completely different; Xterm / Cmder for Windows?

@gvanem Pipenv relies on another package, crayons, to do it's colors.  There isn't much in pipenv to change what blue means unless you change the ansi color it calls in your terminal to something else.  We could avoid using blue, which I am looking into now for windows powershell support out of the box on windows.  I personally use Cmder for windows and it's a lot better at displaying the fancy colors pipenv uses.

Who changed the colors?
On Sun, Nov 19, 2017 at 3:22 PM, Gisle Vanem<EMAIL_ADDRESS>wrote:

change what 'blue' means ..
My response to this really meant, "what change in pipenv can easly do".
I know how to change the colours in Windows console. Duh!
But compared to many other consoles, WinCon is a old dog. I think
displaying those cool emojis would require something completely different;
Xterm / Cmder for Windows?
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/kennethreitz/pipenv/issues/545#issuecomment-345546443,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAHUVaNGvIi1H4VSAHSyiTQkUx1TBxNWks5s4I4PgaJpZM4PXnrm
.

@kennethreitz I'm not sure, gotta look through the commit log.  Were the colors different originally for man?  I can't remember.

Those parts aren't to be touched by anyone but me.
On Sun, Nov 19, 2017 at 3:38 PM, erinxocon<EMAIL_ADDRESS>wrote:

@kennethreitz https://github.com/kennethreitz I'm not sure, gotta look
through the commit log. Were the colors different originally for man? I
can't remember.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/kennethreitz/pipenv/issues/545#issuecomment-345547613,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAHUVVQlJoqp1KJ7WpiZXGOfIEi6-zLgks5s4JGqgaJpZM4PXnrm
.

So if they were changed, please revert the changes back to their original
state.
On Sun, Nov 19, 2017 at 3:38 PM, Kenneth Reitz<EMAIL_ADDRESS>wrote:

Those parts aren't to be touched by anyone but me.
On Sun, Nov 19, 2017 at 3:38 PM, erinxocon<EMAIL_ADDRESS>wrote:

@kennethreitz https://github.com/kennethreitz I'm not sure, gotta look
through the commit log. Were the colors different originally for man? I
can't remember.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/kennethreitz/pipenv/issues/545#issuecomment-345547613,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAHUVVQlJoqp1KJ7WpiZXGOfIEi6-zLgks5s4JGqgaJpZM4PXnrm
.

would it be possible to make nocolors a flag, too, so that pipenv --no-colors run python blah can be called on case-by-case basis rather than turning it off at the system level? Especially on windows10 I find myself using at least three different command lines (cmd, PS, and bash) and for sanity they're all set to different schemes, only one of which really needs no-color to kick in.

@Pomax there is an environment variable you can set. Check the docs ;)

Indeed, that's what I'm currently doing, but it's not exactly convenient. It would just be much nicer to also have a -cb/--color-blind flag, especially as that also makes it possible to do selective color-or-not when it's run as part of a larger build system (for instance, as part of a docker + node + python bootstrap), where an environment variable would be overkill, requiring setting and then unsetting just for a single pipenv run.

`PIPENV_COLORBLIND=1 pipenv run python blah

effectively identical

100% not possible in windows, hence the flag request because that's presumably little effort to code in, while making every (forced or voluntary) windows dev's life much nicer.

While I agree (being a Windows user myself) it would be much simpler to have a flag instead of environ if you want the flag/environ to affect only one command, I don’t really understand the usage of this particular case. Why would you only want to disable colours for this particular command? For me personally, the preference in shell, or even system-based, so I would just set the environ in user preferences, or run set PIPENV_COLORBLIND=1 on shell startup and use the shell without colours.
Note that the environment variable does not affect colours in the shell, only in Pipenv. Or it should, otherwise it’s a bug.

I use cmd and ps, both with blue backgrounds, one with yellow text, one with white text, as main CLI, with several "on the fly" cmd instances that do a thing and then disappear again. Running pipenv will "hijack" the colors, and will not restore them after it's done. For some code runes, that's fine. For others, that's annoying as hell and having to reset the colors manually after every pipenv run is a chore that can easily be fixed with a simple runtime flag so that even different pipenv runs in the same CLI can be told to either allow colors through, or ignore them entirely.

In that case I think maybe we are solving the wrong problem. It sounds more reasonable to me if Pipenv (actually Crayon, I think) could restore the colours after the command.

I'll be honest: a flag in addition to env var hooking would immediately solve my problem, and open up options for others who are currently stuck with only the environment variable solution, whereas going further and further upstream just keeps the problem unsolved for months while people keep going "we could trivially solve this here and now, for every possible tool that might cause this problem upstream, but let's push for fixes in all of those instead". I've told people that same thing in the past, but ultimately the solution that fixes it "for everything" always wins, I'd rather short-circuit that process and still request that flag here in pipenv, simply because it solves this problem immediately, and for every possible tool (including ones I don't use, and ones that others use who do not have github accounts or even file bugs/fr issues because it's just not a thing they do).

I’m sure you’re familiar with the saying about something in software being trivial does not mean you should implement it, especially not quickly. Bad design lasts forever, the trivial solution right now can be maintenance headache for the rest of your life. Well, maybe not yours, but certainly mine (and all Pipenv maintainers). Please understand the maintainers’ hesitance to implement a change, especially in the user interface. The effect lasts long.


SARS-CoV-2 infection in children and adults - Meyer

Primary Wrangler:
Ida
Secondary Wrangler:

Associated files:
Google Drive:
https://drive.google.com/open?id=1uyLOWN3lvfuZcehgzokAFkLmSudlpVz4
Paper:
https://www.medrxiv.org/content/10.1101/2<IP_ADDRESS>53012v1
GEO:
https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE168215
Project in ingest:
https://contribute.data.humancellatlas.org/projects/detail?uuid=1538d572-bcb7-426b-8d2c-84f3a7f87bb0
Key Events

[ ] Send initial instructions about contributing, including questionnaire and T&Cs
[x] Assign the Primary Wrangler
[ ] Receive T&C form filled out by at least 1 representative of group/project
[ ] Send custom metadata template spreadsheet, how-to guide and directions for submitting data, matrix and cell annotation files.
[ ] Receive spreadsheet (to a sufficient standard to submit dataset)
[x] Receive data, matrix and cell annotation files (to a sufficient standard to submit dataset)
[x] Curate metadata spreadsheet with ontologies
[ ] Curate projetct metadata
[ ] Upload sheet to validate metadata
[ ] Run graph validator & Check linking
[ ] Validate data files
[ ] Ask the Secondary Wrangler for an end-to-end review of the project. Ask the Expertise Wrangler to review specific tabs if needed
[ ] Get final approval of submission from the contributor including release date, metadata spreadsheet and data files
[ ] Convert project data to SCEA format following the SCEA conversion SOP if appropriate

Please track the below as well as the key events:

Track date first spreadsheet received and final spreadsheet sent by editing ticket to include
date next to event.
Track spreadsheet iterations by placing asterisks next to receive spreadsheet event.
Track any metadata issues/tickets made for dataset with a bulleted list of links under
received spreadsheet event. Links should be to the ticket in the metadata repo.

This is a collaborative project with some data coming from Kerstin Meyer and other data coming from Chicago.
Chicago processed data is currently in GEO here https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE168215 and raw data will be in dbGaP, no accession provided as yet.
UK processed data will be in ArrayExpress, with raw data in EGA with no accessions provided as yet. processed data is also on covidcellatlas, I believe the top entry here: https://www.covid19cellatlas.org/index.patient.html but doesn't appear to have all data from the paper.

requested new term for 'tracheal brush' https://github.com/EBISPOT/efo/issues/1026
sent email to contributor of geo data to clarify splitting of patients across samples

moving this to stalled as it relies on submission of project without seq files.

I think this project can be moved forward if the process for managed access projects without sequence files is ticked off this release. I will tag @Wkt8 as I think it makes most sense as he has connections to the contributors and I believe this project is somewhat linked to the Haniffa managed access project and the WSSS in general.
Current progress:
US based metadata & matrices
Metadata and processed matrices were extracted from the GEO accession GSE168215
I communicated with Alexander Misharin to clarify some aspects of the study design, email is saved in files_received as Email_From_Alex_Chicago_samples.txt
I have annotated as much information from GEO and the paper as possible into the spreadsheet.
There is not as much metadata available for these donors, it could be worth reaching out to Kerstin to see if any further information could be gained about these donors so they are more comparable to the UK sourced donors.
UK based metadata:
This project is one that I was communicating with Elena about getting their metadata directly into BioSamples and linking to an EGA submission. As of now I am not aware of whether there is a valid EGA Accession for this project. The paper says processed data and metadata will be in ArrayExpress but no accession given.
Elena sent me a spreadsheet with metadata 05/03/2021, the spreadsheet is saved in the 'files_received' folder on the google drive CV001_biosamples_EGA submission_04032021.xlsx
I extracted metadata from this sheet into the spreadsheet. The spreadsheet isn't complete however so more details would need to be sought from Elena.
I am also not 100% sure about the structure of the experiment, for example there are samples CS1A-PBMC & CS1B-PBMC from donor PC2, I am not clear if these were two separate blood draws from the patient, or 2 separate cell suspensions.
The processed matrix from the UK donors is supposed to be on the COVID cell atlas, but when I downloaded the matrix ucl-sanger_covid_airway.submit_to_czi.cellxgene.20200911_obs_metadata.h5ad and looked at the obs metadata, it only had 3 donors present, AP1-N, AP4-N, AP5-N. I extracted the metadata from this file and saved it in files_received as ucl-sanger_covid_airway.submit_to_czi.cellxgene.20200911_obs_metadata.csv
Next steps

Reach out to Elena to see if metadata for the rest of the donors, samples are available and check in on EGA archiving (might need to have a catchup with Tony/Melanie about the next steps for the BioSamples side of this before that)
Check whether there will be a matrix containing all the donors in the paper ~52 donors all up
Check whether there is more metadata available for the US cohort
Confirm structure in terms of specimens/sample/cell suspension splitting

Tagging @Wkt8 in this ticket as I first assigned myself then saw that Wei has links to these contributors.

@Wkt8 this has now a preview of an accepted publication: 10.1038/s41586-021-04345-x

@ESapenaVentura to secondary review

@idazucchi I have looked into it! Overall looks great :)
Just a couple of minor comments about the protocols:
Dissociation protocol

For the Chicago_dissociation_protocol, can we add a bit more info about the protocol? (It says previously reported protocol, but we don’t have that info in the database so it doesn’t make that much sense)
For the UK_nasal_brushing_enzymatic protocol, there are reference numbers in there (e.g. protocol26), we need to take them out

Library preparation protocols

I think the cell barcode read is not indicated well for all the protocols, shouldn’t it be Read 1?

Thank you @ESapenaVentura for catching the error in the library protocols!
I've fixed the errors, expanded the descrption for Chicago_dissociation_protocol and added an ezymatic dissociation protocol.
The project exported without issue and I filled out the export form


Custom pin reverts to built in marked Location pin after map is loaded

Hi there my code currently loads user's location, then loads spots around them. These spots are some I hold in my local DB. However when I leave the move around the map and go back to an already loaded location, if the location is a "recognized place" (i.e. Vons, Albertsons) on the map the pin changes to this:
.
This seems to be the built in marked location pin on iOS. Is there something I am missing or needs to be handled with custom rendering?

I see this is an ongoing issue when IsClusteringEnabled... I am following this thread but it appears no updates have come from it...https://github.com/TorbenK/TK.CustomMap/issues/292


Documentation improvement: Leading plus in from_addr sample

The documentation for the HTTP API provides a JSON sample for a message without a leading plus in the from_addr field. However the actual data sent appears to contain the leading plus.
https://github.com/praekelt/vumi-go/blob/develop/docs/http_api.rst

@jpaidoussi Ready for review.

:+1:


Breaks with latest version of bluebird

When using bluebird 3.0.1:
module.js:338
throw err;
      ^
Error: Cannot find module 'bluebird/js/main/promise'
    at Function.Module._resolveFilename (module.js:336:15)
    at Function.Module._load (module.js:278:25)
    at Module.require (module.js:365:17)
    at require (module.js:384:17)
    at Object.<anonymous> (/Users/brycehamrick/src/appname/node_modules/knex/lib/promise.js:3:15)
    at Module._compile (module.js:460:26)
    at Object.Module._extensions..js (module.js:478:10)
    at Module.load (module.js:355:32)
    at Function.Module._load (module.js:310:12)
    at Module.require (module.js:365:17)
    at require (module.js:384:17)

Resolved by downgrading to<EMAIL_ADDRESS>

I had this issue as well and went nuclear by deleting my node_modules folder and doing a fresh npm install. You may be able to work around it by doing an npm upgrade, though.

I can confirm that @chrisbroome's solution works!

Awesome! Glad it worked. :smile:

Interesting. I wonder why Promise can't be included directly here by require('bluebird')...

The latest version of bluebird, 3.0.2, has promise.js located at bluebird/js/release/promise.js.
Version 2.10.2 has promise.js located at bluebird/js/main/promise.js.
I'm thinking somehow npm is getting confused an trying to load node_modules/bluebird instead of node_modules/knex/node_modules/bluebird.

Closing old stale issue. If still an issue in latest knex, please reopen. (Cannot reproduce)


wayland-protocols: 1.11 -> 1.13

Motivation for this change
1.12 includes the stable XDG Shell Protocol. 1.13 is more minor.
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[x] Fits CONTRIBUTING.md.

Please target staging for this PR because it is a mass rebuild.

Managed to push before changing the base in github (which triggered review, sorry about that).

@grahamc: maybe the labels (except rebuild) would better not be touched until Borg is smarter.

@GrahamcOfBorg eval
I think the problem here is changing the base doesn't trigger a re-eval, and I think this eval ^ will fix all the wrong labels. Do you have additional cases of the labels being definitely wrong, other than the labels not being fixed when the base is changed?

@vcunat ^ if the changed base triggers a re-eval, will that be sufficiently smart?

@grahamc: even if it were against master, I still can't see why the non-rebuild labels were added.

To clarify the steps I took: I moved my branch to staging, cherry picked the wayland-protocol commit and force pushed before changing the base in github.
 Among them I assume there were commits that had new packages, python and cleanup etc.

Ah, right.  I thought you just retargeted the PR without any commit changes.

Given what has happened and that re-targeting branches now triggers evals, would you like the topic-tagging to be dropped?

@grahamc: no, hopefully it will be OK now, we'll see.


Inability to locate an env file doesn't result in error code

We're using env-cmd in our build process, and the most recent change to require a -f parameter for a custom env file location should have failed our builds.
Unfortunately it didn't, it just displays a warning in our build output, and CircleCI still marked it as a passing build, even though we hadn't actually deployed anything:

Looks like this behaviour is already deprecated (DEP0018) and will be fixed in Node.js someday, but in the meanwhile it would be helpful if env-cmd signalled Node to return a non-zero exit status.

Thank you so much for reporting this! This has been fixed and released in 9.0.2