Documentation on using Arduino LMIC libraries missing

Is this a bug or a feature request?
Bug in Documentation
What did you expect?
Advice to how fields in the "device configuration" screen map to the Arduino LMIC settings of APPKEY, DEVEUI, and APPEUI
What happened?
When creating a new device using OTAA, the following fields are available from the console of the app-server:

Network key
Application key

The DEVEUI field is available from the devices overview list on the application screen.
It would be good if there was documentation on how the Network Key and Application Key map to APPKEY and APPEUI, especially as Application Key is listed as optional for Lora 1.0 devices
What version are your using?
2.0.2
How can your issue be reproduced?
Create a new device in the console
NOTE: I'm happy to write the docs, I'm just not sure what the mapping should be for this sample code:
/*******************************************************************************
 * Copyright (c) 2015 Thomas Telkamp and Matthijs Kooijman
 *
 * Permission is hereby granted, free of charge, to anyone
 * obtaining a copy of this document and accompanying files,
 * to do whatever they want with them without any restriction,
 * including, but not limited to, copying, modification and redistribution.
 * NO WARRANTY OF ANY KIND IS PROVIDED.
 *
 * This example sends a valid LoRaWAN packet with payload "Hello,
 * world!", using frequency and encryption settings matching those of
 * the The Things Network.
 *
 * This uses OTAA (Over-the-air activation), where where a DevEUI and
 * application key is configured, which are used in an over-the-air
 * activation procedure where a DevAddr and session keys are
 * assigned/generated for use with all further communication.
 *
 * Note: LoRaWAN per sub-band duty-cycle limitation is enforced (1% in
 * g1, 0.1% in g2), but not the TTN fair usage policy (which is probably
 * violated by this sketch when left running for longer)!

 * To use this sketch, first register your application and device with
 * the things network, to set or generate an AppEUI, DevEUI and AppKey.
 * Multiple devices can use the same AppEUI, but each device has its own
 * DevEUI and AppKey.
 *
 * Do not forget to define the radio type correctly in config.h.
 *
 *******************************************************************************/

#include <lmic.h>
#include <hal/hal.h>
#include <SPI.h>

// This EUI must be in little-endian format, so least-significant-byte
// first. When copying an EUI from ttnctl output, this means to reverse
// the bytes. For TTN issued EUIs the last bytes should be 0xD5, 0xB3,
// 0x70.
static const u1_t PROGMEM APPEUI[8]={ 0xHEX };
void os_getArtEui (u1_t* buf) { memcpy_P(buf, APPEUI, 8);}

// This should also be in little endian format, see above.
static const u1_t PROGMEM DEVEUI[8]={ 0xHEX };
void os_getDevEui (u1_t* buf) { memcpy_P(buf, DEVEUI, 8);}

// This key should be in big endian format (or, since it is not really a
// number but a block of memory, endianness does not really apply). In
// practice, a key taken from ttnctl can be copied as-is.
// The key shown here is the semtech default key.
static const u1_t PROGMEM APPKEY[16] = { 0xHEX };
void os_getDevKey (u1_t* buf) {  memcpy_P(buf, APPKEY, 16);}

static uint8_t mydata[] = "Hello, world!";
static osjob_t sendjob;

// Schedule TX every this many seconds (might become longer due to duty
// cycle limitations).
const unsigned TX_INTERVAL = 60;

// Pin mapping
const lmic_pinmap lmic_pins = {
    .nss = 10,
    .rxtx = LMIC_UNUSED_PIN,
    .rst = 0,
    .dio = {4, 5, 7},
};

void onEvent (ev_t ev) {
    Serial.print(os_getTime());
    Serial.print(": ");
    switch(ev) {
        case EV_SCAN_TIMEOUT:
            Serial.println(F("EV_SCAN_TIMEOUT"));
            break;
        case EV_BEACON_FOUND:
            Serial.println(F("EV_BEACON_FOUND"));
            break;
        case EV_BEACON_MISSED:
            Serial.println(F("EV_BEACON_MISSED"));
            break;
        case EV_BEACON_TRACKED:
            Serial.println(F("EV_BEACON_TRACKED"));
            break;
        case EV_JOINING:
            Serial.println(F("EV_JOINING"));
            break;
        case EV_JOINED:
            Serial.println(F("EV_JOINED"));

            // Disable link check validation (automatically enabled
            // during join, but not supported by TTN at this time).
            LMIC_setLinkCheckMode(0);
            break;
        case EV_RFU1:
            Serial.println(F("EV_RFU1"));
            break;
        case EV_JOIN_FAILED:
            Serial.println(F("EV_JOIN_FAILED"));
            break;
        case EV_REJOIN_FAILED:
            Serial.println(F("EV_REJOIN_FAILED"));
            break;
            break;
        case EV_TXCOMPLETE:
            Serial.println(F("EV_TXCOMPLETE (includes waiting for RX windows)"));
            if (LMIC.txrxFlags & TXRX_ACK)
              Serial.println(F("Received ack"));
            if (LMIC.dataLen) {
              Serial.println(F("Received "));
              Serial.println(LMIC.dataLen);
              Serial.println(F(" bytes of payload"));
            }
            // Schedule next transmission
            os_setTimedCallback(&sendjob, os_getTime()+sec2osticks(TX_INTERVAL), do_send);
            break;
        case EV_LOST_TSYNC:
            Serial.println(F("EV_LOST_TSYNC"));
            break;
        case EV_RESET:
            Serial.println(F("EV_RESET"));
            break;
        case EV_RXCOMPLETE:
            // data received in ping slot
            Serial.println(F("EV_RXCOMPLETE"));
            break;
        case EV_LINK_DEAD:
            Serial.println(F("EV_LINK_DEAD"));
            break;
        case EV_LINK_ALIVE:
            Serial.println(F("EV_LINK_ALIVE"));
            break;
         default:
            Serial.println(F("Unknown event"));
            break;
    }
}

void do_send(osjob_t* j){
    // Check if there is not a current TX/RX job running
    if (LMIC.opmode & OP_TXRXPEND) {
        Serial.println(F("OP_TXRXPEND, not sending"));
    } else {
        // Prepare upstream data transmission at the next possible time.
        LMIC_setTxData2(1, mydata, sizeof(mydata)-1, 0);
        Serial.println(F("Packet queued"));
    }
    // Next TX is scheduled after TX_COMPLETE event.
}

void setup() {
    Serial.begin(9600);
    Serial.println(F("Starting"));

    #ifdef VCC_ENABLE
    // For Pinoccio Scout boards
    pinMode(VCC_ENABLE, OUTPUT);
    digitalWrite(VCC_ENABLE, HIGH);
    delay(1000);
    #endif

    // LMIC init
    os_init();
    // Reset the MAC state. Session and pending data transfers will be discarded.
    LMIC_reset();

    // Start job (sending automatically starts OTAA too)
    do_send(&sendjob);
}

void loop() {
    os_runloop_once();
}

If you you're willing to create a pull-request with LMIC specific documentation, that would be great! Please note that what you need for OTAA:
(LMIC --> LoRa App Server)

DevEUI --> DevEUI
AppKey --> NwkKey (Per LoRaWAN 1.1 specification, the LoRaWAN 1.0 AppKey is now called NwkKey)
App EUI --> no need to set this in LoRa App Server

OK, thanks,
So at the moment, I have the following in the console:
DevEUI: 00415ee3b9a02d4a
Network Key (made up by me!): 06647d9527683ccfa457c70ab27b4606
Application Key: ALL ZEROS
In the code, I have the following:
// This EUI must be in little-endian format, so least-significant-byte
// first. When copying an EUI from ttnctl output, this means to reverse
// the bytes. For TTN issued EUIs the last bytes should be 0xD5, 0xB3,
// 0x70.
static const u1_t PROGMEM APPEUI[8];
void os_getArtEui (u1_t* buf) { memcpy_P(buf, APPEUI, 8);}

// This should also be in little endian format, see above.
static const u1_t PROGMEM DEVEUI[8]={ 0x4A, 0x2D, 0xA0, 0xB9, 0xE3, 0x5E, 0x41, 0x00 }; // 00415EE3B9A02D4A
void os_getDevEui (u1_t* buf) { memcpy_P(buf, DEVEUI, 8);}

// This key should be in big endian format (or, since it is not really a
// number but a block of memory, endianness does not really apply). In
// practice, a key taken from ttnctl can be copied as-is.
// The key shown here is the semtech default key.
static const u1_t PROGMEM APPKEY[16] = { 0x06, 0x46, 0x7B, 0xB2, 0x0A, 0xC7, 0x57, 0xA4, 0xCF, 0x3C, 0x68, 0x27, 0x95, 0x7D, 0x64, 0x06 };
void os_getDevKey (u1_t* buf) {  memcpy_P(buf, APPKEY, 16);}

The serial port shows the "JOIN" message being sent, but the join does not take place.
I'm sure I'm missing something obvious here, but I can't work out what it is!

@brocaar I'm now wondering whether this is due to the "endian-ness"  of the hex that I'm putting in to the LoraServer console.
In the TTN console, there is the option to select the code as straight hex, or as an array that is either LSB or MSB first.
It is this LSB array that I should be using with the LMIC code, but I can't work out whether I need to perform this same transformation on the hex string from LoraServer that I need to for TTN.

All values in the LoRa App Server interface are presented big endian.

The AppEUI can be set to any value you like. This is only going to matter when you're using an external join-server to activate the OTAA request.

DevEUI "0102030405060708" means you need to use 0x08, 0x07, 0x06, 0x05, 0x04, 0x03, 0x02, 0x01 as the above comment mentions LMIC expects this value as little endian.

AppKey must be used in the original byte order, as the above comment mentions LMIC expects this value as big endian.

There is an open issue to improve this input in LoRa App Server, if you would like to contribute on this.... :-) https://github.com/brocaar/lora-app-server/issues/74

I've kept the above values, and made sure that the "endian-ness" is right:
static const u1_t PROGMEM APPEUI[8]= { 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00 };
static const u1_t PROGMEM DEVEUI[8]={ 0x4A, 0x2D, 0xA0, 0xB9, 0xE3, 0x5E, 0x41, 0x00 };
static const u1_t PROGMEM APPKEY[16] = { 0x06, 0x64, 0x7d, 0x95, 0x27, 0x68, 0x3c, 0xcf, 0xa4, 0x57, 0xc7, 0x0a, 0xb2, 0x7b, 0x46, 0x06 };

I can see the join request on the gateway:
Message Type = Join Request
AppEUI =<PHONE_NUMBER>000000
DevEUI = 00415EE3B9A02D4A
DevNonce = F07B
MIC = AE6A616A

and the gateway is clearly sending packets to LoraServer because I can see it in the Lora Server Console.
I do see a "join accept" packet sent back, but the device doesn't appear to acknowledge it:
Message Type = Join Accept
AppNonce = 180846
NetID = D73F5E
DevAddr = E88BBA7D
MIC = 06C3747C

@brocaar looks like the issue isn't with the LMIC library, but with my setup somewhere:
Aug 22 19:32:21 mbc-loranode1 loraserver[4761]: time="2018-08-22T19:32:21Z" level=info msg="backend/gateway: rx packet received"
Aug 22 19:32:21 mbc-loranode1 loraserver[4761]: time="2018-08-22T19:32:21Z" level=info msg="packet(s) collected" dev_eui=00415ee3b9a02d4a gw_count=1 gw_macs=<GWMAC> mtype=JoinRequest
Aug 22 19:32:21 mbc-loranode1 lora-app-server[4747]: time="2018-08-22T19:32:21Z" level=info msg="js: request received" message_type=JoinReq receiver_id=0000000000000000 sender_id=010203 transaction_id=3455401306
Aug 22 19:32:21 mbc-loranode1 lora-app-server[4747]: time="2018-08-22T19:32:21Z" level=info msg="device-keys updated" dev_eui=00415ee3b9a02d4a
Aug 22 19:32:21 mbc-loranode1 lora-app-server[4747]: time="2018-08-22T19:32:21Z" level=info msg="js: sending response" message_type=JoinAns receiver_id=010203 result_code=Success sender_id=0000000000000000 transaction_id=3455401306
Aug 22 19:32:21 mbc-loranode1 loraserver[4761]: time="2018-08-22T19:32:21Z" level=info msg="device-queue flushed" dev_eui=00415ee3b9a02d4a
Aug 22 19:32:21 mbc-loranode1 loraserver[4761]: time="2018-08-22T19:32:21Z" level=info msg="device-session saved" dev_addr=07ab7b1f dev_eui=00415ee3b9a02d4a
Aug 22 19:32:21 mbc-loranode1 loraserver[4761]: time="2018-08-22T19:32:21Z" level=info msg="device-activation created" dev_eui=00415ee3b9a02d4a id=16
Aug 22 19:32:21 mbc-loranode1 loraserver[4761]: time="2018-08-22T19:32:21Z" level=info msg="backend/gateway: publishing tx packet" qos=0 topic=gateway/<MAC>
```

I can see the LoraFrames coming in to the UI, as request/accept JOIN, I just don't see the device ever connect.

I'll close this issue and move it to the forums, once I've got LoraServer connecting properly I'll revisit this and update the docs.


Monitor Tree Can't View Server Status

Click Monitor Tree-> Servers-> Can display Server Status in the Server List, if you click inside server,the status is null.
When you create the Dash board, the Dash board get the server status from the server inside status information, it will display null.
Platform: Windows 10
Weblogic Version: <IP_ADDRESS>.4.0
Ext version:4.0
Version:2.4.4

self resolved

self resolved


UX: when only hour has been selected, min is displayed but the selection is not taken into account

Hi,
So if we just select an hour, the clock will display eg. '13:00', and if we click outside the overlay, the value will not be taken into account.
I think it should either display '13:__' to make obvious that minutes need to be selected, or valid the fact that minutes default to 00 without user action.
What do you think?

I need this :/. It's a bit annoying the fact that you have to select minutes even though you just want to keep the same value.


[Removal]:  Tame Rainbow Sloth

Hotspot Name
tame-rainbow-sloth
Hotspot b58 Address
112sWWYWje8PBXQzfqFQXbyKLBcH3DGNNp3u4gbkAGYSqGknzy8q
Discord Handle
No response
Hotspot Manufacturer
Pisces
Removal Reason
I bought it from the second hand market and don't know why it's on the blacklist
Modifications
No
Extra forwarders
No
Extra antennas
No
Additional Information
No response

We have taken into account this removal request in the next iteration of the denylist. It may take some time before it's processed. This is an automated comment due to the volume of addition and removal requests. A future tool is in the works to provide more insight into the analysis approach made by the Helium team and community members that maintain this list.


running gulp

Error: Cannot find module 'fabric-client/lib/Peer.js
  at Function.Module._resolveFilename (module.js:470:15)     at Function.Module._load (module.js:418:25)     at Module.require (module.js:498:17)     at require (internal/module.js:20:19)     at module.exports (/Users/nasimk/git/marbles/utils/fc_wrangler/deploy_cc.js:8:13)     at module.exports (/Users/nasimk/git/marbles/utils/fc_wrangler/index.js:6:43)     at Object.<anonymous> (/Users/nasimk/git/marbles/app.js:34:50)     at Module._compile (module.js:571:32)     at Object.Module._extensions..js (module.js:580:10)     at Module.load (module.js:488:32)

your SDK install was not successful or something like that.  did you run both npm install steps?

i'll give it another shot - i'm on a mac.

In my mac environment I need to use node v6.9.1 . I fail "npm install" with node v7.
I used nvm to use lower node version.

closing, inactivity


Startup Replacement

Here is the project:
JSONClientSRC.zip
signtool.zip
[Important]Must to be signed as system app to run.
Question is:

who called JSONClient during the startup procedure?
what else has been done before/after activation?
what is happening while activating? Some critical part is missing in decompilated code.
protected void onPreExecute() {
}
//Activate code authentication happens here
protected Object doInBackground(Object[] params_obj) {
// :( Parsing error. Please contact me.
}
Also String.format('/sbin/eid recv {0}', result), this result is definitely missing, needs some debugging.
adb needs to be enabled in hacked boot.img for debugging.

Conclusion: Emulate the startup process means replace the original components without protection. JSONClient.apk might be just a part of it.

great job!
for 3: maybe try another smali decompiler to see those part. maybe https://bytecodeviewer.com/ ?

@ziegfeld wow, thx, this information is cool, seems to have some scripts inside.

attatched partition mapping:
lrwxrwxrwx root     root              2018-12-14 16:31 DTIM -> /dev/block/mmcblk0p4
lrwxrwxrwx root     root              2018-12-14 16:31 MEP2 -> /dev/block/mmcblk0p3
lrwxrwxrwx root     root              2018-12-14 16:31 MRD -> /dev/block/mmcblk0p1
lrwxrwxrwx root     root              2018-12-14 16:31 MRD1 -> /dev/block/mmcblk0p2
lrwxrwxrwx root     root              2018-12-14 16:31 NVM -> /dev/block/mmcblk0p12
lrwxrwxrwx root     root              2018-12-14 16:31 boot -> /dev/block/mmcblk0p8
lrwxrwxrwx root     root              2018-12-14 16:31 cache -> /dev/block/mmcblk0p14
lrwxrwxrwx root     root              2018-12-14 16:31 ddat -> /dev/block/mmcblk0p16
lrwxrwxrwx root     root              2018-12-14 16:31 diag -> /dev/block/mmcblk0p15
lrwxrwxrwx root     root              2018-12-14 16:31 misc -> /dev/block/mmcblk0p10
lrwxrwxrwx root     root              2018-12-14 16:31 pbootloader -> /dev/block/mmcblk0p7
lrwxrwxrwx root     root              2018-12-14 16:31 radio -> /dev/block/mmcblk0p11
lrwxrwxrwx root     root              2018-12-14 16:31 rbootloader -> /dev/block/mmcblk0p5
lrwxrwxrwx root     root              2018-12-14 16:31 recovery -> /dev/block/mmcblk0p6
lrwxrwxrwx root     root              2018-12-14 16:31 secure -> /dev/block/mmcblk0p13
lrwxrwxrwx root     root              2018-12-14 16:31 system -> /dev/block/mmcblk0p9
lrwxrwxrwx root     root              2018-12-14 16:31 userdata -> /dev/block/mmcblk0p17

@sekkit, cc @HappyZ  @p4sw2wd
Hi, if you read Chinese, here's a thread about bypassing Kindle Android boot activation from Taobao kdroid, hopefully it would help us here for DPT:
https://www.hi-pda.com/forum/viewthread.php?tid=2313206&extra=page%3D1

this is great news. But I can't get access to hi-pda. And right now DPT has been returned to retailer because of WiFi issue. I will try reimplement that method when next DPT arrived.

This topic is about how to crack a kindle which installed android by the same tabao seller who crack dpt. The main step is:
1.mount system image.
2.modify /system/framework/services.jar to bypass signature check.
3.modify JSONClient.jar so any password can pass password check.

@sekkit please do register an account for access, or let me know what the problem is and let me see how to help.
Because, from that bbs thread, updates (esp. from post no. 110) indicated the screenshot versiom is not 100% accurate and pointed out some errors/typos.

@ziegfeld I registered for almost half a year, but not approved by admin yet.
After flashing the third boot.img from another issue, my DPT can no longer connect to WiFi by DigitalPaperApp, even restored to original firmware. Not a clue for what might've happened.

@ziegfeld can u export that page as pdf so information could be more specific?

@ziegfeld hi Im going to repimplement these steps, could u export post no. 110 as pdf and send it to me? seckkit gmail#com

@sekkit sorry for late reply. I forgot it then the holiday came. Glad that you got it! You can ask questions directly there this way!

Tried to mod the apk. But the official activation network is down. Even with the original firmware. Anyone has the fw from /dpt2-user ? Just need to clarify whether the author opened another service.
More mods need to be done.


Introduce tracking of individual sources, using the Remote Asset API

See original issue on GitLab
In GitLab by [Gitlab user @sstriker] on Mar 25, 2020, 22:08
We can reduce the load and reliance on additional services by leveraging the Remote Asset API.  For example, instead of having all clients poll git services, the FetchService.FetchDirectory API is used to resolve the commit at a certain branch.  As clients all track to the same revision, in cache hits are more likely for sources, artifacts and actions.
To support this we need to extend the Source Plugin API to return the list of URIs and qualifiers as needed by the FetchService.  Specifically:

FetchDirectoryRequest.uris is the complete set of URLs that represent the content of the source.  This is the full set after alias expansion.  For example: git.example.com/foo/bar.git and git-mirror.example.com/foo/bar.git.
FetchDirectoryRequest.qualifiers is the minimal set of qualifiers that identifies the source.  This must exclude the commit for the branch.  For example:

vcs.branch = master
resource_type = application/x-git

In the response the client will learn the Digest of the source as well as all other qualifiers the service knows about.  This would include identifying information the source plugin would use in its ref. For example:

FetchDirectoryResponse.uri is the URL that matched, that represent the content of the source.  For example: git.example.com/foo/bar.git.
FetchDirectoryResponse.qualifiers is the complete set of qualifiers associated with the source.  For example:

vcs.commit = b5123b1bb2853393c7b9aa43236db924d7e32d61
resource_type = application/x-git
vcs.branch = master

Behavior should be configurable to support the following use cases:

client does not use the Remote Asset API to track sources, and only uses the source plugin native track
client uses the Remote Asset API to track sources, and falls back to using the source plugin native track
client uses the Remote Asset API to track sources, and does not fall back to using the source plugin native track
client uses the Remote Asset API to push sources, after using the source plugin native track and fetch
client does not use the Remote Asset API to push sources
client is configured to only accept results younger than a certain age.

See also:https://mail.gnome.org/archives/buildstream-list/2020-February/msg00000.html

In GitLab by [Gitlab user @sstriker] on Mar 25, 2020, 22:08
Given the Source Plugin API changes suggesting this for 2.0

In GitLab by [Gitlab user @sstriker] on Mar 25, 2020, 22:08
Given the Source Plugin API changes suggesting this for 2.0

In GitLab by [Gitlab user @cs-shadow] on Mar 26, 2020, 12:18
[Gitlab user @sstriker] thanks for the write-up.
How do you envision this working for SourceTransforms (like pip source etc) that require access to other sources listed for that element? Or would such sources have to resort back to how they are handled now?

In GitLab by [Gitlab user @cs-shadow] on Mar 26, 2020, 12:18
[Gitlab user @sstriker] thanks for the write-up.
How do you envision this working for SourceTransforms (like pip source etc) that require access to other sources listed for that element? Or would such sources have to resort back to how they are handled now?

In GitLab by [Gitlab user @sstriker] on Mar 26, 2020, 21:28
Excellent question.
It really depends on how specialized of a source plugin we are willing to make.  And how much there is to gain when tracking these types of sources - in the pip source case, I imagine there might be quite a bit of work that can be avoided.  That is, going from a requirements.txt to a frozen set of requirements.
I imagine that during tracking source plugins that have indicated they need the previous sources, to be passed the Tree (vDirectory?) of the previous sources at the start of tracking.  The pip source plugin could use the digest of the requirements.txt file as a qualifier, and then maybe use these for tracking:

buildstream.build.plugins.pip.requirements_digest = digest
buildstream.build.plugins.pip.additional_packages = package1,package2,...

The ref for a pip source plugin is the frozen requirements, I think that would result in push qualifiers would be:

buildstream.build.plugins.pip.requirements_digest = digest
buildstream.build.plugins.pip.additional_packages = package1,package2,...
buildstream.build.plugins.pip.frozen = frozen_requirements

In a setup like this, you would have a central BuildStream instance be taking care of the tracking:

whenever the source changes, and/or
periodically, as the requirements.txt may resolve to a new frozen set, and/or
triggered because it is known that a package in requirements.txt for this project has a new version

And each other client would be tracking to the same set of frozen deps the central instance has.  For the pip source plugin this has two implications:

BuildStream clients will only spend time doing native tracking when the requirements.txt or element configuration changes for this source

BuildStream clients will only then need file content of the previous sources
BuildStream clients will only then hit pypi endpoints

In short, I think that there might be source plugins that require previous sources, where it makes sense to resort to native tracking.  In other cases, the specialized handling may be worth it.
Make sense?

In GitLab by [Gitlab user @sstriker] on Mar 26, 2020, 21:28
Excellent question.
It really depends on how specialized of a source plugin we are willing to make.  And how much there is to gain when tracking these types of sources - in the pip source case, I imagine there might be quite a bit of work that can be avoided.  That is, going from a requirements.txt to a frozen set of requirements.
I imagine that during tracking source plugins that have indicated they need the previous sources, to be passed the Tree (vDirectory?) of the previous sources at the start of tracking.  The pip source plugin could use the digest of the requirements.txt file as a qualifier, and then maybe use these for tracking:

buildstream.build.plugins.pip.requirements_digest = digest
buildstream.build.plugins.pip.additional_packages = package1,package2,...

The ref for a pip source plugin is the frozen requirements, I think that would result in push qualifiers would be:

buildstream.build.plugins.pip.requirements_digest = digest
buildstream.build.plugins.pip.additional_packages = package1,package2,...
buildstream.build.plugins.pip.frozen = frozen_requirements

In a setup like this, you would have a central BuildStream instance be taking care of the tracking:

whenever the source changes, and/or
periodically, as the requirements.txt may resolve to a new frozen set, and/or
triggered because it is known that a package in requirements.txt for this project has a new version

And each other client would be tracking to the same set of frozen deps the central instance has.  For the pip source plugin this has two implications:

BuildStream clients will only spend time doing native tracking when the requirements.txt or element configuration changes for this source

BuildStream clients will only then need file content of the previous sources
BuildStream clients will only then hit pypi endpoints

In short, I think that there might be source plugins that require previous sources, where it makes sense to resort to native tracking.  In other cases, the specialized handling may be worth it.
Make sense?

In GitLab by [Gitlab user @sstriker] on Mar 27, 2020, 20:51
[Gitlab user @cs-shadow]: After a night's sleep, an updated answer
A not unimportant benefit I left out for this pip source example:

BuildStream clients wouldn't need to have python host tools to track (caveat: only when not falling back to native tracking)

Now, if we wanted to make the example of pip source more generic, we could document qualifiers in the Remote Asset API spec.
Let's assume that for the Remote Asset API in combination with PyPI, we use the following convention:

We use FetchDirectory/PushDirectory
The root_directory_digest refers to a directory in CAS with all the downloaded sdists from PyPI
The uris are the PyPI index URLs we can retrieve these sdists from
We have the following generalized qualifiers

pypa.pip.requirements: sorted list of packages with optional version information, eg. package1,package2,package3==0.5.4
pypa.pip.contraints: sorted list of packages with version information, eg. package1==0.1.2,package2==2.4.0
pypa.pip.requirements.frozen: sorted list of packages with version information, eg. package1==0.1.2,package2==2.4.0,package3==0.5.4

We have the following BuildStream specific qualifiers (these may be made generic as well, open to suggestions)

buildstream.build.plugins.pip.requirements.digest: digest of the requirements.txt
buildstream.build.plugins.pip.constraints.digest: digest of constraints.txt

fetch
On a bst fetch operation we already have a "ref", which we map to the pypa.pip.requirements.frozen qualifier.  We call FetchDirectory with only that qualifier.
track
On a bst track operation we want the following to happen.

We attempt FetchDirectory with qualifiers specialized for previous sources:

buildstream.build.plugins.pip.requirements.digest
buildstream.build.plugins.pip.constraints.digest
If we get a result back we're done, and can use the returned pypa.pip.requirements.frozen as ref

We attempt FetchDirectory with general qualifiers:

pypa.pip.requirements
pypa.pip.contraints
For this to work we do actually need to get the file contents for requirements.txt and constraints.txt.  These should be available from CAS, and auto-hydrated when accessed through vDirectory(?).
If we get a result back we're done, and can use the returned pypa.pip.requirements.frozen as ref

We fall back to native tracking

As you can see this reveals we do need a protocol for the Source Plugin API in combination with Remote Asset tracking and previous sources.  Step 1 would not be relevant to plugins that don't have previous sources.
push
On a bst push operation we send all of the qualifiers:

pypa.pip.requirements
pypa.pip.contraints
buildstream.build.plugins.pip.requirements.digest
buildstream.build.plugins.pip.constraints.digest
pypa.pip.requirements.frozen

In GitLab by [Gitlab user @sstriker] on Mar 27, 2020, 20:51
[Gitlab user @cs-shadow]: After a night's sleep, an updated answer
A not unimportant benefit I left out for this pip source example:

BuildStream clients wouldn't need to have python host tools to track (caveat: only when not falling back to native tracking)

Now, if we wanted to make the example of pip source more generic, we could document qualifiers in the Remote Asset API spec.
Let's assume that for the Remote Asset API in combination with PyPI, we use the following convention:

We use FetchDirectory/PushDirectory
The root_directory_digest refers to a directory in CAS with all the downloaded sdists from PyPI
The uris are the PyPI index URLs we can retrieve these sdists from
We have the following generalized qualifiers

pypa.pip.requirements: sorted list of packages with optional version information, eg. package1,package2,package3==0.5.4
pypa.pip.contraints: sorted list of packages with version information, eg. package1==0.1.2,package2==2.4.0
pypa.pip.requirements.frozen: sorted list of packages with version information, eg. package1==0.1.2,package2==2.4.0,package3==0.5.4

We have the following BuildStream specific qualifiers (these may be made generic as well, open to suggestions)

buildstream.build.plugins.pip.requirements.digest: digest of the requirements.txt
buildstream.build.plugins.pip.constraints.digest: digest of constraints.txt

fetch
On a bst fetch operation we already have a "ref", which we map to the pypa.pip.requirements.frozen qualifier.  We call FetchDirectory with only that qualifier.
track
On a bst track operation we want the following to happen.

We attempt FetchDirectory with qualifiers specialized for previous sources:

buildstream.build.plugins.pip.requirements.digest
buildstream.build.plugins.pip.constraints.digest
If we get a result back we're done, and can use the returned pypa.pip.requirements.frozen as ref

We attempt FetchDirectory with general qualifiers:

pypa.pip.requirements
pypa.pip.contraints
For this to work we do actually need to get the file contents for requirements.txt and constraints.txt.  These should be available from CAS, and auto-hydrated when accessed through vDirectory(?).
If we get a result back we're done, and can use the returned pypa.pip.requirements.frozen as ref

We fall back to native tracking

As you can see this reveals we do need a protocol for the Source Plugin API in combination with Remote Asset tracking and previous sources.  Step 1 would not be relevant to plugins that don't have previous sources.
push
On a bst push operation we send all of the qualifiers:

pypa.pip.requirements
pypa.pip.contraints
buildstream.build.plugins.pip.requirements.digest
buildstream.build.plugins.pip.constraints.digest
pypa.pip.requirements.frozen

In GitLab by [Gitlab user @cs-shadow] on Apr 1, 2020, 22:39
Many thanks for the detailed response.
The overall plan seems good to me. On the high-level design, I only have one comment/quesiton.
I'm unsure about adding the input requirements as a qualifier. When tracking the same exact same requirements at different times, we are not guaranteed the same result. This will depend on how often the dependencies change, but will see heavy churn with lots of unbounded dependencies.
This is because pip will pick the latest version each time. So, if there is a new release of any dependency between two track operations, the output will be different.
Imagine this scenario:

Package Ponies releases 1.0.
User had Ponies >= 1.0 as their only requirement in element Unicorn.
User tracks Unicorn and pushes Ponies to the cache.
Ponies now releases 2.0.

Now, if we use pypa.pip.requirements as a qualifier, we will get back the cached version (version 1.0) in form of pypa.pip.requirements.frozen. However, if the user does native tracking at this point, they will get version 2.0. As such, it is not good for reproducibility.
This is not a general problem with the plan itself, but specifically with pip source. However I think it may extend to other popular package managers as well, since most of them pick the latest available version.
Having said that some other package managers (like the one in Go) aim to provide this guarantee by picking the oldest allowed version.

A couple of minor comments:

BuildStream clients wouldn't need to have python host tools to track (caveat: only when not falling back to native tracking)

This is pretty neat.

sorted list of packages with optional version information, eg. package1,package2,package3==0.5.4

I don't think it matters here but I'd just mention it that we will likely have duplicates in this list. When merging differnet requirement files and inline requirements, BuildStream relies on pip's logic to satisfy all the constraints. So, a single packge may appear multiple times, like (package1,package1>0.1 etc).

pypa.pip.contraints: sorted list of packages with version information

Maybe I'm missing something but I'm not sure if I understand what are you referring to by "constraints" here.
The way I understand it, pypa.pip.requirements is the set of input requirements and pypa.pip.requirements.frozen is the result of tracking on the input requirements. What are these constraints then?
Are they just additional input, or something else?

In GitLab by [Gitlab user @cs-shadow] on Apr 1, 2020, 22:39
Many thanks for the detailed response.
The overall plan seems good to me. On the high-level design, I only have one comment/quesiton.
I'm unsure about adding the input requirements as a qualifier. When tracking the same exact same requirements at different times, we are not guaranteed the same result. This will depend on how often the dependencies change, but will see heavy churn with lots of unbounded dependencies.
This is because pip will pick the latest version each time. So, if there is a new release of any dependency between two track operations, the output will be different.
Imagine this scenario:

Package Ponies releases 1.0.
User had Ponies >= 1.0 as their only requirement in element Unicorn.
User tracks Unicorn and pushes Ponies to the cache.
Ponies now releases 2.0.

Now, if we use pypa.pip.requirements as a qualifier, we will get back the cached version (version 1.0) in form of pypa.pip.requirements.frozen. However, if the user does native tracking at this point, they will get version 2.0. As such, it is not good for reproducibility.
This is not a general problem with the plan itself, but specifically with pip source. However I think it may extend to other popular package managers as well, since most of them pick the latest available version.
Having said that some other package managers (like the one in Go) aim to provide this guarantee by picking the oldest allowed version.

A couple of minor comments:

BuildStream clients wouldn't need to have python host tools to track (caveat: only when not falling back to native tracking)

This is pretty neat.

sorted list of packages with optional version information, eg. package1,package2,package3==0.5.4

I don't think it matters here but I'd just mention it that we will likely have duplicates in this list. When merging differnet requirement files and inline requirements, BuildStream relies on pip's logic to satisfy all the constraints. So, a single packge may appear multiple times, like (package1,package1>0.1 etc).

pypa.pip.contraints: sorted list of packages with version information

Maybe I'm missing something but I'm not sure if I understand what are you referring to by "constraints" here.
The way I understand it, pypa.pip.requirements is the set of input requirements and pypa.pip.requirements.frozen is the result of tracking on the input requirements. What are these constraints then?
Are they just additional input, or something else?


Question: The task includes an option with an undefined variable.

Here is my config
- hosts: servers
  become: false
  vars:
    rs_port:                "27017"
    mongodb_type:           "daemon"
    replica_set_name:       "rs0"
    cluster_role:           "shardsvr"
    rs_server_group_id:     "rs_mongo"
    app_users:
      - { db_name: "", user: "", pwd: "", roles: ["readWrite", "userAdmin"] }
  roles:
    - { role: pgkehle.mongodb, flags: ['init'] }
    - { role: pgkehle.mongodb, flags: ['config_save'] }
    - { role: pgkehle.mongodb, flags: ['storage_init'] }
    - { role: pgkehle.mongodb, flags: ['rs_init'] }

I'm getting the following error
https://asciinema.org/a/CksDLsDWChibot2Ay7P1WQRpI
I'm wondering if I need to structure my inventory file in specific way, to include missing variables as attributes?

Yes, you do have to configure your inventory file properly.   This playbook is intended to use a group of servers from your inventory, and allow you to designate which group you would like to use, via the rs_server_group_id variable.
There will be more info on how to do that on the ansible site of course, but as a quick and dirty example:
Inventory Directory Structure
inventories
├── app-servers
├── group_vars
|   ...
│   ├── app-cfg-servers.yml
│   ├── app-shard-servers.yml
│   ├── app-shd00-servers.yml
│   ├── app-shd01-servers.yml
│   ...

Set variables for one specific group example
> cat inventories/group_vars/app-shd00-servers.yml
---

# MongoDB Cluster Definitions - Group Settings
replica_set_name:     "app-shd00"
replica_set_group_id: "app-shd00-servers"

Defining the host names in the groups
> cat inventories/app-servers

...
[app-cfg-servers]
app-cfg-rs0
app-cfg-rs1
app-cfg-rs2

[app-shd00-servers]
app-shd00-rs0
app-shd00-rs1
app-shd00-rs2

[app-shd01-servers]
app-shd01-rs0
app-shd01-rs1
app-shd01-rs2

[app-shard-servers:children]
app-shd00-servers
app-shd01-servers

...

Thus for that example of the shard server 0 replica set:
- hosts: servers
  become: false
  vars:
    rs_port:                "27017"
    mongodb_type:           "daemon"
    replica_set_name:       "rs0"
    cluster_role:           "shardsvr"

-- Do not set this here.  It is set in prep-vars.yml
--    rs_server_group_id:     "rs_mongo"

-- These are set in 'app-shd00-servers.yml'
--	replica_set_name:     	"app-shd00"
--	replica_set_group_id: 	"app-shd00-servers"

    app_users:
      - { db_name: "", user: "", pwd: "", roles: ["readWrite", "userAdmin"] }
  roles:
    - { role: pgkehle.mongodb, flags: ['init'] }
    - { role: pgkehle.mongodb, flags: ['config_save'] }
    - { role: pgkehle.mongodb, flags: ['storage_init'] }
    - { role: pgkehle.mongodb, flags: ['rs_init'] }

...which is used in prep-vars.yml:
  - name: Set the rs group name from cfg_server_name
    set_fact:
      rs_server_group_id: "{{ cfg_server_group_id }}"
    when:
      - cfg_server_group_id is defined

  - name: Set the rs group name from replica_set_name
    set_fact:
      rs_server_group_id: "{{ replica_set_group_id }}"
    when:
      - replica_set_group_id is defined

It's been 11 months or so since this was opened.  I'm not sure if this is still an outstanding issue.  Please confirm.

Hi @pgkehle . Could you provide path where will place this configuration inside ansible project, thank you
cfg_server:
  name: "my-cfg" # (Required)
  group: "my-cfg-servers" # (Required) Always pass in the group id used for the config servers

replica_set:
  name: "my-cfg" # name of the replica set for the config server (prefix of fqdn)
  group: "my-cfg-servers" # group name for all servers in the replica set

Replica Set Only
cluster_role: "replicaSet"

Router Server
cluster_role: "router"

Config Server
cluster_role: "config"
replica_set:
  name: "my-cfg" # name of the replica set for the config server (prefix of fqdn)
  group: "my-cfg-servers" # group name for all servers in the replica set

Shard Server
cluster_role: "shard"
replica_set:
  name: "db-data" # name of the replica set for the shard server (prefix of fqdn)
  group: "db-data-servers" # group name for all servers in the replica set

What is the error message you are seeing, @PhanSon95?
Have you configured your inventories appropriately, and are targeting one server?  ie logs would help.


COM-60: Additional features for Tabs

This PR adds new Features to the Tabs and RouterTabs Components. (Router-)Tabs can now contain the Divider component as children to separate Tabs. Additionally, one Tab or RouterTab can now show a tab icon, a status with icon and a tooltip.
Acceptance criteria

[x] Add changeset

Screenshots
Implementation
Tabs

RouterTabs

@jomunker, @thomasdax98, @johnnyomair what's the status here? I would like to use this feature in a project and could help out, if something still needs to be done.

@jomunker Can you please rebase this branch and fix the conflicts?

This PR has commits from #1351 -> maybe you need a rebase? 🤔

Rebase doesn't work since it has bin squashed 🤷🏻♂  Should I just leave it like this? Won't change anything I guess and we're squashing here anyways...

The icons are also added in #1462, right?

Nope

We discussed this in today's meeting: There will be some UX changes (for instance, no tooltip), which need to be addressed in this PR before merging. Marking as draft for now.


Add code example for in model.destroy

Could you please add this example to the JSDoc doclet in src/model.js so we keep parity with upcoming generated docs? Thanks.

No problem, I have updated the file src/model.js with the example.
Side note, I tried to generate the docs on my local machine using npm run babel && rm -rf ./docs/html/* && jsdoc --configure ./scripts/jsdoc.config.json and it erased all my changes. Do you have any advice for generating the docs on a local machine?

...it erased all my changes.

Your changes to what?

All of my changes I made in the file src/model.js. Then I made the changes again, but did not run the jsdoc command, then resubmitted the pull request.

@stridentbean, I think that you are mistaken. Calling npm run jsdoc will update /lib/model.js, but leaves /src/model.js unchanged. I just checked out your branch and confirmed that this is the case locally. Could you try that again and commit the changes to /lib/model.js too?

@rhys-vdw, I get it now. Thanks for your patience!

Thanks, @stridentbean.


Unit split form: add ability to split unit into more than 2 child records

Currently, split form only allows the creation of 2 child records.
We would like to enable the ability to split into any number of child records, but still enforce the same validation that all of the child record unit counts add up to the total unit count of the original record.
It would be nice to follow similar UI standards where users can add as many split child records as possible, but it defaults to 2 records.

@ccornick this is actually a major refactor on both the UI and the API. There was a lot of logic that assumed split = 2. Is this something you really want us to do? The original requirement was that we only spit into 2 units.


[WEB][Image]

When we insert an image in a media card (in a story), it is supposed to be displayed then when the story is called.
But the current situation is that the bot doesn't display it at all we get an error 404.
The loophole would be to go in the "document section" where the image got uploaded, take it's link and insert the link in an <a href>, but that's not what we want, this solution is a bit too technical for métier teams.

It is not displayed in the bot ? With which connector ? Because it works for other bots :)

It's on the web connector, here is one example of what I get as output:

When I click on the link in the first picture, it does display correctly:

Maybe then this part wasn't made to handle pdf content, so I think it should be restricted to images as in .png, .jpeg and so on.
To avoid any confusion (if the previous sentence is correct)


Local file persistence adapter

Other than IndexedAdapter, there are no persistence adapters ready OOTB.
Would you consider adding generic adapters, such as to a local file?

My bad, didn't see the saveDatabase function...
Maybe make it stand out more in the lokijs.org docs?

So, eventually, you set the adapter in the Loki ctor? Am I the only one who got this one wrong at first?

Hey @moshewe,
Normally Loki picks a available adapter depending on the Enviroment (e.g. localstorage in the browser or an Filesystem adapter in nodejs).
But you can also provide your own adapter.
For example the LokiIndexAdapter:
var db = new loki('MyDb', {
  persistenceAdapter: new LokiIndexedAdapter('MyDb')
});

I understood that, eventually, but I don't think it was clear enough from the docs.

Our documentation is currently split between a very nice 'reference' type page here :
http://lokijs.org/#/docs
As well as our wiki pages, for instance this one in particular for your issue :
https://github.com/techfort/LokiJS/wiki/LokiJS-persistence-and-adapters
We are currently in the process of tweaking jsdoc syntax so that may be the best place to recommend changes to our 'reference' style documentation.  Feel free to contribute PRs to loki or or lokijs.org github or offer suggestions for wiki pages.

closing this, we have new jsdoc preview as well


Missing word "through" in "Conclusion" paragraph && "submit" misspelled

Request type

[ ] New documentation
[X] Correction or update

Details
URL:
https://developer.mozilla.org/en-US/docs/Learn/HTML/Forms/Sending_and_retrieving_form_data
Grammer: missing word "through" in "Conclusion" paragraph && "submit" misspelled
"If you've worked your way ##through## these tutorials in order, you now know how to markup, style, and ##sumbit## a form.
hth,
C K Ashby

Fixed, and fixed ... thanks for reporting!


Speeding up vrf computation by directly producing output without proof

In Directory::publish(), we are doing some unnecessary computation when evaluating VRFs, by first generating the proof, then turning the proof into the VRF output. It is faster to just generate the output directly. Also added a benchmark for a single VRF evaluation on a random label, plus some small performance improvements to generating ECVRF proofs.
See the following benchmarks...
 Nice catch! (Just needs a clippy adjustment and good-to-go!)


fix the issue of index directory containing schema

What changes were proposed in this pull request?
Currently, if OapConf.OAP_INDEX_DIRECTORY contain the schema, for example: file:/tmp, the final index directory path is not  "indexDirectory+tablePath" or "indexDirectory+tablePath+partitionPath".
Reason in IndexUtils#generateTempIndexFilePath, we get the wrong tablePath, which cause the wrong partitionPath in the following:
val tablePath= Path.getPathWithoutSchemeAndAuthority( outputPath).toString.replaceFirst(indexDirectory.toString, "")
For example indexDirectory = file:/tmp, outputPath = file:/tmp/tablePath
tablePath = "/tmp/tablePath".replace("file:/tmp", ""). so the tablePath is "/tmp/tablePath" not "/tablePath". So we should remove the schema of indexDirectory when get the tablePath:
val tablePath= Path.getPathWithoutSchemeAndAuthority( outputPath).toString.replaceFirst(Path.getPathWithoutSchemeAndAuthority(new Path(indexDirectory)).toString, "")
How was this patch tested?
UT in OapDDLSuite.scala

@LuciferYang@linhong-intel  @carsonwang @lee-lei @gczsjdy , please help me review. Thanks.

LGTM

Merging this. Thanks all!


test: Add parens to print for Python3 compatibility

Missing parens in inline Python calls from tests.
/cc: @compnerd

@swift-ci please test


router: missing logic for peering links

Since moving to the new heaver v2, router's logic cannot handle forwarding on peering links. Specifically, routers cannot discover the other side's interface IDs, therefore no paths across a peering link can be assembled.
@matzf explained me why:
"The link liveness used to be checked by "IFID" keep-alive messages sent between the control services of neighboring ASes; these keep-alive messages served the double purpose to announce the local interface ID to the neighbor. For peering links, this remote interface ID needs to be known during beaconing (because the path segments entries will be matched up based on these interface IDs when stitching the path segments together for a peering shortcut).
The new keep-alive mechanism only works on the level of border routers (called bidirectional forwarding detection); the control service was removed from this (which makes a lot of sense!). Now there is no longer any mechanism to announce these interface IDs to be used for beaconing for peering links and so this is now missing. I'm not sure whether there is an existing design to add this missing piece back -- maybe the idea is that this remote interface ID just needs to be supplied in the configuration, idk.
Also, the (open source) router currently just doesn't handle forwarding for the peering link case -- for the "new" header v2, this logic changed significantly and it has not been implemented yet. But here, it's quite clear what would need to be done, I believe. "

Yes, in the legacy keep alive mechanism we involved both the control service and the border routers.
It had multiple responsibilities:

Keeping the link alive by suppressing revocations.
Discovering the interface ID of the far-side of each link
Deciding what peer entries are included in the constructed segments (revoked peer links were ignored)

With the move to the current SCION header design, we drop separated the concerns.
It turned out that knowing the far-side interface ID is only required for the peer links. Regular child and core links do not need this because of the MAC chaining.
Because we have not implemented peer links so far, there also has not been a need for configuring the far-side interface IDs.
However, we discussed this internally some time ago. It is reasonable to just make this a static configuration.
Think about it this way: Changing the interface ID dynamically will result in a lot of stale segments that still reference the old identifier. We do not have a global way of invalidating the caches -> havoc.
Dynamically changing an interface ID would require very careful operation. IMO it is far easier to simply create a new interface pair and move traffic to there by stopping to advertise the old interface pair. But even this needs a "don't advertise" mode for an interface pair that we currently do not support in code.

Hi @shitz, @oncilla. Unfortunately I am currently not sufficiently familiar with the codebase to easily find the best place to get started with this. In your own best guesstimate, how much work are we talking about? Would it indeed be something that should be manageable as a side project?

Okay, so I did get peering to work again, but only on single hop paths with source and destination in directly adjacent peering ASes and without MAC validation. With any longer paths, there are apparently issues related to processing the reverse path.
Here is an incomplete list of the steps I took to get to this point.

added a remote_if_id,omitempty json field to the BRInterface struct, allowing for remote interface IDs defined in the topology.json files to again be known in the dataplane
added some logic to topo.py to automatically generate the relevant json entry if the interface is marked as a peer
(crudely) propagate this RemoteIFID via topology.go to dataplane.go
let validateEgressID() succeed in case egress == topology.Peer
disabled path increment in processEgress() when p.infoField.Peer == true (shot in the dark that helped)
modified egressInterface() to return the proper local peering IF
lobotomized verifyCurrentMAC because peer field validation is apparently somehow different from hop field validation
fudged GetStatuses in pathprobe/path.go to simply mark all links as alive

This got SCMP pings to work between two adjacent peering ASes and also the bwtest app could run successfully over the shortcut.
Given the lack of help and the insufficient documentation, I'm giving up beyond this point. Best of luck to anyone wishing to pursue this further.

Hi @benthor
Sorry for not responding, I have been on vacation.
Do you have a branch that we can take a look at?
Overall, the approach does not look that far off for getting a prototype working.
But it is hard to judge what is going on from this high level.
Also, always feel free to reach out to us on https://scionproto.slack.com/

Hi @oncilla,
See this branch comparison. This is where I gave up.
With my changes, you can locally verify that peering is working between adjacent ASes by issuing the following:
make
./scion.sh topology -c topology/peering-test.topo
./scion.sh run
echo "0" | ./bin/scion ping -c 3 --sciond $(./scion.sh sciond-addr 112) -i 2-ff00:0:212,<IP_ADDRESS>

Sample output:
Available paths to 2-ff00:0:212:
2 Hops:
[ 0] Hops: [1-ff00:0:112 42>23 2-ff00:0:212] MTU: 1472 NextHop: <IP_ADDRESS>:31018
4 Hops:
[ 1] Hops: [1-ff00:0:112 1>2 1-ff00:0:110 3>3 2-ff00:0:210 2>7 2-ff00:0:212] MTU: 1400 NextHop: <IP_ADDRESS>:31014
Choose path: Resolved local address:
  <IP_ADDRESS>
Using path:
  Hops: [1-ff00:0:112 42>23 2-ff00:0:212] MTU: 1472 NextHop: <IP_ADDRESS>:31018

PING 2-ff00:0:212,<IP_ADDRESS>:0 pld=0B scion_pkt=88B
96 bytes from 2-ff00:0:212,<IP_ADDRESS>: scmp_seq=0 time=1.035ms
96 bytes from 2-ff00:0:212,<IP_ADDRESS>: scmp_seq=1 time=972µs
96 bytes from 2-ff00:0:212,<IP_ADDRESS>: scmp_seq=2 time=915µs

--- 2-ff00:0:212,<IP_ADDRESS> statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2.002079s

Child ASes of the peering ASes can not ping each other via the peering link:
echo "0" | ./bin/scion ping -i --sciond $(./scion.sh sciond-addr 113) 2-ff00:0:213,<IP_ADDRESS> -c 3

Output:
Available paths to 2-ff00:0:213:
4 Hops:
[ 0] Hops: [1-ff00:0:113 3>3 1-ff00:0:112 42>23 2-ff00:0:212 3>3 2-ff00:0:213] MTU: 1472 NextHop: <IP_ADDRESS>:31022
6 Hops:
[ 1] Hops: [1-ff00:0:113 3>3 1-ff00:0:112 1>2 1-ff00:0:110 3>3 2-ff00:0:210 2>7 2-ff00:0:212 3>3 2-ff00:0:213] MTU: 1400 NextHop: <IP_ADDRESS>:31022
Choose path: Resolved local address:
  <IP_ADDRESS>
Using path:
  Hops: [1-ff00:0:113 3>3 1-ff00:0:112 42>23 2-ff00:0:212 3>3 2-ff00:0:213] MTU: 1472 NextHop: <IP_ADDRESS>:31022

PING 2-ff00:0:213,<IP_ADDRESS>:0 pld=0B scion_pkt=112B

--- 2-ff00:0:213,<IP_ADDRESS> statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 4.00166s
Error: no reply packet received

Hi. Just checking in to say that we have not forgotten about this, and I will be provide feedback soon.

Hi @benthor
I finally found time to give you some feedback on the peering link code. You can find it on a draft PR that I created based on your work (https://reviewable.io/reviews/scionproto/scion/4249)
The main reason the code did only work for direct neighbors was a misunderstanding of how the Peer flag in the info field works. You can find an explanation here: https://reviewable.io/reviews/scionproto/scion/4249#-NAkSrXY1ZDQyUtzl7md:-NAkSrXY1ZDQyUtzl7me:bssnr09
With small changes to the code, I got a fully functioning topology. See here: https://reviewable.io/reviews/scionproto/scion/4249#-NAkMRCmDy8kyUxEI_If:-NAkMRCmDy8kyUxEI_Ig:bp0jb2w
I think that can be used as a starting point for a pull request that would introduce this feature.
What still is missing are:

Get scion showpath working with peering links
Acceptance test cases: https://github.com/scionproto/scion/blob/master/tools/braccept/main.go
Removing unnecessary code

There was also a small bug in the path combinator that made everything even more confusing:
diff --git a/private/path/combinator/graph.go b/private/path/combinator/graph.go
index 3eb3e6b8a..e62ca9461 100644
--- a/private/path/combinator/graph.go
+++ b/private/path/combinator/graph.go
@@ -496,6 +496,9 @@ func calculateBeta(se *solutionEdge) uint16 {
 		}
 	} else {
 		index = len(se.segment.ASEntries) - 1
+		if index == se.edge.Shortcut && se.edge.Peer != 0 {
+			index++
+		}
 	}
 	beta := se.segment.Info.SegmentID
 	for i := 0; i < index; i++ {

This would lead to MAC verification failures in the data plane if the source AS was also the peering AS (i.e., first hop is already using the peering link) because the SegID was initialized with beta_i instead of beta_i+1


[Test] Disable dysfunctional catalog extension test

Notes for Reviewers
This PR disables a test that is incomplete and not ready for inclusion and consideration of whether a PR should merge or not.
Signed commits

[x] Yes, I signed my commits.

Wait, one test is still failing...

@jerensl do we need more tests disabled?

oh, I see. This extension is supposed to be enabled by default, but it users are supposed to be allowed to disable it.

@leecalcote this one is a bug from the API when u try to toggle Meshery Catalog Content button from this page https://playground.meshery.io/user/preferences, it will be always enabled

Let's get to a passing state and work forward from there.


WebView doesn't report errors

[x] I have reviewed the documentation
[x] I have searched existing issues
[x] I am using the latest React Native version
The RN WebView doesn't seem to report errors that native WebViews would. For example, it does not seem to report an errors when given an invalid URL.
Environment
  OS:  macOS Sierra 10.12.6
  Node:  9.8.0
  Yarn:  1.3.2
  npm:  5.6.0
  Watchman:  4.6.0
  Xcode:  Xcode 9.2 Build version 9C40b
  Android Studio:  2.3 AI-162.4069837

I've reproduced in Expo with React Native v0.54.2 (see below).
Steps to Reproduce
https://snack.expo.io/S1_nemasz
Expected Behavior
I expected the onError callback to be called.
Actual Behavior
No error is reported.

Updated with environment information!


Improved css styling of donation button and removed an extra learn more us button #626

Issue - #626
Improved css styling of donation button
https://github.com/user-attachments/assets/ba8319bf-19cc-4695-9d55-fae0d8cbcce9
removed an extra learn more us button

Resolve the conflicts in the mentioned files. Ensure you pull all the latest changes before pushing your feature/bug fixes to the main branch. After resolving the conflicts, please open a new PR for review.
Thank you!


small fix causing app to crash

configuration has been renamed

+1, I think the commit message could be better though - this is just a build issue due to the function being deprecated, nothing related to a crash. See https://developer.apple.com/documentation/arkit/arworldtrackingconfiguration.


Getting unexpected login dialog box

Core Library
MSAL.js v2 (@azure/msal-browser)
Core Library Version
2.13.1
Wrapper Library
Not Applicable
Wrapper Library Version
NA
Description
App is using latest MSAL 2.0 library and passing token to Graph library using acquireTokenSilent. Randomly acquireTokenSilent is returning "InteractionRequiredAuthError" which ends up calling acquireTokenPopup and user sees login dialog box.
I understand that if refresh token is expired (which should happen only after 90 days), then silent token method may fail but some user are getting within  4/5 days after initial login and getting fresh token.
I feel SDK is sometime randomly sending "InteractionRequiredAuthError" error even though refresh token is valid.
We also verified that 3rd party cookies are not blocked (if it is blocked I actually get "InteractionRequiredAuthError" error within 24 hours).
Error Message
No response
Msal Logs
No response
MSAL Configuration
this.msalauthService = new PublicClientApplication({
      auth: {
        clientId: this.getClientId(),
        redirectUri: window.location.origin + '/office365/login/redirect',
      },
      cache: {
        cacheLocation: 'localStorage',
        storeAuthStateInCookie: false, // set to true for IE 11
      },
      system: {
        iframeHashTimeout: 10000,
      },
    });

Relevant Code Snippets
private initGraphClient() {
    this.graphClient = MSGraphClient.init({
      authProvider: async (done) => {
        // Get the token from the auth service
        let token = await this.getAccessToken().catch((reason) => {
          done(reason, null);
        });

        if (token) {
          done(null, token);
        } else {
          done('Could not get an access token', null);
        }
      },
    });
  }

  private async getAccessToken(): Promise<string> {
    let InteractionRequiredAuth = false;
    let result = await this.msalauthService.acquireTokenSilent({ scopes: this.tokenScope }).catch((reason) => {
      if (reason + ''.includes('InteractionRequiredAuthError')) {
        Logging.info('Microsoft Enterprise Account: Interaction required ');
        InteractionRequiredAuth = true;
      } else {
        Logging.error('Microsoft Enterprise Account: ms token aquire first trial failed:' + reason);
      }
    });

    if (InteractionRequiredAuth) {
      // use aquiretoken popup instead of login popup to avoid
      // issue of poping login window every day.
      result = await this.msalauthService.acquireTokenPopup({ scopes: this.tokenScope }).catch((reason) => {
        Logging.error('Microsoft Enterprise Account: ms token aquire second trial failed:' + reason);
      });
    }

    if (result) {
      return result.accessToken;
    }
    return '';
  }

Reproduction Steps
1.Login into O365
2.keep using app every day to make sure token are renewed every day
3.wait for "InteractionRequiredAuthError" within few days (it is random but happen within few days which is less than 90days)
Expected Behavior
If user is keep using client, then error should not happen ever because it should renew refresh token every day but in worst case error should not happen before 90days from the first login.
Identity Provider
ADFS
Browsers Affected (Select all that apply)
Chrome
Regression
No response
Source
External (Customer)

@pashvin Refresh tokens issued to "SPA" apps have a 24 hour, non adjustable lifetime as a security consideration given that the tokens are stored in the browser. This is documented here. When the RT is expired MSAL.js will attempt to acquire a new set of access and refresh tokens using your session cookies on the service, which explains why this happens after a few days when 3rd party cookies are not blocked, but after only 24 hours when they are blocked.

@tnorling  thanks for the quick response
How this library is getting 24 hours token because as per MS document this RT timeout is fixed 90 days or can be adjusted by only AD admin for their tenant (and we don't have any custom setting so it should be 90 days for us)
Ref - https://docs.microsoft.com/en-us/azure/active-directory/develop/active-directory-configurable-token-lifetimes
Also if library is defaulting to 24 hours , then is there any way for MSAL library consumer to increase refresh token timeout using any SDK settings because 24 hours is lot less.

@pashvin This is dictated by the service for all reply urls of type "SPA" (which is required to use MSAL.js v2 or any other client-side auth library) and is not configurable. See the note in the refresh token document. If you need an RT with a longer lifetime you'll need to use a server side library (msal-node, msal.net, etc) and do your authentication server-side. Client-side auth in the browser only supports 24 hour RTs.

@tnorling thanks for confirmation that RT is only for 24 hours. I also got some more detail from one of the user. The user is putting his laptop in sleep over night while application running and then weak-up in morning which is less than 24 hours but I see in that case MSAL is sending "InteractionRequiredAuthError" while renewing token in morning. Is it possible is we try to call acquireTokenSilent while network is not ready (it takes some times on some laptop to connect with network after weakup) may fail renew token and then ask for interactive token renewal?
Anyway I am going to add verbose logs for that user and can find out more detail.

@pashvin No, InteractionRequiredAuthError is thrown as a result of very specific error codes returned by the server (token expired, consent required, MFA required, login required, etc.) If the network is unavailable you should get a no_network_connectivity error or get_request_failed/post_request_failed error.
To further clarify the RT is valid for 24 hours from the time it was obtained and is not renewable. If you make an acquireTokenSilent call after the RT has expired MSAL.js will attempt to acquire a new 24 hr RT silently (if 3rd party cookies are blocked this will fail). You can also obtain a new 24 hr RT when you make an interactive request. To further illustrate this: if your user logs in at 9am and only makes acquireTokenSilent requests all day the RT will expire at 9am the following day. At this point if 3rd party cookies are blocked or the server session has expired acquireTokenSilent will throw an InteractionRequiredAuthError which you should handle by invoking interaction.
If you are certain the original RT was obtained less than 24 hours ago and the server is still expecting interaction it may be expecting a different type of interaction, such as consent. You can email me a fiddler trace if you're unsure what kind of interaction is expected.

@tnorling Yes, I understand that RT is valid from the time it is obtained but here is my understanding First time login MSAL obtain access token (1 hour validity) and refresh token (24 hour validity) and app continue to use access token till hour to make MS graph calls. After 1 hour MS Graph request will find access token is not valid and that that point we request acquireTokenSilent and that gets new access token and refresh token (which should be valid for next 24 hour) so if user is just putting machine in sleep overnight, then RT should be valid. If user machine down little over 24 hour, then RT will be invalid.
So  does acquireTokenSilent renew (or get new one) refresh token or not?  if not then is there any way to force it?

I am assuming acquireTokenSilent is getting new RT because if that is not the case , then most of user must be reporting for interactive prompt after every weekend because many of them not running app in weekend and that is more than 24 hours.

@pashvin No, again the RT is not renewable, acquireTokenSilent does not obtain a new RT on every call. There are 3 ways to get a new RT:

Call an interactive acquireToken API
Call ssoSilent API
Call acquireTokenSilent API after RT has expired

Methods 2 and 3 above are essentially the same as acquireTokenSilent calls ssoSilent under the hood when it needs a new RT. However, these methods will not always succeed and you should fallback to interaction when they throw InteractionRequiredAuthError. A few reasons why they might fail are:

3rd party cookies are blocked in the browser
The user does not have an existing, valid session cookie from the server
The user has not previously consented to the scopes requested
Multi factor authentication is required

Our recommendation is to call acquireTokenSilent each time you need a token and fallback to interaction when you receive InteractionRequiredAuthError. Unfortunately, if your goal is to avoid the InteractionRequiredAuthError in the first place, that won't be possible. Users can reduce the frequency with which they are prompted for interaction beyond 24 hours by ensuring 3P cookies are not blocked in their browser but they will still be asked to sign in again from time to time when their server session expires.

@tnorling most of our user has MFA enabled and they are in AD network so do you think when "acquireTokenSilent" fails with Interaction Required error (assuming user consent is done), then instead of calling acquireTokenPopup directly , if I call ssoSilent first, then it may reduce frequency of popup?
Also now I know that this popup may come and in some cases I can't avoid it, then is there any way to find out that popup already open and waiting for user input because I don't have that check and sometime more than one popup are opening (I have code which just calls acquireTokenPopup on InteractionRequiredAuth error

@pashvin

so do you think when "acquireTokenSilent" fails with Interaction Required error (assuming user consent is done), then instead of calling acquireTokenPopup directly , if I call ssoSilent first, then it may reduce frequency of popup?

No, if you receive InteractionRequiredAuthError you must invoke interaction to resolve. ssoSilent will also fail.

is there any way to find out that popup already open and waiting for user input because I don't have that check and sometime more than one popup are opening

MSAL.js should throw an interaction_in_progress error if you try to invoke acquireTokenPopup while another is already in progress. When this happens do you perhaps have multiple tabs/windows open that may each be invoking their own popup? Can you please share a .gif showing multiple popups opening?
The API returns a promise, if you need to block other code from running until it's complete you can await the promise.

acquireTokenPopup does throw interaction_in_progress but it also shows popup so it is too late because I see those error in log but also see multiple popup. Do you think it should just throw error and not show popup and it is MSAL bug?
I am thinking to introduce some flag on my side to avoid duplicate unless there is some flag already which I can use it.

Few more update from my app users. Some users are saying they get popup (and based on our discussion that is due to access and RT both expired) and some user just tried to just close popup and they are able to get data (my app gets user meetings using graph API which uses token from MSAL). I don't understand how the token is refreshed without user respond to interactive popup?

acquireTokenPopup does throw interaction_in_progress but it also shows popup so it is too late because I see those error in log but also see multiple popup. Do you think it should just throw error and not show popup and it is MSAL bug?

It should just throw the error and not open a 2nd popup. Can you please provide a video/.gif of this behavior and/or a sample app that reproduces this?

Some users are saying they get popup (and based on our discussion that is due to access and RT both expired) and some user just tried to just close popup and they are able to get data (my app gets user meetings using graph API which uses token from MSAL). I don't understand how the token is refreshed without user respond to interactive popup?

If a user closes the popup the API will throw a user_cancelled error. I can't speculate as to what's going on here, I would need a sample app that reproduces this behavior to debug further.

I don't have small sample app yet but I have some internal product screenshot which shows two dialog box (and logs shows two fast calls and then exception and later each call has exception). Most of user reported two and not more than that. Here is log statement which I quickly placed for one user to track down. I don't know when and how the duplicate popup check is done but looks like it may be skipping some early calls if those happen too fast.
"2021-09-13T08:23:45.858Z INFO App Microsoft Enterprise Account: Interaction required  ",
"2021-09-13T09:25:13.073Z INFO App Microsoft Enterprise Account: Interaction required  ",
"2021-09-13T09:25:13.077Z ERROR App Microsoft Enterprise Account: ms token aquire second trial failed:BrowserAuthError: interaction_in_progress: Interaction is currently in progress. Please ensure that this interaction has been completed before calling an interactive API.  For more visit: microsoft-authentication-library-for-js/errors.md at dev · AzureAD/microsoft-authentication-library-for-js . ",
"2021-09-13T11:26:32.062Z INFO App Microsoft Enterprise Account: Interaction required  ",
"2021-09-13T11:26:32.063Z ERROR App Microsoft Enterprise Account: ms token aquire second trial failed:BrowserAuthError: interaction_in_progress: Interaction is currently in progress. Please ensure that this interaction has been completed before calling an interactive API.  For more visit: microsoft-authentication-library-for-js/errors.md at dev · AzureAD/microsoft-authentication-library-for-js . ",
"2021-09-13T11:26:35.980Z INFO App Microsoft Enterprise Account: Interaction required  ",
"2021-09-13T11:26:35.980Z ERROR App Microsoft Enterprise Account: ms token aquire second trial failed:BrowserAuthError: interaction_in_progress: Interaction is currently in progress. Please ensure that this interaction has been completed before calling an interactive API.  For more visit: microsoft-authentication-library-for-js/errors.md at dev · AzureAD/microsoft-authentication-library-for-js . ",
"2021-09-13T12:21:19.040Z INFO App Microsoft Enterprise Account: Interaction required  ",
"2021-09-13T12:21:19.128Z ERROR App Microsoft Enterprise Account: ms token aquire second trial failed:BrowserAuthError: interaction_in_progress: Interaction is currently in progress. Please ensure that this interaction has been completed before calling an interactive API.  For more visit: microsoft-authentication-library-for-js/errors.md at dev · AzureAD/microsoft-authentication-library-for-js . ",
"2021-09-13T12:47:32.484Z INFO App Microsoft Enterprise Account: Interaction required  ",
"2021-09-13T12:47:32.539Z ERROR App Microsoft Enterprise Account: ms token aquire second trial failed:BrowserAuthError: interaction_in_progress: Interaction is currently in progress. Please ensure that this interaction has been completed before calling an interactive API.  For more visit: microsoft-authentication-library-for-js/errors.md at dev · AzureAD/microsoft-authentication-library-for-js . ",
"2021-09-13T12:47:36.040Z INFO App Microsoft Enterprise Account: Interaction required  ",
"2021-09-13T12:47:36.090Z ERROR App Microsoft Enterprise Account: ms token aquire second trial failed:BrowserAuthError: interaction_in_progress: Interaction is currently in progress. Please ensure that this interaction has been completed before calling an interactive API.  For more visit: microsoft-authentication-library-for-js/errors.md at dev · AzureAD/microsoft-authentication-library-for-js . ",
"2021-09-13T13:46:02.964Z INFO App Microsoft Enterprise Account: Interaction required  ",
"2021-09-13T13:46:02.965Z ERROR App Microsoft Enterprise Account: ms token aquire second trial failed:BrowserAuthError: interaction_in_progress: Interaction is currently in progress. Please ensure that this interaction has been completed before calling an interactive API.  For more visit: microsoft-authentication-library-for-js/errors.md at dev · AzureAD/microsoft-authentication-library-for-js . ",

When user cancel dialog box, then I see "user_cancelled" error in the log but also see calls to graph API is fine.
By the way is there any impact on "Keep me signed" checked by user for 24 hour RT expiration as mentioned in following web page.
Ref: https://www.zubairalexander.com/blog/azure-active-directorys-configurable-token-lifetimes/

One of the first things acquireTokenPopup does is checks sessionStorage for an interaction_in_progress value. If it's there it throws and if it's not it creates it. If you make 2 acquireTokenPopup calls at virtually the same time it is possible that they both make it through that check and open their own popups. Are you sure the interaction_in_progress error you are seeing isn't being thrown by a 3rd or 4th, etc. invocation of acquireTokenPopup? In any event you should debug your app to find out why acquireTokenPopup is being invoked multiple times back to back and suppress that behavior.

When user cancel dialog box, then I see "user_cancelled" error in the log but also see calls to graph API is fine.

At what point is the popup closed? Is it expecting credentials when the user closes it? Or has the user already entered their credentials? Like I said, it's hard to speculate without a reproduction of this behavior, but the most reasonable I can think of is that the user successfully renewed their server session before closing the popup and subsequent acquireTokenSilent calls in the background start succeeding using the newly renewed session cookie.

By the way is there any impact on "Keep me signed" checked by user for 24 hour RT expiration as mentioned in following web page.

KMSI only affects the session cookie, not the RT. You may see improvements in the frequency with which you are prompted via popup if you check the KMSI box as it will extend the server session. However, this will have no affect in browsers that block 3P cookies as this session cookie is used after the 24 hour RT expiration and will be unavailable in those browsers.

@tnorling By the way thanks for all your valuable time and advice. Appreciate your prompt response.
For multiple popup , yes may be my app is making quick multiple call graph call on startup which may be getting integration required error and creating race condition and I will fix that in my app.
For canceling popup, user are not entering username or password, they are just closing it. I am adding verbose level log so will see if that gives any detail.
For 24 hour RT expiration, if I obtain refresh token my self (by renewing every hour), then is there any way I can update and hand over new RT to MSAL lib?
I have one old app (written before MSAL lib) where I was renewing refresh token with access token so may be I can use that trick to renew refresh token and save in the storage where MSAL can use it.
oAuth_refreshAccessToken: function() {

   /* jshint ignore:start */

   var dfd = can.Deferred();

   var url = 'https://login.microsoftonline.com/common/oauth2/v2.0/token';

   fetch(url,

   {

       crossDomain: true,

       headers: {

       'Accept': 'application/json',

       'Content-Type': 'application/x-www-form-urlencoded'

       },

       method: 'POST',

       body: 'client_id=test-id&scope=https%3A%2F%2Foutlook.office.com%2FEWS.AccessAsUser.All&refresh_token='+localStorage.getItem('microsoft_refresh_token')+'&grant_type=refresh_token'

   })

   .then(response => response.json())

   .then(responseJson => {

       if(responseJson.access_token !== undefined && responseJson.refresh_token !== undefined) {

           console.log('Microsoft OAuth: Renewing Access and Refresh token');

           localStorage.setItem('microsoft_access_token',responseJson.access_token );

For 24 hour RT expiration, if I obtain refresh token my self (by renewing every hour), then is there any way I can update and hand over new RT to MSAL lib?

There is not.

I have one old app (written before MSAL lib) where I was renewing refresh token with access token so may be I can use that trick to renew refresh token and save in the storage where MSAL can use it.

What you have here is very similar to what MSAL does to renew the access token. I would like to reiterate that the refresh token behavior is not an MSAL.js limitation but rather a result of very deliberate protocol designs implemented on the server to mitigate several attack vectors. You can use any Oauth compliant library you wish or even write your own, but as long as you are targeting AAD with a replyUrl of type "SPA" you will receive a 24 hr fixed lifetime RT. The only way to get a fresh RT is to make a new call to the /authorize endpoint which requires either cookies or interaction. The /token endpoint will issue a "new" RT but its lifetime will be equal to the remaining lifetime of the original RT. I would caution you not to try to work around this as these decisions were made for really important security related reasons. If you did manage to get a new 24 hr RT from the /token endpoint with replyUrl of type "SPA" please report it to MSRC as that is something we would certainly want to fix.

Ok I understand 24 hour rule and it is MS enforced but user are reporting before 24 hours. Now myself can't reproduce so it is working for me beyond 24 hours so I have few choices

spend time to create sample app and reproduce and as I said it never happens on my machine so will not have scucess
in old app without using MSAL don't have issue so find out how that works but not MSAL lib.
generate token on server side which is 90days and pass to MSAL

@pashvin

Ok I understand 24 hour rule and it is MS enforced but user are reporting before 24 hours.

If it's happening before 24 hours it's very likely due to something other than RT expiration. Maybe the RT was removed from browser storage or MFA is required. Best way to confirm is by collecting network traces or providing me with a correlationId and timestamp when this happens.

generate token on server side which is 90days and pass to MSAL

If your app has a server side component that can acquire tokens you should be acquiring tokens server side using a server side library such as msal-node. Server side flows are not subject to the same restrictions that browser based flows are. Please do not acquire tokens server side and store in the browser, especially long lived refresh tokens.

We have all user enabled with MFA but only few has this issue. I asked one user who can reproduce always to capture new logs.

While we are waiting for user logs, I have one more question. Is there any way to get popup window object handle? Sometime this popup comes at startup and it goes behind the main app (because user may accidentally click on main window) so popup is waiting for user input and user has main window and it is waiting for data. If we get handle on popup , then I can register on unfocus event and put back in focus.

You can use the event API and listen to the POPUP_OPENED event. The payload is the popup window object. You can read more about the event API usage here

Yes, I have hook for those event but those event does not provide popup window object so I can't hookup to find out popup is going in/out focus or can't put back in focus. I am looking a way to control popup and put in user visibility or make it always on top so user sees it.

Can you please share the code you are using to listen to the event? Like I said the payload for the POPUP_OPENED event is the popup window object. See here

It was my mistake. I was adding following code in LOGIN START event. I see popup window object in POPUP_OPENED event but still can't force to focus because it is a cross origin popup so I don't get blur event in my handler so I don't think I can put in focus.
this.msalauthService.addEventCallback((event: EventMessage) => {
if (event.eventType === EventType.POPUP_OPENED) {
const popup: Window = event.payload['popupWindow'];
if(popup) {
popup.onblur = function(event) {
popup.focus();
}
}
}
}

@tnorling I got one network capture from user but during overnight sleep test there was internet outage so if this has not good info, then I am going to get one more clean test tomorrow.
test-1.har.zip

Did this fail due to InteractionRequired? I'm only seeing a gateway timeout (likely due to the internet outage) which does not result in InteractionRequired.
Side note: please email me (email on my profile) the next trace instead of posting publicly due to the sensitive nature of what is inside.

So user put machine in the sleep overnight and in morning he saw MS login window but that was blank and then internet connection was back and popup window got login ui so it got the popup (which normally happen due to interaction required) but anyway I don't want to go in wrong direction because of internet outage so if something does not make sense, then don't spend time and will get next set of logs tomorrow.

@tnorling sent some new logs to your email address.

Still under monitor and waiting for new logs

Closing issue as after upgrading to 2.17 issue is not noticed


Implement and train a point cloud autoencoder.

Input point cloud with categorical features (or perhaps other features, e.g., atom types).
Output identical point cloud.
Should be equivariant w.r.t. input cloud.
To-do:

implement equivariant encoder & decoder
select a point cloud reconstruction loss

Finished in equivariant branch.


Basic Cirrus Terminal implementation

Things that still need to be done:

write tests
implement proper error funneling (there are places where we just log the error, but we could pass it to the caller instead)
complete CI configuration
write README.md and include an architecture diagram

write tests

Added unit tests in https://github.com/cirruslabs/terminal/pull/1/commits/899736cebd228cdf10bffb77638439d498963b97 and end-to-end tests in https://github.com/cirruslabs/terminal/pull/1/commits/3a76ca596eca376025049faab48048efd05a6718.

write tests

Added unit tests in 899736c and end-to-end tests in 3a76ca5.

Will do the rest as a separate PRs.


[7.x] Fix workbench:install generating migrate-fresh command with incorrect options

fixes https://github.com/orchestral/workbench/issues/65

coverage: 93.076% (-0.1%) from 93.193%
when pulling 562327a56cf719403261cd1ec95141bde012258e on 7/issue-65
into 4d9128207c8222324ccad912ef600b217a27938f on 7.x.


Mega 0.4.0 spike - #17 #18 #19 #21 #22

Turned out this one touched all the relevant bits for a bunch of things.
Resolves #17 #18 #19 #21
Needs tests, and maybe a refactor.


Missing some crucial events

Hello!
I'm curious why there are no events like:
https://github.com/fluxcd/helm-controller/blob/56e36da8c15fe4aef06d257c3295044e53874cab/internal/controller/helmrelease_controller.go#L410
https://github.com/fluxcd/helm-controller/blob/56e36da8c15fe4aef06d257c3295044e53874cab/internal/controller/helmrelease_controller.go#L416
in the new version of the helm controller.
Today, we were surprised when we upgraded, and there weren't any of these events.
We built a notification system based on these events that is critical for us and our internal processes. It's really convenient to have granular events to know when the install or upgrade starts.
Don't you mind if we contribute something like this to the new version of the controller?
Thanks!

Hello @darkowlzz!
First of all, thanks for such a detailed explanation!
We've decided to use the kstatus approach with conditions. The manifest already has all the information we need, so we'll build a solution on top of it.
Thanks for the clarification!


In mention username with dot is not detecting

In mentioned username, it is not detecting full username with the dot.
For example, username like @hb.mehulmodi is only detecting hb

Need to update the mention regex as below :
static let mentionPattern =<EMAIL_ADDRESS>

I am having the same issue, did you edit the package files with this update to fix?


Investigate slow loading feed

seen by Ben and Davis (android)

I expect this is closed by https://github.com/daviswhitehead/shayr/issues/56
will reopen if not


"error return without exception set" then try to set cshigh to False.

Raspberry PI Zero W.
Works good on Linux raspberrypi 4.19.75+ #1270 Tue Sep 24 18:38:54 BST 2019
Fails on Raspberry Pi OS (32-bit) Lite May 2020 (Release date:2020-05-27)
import spidev spi = spidev.SpiDev() spi.open(0, 1) spi.cshigh = False

I think the problem is here: https://github.com/doceme/py-spidev/blob/a5d82b88bd1c95c9f9ec46e9f7afe08c3323e625/spidev_module.c#L904-L906
__spidev_set_mode returns -1 without calling PyErr_Set*, which is then returned by SpiDev_set_cshigh.
Running this test code:
#!/usr/bin/env python3
import spidev

spi = spidev.SpiDev()
spi.open(0, 0)

spi.cshigh = False

I see only these relevant syscalls in strace:
openat(AT_FDCWD, "/dev/spidev0.0", O_RDWR|O_LARGEFILE) = 3
ioctl(3, SPI_IOC_RD_MODE, 0xbea7c23f)   = 0
ioctl(3, SPI_IOC_RD_BITS_PER_WORD, 0xbea7c23f) = 0
ioctl(3, SPI_IOC_RD_MAX_SPEED_HZ, 0xbea7c248) = 0
ioctl(3, SPI_IOC_WR_MODE, 0xbea7d1d7)   = 0
ioctl(3, SPI_IOC_RD_MODE, 0xbea7d1db)   = 0

So because none of the ioctl calls fail, I have to assume that the attempt to set the SPI_CS_HIGH bit was refused by the kernel driver. So test, the readback value, didn't match mode, the value attempted to be set.

cshigh in fact is True by default because if you print(device.cshigh) you'll get a True, however I'm having a low-active behavior:

Why is this?


[Flutter] Add keyboard shortcuts for "Run" and "Format".

These two actions are done a lot when writing Flutter code. It would be nice to have shortcuts for these two things.
On MacOS, I'm instinctively expecting "Command + S" and "Command + Shift + F" respectively, although that might just be my personal instincts.

There are two issues here:

We don't expose the list of keyboard shortcuts on the embeds as they are on the front page. There actually is a shortcut for run: Cmd-Enter.
We don't have a shortcut for format, which just seems bonkers.

It's related to https://github.com/dart-lang/dart-pad/issues/1134


code blocks have trailing blank lines

See this screenshot:

there are blank lines in code blocks that use the triple-backtick syntax

Is this really an issue with the markdown package?
cc @srawlins who has been helping work on markdown

This should be fixed by 4321be8c8415d424a23a447f1cbbcd2c81a6848e


Intellisense for the terminal/debug console

Is your feature request related to a problem? Please describe.
With the release of 1.3.0, being able to quickly run commands in the terminal/debug console is essential to debugging faster. Manually typing symbols, especially when they have many characters is cumbersome and can easily introduce typing errors.
Describe the solution you'd like
Adding intellisense to the terminal and/or debug console can help users debug more efficiently.
Describe alternatives you've considered
We can debug through the connected MATLAB instance, though this defeats the purpose of debugging in VS Code.
Additional context
N/A
Thanks so much!

+1


Missing import codefix: Take scoped packages (@foo/bar) into consideration

Fixes #15223
When a package name has a scope, like @angular/core, we weren't able to locate package.json correctly, only looking under the @angular directory. Which meant that we never read the 'main/`typings property in package.json which would have helped us simplify the import module specifier.
This fixes the parsing of the path so we can find package.json.

LGTM


account Id parameter update for getUser method

Please add options.accountId as parameters to query for getUser method.

Thanks for the issue! We will add these function in the next release!


Why does asking a question once result in two requests?

Bug Report
Why does asking a question once result in two requests?
Description
asking a question once result in two requests

Open WebUI Version: [0.2-0.3.1]
**OpenAi API

Title generation is a separate prompt.


How to get Y value on click

chart.setOnClickListener(view -> {
    //How to get Y value
});

or OnTouchListener with MotionEvent?

hi @paradoxfm , you need to create a class that will implement ChartOnValueSelectListener for you to be able to override the onValueSelected method.
here is the method signature:
public void onValueSelected(int lineIndex, int pointIndex, PointValue value) {}
inside the mentioned method, you can call value.getY();


Guide on how to use this project

So, depending on your system, you either can use Docker or the Windows build (Or compile on your own). Well, the thing is that the service just needs a working appsettings.json file in the main folder and that's it. For docker, you can follow this project https://github.com/SeppPenner/SimpleMqttServer/blob/master/HowToUse.md (Replace the names of course).
Then, you need the valid settings file:
{
    "AllowedHosts": "*",
    "CalDavSynologySyncer": {
        "ServiceDelayInMilliSeconds": 30000, // Delay how often the service syncs
        "HeartbeatIntervalInMilliSeconds": 30000, // Delay how often the service logs a heartbeat message
        "CalendarUrls": [
            "https://ical.de/ical1123" // A list of source calendar urls (e.g. from the web)
        ],
        "SynologyCalendarUrl": "http://<IP_ADDRESS>/caldav.php/user/someid", // Your URL to your Synology calendar
        "SynologyCalendarId": "/caldav.php/user/uniqueid/", // The id of the Synology calendar
        "SynologyUserName": "test", // The Synology user name
        "SynologyPassword": "password", // The Synology password
        "TelegramBotToken": "111111111:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA", // A bot token from Telegram to send error logs to a chat, follow https://core.telegram.org/bots/api
        "TelegramChatId": "2222222" // The chat id on Telegram
    }
}

I will add a readme soon and try to make Telegram optional.

Feel free to ask further questions if anything is unclear.

So, depending on your system, you either can use Docker or the Windows build (Or compile on your own). Well, the thing is that the service just needs a working appsettings.json file in the main folder and that's it. For docker, you can follow this project https://github.com/SeppPenner/SimpleMqttServer/blob/master/HowToUse.md (Replace the names of course).
Then, you need the valid settings file:
{
    "AllowedHosts": "*",
    "CalDavSynologySyncer": {
        "ServiceDelayInMilliSeconds": 30000, // Delay how often the service syncs
        "HeartbeatIntervalInMilliSeconds": 30000, // Delay how often the service logs a heartbeat message
        "CalendarUrls": [
            "https://ical.de/ical1123" // A list of source calendar urls (e.g. from the web)
        ],
        "SynologyCalendarUrl": "http://<IP_ADDRESS>/caldav.php/user/someid", // Your URL to your Synology calendar
        "SynologyCalendarId": "/caldav.php/user/uniqueid/", // The id of the Synology calendar
        "SynologyUserName": "test", // The Synology user name
        "SynologyPassword": "password", // The Synology password
        "TelegramBotToken": "111111111:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA", // A bot token from Telegram to send error logs to a chat, follow https://core.telegram.org/bots/api
        "TelegramChatId": "2222222" // The chat id on Telegram
    }
}

I will add a readme soon and try to make Telegram optional.

Forget about Docker, the tool is only available as bare metal tool for now.

Thanks a lot. Great work. Looking forward to a docker version.


testapp: allow to render text with SDL_TextEngine (key 'x')

@slouken
this allow to use testapp with textengine and compare none/surface/renderer
allow, it seems "TE renderer" asserts with:
Assertion failure at CreateDrawSequence (//SDL_ttf_git/src/SDL_renderer_textengine.c:540), triggered 1 time:
  'num_ops > 0'

while "TE surface" doesn't assert

This seems fine, but can you give me a standalone repro for the num_ops assertion?

@slouken
it seems this happens with the first rendering of testapp :
fonts/FletcherGothicFLF.ttf
and this is string in  test_strings[0]
just do ./testapp and press x x and it asserts.
you need the fonts file also in "./fonts/FletcherGothicFLF.ttf"

I'm not reproducing this with the current head. Has it been fixed?

@slouken
this is still present in current HEAD.
renderer_surface would also assert, but the is no "SDL_assert(num_ops > 0);"
after, Render_Line_TextEngine num_ops is still 0
Render_Line_TextEngine:
pos_len is 4. The 4 glyphs are found, but  widths and rows are 0. so no op_index++ glyph->sz_width == glyph->sz_rows == 0and lastglyph_width == glyph_rows == 0`
the string is of course wrong because testapp is meant to break rendering with corner case.
I compile with SDL_ttf:
CMAKE_FLAGS+=" -DSDLTTF_VENDORED=ON "
        CMAKE_FLAGS+=" -DSDLTTF_HARFBUZZ=ON "

I'm not seeing this here at all. Maybe my version of the font is different? Can you create an issue and provide a modification to showfont and attach a link to the font you're using so I can repro in a debugger?

./showfont -textengine renderer fonts/FletcherGothicFLF.ttf  20 "纸"

then font I have FletcherGothicFLF.ttf.txt

@slouken here it is


[MGW 3.2.0] subscriptions validation process

Description:
A combination of the following config at the micro-gw.conf will allow the mgw for subscription validation. Better if we could improve the documentation to explaining the end to user scenario.  As of now how to config have scattered across multiple documentation
[[jwtTokenConfig]]
issuer = "https://localhost:9443/oauth2/token
certificateAlias = "wso2apim310"
validateSubscription = true
consumerKeyClaim = "aud"
and
[apim.eventHub]
enable = true
serviceUrl = "https://localhost:9443"
internalDataContext="/internal/data/v1/"
username="admin"
password="admin"
eventListeningEndpoints = "amqp://admin:admin@carbon/carbon?brokerlist='tcp://localhost:5672'"
Steps to reproduce:
Configure micro gateway default mode, then allow mgw to validate subscription flow, cross-check information available in the wso2 migro gateway documenation

This issue is fixed already with the following PR [1]. Hence we can close this issue.
[1]. https://github.com/wso2/docs-mg/pull/67

This issue is fixed already with the following PR [1]. Hense we can close this issue.
[1]. https://github.com/wso2/docs-mg/pull/67


Using Keep and Drop modifiers together can duplicate drops

Describe the bug
When using both keep and drop modifiers on the same dice, you can have the same dice being dropped by both modfiers, which causes the d flag to be added twice.
This is because both the kh and dl mark the same roll as dropped.
To Reproduce
Steps to reproduce the behavior:

Roll 3d6kh2dl1
See if the lowest roll only appears once (e.g. If the result is [2, 4, 5] the lowest roll, 2, appears once)
See that the lowest roll has two drop (d) flags: 3d6kh2dl1: [2dd, 4, 5] = 9

Expected behavior
I would expect the lowest roll to have only been dropped once: 3d6kh2dl1: [2d, 4, 5] = 9

This has been fixed and will go out in v4.2.0

I've just pushed this out in v4.1.1, rather than wait for the v4.2.0 release.