Cannot write (rawOptolink)

Installation specifics

Heating type: Vitodens 200
Protocol: P300
Board: Arduino Pro Mini ATmega328 5v 16MHz

Hi,
I am trying to write using the OptolinkP300 library and I am unable to do so. If I only try to read everything works ok. But when I try to write It seems to get stuck in busy state (the lcd prints quickly changing numbers + "busy" as it tries to do something and doesn't pass the !myOptolink.isBusy() condition.)
I'm working on another code and it does read ok until it tries to write, then it gets stuck in the busy state.
I couldn't find any example of how to write using the Optolink library.
BTW I had to modify few lines in the library to make it work with Arduino Pro Mini (I am using VisualStudio with Arduino plugin - maybe it's specific just to VisualStudio):

In OptolinkP300.hpp

class OptolinkP300 {
 public:
  OptolinkP300();
//#ifdef ARDUINO_ARCH_ESP32
//  void begin(HardwareSerial* serial, int8_t rxPin, int8_t txPin);
//#endif
//#ifdef ESP8266
  void begin(HardwareSerial* serial);
//#endif
  void loop();

In OptolinkP300.cpp

// begin serial @ 4800 baud, 8 bits, even parity, 2 stop bits
//#ifdef ARDUINO_ARCH_ESP32
//void OptolinkP300::begin(HardwareSerial* serial, int8_t rxPin, int8_t txPin) {
//  serial->begin(4800, SERIAL_8E2, rxPin, txPin);
//  _stream = serial;
//  // serial->flush();
//}
//#endif
//#ifdef ESP8266
void OptolinkP300::begin(HardwareSerial* serial) {
  serial->begin(4800, SERIAL_8E2);
  _stream = serial;
  // serial->flush();
}
//#endif

Example:
(Tries to write 27 °C to the set temperature register and then reads it).
#include <LiquidCrystal_I2C.h>
#include <Wire.h>
#include <OptolinkP300.hpp>
#include <Constants.hpp>

OptolinkP300 myOptolink;
#define lcd_width 16
LiquidCrystal_I2C lcd(0x27, lcd_width, 2);
#define room_temperature_day_address 0x2306 //(3..37) °C
#define room_temperature_day_length 1

#define pin_LED_alarm 5
#define pin_LED_communication 7
#define pin_LED_DEBUG 8
#define pin_LED_write LED_BUILTIN

uint32_t lastMillis = 0;
bool getValues = false;
boolean writeValues = false;
boolean readRequested = false;

void setup() {
    pinMode(pin_LED_alarm, OUTPUT);
    pinMode(pin_LED_communication, OUTPUT);
    pinMode(pin_LED_DEBUG, OUTPUT);
    pinMode(pin_LED_write, OUTPUT);

    myOptolink.begin(&Serial);
    lcd.init();
    lcd.backlight();
    lcd.setCursor(0, 0);
    lcd.print("boot");

    //blink all LEDs top indicate the boot
    digitalWrite(pin_LED_alarm, HIGH);
    digitalWrite(pin_LED_communication, HIGH);
    digitalWrite(pin_LED_DEBUG, HIGH);
    digitalWrite(pin_LED_write, HIGH);
    delay(500);
    digitalWrite(pin_LED_alarm, LOW);
    digitalWrite(pin_LED_communication, LOW);
    digitalWrite(pin_LED_DEBUG, LOW);
    digitalWrite(pin_LED_write, LOW);

}

// the loop function runs over and over again until power down or reset
/*
//The loop has two states - getValues and writeValues
void loop() {
    myOptolink.loop();

    if (millis() - lastMillis > 10 * 1000UL) {
        lastMillis = millis();
        //RESET the LEDs
        digitalWrite(pin_LED_alarm, LOW);
        digitalWrite(pin_LED_communication, LOW);
        digitalWrite(pin_LED_DEBUG, LOW);
        lcd.clear();

        if (writeValues) { //sets state to get
            getValues = true;
            readRequested = false;
            writeValues = false;
            lcd.setCursor(0, 1);
            lcd.print("get");
        }
        else { //sets state to write
            getValues = false;
            writeValues = true;
            lcd.setCursor(0, 1);
            lcd.print("write");
        }

        digitalWrite(pin_LED_write, writeValues); //write LED is on when writing
    }

    if (getValues) {
        if (!myOptolink.isBusy()) {
            if (!readRequested) {
                digitalWrite(pin_LED_communication, HIGH);
                myOptolink.readFromDP(room_temperature_day_address, room_temperature_day_length);
                readRequested = true;
            }
        }
        else {
            lcd.setCursor(0, 0);
            lcd.print(millis());
            lcd.print("Busy");
        }
        if (readRequested) {
            if (myOptolink.available() > 0) {
                uint8_t value[4] = { 0 };
                myOptolink.read(value);

                if (value[0] == 27) {
                    digitalWrite(pin_LED_DEBUG, HIGH); //read data matches the written data
                }
                getValues = false;
                digitalWrite(pin_LED_communication, LOW);

            }
        }
    }

    if (writeValues) {
        if (!myOptolink.isBusy()) {
            digitalWrite(pin_LED_communication, HIGH);
            byte value[4] = { 27,0,0,0 };
            boolean err = !myOptolink.writeToDP(room_temperature_day_address, room_temperature_day_length, value);
            if (err) {
                digitalWrite(pin_LED_alarm, HIGH);
                lcd.setCursor(0, 0);
                lcd.print(millis());
                lcd.print("Write error");
            }
            digitalWrite(pin_LED_communication, LOW);
        }
        else {
            lcd.setCursor(0, 0);
            lcd.print(millis());
            lcd.print("Busy");
        }
    }

    if (myOptolink.available() < 0) { //error
        digitalWrite(pin_LED_alarm, HIGH);
        lcd.setCursor(0, 0);
        lcd.print(millis());
        lcd.print("error");
        lcd.print(myOptolink.readError());
    }
}

Edit: I tried to compile the same code with Arduino IDE and it does exactly the same thing.

Is it possible to have debug outputs?
--> set printer in Optolink, uncomment the setState and setAction methods in OptolinkP300.hpp
I must admit it has been a while since I looked at the current code. I'm working on a brand new version, much easier to work with.

I'd have to use SoftwareSerial to send the debug data as I'm using Arduino Pro Mini that only has one hardware UART. I'll try it tomorrow.

Here is what I got from the debug printer (going trought the loop for about a minute):
Optolink state: 1
Optolink state: 2
Optolink state: 3
Optolink state: 0
Optolink state: 1
Optolink state: 2
Optolink state: 3
Optolink state: 4
Optolink action: 1
Optolink state: 5
Optolink state: 6
WRITE<PHONE_NUMBER>06011b4d
ack
Optolink state: 7
RCV<PHONE_NUMBER>060132
Optolink state: 8
ack
Optolink state: 4
Optolink action: 2
Optolink state: 2
Optolink state: 3
Optolink state: 4
Optolink state: 2
Optolink state: 3
Optolink state: 4
Optolink state: 2
Optolink state: 3
Optolink state: 4
Optolink state: 2
Optolink state: 3
Optolink state: 4
Optolink state: 2
Optolink state: 3
Optolink state: 4

I restarted the boiler because I was using KW that could affect it before. I does exactly the same thing.
18:58:11.261 -> Optolink state: 1
18:58:11.261 -> Optolink state: 2
18:58:11.261 -> Optolink state: 3
18:58:12.282 -> Optolink state: 0
18:58:12.282 -> Optolink state: 1
18:58:12.353 -> Optolink state: 2
18:58:12.353 -> Optolink state: 3
18:58:12.353 -> Optolink state: 4
18:58:19.670 -> Optolink action: 1
18:58:19.670 -> Optolink state: 5
18:58:19.707 -> Optolink state: 6
18:58:19.707 -> WRITE<PHONE_NUMBER>06011b4d
18:58:19.741 -> ack
18:58:19.741 -> Optolink state: 7
18:58:19.741 -> RCV<PHONE_NUMBER>060132
18:58:19.778 -> Optolink state: 8
18:58:19.778 -> ack
18:58:19.778 -> Optolink state: 4
18:58:19.778 -> Optolink action: 2
18:58:34.819 -> Optolink state: 2
18:58:34.819 -> Optolink state: 3
18:58:34.819 -> Optolink state: 4
18:58:49.806 -> Optolink state: 2
18:58:49.842 -> Optolink state: 3
18:58:49.842 -> Optolink state: 4
18:59:04.832 -> Optolink state: 2
18:59:04.866 -> Optolink state: 3
18:59:04.866 -> Optolink state: 4
18:59:19.881 -> Optolink state: 2
18:59:19.881 -> Optolink state: 3
18:59:19.919 -> Optolink state: 4
18:59:34.928 -> Optolink state: 2
18:59:34.928 -> Optolink state: 3
18:59:34.928 -> Optolink state: 4
18:59:49.928 -> Optolink state: 2
18:59:49.962 -> Optolink state: 3
18:59:49.962 -> Optolink state: 4

After writing you also have to "reset" by reading the optolink like you do on if (readRequested) { ...
v2.0.0 will be a lot more easy to use. So if you're adventurous, you can take a look at the v2.0.0 branch I don't know if it is working on a AVR though. I don't know if you're able to use a std::queue.

Thank you. I'll try it and let you know if it works.
Should I do it exactly the same way as reading?
if (myOptolink.available() > 0) myOptolink.read(value);
What is it supposed to read?
It would be nice to include this into the rawOptolink example.

It just returns the written value or the error, in case.
if (myOptolink.available() > 0) {
  uint8_t value[4] = {0};
  myOptolink.read(value);
} else if (myOptolink.available() < 0) {
  myOptolink.readError();
}

Thank you. I don't have enough time to try it today so I'll tomorrow.

Hi, I modified my code and it works:
#include <SoftwareSerial.h>
#include <LiquidCrystal_I2C.h>
#include <Wire.h>
#include <OptolinkP300.hpp>
#include <Constants.hpp>

OptolinkP300 myOptolink;
#define lcd_width 16
LiquidCrystal_I2C lcd(0x27, lcd_width, 2);
SoftwareSerial swSerial(9,10); //TX:10

#define room_temperature_day_address 0x2306 //(3..37) °C
#define room_temperature_day_length 1

#define pin_LED_alarm 5
#define pin_LED_communication 7
#define pin_LED_DEBUG 8
#define pin_LED_write LED_BUILTIN

uint32_t lastMillis = 0;
bool getValues = false;
boolean writeValues = false;
boolean readRequested = false;
boolean state = false; //0: read 1:write

// the setup function runs once when you press reset or power the board
void setup() {
    pinMode(pin_LED_alarm, OUTPUT);
    pinMode(pin_LED_communication, OUTPUT);
    pinMode(pin_LED_DEBUG, OUTPUT);
    pinMode(pin_LED_write, OUTPUT);

    myOptolink.begin(&Serial);
    swSerial.begin(57600);
    myOptolink.setLogger(&swSerial);
    lcd.init();
    lcd.backlight();

    lcd.setCursor(0, 0);
    lcd.print("boot");

    //blink all LEDs top indicate the boot
    digitalWrite(pin_LED_alarm, HIGH);
    digitalWrite(pin_LED_communication, HIGH);
    digitalWrite(pin_LED_DEBUG, HIGH);
    digitalWrite(pin_LED_write, HIGH);
    delay(500);
    digitalWrite(pin_LED_alarm, LOW);
    digitalWrite(pin_LED_communication, LOW);
    digitalWrite(pin_LED_DEBUG, LOW);
    digitalWrite(pin_LED_write, LOW);

}

// the loop function runs over and over again until power down or reset
/*
The loop has two states - getValues and writeValues

*/

void loop() {
    myOptolink.loop();

    if (millis() - lastMillis > 10 * 1000UL) {
        lastMillis = millis();
        //RESET the LEDs
        digitalWrite(pin_LED_alarm, LOW);
        digitalWrite(pin_LED_communication, LOW);
        digitalWrite(pin_LED_DEBUG, LOW);
        lcd.clear();

        if (state) { //sets state to get
            getValues = true;
            readRequested = false;
            writeValues = false;
            state = false;
            lcd.setCursor(0, 1);
            lcd.print("get");
        }
        else { //sets state to write
            getValues = false;
            writeValues = true;
            state = true;
            lcd.setCursor(0, 1);
            lcd.print("write");
        }

        digitalWrite(pin_LED_write, writeValues); //write LED is on when writing
    }

    if (getValues) {
        if (!myOptolink.isBusy()) {
            if (!readRequested) {
                digitalWrite(pin_LED_communication, HIGH);
                myOptolink.readFromDP(room_temperature_day_address, room_temperature_day_length);
                readRequested = true;
            }
        }
        else {
            lcd.setCursor(0, 0);
            lcd.print(millis());
            lcd.print("Busy");
        }
        if (readRequested) {
            if (myOptolink.available() > 0) {
                uint8_t value[4] = { 0 };
                myOptolink.read(value);

                if (value[0] == 27) {
                    digitalWrite(pin_LED_DEBUG, HIGH); //read data matches the written data
                }
                getValues = false;
                digitalWrite(pin_LED_communication, LOW);

            }
        }
    }

    if (writeValues) {
        if (!myOptolink.isBusy()) {
            digitalWrite(pin_LED_communication, HIGH);
            byte value[4] = { 27,0,0,0 };
            boolean err = !myOptolink.writeToDP(room_temperature_day_address, room_temperature_day_length, value);
            writeValues = false; //to prevent multiple writing
            while (myOptolink.available() == 0) {
                myOptolink.loop();
            }
            if (myOptolink.available() > 0) {
                uint8_t value[4] = { 0 };
                myOptolink.read(value);
                if (value[0] != 27) {
                    err = true;
                }
            }
            if (myOptolink.available() < 0) {
                myOptolink.readError();
                err = true;
            }

            if (err) {
                digitalWrite(pin_LED_alarm, HIGH);
                lcd.setCursor(0, 0);
                lcd.print(millis());
                lcd.print("Write error");
            }
            digitalWrite(pin_LED_communication, LOW);
        }
        else {
            lcd.setCursor(0, 0);
            lcd.print(millis());
            lcd.print("Busy");
        }
    }

    if (myOptolink.available() < 0) { //error
        digitalWrite(pin_LED_alarm, HIGH);
        lcd.setCursor(0, 0);
        lcd.print(millis());
        lcd.print("error");
        lcd.print(myOptolink.readError());
    }
}

Here is the debug output I got:
13:51:23.315 -> Optolink state: 1
13:51:23.820 -> Optolink state: 0
13:51:23.820 -> Optolink state: 1
13:51:24.300 -> Optolink state: 0
13:51:24.300 -> Optolink state: 1
13:51:24.806 -> Optolink state: 0
13:51:24.806 -> Optolink state: 1
13:51:25.302 -> Optolink state: 2
13:51:25.302 -> Optolink state: 3
13:51:25.302 -> Optolink state: 4
13:51:31.765 -> Optolink action: 1
13:51:31.765 -> Optolink state: 5
13:51:31.765 -> Optolink state: 6
13:51:31.765 -> WRITE<PHONE_NUMBER>06011b4d
13:51:31.765 -> ack
13:51:31.765 -> Optolink state: 7
13:51:31.765 -> RCV<PHONE_NUMBER>060132
13:51:31.810 -> Optolink state: 8
13:51:31.810 -> ack
13:51:31.810 -> Optolink state: 4
13:51:31.810 -> Optolink action: 2
13:51:31.810 -> Optolink action: 0
13:51:41.725 -> Optolink action: 1
13:51:41.725 -> Optolink state: 5
13:51:41.778 -> Optolink state: 6
13:51:41.778 -> READ<PHONE_NUMBER>060130
13:51:41.828 -> ack
13:51:41.828 -> Optolink state: 7
13:51:41.828 -> RCV<PHONE_NUMBER>06011b4d
13:51:41.828 -> Optolink state: 8
13:51:41.828 -> ack
13:51:41.828 -> Optolink state: 4
13:51:41.828 -> Optolink action: 2
13:51:41.862 -> Optolink action: 0
13:51:51.736 -> Optolink action: 1
13:51:51.736 -> Optolink state: 5
13:51:51.736 -> Optolink state: 6
13:51:51.736 -> WRITE<PHONE_NUMBER>06011b4d
13:51:51.774 -> ack
13:51:51.774 -> Optolink state: 7
13:51:51.774 -> RCV<PHONE_NUMBER>060132
13:51:51.835 -> Optolink state: 8
13:51:51.835 -> ack
13:51:51.835 -> Optolink state: 4
13:51:51.835 -> Optolink action: 2
13:51:51.835 -> Optolink action: 0
13:52:01.762 -> Optolink action: 1
13:52:01.762 -> Optolink state: 5
13:52:01.762 -> Optolink state: 6
13:52:01.762 -> READ<PHONE_NUMBER>060130
13:52:01.801 -> ack
13:52:01.801 -> Optolink state: 7
13:52:01.801 -> RCV<PHONE_NUMBER>06011b4d
13:52:01.801 -> Optolink state: 8
13:52:01.801 -> ack
13:52:01.850 -> Optolink state: 4
13:52:01.850 -> Optolink action: 2
13:52:01.850 -> Optolink action: 0
13:52:11.758 -> Optolink action: 1
13:52:11.758 -> Optolink state: 5
13:52:11.758 -> Optolink state: 6
13:52:11.758 -> WRITE<PHONE_NUMBER>06011b4d
13:52:11.758 -> ack
13:52:11.758 -> Optolink state: 7
13:52:11.803 -> RCV<PHONE_NUMBER>060132
13:52:11.803 -> Optolink state: 8
13:52:11.803 -> ack
13:52:11.803 -> Optolink state: 4
13:52:11.803 -> Optolink action: 2
13:52:11.803 -> Optolink action: 0
13:52:21.738 -> Optolink action: 1
13:52:21.738 -> Optolink state: 5
13:52:21.738 -> Optolink state: 6
13:52:21.773 -> READ<PHONE_NUMBER>060130
13:52:21.808 -> ack
13:52:21.808 -> Optolink state: 7
13:52:21.808 -> RCV<PHONE_NUMBER>06011b4d
13:52:21.808 -> Optolink state: 8
13:52:21.808 -> ack
13:52:21.857 -> Optolink state: 4
13:52:21.857 -> Optolink action: 2
13:52:21.857 -> Optolink action: 0
13:52:31.762 -> Optolink action: 1
13:52:31.762 -> Optolink state: 5
13:52:31.762 -> Optolink state: 6
13:52:31.762 -> WRITE<PHONE_NUMBER>06011b4d
13:52:31.762 -> ack
13:52:31.762 -> Optolink state: 7
13:52:31.808 -> RCV<PHONE_NUMBER>060132
13:52:31.808 -> Optolink state: 8
13:52:31.808 -> ack
13:52:31.808 -> Optolink state: 4
13:52:31.808 -> Optolink action: 2
13:52:31.808 -> Optolink action: 0

I don't need to do anything else while writing so I solved it with a while loop - it waits for some data to be received and then continues by reading it and comparing it to the data it wrote.
I'll modify the code I'm working on and close this issue if everything works as expected.
Thank you.

 Succes!

Everything works just fine. Thank you for your time.

After writing you also have to "reset" by reading the optolink like you do on if (readRequested) { ...

It just returns the written value or the error, in case.
if (myOptolink.available() > 0) {
  uint8_t value[4] = {0};
  myOptolink.read(value);
} else if (myOptolink.available() < 0) {
  myOptolink.readError();
}

It would be nice to include this as a comment in the rawOptolink example, or somewhere else in the documentation.


fix: Fix change month with invalid date

The purpose of this Pull Request is to fix a BUG.
Steps to reproduce the bug:

Select the 31st of any month;
Change the month to any other than doesn't have the 31st.

https://user-images.githubusercontent.com/3704377/223242442-b2aa943b-f986-428c-a8b8-b48b9ef5e7c7.mov

Thanks for reporting this!  The screencast is very useful.  And thanks too for the patch.
The problem is that when the month and year dropdowns change, dateFromMonthYearSelectsAndDayGrid() reads the day from the pre-change grid, which is obsolete.
My first instinct would be to delegate the fixing up of the day to IsoDate's constructor.  But actually I feel that is out of its scope; it's the caller's responsibility to give it a valid date.
So I like your suggestion to adjust the day value in the controller.
Given that the day can only have overflowed, I think it would be better to set it to IsoDate.days_in_month(month, year) in this situation.
const daysInMonth = IsoDate.days_in_month(month, year)
if (day > daysInMonth) day = daysInMonth

Thanks, @airblade!
I liked your suggestion, I did the changes and it's working now.


cookie_samesite breaks setCookie

cookie_samesite=none in grafana.ini breaks login, Set-Cookie header blocked since it lacks secure attribute.
I changed it to be commented out in /etc/grafana/grafana.ini inside the docker container (so it defaults to Lax), and that solved the issue.

@philhawthorne
I saw a new latest is published, however this is still an open issue.
This broke in a recent commit 85dc9b6af5ba35b331da87f4158315bedd858256 on Aug 17th.

Ah thanks @jsBlackBelt
I've just pushed an update to the edge tag. I'll test it with Home Assistant to make sure it works. Strangely the cookie samesite none didn't break for me.
If Lax still works for Home Assistant and is fixed for you, I'll do another merge to master.
Cheers

Thanks @philhawthorne ,
Apparently, this is a known issue:
https://community.grafana.com/t/error-logging-in-user-token-not-found/14996


Updates to support packaging and more config support

Three commits (package.json has the latest versions of all dependencies):

Make workdir required and add owner to github-glue
Make workdir required for CIHelper (always passed anyway).  Also set some values using the config instead of assuming git.
Make workdir required for GitHubGlue and add owner for authentication use.  Having owner removes the need to authenticate ahead of any other calls.
Note: Both workdir and repo are iffy as class members.  eg. annotateCommit could be in CIHelper with just the check create in this class.

Add mailparser to package.json
All external packages must be identified in order for gitgitgadget to be used as an npm package.

Change include 'require' to 'from'
The ncc tool does not seem to support 'require' on the import statement.  Changing to 'from', which works with current tooling (tsc, ncc).

Thank you for your tireless work on improving GitGitGadget!

Thanks for straightening out my little mess.  I did a rebase and somehow lost the merges when it stopped for the package.json conflict.  Then everything showed up as new commits, which I did not notice before pushing.

Thanks for straightening out my little mess.

No worries. I'm happy if I can contribute using my modest skills (for which I usually make up via tenacity).


Unable to read server response to the HEAD request

Program below works well for all requests except HEAD
import "dart:io";
import "dart:convert";

var localhost = InternetAddress.LOOPBACK_IP_V4.address;

test(String method) async {
  String helloWorld = "Hello test world!";
  HttpServer server = await HttpServer.bind(localhost, 0);
  server.listen((HttpRequest request) {
    request.response.write(helloWorld);
    request.response.close();
    print("$method request processed");
    server.close();
  });

  HttpClient client = new HttpClient();
  client.open(method, localhost, server.port, "")
      .then((HttpClientRequest request) {
    print("$method request sent");
    return request.close();
  }).then((HttpClientResponse response) {
    print("$method responce received");
    response.transform(UTF8.decoder).listen((content) {
      print("$method response read: $content"); // Why response cannot be read???
    });
  });
}

main() {
  test("HEAD");
}

The above produces
HEAD request sent
HEAD request processed
HEAD response received

Why server responce to the HEAD request cannot be read?
Tested on Dart VM version: 2.0.0-dev.10.0 (Mon Dec 4 17:07:49 2017 +0100) on "windows_x64"

Isn't this the point of a HEAD request to only get headers but no content?


[pulsar-client] Fix pending queue-size stats for batch messages

Motivation
Right now, pulsar-client prints stats with incorrect pending queue size for the batch messages. if the user configures batch-message size 100 (or batch delay is 10ms) and publishes 1000 messages then the producer prints stats with  < 10 pending messages instead 1000 messages which is published by user. It confuses the user because the user sees a different counter than actual published messages. it creates confusion when the user gets ProducerQueueFull Error and stats prints queue size is 10 and allowed pending-messages are 1000.
Modification
Show correct pending messages for batch messages.

@rdhabalia seems that this is a bug fix and does not affect the user guide, so no need to update docs here?

@Anonymitaet it won't require doc change.

@rdhabalia thanks. when submitting a PR, can you help provide a doc label (tick the box)? Thanks


[BUG] Missing disableConnectionStringValidation property which blocks migration

Description/Screenshot
I'm using application insights with an instrumentationkey currently.  My configuration looks like this:
 this._applicationInsights = new ApplicationInsights({
      config: {
        instrumentationKey: '%TEMP%', // Will be replaced in the BFF
        disableInstrumentationKeyValidation: true,
        enableAutoRouteTracking: true,
        loggingLevelTelemetry: SeverityLevel.Information,
        endpointUrl: 'track', // Send telemetry to the BFF as a reverse proxy so we don't need to expose the INSTRUMENTATION KEY to the front-end.
      },
    });
    this._applicationInsights.loadAppInsights();

The disableInstrumentationKeyValidation is extremely important for me. I forward all app insights to my reverse proxy which then replaces the %temp% value with the real one. This way I don't need to expose it in the client, which is far more secure❗
I'm currently migrating to the new connectionString property with the same %TEMP value, but there is no disableConnectionStringValidation property available. This causes application insights to load incorrectly...
I need to be able to disable connectionstring validation to keep this process working.
I would really appreciate a fix or workaround!
Steps to Reproduce

OS/Browser: Firefox
SDK Version [e.g. 22]: ^3.0.5
How you initialized the SDK: See code above

Expected behavior
I want to be able to disable connectionstring validation.
Additional context

There is no need for any disableConnectionStringValidation, you can still use the disableInstrumentationKeyValidation (as long as the connection string is well formed)
This is because when we parse the connectionString we populate the normal instrumentationKey property for you from the connectionString and from that point on the validation occurs on the instrumentationKey as normal.

as long as the connectionstring is well formed

Well, it isn't in my case ;)
Regardless of that, though. InstrumentationKey support is going to be deprecated and removed in the future, probably including the disableInstrumentationKeyValidation. So I do still think we need to be able to disable validation of the new connectionstring?

Not exactly, while "InstrumentationKey" support is going away, this is just for directly setting the instrumentationKey, the actual InstrumentationKey while still exist, just encoded with the connectionString (like it is today) and it will still require validation.

So I do still think we need to be able to disable validation of the new connectionstring?

No, we will (and are planning) to keep the existing disableInstrumentationKeyValidation for handling the validation of both the instrumentationKey (as per today) and for when only the connectionString (with the instrumentation key included -- also as per today).

Thanks for your timely response, it's immensely appreciated!
I will attempt your suggestions tomorrow and report back if it works.
As far as I can remember, setting connectionString to a value and disableInstrumentationKey to true caused errors, but maybe I just remember that wrong.

One step forwards, one step back!
I've gotten my application to load when I change the connectionString to a more real-looking value:
this._applicationInsights = new ApplicationInsights({
  config: {
    connectionString: 'InstrumentationKey=xxxx;', // Will be replaced in the BFF
    disableInstrumentationKeyValidation: true,
    enableAutoRouteTracking: true,
    loggingLevelTelemetry: SeverityLevel.Information,
    endpointUrl: 'track', // Send telemetry to the BFF as a reverse proxy so we don't need to expose the INSTRUMENTATION KEY to the front-end
  },
});
this._applicationInsights.loadAppInsights();

However, the telemetry is now sent to https://dc.services.visualstudio.com/v2/track instead of track on my own domain, which was normally done by setting the endpointUrl:
Before (Red is my own domain):

After (Blocked by CSP because the browser would never communicate with App Insights servers directly)

This is the most important part, because being able to send it to my own domain means I can insert the real connectionstring there before forwarding it myself to https://dc.services.visualstudio.com/v2/track.
I think this might have been an (undocumented) breaking change.
I used to use "@microsoft/applicationinsights-web": "^2.7.1" and I now switched to "@microsoft/applicationinsights-web": "^3.0.5".
If this is a bug in the SDK, I can simply deal with it by using ^2.7.1 for now and migrate to ^3.0.x when this is fixed. Otherwise I would like to get some guidance how to get this to work in the new SDK version.
Thanks!

If your connection string includes the ingestionendpoint, then this will replace any endpointUrl you provide in your configuration. Just removing that from the connection string you provide should work.
https://github.com/microsoft/ApplicationInsights-JS/blob/dce1e44afd47762fc4021885669852a6e37df8c7/AISKU/src/AISku.ts#L196-L201

All "keys" from the parsed connection string become lowercase -- so the one you provide doesn't have to have it all in lowercase.

I have found a solution, but it goes against your advice, therefore I'm posting this comment.
The connectionstring I provided in my latest comment was:
InstrumentationKey=xxxx;

If I understand you correctly, this means that ingestionEndpoint is null and that my endpointUrl should be used, making my use-case work. The code that you linked to agrees with this: https://github.com/microsoft/ApplicationInsights-JS/blob/dce1e44afd47762fc4021885669852a6e37df8c7/AISKU/src/AISku.ts#L198-L199
I fully understand what you're saying here, and I hope we can agree on the things said above. I'm grateful for your patience.
The issue
However, as you can see in my latest comment, this is not the case because it still overrides the endpointUrl with the https://dc.services.visualstudio.com/v2/track url, so endpointUrl isn't used.
I looked a bit further into the parseConnectionString function and found the following:
https://github.com/microsoft/ApplicationInsights-JS/blob/dce1e44afd47762fc4021885669852a6e37df8c7/shared/AppInsightsCommon/src/ConnectionStringParser.ts#L32-L39
The way I interpret this is that, whilst parsing the connectionstring, the ingestionendpoint is null, so it will be given the DEFAULT_BREEZE_ENDPOINT value, which means that ingestionendpoint can never be null, meaning that endpointUrl will never be used (in the code you linked to above):
https://github.com/microsoft/ApplicationInsights-JS/blob/dce1e44afd47762fc4021885669852a6e37df8c7/AISKU/src/AISku.ts#L198-L199
Workaround
I can work around this by using the following connectionstring.
'instrumentationkey=xxxx;ingestionendpoint=/'
(app insights config) endpointUrl: 'track'

When running on https://localhost:5012, this will send all requests to https://localhost:5012/track again. However, I do not really understand why this works. If I read the code you linked, it should now append the DEFAULT_BREEZE_PATH, making the final url https://localhost:5012/v2/track. But apparently it DOES use my endpointUrl:
https://github.com/microsoft/ApplicationInsights-JS/blob/dce1e44afd47762fc4021885669852a6e37df8c7/AISKU/src/AISku.ts#L198-L199
What's going on here?
Extra
I do not understand how "lowercase" plays a role here, as this is the exact spelling as showcased in the Azure Portal.
In fact, changing my working example (all lowercase) to this:
InstrumentationKey=xxxx;IngestionEndpoint=/

results in a working code sample, so I don't understand why the lowercase is required here.

Well bother!
Lets tag this issue as a bug so that we can "update" the parseConnectionString to "pass-in" the "user default endpointUrl", this will allow the caller (the AISku.ts) to pass the config value of this default value.
so it would become something like
const cs = parseConnectionString(_config.connectionString, _config.endpointUrl);

and then in the parse this bit can use that
export function parseConnectionString(connectionString?: string, userDefaultEndpointUrl?: string): ConnectionString {
...
        // apply the default endpoints
        result.ingestionendpoint = result.ingestionendpoint || userDefaultEndpointUrl || DEFAULT_BREEZE_ENDPOINT;

Seems like a great fix! Perhaps it would be worth creating a new issue for this, as the current issue is kind of all over the place.
Also, I'm just wondering if this needs to be applied to other App Insights SDK's as well. Couldn't a Java/Python/.NET/etc.. SDK have the same issue as well?

@MSNev I'm trying to reproduce my workaround but now I can't get that to work, sadly. Very frustrating.
Do you have any idea if this will be solved before 2023-12-08? I'm planning on publishing a blog post on this topic on that date 😅 .  If not, could you help me with figuring out a stable workaround for this issue?

For future readers:
Until this bug is fixed, if you use @microsoft/applicationinsights-web": "^3.0.5 with the connectionString approach and want to change the ingestionEndpoint to anything that isn't the same domain as the application is running on (like https://localhost:3434 for example), you'll run into a bug you can't work around.
If you have set up your endpointUrl to be /hello (for example), the bug is that the url will ALWAYS be /hello/v2/track. The library currently forcibly adds /v2/track  to the URL and you can't do anything about it. For example: https://localhost:3434/v2/track.


[build/android] Add missing arg in build.gradle

Add missing arg NNSTREAMER_EDGE_ROOT in build.gradle

Signed-off-by: Yongjoo Ahn<EMAIL_ADDRESS>

:memo: TAOS-CI Version: 1.5.20200925. Thank you for submitting PR #228. Please a submit 1commit/1PR (one commit per one PR) policy to get comments quickly from reviewers. Your PR must pass all verificiation processes of cibot before starting a review process from reviewers. If you are new member to join this project, please read manuals in documentation folder and wiki page. In order to monitor a progress status of your PR in more detail, visit http://ci.nnstreamer.ai/.


Flutter DropdownButtonFormField widget does not provide a way to center text vertically if constrained height is set

Use case
There is currently no method to center the item text vertically within a bordered-style form field appropriately, contentPadding property is not sufficient for customization if the widget is wrapped with constrained upper height.
String? val;
List<String> samples = ['A', 'B', 'C'];

LimitedBox(
     maxHeight: 30,
     child: DropdownButtonFormField<String?>(
          decoration: InputDecoration(
              contentPadding: EdgeInsets.only(bottom: 10.5),
                focusedBorder: UnderlineInputBorder(
                    borderSide:
                        BorderSide(color: Colors.transparent)),
                enabledBorder: UnderlineInputBorder(
                    borderSide:
                    BorderSide(color: Colors.transparent)),
            ),
            isExpanded: true,
            value: val,
            alignment: AlignmentDirectional.centerStart,
            items: samples
                .map((j) => DropdownMenuItem(
                    value: j,
                    child: Text(j)))
                .toList(),
            icon: Padding(
                padding: EdgeInsets.only(right: 10),
                child: Image.asset(
                   'images/dropdown_arrow.png',
                    scale: 2.0,
                ),
            ),
            onChanged: (newVal) => setState(() => val = newVal),
            hint: Text('Unselected'),
          )
)

Proposal
DropdownButtonFormField(
        ...
        itemPosition: Alignment.centerLeft, // New property for setting item offset position, type AlignmentGeometry
)

Hi @henry-chan-ftd, Thanks for filing the issue. You can wrap your DropdownMenuItem.child in a Align widget that should allow you to align them as desired.
items: samples
    .map((j) => DropdownMenuItem(
        value: j,
        child: Align(
            alignment: Alignment.centerLeft,
            child: Text(j),
         )
     )
   ).toList(),

complete code sample
import 'package:flutter/material.dart';

void main() {
  runApp(const MyApp());
}

class MyApp extends StatelessWidget {
  const MyApp({Key? key}) : super(key: key);
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Flutter Demo',
      theme: ThemeData(
        primarySwatch: Colors.blue,
      ),
      home: const MyHomePage(title: 'Flutter Demo Home Page'),
    );
  }
}

class MyHomePage extends StatefulWidget {
  const MyHomePage({Key? key, required this.title}) : super(key: key);
  final String title;
  @override
  State<MyHomePage> createState() => _MyHomePageState();
}

class _MyHomePageState extends State<MyHomePage> {
  String? val;
  List<String> samples = ['A', 'B', 'C'];

  @override
  Widget build(BuildContext context) {
    return Scaffold(
        body: Center(
      child: LimitedBox(
          maxHeight: 30,
          child: DropdownButtonFormField<String?>(
            decoration: InputDecoration(
              contentPadding: EdgeInsets.only(bottom: 10.5),
              focusedBorder: UnderlineInputBorder(
                  borderSide: BorderSide(color: Colors.transparent)),
              enabledBorder: UnderlineInputBorder(
                  borderSide: BorderSide(color: Colors.transparent)),
            ),
            isExpanded: true,
            value: val,
            alignment: AlignmentDirectional.centerStart,
            items: samples
                .map((j) => DropdownMenuItem(
                    value: j,
                    child: Align(
                      alignment: Alignment.centerLeft,
                      child: Text(j),
                    )))
                .toList(),
            icon: Icon(Icons.arrow_downward),
            onChanged: (newVal) => setState(() => val = newVal),
            hint: Text('Unselected'),
          )),
    ));
  }
}

Closing as this is not a bug and can be achieved with existing apis.

Thank you for suggesting, it did sound like an accurate solution but still it incurs unexpected behavior after adopting your instruction (Align widget over child of DropdownMenuItem):

it does not center vertically if the dropdown box height is set less than kMinInteractiveDimension, 2) Text clipping

Screenshots:
1a)
1b)


feat: add suport to Syscoin

Implement and deploy on Syscoin Testnet:

[x] Price-SYS/USDT-6
[ ]

Superseeded by https://github.com/witnet/witnet-price-feeds/pull/132


Incorrect image when installing chart dependency with previously used (and purged) release name

Incorrect redis image installed as sub dependency
Output of helm get:
kind: Deployment
metadata:
  name: api-redis
  labels:
    app: api-redis
    chart: "redis-1.1.20"
    release: "api"
    heritage: "Tiller"
spec:
  template:
    metadata:
      labels:
        app: api-redis
    spec:
      securityContext:
        fsGroup: 1001
        runAsUser: 1001
      containers:
      - name: api-redis
        image: "bitnami/redis:3.2.9-r2"
        imagePullPolicy: "IfNotPresent"

The chart dep specifies 1.1.20 which is correct, but it should use redis:4.0.8-r7
Strangely, this only happened when after doing a delete --purge, and then install using the same name. If I install using a new name, it uses the new chart/image. Could it be grabbing something from old release data?
Also, the charts/redis-1.1.20.tgz file has the correct redis version.
Potentially related: https://github.com/kubernetes/helm/issues/3249 (I did not run a helm dep update before the install)
Output of helm version:
Client: &version.Version{SemVer:"v2.8.2", GitCommit:"a80231648a1473929271764b920a8e346f6de844", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.8.2", GitCommit:"a80231648a1473929271764b920a8e346f6de844", GitTreeState:"clean"}

Output of kubectl version:
Client Version: version.Info{Major:"1", Minor:"8", GitVersion:"v1.8.4", GitCommit:"9befc2b8928a9426501d3bf62f72849d5cbcd5a3", GitTreeState:"clean", BuildDate:"2017-11-20T05:28:34Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"8+", GitVersion:"v1.8.9-gke.1", GitCommit:"c205eb8319d25aa46b1eb39b6c1d2bc5d1a49fae", GitTreeState:"clean", BuildDate:"2018-03-13T17:49:52Z", GoVersion:"go1.8.3b4", Compiler:"gc", Platform:"linux/amd64"}

Cloud Provider/Platform (AKS, GKE, Minikube etc.): GKE

I'd check if there are orphaned releases (ConfigMaps) in kube-system. If the output of helm template shows the correct info but helm install installs something else, then there's some external actor in play here.

Could it be that I had a old version cached locally? (not doing a helm repo update prior to the install)

Closing due to inactivity. Please open a new ticket against Helm 3 if this issue persists. Thanks!


Add basic tests for tilerator

Configure travis-ci to run npm install and linting
in a docker container. It cannot run the tests because it
would require postgres, redis, and cassandra. That's heavy
to setup in docker but it can be done eventually.
The tests can still be run locally when the dependencies are setup correctly.
The created spec.yaml can also be used for monitoring in prod as a next step.

@stephanebisson you don't need postgres or cassandra to do a tilerator test. Both of them are required only if you want to test a specific data supplier (tilelive-vector), or tile storage (tilelive-postgress).  Instead, you can run most integration tests with simpler setup, e.g. using tilelive-file. This way, just make tilerator copy some tiles from one file to another. Redis would be the only real requirement there to coordinate jobs, but even that can probably be simulated with a simple key-value store.

P.S. Or use redis-mock


HFRM parameters problems after DWT

Hey, in-dimension in self.enhance0 might have some problems. It should be set to 12 instead of 3, I think.
I am a little bit confused on this, so I pulled this issue for help.

We concentrate the components after dwt in the batch dimension, so the channel dimension should be 3.


Microphone getting off in IOS Devices

Hello,
Currently, I am working on video calling project in a flutter. I have used livekit for video calling. but I am facing one issue between android and IOS video calling if android and IOS users do the first call audio will be fine but if they do the second call android device user  cant get IOS  user’s audio.
recently i noticed that when audio is not coming through in android on IOS devices MICMODE is getting Turned off means microphone is getting turned off .and when audio is coming in IOS devices MICMODE is getting on. like below images

.

@sumitanaiya which version of the flutter SDK are you using?

flutter SDK : Channel stable, 3.7.6.
livekit_client: ^1.2.1

flutter SDK : Channel stable, 3.7.6.
livekit_client: ^1.2.1

+1
If one of the phones calls
room.localParticipant?.setMicrophoneEnabled(true);
// ......
room.localParticipant?.setMicrophoneEnabled(false);
The other phone calls
room.localParticipant?.setMicrophoneEnabled(true);
Unable to turn on microphone

No new reports, so close this one


print ubxlib output to usb on nrf52840 dongle using Zephyr

Hi there,
I have a nrf52840 dongle and I'm using the usb console example to print something to the terminal. ->This works fine.
When I import ubxlib to my project as explained here  it compiles and flashes, but the com-port that is enabled by the usb-cdc-acm-console seems to be lost.
Does ubxlib require a specific hardware block to write the output on?
I'm fairly new to Zephyr and ubxlib, so any help/improvements would be highly appreciated.
app.overlay:
`
/ {
chosen {
zephyr,console = &cdc_acm_uart0;
};
};
&zephyr_udc0 {
cdc_acm_uart0: cdc_acm_uart0 {
compatible = "zephyr,cdc-acm-uart";
};
};
&i2c0 {
compatible = "nordic,nrf-twim";
status = "okay";
pinctrl-0 = <&i2c0_default>;
pinctrl-1 = <&i2c0_sleep>;
pinctrl-names = "default", "sleep";
mysensor: mysensor@42{
compatible = "i2c-device";
reg = < 0x42 >;
label = "MYSENSOR";
};
};
&pinctrl {
i2c0_default: i2c0_default {
group1 {
psels = <NRF_PSEL(TWIM_SDA, 0, 31)>,
<NRF_PSEL(TWIM_SCL, 0, 29)>;
};
};
i2c0_sleep: i2c0_sleep {
group1 {
psels = <NRF_PSEL(TWIM_SDA, 0, 31)>,
<NRF_PSEL(TWIM_SCL, 0, 29)>;
low-power-enable;
};
};
};`
prj.conf:
`CONFIG_USB_DEVICE_STACK=y
CONFIG_USB_DEVICE_PRODUCT="Zephyr USB console sample"
CONFIG_USB_DEVICE_PID=0x0004
CONFIG_USB_DEVICE_INITIALIZE_AT_BOOT=n
CONFIG_SERIAL=y
CONFIG_CONSOLE=y
CONFIG_UART_CONSOLE=y
CONFIG_UART_LINE_CTRL=y
CONFIG_GPIO=y
CONFIG_I2C=y
CONFIG_GNSS=y
CONFIG_GNSS_SATELLITES=y
CONFIG_GNSS_NMEA0183=y
CONFIG_UBXLIB=y
CONFIG_UBXLIB_GNSS=y
CONFIG_DEBUG=y
CONFIG_ASSERT=y
CONFIG_INIT_STACKS=y
CONFIG_THREAD_STACK_INFO=y
CONFIG_THREAD_NAME=y
CONFIG_KERNEL_MEM_POOL=y
CONFIG_HEAP_MEM_POOL_SIZE=20000
CONFIG_MINIMAL_LIBC_MALLOC=n`

Hi, and thanks for posting.  So that I have some context, you say that you are using the nrf52840 dongle: I guess this means that you are compiling Zephyr for nRF52840, running the compiled Zephyr code on that nrf52840 dongle, and when you say "the com-port that is enabled by the usb-cdc-acm-console seems to be lost" that you were receiving debug output from Zephyr at a PC or some such over a USB-serial interface and that, once you added the ubxlib and Need the following for U-Blox as well CONFIG_ parameters you have listed above, that debug stream is no longer present.  I assume also, since you've not mentioned any, that this is without calling any ubxlib code yet, so there is no run-time difference, purely a compile-time difference.
ubxlib, as far as logging is concerned, just calls printf(), nothing fancy or HW-related there.  Nothing in what you've posted above stands out to me like it would affect Zephyr's logging interface but it is worth checking the start of your Zephyr build output (maybe do a clean build to make sure that it is performing the device tree configuration step) to make sure that it is not warning about one CONFIG_ item overriding another.
Since you are not running any ubxlib code yet, you could try removing CONFIG_ items until your debug output begins working, to try to narrow it down.

Hi @RobMeades,
Thank you for your prompt reply!
I'm on a steep learning curve from Arduino to Zephyr. Any help is appreciated!
Since the issue arrose from importing ubxlib, I thought about starting here.
I've started from a clean console printing 'hello world' example project. -> compiles and runs
I'm including the library in CmakeLists.txt with
set(ENV{UBXLIB_DIR} ${CMAKE_CURRENT_LIST_DIR}/ubxlib)
include($ENV{UBXLIB_DIR}/zephyr/ubxlib.cmake)

were receiving debug output from Zephyr at a PC or some such over a USB-serial interface

-Yes, that is correct.

so there is no run-time difference, purely a compile-time difference.

-Seems like it, yes
When setting CONFIG_UBXLIB=y
I get the following compile error:
`c:/ncs/toolchains/c57af46cb7/opt/zephyr-sdk/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/12.2.0/../../../../arm-zephyr-eabi/bin/ld.bfd.exe: app/libapp.a(u_port.c.obj): in function `ubxlib_preinit':
C:/Users/ ... /Zephyr/m8q_print/ubxlib/port/platform/zephyr/src/u_port.c:208: undefined reference to `k_thread_system_pool_assign'`

So I also enabled the following from the default.conf.
CONFIG_INIT_STACKS=y
CONFIG_THREAD_STACK_INFO=y
CONFIG_THREAD_NAME=y
CONFIG_HEAP_MEM_POOL_SIZE=20000

This does compile but then the com-port does not pop up when the code is flashed to the nRF52840.
Am I understanding correctly that the conf of default.conf is used for compiling when I include it in CmakeLists?
If that is the case, why do I need to set CONFIG_UBXLIB=y?
During compiling I see the following message.
UBXLIB_FEATURES will be "u_lib;short_range;cell;gnss"
I only need GNSS for my application.
Could you maybe point me how I can configure to only compile gnss specific parts?
Thanks in advance for your help!

I don't use Zephyr on a day to day basis myself, so I also find it very difficult to follow; it is extremely complicated because it is trying to support pretty much everything there is and remain small and remain flexible; an impossible task without accidentally appending "and extremely complicated" on the end :-(.

Am I understanding correctly that the conf of default.conf is used for compiling when I include it in CmakeLists?

The line include($ENV{UBXLIB_DIR}/zephyr/ubxlib.cmake), which you have added to your CMakeLists.txt, brings the file ubxlib.cmake into your CMakeLists.txt file (assuming, of course, that you have an environment variable UBXLIB_DIR pointing to the ubxlib directory).  The file ubxlib.cmake adds ubxlib to ZEPHYR_EXTRA_MODULES and, unless you have deliberately defined UBXLIB_NO_DEF_CONF yourself, it should append port/platform/zephyr/default.conf to your configuration variables.  But that default.conf file adds CONFIG_UBXLIB=y and all the other things already, you should have no need to add them yourself, so it plainly isn't doing what it apparently should be doing.  It may depend upon where you have put the include($ENV{UBXLIB_DIR}/zephyr/ubxlib.cmake) line in your CMakeLists.txt file; order is important in CMake, things take effect from the moment they are added and for subsequent lines only and it is not always clear what any one line is doing.
Adding ubxlib to ZEPHYR_EXTRA_MODULES is the critical thing though, because that brings in the file module.yml (more abstraction I'm afraid) which the Zephyr west build tool understands, where the most important lines are:
  cmake: port/platform/zephyr/
  kconfig: port/platform/zephyr/Kconfig

Those lines enable west to find the ubxlib Zephyr CMakeLists.txt and KConfig files, which do most of the useful stuff.

When setting CONFIG_UBXLIB=y I get the following compile error...

Yes, just adding CONFIG_UBXLIB=y alone isn't sufficient: Zephyr does not, by default, allow task stacks to be allocated at task creation, only statically at compile time, hence we need those extra things (CONFIG_INIT_STACKS=y, CONFIG_THREAD_STACK_INFO=y etc.) to make it do that.

Could you maybe point me how I can configure to only compile gnss specific parts?

If you look at the detail of the ubxlib Zephyr CMakeLists.txt you will see that it adds ubxlib .c files based on the CMake variable UBXLIB_FEATURES.  If you don't define UBXLIB_FEATURES yourself then it brings all of them in, so to chose a subset you should set UBXLIB_FEATURES in your CMakeLists.txt file before you do the include($ENV{UBXLIB_DIR}/zephyr/ubxlib.cmake) thang (order again).  If all you need is gnss then you might do something like set(UBXLIB_FEATURES gnss).

Just to add, I don't see how any of the above could relate to an nRF52 dongle's USB interface suddenly not working.  You say "then the com-port does not pop up".  Not "popping up" I guess means that the USB interface is not enumerating.  I was going to say that this sounds like the nRF52 is not booting but the nRF52 doesn't have a USB interface, there must be some other chip on the dongle which provides that interface, and that must be booting, otherwise you would not be able to re-program the nRF52.  Why would the USB interface chip suddenly not enumerate its output?
All I can suggest is backing-out changes until things work, so that you can pin-point what is going wrong.
Can you confirm that the ubxlib files are actually being built into your image (they should be listed in the build output)? It may or may not make a difference but, for the system thread pool stuff to work, SYS_INIT() must be called, so it might be if, for some reason, the ubxlib files aren't being built into your image then SYS_INIT() would not be being called; not sure what effect that would have.

Hi @RobMeades,
Thanks for helping!
This is my CmakeLists.txt.
cmake_minimum_required(VERSION 3.20.0)

# Include UBXLIB
set(UBXLIB_FEATURES gnss)
set(ENV{UBXLIB_DIR} ${CMAKE_CURRENT_LIST_DIR}/ubxlib)
include($ENV{UBXLIB_DIR}/zephyr/ubxlib.cmake)
set(ENV{ZEPHYR_EXTRA_MODULES} "${CMAKE_CURRENT_LIST_DIR}/ubxlib")

find_package(Zephyr REQUIRED HINTS $ENV{ZEPHYR_BASE})
project(m8q_print)

include(${ZEPHYR_BASE}/samples/subsys/usb/common/common.cmake)
FILE(GLOB app_sources src/*.c)
target_sources(app PRIVATE ${app_sources})

Adding ubxlib to ZEPHYR_EXTRA_MODULES is working.

ubxlib files are actually being built into your image

-Yes that seems to be done:

This is when I include the lines for ubxlib in CmakeLists.txt and include the ubxlib lines in proj.conf.
However, when I include these lines, the com-port is not opened after flashing.
This is without including the ubxlib header or using any of it's functions.
When I remove the ubxlib lines in proj.conf, the com-port opens and serial data is being written to my terminal.
For completeness, I'm using this dongle.
There is communication over USB, but no debugging unfortunately.

Where is the picture you show above from?  I agree that it shows the ubxlib files but the important thing is that the Zephyr build process shows the ubxlib files: it might be an idea to post that build output, from a clean build, as an attachment here so that we can be completely sure.
From what you say I understand that the USB com port over which debug arrives is enumerating but nothing is being sent over it, which takes us back to the possibility that, for whatever reason the nRF52 is not booting.  Is there any other way you could tell whether the nRF52 was booting?  For instance, could you light an LED or some such so that you could determine if it has entered your main application?  With no debug output, LED's are the thing I'm afraid, we've all been there :-).
As indicated above, since you are using no ubxlib code whatsoever so far, taking out the additional CONFIG_ variables you have added until something works may be another way to divide the problem.  The thing I don't understand, though, is that all of the CONFIG_ variables you have added should already have been added because you have included ubxlib.cmake, see these lines:
https://github.com/u-blox/ubxlib/blob/1b94ec51334a5a326328341ad1e5a078692c2f46/zephyr/ubxlib.cmake#L7-L15
Not that it should make any difference but, FYI, you have:
include($ENV{UBXLIB_DIR}/zephyr/ubxlib.cmake)
set(ENV{ZEPHYR_EXTRA_MODULES} "${CMAKE_CURRENT_LIST_DIR}/ubxlib")

...but including the ubxlib file ubxlib.cmake will already add ubxlib to ZEPHYR_EXTRA_MODULES, so you don't need the second line (doing it twice should do no harm though).

To be clear, we build/run our regression tests on nRF52840 constantly, so there is nothing fundamentally wrong here, which is what makes it all the more strange.

I understand from one of my colleagues that this board in fact uses a USB block inside the nRF52 itself, for which driver code needs to be included etc., which I am sure the sample you are using is doing.  He's going to see if he can dig up one of these boards himself to see what happens (he will dive in here with any input when he does).
I don't know how long that might take though so I would encourage you to (a) get an LED on the board (assuming it has one) operating so that you can flash it in the working case and then (b) flashing that LED at entry to your application in the non-working case to see if the code is getting that far.

Please take a look here: https://devzone.nordicsemi.com/f/nordic-q-a/88527/how-to-config-zephyr-if-want-to-output-debug-message-through-usb-port-on-nrf52840-dongle

@RobMeades, I will check with some LED sample code to see if the application starts.
I think that could be the reason the usb port is not listed, since it starts it during runtime.
@plerup, my journey started at this post. I'm using the one from Zephyr. The only difference from the Zephyr example is that they also include a usbd_next.

the Zephyr build process shows the ubxlib

-Yes they seem to be included when building.
Thing is that Vscode goes over it and removes the line. I've attached the build output but it does not show since only the last line of the build is presented [240/240]
build.txt

taking out the additional CONFIG_ variables you have added until something works may be another way to divide the problem

-When I do that, the "Hello World! arm" message is printed. Here I have included ubxlib in the CmakeLists.txt. Then it does not show in the build logs or in the build source files in vscode (like the image included before).

@RobMeades,
The succes case: (printing to the terminal over USB)
This is the build log with including ubxlib in the CMakeLists.txt, without including it in the CONFIG_.
build_logs_no_CONF.txt
As you can see, ubxlib is not included in the build. You mention it should so there is something else going wrong here.
The non-succes case: (not printing to the terminal over USB)
This is the build log with including ubxlib in the CMakeLists.txt, with including it in the CONFIG_.
build_logs_with_CONF.txt
I"m including ubxlib in CMakeLists.txt as follows:
cmake_minimum_required(VERSION 3.20.0)

# Include UBXLIB
set(UBXLIB_FEATURES gnss)
set(ENV{UBXLIB_DIR} ${CMAKE_CURRENT_LIST_DIR}/ubxlib)
include($ENV{UBXLIB_DIR}/zephyr/ubxlib.cmake)

find_package(Zephyr REQUIRED HINTS $ENV{ZEPHYR_BASE})
project(console)

include(${ZEPHYR_BASE}/samples/subsys/usb/common/common.cmake)
FILE(GLOB app_sources src/*.c)
target_sources(app PRIVATE ${app_sources})

it is only explicitly adding certain CONFIG_ lines directly in your CMakeLists.txt file that makes the problem occur?

-Well, I'm including ubxlib in the  CONFIG_ as follows (is this what you mean?):
# re-directing uart output to usb (from console example)
CONFIG_USB_DEVICE_STACK=y
CONFIG_USB_DEVICE_PRODUCT="Zephyr USB console sample"
CONFIG_USB_DEVICE_PID=0x0004
CONFIG_USB_DEVICE_INITIALIZE_AT_BOOT=n

CONFIG_SERIAL=y
CONFIG_CONSOLE=y
CONFIG_UART_CONSOLE=y
CONFIG_UART_LINE_CTRL=y

# ubxlib
CONFIG_UBXLIB=y

# Required for uPortTaskStackMinFree() to work
CONFIG_INIT_STACKS=y
CONFIG_THREAD_STACK_INFO=y
CONFIG_THREAD_NAME=y
CONFIG_HEAP_MEM_POOL_SIZE=20000

Thanks for the clarification.  The line:
include($ENV{UBXLIB_DIR}/zephyr/ubxlib.cmake)
...should have the effect of bringing in all of the ubxlib stuff and making it a Zephyr module.  If you look inside the file port/platform/zephyr/CMakeLists.txt, which will be brought in by the inclusion, you will see that the whole thing is gated on CONFIG_UBXLIB being set.
Can you try just defining CONFIG_UBXLIB=y for your build, i.e. include none of the other defines that you are adding on behalf of ubxlib?  That should bring in the ubxlib stuff in the way that I would expect, if nothing else, so the ubxlib files should appear in your build output.

Sorry, I see you did the originally and that cause the compilation error:
:/Users/ ... /Zephyr/m8q_print/ubxlib/port/platform/zephyr/src/u_port.c:208: undefined reference to `k_thread_system_pool_assign'`

I don't understand any of this, let me try doing exactly what you are doing myself; I don't have one of those boards but I can try to see why the right definitions aren't getting to the right places.

@RobMeades,
I've included a GPIO led in my main. It is blinking when I do not include ubxlib in the CONFIG_.
When I do include ubxlib there, the LED does not start blinking, nor do I see the com-port.
I've added my code in the zip below.
source.zip
Many thanks for the help so far!

Got it, so one or more of the CONFIG_ variables (since that's all that is really being added in this case, given that no ubxlib code is being called) prevents this particular USB-interfacing nRF52 board from booting.  That is nice and clear.  Next question is, which one?  Lemme think...

From a blank environment, to reproduce your build locally, I have done the following:

Set a few environment variables, so that west knows where Zephyr and my copy of gcc are (obviously just I need to do this, you will already have this set up):

ZEPHYR_BASE=C:\Users\rmea\.ubxlibpkg\nrfconnectsdk-v2.5.0\zephyr
ZEPHYR_TOOLCHAIN_VARIANT=gnuarmemb
GNUARMEMB_TOOLCHAIN_PATH=\Users\rmea\.ubxlibpkg\arm_embedded_gcc-10-2020-q4-major

Copied the sample you are using from samples/subsys/usb/console into a directory I can play with, C:\temp\console.
Inserted into the CMakeLists.txt file of this copy of the sample, after the cmake_minimum_required(VERSION 3.20.0) line, the additional lines:

set(UBXLIB_FEATURES gnss)
set(ENV{UBXLIB_DIR} /projects/ubxlib_priv)
include($ENV{UBXLIB_DIR}/zephyr/ubxlib.cmake)

Appended to the end of the prj.conf file of this copy of the sample the single line:

CONFIG_UBXLIB=y

While in the C:\TEMP\console directory, executed the following command:

west build -p auto -b nrf52840dongle_nrf52840 --build-opt=-v

[The command on the end to show the verbose build output].
This built to completion, output from west attached.  It seems to have done the kind of thing I was expecting, built the bits of ubxlib I was expecting, didn't throw a compiler error.
Attached find the zephyr.elf file it built (renamed to .txt so that Github will allow me to attach it here).  This will, of course, be an awfully long-winded way of debugging the problem, but it might be worth you downloading this file, renaming it to zephyr.elf and loading it onto your board to see if it, at least, boots.

Hi @RobMeades,
Strange that it does want to compile just with CONFIG_UBXLIB=y.
Could you add the hex instead of the elf file?

Checking the end of your build_logs_with_CONF.txt against the build output I get, they are quite different.  You get:
Memory region         Used Size  Region Size  %age Used
           FLASH:       43072 B      1020 KB      4.12%
             RAM:       37280 B       256 KB     14.22%
        IDT_LIST:          0 GB        32 KB      0.00%

I get:
Memory region         Used Size  Region Size  %age Used
           FLASH:       76584 B      1020 KB      7.33%
             RAM:      157632 B       256 KB     60.13%
        IDT_LIST:          0 GB         2 KB      0.00%

Now I would expect the RAM to be quite different, as you are only defining a small heap, but the flash size is half in your build versus mine, and we're using the same optimisation settings etc. as far as I know...?

Hi @RobMeades,
That worked! It's printing 'Hello World! arm'.
I'm using nrfConnect v2.5.2
With Zephyr v3.5.99

Well, I suppose that is kind of good news; I am using the default settings in this "console" example, so should all be the same as yours.  Let me go grab nRFConnect 2.5.2 and see if that is any different.

I have now repeated the procedure, only difference being that I have pointed the build at the version of Zephyr that comes with nRFConnect version 2.5.2:
ZEPHYR_BASE=C:\Users\rmea\.ubxlibpkg\nrfconnectsdk-v2.5.2\zephyr

...and have freshly copied samples/subsys/usb/console from there, applying the same changes as above to CMakeLists.txt and prj.conf.  Outcome was the same, new build output and  zephyr.hex (the latter renamed to have .txt on the end) attached: let me know what happens when you run that one.
One thing to think about is, is there enough heap?  You have set it 20,000: is it clear that is sufficient for whatever the build is doing?  You might try a bigger number.  It is also strange that your build output complains that both "Debug" and -Os have been applied; I don't know whether that's a real issue or not, I would guess not, since it is a warning rather than an error.

@RobMeades,
Succes! I uninstalled everything and installed the toolchain through the nrf extension pack in VScode.
The code compiles and runs now. Thank you so much for your help!
However..I am now getting a -5 error when opening the device.
Hello World! arm
Opened device with return code -5.
Unable to open GNSS!
Done.

Do you know what it could be?
There is not much description here..

Excellent!  If uDeviceOpen() is returning U_ERROR_COMMON_INVALID_PARAMETER then that will mean that one or more of the parameters you passed into it, in the configuration structure, is invalid.  Can you paste here what you have passed in?

@RobMeades,
I currently have the following:
static const uDeviceCfg_t gDeviceCfg = {
    .deviceType = U_DEVICE_TYPE_GNSS,
    .deviceCfg = {
        .cfgGnss = {
            .moduleType = U_GNSS_MODULE_TYPE_M8,
            .pinEnablePower = U_CFG_APP_PIN_GNSS_ENABLE_POWER,
            .pinDataReady = -1, // Not used
				},
		},
		.transportType = U_DEVICE_TRANSPORT_TYPE_I2C,
    .transportCfg = {
        .cfgI2c = {
            .i2c = U_CFG_APP_GNSS_I2C,
            .pinSda = U_CFG_APP_PIN_GNSS_SDA,
            .pinScl = U_CFG_APP_PIN_GNSS_SCL
				},
		},
};

Thanks, a few comments/questions:

in the device tree fragment you posted at the top of this issue you seem to be setting up i2c0 as the I2C bus where the GNSS device is connected: I guess you have changed this to use i2c1 if you are setting .transportCfg.cfgI2c.i2c to 1?
as explained here, Zephyr, unlike all the other platforms we support, doesn't allow I2C (or UART or SPI) pins to be passed-in by an application, the application must define them in the Zephyr tree, so do make sure that the I2C pins .pinSda and .pinScl are set to -1.

Otherwise the structure looks fine.  During device open the ubxlib code will be communicating with the GNSS device, of course: and by default you should see debug prints of that communication process: could you paste those here?

@RobMeades,
I see, thank you for the support. This is my current overlay (maybe it's usefull for other people in the future):
/ {
	chosen {
		zephyr,console = &cdc_acm_uart0;
	};
};

&zephyr_udc0 {
	cdc_acm_uart0: cdc_acm_uart0 {
		compatible = "zephyr,cdc-acm-uart";
	};
};

&i2c0 {
	status = "okay";
	/* Otherwise board default settings are good. */
};

&pinctrl {
  i2c0_default: i2c0_default {
	  group1 {
		  psels = <NRF_PSEL(TWIM_SDA, 0, 31)>,
			  <NRF_PSEL(TWIM_SCL, 0, 29)>;
	  };
  };

  i2c0_sleep: i2c0_sleep {
	  group1 {
		  psels = <NRF_PSEL(TWIM_SDA, 0, 31)>,
			  <NRF_PSEL(TWIM_SCL, 0, 29)>;
		  low-power-enable;
	  };
  };
};

u_cfg_app_platform_specific.h
U_CFG_APP_GNSS_I2C                        0
U_CFG_APP_PIN_GNSS_SDA               -1
U_CFG_APP_PIN_GNSS_SCL                -1

This is what I get printend in the terminal:
---- Opened the serial port COM9 ----
USB init and terminal opened
U_GNSS: initialising with ENABLE_POWER pin not connected, transport type I2C.
U_GNSS: sent command b5 62 0a 06 00 00 10 3a.

U_GNSS: sent command b5 62 06 04 04 00 00 00 09 00 17 76.
U_GNSS: sent command b5 62 0a 06 00 00 10 3a.

U_GNSS: sent command b5 62 0a 28 00 00 32 a0.
U_GNSS: decoded UBX response 0x0a 0x28: 00 0f 03 03 03 00 00 00 [body 8 byte(s)].
Opened device with return code 0.
Starting position stream.
U_GNSS: sent command b5 62 06 08 00 00 0e 30.
U_GNSS: decoded UBX response 0x06 0x08: e8 03 01 00 01 00 [body 6 byte(s)].
U_GNSS: sent command b5 62 06 01 02 00 01 07 11 3a.
U_GNSS: decoded UBX response 0x06 0x01: 01 07 00 00 00 00 00 00 [body 8 byte(s)].
U_GNSS: sent command b5 62 06 01 08 00 01 07 01 00 00 00 00 00 18 e2.
U_GNSS: decoded UBX response 0x05 0x01: 06 01 [body 2 byte(s)].
Waiting up to 60 seconds for 5 position fixes.
U_GNSS: sent command b5 62 06 01 08 00 01 07 00 00 00 00 00 00 17 dc.
U_GNSS: decoded UBX response 0x05 0x01: 06 01 [body 2 byte(s)].
Done.

I did expect the callback to print out the position. Is that correct?
This is (part of) my main:
static const uDeviceCfg_t gDeviceCfg = {
    .deviceType = U_DEVICE_TYPE_GNSS,
    .deviceCfg = {
        .cfgGnss = {
            .moduleType = U_GNSS_MODULE_TYPE_M8,
            .pinEnablePower = U_CFG_APP_PIN_GNSS_ENABLE_POWER,
            .pinDataReady = -1, // Not used
				},
		},
		.transportType = U_DEVICE_TRANSPORT_TYPE_I2C,
    .transportCfg = {
        .cfgI2c = {
            .i2c = U_CFG_APP_GNSS_I2C,
            .pinSda = U_CFG_APP_PIN_GNSS_SDA,
            .pinScl = U_CFG_APP_PIN_GNSS_SCL
				},
		},
};

typedef struct {
    char *pSomething;
    int32_t somethingElse;
} MyContext_t;

// Count of the number of position fixes received
static size_t gPositionCount = 0;

static char latLongToBits(int32_t thingX1e7,
                          int32_t *pWhole,
                          int32_t *pFraction)
{
    char prefix = '+';

    // Deal with the sign
    if (thingX1e7 < 0) {
        thingX1e7 = -thingX1e7;
        prefix = '-';
    }
    *pWhole = thingX1e7 / 10000000;
    *pFraction = thingX1e7 % 10000000;

    return prefix;
}

// Callback for position reception.
static void callback(uDeviceHandle_t gnssHandle,
                     int32_t errorCode,
                     int32_t latitudeX1e7,
                     int32_t longitudeX1e7,
                     int32_t altitudeMillimetres,
                     int32_t radiusMillimetres,
                     int32_t speedMillimetresPerSecond,
                     int32_t svs,
                     int64_t timeUtc)
{
    char prefix[2] = {0};
    int32_t whole[2] = {0};
    int32_t fraction[2] = {0};

    // Pick up our user context
    MyContext_t *pContext = (MyContext_t *) pUDeviceGetUserContext(gnssHandle);

    // Not using these, just keep the compiler happy
    (void) gnssHandle;
    (void) altitudeMillimetres;
    (void) radiusMillimetres;
    (void) speedMillimetresPerSecond;
    (void) svs;
    (void) timeUtc;
    // We don't actually use the user context here, but
    // of course _you_ could
    (void) pContext;

    if (errorCode == 0) {
        prefix[0] = latLongToBits(longitudeX1e7, &(whole[0]), &(fraction[0]));
        prefix[1] = latLongToBits(latitudeX1e7, &(whole[1]), &(fraction[1]));
        uPortLog("I am here: https://maps.google.com/?q=%c%d.%07d,%c%d.%07d\n",
                 prefix[1], whole[1], fraction[1], prefix[0], whole[0], fraction[0]);
        gPositionCount++;
    }
}

int main(void)
{
	const struct device *const dev = DEVICE_DT_GET(DT_CHOSEN(zephyr_console));
	uint32_t dtr = 0;

#if defined(CONFIG_USB_DEVICE_STACK_NEXT)
	if (enable_usb_device_next()) {
		return 0;
	}
#else
	if (usb_enable(NULL)) {
		return 0;
	}
#endif

	/* Poll if the DTR flag was set */
	while (!dtr) {
		uart_line_ctrl_get(dev, UART_LINE_CTRL_DTR, &dtr);
		/* Give CPU resources to low priority threads. */
		k_sleep(K_MSEC(100));
	}
	printk("USB init and terminal opened\n");

	uDeviceHandle_t devHandle = NULL;
	int32_t returnCode;
	int32_t guardCount = 0;
	MyContext_t context = {0};

	// Initialise the APIs we will need
	uPortInit();
	uPortI2cInit(); // You only need this if an I2C interface is used
	uDeviceInit();

	// Open the device
	returnCode = uDeviceOpen(&gDeviceCfg, &devHandle);
	uPortLog("Opened device with return code %d.\n", returnCode);

	if (returnCode == 0) {
			// Since we are not using the common APIs we do not need
			// to call uNetworkInteraceUp()/uNetworkInteraceDown().

			// If you need to pass context data to
			// uGnssPosGetStreamedStart(), which does not include
			// a user parameter in its function signature, set
			// a user context for ourselves in the device, which
			// we can pick up in callback()
			uDeviceSetUserContext(devHandle, (void *) &context);

			// Start to get position
			uPortLog("Starting position stream.\n");
			returnCode = uGnssPosGetStreamedStart(devHandle,
																						U_GNSS_POS_STREAMED_PERIOD_DEFAULT_MS,
																						callback);
			if (returnCode == 0) {

					uPortLog("Waiting up to 60 seconds for 5 position fixes.\n");
					while ((gPositionCount < 5) && (guardCount < 60)) {
							uPortTaskBlock(1000);
							guardCount++;
					}
					// Stop getting position
					uGnssPosGetStreamedStop(devHandle);

			} else {
					uPortLog("Unable to start position stream!\n");
			}

			// Close the device
			// Note: we don't power the device down here in order
			// to speed up testing; you may prefer to power it off
			// by setting the second parameter to true.
			uDeviceClose(devHandle, false);

	} else {
			uPortLog("Unable to open GNSS!\n");
	}

	// Tidy up
	uDeviceDeinit();
	uPortI2cDeinit(); // You only need this if an I2C interface is used
	uPortDeinit();

	uPortLog("Done.\n");
}

From a quick check that code looks OK.  You are calling uGnssPosGetStreamedStart() and, I guess, giving it your callback() function, then the code is waiting for a minute for the callback() function to be called before it eventually calls uGnssPosGetStreamedStop().
As explained above the function signature for uGnssPosGetStreamedStart():
https://github.com/u-blox/ubxlib/blob/1b94ec51334a5a326328341ad1e5a078692c2f46/gnss/api/u_gnss_pos.h#L245-L259
...you will get a timeout error if the GNSS device has not been able to establish position.
While doing streamed position we switch off printing-out of the messages arriving from the GNSS device as they would get in the way but, if you wish to see them temporarily to confirm that they really are arriving, after your call to uGnssPosGetStreamedStart() you could call uGnssSetUbxMessagePrint(deHandle, true) to switch message printing back on again.

Hi @alehanderoo: were you able to get a GNSS fix in the end?


[kubecost] add helmfile

WIP
what

kubecost added
kube prometheus upgraded to 0.17.0

why

kubecost is awesome dashboard
kubecost is using metrics provided in v0.16.0, so have to upgrade

todo

[x] add support for ingress
[ ] add support for IAP
[x] add support for DNS
[x] add support for our grafana
[x] add support for our prometheues (in progress)

Replaced by #135


python3Packages.discordpy: 1.7.1 -> 1.7.2

Motivation for this change
Update to latest upstream release 1.7.2
Change log: https://discordpy.readthedocs.io/en/stable/whats_new.html#v1-7-2
1.7.2 contains only bugfixes.
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 121792 run on x86_64-linux 1

2 packages built:

python38Packages.discordpy
python39Packages.discordpy

Result of nixpkgs-review pr 121792 at 2abf0fdf run on aarch64-linux 1

2 packages built successfully:

python38Packages.discordpy
python39Packages.discordpy

Looks like semver from the changelog, so should be non-breaking, even though home-assistant pins 1.6.0.

websockets removed. Thanks for checking.


[Backport 1.3] Fix dependency for HDFS, Manually update com.google.jimfs:jimfs from 1.2 to 1.3.0

Description
Potentially fixes CVEs for 1.3

CVE-2023-34453
CVE-2023-34455
CVE-2023-2976

Related Issues
Resolves #[Issue number to be closed when this PR is merged]

Check List

[ ] New functionality includes testing.

[ ] All tests pass

[ ] New functionality has been documented.

[ ] New functionality has javadoc added

[x] Commits are signed per the DCO using --signoff
[ ] Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

 What went wrong:
Could not determine the dependencies of task ':qa:evil-tests:thirdPartyAudit'.
> Could not resolve all files for configuration ':qa:evil-tests:testCompileClasspath'.
   > Could not resolve com.google.guava:guava:32.1.1-jre.
     Required by:
         project :qa:evil-tests > com.google.jimfs:jimfs:1.3.0
      > The consumer was configured to find an API of a library compatible with Java 8, preferably in the form of class files, and its dependencies declared externally. However we cannot choose between the following variants of com.google.guava:guava:32.1.1-jre:
          - androidApiElements
          - jreApiElements
        All of them match the consumer attributes:
          - Variant 'androidApiElements' capabilities com.google.collections:google-collections:32.1.1-jre and com.google.guava:guava:32.1.1-jre and com.google.guava:listenablefuture:1.0 declares an API of a library compatible with Java 8, packaged as a jar, and its dependencies declared externally:
              - Unmatched attributes:
                  - Provides attribute 'org.gradle.jvm.environment' with value 'android' but the consumer didn't ask for it
                  - Provides release status but the consumer didn't ask for it
          - Variant 'jreApiElements' capabilities com.google.collections:google-collections:32.1.1-jre and com.google.guava:guava:32.1.1-jre and com.google.guava:listenablefuture:1.0 declares an API of a library compatible with Java 8, packaged as a jar, and its dependencies declared externally:
              - Unmatched attributes:
                  - Provides attribute 'org.gradle.jvm.environment' with value 'standard-jvm' but the consumer didn't ask for it
                  - Provides release status but the consumer didn't ask for it

Base JDK is incompatible with newer versions


Carthage support

The Carthage doesn't quite work.
Project "FastImageCacheDemo.xcodeproj" has no shared schemes

Just need to the check the box to share the framework scheme. Thanks!

cc @RuiAAPeres

The scheme was shared in this commit: https://github.com/path/FastImageCache/commit/ea7650d78143015e57f32d4df0e9a9e927e9fbac

So the issue, is that by default carthage will point to the latest release and not the latest commit. This fixes the issue @soffes:
github "path/FastImageCache" "8165e0a9bbf0f643805f609184a5341f966b6724"

I think this was technically done in https://github.com/path/FastImageCache/pull/99, but I can't seem to get the Carthage output to work. https://github.com/path/FastImageCache/pull/122 fixes it, once that's merged this could be closed.

@timonus for what it's worth, it works in my fork

:+1: nice


Revert "Update readme.md"

@Bigrig7272gg what's the reason for reverting https://github.com/transitive-bullshit/chatgpt-api/pull/32 by @coskuncay?
I know it's not exactly using this package, but it's a direct port.

@Bigrig7272gg please stop creating PRs that undo other people's PRs. They will not be merged, especially if you don't include an explanation.

@transitive-bullshit Thank you!🚀


Implement cosmetic (css/js/extcss) rules logic

Subj

JNI done, now java part remains.

testGenericHideRule failed

testUblockCssInjectionSyntaxSupport failed

testCssInjectionRules failed

All failing java tests are fixed in commit 06b0d84c8033f89586af6bbda185ee1f84d8714c.

testExtendedCssGenericRules failed.
The thing is that ExtCss rules shouldn't be grouped like we do it for regular CSS rules.
Regular:
selector1, selector2 { style }
ExtCss: selector1 { style } selector2 { style }
The fix should be more or less trivial.

Fixed

Tests for NativeJsFilter fail, there's garbage in the end of the buildScript result

Added line termination:
    if (result.used) {
        if (!add_to_str(&result, "", 1)) {
            return NULL;
        }
    }

Was merged in commit 5c4bebeae866ef9bd60370209a387f1ea389a878.


Add npm too

Hello @aarondfrancis,
I have created pull request for same(https://github.com/aarondfrancis/pulse-outdated/pull/11).


Client account - account settings should be change to profile

Expected behavior
Per the US #516

Change current OOTB label 'Account Settings' to 'Profile' and move to the front of the tabs list.

Actual behavior
Still displays "account settings'" and is not in the front

Steps to reproduce behavior
Login to client account, view tabs.

https://razoyo.atlassian.net/browse/TST-129
PR #200 is pending Jared to push out on 7/27 5:47pm

7/28 SH - Label has been changed to Profile, however was not moved to the front of the list. Jen is going to do this and send in another PR or let me know via teams if it's not possible.

7-29 Profile has been moved to the front of the list now.


Clicking arrows doesn't update ng-dirty

When modifying time in the inline timepicker using the arrows, the form state doesn't change (ng-pristine/ng-dirty are not applied).
When modifying the time by mouse/keyboard, it updates the form state as expected.
Angular 1.6.4

Cannot reproduce in 1.0.7


add direct formatting option to ImageKeyword struct with tests

For example usage, see the code in the test files ImCreate_cube.c and ImCreate_img.c.
Key points:

specify F keyword type for formatted keywords
Format as added in the place of the space-padding uint64 in the ImageKeyword struct. The number of bytes is, by default, set to 8 to match the original padding of the struct.

This should be sufficient for all format strings as they are %-16.16s, which is 8 bytes, but if we need breathing room, we can save the format strings without the % since it is redundant.

Struct version incremented to 2.00 -> 2.01; this is not a breaking change, just a feature addition.

To fully implement these changes, code will need to be changed in milk (save_fits.c) and camstack.

Unfortunately, I have comments...
Changing type to 'F' is breaking
--> We need to hang on to native formats everywhere, and at the very last step of FITS-ifying we need to figure out that .format is not a null. Otherwise, all the other stuff loses capability to distinguish D, F and S.
Now, how to know a keyword is formattable?

Having non null format field: yes but then everytime a keyword is reused, the format field MUST positively be cleared

Add new formatters 'd' 'f' 's' for formatted int, float, string: that somewhat works, and lower/upper lets the FITS writer know whether the format is current or stale, and lets camstack set the formatters to lowercase based on the knowledge that there indeed is a valid formatter. HOWEVER, every single code that performs a kw.type check against 'D', 'S', 'F' must be changed to compare (0x7f & kw.type == kw.type.upper) instead.


Remove Crystal::Digest implementations

Crystal::Digest::MD5 is an internal implementation used by the compiler, in only one location:
https://github.com/crystal-lang/crystal/blob/33c97d419c88b4c6595786597f0ed04c7973b283/src/compiler/crystal/compiler.cr#L644
But this use case doesn't need MD5 at all. The purpose is to restrict long names to 50 characters while still having a unique identifier. So there's really no need for a cryptographic hash function.
The same can be achieved much simpler. For example String#hash should do as a hash function. Or it could use a counter and names get assigned sequential numeric identifiers.
If we change that, we can remove Crystal::Digest::MD5 from the code base.

There's also Crystal::Digest::SHA1 which has a similar purpose only for the compiler's playground when compiled without openssl.
https://github.com/crystal-lang/crystal/blob/33c97d419c88b4c6595786597f0ed04c7973b283/src/http/web_socket/protocol.cr#L330-L332
Techincally it's also used for any program that uses the websocket implementation is compiled without openssl. But that's not really an intended use case.
Since the playground is only a development tool security concerns really don't matter much. But it seems impossible to avoid because the Sec-WebSocket-Accept header with a SHA1 hashed value is mandatory for establishing a websocket connection.

Ideally any program wanting to use WebSocket should compile with openssl, for security reasons and also performance.
The playground should not be needed to bootstrap the compiler. An option is to not have the Playground at all when openssl is not available. Then, no Digest will be needed anymore.

Ideally any program wanting to use WebSocket should compile with openssl, for security reasons and also performance.
The playground should not be needed to bootstrap the compiler. An option is to not have the Playground at all when openssl is not available. Then, no Digest will be needed anymore.

Compiler builds are usually not linked against openssl (AFAIK because static linking is difficult). So that would mean the playground would generally not be available. So that would be a major change and basically drop the playground from the compiler entirely.

Compiler builds are usually not linked against openssl (AFAIK because static linking is difficult). So that would mean the playground would generally not be available. So that would be a major change and basically drop the playground from the compiler entirely.

There's simply no rush at all to change this.

There's simply no rush at all to change this.

You're both right. Then, for the moment, just changing the MD5 use in crystal/src/compiler/crystal/compiler.cr will be good.
I don't know how the playground could possibly use the local host's openssl library after being compiled.
As a side note, if digests are dropped from this repository, could they be moved in a repository?

You're both right. Then, for the moment, just changing the MD5 use in crystal/src/compiler/crystal/compiler.cr will be good.
I don't know how the playground could possibly use the local host's openssl library after being compiled.
As a side note, if digests are dropped from this repository, could they be moved in a repository?

@asterite No rush, yes. But there should be an issue to track this.

As a side note, if digests are dropped from this repository, could they be moved in a repository?

This is only about internal implementations. They're not intended for public use. So there's no reason to make it available in separate repo. At least not in crystal-lang, if somebody wants to publish a shard there's noone stopping you. I wouldn't recommend it though.

It would be good for educational purposes, and some non security related cases. Their implementations could even be improved and audited, who knows.

Why would you remove those digests? It's kinda useful to be able to compute a MD5/SHA* sum in order to veryfy a download, in a os-agnostic way and without having to write to file.

These are pure Crystal implementations and neither optimized for performance nor security (not that the latter matters too much anymore).
Both digest methods are available in stdlib via bindings to ˋlibcryptoˋ in the ˋOpenSSLˋ namespace. So as long as you can link ˋlibcryptoˋ, you already have a much better implementation available and there's no need for primitive implementation in pure Crystal.


Suggestions: Dark theme and string examples

Great job on this app, it really simplifies looking for groups in 840. But I'm wondering if its possible to add a few new features such as:

A dark theme for us nightcrawlers
Example strings, wiking regular expressions is fine, but fairly hard to understand for non coders.
Clear chat button

had to do some research to make this string since im not using it for rota etc;
^(?!.(vagan|elreon|zana|vorici|haku|catarina|tora|trail|uber|lab|rota)).(LF|war|vaal|box).*$

Here you go: https://github.com/Bunni/global820/releases/tag/1


fix: #8040 symlink name file doesn't appear

Resolved / Related Issues
Items resolved / related issues by this PR.

Related #8040

Details of Changes
Add details of changes here.

Use ItemNameRaw instead of his extension

Validation
How did you test these changes?

[x] Built and ran the app
[x] Tested the changes for accessibility

Screenshots (optional)

@acesyde before we merge this, can you add your agreement to #7625?

@acesyde thank you!


[Env] Fix SMAC

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (492d1c6) 87.42% compared to head (59ea12c) 45.06%.

Additional details and impacted files
@@             Coverage Diff             @@
##             main      #40       +/-   ##
===========================================
- Coverage   87.42%   45.06%   -42.36%
===========================================
  Files          56       56
  Lines        2576     2576
===========================================
- Hits         2252     1161     -1091
- Misses        324     1415     +1091

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


cannot upload files or fetch web in production [BUG]:

How are you running AnythingLLM?
Not listed
What happened?
I know you have mentioned prod issues wont be supported  but I really need a help as I'm doing modifications on translating some labels.
I am running in production without docker.
Everything looks as good as in devlopment  except uploading files. I cannot click to upload. This doesn't happen in development. Everything goes well in development.
I noticed that prod:collector is missing in package.json. Then I follow the BARE_METAL.md to manually start collector but it reports the same error as https://github.com/Mintplex-Labs/anything-llm/issues/1589
Are there known steps to reproduce?
export STORAGE_DIR=$HOME/anythingllm
touch $HOME/anythingllm/.env
yarn prod:frontend
cp -R frontend/dist server/public
yarn prod:server
Because I couldn't click to upload, I added a step:
cd collector && NODE_ENV=production node index.js &
AnythingLLM was running on http://localhost:3001 but I got the same issue as https://github.com/Mintplex-Labs/anything-llm/issues/1589
The collector didn't output any info on shell prompt.
Any workaround please?

Here is a simple helper script:
echo "Rebuilding Frontend"
cd /home/ubuntu/anything-llm/frontend && yarn && yarn build && cd /home/ubuntu/anything-llm

echo "Copying to Sever Public"
rm -rf server/public
cp -r frontend/dist server/public

echo "Killing node processes"
pkill node

echo "Installing collector dependencies"
cd /home/ubuntu/anything-llm/collector && yarn

echo "Installing server dependencies & running migrations"
cd /home/ubuntu/anything-llm/server && yarn
cd /home/ubuntu/anything-llm/server && npx prisma migrate deploy --schema=./prisma/schema.prisma
cd /home/ubuntu/anything-llm/server && npx prisma generate

echo "Booting up services."
truncate -s 0 /logs/server.log
truncate -s 0 /logs/collector.log

cd /home/ubuntu/anything-llm/server
(NODE_ENV=production node index.js) &> /logs/server.log &

cd /home/ubuntu/anything-llm/collector
(NODE_ENV=production node index.js) &> /logs/collector.log &

notice that we have to copy the frontend to server public then run the equivalent of prod:server. This is because in production the frontend is served from the server public folder. The we start the collector and server is two processes.


Space in extension path: Multiworkspace failures

Module Installer
Validate pip install arguments  without a resource:
Invalid command sent to terminal for installation.

expected - actual

-"/home/runner/work/vscode-python/vscode-python/path with spaces/pythonFiles/pyvsc-run-isolated.py" pip install -U xyz --user
+/home/runner/work/vscode-python/vscode-python/path with spaces/pythonFiles/pyvsc-run-isolated.py pip install -U xyz --user
at Context. (out/test/common/moduleInstaller.test.js:356:47)
at processTicksAndRejections (internal/process/task_queues.js:94:5)

Module Installer
Validate Conda install arguments  without a resource:
Invalid command sent to terminal for installation.

expected - actual

-"/home/runner/work/vscode-python/vscode-python/path with spaces/pythonFiles/pyvsc-run-isolated.py" pip install -U xyz
+/home/runner/work/vscode-python/vscode-python/path with spaces/pythonFiles/pyvsc-run-isolated.py pip install -U xyz
at Context. (out/test/common/moduleInstaller.test.js:379:47)
at processTicksAndRejections (internal/process/task_queues.js:94:5)

Module Installer
Validate pip install arguments  with a resource:
Invalid command sent to terminal for installation.

expected - actual

-"/home/runner/work/vscode-python/vscode-python/path with spaces/pythonFiles/pyvsc-run-isolated.py" pip install -U xyz --user
+/home/runner/work/vscode-python/vscode-python/path with spaces/pythonFiles/pyvsc-run-isolated.py pip install -U xyz --user
at Context. (out/test/common/moduleInstaller.test.js:356:47)
at processTicksAndRejections (internal/process/task_queues.js:94:5)

Module Installer
Validate Conda install arguments  with a resource:
Invalid command sent to terminal for installation.

expected - actual

-"/home/runner/work/vscode-python/vscode-python/path with spaces/pythonFiles/pyvsc-run-isolated.py" pip install -U xyz
+/home/runner/work/vscode-python/vscode-python/path with spaces/pythonFiles/pyvsc-run-isolated.py pip install -U xyz
at Context. (out/test/common/moduleInstaller.test.js:379:47)
at processTicksAndRejections (internal/process/task_queues.js:94:5)

Multiroot Linting
Enabling Pylint in root and also in Workspace, should return errors:
AssertionError [ERR_ASSERTION]: No errors returned by linter

expected - actual

-false
+true
at testLinterInWorkspaceFolder (out/test/linters/lint.multiroot.test.js:67:16)
at runMicrotasks ()
at processTicksAndRejections (internal/process/task_queues.js:94:5)
at runTest (out/test/linters/lint.multiroot.test.js:94:9)
at Context. (out/test/linters/lint.multiroot.test.js:70:9)

Multiroot Linting
Disabling Pylint in root and enabling in Workspace, should return errors:
AssertionError [ERR_ASSERTION]: No errors returned by linter

expected - actual

-false
+true
at testLinterInWorkspaceFolder (out/test/linters/lint.multiroot.test.js:67:16)
at runMicrotasks ()
at processTicksAndRejections (internal/process/task_queues.js:94:5)
at runTest (out/test/linters/lint.multiroot.test.js:94:9)
at Context. (out/test/linters/lint.multiroot.test.js:76:9)

Multiroot Linting
Enabling Flake8 in root and also in Workspace, should return errors:
AssertionError [ERR_ASSERTION]: No errors returned by linter

expected - actual

-false
+true
at testLinterInWorkspaceFolder (out/test/linters/lint.multiroot.test.js:67:16)
at runTest (out/test/linters/lint.multiroot.test.js:94:9)
at Context. (out/test/linters/lint.multiroot.test.js:79:9)

Multiroot Linting
Disabling Flake8 in root and enabling in Workspace, should return errors:
AssertionError [ERR_ASSERTION]: No errors returned by linter

expected - actual

-false
+true
at testLinterInWorkspaceFolder (out/test/linters/lint.multiroot.test.js:67:16)
at runTest (out/test/linters/lint.multiroot.test.js:94:9)
at Context. (out/test/linters/lint.multiroot.test.js:85:9)

Unit Tests - debugging
Debugger should start (nosetest):
Incorrect number of test files

expected - actual

-0
+2
at testStartingDebugger (out/test/testing/debugger.test.js:95:23)
at runMicrotasks ()
at processTicksAndRejections (internal/process/task_queues.js:94:5)
at Context. (out/test/testing/debugger.test.js:126:9)

Unit Tests - debugging
Debugger should stop when user invokes a test discovery (nosetest):
Incorrect number of test files

expected - actual

-0
+2
at testStoppingDebugger (out/test/testing/debugger.test.js:132:23)
at runMicrotasks ()
at processTicksAndRejections (internal/process/task_queues.js:94:5)
at Context. (out/test/testing/debugger.test.js:154:9)

Unit Tests - debugging
Debugger should not stop when test discovery is invoked automatically by extension (nosetest):
Incorrect number of test files

expected - actual

-0
+2
at testDebuggerWhenRediscoveringTests (out/test/testing/debugger.test.js:160:23)
at runMicrotasks ()
at processTicksAndRejections (internal/process/task_queues.js:94:5)
at Context. (out/test/testing/debugger.test.js:190:9)

Unit Tests - nose - discovery against actual python process
Discover Tests (single test file):
AssertionError [ERR_ASSERTION]: Incorrect number of test files

expected - actual

-0
+2
at Context. (out/test/testing/nosetest/nosetest.test.js:63:16)
at runMicrotasks ()
at processTicksAndRejections (internal/process/task_queues.js:94:5)

Unit Tests re-discovery
Re-discover tests (nosetest):
Incorrect number of test files

expected - actual

-0
+2
at discoverUnitTests (out/test/testing/rediscover.test.js:61:23)
at runMicrotasks ()
at processTicksAndRejections (internal/process/task_queues.js:94:5)
at Context. (out/test/testing/red

Steps to repro:

clone into path with a space
Run multiWorkspace test

Or if that doesn't work, use a github action as done in this PR
https://github.com/microsoft/vscode-python/pull/14113

Not able to repro in main.


Claim language in test cases is incorrect

https://github.com/0xcert/ethereum-erc721/blob/master/test/tokens/NFToken.test.js#L30-L32

'throws when trying to mint 2 NFTs with the same claim'

There are no claims in erc-721 baseline implementation.

#139


Revert "Update github.com/miekg/dns to pull in bug fixes."

@fabxc have to bother you again. Apparently, the updated miekg/dns packages doesn't deal well with long responses. That looks like a bad regression. On the other hand, we discussed now and then if we can finally use the standard Go DNS library again. I don't remember the details, and for a bugfix release, I prefer for now to just keep the DNS package where it was, even if that means accepting other known bugs. We need to revisit the issue separately later.
This reverts commit 590446434f3d840dc0b20cd2686c0c9e7790be23.
Sadly, there's a problem with truncated messages:
WARN[0684] resolving all.telemetry.pm.prod.bazooka.dd.srv.int.s-cloud.net failed: dns: failed to unpack truncated message  source=dns.go:187
We have to revisit the SRV DNS part later.

Note: I am running an "internal RC" with a quite heavy load to find bugs like this one.

:+1:

Thanks @brian-brazil .
I have also re-opened https://github.com/prometheus/prometheus/issues/360 to not forget about the issue.


The updateSql command doesn't invoke the generateStatements method on any instances of CustomSqlChange.

Search first

[X] I searched and no similar issues were found

Description
The updateSql command doesn't invoke the generateStatements method on any instances of CustomSqlChange.  Therefore, When running the updateSql command, it's not possible to see the statements generated by any instances of CustomSqlChange.
Steps To Reproduce
Run the updateSql command when you have some changesets that use a CustomSqlChange.
Expected/Desired Behavior
the generateStatements method should be invoked and the sql generated with the updateSql command.
Liquibase Version
4.21.1
Database Vendor & Version
No response
Liquibase Integration
No response
Liquibase Extensions
No response
OS and/or Infrastructure Type/Provider
No response
Additional Context
No response
Are you willing to submit a PR?

[ ] I'm willing to submit a PR (Thank you!)

Had the same issue, worked around by writing a custom liquibase change (as described here: https://contribute.liquibase.com/extensions-integrations/extensions-overview/your-first-extension/)


Added support for nga11y-form if there is multiple forms in the view.

nga11y-form directive will not work properly if we have more than one form in the view. It will set the focus on the last form on the view even though we submit first form. This commit should fix that issue.

@dylanb  Could you please go through this pull request ?

Hi @chandramuralis, thanks for your contribution! How about adding a test for multiple forms? I can help, if needed.

Sure I can create some unit tests if you can point me to which file I should add my cases.

@chandramuralis it looks like we would need to add test/nga11yforms.js, as it doesn't exist yet. I created a basic skeleton and pushed it to a branch to save you some time: https://github.com/dequelabs/ngA11y/blob/wip-forms-tests/test/nga11yforms.js

I've written a test for this and made some changes, @chandramuralis do you mind checking out #28?

Seems like this is taken care as part of #28. Closing the pull request

This change is not being released, is there any plan to release this change?