can i use mqtt with this library?

I can use the MQTT protocol to communicate nrf24l01 from a raspberry to an esp8266, it is that I want to implement a sensor network.
Thank you

Hi @potier97,
You can node-red with this module to build a bridge between nrf24l01 devices to mqtt without much effort: With nrf24 input and output nodes routed to some logic nodes and mqtt nodes provided by node-red by default. If the logic it will take only some minutes to do it.


feat: Name & WriteLowPri methods

Added Name() for easier debugging & and WriteLowPri() for pruning.

Codecov Report

Merging #35 (3384b47) into main (fcfa67d) will increase coverage by 13.06%.
The diff coverage is 64.52%.

@@             Coverage Diff             @@
##             main      #35       +/-   ##
===========================================
+ Coverage   52.60%   65.66%   +13.06%
===========================================
  Files          29       32        +3
  Lines        1785     1404      -381
===========================================
- Hits          939      922       -17
+ Misses        740      421      -319
+ Partials      106       61       -45

Impacted Files
Coverage Δ

badgerdb/batch.go
96.15% <0.00%> (-3.85%)
:arrow_down:

boltdb/batch.go
80.55% <0.00%> (-2.31%)
:arrow_down:

cleveldb/batch.go
77.14% <0.00%> (-2.27%)
:arrow_down:

goleveldb/batch.go
90.00% <0.00%> (-3.11%)
:arrow_down:

memdb/batch.go
90.90% <0.00%> (-2.85%)
:arrow_down:

memdb/db.go
55.88% <0.00%> (-0.84%)
:arrow_down:

metadb/db.go
47.05% <ø> (ø)

rocksdb/batch.go
68.29% <0.00%> (-14.07%)
:arrow_down:

badgerdb/db.go
50.00% <50.00%> (-0.73%)
:arrow_down:

boltdb/db.go
44.08% <50.00%> (-0.48%)
:arrow_down:

... and 10 more

Impacted Files
Coverage Δ

badgerdb/batch.go
96.15% <0.00%> (-3.85%)
:arrow_down:

boltdb/batch.go
80.55% <0.00%> (-2.31%)
:arrow_down:

cleveldb/batch.go
77.14% <0.00%> (-2.27%)
:arrow_down:

goleveldb/batch.go
90.00% <0.00%> (-3.11%)
:arrow_down:

memdb/batch.go
90.90% <0.00%> (-2.85%)
:arrow_down:

memdb/db.go
55.88% <0.00%> (-0.84%)
:arrow_down:

metadb/db.go
47.05% <ø> (ø)

rocksdb/batch.go
68.29% <0.00%> (-14.07%)
:arrow_down:

badgerdb/db.go
50.00% <50.00%> (-0.73%)
:arrow_down:

boltdb/db.go
44.08% <50.00%> (-0.48%)
:arrow_down:

... and 10 more


Regenerate vision client

This PR was generated using Autosynth. :rainbow:
Here's the log from Synthtool:
synthtool > You are running the synthesis script directly, this will be disabled in a future release of Synthtool. Please use python3 -m synthtool instead.
synthtool > Ensuring dependencies.
synthtool > Pulling artman image.
latest: Pulling from googleapis/artman
Digest: sha256:2f6b261ee7fe1aedf238991c93a20b3820de37a343d0cacf3e3e9555c2aaf2ea
Status: Image is up to date for googleapis/artman:latest
synthtool > Cloning googleapis.
synthtool > Running generator for google/cloud/vision/artman_vision_v1.yaml.
synthtool > Generated code into /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java.
synthtool > Running java formatter on 62 files
synthtool > Running java formatter on 2 files
synthtool > Running java formatter on 162 files
synthtool > Running generator for google/cloud/vision/artman_vision_v1p1beta1.yaml.
synthtool > Generated code into /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java.
synthtool > Running java formatter on 62 files
synthtool > Running java formatter on 1 files
synthtool > Running java formatter on 67 files
synthtool > Running generator for google/cloud/vision/artman_vision_v1p2beta1.yaml.
synthtool > Generated code into /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java.
synthtool > Running java formatter on 62 files
synthtool > Running java formatter on 1 files
synthtool > Running java formatter on 91 files
synthtool > Running generator for google/cloud/vision/artman_vision_v1p3beta1.yaml.
synthtool > Generated code into /home/kbuilder/.cache/synthtool/googleapis/artman-genfiles/java.
synthtool > Running java formatter on 62 files
synthtool > Running java formatter on 2 files
synthtool > Running java formatter on 165 files
synthtool > Cleaned up 0 temporary directories.

This should be fixed now, if synthtool is run again. My local regen: https://github.com/andreamlin/google-cloud-java/pull/2


The remote? method should allow for files with http in their names

If a filename has an http(s) in the name it is mistaken for a remote:
# filename: /tmp/movie_of_http_webpage.mov

# Run:

     Failure/Error: response = http.request_head(url.request_uri)

     NoMethodError:
       undefined method `request_uri' for #<URI::Generic:0x00005640e7574d78>

The search for http(s) should be at the start of the string.

@streamio Do you need any more information on this PR?


MEITNER milestone M2.1 report needs review

Hello @mitchellwhalen.
Finally, the report is ready. You help is very appreciated.
In case you wanna compile by your own: https://github.com/andrewryh/meitner-task-2/tree/master/doc/2019-06-q4-report
Otherwise, just use PDF.
2019-06-q4-report.pdf

Hello @mitchellwhalen
What's the status?


merge latest libnetwork

This change brings in latest libnetwork into godeps.
This is required for working with latest experimental docker due to remote driver REST api format change(They removed multiple interfaces per endpoint).
There is no logic change.
If someone can do a quick LGTM that'll be great.

@shaleman
changes LGTM, but the sanity failed due to a compilation error in netlink library, can you please check?

Centos compilation seem to be failing for some reason. Ubuntu compilation went thru fine.
I haven’t tried Centos in my testbed yet. I’ll look into it.
-sukhesh
From: Madhav Puri <notifications<EMAIL_ADDRESS>Reply-To: contiv/netplugin <reply<EMAIL_ADDRESS>Date: Monday, September 28, 2015 at 8:31 AM
To: contiv/netplugin <netplugin<EMAIL_ADDRESS>Cc: Sukhesh Halemane <shaleman<EMAIL_ADDRESS>Subject: Re: [netplugin] merge latest libnetwork (#152)
@shalemanhttps://github.com/shaleman
changes LGTM, but the sanity failed due to a compilation error in netlink library, can you please check?
—
Reply to this email directly or view it on GitHubhttps://github.com/contiv/netplugin/pull/152#issuecomment-143779359.


[new feature request] : The ability to resume downloading from where it left off

can you implement/add a new feature the ability to resume incomplete downloads and append data to an existing file?
have a look at this: https://dev.to/rlazom/resume-downloads-in-flutter-with-dio-abc
I know you are using the HTTP package in this package. can you do it with the HTTP package? (when you have time 🙂)?

@Shahmirzali-Huseynov, do you have an issue with resuming? I use my other package named download_task based on http, and it supports the resuming, as well as this package do.

I'm not meaning normal resume and pause functionality.
**Current issue**
I was facing a problem with my current project. I need to handle large video files, sometimes the download would not complete and every time the user accesses this specific view the download starts again from the beginning.

At the time of writing this post, the dio or http does not have the ability to append data to an existing file during the download.

@Shahmirzali-Huseynov, there is a link to my package based on http, and yes, it can append data to existing file.

@starkdmi for example, I start downloading and its progress is 50%. at his time I close the app and after a while opened it again. Will the download start from where it left off? ( I mean from 50%) ?

@Shahmirzali-Huseynov, if the file wasn't deleted - it should.

I tested your example sample. yeah, your right but for a single download.
but for multiple downloads (https://github.com/starkdmi/flutter_download_manager/issues/1#issuecomment-1592085966) it's not resumed. it starts from 0.  -> https://youtube.com/shorts/q7R5RVGYwG0?feature=share

@Shahmirzali-Huseynov, I do remember you had code for cleaning the downloading directory, did you disabled it?
Let's say 20 of 100 files are downloaded, you closed the app, restart the app and downloading - you will pass 100 links to DownloadManager which will process them in queue using available Isolates, if file exists, it tries to continue downloading. Now it depends on server response, if server sent file size which is equals to downloaded - done, let's move to next link, but if your device have just a part of file - it continues.
The main thing here what you still will have summary progress 0% on restart. As you know there is no such a thing as summary progress in my code, so the logic we used to calculate it can be changed to take in calculations amount of downloaded file. For instance, I wouldn't pass the already downloaded files to the DownloadManager, I will remove those from all links when file is downloaded.

@starkdmi I wanted to clear when there is an error or the user didn't finish downloading. but now I wanna resume where the download left off.

What do you think is there a workaround or solution for multiple downloads? (https://github.com/starkdmi/flutter_download_manager/issues/1#issuecomment-1592085966)

by the way, I removed the clear function from init. but same starting from again(0).
static Future clean({removeDirectory = false}) async {
// remove files in default directory
final dir = Directory(directory);
if (await dir.exists()) {
await for (final file in dir.list()) {
if (await file.exists()) {
await file.delete();
}
}
if (removeDirectory) {
dir.delete();
}
}
}

and what must happen if the URL/file is downloaded before ? this control up to us or? package doing something for it?

@Shahmirzali-Huseynov, it's totally on you, the package doesn't store the downloaded files in db, so it cannot skip already downloaded files on re-run, but will check if it's downloaded fully and continue if needed.

@Shahmirzali-Huseynov, if I re-run the downloading and the file is partially stored on the device, it continues for me. But it should be the same directory (temp directory can change between the runs).

for single downloading yes, but not for multiple.

@Shahmirzali-Huseynov, to show the correct progress in your case you can just get the amount of files stored in the directory before running the downloading. Some of them may be incomplete, but the amount of them isn't greater than the isolates amount.

@Shahmirzali-Huseynov, if I re-run the downloading and the file is partially stored on the device, it continues for me. But it should be the same directory (temp directory can change between the runs).

for single downloading yes, but not for multiple.

@Shahmirzali-Huseynov, the downloading is fully separated, each runs on its own Isolate in parallel. There should be no difference between one or many.

@starkdmi then the problem is about calculating total progress?

Open Ai chat gpt 4 answer to my question (file is downloaded completely 100% or not) :
You want to ensure that the MP3 file is completely downloaded before playing it. This can be a bit more challenging because the file system doesn't inherently know if the file was fully downloaded or not.
One common approach to solve this issue is by downloading the file with a temporary name or extension. Once the file is fully downloaded, you can then rename it to its final name. Here's a simple illustration of this strategy:
import 'dart:io';
import 'package:http/http.dart' as http;
import 'package:path/path.dart';

void downloadFile(String url, String filePath) async {
  var request = await HttpClient().getUrl(Uri.parse(url));
  var response = await request.close();
  var bytes = await consolidateHttpClientResponseBytes(response);
  File file = File(filePath + ".temp");
  await file.writeAsBytes(bytes);

  // Rename the file to indicate it has finished downloading
  await file.rename(filePath);
}

void playFile(String filePath) async {
  File file = File(filePath);

  // Check if the file exists, if not, it means the file hasn't finished downloading
  if (await file.exists()) {
    // Play the file
  } else {
    print("File not found, it may not have finished downloading.");
  }
}

In this code, we download the file with a .temp extension, and once the file has fully downloaded, we rename it to its final name. When you want to play the file, you first check if the file with the final name exists. If it does, you can play it. If not, it means the file hasn't finished downloading.
This is a simple solution and might not cover all edge cases, but it should give you a starting point to solve your issue.

@starkdmi there is another problem 😩  I think each download is being appended to the file. it's problem. ceause normally mp3 is 1 hour but when I resume 3,4 times it is 2,3 hour 😕.
I calculated the megabyte file size.
normal download: (file size 47 MB)
https://github.com/starkdmi/download_manager/assets/49102327/4c557e06-8285-410a-ad3c-b60373ceb726
with resuming: (file size 180 MB)
https://github.com/starkdmi/download_manager/assets/49102327/93f7c61d-d92a-49a2-b36a-504e469d950e
my function :
Future<void> startSurahDownload({required int surahNumber}) async {
    final downloader = DownloadManager.instance;
    log("Globals.directory ${Globals.directory}");
    final String tempPath = '${Globals.directory}/abdullaah_3awwaad_al-juhaynee_$surahNumber.mp3.temp';
    final String finishedPath = '${Globals.directory}/abdullaah_3awwaad_al-juhaynee_$surahNumber.mp3';
    final request = downloader.download(
      'https://download.quranicaudio.com/quran/abdullaah_3awwaad_al-juhaynee//${getFormattedNumber(surahNumber)}.mp3',
      path: tempPath,
    );

    request.events.listen(
      (event) async {
        if (event is double) {
          // Calculate the progress
          final totalProgress = event;

          state = (totalProgress, null);

          print("Total progress: ${totalProgress}");
        }
        if (event == DownloadState.finished) {
          print("Download finished and renamed");
          File oldFilePath = File(tempPath);
          await oldFilePath.rename(finishedPath);

          ByteData data = await rootBundle.load(finishedPath);

          // Get file size in bytes
          final megabytes = data.lengthInBytes ~/ (1024 * 1024);
          state = (1.0, megabytes);
        }
      },
      onError: (error) {
        log("onError: $error");
      },
    );
  }

@Shahmirzali-Huseynov, could you try to set the  safeRange to false? It's an option which reflects server response headers.

@starkdmi Yeah you are right, I add safeRange to false, it's working as excepted. can you explain a bit more about what is the safeRange ?

@Shahmirzali-Huseynov, whenever you want to continue downloading to existing file you need to known the full size of the file on server. This info sent to server in a Range header via bytes=$from-$size. By setting safeRange to false you allow HTTP client to use bytes=$from- header, which has no ending point. The code for this parameter is located here. The download_task package have some options to be compatible with different server configurations and this is the one of them.

What must happen when safeRange to true or null? @starkdmi

@Shahmirzali-Huseynov, when the safeRange is set to true, and no file length is known the Range header is not set. That's common for an initial response, not a continuing one, but may be used to enforce the file length confirmation on the server side.

@Shahmirzali-Huseynov, as for a null, the default in DownloadManager instance is true. You can set it globally for a manager or pass it to each separate download request.

I think My situation was safeRange true and i have fileSize content length from server. Only this time you didn't tell what should happen))? @starkdmi

@Shahmirzali-Huseynov, the Range header is calculated and sent to the server before any response. Here is the code for the safeRange, and the full code is just a 300+ lines for a resumable http client, you may scroll it to be in a context 🧑💻

sorry I was on the phone. I will look at the codes. thanks.


layer was never built but weights file lists have variables

I've simple model built in tf.keras 2.12. To move keras v3, I tried

save weight from tf.keras as .save_weights("model.weights.h5")
translate tf.keras to keras v3
tried to load weight model.load_weights("model.weights.h5")

But it gives error, saying some layers were not built but the weights have states. Following are the model (translated to keras v3) and attempt.
import tensorflow as tf
import keras
from keras import layers
from keras import ops

class SubModel(keras.Model):
    def __init__(self, num_layer_norm):
        super().__init__()
        self.dense_layers = [
            layers.Dense(64, activation='relu') for _ in range(num_layer_norm)
        ]
        self.layer_norms = [
            layers.LayerNormalization() for _ in range(num_layer_norm)
        ]
        self.output_layer = layers.Dense(32, activation='relu')

    def call(self, inputs):
        x = inputs
        for i, (dense, layer_norm) in enumerate(zip(self.dense_layers, self.layer_norms)):

            if i < 2:
                continue

            x = dense(x)
            x = layer_norm(x)
        output = self.output_layer(x)
        return output

class CombinedModel(keras.Model):
    def __init__(self, num_submodels, num_layer_norm):
        super().__init__()
        self.submodels = [
            SubModel(num_layer_norm) for _ in range(num_submodels)
        ]
        self.combine_layer = layers.Dense(
            10, activation='softmax'
        )

    def call(self, inputs):
        submodel_outputs = [submodel(inputs) for submodel in self.submodels]
        combined_output = ops.concatenate(submodel_outputs, axis=-1)
        final_output = self.combine_layer(combined_output)
        return final_output

num_submodels = 5
num_layer_norm = 10
model = CombinedModel(num_submodels, num_layer_norm)
model.build(input_shape=(None, 100))
model.summary()

/opt/conda/lib/python3.10/site-packages/keras/src/layers/layer.py:357: UserWarning: build() was called on layer 'combined_model_2', however the layer does not have a build() method implemented and it looks like it has unbuilt state. This will cause the layer to be marked as built, despite not being actually built, which may cause failures down the line. Make sure to implement a proper build() method.

model.load_weights(
    'model.weights.h5',
)

ValueError: Layer 'dense_167' was never built and thus it doesn't have any variables. However the weights file lists 2 variables for this layer.
In most cases, this error indicates that either:

The layer is owned by a parent layer that implements a build() method, but calling the parent's build() method did NOT create the state of the child layer 'dense_167'. A build() method must create ALL state for the layer, including the state of any children layers.

You need to implement the def build_from_config(self, config) method on layer 'dense_167', to specify how to rebuild it during loading. In this case, you might also want to implement the method that generates the build config at saving time, def get_build_config(self). The method build_from_config() is meant to create the state of the layer (i.e. its variables) upon deserialization.

Comments

The same model works just fine in tf.keras 2.12, saving and reloading, no complain.
Skipping some layer computation in SubModel might cause this issue.
If such approach is valid for tf.keras 2.12, this should also valid for keras v3. Was it bug or is it now?
The above error log messages give some intuition ( override the build method, etc.), but this is expensive to comply on actual model.
Translating is not going smooth!

Keras 3 is more strict than Keras 2 about state creation. In all cases the solution is to implement a proper build() method on your layers/models.
If you can't be bothered writing a symbolic one, just use build() to create dummy data and then call the model eagerly.

Thanks for the response.  Actually, writing a new model with new API is okay. But adding new components to old models can be expensive, especially if the amount is big. Keras 3 becomes more strict for state creating, I think that's because of supporting all backend. That's okay, I will revisit this issue soon, report you back. Thank you for all the support.

I think that's because of supporting all backend

It's for JAX mainly. TF can create new state during function tracing, but in JAX all state must be created in advance of any tracing. So Keras needs to be able to create all model state in advance.

Thanks @fchollet
I rewrite the above sample model to keep align with new API. The model build well and weights.h5 saving and reloading also work. But .keras reloading gives above no-yet-build error.
class SubModel(keras.Model):
    def __init__(self, num_layer_norm, **kwargs):
        super().__init__(**kwargs)
        self.num_layer_norm = num_layer_norm

    def build(self, input_shape):
        self.dense_layers = [
            layers.Dense(64, activation='relu') for i in range(self.num_layer_norm)
        ]
        self.layer_norms = [
            layers.LayerNormalization() for _ in range(self.num_layer_norm)
        ]
        self.output_layer = layers.Dense(32, activation='relu')

    def call(self, inputs):
        x = inputs
        for i, (dense, layer_norm) in enumerate(
            zip(self.dense_layers, self.layer_norms)
        ):
            x = dense(x)
            x = layer_norm(x)
        output = self.output_layer(x)
        return output

    def get_config(self):
        config = super().get_config()
        config.update(
            {
                "num_layer_norm": self.num_layer_norm,
            }
        )
        return config

@keras.saving.register_keras_serializable('combined.model')
class CombinedModel(keras.Model):
    def __init__(self, num_submodels, num_layer_norm, **kwargs):
        super().__init__(**kwargs)
        self.num_submodels = num_submodels
        self.num_layer_norm = num_layer_norm

    def build(self, input_shape):
        self.submodels = [
            SubModel(self.num_layer_norm) for _ in range(self.num_submodels)
        ]
        self.combine_layer = layers.Dense(
            10, activation='softmax', name='combined_dense', dtype='float32'
        )

    def call(self, inputs):
        submodel_outputs = [submodel(inputs) for submodel in self.submodels]
        combined_output = ops.concatenate(submodel_outputs, axis=-1)
        final_output = self.combine_layer(combined_output)
        return final_output

    def get_config(self):
        config = super().get_config()
        config.update(
            {
                "num_submodels": self.num_submodels,
                "num_layer_norm": self.num_layer_norm
            }
        )
        return config

Test
num_submodels = 5
num_layer_norm = 10
model = CombinedModel(num_submodels, num_layer_norm)
model(tf.ones(shape=(2, 100)))
model.summary() # OK

State saving and reloading works.
# OK
model.save_weights('custom.weights.h5')
new_model = CombinedModel(num_submodels, num_layer_norm)
new_model(tf.ones(shape=(2, 100)))
new_model.load_weights("custom.weights.h5")

But, saving in .keras format, it breaks on self.combine_layer (built-in dense layer) of CombinedModel model above.
# NOT OK
model.save('custom.keras')
check = keras.models.load_model(
   'custom.keras'
)

ValueError: Layer 'combined_dense' was never built and thus it doesn't have any variables. However the weights file lists 2 variables for this layer.
In most cases, this error indicates that either:

The layer is owned by a parent layer that implements a build() method, but calling the parent's build() method did NOT create the state of the child layer 'combined_dense'. A build() method must create ALL state for the layer, including the state of any children layers.
You need to implement the def build_from_config(self, config) method on layer 'combined_dense', to specify how to rebuild it during loading. In this case, you might also want to implement the method that generates the build config at saving time, def get_build_config(self). The method build_from_config() is meant to create the state of the layer (i.e. its variables) upon deserialization.


GOT RATTED DONT USE DEV9998 IS KNOWN FOR DOING THIS STUFF!!!!!!!!!!!!!!!

DEV YOU WILL BE STOPPED

nuh uh


Suppress console errors on cancelled fetch

Fixes #1595

coverage: 94.179% (-0.02%) from 94.197% when pulling 50baaa4899a86e804f891ffae6d6260247b93295 on Paul-Taiwo:paul-suppress-console-errors into 3df509229aaf00da1ff55e8100e7a2e8aeffa89c on medplum:main.


Generator expressions with filtering in v0.5 break highlighting

e.g.
[e for e in x if e > 0]

Same as #76, already fixed on latest version.


Rare PathTimeout errors in provider.realize(...)

#20 did indeed fix the ~dozens of returning a symbolic type errors that I was getting before, but in a few rare cases it looks like that's uncovered a new failure mode: this CI job has three tests failing with
  File ".../hypothesis_crosshair_provider/crosshair_provider.py", line 293, in realize
    return self.export_value(value)
  File ".../hypothesis_crosshair_provider/crosshair_provider.py", line 280, in export_value
    return deep_realize(value)
  File ".../crosshair/core.py", line 258, in deep_realize
    return deepcopyext(value, CopyMode.REALIZE, {})
  File ".../crosshair/copyext.py", line 47, in deepcopyext
    obj = obj.__ch_realize__()  # type: ignore
  File ".../crosshair/libimpl/builtinslib.py", line 3819, in __ch_realize__
    return bytes(tracing_iter(self.inner))
  File ".../crosshair/tracers.py", line 463, in tracing_iter
    value = next(itr)
  File ".../crosshair/libimpl/builtinslib.py", line 2537, in __iter__
    if not space.smt_fork(idx < my_smt_len):
  File ".../crosshair/statespace.py", line 1043, in smt_fork
    return self.choose_possible(expr, probability_true)
  File ".../crosshair/statespace.py", line 854, in choose_possible
    self.check_timeout()
  File ".../crosshair/statespace.py", line 841, in check_timeout
    raise PathTimeout
crosshair.util.PathTimeout

I'm not really sure what we should do here.  Maybe we should have a known exception to indicate that the current test case can't be realized so that Hypothesis can skip it as if for assume(False)?  (raised/caught here)  Increasing the timeout will only make the problem less common rather than fixing it, but at the cost of rather slow tests.  Maybe that - or removing it entirely - is worth it though?

So,  it would be really great to have a designated exception or some other way to indicate that I need to abort the current iteration. And, as you're seeing here, that can happen not only when the test case runs, but also everywhere we realize values. How difficult would this be to support?  I'm already doing a few horrible things in the plugin to work around this that have their own edge-case failure scenarios.
In the specific case of this timeout, I suspect we want to retain it, though perhaps ideally it'd be 2x the current test's deadline or something like that.

So, it would be really great to have a designated exception or some other way to indicate that I need to abort the current iteration. And, as you're seeing here, that can happen not only when the test case runs, but also everywhere we realize values. How difficult would this be to support?

Perfectly reasonable, it'll just be a little fiddly to implement - but easy enough to test using a provider that fails on the 1,2,3,4,5,...th interaction to check that they're all handled properly.

In the specific case of this timeout, I suspect we want to retain it, though perhaps ideally it'd be 2x the current test's deadline or something like that.

Makes sense!  With timeout=None defaulting to ten seconds or something.

I think fixed by https://github.com/pschanely/hypothesis-crosshair/commit/dd7d1aeb8714fa7cd34e14fd71b495464a2c0473 (was #23), will reopen if I see it again 🙂


Change getAttachedTargetGroupPolicy to a generic method getAttachedPolicy()

Changes:

Change getAttachedTargetGroupPolicy to a generic method getAttachedPolicy()

What type of PR is this?

Which issue does this PR fix:
What does this PR do / Why do we need it:
If an issue # is not available please add repro steps and logs from aws-gateway-controller showing the issue:
Testing done on this change:

Added extra unit tests
e2etest WIP

Automation added to e2e:

Will this PR introduce any new dependencies?:

Will this break upgrades or downgrades. Has updating a running cluster been tested?:
Does this PR introduce any user-facing change?:

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

41 of 48 (85.42%)  changed or added relevant lines in 2 files are covered.
2 unchanged lines in 1 file lost coverage.
Overall coverage increased (+0.2%) to 38.9%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

pkg/gateway/utils.go
34
41
82.93%

Files with Coverage Reduction
New Missed Lines
%

pkg/gateway/model_build_targetgroup.go
2
80.0%

Totals

Change from base Build<PHONE_NUMBER>:
0.2%

Covered Lines:
3997

Relevant Lines:
10275

💛  - Coveralls


Deactivate Shadow Page dont show content/seo/excerpt tab

When you activate and then deactivate Shadow Page the other tabs will not shown again.

@alexander-schranz could you please check this issue? after save they should be there in the current version!

looks fine


Using prepared fails sometimes with "Acquiring waiter that is already in use "

Sometimes when using a lot of prepared statements it will fail with "Acquiring waiter that is already in use" from down in "libasync", which causes the application to crash with an "AssertError".
I'm using vibe.d and MySQLPool and to retrieve a connection I use "lockConnection()" which I pass to "prepare":
Example:
auto connection = pool.lockConnection();
auto prepared = prepare(connection, sql);

This should be fixed in v2.0.0 as long as you don't use prepareBackwardCompat or BackwardCompatPrepared. Please re-open if you still experience this problem with v2.x.

Apparently still experienced, so reopening:
https://forum.dlang.org/thread/p7djc3$21hp$1@digitalmars.com?page=2#post-wzpqtxvtrgqqqrpfoukh:40forum.dlang.org

I've just tagged a new 'v2.2.1', please try that and also (as discussed on the NG) check that your code isn't accidentally using a Connection or a ResultRange after the original LockedConnection has gone out of scope. Let me know whether that tasks care of it..

As far as I can see this is no longer an issue, at least not for me. I'll close this for now.


Add EOL Status pie charts

Server (Operating System and Hardware)
** Application Page
** Rollups (org unit etc)

Database
** Application Page

just adding screenshots (for use in the release notes)


Web app for project

Hey! Wouldn't it be a good extension to the project to create a basic web application that based on the user input in the front end generates the ER model images in the back end and then displays it to the user? Maybe something similar to what already exists for markdown processing (https://dillinger.io/ for example), but using the dbdia syntax to create them.
Thanks!

Yes, good idea, I also have that in mind. It'll have to wait though ...


Android: Update project setup

Versions of various build tools are rather outdated, there are newer setups being used now.
The things to update are: android build plugin, gradle wrapper, build repositories.

Just make sure that android-release-aar.gradle will still work, because it's important for deployment to jcenter. The scripts/deploy-build.sh is internally calling gradlew clean build generateRelease to prepare a ZIP file with build. The ZIP must be prepared to specific path.


Recursive Voronoi

This is not an issue but I wanted to share a nice picture made with shapely (+ scipy and matplotlib) and did not find where to post. Code is available from https://github.com/rougier/recursive-voronoi (it is part of an upcoming book on matplotlib).

Thanks for sharing, fun to see :+1:

It's beautiful, @rougier !


WIP: POC of using Modal in two separate ways

Fixes JIRA ticket DSD-703
This PR does the following:

Is an exploration for the Modal component and how devs might/should use it.

How has this been tested?
This is just a POC so we'll close this eventually once we agree on how to move forward. So, ignore the failing GA tests.
Accessibility concerns or updates

Checklist:

[ ] I have updated the Storybook documentation accordingly.
[ ] I have added relevant accessibility documentation for this pull request.
[ ] All new and existing tests passed.

Front End Review:

[ ] View the example in Storybook

Tugboat has finished building the preview for this pull request!
Link:

https://pr839-lll4qwnkt7xviitqcfqpnq7735kciohu.tugboat.qa

Dashboard:

https://dashboard.tugboat.qa/61dc5a737499db652205bde9

Tugboat has finished building the preview for this pull request!
Link:

https://pr839-lll4qwnkt7xviitqcfqpnq7735kciohu.tugboat.qa

Dashboard:

https://dashboard.tugboat.qa/61dc5a737499db652205bde9

Tugboat has finished building the preview for this pull request!
Link:

https://pr839-lll4qwnkt7xviitqcfqpnq7735kciohu.tugboat.qa

Dashboard:

https://dashboard.tugboat.qa/61dc5a737499db652205bde9

Closing but using code for final DSD-412 ticket.

@wluisi @bigfishdesign13  let's talk about this soon and see if it'll work in the upcoming work Bill is doing.

Tugboat has finished building the preview for this pull request!
Link:

https://pr839-gt9wbfjexcxmm0ivt9qtpeb37soj5xfr.tugboat.qa

Dashboard:

https://dashboard.tugboat.qa/626190762e91387b17150894

From meeting:

Expose the onClose function in the useModal hook
allow jsx for the body content
allow "full" size style only for mobile

This is ready to be tested now. I added an example and you can create your custom open and close buttons. I'll fix the tugboat issue later.

Tugboat has finished building the preview for this pull request!
Link:

https://pr839-igsfimtu3yrifbrtmguioescildxk2ye.tugboat.qa

Dashboard:

https://dashboard.tugboat.qa/626304fb2e91387b1726e343

Updated.

Tugboat has finished building the preview for this pull request!
Link:

https://pr839-igsfimtu3yrifbrtmguioescildxk2ye.tugboat.qa

Dashboard:

https://dashboard.tugboat.qa/626304fb2e91387b1726e343


[DOCUMENTATION] Mechanics Design Documents (Hulls, Weapons, Station)

What needs to be changed or added.
A clear and concise description of what is required. Ex. We need information on mechanic [...]

This should have been closed last semester, refer to the entire machinery section of the wiki for the documentation in question.


Add missing changelog entries

Issue #, if available:
Description of changes:
Added missing changelog entries (copied from github releases).
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Codecov Report

Merging #91 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master      #91   +/-   ##
=======================================
  Coverage   97.72%   97.72%
=======================================
  Files          19       19
  Lines         659      659
  Branches       48       48
=======================================
  Hits          644      644
  Misses          9        9
  Partials        6        6

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 01b42ff...8aec6fe. Read the comment docs.


The variable yellow is not converted when using the minified output style

I have this scss-code:
$colors: (purple: #7B81CB, yellow: #FEC051);

@each $color-name in map-keys($colors) {
  $color-text: map-get($colors, $color-name);

  .color.#{$color-name} {
    color: $color-text;
  }
}

When I run node-sass app.scss app.css, I get this (as expected):
.color.purple {
  color: #7B81CB; }

.color.yellow {
  color: #FEC051; }

But when I run node-sass --output-style compressed app.scss app.css, I get this (not expected):
.color.purple{color:#7B81CB}

This is what I am expecting:
.color.purple{color:#7B81CB}.color.yellow{color:#FEC051}

I solved this problem in my app by changing yellow to yellows.

I also tested it in sassc and there it works, this is the current node-sass version:
$ node-sass --version
node-sass	3.4.2	(Wrapper)	[JavaScript]
libsass  	3.3.2	(Sass Compiler)	[C/C++]

OS: Mac OS X 10.11.2

This is fixed in the upcoming beta. Subscribe #1349 for updates on that release.


Broken due to major update - <IP_ADDRESS>

This no longer works since ASF released <IP_ADDRESS>

I'll release a new version of the plugin later today.  In the meantime, there's a pre-release version available which may support ASF <IP_ADDRESS>:
https://github.com/Citrinate/FreePackages/releases/tag/1.3.1

This no longer works since ASF released <IP_ADDRESS>

Duplicate of:

#15

A proper release with V5.5 support is now posted

Thank you :D


Implement Enumerable Module

Enumerable module in Ruby originally was included in Array class and it has pre-defined methods such as the #map, #select and #reduce. In original Ruby, it requires also to define the #each method when included Enumerable module.

Hint: * means if no block given, then returns the Enumerator itself
Method list implementation:

[ ] Enumerable#all? Returns true if the block never returns false or nil
[ ] Enumerable#any? Returns true if the block ever returns a value other than false or nil
[ ] *Enumerable#chunk Enumerates over items and chunking them together based on the return value of block
[ ] Enumerable#chunk_while Creates a enumerator for each chunked elements
[ ] *Enumerable#map and *Enumerable#collect
[ ] *Enumerable#flat_map and *Enumerable#collect_concat
[ ] Enumerable#count

Returns the number of items in Enumerator
If argument is given, count the number of argument value in enumerator
If block is given, counts the number of elements yielding true value

[ ] *Enumerable#cycle Default runs forever, if specified number n then runs enumerator n times repeated
[ ] *Enumerable#detect Enumerator#find
[ ] Enumerable#drop Drops first n elements from enumerator and return rest elements in an array
[ ] *Enumerable#drop_while Drops elements up to, but not including, the first element for which the block returns nil or false and returns an array containing the remaining elements
[ ] *Enumerable#each_cons and Enumerable#each_slice Iterates the given block for each array of consecutive n elements
[ ] *Enumerable#each_entry Converting multiple values from yield to an array
[ ] *Enumerable#each_with_index
[ ] *Enumerable#each_with_object
[ ] Enumerable#entries and Enumerable#to_a Returns array containing items in enumerator
[ ] Enumerable#empty? Returns true if no elements
[ ] *Enumerable#find_all and *Enumerable#select
[ ] *Enumerable#find_index

Returns the index for the first for which the evaluated value is not false
If pass in argument, return the index of the value which is equal to the argument

[ ] Enumerable#first Returns first n element (default n is 1)
[ ] Enumerable#grep

Range input as an argument will enumerates self and returns map of values which is included in range
RegExp input as an argument will returns map of values which matches the regex

[ ] Enumerable#grep_v Inverted version of #grep
[ ] *Enumerable#group_by Groups collection by the result of the block which returns a hash
[ ] Enumerable#include? and Enumerable#member?
[ ] Enumerable#inject and Enumerable#reduce
[ ] Enumerable#lazy (Lazy Enumerator need more survey)
[ ] Enumerable#max Returns the object of enum with n maximun value (default n is 1)
[ ] *Enumerable#max_by
[ ] Enumerable#min Inversed version of #max
[ ] *Enumerable#min_by Inversed version of #max_by
[ ] Enumerable#minmax (???)
[ ] *Enumerable#minmax_by (???)
[ ] Enumerable#none? Returns true if none of the block value results true ([nil] is none but not empty!)
[ ] Enumerable#one? Returns true if block result returns true exactly once
[ ] Enumerable#partition Returns two array elements in an array which the first array stores the value that the block evaluates true while the second array stores value that the block evaluates false
[ ] *Enumerable#reject Returns array of all elements for which block result false
[ ] *Enumerable#reverse_each
[ ] Enumerable#slice_after
[ ] Enumerable#slice_before
[ ] Enumerable#slice_when
[ ] Enumerable#sort
[ ] *Enumerable#sort_by
[ ] Enumerable#take Returns first n elements (different from #first, #take needs to specifiy argument as an integer)
[ ] *Enumerable#take_while
[ ] Enumerable#to_a
[ ] Enumerable#to_h
[ ] Enumerable#unique Returns a new array by removing duplicated value in self
[ ] Enumerable#zip

@Maxwell-Alexius Are you still interested in working on this?

I'm not sure I can handle this ton of works and I haven't been using Ruby for a long time since working on JS related things, it might take me time to pick up and inspect the Enumerable mechanism in Ruby again.
I would prefer others can pick up this issue and build the main Enumerator class.
If the main class is built, I think I can assist in implementing the Enumerable#method part

Seeing Maxwell's recent comment, I've been surveying around Enumerator in Ruby and Goby, and I built a very primitive Enumerator class as a mock on my local rep that can be compiled so far. I think it is possible to build.

But now I'm wondering how to proceed farther. In other words, I think we need a grand design around organizing Enumerator and Enumerable (and perhaps Lazy and Block) classes in Goby.

I wonder if it is certainly good for us to inherit the architecture or design of Ruby's Enumerator and Enumerable. Sharing methods for iteration via including Enumerable module and implementing each is good for consistency, but I also guess that Enumerator class might be just for instantiating Enumerator as Block class does like that.
I suppose the current LazyEnumerator class that is written in Goby is just a temporal and they should be finally integrated into the upcoming Enumerator.
I recognize that st0012 does not want to make the inheritance path more fat.

This is just a draft memorandum and is not final. If needed, I create a new issue for that.
I'd be glad if @st0012 would show ideas around this.
Thank you,

@hachi8833 sorry for the late response
I don't use either Enumerator or Enumerable directly very often, so I have limited understanding about those 2. I think we can start by

creating the Enumerable module
moving most of the Array's methods into Enumerable and make Array include it
Implement Enumerator class

And from the screenshot, you uploaded your implementation looks cool! Can you open a PR for it?

@st1102 thank you for the checking!
I just wanted a grand design from you around Enumerable😊
I'd send the WIP PR

@Maxwell-Alexius I hope you could check https://github.com/goby-lang/goby/pull/801


Error (CSV_INVALID_CLOSING_QUOTE): Invalid Closing Quote: got "A" at line 231 instead of delimiter, record delimiter, trimable character (if activated) or comment

Summary
I get error when running
sf data query -b -f fields.soql -w 30 -r csv > fields.csv
Error (CSV_INVALID_CLOSING_QUOTE): Invalid Closing Quote: got "A" at line 231 instead of delimiter, record delimiter, trimable character (if activated) or comment

Steps To Reproduce
Unsure, there are 1M+ rows I am trying to get
Why would cli try to parse CSV in first place when API already provides CSV file? I suppose results come in multiple files and one way of merging them is parsing...
Expected result

Actual result

Additional information

System Information
Y{
  "architecture": "darwin-arm64",
  "cliVersion"<EMAIL_ADDRESS>  "nodeVersion": "node-v20.13.1",
  "osVersion": "Darwin 24.1.0",
  "rootPath": "/Users/ihazgrog/.nvm/versions/node/v20.13.1/lib/node_modules/@salesforce/cli",
  "shell": "zsh",
  "pluginVersions": [
    "@oclif/plugin-autocomplete 3.2.7 (core)",
    "@oclif/plugin-commands 4.1.5 (core)",
    "@oclif/plugin-help 6.2.16 (core)",
    "@oclif/plugin-not-found 3.2.24 (core)",
    "@oclif/plugin-plugins 5.4.15 (core)",
    "@oclif/plugin-search 1.2.13 (core)",
    "@oclif/plugin-update 4.6.8 (core)",
    "@oclif/plugin-version 2.2.15 (core)",
    "@oclif/plugin-warn-if-update-available 3.1.20 (core)",
    "@oclif/plugin-which 3.2.16 (core)",
    "@salesforce/cli 2.65.8 (core)",
    "apex 3.5.5 (core)",
    "api 1.3.1 (core)",
    "auth 3.6.70 (core)",
    "custom-metadata 3.3.6 (user) published 189 days ago (Sun May 05 2024) (latest is 3.3.37)",
    "data 3.9.0 (core)",
    "deploy-retrieve 3.15.4 (core)",
    "devops-center 1.2.9 (user) published 241 days ago (Fri Mar 15 2024) (latest is 1.2.26)",
    "functions 1.23.0 (user) published 250 days ago (Wed Mar 06 2024)",
    "info 3.4.15 (core)",
    "lightning-dev 1.3.0 (user) published 51 days ago (Sat Sep 21 2024) (latest is 1.9.3)",
    "limits 3.3.37 (core)",
    "marketplace 1.3.2 (core)",
    "org 5.0.2 (core)",
    "packaging 1.27.1 (user) published 375 days ago (Wed Nov 01 2023) (latest is 2.9.0)",
    "schema 3.3.39 (core)",
    "settings 2.4.2 (core)",
    "signups 2.3.0 (user) published 188 days ago (Tue May 07 2024) (latest is 2.6.1)",
    "sobject 1.4.44 (core)",
    "telemetry 3.6.18 (core)",
    "templates 56.3.26 (core)",
    "trust 3.7.38 (core)",
    "user 3.6.0 (core)",
    "@salesforce/sfdx-scanner 4.1.0 (user) published 194 days ago (Wed May 01 2024) (latest is 4.7.0)"
  ]
}

I'm pretty this is caused by this is caused by the qty of records (sf can run out of memory while downloading the CSV chunks from the API and parse them).
Can you try with the data export bulk command? it can handle million of records and output them to a file in csv or json:
https://developer.salesforce.com/docs/atlas.en-us.sfdx_cli_reference.meta/sfdx_cli_reference/cli_reference_data_commands_unified.htm#cli_reference_data_export_bulk_unified

Thanks @cristiand391, that sort of makes sense. With sfdx I used to use bulk export and that worked pretty well.
I'm still somewhat new to sf and since it has --bulk api flag I assumed the commands were merged.
Would be nice if command would work or at least suggest using different command when too much data is detected (i.e. Illuminated Cloud IDE always warns me when there's tons of rows about to be returned so it wouldn't crash itself).
Regardless I've played with an LLM and managed to create independent bash script that downloaded data that  I needed...

Would be nice if command would work or at least suggest using different command when too much data is detected (i.e. Illuminated Cloud IDE always warns me when there's tons of rows about to be returned so it wouldn't crash itself).

Yes, we actually plan to deprecate the bulk functionality from data query so there'll be just one command for bulk queries to avoid confusion.
I'll close this issue, please let us know if you have any issue with data export bulk.


Integration with Frontmatter Title Plugin

Hello! Could you integrate this plugin with https://github.com/snezhig/obsidian-front-matter-title so that we can see the title property rather than the filename?
This plugin provides an API : https://github.com/snezhig/front-matter-plugin-api-provider

Good idea! I'm afraid that integrations with other plugins, however, might have to be postponed for a little while. I'll look into it after fixing all known issues and getting the core functionality done. Please be patient.

Good idea! I'm afraid that integrations with other plugins, however, might have to be postponed for a little while. I'll look into it after fixing all known issues and getting the core functionality done. Please be patient.

Of course! No worry, take your time ; )

This feature request started on the wrong foot. The issue is not integrating Vertical Tabs with another plugin; it's about displaying a note's frontmatter title. This functionality relies on standard YAML frontmatter, a core feature of Obsidian, and doesn't require a third-party plugin to access. This feature would greatly enhance Vertical Tabs for users who decoupled or 'internalized' titles" from filenames. Title should be: "option to display front matter title instead of file name"

Hi @attilaszkrs ! May I ask which solution do you use or prefer? With or without the frontmatter title plugin?
The advantage of Frontmatter Title is that it comes with more features such as replacing titles in the file browser. Otherwise Vertical Tabs has to reimplement some features.

For me both solutions work, since the result is "show title instead of filename".
Of course, I understand that some users don't want to rely on additional plugins -
First, however, as @oxdc said, the Frontmatter title plugin implements some features in its own right, and integrating with it avoids the need to reinvent the wheel.
Examples include customizable title frontmatter property name, fallback frontmatter name, and more complex function handling:

For example, relying on the Function V2 Processor, I can display the name of "Today's DailyNote" as ⬇ Today:

This is easier to identify. And because it's dynamic, it can not be achieved by a fixed title.

Secondly, I personally think that the need to "display title" should be unified.
If a person wants to display the title in vertical tabs, then it is likely that he will also want to display the title in other places - tabs, title bars (etc.), so this is to "display title" as a special problem, with a separate solution to unify - Vertical tabs are only compatible with this solution.

Still, both solutions are acceptable for me! It depends on what @oxdc thinks 😊


Update README.md by removing DML information from CI status table

PTAL, thanks! @BruceDai @fujunwei

Why do not you add the CI for the dml backend?

Close it, will add the CI files.

Reopen this PR to delete CI information for DML backend. @fujunwei @BruceDai

Merge it, thanks.


python3.10 -> python3.11

Flake.lock needs another bump, but (pending open PR to fix pynetdicom tests) everything builds with python3.11 except debugpy, and, annoyingly, catboost.
This PR might seem superfluous and/or premature but python 3.11 being the default python3 will soon hit master in time for NixOS 23.11, so we need to update or face the usual issue of Python CLI tools breaking the devShells (or hold back Nixpkgs until we make this update, clearly undesirable).

do you know what's up with catboost? We can start migrating projects to 3.11, TBI should be no problem. ICH needs catboost, but we can either try to fix locally+upstream it or wait for it to get solved upstream.

Nixpkgs has an old version of catboost which doesn't build under 3.11 since (1) catboost used to have a somewhat obscure build using ymake that needed a bunch of patching and no-one was maintaining it; (2) they now switched to cmake but sadly use conan for dependencies, which is not compatible with Nix.

I guess we can merge the final version of this PR into main and then revert it on the ICH branch.

So new catboost is hopeless because of conan, or is there a painful workaround? And we won't even need to revert ICH, I just won't pull

They don't use a huge number of dependencies so it might theoretically be possible for someone proficient in CMake to patch them out.

Just waiting for PR 257792 to land upstream.

Just waiting for PR 257792 to land upstream.

It's on master now, so will start a build shortly.

@cfhammill there are a few packages broken (nilearn, snakemake, plotnine) currently but all the TBI stuff is good to go.

Now waiting on nixpkgs/263799

Ready to go, tbi tests pass.

I'm ok with either targeting main or tbi-deps, I can selectively not pull into ICH until I get a chance to validate there.

I'm ok with either targeting main or tbi-deps, I can selectively not pull into ICH until I get a chance to validate there.

maybe tbi-deps makes more sense  / maybe there isn't much point in having a main ?
Also remember, it won't work with ICH until someone fixes catboost with python 3.11.


The audio is very sizzle

It's very interesting and I don't know why is happening(sizzle/cracking) when streaming audio... For example riftcat working perfectly... so theorically the problem isn't my hardware.

Some here. I want to know the solution too.


I have a problem with the accounts five minutes after they enter the room they go out what is the solution

I have a problem with the accounts five minutes after they enter the room they go out what is the solution

Hi,
Please describe your issue

I have a problem every time I enter the accounts the room by my tokens through your script after running the script the accounts come out after five minutes only

Is there a way to determine how long the accounts stay in the room by your code

the name of script is : join-room

Join a room with a given room id and user auth token.
Author: Ehsan Ghaffar
This file is part of the spacepy.
import requests
import json
api_url = "https://www.clubhouseapi.com/api/"
token = "YOUR_TOKEN_HERE"
channel_id = ""
def join_room(t, url, ch):
headers = {
"CH-Languages": "en-US",
"CH-Locale": "en_US",
"Accept": "application/json",
"Accept-Encoding": "gzip, deflate",
"CH-AppBuild": "2576",
"CH-AppVersion": "1.0.0",
"CH-UserID": "",
"User-Agent": "clubhouse/490 (iPhone; iOS 14.4; Scale/2.00)",
"Connection": "alive",
"Content-Type": "application/json; charset=utf-8",
"Authorization": "Token " + token
}
end_point = url + 'join_channel'
payload = {
    "channel": ch
}

response = requests.request(
    "POST", end_point, json=payload, headers=headers)
room_res = response.json()
dumped_res = json.dumps(room_res, indent=2)
print(room_res)

call join rooom function
join_room(token, api_url, channel_id)

"???

"???

Soon ...

What do you mean by soon??


Fix API document

Fix some problems in API document.

Document URLs (private preview)

C++ API Reference http://docs.qulacs.org/en/fix_doc/api/cpp_library_root.html
Python API Reference http://docs.qulacs.org/en/fix_doc/pyRef/modules.html

Update Sphinx-related libraries

Fix a problem that some classes (e.g. QuantumCircuit) did not appear on the reference

Use header files from the submodule qulacs/ instead of the copied version qulacs-src/

Note that updates made in #13 are not reflected, so this version is not ready to be made public yet

Include vqcsim as well as cppsim

This looks good to me!
Just as a note, this repo has a
qulacs-src/cppsim dir because we translated the cpp comments at the time, but don't want to create
difficult to resolve conflicts with the dev branch of the upstream qulacs codes
It would be nice if we can put the translations into the source code and delete qulacs-src/cppsim in the future


feat: update args hash to be a flat poseidon [WIP]

We used before a sort of tree structure for args hash to make an incremental hash. Poseidon is incremental in noir, so we can get rid of it and just use a flat poseidon hash.

Benchmark results
Metrics with a significant change:

avm_simulation_time_ms (FeeJuice:_increase_public_balance): 42.8 (-68%)
avm_simulation_time_ms (FPC:constructor): 48.5 (-34%)
avm_simulation_time_ms (Token:mint_public): 242 (+461%)
avm_simulation_time_ms (Token:assert_minter_and_mint): 48.2 (-81%)
avm_simulation_time_ms (FPC:prepare_fee): 57.8 (-33%)
avm_simulation_time_ms (Token:_increase_public_balance): 49.6 (+30%)
avm_simulation_bytecode_size_in_bytes (Token:constructor): 7,451 (-30%)
avm_simulation_bytecode_size_in_bytes (FPC:constructor): 4,853 (-39%)
avm_simulation_bytecode_size_in_bytes (Token:transfer_public): 5,562 (-36%)

Detailed results
All benchmarks are run on txs on the Benchmarking contract on the repository. Each tx consists of a batch call  to create_note and increment_balance, which guarantees that each tx has a private call, a nested private call, a public call, and a nested public call, as well as an emitted private note, an unencrypted log, and public storage read and write.
This benchmark source data is available in JSON format on S3 here.
Proof generation
Each column represents the number of threads used in proof generation.

Metric
1 threads
4 threads
16 threads
32 threads
64 threads

proof_construction_time_sha256_ms
5,767
1,541
701
749 (-1%)
773

proof_construction_time_sha256_30_ms
11,519
3,067
1,361
1,421
1,461 (+1%)

proof_construction_time_sha256_100_ms
45,395
11,919 (+1%)
5,377
5,407
5,363

proof_construction_time_poseidon_hash_ms
79.0 (+1%)
34.0
34.0
58.0 (-2%)
87.0

proof_construction_time_poseidon_hash_30_ms
1,537
416
199
225
267 (-1%)

proof_construction_time_poseidon_hash_100_ms
5,674
1,505
669
726 (-1%)
749 (+1%)

L2 block published to L1
Each column represents the number of txs on an L2 block published to L1.

Metric
4 txs
8 txs
16 txs

l1_rollup_calldata_size_in_bytes
4,612
8,260
15,524

l1_rollup_calldata_gas
52,816
97,584
186,812

l1_rollup_execution_gas
821,812
1,557,988
3,348,651

l2_block_processing_time_in_ms
251 (-2%)
442 (+2%)
793 (+2%)

l2_block_building_time_in_ms
9,173
18,036
36,145

l2_block_rollup_simulation_time_in_ms
9,173
18,036
36,145

l2_block_public_tx_process_time_in_ms
7,806
16,636
34,702

L2 chain processing
Each column represents the number of blocks on the L2 chain where each block has 8 txs.

Metric
3 blocks
5 blocks

node_history_sync_time_in_ms
3,034
3,966 (+2%)

node_database_size_in_bytes
14,225,648
18,223,344

pxe_database_size_in_bytes
16,258
26,818

Circuits stats
Stats on running time and I/O sizes collected for every kernel circuit run across all benchmarks.

Circuit
simulation_time_in_ms
witness_generation_time_in_ms
input_size_in_bytes
output_size_in_bytes
proving_time_in_ms

private-kernel-init
69.5
377 (+1%)
20,971
44,933
N/A

private-kernel-inner
138
685
71,782
45,067
N/A

private-kernel-reset-tiny
318 (-1%)
688
64,268
44,922
N/A

private-kernel-tail
149 (-1%)
129
50,548
57,178
N/A

base-parity
6.03
N/A
160
96.0
N/A

root-parity
35.5
N/A
73,964
96.0
N/A

base-rollup
2,084 (-1%)
N/A
189,128
664
N/A

block-root-rollup
42.1
N/A
58,217
2,448
N/A

public-kernel-setup
83.9 (-2%)
N/A
114,433
79,670
N/A

public-kernel-app-logic
98.0
N/A
114,251
79,670
N/A

public-kernel-tail
566
N/A
487,098
16,414
N/A

private-kernel-reset-small
289
N/A
66,345
45,629
N/A

private-kernel-tail-to-public
788
599 (-2%)
447,469
1,825
N/A

public-kernel-teardown
83.2 (-1%)
N/A
114,697
79,670
N/A

merge-rollup
20.3 (-1%)
N/A
38,182
664
N/A

undefined
N/A
N/A
N/A
N/A
104,865

Stats on running time collected for app circuits

Function
input_size_in_bytes
output_size_in_bytes
witness_generation_time_in_ms

ContractClassRegisterer:register
1,344
11,731
342

ContractInstanceDeployer:deploy
1,408
11,731
18.2 (-4%)

MultiCallEntrypoint:entrypoint
1,920
11,731
421

FeeJuice:deploy
1,376
11,731
383 (-2%)

SchnorrAccount:constructor
1,312
11,731
62.7 (-2%)

SchnorrAccount:entrypoint
2,336
11,731
370

FeeJuice:claim
1,344
11,731
33.8 (-14%)

Token:privately_mint_private_note
1,280
11,731
70.9 (-6%)

FPC:fee_entrypoint_public
1,344
11,731
20.5 (-26%)

Token:transfer
1,312
11,731
205 (-5%)

Benchmarking:create_note
1,344
11,731
76.3 (-1%)

SchnorrAccount:verify_private_authwit
1,280
11,731
24.9 (-8%)

Token:unshield
1,376
11,731
489 (-9%)

FPC:fee_entrypoint_private
1,376
11,731
648 (-9%)

AVM Simulation
Time to simulate various public functions in the AVM.

Function
time_ms
bytecode_size_in_bytes

FeeJuice:_increase_public_balance
:warning: 42.8 (-68%)
1,197

FeeJuice:set_portal
10.6 (+2%)
1,179

Token:constructor
256 (-9%)
:warning: 7,451 (-30%)

FPC:constructor
:warning: 48.5 (-34%)
:warning: 4,853 (-39%)

FeeJuice:check_balance
37.2 (+1%)
1,004

Token:mint_public
:warning: 242 (+461%)
2,122

Token:assert_minter_and_mint
:warning: 48.2 (-81%)
1,434

AuthRegistry:set_authorized
45.7 (+1%)
501

FPC:prepare_fee
:warning: 57.8 (-33%)
1,379

Token:transfer_public
27.5 (-5%)
:warning: 5,562 (-36%)

FPC:pay_refund
39.9 (-9%)
1,915

Benchmarking:increment_balance
980
1,571

Token:_increase_public_balance
:warning: 49.6 (+30%)
1,168

FPC:pay_refund_with_shielded_rebate
51.9
1,919

Public DB Access
Time to access various public DBs.

Function
time_ms

get-nullifier-index
0.163 (-1%)

Tree insertion stats
The duration to insert a fixed batch of leaves into each tree type.

Metric
1 leaves
16 leaves
64 leaves
128 leaves
256 leaves
512 leaves
1024 leaves

batch_insert_into_append_only_tree_16_depth_ms
2.17 (+1%)
3.84 (+1%)
N/A
N/A
N/A
N/A
N/A

batch_insert_into_append_only_tree_16_depth_hash_count
16.7
31.8
N/A
N/A
N/A
N/A
N/A

batch_insert_into_append_only_tree_16_depth_hash_ms
0.113 (+1%)
0.108 (+1%)
N/A
N/A
N/A
N/A
N/A

batch_insert_into_append_only_tree_32_depth_ms
N/A
N/A
11.1 (+1%)
17.7 (+1%)
30.8 (-4%)
61.3 (+5%)
113

batch_insert_into_append_only_tree_32_depth_hash_count
N/A
N/A
95.9
159
287
543
1,055

batch_insert_into_append_only_tree_32_depth_hash_ms
N/A
N/A
0.106 (+1%)
0.103 (+1%)
0.0998 (-4%)
0.106 (+5%)
0.101

batch_insert_into_indexed_tree_20_depth_ms
N/A
N/A
14.3 (+1%)
26.3 (+2%)
43.9
84.1 (+2%)
161 (+1%)

batch_insert_into_indexed_tree_20_depth_hash_count
N/A
N/A
109
207
357
691
1,363

batch_insert_into_indexed_tree_20_depth_hash_ms
N/A
N/A
0.109
0.107 (+3%)
0.105 (-1%)
0.104 (+2%)
0.101 (-1%)

batch_insert_into_indexed_tree_40_depth_ms
N/A
N/A
16.0
N/A
N/A
N/A
N/A

batch_insert_into_indexed_tree_40_depth_hash_count
N/A
N/A
129
N/A
N/A
N/A
N/A

batch_insert_into_indexed_tree_40_depth_hash_ms
N/A
N/A
0.105
N/A
N/A
N/A
N/A

Miscellaneous
Transaction sizes based on how many contract classes are registered in the tx.

Metric
0 registered classes
1 registered classes

tx_size_in_bytes
72,419
668,302

Transaction size based on fee payment method
| Metric |  |
| - |  |


Create a new python example script - submit_and_read_block

Description of change
Existing examples are not fully-emcompassing. This example script aims to give the user a full picture of what they are doing and why.
This script has three steps:

Prepare the payload for a block
Submit the block to the Shimmer test network
Use the block ID to read the payload back from the network

It also provides the url to click and go to the explorer.
Might be worth to make the other examples more verbose also. I didn't want to add too much into this PR though so I left that for now.
Type of change

Documentation Fix

How the change has been tested
Ran it in terminal, it outputs what is expected.
Change checklist
Tick the boxes that are relevant to your changes, and delete any items that are not.

[ ] I have followed the contribution guidelines for this project
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] I have checked that new and existing unit tests pass locally with my changes

Thanks!


How to disable color in the browser?

I use debug in several libraries that I put together via browserify(). Now I would like to insert a mechanism so all the logs get (also) stored into a JS variable so they can be later sent somewhere. And obviously I plan to override the global console.xxxxx() methods for that.
So I would prefer to disable colorized output (%c symbols).
BTW, any other suggestion for what I want to achieve? Thanks a lot.

Would love this as well. Doesn't look possible: https://github.com/visionmedia/debug/blob/master/browser.js#L44
@TooTallNate would you accept a PR to fix this?

Yea
On Tuesday, March 3, 2015, JP Richardson<EMAIL_ADDRESS>wrote:

Would love this as well. Doesn't look possible:
https://github.com/visionmedia/debug/blob/master/browser.js#L44
@TooTallNate https://github.com/TooTallNate would you accept a PR to
fix this?
—
Reply to this email directly or view it on GitHub
https://github.com/visionmedia/debug/issues/179#issuecomment-76985161.

@TooTallNate There's a PR that fixes this #247. For your consideration.

Check out middleware proposal here: https://github.com/visionmedia/debug/issues/370


Update copy on student UI calendar tabs

Reassigning to @Coenego for review.

Coverage remained the same at 100.0% when pulling dd1c3bfef2a97df255d8591bcb5b6f1fe1716e20 on bp323:issue-211 into 78455599a6802b63795877f9417813b0f2919714 on CUL-DigitalServices:master.


Sent money/requested money message disappears on refresh from LHN - Reported by @Puneet-here

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Action Performed:

Go to a chat and send money
Refresh the page

Expected Result:
The message shouldn't disappear
Actual Result:
The message disappear
Workaround:
unknown
Platform:

Where is this issue occurring?

Web
iOS
Android
Desktop App
Mobile Web

Version Number: 1.1.89-0
Reproducible in staging?: Y
Reproducible in production?: Y
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Notes/Photos/Videos: Any additional supporting documentation
https://user-images.githubusercontent.com/43995119/187111133-13786dbb-6ded-4fdf-bf27-44ec4f4685eb.mov
Expensify/Expensify Issue URL:
Issue reported by: @Puneet-here
Slack conversation: https://expensify.slack.com/archives/C01GTK53T8Q/p1660070110835229
View all open jobs on GitHub

Sorry I was looking at the dashboard. The copy looks good!

@cdraeger11 Did you already triage this? Or did you reapply the triage label to have someone else triage?

Double checking with @marcaaron Do you think it can be external?

I would guess it's related to an API refactor. But without knowing the root cause a bit hard to say. Maybe we can put an Internal label on it for now and then External after someone looks into it a bit more.

We had a similar issue in past (https://github.com/Expensify/App/issues/5712) and it was fixed by this PR: https://github.com/Expensify/Web-Expensify/pull/31925
Just linking it here hoping it will help in the current issue.

This is not reproduceable on staging - just tested refreshing and the send money request is still there:

Hi @zanyrenney, the iou message disappears at the LHN (preview of message) not from the chat.

bump @zanyrenney

Bump @zanyrenney

This is reproducible even without refreshing.

Grabbing this one as internal, from @Puneet-here's comment here seems we're missing some data in Onyx.

Awesome, thanks for grabbing it @marcochavezf - let me know how I can help.

@marcochavezf Are there any additional PRs coming for this issue? The only linked PR I see hit production some time ago.

Yeah that was the only PR needed to fix this issue. Tested real quick and looks good:
https://user-images.githubusercontent.com/6829422/209414182-74f8d1b8-1433-4ced-9dc5-8a67fd914c28.mov
We can close this out.

Hi @JmillsExpensify, I think it's eligible for the reporting bonus.

Oh thanks. @zanyrenney is OOO so I'll take this over for her in the meantime. I just invited you to the Upwork job.

@JmillsExpensify, applied!

Thanks! Offer sent

Hello @JmillsExpensify, I know it's quite late to say this, but I had also reported the exact same issue:
https://github.com/Expensify/App/issues/8125
Can you check if I'm still eligible for reporting compensation?

Huh, I'm glad you said something. Given that your issue was reported much earlier, and the reproduction steps are the same (in fact more thorough), I'm inclined to award you. I'm also not clear how this duplicate wasn't caught earlier. cc @mallenexpensify for a second opinion.

Fun one!  I think there are a few likely reasons here

#8125 was closed cuz it wasn't reproducible.  Maybe it was still reproducible but under a different set of circumstances.
There's something new going on and it just so happens to mirror the older issue/s
There's a regression which caused this bug

I don't know the answer to the above but, I think @Puneet-here should be compensated for reporting because, had they not, we might not have fixed the bug (ie. commenting on an issue that was closed 6 months ago isn't likely to get much traction) .  I think we should also compensate @Tushu17 unless someone wants to go digging to somehow show #8125 isn't directly related (also... it's only $250, I'm impressed @Tushu17 happened to find this one, I appreciate the 👀 on repo issues and 🐛🐛).
@Tushu17 and @Puneet-here , please accept and Mills or I will get you paid https://www.upwork.com/jobs/~01f948c7e31e63a26d

Nice, I agree with that. Both offers are pending in Upwork, so we'll close the loop on this one shortly.

Thank you for the consideration😄

I have accepted the offer

@JmillsExpensify @mallenexpensify Thank you; Offer accepted, and I am grateful.

@Puneet-here and @Tushu17 have been paid! Thanks!  re-re-closing :)


Error on tye run with a Docker folder on project solution

Describe the bug
We have a problem when we run tye run with a custom tye.yaml and in the folder we have a "Docker" folder.
We found the following Exception:
An unhandled exception has occurred, how unseemly:
System.UnauthorizedAccessException: Access to the path 'C:\git\tc\BackEnd\ENC.TEAMSCHAMP\docker' is denied.
at System.IO.FileSystem.DeleteFile(String fullPath)
at System.IO.File.Delete(String path)
at Microsoft.Tye.Hosting.ReplicaRegistry.DeleteStore(String storeName) in //src/Microsoft.Tye.Hosting/ReplicaRegistry.cs:line 62
at Microsoft.Tye.Hosting.DockerRunner.PurgeFromPreviousRun() in //src/Microsoft.Tye.Hosting/DockerRunner.cs:line 469
at Microsoft.Tye.Hosting.DockerRunner.StartAsync(Application application) in //src/Microsoft.Tye.Hosting/DockerRunner.cs:line 37
at Microsoft.Tye.Hosting.AggregateApplicationProcessor.StartAsync(Application application) in //src/Microsoft.Tye.Hosting/AggregateApplicati
onProcessor.cs:line 24
at Microsoft.Tye.Hosting.TyeHost.StartAsync() in //src/Microsoft.Tye.Hosting/TyeHost.cs:line 104
at Microsoft.Tye.Hosting.TyeHost.RunAsync() in //src/Microsoft.Tye.Hosting/TyeHost.cs:line 70
at Microsoft.Tye.Hosting.TyeHost.RunAsync() in //src/Microsoft.Tye.Hosting/TyeHost.cs:line 76
at Microsoft.Tye.Program.<>c__DisplayClass9_0.<b__0>d.MoveNext() in //src/tye/Program.RunCommand.cs:line 97
--- End of stack trace from previous location where exception was thrown ---
at Microsoft.Tye.Program.<>c__DisplayClass9_0.<b__0>d.MoveNext() in /_/src/tye/Program.RunCommand.cs:line 97
--- End of stack trace from previous location where exception was thrown ---
at System.CommandLine.Invocation.CommandHandler.GetResultCodeAsync(Object value, InvocationContext context)
at System.CommandLine.Invocation.ModelBindingCommandHandler.InvokeAsync(InvocationContext context)
at System.CommandLine.Invocation.InvocationPipeline.<>c__DisplayClass4_0.<b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c.<b__19_0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass14_0.<b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass22_0.<b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c.<b__10_0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass12_0.<b__0>d.MoveNext()
To Reproduce

Create a "Docker" folder in the same folder that we have the tye.yaml file and the other projects folder.
Run tye run

You will find the following exception:

Further technical details

Include the output of tye --version
0.1.0-alpha.20209.5+e3fc0045bd1e5913da935241874761929f1e8465
0.2.0-alpha.20212.1+a55639072e3b0f17cfe9fdf1e243c4dd61993731
The platform (Linux/macOS/Windows)
Windows

Same issue here. I could reproduce it and my tye version is 0.1.0-alpha.20209.5+e3fc0045bd1e5913da935241874761929f1e8465

Hmm, do you have a .tye folder next to the solution?

Oh wait I just saw the repro, I'll take a look.

Great!


bumping js-slang to 0.4.54

Description
Bumps js-slang to 0.4.54
Type of change

[ ] Bug fix (non-breaking change which fixes an issue)
[ x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update
[ ] Code quality improvements

How to Test
Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration
Checklist
Please delete options that are not relevant.

[ ] I have tested this code
[ ] I have updated the documentation

Pull Request Test Coverage Report for Build 6402

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 32.259%

Totals

Change from base Build 6390:
0.0%

Covered Lines:
1869

Relevant Lines:
5405

💛  - Coveralls

There was an update to the function signature of typeCheck:
https://github.com/source-academy/js-slang/blob/master/src/typeChecker/typeChecker.ts#L218-L221
Will need to pass in a Context object as well

There was an update to the function signature of typeCheck:
https://github.com/source-academy/js-slang/blob/master/src/typeChecker/typeChecker.ts#L218-L221
Will need to pass in a Context object as well:
const typeErrors = parsed && typeCheck(validateAndAnnotate(parsed!, context), context)[1];

Yes that did it. Thanks!


Remove normalisation of the password when using SCRAM auth

KafkaJS was incorrectly normalised the SCRAM password, resulting in , and = characters being encoded causing authentication failures.
According to the cited RFC in the source code (https://tools.ietf.org/html/rfc5802#section-5.1), just the username needs to be normalised.
Closes #506

Java client for reference: https://github.com/axbaretto/kafka/blob/master/clients/src/main/java/org/apache/kafka/common/security/scram/ScramFormatter.java#L101

Thank you very much for your contribution! I just want to reproduce this issue myself, in order to see that the proposed fix works, and then I'll merge it. :)

Thanks. I think I want to add some tests for special characters in the password but want some direction on how I should include something like that 👍

What I did was to simply change the password in scripts/createScramCredentials.sh to include =, update the passwords to match in testHelpers/index.js, and then run the existing tests. Without your fix, the tests failed, and with the fix, they pass. 👍

Perfect, thanks for that

This fix will be available in pre-release version 1.11.0-beta.12 as soon as the build is done.


SoC2023 RFSB509

About
In the context of this years Summer of Code I have implemented the RFSB509 cryptographic hash function [1] in this branch. This function depends on the SHA256 algorithm and AES128 cipher procedure. A SHA256 implementation already exists in this library so this branch only adds the AES128 (Inv_)Cipher procedure. Additionally, because I initially misread the RFSB509 paper I also added the AES256 (Inv_)Cipher procedure.  The AES procedures were implemented according to the NIST FIPS 197 standard and optimized using information from [2]. All subprograms are implemented to avoid timing attacks, this  includes no integer multiplications, no table lookups and no branching based on secret data.
References

Bernstein, D.J., Lange, T., Peters, C., Schwabe, P. (2011). Really Fast
Syndrome-Based Hashing. In: Nitaj, A., Pointcheval, D. (eds) Progress in
Cryptology – AFRICACRYPT 2011. AFRICACRYPT 2011. Lecture Notes in
Computer Science, vol 6737. Springer, Berlin, Heidelberg.
https://doi.org/10.1007/978-3-642-21969-6_9
Satoh, A., Morioka, S., Takano, K., Munetoh, S. (2001).  A Compact
Rijndael Hardware Architecture with S-Box Optimization.  In: Boyd, C.
(eds) Advances in Cryptology — ASIACRYPT 2001. ASIACRYPT 2001.  Lecture
Notes in Computer Science, vol 2248. Springer, Berlin, Heidelberg.
https://doi.org/10.1007/3-540-45682-1_15

We'll squash after review.

Still fails on both M1 and Intel macOS with
   [link]         testall.adb
Undefined symbols for architecture arm64:
  "_rfsb509_crypto_hash", referenced from:
      __ada_rfsb509_supercop_regression in rfsb509_supercop_regression.o
ld: symbol(s) not found for architecture arm64
collect2: error: ld returned 1 exit status
gprbuild: link of testall.adb failed

Build of "tests" now looks good on both M1 and Intel macOS

I too was able to get a successful build and passing tests on M1 w/ GNAT 12.2.1 and gprbuild 22.0.1.

Which version of the RISC-V cross compiler are you using to compile the performance tests? I think I need an upgrade... mine is very old...

Hmmm.. I installed the risc64-elf 12.2.1 toolchain (using alr 1.2.2) on my Linux VM.
Now in perf//aes256 I get:
rchapman@buster:~/prog/maxnacl/perf/aes256$ make
gprbuild -j4 -Paes256 -v -XSPARKNACL_RUNTIME_MODE=zfp -XSPARKNACL_RUNTIME_CHECKS=disabled -XSPARKNACL_CONTRACTS=disabled -XSPARKNACL_BUILD_MODE=O2 -XSPARKNACL_TARGET_ARCH=rv32imc_a4
gprconfig: can't find a toolchain for the following configuration:
gprconfig: language 'ada', target 'riscv64-elf', runtime 'zfp-rv32imac'

==============Messages for file: /home/rchapman/prog/maxnacl/sparknacl.gpr
     1. library project Sparknacl is
                        |
        >>> no compiler for language "Ada", cannot compile "sparknacl-mac.adb"
gprbuild: *** compilation phase failed
make: *** [makefile:12: obj/aes256] Error 4

Yes.. PATH looks like this:
/home/rchapman/gnat/alr1.2.2/bin:
/home/rchapman/.config/alire/cache/dependencies/gprbuild_22.0.1_24dfc1b5/bin:
/home/rchapman/gnat/spark-pro-23.0w-20220718-x86_64-linux-bin/bin:
/home/rchapman/.config/alire/cache/dependencies/gnat_riscv64_elf_12.2.1_fc4a4d08/bin:
/home/rchapman/.config/alire/cache/dependencies/gnat_native_12.1.1_df1f36b3/bin:
/media/psf/Home/Desktop/prog/AdaCore/gnatstack-20.2-20200603-1DACB-src/bin

rchapman@buster:~/prog/maxnacl/perf/aes256$ gprconfig --target=riscv64-elf
--------------------------------------------------
gprconfig has found the following compilers on your PATH.
Only those matching the target and the selected compilers are displayed.
   1. GNAT for Ada in /home/rchapman/.config/alire/cache/dependencies/gnat_riscv64_elf_12.2.1_fc4a4d08/bin/ version 12.2 (embedded-polarfiresoc runtime)
   2. GNAT for Ada in /home/rchapman/.config/alire/cache/dependencies/gnat_riscv64_elf_12.2.1_fc4a4d08/bin/ version 12.2 (light-hifive1 runtime)
   3. GNAT for Ada in /home/rchapman/.config/alire/cache/dependencies/gnat_riscv64_elf_12.2.1_fc4a4d08/bin/ version 12.2 (light-polarfiresoc runtime)
   4. GNAT for Ada in /home/rchapman/.config/alire/cache/dependencies/gnat_riscv64_elf_12.2.1_fc4a4d08/bin/ version 12.2 (light-rv32i runtime)
   5. GNAT for Ada in /home/rchapman/.config/alire/cache/dependencies/gnat_riscv64_elf_12.2.1_fc4a4d08/bin/ version 12.2 (light-rv32iac runtime)
   6. GNAT for Ada in /home/rchapman/.config/alire/cache/dependencies/gnat_riscv64_elf_12.2.1_fc4a4d08/bin/ version 12.2 (light-rv32im runtime)
   7. GNAT for Ada in /home/rchapman/.config/alire/cache/dependencies/gnat_riscv64_elf_12.2.1_fc4a4d08/bin/ version 12.2 (light-rv32imac runtime)
   8. GNAT for Ada in /home/rchapman/.config/alire/cache/dependencies/gnat_riscv64_elf_12.2.1_fc4a4d08/bin/ version 12.2 (light-rv32imafc runtime)
   9. GNAT for Ada in /home/rchapman/.config/alire/cache/dependencies/gnat_riscv64_elf_12.2.1_fc4a4d08/bin/ version 12.2 (light-rv32imafdc runtime)
  10. GNAT for Ada in /home/rchapman/.config/alire/cache/dependencies/gnat_riscv64_elf_12.2.1_fc4a4d08/bin/ version 12.2 (light-rv64im runtime)
  11. GNAT for Ada in /home/rchapman/.config/alire/cache/dependencies/gnat_riscv64_elf_12.2.1_fc4a4d08/bin/ version 12.2 (light-rv64imac runtime)
  12. GNAT for Ada in /home/rchapman/.config/alire/cache/dependencies/gnat_riscv64_elf_12.2.1_fc4a4d08/bin/ version 12.2 (light-rv64imafc runtime)
  13. GNAT for Ada in /home/rchapman/.config/alire/cache/dependencies/gnat_riscv64_elf_12.2.1_fc4a4d08/bin/ version 12.2 (light-rv64imafdc runtime)
  14. GNAT for Ada in /home/rchapman/.config/alire/cache/dependencies/gnat_riscv64_elf_12.2.1_fc4a4d08/bin/ version 12.2 (light-rv64imc runtime)
  15. GNAT for Ada in /home/rchapman/.config/alire/cache/dependencies/gnat_riscv64_elf_12.2.1_fc4a4d08/bin/ version 12.2 (light-rv64imfc runtime)
  16. GNAT for Ada in /home/rchapman/.config/alire/cache/dependencies/gnat_riscv64_elf_12.2.1_fc4a4d08/bin/ version 12.2 (light-spike runtime)
  17. GNAT for Ada in /home/rchapman/.config/alire/cache/dependencies/gnat_riscv64_elf_12.2.1_fc4a4d08/bin/ version 12.2 (light-tasking-polarfiresoc runtime)
  18. GCC-ASM for Asm in /home/rchapman/.config/alire/cache/dependencies/gnat_riscv64_elf_12.2.1_fc4a4d08/bin/ version 12.2.0
  19. GCC-ASM for Asm2 in /home/rchapman/.config/alire/cache/dependencies/gnat_riscv64_elf_12.2.1_fc4a4d08/bin/ version 12.2.0
  20. GCC-ASM for Asm_Cpp in /home/rchapman/.config/alire/cache/dependencies/gnat_riscv64_elf_12.2.1_fc4a4d08/bin/ version 12.2.0
  21. LD for Bin_Img in /home/rchapman/.config/alire/cache/dependencies/gnat_riscv64_elf_12.2.1_fc4a4d08/bin/ version 2.39
  22. GCC for C in /home/rchapman/.config/alire/cache/dependencies/gnat_riscv64_elf_12.2.1_fc4a4d08/bin/ version 12.2.0
Select or unselect the following compiler (or "s" to save): ^C
r

Ah... I think I need to update Ada_Drivers_Library so it specified the "light" runtime rather than "zip"...

How come SPARKNaCl.Hashing.RFSB509.Hash takes a "Key" formal parameter, but SUPERCOP_RFSB509_API.SUPERCOP_RFSB509_Hash does not? This needs to be explained...

Both please!

OK... Understood. I think a comments in the package spec to say "this is the keyed variant", and "Use (others => 0) as the key to get same behaviour as the SUPERCOP reference code" should suffice.  That also explains why that value for Key is used in the test program...

So add that link and explanation as a comment just above the first declaration.


S_EnvObj Generation unclear

Hello Dr. Bao,
Sorry first up for more questions. I spent the last weeks trying to get TexSdG working, and while your previous clarification about the Material mapping definitely helped, I'm still facing issues.
I ported the marlab Code over to python for my own experiments, but confirmed the optimization produces the same results (and all the steps in between as well). So far so good. (I can also recommend this, as some scipy optimizers are much quicker for me than matlab itself)
I teste this on the real world Scene11 and got great results!
But on the simulated scene I simply cannot get all materials to properly 'estimate'.
Scene 7, first frame for example, most materials are correct but the trees and flowers/Grass tufts simply always come out as water. The Grass surfaces on the other Hand works fine.

Please ingore Lack of textur, etc. I have not ported some of the visualization modes/details)
As scene 11 works I think this must be due to the environment object estimation (or maybe temperature or radiance parameters, but I've tested a lot there and have similar results), as you provide a ground truth environment estimation for that scene (which I used for testing)
Other scene Show similar errors with a few materials being categorically wrong.
In your follow up "why are thermal images blurry" you weite that you use K-means to group environment objects and the average channel wise across each object. Do I understand that part correctly?
I get very clean clusters in the aforementioned scene (1 cluster being the sky and stones, the other being everything else) but I feel like I'm doing sth wrong still.

I then take each cluster and calculate the mean for each of the 54 channels.
Any idea on why this might not work as expected?
Ps: I'm sorry I cannot post proper screenshots, these Photos are all I can provide.

Hi Daepilin,
Thanks for sharing your thoughts about a python version of TeX-SGD. I'm definitely interested and want to have a test if you've released it somewhere.
Since you have good results for Scene 11. I think the error you met with synthetic scenes is not a bug of your codes. Environment estimation does matter. K-means clustering is the best solution we have for the moment, but there are many remaining aspects deserving further exploration.
For objects in crowded scenes, the non-uniformity of an object's environment starts to dominate over. In principle, more environmental terms are needed for crowded scenes (e.g., scene 7 or scene 1) than open scenes (like experimental scene 11 or synthetic scene 9). But 'with more environmental terms' immediately requires more spectral bands to solve the problem, and the solution is less robust (trapped in local minima). Also note that the synthetic data contains numeric error as well in Monte Carlo simulations (finite sampling).
We think using spatial+spectral information for TeX decomposition (i.e., TeX-Net) is a promising way to go in the future.
Hope this can help!

Definitely interesting input!
Unfortunately for scene 7 I could, even with 6 en objects, not quite reproduce correct results.
But your opinion and my experiments seem to confirm my idea that it's the environmental radiation.
Except for temperature. The matlab example uses 10-20C for the real scene, but for simulated data this is obviously not correct. Would it be possible to know that range for those as well?
As for sharing code: I'm not sure i will be able to share everything, but I will do what I can when I can :)

Yes, you can check the tMap in the ground truth folder for the ground truth temperatures. You can set upper and lower bounds accordingly.
'Scipy slsqp optimizer' is the information I would like to know. That is sufficient. Thanks.

Glad I could provide some useful info. I'm also still experimenting with other optimizer algorithms but the final results seem very similar right now.
Maybe 2 more things on this:

what did you use for estimating the environment in scene 11,if you can share thst info? I've compared the GT you provide with my own K-means and the estimations are quite significantly different. Do you provide your own cluster centers or do you use an automatic approach?

Do you do further processing with the cluster means before using them in later steps? Right now I'm basically simply using
cluster_idx = kmeans(reshape(HSI, [], 49) 2); Env1 = mean(reshape(HSI, [], 49)(cluster_idx==1, :)); Env2 = mean(reshape(HSI, [], 49)(cluster_idx==2, :)); S_EnvObj = cat(1,Env1,Env2);   (example what I tried for comparing your scene 11 GT with my Code)

Of course only if you want/can answer this. I know you wrote in other issues that you plan further releases so I dont want to pry too much.

Yeah, we plan to confirm everything is correct and make the process more robust, before we can release it.
For Scene 11, mostly the sky is not included in the images. So, we estimated the sky radiance from the reflection of the checkerboard marker. 2 out of 4 squares of the marker are highly reflective and diffusive.

Thank you very much :)

the input data provided by the author has a size of 1088*1920 I cropped it to a multiple of 16 to be able to train.
the loss becomes nan during the training process.
have you encountered this type of problem?

2. Env1

May I ask which part of the original code you are replacing with this code?
I fail to find the k-means related code in the  SGD_and_TeX_vision_matlab_code_package https://purdue0-my.sharepoint.com/:f:/g/personal/sjape_purdue_edu/EikkN6bcJJNLoAP-ks_DqVQBIzIrE_LnN8bGEjtlKz6jlA?e=UOAlvE
I'm not very good at matlab code. It would help me a lot if you could reply to me.

Glad I could provide some useful info. I'm also still experimenting with other optimizer algorithms but the final results seem very similar right now.
Maybe 2 more things on this:

what did you use for estimating the environment in scene 11,if you can share thst info? I've compared the GT you provide with my own K-means and the estimations are quite significantly different. Do you provide your own cluster centers or do you use an automatic approach?
Do you do further processing with the cluster means before using them in later steps? Right now I'm basically simply using
cluster_idx = kmeans(reshape(HSI, [], 49) 2);
Env1 = mean(reshape(HSI, [], 49)(cluster_idx==1, :));
Env2 = mean(reshape(HSI, [], 49)(cluster_idx==2, :));
S_EnvObj = cat(1,Env1,Env2);    (example what I tried for comparing your scene 11 GT with my Code)

Of course this does not guarantee me which of the 2 (or more) clusters is the sky, but from my understanding thst is only relevant for visualization?
Of course only if you want/can answer this. I know you wrote in other issues that you plan further releases so I dont want to pry too much.

May I ask which part of the original code you are replacing with this code?
I fail to find the k-means related code in the SGD_and_TeX_vision_matlab_code_package https://purdue0-my.sharepoint.com/:f:/g/personal/sjape_purdue_edu/EikkN6bcJJNLoAP-ks_DqVQBIzIrE_LnN8bGEjtlKz6jlA?e=UOAlvE
I'm not very good at matlab code. It would help me a lot if you could reply to me.


Different behavior for local build and the deployed build

BTW, I'm not sure if it's now-cli exactly, maybe it's related to the platform itself and I should send the feedback via some Zeit contact form instead. Then just redirect me and sorry for bothering

Local one (it's successfully built):

On the now.sh (build failed):

Whoops, my bad. This were some necessery files that by mistake were added to .nowignore

Particularly, it was types folder, with, well... types :)
By some reason it was working with Next v8 + withTypescript plugin. But with build-in typescript support on Next v9 it works in a bit different way. Shouldn't ignore types for deployments.


RedundantBlockCall ignores asymptotic complexity

The asymptotic complexity of yield and call is equal (depending on size of the executed code or how often it's executed). Actually call just adds a constant value to the needed time.
So even when you run call in a loop the performance of the actual yield/call is never worse than by a constant factor. I did some tests and always got just a constant ~30% performance drop for the actual yield/block call (nothing like an exponential or higher decrease in performance appeared).
Example: Your software needs overall 10 seconds, every call wastes 0.000001 or 10^-6 seconds (compared to yield) and call comes up 1000 times. Then the software will need just 0.01 seconds more compared to yield.

Even worse, RedundantBlockCall also fails when there's just a constant value added to the whole method.
I did some performance tests for such a scenario and could get absolutely no difference between yield and call. (the difference was always +/- 1 %)
def fun_yield
  yield
end
fun_yield {<PHONE_NUMBER>.times { print('') } }

def fun_call(&block)
  block.call
end
fun_call {<PHONE_NUMBER>.times { print('') } }

Any progress on this?


Billing - periodic operation -  some fixes

For a DRAFT invoice, the final invoice date is only known when it gets finalized. The invoice synchronization takes now the finalized_at property into consideration.
The side effect of that change is that the initial date gets updated during the periodic billing process. This as an impact on the pagination logic in place when going thru the list of invoices of the previous month. The pagination logic has been updated to make sure the periodic billing process does not 'forget' any invoices !

The billing tests are failing with HTTP Status Code 595 - User account Changed!
I do not have the problem when running the tests locally!?!
@LucasBrazi06 :
I'm changing the user locale while running the Billing tests. Is this forbidden?
Thanks,
Calude
The check of the user hash is not new!. Rigth
?
// Check User's Hash
if (userHashID !== this.buildUserHashID(user)) {
throw new AppError({
source: Constants.CENTRAL_SERVER,
errorCode: HTTPError.USER_ACCOUNT_CHANGED,
message: 'User has been updated and will be logged off',
module: MODULE_NAME,
method: 'isSessionHashUpdated',
user: req.user
});
}

@LucasBrazi06 :
I'm changing the user locale while running the Billing tests. Is this forbidden?

You'll have to relog in that case and get the new token ;-)

@LucasBrazi06 :
I'm changing the user locale while running the Billing tests. Is this forbidden?

You'll have to relog in that case and get the new token ;-)

Ok ... i'll remove these locale tests for now. I do not know how to 'relog' with the test helpers that we have so far!

Tests are still failing but not the ones for the billing!

Car
Success cases
With component Car (utcar)
Where admin user
Should be able to get car catalogs:
TypeError: Cannot read property 'id' of undefined
at Context. (/home/runner/work/ev-server/ev-server/test/api/CarTest.ts:94:43)

I'm lost!


[FEATURE] Sign and notarise the releases.

I would be happy to sponsor a year's Apple developer subscription for the author to sign releases and have Apple Notarise the DMG to lower the barrier for take-up amongst non-technical users on Apple.

FWIW @sdatkinson I am working on new CI scripts for the iPlug2OOS repo which will do all the signing etc in the cloud

Github actions can do it excellently with the existing script, I should think, or alternatively, there is a specific action for it.

I understand. If you not are actively working on it, I'm happy also to contribute code there. Or on the port to Cmake that I saw was being worked on, if it is still active.

oops duplicate -> #365 but this is really important

Let's close this and continue w/ #365 to keep things deduped 👍🏻


Travis build fails with yarn

^

https://github.com/hackjutsu/Lepton/commit/2a653113d517830c9a609932a2c790860cb4fa9a


Improve AssetEvent docs

The docs for AssetEvent are a bit vague on what exactly it means for an asset to be "added", or "modified" or "loaded with dependencies".
For instance, when hot-reloading an asset, we expect Modified to fire, but what about LoadedWithDependencies? Will that fire? What if the newly modified asset has additional dependencies?
It should be easy to deduce exactly when an asset event will fire for a particular sequence of asset operations.

Another example: Do Unused assets not get automatically Removed? When would one want to use one over the other?

Unused assets will get removed automatically, if they were not already removed earlier. Unused is mainly for RenderAsset and RenderAssetUsage, so that we can remove the asset data from the Assets collection in RAM, but know when to unload the corresponding GPU data in VRAM when the asset is no longer actually used.
Tldr Unused = last handle dropped, Removed = removed from Assets resource.


Code quality

Hi @jannejava!
Thanks for your package! But I cannot use it in my projects, because code quality is not good. So, I wrote one more wrapper (json), and maybe you can take some ideas for your code refactoring.
Link: https://github.com/tarampampam/wrappers-php

Sorry for my english.

Why not submit a PR to improve the code quality of this existing package then? 🙂

Because PR would overwrite 90% of existing code :(

You mean like #2? 😉

Yeah :D

Come on! Don't be shy! It has already happened once :-)
In essence, we are only talking about variants of:

$array = json_decode($string);

if(json_last_error() != JSON_ERROR_NONE) {
   throw \Exeception
}


Synchronize writing to connection and aborting connection

Fixes https://github.com/aspnet/AspNetCore/issues/6701
Verified perf didn't change.

Help me understand this change? The description of the fix is a bit light on explanation.

Servers could get error logs saying "Writing is not allowed after writer was completed" if the connection was closed while writing, especially when writing the ping message. This synchronizes the abort with the writing so we wont see error logs when writing would be expected to fail.

Unit test?

Unit test?

It's a race and there isn't anywhere to inject our own test code to force it.

Do we need to backport this?

Do we need to backport this?

Worst case it's an erroneous error log. There is no functional issue here. Based on that I don't think we should, but open to other opinions.

Looks like a legit test failures:

Microsoft.AspNetCore.SignalR.Tests.HubConnectionHandlerTests.ConnectionTimesOutIfInitialPingAndThenNoMessages

Microsoft.AspNetCore.SignalR.Tests.HubConnectionHandlerTests.AbortFromHubMethodForcesClientDisconnect

@aspnet-hello triage https://dev.azure.com/dnceng/public/_build/results?buildId=186945&view=logs

This comment was made automatically. If there is a problem contact<EMAIL_ADDRESS>I've triaged the above build. I've created/commented on the following issue(s)
https://github.com/aspnet/AspNetCore-Internal/issues/2511


implement sphinx doc and read the docs

Hello @Remi-Gau! Thanks for opening this PR. We checked the lines you've touched for PEP 8 issues, and found:

In the file docs/source/conf.py:

Line 34:28: W291 trailing whitespace
Line 91:2: W292 no newline at end of file

Error in CI on checking the markdown:
$ npm install `cat npm-requirements.txt`

Unhandled rejection Error: EACCES: permission denied, mkdir '/home/travis/.npm/_cacache/content-v2/sha512/a4/f5'

Unhandled rejection Error: EACCES: permission denied, mkdir '/home/travis/.npm/_cacache/content-v2/sha512/33/9a'

Unhandled rejection Error: EACCES: permission denied, mkdir '/home/travis/.npm/_cacache/content-v2/sha512/ce/b2'

Unhandled rejection Error: EACCES: permission denied, mkdir '/home/travis/.npm/_cacache/content-v2/sha512/d2/66'

Unhandled rejection Error: EACCES: permission denied, mkdir '/home/travis/.npm/_cacache/content-v2/sha512/a2/ce'

npm ERR! cb() never called!

npm ERR! This is an error with npm itself. Please report this error at:

npm ERR!     <https://npm.community>

npm ERR! A complete log of this run can be found in:

npm ERR!     /home/travis/.npm/_logs/2020-11-08T10_35_52_601Z-debug.log

The command "npm install `cat npm-requirements.txt`" failed and exited with 1 during .

Merging just to provide all functions with a template help section


Optimize validate_arguments

Summary
Minor optimizations to validate_arguments, to avoid duplicate lookups, pre-allocate the vector, etc.

This could in theory be merged upstream, what's the workflow for that these days?

It's complicated :D You can create a new branch by forking from upstream rather our version. Make your changes and submit your PR to RustPython.
After merge: Merge the changes downstream by using git rebase --rebase-merges. This will probably pull in many larger changes