[Feature Request] FilePicker should allow to pick folder files

Is your feature request related to a problem? Please describe.
Some MS vendor formats (e.g. those of Bruker) report the run data as a folder instead of a single file. For this it would be great to have the filepicker be able to also select single or multiple folders.
Describe the solution you'd like
Similar behaviour to picking files for folders

Sorry for not providing a better description. I had the latter case in mind, specifically referencing the folder.

This is very difficult to implement, as websites do not have access to path. So at best i could return the name of the folder. But a quick search showed even this to be rather difficult.
Let me know what exactly you would need with an example and which alternatives you would be okay with. Thanks in advance!

I think the name of the folder would be sufficient. I think a function like this would be needed since e.g. Bruker does not create a single file as an output for a MS run but rather a folder with many files.
The resulting data would look like this:

This is how I like to reference it in the workbook sheet:

For the user a workaround could be to simply zip the folder. However this is rather impractical since most tools only accept it unzipped.

So here is the thing. There is an API to upload folders instead of files, but.. This API MUST upload the files inside the folder and then i can get the folder name from the relative paths of these files. For me on opera this process triggered a warning, asking me if i really want to upload all these files. In addition i think the file size can get quite large for something as simple as a folder name.
@ZimmerD of course you can give me additional input on this topic, but currently i think this is simply not possible to do in Swate. But(!) Swate in ARCitect should be able to solve this issue with Electron features!

Added Relative paths and folder pick option for ARCitect ✨ Thanks to @JonasLukasczyk for the ARCitect side help


Widgets are left Justified

From the screenshots I've seen the top half is usually at the right of the screen.  Not sure if there is something I need to configure.  Screenshot attached to show version / my view

This might be caused by having other conflicting widget maybe? I am not sure how to replicate this error.


Wrong Window Title

VSCode Version: 1.23
OS Version: Windows

Steps to Reproduce:

When I open a file the window title shows the path and file name.
If there are multiple files open, the window title does not change when I select a different tab.
The title doesn't update until I switch to another application then back to VSCode.

@wjmsrv when this happens and you resize the window, does the title then update?


Battle Items

Such as Toxic Orb, Flame Orb, Life Orb.

help wanted: how do I make a held item that's recognized by the battle system?

I noticed that evidently you can apply the effects from an item in the showdown folder to any item and it may juay work? Just something I read in the dev server, probably worth checking it out, unless you already knew about it.

I noticed that evidently you can apply the effects from an item in the showdown folder to any item and it may just work? Just something I read in the dev server, probably worth checking it out, unless you already knew about it.

Yep, that's what I'd like to do. Short of a code example, this is just repeating what I would like to do. Adding items has been paused until 1.4 comes out, but I may get the speed cutting in there before then.


Internal server error toggling OpenID visibility

Current master version 1.1.0+491-g5707ad3b
User settings, OpenID, Show in profile -> 500
user/settings/openid/toggle_visibility

What error do you get in log?

I just tried to reproduce locally but failed. The above was from http://gitea.debian.net so I don't have access to logs.

Cannot reproduce on try.gitea.io (with 2c3a229) either.
@MTecknology I wonder if my latest commit (you are 2 commits behind) accidentally fixed this bug.
It will help to know more (logs, backend...)

Cannot reproduce locally with 5707ad3bb72179e90adea16174102303564a22a9 either. That's supposedly the same version running on gitea.debian.net - I'm using a PostgreSQL backend here.

@strk It looks like it was probably some of the screwing around I was doing that was causing the 500's. It looks like you may have hit it when I was trying to add some caching to nginx.

@strk @MTecknology can this be closed than?

I'll close, OpenID is now disabled on there anyway.


Bug 1572129 - recover in-progress mediawiki provision and deprovision jobs

@jmrodri did we create a issue to track the refactor of recover?
I was planning on putting together a proposal soon for this but I think it would be worth linking this PR to that issue?


rm backtester and some code that seemed to be only called by it

This PR
https://trello.com/c/nYgDEl3M/385-ability-to-query-raw-data-by-msa

LGTM.


Mode button not working during tests

It has been reported that the mode button does not work in any kind of test in order to interrupt it before the natural termination.
The problem emerged after the refactoring concerning the plotting library, switching from matplotlib to pyqtgraph library.

This issue seems not to exist anymore. Thus, it will be closed.


[SPARK-9144] Remove DAGScheduler.runLocallyWithinThread and spark.localExecution.enabled

Spark has an option called spark.localExecution.enabled; according to the docs:

Enables Spark to run certain jobs, such as first() or take() on the driver, without sending tasks to the cluster. This can make certain jobs execute very quickly, but may require shipping a whole partition of data to the driver.

This feature ends up adding quite a bit of complexity to DAGScheduler, especially in the runLocallyWithinThread method, but as far as I know nobody uses this feature (I searched the mailing list and haven't seen any recent mentions of the configuration nor stacktraces including the runLocally method).  As a step towards scheduler complexity reduction, I propose that we remove this feature and all code related to it for Spark 1.5.

Ping @tdas, who helped come up with the idea of removing this after we noticed that some messiness in error-handling in runLocallyWithinThread led to a problem in #7385.
Also, ping @kayousterhout or @markhamstra to see if they can think of any reasons why we should save this feature.

Merged build triggered.

Merged build started.

Test build #37675 has started for   PR 7484 at commit eec39fa.

If nobody is going to miss it, I'd be happy to be rid of localExecution.  But this PR should really be "Deprecate DAGScheduler.runLocallyWithinThread and spark.localExecution.enabled" since the actual removal can't occur until later.

Fair enough RE: deferring the actual removal to 1.6.  I can re-work this to only deprecate the setting and the public methods that expose the allowLocal flag.

Test build #37675 has finished for   PR 7484 at commit eec39fa.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
class NaiveBayes(override val uid: String)
class PCA(JavaEstimator, HasInputCol, HasOutputCol):
class PCAModel(JavaModel):
abstract class Expression extends TreeNode[Expression] with Product
trait Generator extends Expression
trait NamedExpression extends Expression
abstract class Attribute extends LeafExpression with NamedExpression
abstract class LogicalPlan extends QueryPlan[LogicalPlan] with Logging with Product
abstract class SparkPlan extends QueryPlan[SparkPlan] with Logging with Product with Serializable

Merged build finished. Test FAILed.

I think the idea is to run stuff like take much faster. Why do we want to remove this?

@rxin, it turns out that this optimization for local actions is guarded behind a feature flag which is off by default.  Although this path gets tested in DAGSchedulerSuite, I think it's somewhat unlikely that it ends up getting used in most production deployments.

Also, there is a bit of messiness in how runLocallyInThread handles exceptions that led to a flaky test in one of my other patches, so I wanted to see whether we could consider just removing that codepath entirely rather than going through the work to produce and review fixes for that rare corner-case in such a rarely-used feature.

FYI, this used to be on by default (and not flagged) until August last year, when this commit turned it off<EMAIL_ADDRESS>It doesn't seem horrible to keep it off forever since it created some problems before. The main place it helped was if you call first(), take() etc on a dataset when working interactively, but maybe sending one task isn't that bad.

BTW it would also be nice to test the difference this makes before deciding, though the optimization only helps in somewhat limited cases (e.g. it won't help much if you do a shuffle).

I think the main time when this can help a lot is when you are connecting to a busy cluster, and in that case, it can take a while to get something scheduled.
If the cluster is idle, it takes just a few ms to launch a task, and as a result users won't be able to tell the difference at all.

I submitted https://github.com/apache/spark/pull/7585 to bring this up to date. We can merge that one.

Closing since this was done in #7585.


getusermedia

How do you use this in a headless server? The latest browsers (firefox/chrome) want a https connection. So instead of localhost, I just enter IP of server on the LAN. Where to install the certificates at?
vad error {"message":"getUserMedia is not implemented in this browser"}
This is nice though!

Hey,
We should probably put together guide for how to do this, but check out this thread https://www.reddit.com/r/homelab/comments/uaqy88/ssl_certificates_for_local_network_home_domain/ has some script that should work. Basically, you will want to have your server configured to use the self signed cert, and on your device import the cert.

@cake2 for me I did not want to bother with that, I have my phone on my home vpn via NAS, and just connect to my main machine. Now of course there is that thingie that it wants https, but with chrome (which I genearlly don't use) there is option to treat insecure origins as secure (for specific hosts), so I just set that to my machines lan ip and it works with no certs. And I can chat hands free on my lunch walk or what no :)

Arbius has a $200 AIUS bounty for this issue!
Brief: Fix/Complete Issue, add function calling from Ollama, but do this in a way that function calling can be extended to using other things other than Ollama in the future.
Please read carefully:
To begin work on a bounty, reply by saying “I claim this bounty” - you will have 48 hours to submit your PR before someone else may attempt to claim this bounty.
To complete the bounty, within 48 hours of claiming, reply with a link to your PR referencing this issue and an Ethereum address. You must comply with reviewers comments and have the PR merged to receive the bounty reward. Please be sure to focus on quality submissions to minimize the amount of time reviewers must take.


Simple python program executes more slowly with codon than with python 3.11

Hi,
I tested codon on a simple, self-contained python program (code below), which I thought might be type of algorithm to benefit from compilation. To my surprise, the runtime increased. Since it was suggested in the FAQ to open an issue in such instances, here we go.
Best regards,
Peter
$ time python murphy.py
5.797

real	1m9.516s
user	1m9.389s
sys	0m0.063s

$ codon build -release -exe murphy.py
ld: warning: dylib (/Users/peter/.codon/lib/codon/libcodonrt.dylib) was built for newer macOS version (11.7) than being linked (11.3)
ld: warning: object file (murphy.o) was built for newer macOS version (13.0) than being linked (11.3)
ld: warning: dylib (/Users/peter/.codon/lib/codon/libomp.dylib) was built for newer macOS version (11.7) than being linked (11.3)

$ time ./murphy
5.797

real	1m25.066s
user	1m30.261s
sys	1m40.128s

import random
from collections import defaultdict
from functools import partial
from itertools import combinations

NUM_CARDS = 12
NUM_CHOSEN = 4
ROW_LENGTH = 3

def zip_fun(*fs):
    def result(x):
        return tuple(f(x) for f in fs)

    return result

def combs(k=NUM_CHOSEN):
    for s in combinations(range(NUM_CARDS), k):
        # yield frozenset(s)
        yield set(s)

def display(s):
    result = ["X" if i in s else "-" for i in range(NUM_CARDS)]
    result = [
        "".join(result[(ROW_LENGTH * i) : (ROW_LENGTH * i + ROW_LENGTH)])
        for i in range(NUM_CARDS // ROW_LENGTH)
    ]
    result = "\n".join(result)
    return result

def minimal(f, xs):
    min_value = None
    result = None
    for x in xs:
        f_x = f(x)
        if min_value is None or f_x < min_value:
            min_value = f_x
            result = x
    return result

def quality1(x):
    a = max(i % ROW_LENGTH for i in x) - min(i % ROW_LENGTH for i in x)
    b = max(i // ROW_LENGTH for i in x) - min(i // ROW_LENGTH for i in x)
    return a * b

def quality2(x):
    return len(set(i % ROW_LENGTH for i in x)) + len(set(i // ROW_LENGTH for i in x))

def score_prob(scenarios, subset):
    """
    lower is better
    """
    d = defaultdict(int)
    for s in scenarios:
        d[len(s & subset)] += 1
    return sum(v**2 for v in d.values())

def score_worst(scenarios, subset):
    """
    lower is better
    """
    d = defaultdict(int)
    for s in scenarios:
        d[len(s & subset)] += 1
    return max(d.values())

def suggest_question(scenarios, n):
    # prob:         5.797  5.8025  5.812
    # prob + worst: 5.838  5.842
    # worst + prob: 5.891
    # worst:        6.189
    return minimal(
        zip_fun(
            partial(score_prob, scenarios),  # probabilistic
            # partial(score_worst, scenarios),  # worst case
            quality1,
            quality2,
        ),
        combs(n),
    )

def filter_scenarios(scenarios, choice, a):
    for s in scenarios:
        if len(s & choice) == a:
            yield s

def play(dice_fun, answer_fun):
    scenarios = list(combs())
    while len(scenarios) > 1:
        n = dice_fun()
        choice = suggest_question(scenarios, n)
        a = answer_fun(choice)
        scenarios = list(filter_scenarios(scenarios, choice, a))
    assert len(scenarios) == 1
    return scenarios[0]

# def user_dice():
#     return int(input("\nWürfel: "))

# def user_answer(choice):
#     return int(input(f"{display(choice)}: "))

# def main():
#     s = play(user_dice, user_answer)
#     print("\nLösung:")
#     print(display(s))

def test_murphy():
    solutions = list(combs())
    NUM_EXPERIMENTS = 1_000
    MURPHY_DICE = (2, 3, 3, 4, 4, 5)

    total = 0
    random.seed(0)
    for _ in range(NUM_EXPERIMENTS):
        turns = 0
        solution = random.choice(solutions)

        def dice():
            nonlocal turns
            turns += 1
            return random.choice(MURPHY_DICE)

        def answer(choice):
            nonlocal solution
            return len(solution & choice)

        s = play(dice, answer)
        assert solution == s
        total += turns
    print(total / NUM_EXPERIMENTS)

if __name__ == "__main__":
    # main()
    test_murphy()

PS:
codon is in the lead slightly on fedora 38. However, I can produce a variant of the code where codon performance is worse than python's on fedora as well.
$ codon build -release -exe murphy.py
$ time ./murphy
5.797

real	2m23.875s
user	2m21.629s
sys	0m0.157s
$ python
Python 3.11.5 (main, Aug 28 2023, 00:00:00) [GCC 13.2.1 20230728 (Red Hat 13.2.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>
$ time python murphy.py
5.797

real	2m29.715s
user	2m27.071s
sys	0m0.054s
$ cat /etc/redhat-release
Fedora release 38 (Thirty Eight)

I tried memorizing suggest_question. This was done manually, as the lru_cache function decorator does not stringify the arguments; e.g. TypeError: unhashable type: 'list'.
SUGGEST_DICT = dict()
def suggest_question(scenarios, n):
    # prob:         5.797  5.8025  5.812
    # prob + worst: 5.838  5.842
    # worst + prob: 5.891
    # worst:        6.189
    try:
        return SUGGEST_DICT[f"{scenarios} {n}"]
    except:
        SUGGEST_DICT[f"{scenarios} {n}"] = minimal(
            zip_fun(
                partial(score_prob, scenarios),  # probabilistic
                # partial(score_worst, scenarios),  # worst case
                quality1,
                quality2,
            ),
            combs(n),
        )
        return SUGGEST_DICT[f"{scenarios} {n}"]

This demonstration runs 6 times faster with the change.

I tried memorizing suggest_question. This was done manually, as the lru_cache decorator does not stringify the arguments; e.g. TypeError: unhashable type: 'list'.
SUGGEST_DICT = dict()
def suggest_question(scenarios, n):
    # prob:         5.797  5.8025  5.812
    # prob + worst: 5.838  5.842
    # worst + prob: 5.891
    # worst:        6.189
    try:
        return SUGGEST_DICT[f"{scenarios} {n}"]
    except KeyError:
        SUGGEST_DICT[f"{scenarios} {n}"] = minimal(
            zip_fun(
                partial(score_prob, scenarios),  # probabilistic
                # partial(score_worst, scenarios),  # worst case
                quality1,
                quality2,
            ),
            combs(n),
        )
        return SUGGEST_DICT[f"{scenarios} {n}"]

The change enables the demonstration to complete in 1/6th the time.

Thank you very much, @marioroy, that is an excellent idea. After memoization, the codon version is 31% faster than python.

@marioroy , I'm generating Fibonacci series with python and the performance results are similar.
Python native is 16X faster than codon.
~/r/p/c/try-codon ❯❯❯ time python fib.py
0 1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987

________________________________________________________
Executed in   46.95 millis    fish           external
   usr time   12.97 millis    0.07 millis   12.90 millis
   sys time   12.87 millis    3.67 millis    9.20 millis

~/r/p/c/try-codon ❯❯❯ time codon run fib.py
0 1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987

________________________________________________________
Executed in  756.86 millis    fish           external
   usr time  638.98 millis    0.08 millis  638.90 millis
   sys time   56.15 millis    3.37 millis   52.79 millis

Code :
def fib(n):
    a, b = 0, 1
    while a < n:
        print(a, end=' ')
        a, b = b, a+b
    print()

fib(1000)

@iamshreeram Compile and run with optimizations with the -release option:
time codon run -release fib.py

@iamshreeram Note that time codon run -release fib.py measures the time to compile and run the program. You should build it first and then time the execution.
Build:
codon build -release fib.py
Time:
time ./fib

Completed in #534 .


SPARKC-387: Have columntype dependent on C* Protocol Version

There is an error where the connector will attempt to use types that are not valid for the C* version which is connected to. This patch makes all Type decisions protocol version aware to fix backwards compatibility.

Build against Scala 2.11 finished with failure

Build against Scala 2.10 finished with failure

Build against Scala 2.11 finished with failure

Build against Scala 2.10 finished with failure

Build against Scala 2.11 finished with failure

Build against Scala 2.10 finished with failure

Build against Scala 2.10 finished with success

Build against Scala 2.11 finished with success

Build against Scala 2.10 finished with success

Build against Scala 2.11 finished with success

Build against Scala 2.11 finished with success

Build against Scala 2.10 finished with success

Build against Scala 2.10 finished with failure

Build against Scala 2.11 finished with failure

Build against Scala 2.10 finished with failure

Build against Scala 2.11 finished with failure

@jacek-lewandowski I think this is ready for review now, in the 2.1.15 build we should have 2 Failing tests, which I can fix in https://datastax-oss.atlassian.net/projects/SPARKC/issues/SPARKC-402 As these are not regressions. They are simply bugs in the code that previously existed

Build against Scala 2.11 finished with failure

Build against Scala 2.10 finished with failure

Build against Scala 2.10 finished with success

Build against Scala 2.11 finished with success

+1

can we merge this together with SPARKC-402 so that we have repo history without commits that do not build?

Merging with SPARKC-402


Fix extraButtons typescript define not correct

Brief description of the changes
extraButtons in typescript define not correct. Due to document, this option allow an array of ExtraButtonsOptions
What browsers and operating systems have you tested these changes on?
NA
Have you written unit tests? If not, explain why.
NA

@maivantan1992  thanks for your contribution


tap-29

Add dl4j to enable ML capabilities

@andrewresearch those stylistic changes were done with the latest push.


Bump identity carbon auth rest version

PR builder started
Link: https://github.com/wso2/product-is/actions/runs/2541586082

PR builder completed
Link: https://github.com/wso2/product-is/actions/runs/2541586082
Status: success


Error: Call to undefined function Patchwork\replace() after composer update

Hi again.
I have a weird situation here: in a plugin I defined Monkey as dev dependency like so:
  "require": {
    "php": ">=5.3.0"
  },
  "autoload": {
    "psr-4": {
      "ShortcodeReplace\\": "src/"
    }
  },
  "autoload-dev": {
    "psr-4": {
      "ShortcodeReplace\\Test\\Helper\\": "tests/Helper/"
    }
  },
  "require-dev": {
    "php": ">=5.4.0",
    "brain/monkey": "~1.2"
  }

When I installed the dependencies in the past, I got Monkey at 1.2.1 and Patchwork at 1.4.0.
Today I wanted to update the dependencies:
david@vaio:/var/www/.../wp-content/plugins/edit-post-shortcode-replace$ composer update
Loading composer repositories with package information
Updating dependencies (including require-dev)
  - Removing antecedent/patchwork (1.4.0)
  - Installing antecedent/patchwork (1.3.5)
    Loading from cache

  - Removing brain/monkey (1.2.1)
  - Installing brain/monkey (1.3.2)
    Loading from cache

Writing lock file
Generating autoload files

Now when I'm trying to run the tests, I see this:
$ phpunit
PHPUnit 4.8.23 by Sebastian Bergmann and contributors.

Fatal error: Call to undefined function Patchwork\replace() in /var/www/projects/inpsyde/mi-cockpit-ae/wp-content/plugins/edit-post-shortcode-replace/vendor/brain/monkey/src/Monkey/Functions.php on line 77

My phpunit bootstrap file is just vendor/autoload.php. The problem still persists after I cleared the composer cache.
Any ideas, what this is about?

Hm, at the moment I cannot eliminate that this is an issue of weird caching. See https://github.com/antecedent/patchwork/issues/39 I'll have a closer look on that.

I don't think it is something related to cache.
The reason why I moved Patchwork from 1.4 to 1.3 is that 1.4 raises errors when some functions is defined before Patchwork is loaded.
I think your problem are more related to the changes in #6
I want to ask:

are you relying on Brain Monkey to load Patchwork or are you loading it manually in your bootstrap?
in the first case, can you try if loading Patchwork manually solve the issue?
in the second case, can you post the relevant part of your bootstrap file?

Just noticed the problem was not limited to the latest release of Brain Monkey. I specified the version constraint to ~1.2.0 and ran composer update (because I have had to add other dependencies) and got the same issue.

are you relying on Brain Monkey to load Patchwork or are you loading it manually in your bootstrap?
I'm relying on Brain Monkey.

in the first case, can you try if loading Patchwork manually solve the issue?
The problem is solved when I do this in my bootstrap.php (as described in the readme.md):

<?php

$autoload  = $base_dir . '/vendor/autoload.php';
$patchwork = $base_dir . '/vendor/antecedent/patchwork/Patchwork.php';

# require patchwork first
if ( file_exists( $patchwork ) )
	require_once $patchwork;

if ( file_exists( $autoload ) )
	require_once $autoload;

It solves the problem for both 1.2.1 and 1.3.2 of Brain Monkey.

Wait, I withdraw! I noticed (way to late) that my tests didn't even call Brain\Monkey::setUp() neither setUpWp() nor tearDown().  This is awkward. I'm sorry!

:D no problem, glad it is solved.


Uncaught TypeError: Cannot read property 'bgColor' of undefined

Hi Allen,
I am using 2.5.1. version.There I have to toggle the row selection basis some condtion.So I am writing like below.
let selectRowProp;
if(myCondition){
  selectRowProp = {
     mode: "checkbox",
     clickToSelect: false,
     bgColor: "transparent",
     unselectable: unselectableTripExp,
     onSelect: this.onTRFExpRowSelect,
     onSelectAll : this.onTRFExpRowSelectAll
  }
} else {
  selectRowProp = {};
}
<BootstrapTable
    selectRow={selectRowProp}
    data={tripExpenses}>
    .....Table columns......
</BootstrapTable>

So initially it is in normal mode.Upon a button click I made it selection mode.But after some row selection when I am making it in non-selection mode it throwing me bgColor of undefined error.
Can you please suggest me what needs to be done to avoid this error.
Thanks in advance.
Satya

I got the solution.Instead of passing a blank object if I am passing undefined it works without any error.


Update fit profile and documentation

This pull request

Updates the fit profile to version 20.76
Updates documentation link in README.md to point to the pythonhosted documentation
Updates the Python requirements to anything > Python 2.5 since I believe this project now supports Python 3

The incorrect pip install line should update when the next release to pypi is published.

I just noticed the following test is failing:

FAIL: test_int_long (test.FitFileTestCase)
Test that ints are properly shifted and scaled
Traceback (most recent call last):
File "/Users/anthony/Development/python-fitparse/tests/test.py", line 398, in test_int_long
assert f.messages[-1].fields[1].raw_value == 1739.486328125
AssertionError

Do you think this could have been caused by updating the profile?


Code Coverage Tool

On other SEG repos, there are tools which monitor code coverage from unit tests. It would be good to implement a tool for this repo, though this should be done if there's time to do so at the end of the project.

Work for this merged into master, but wasn't closed automatically.

Work for this merged into master, but wasn't closed automatically.


builtins: add a builtin to convert the hlc.Timestamp decimal to timestamp

PR: https://github.com/cockroachdb/cockroach/pull/51662
From release notes:

Added the builtin function crdb_internal.approximate_timestamp to convert the DECIMAL returned from the crdb_internal_mvcc_timestamp system column into a TIMESTAMP. [#51662][#51662] {% comment %}doc{% endcomment %}

This is already documented.


Read / Write Concerns Problem

I am having problems getting the read after write to work consistently. I've passed j: true and w: :majority in through multiple methods:
Mongo.start_link([name: :mongo, seeds: ["mongo:27017"], database: "events", pool: DBConnection.Poolboy, w: :majority, j: true])

and
Mongo.insert_one(:mongo, "events", document, [pool: DBConnection.Poolboy, j: true, w: :majority])

Neither seem to work for me, though the documentation on this hasn't been touched in 2 years and there seems to have been a move to db_connection since then, would this affect that?

This appears to have been a problem with my replicaset


Can't start created App in debug

I've created an App and added an empty View+Viewmodel (according to usage-guide). When I try to start the app in debug i get following Output:
QML debugging is enabled. Only use this in a safe environment.
QQmlApplicationEngine failed to load component
qrc:/App.qml:2 plugin cannot be loaded for module "de.skycoder42.QtMvvm.Quick": Die Bibliothek C:\Qt\5.11.1\msvc2017_64\qml\de\skycoder42\QtMvvm\Quick\declarative_mvvmquickd.dll kann nicht geladen werden: Eine DLL-Initialisierungsroutine ist fehlgeschlagen.

Starting in Release works

Thats kind of surprising as there are no custom DLL initialization routines in that plugin. I will try and see if I can reproduce that error

I have Qt Version 5.11.1 msvc2017_64 with the QtMvvm Plugin installed using you plugin source with the Qt Maintenance Tool

Ok. So I manage to reproduce the issue. The reason seems to be that the Qt5MvvmQuick.dll is only loaded by the qml plugin and not by the application itself which leads to the error. Allthough I have no Idea why exactly thats the case. Could be a bug in Qt...
Now as a workaround, all you need to do is to reference any class from the QtMvvmQuick module in your cpp code to ensure that windows actually loads the Qt5MvvmQuick.dll when your application is started. This way the DLL is loaded before the QML plugin and the error does not appear.
Please try adding the following to your main.cpp:
QtMvvm::QuickPresenter::getInputViewFactory(); //does nothing, except referencing the DLL
QQmlApplicationEngine engine;
// ...

This really seems to be a bug in QT. See QTBUG-69963.


Support non-zero exit codes as success

Description of the new feature/enhancement
Some applications return non-zero codes that are not errors. There should be a way for the client to know the difference between these codes and actual errors.
Proposed technical implementation details (optional)
A key could be added to the manifest syntax to indicate which non-zero codes should be treated as success. The client could then treat these non-zero return codes as success.

This is the case with installers wrapped in a WinRAR self-extracting archive, for example (e.g. https://github.com/microsoft/winget-pkgs/pull/535). RARSFX will return exit code 1000 if the installation succeeded, and any other code indicates an error.

This also impacts adobe digital editions, and there are plenty more documented in installers listed on chocolatey/scoop https://chocolatey.org/packages/adobedigitaleditions#files.

https://github.com/microsoft/winget-pkgs/pull/132

fixes microsoft/winget-pkgs#132

This feature is complete with the v1.0 schema and the v0.3 client release.


Adding Private Tab

Is it possible to add option to open private tab?

It is difficult because of depending on API limitations.

Sorry to hear that, private tab is only feature  missing for me. Can you give more details about API limitations?

The Cookie API can be switched only on an application basis. In 4.1.0, I added Cookie setting in pattern match. However, It can not protect all cookie data. I think private tab should protect all privacy data, so I cannot add it.


db: document schema

This PR tries to somewhat document how the current db schema works;
and adds a few type hints.

btw, the db schema of electrs might be interesting to see for comparison:
https://github.com/romanz/electrs/blob/3c4cf729600e94f83ee3104adf797306e10f77ba/doc/schema.md


VSCode redownloads all extensions I just deleted every time I log in using remote tunnels

Does this issue occur when all extensions are disabled?: No

VS Code Version: 1.88.1
OS Version: Windows 11 Pro 23H2 (actually anything)

Steps to Reproduce:

Remove tens of extensions
Open vscode.dev in several days, open remote tunnel
Open main laptop
Vscode says "Time to settings sync boiiiii" @ downloads all my extensions back
Remove all extensions back again
Force settings sync
Open vscode.dev again, log into remote tunnel again
"Hi, how about I sync all your deleted extensions again?"  Skipped: ["gigacode.gigacode-vscode"].
2024-04-26 20:13:51.576 [info] Profiles: No changes found during synchronizing profiles.
2024-04-26 20:13:51.583 [info] Sync done. Took 137845ms

Here is what happened on ipad (i used blink app that integrates web vscode, but with vscode.dev I get absolutely similar results:
2024-04-26 20:08:18.509 [info] Using settings sync service https://vscode-sync.trafficmanager.net/
2024-04-26 20:08:18.509 [info] Auto Sync is disabled.
2024-04-26 20:10:16.651 [info] Sync started.
2024-04-26 20:10:16.938 [info] Did reset the local sync state.
2024-04-26 20:10:16.952 [info] Settings: Migrating last sync user data. Resource does not exist.
2024-04-26 20:10:16.952 [info] Settings: Last sync data state does not exist.
2024-04-26 20:10:17.067 [info] Settings: Migrating last sync user data. Resource does not exist.
2024-04-26 20:10:17.067 [info] Settings: Last sync data state does not exist.
2024-04-26 20:10:17.231 [info] Keybindings: Migrating last sync user data. Resource does not exist.
2024-04-26 20:10:17.231 [info] Keybindings: Last sync data state does not exist.
2024-04-26 20:10:17.318 [info] Snippets: Migrating last sync user data. Resource does not exist.
2024-04-26 20:10:17.318 [info] Snippets: Last sync data state does not exist.
2024-04-26 20:10:17.395 [info] Tasks: Migrating last sync user data. Resource does not exist.
2024-04-26 20:10:17.395 [info] Tasks: Last sync data state does not exist.
2024-04-26 20:10:17.496 [info] GlobalState: Migrating last sync user data. Resource does not exist.
2024-04-26 20:10:17.496 [info] GlobalState: Last sync data state does not exist.
2024-04-26 20:10:17.677 [info] Extensions: Migrating last sync user data. Resource does not exist.
2024-04-26 20:10:17.677 [info] Extensions: Last sync data state does not exist.
2024-04-26 20:10:18.910 [info] Profiles: Migrating last sync user data. Resource does not exist.
2024-04-26 20:10:18.910 [info] Profiles: Last sync data state does not exist.
2024-04-26 20:10:18.991 [info] Profiles: Migrating last sync user data. Resource does not exist.
2024-04-26 20:10:18.991 [info] Profiles: Last sync data state does not exist.
2024-04-26 20:10:23.376 [info] GlobalState: Migrating last sync user data. Resource does not exist.
2024-04-26 20:10:23.376 [info] GlobalState: Last sync data state does not exist.
2024-04-26 20:10:23.550 [info] GlobalState: Migrating last sync user data. Resource does not exist.
2024-04-26 20:10:23.550 [info] GlobalState: Last sync data state does not exist.
2024-04-26 20:10:38.519 [info] Settings: Updated local settings
2024-04-26 20:10:38.600 [info] Settings: Updated local settings
2024-04-26 20:10:38.624 [info] Settings: Migrating last sync user data. Resource does not exist.
2024-04-26 20:10:38.624 [info] Settings: Last sync data state does not exist.
2024-04-26 20:10:38.625 [info] GlobalState: Migrating last sync user data. Resource does not exist.
2024-04-26 20:10:38.625 [info] GlobalState: Last sync data state does not exist.
2024-04-26 20:10:38.872 [info] Settings: Updated remote settings
2024-04-26 20:10:38.878 [info] Settings: Updated last synchronized settings
2024-04-26 20:10:38.923 [info] Request failed {"url":"https://vscode-sync.trafficmanager.net/v1/resource/settings","status":412,"execution-id":"97074994-963a-479a-bf72-96b7ece65edf"} Precondition failed
2024-04-26 20:10:38.923 [error] o@https://main.vscode-cdn.net/stable/e170252f762678dec6ca2cc69aba1570769a5d39/out/vs/workbench/workbench.web.main.js:770:32657
n@https://main.vscode-cdn.net/stable/e170252f762678dec6ca2cc69aba1570769a5d39/out/vs/workbench/workbench.web.main.js:770:32893
@https://main.vscode-cdn.net/stable/e170252f762678dec6ca2cc69aba1570769a5d39/out/vs/workbench/workbench.web.main.js:786:135495
2024-04-26 20:10:38.923 [error] settings: POST request 'https://vscode-sync.trafficmanager.net/v1/resource/settings' failed because of Precondition Failed (412). There is new data for this resource. Make the request again with latest data.
2024-04-26 20:10:38.925 [info] Keybindings: Updated local keybindings
2024-04-26 20:10:38.927 [info] Keybindings: Updated last synchronized keybindings
2024-04-26 20:10:38.928 [info] Snippets: No changes found during synchronizing snippets.
2024-04-26 20:10:38.929 [info] Snippets: Updated last synchronized snippets
2024-04-26 20:10:38.929 [info] Tasks: Updated local tasks
2024-04-26 20:10:38.931 [info] Tasks: Updated last synchronized tasks
2024-04-26 20:10:39.015 [info] GlobalState: Updated global state ["recently.opened","github-nurumaik","microsoft-nurumaik@gmail.com","nps/lastSessionDate","nps/sessionCount","cpp.1.lastSessionDate","cpp.1.sessionCount","java.2.lastSessionDate","java.2.sessionCount","javascript.1.lastSessionDate","javascript.1.sessionCount","typescript.1.lastSessionDate","typescript.1.sessionCount","ces/skipSurvey","fileBasedRecommendations/promptedRecommendations","cpp.1.editedCount","cpp.1.editedDate","commandPalette.mru.cache","commandPalette.mru.counter","terminal.integrated.showTerminalConfigPrompt","expandSuggestionDocs","nps/isCandidate","nps/skipVersion","typescript.1.editedCount","typescript.1.editedDate","hasNotifiedOfSettingsAutosave","cpp.1.isCandidate","cpp.1.skipVersion","editor.contrib.largeFileOptimizationsWarner","workbench.statusbar.hidden","javascript.1.editedCount","javascript.1.editedDate","tabs-list-width-horizontal","workbench.view.extension.references-view.state.hidden","remote.explorerType","workbench.view.sync.state.hidden","http.linkProtectionTrustedDomains","workbench.welcomePage.hiddenCategories","fileBasedRecommendations/promptedFileExtensions","workbench.view.extension.cmake__viewContainer.state.hidden","memento/workbench.editor.keybindings","refactorPreview.hidden","typescript.1.isCandidate","typescript.1.skipVersion","workbench.views.service.panel.c61285b6-7bb1-4a11-b9bd-e3cae7bf8f12.state.hidden","workbench.view.extension.code-time-sidebar.state.hidden","workbench.view.extension.github-pull-request.state.hidden","workbench.view.extension.jupyter-variables.state.hidden","memento/notebookGettingStarted2","notebook.controller2TypeBindings","workbench.sideBar.position","workbench.activityBar.hidden","workbench.statusBar.hidden","workbench.view.extension.gitlensPanel.state.hidden","remote.tunnels.toRestore.codespaces+nurumaik-psychic-barnacle-4jqpgwxh576v.961825561","workbench.view.extension.gradleContainerView.state.hidden","remote.tunnels.toRestore.codespaces+nurumaik-yapracticum-976gqjp397qv.-384137841","javascript.1.isCandidate","javascript.1.skipVersion","workbench.auxiliarybar.pinnedPanels","workbench.views.service.auxiliarybar.243f713a-e9ed-4f85-b169-73ffcd8720b5.state.hidden","extensionTips/promptedExecutableTips","views.customizations","workbench.view.extension.jupyter.state.hidden","views.cachedViewPositions","java.2.editedCount","java.2.editedDate","remoteTunnelServicePromptedPreview","remoteTunnelServiceUsed","workbench.view.extension.outlook-unofficial.state.hidden","remote.tunnels.toRestore.tunnel+retardedgames.892394792","remote.tunnels.toRestore.tunnel+retardedgames.951156358","remote.tunnels.toRestore.tunnel+retardedgames.499270414","remote.tunnels.toRestore.ssh-remote+retarded.games.-285647964","remote.tunnels.toRestore.tunnel+retardedgames.-285647964","remote.tunnels.toRestore.tunnel+retardedgames.1115380343","csharp.1.lastSessionDate","csharp.1.sessionCount","workbench.view.extension.PowerShell.state.hidden","extensionsAssistant/deprecated","workbench.view.extension.cmake-view.state.hidden","snippets.usageTimestamps","workbench.view.extension.vscode-edge-devtools-view.state.hidden","remote.tunnels.toRestore.wsl+Ubuntu.-111354861","workbench.panel.testResults.state.hidden","csharp.1.editedCount","csharp.1.editedDate","workbench.view.extension.gigacode-sidebar-view.state.hidden","workbench.view.extension.gitlab-workflow.state.hidden","remote.tunnels.toRestore.wsl+Ubuntu-22.04.-2076757131","remote.tunnels.toRestore.wsl+ubuntu-22.04.-2076757131","remote.tunnels.toRestoreExpiration.wsl+Ubuntu-22.04.-2076757131","remote.tunnels.toRestoreExpiration.wsl+ubuntu-22.04.-2076757131","remote.tunnels.toRestore.wsl+Ubuntu-22.04.-111354861","remote.tunnels.toRestoreExpiration.wsl+Ubuntu-22.04.-111354861","workbench.panel.output.hidden","workbench.explorer.views.state.hidden","workbench.scm.views.state.hidden","workbench.activity.pinnedViewlets2","workbench.panel.pinnedPanels","workbench.view.debug.state.hidden","workbench.view.remote.state.hidden","workbench.welcomePage.walkthroughMetadata","colorThemeData","workbench.panel.alignment","workbench.view.extensions.state.hidden","memento/gettingStartedService","workbench.view.extension.test.state.hidden","workbench.view.extension.liveshare.state.hidden","workbench.view.extension.github-pull-requests.state.hidden","workbench.view.extension.gitlens.state.hidden","workbench.view.extension.package-explorer.state.hidden","workbench.view.extension.gitlensInspect.state.hidden","workbench.view.extension.yandex-music-explorer.state.hidden"]
2024-04-26 20:10:39.016 [info] GlobalState: Updated local ui state
2024-04-26 20:10:39.028 [info] GlobalState: Updated last synchronized ui state
2024-04-26 20:10:40.379 [info] Extensions: Skipped synchronizing extension because the extension is not found. gigacode.gigacode-vscode
2024-04-26 20:10:41.417 [info] Extensions: Disabled extension fredericbonnet.cmake-test-adapter 0.17.3
2024-04-26 20:10:41.418 [info] Extensions: Disabled extension twxs.cmake 0.0.17
2024-04-26 20:10:41.445 [info] Extensions: Disabled extension llvm-vs-code-extensions.vscode-clangd 0.1.28
2024-04-26 20:10:41.482 [info] Extensions: Skipped synchronizing extension because it cannot be installed. Remote - SSH: Editing Configuration Files
2024-04-26 20:10:41.507 [info] Extensions: Migrating last sync user data. Resource does not exist.
2024-04-26 20:10:41.507 [info] Extensions: Last sync data state does not exist.
2024-04-26 20:10:41.753 [info] Extensions: Skipped synchronizing extension because it cannot be installed. Dev Containers
2024-04-26 20:10:41.825 [info] Extensions: Skipped synchronizing extension because it cannot be installed. Remote - SSH
2024-04-26 20:10:44.034 [info] Extensions: Migrating last sync user data. Resource does not exist.
2024-04-26 20:10:44.034 [info] Extensions: Last sync data state does not exist.
2024-04-26 20:10:59.800 [info] Extensions: Installed extension. ms-vscode-remote.vscode-remote-extensionpack 0.25.0
2024-04-26 20:10:59.800 [info] Extensions: Installed extension. vscodevim.vim 1.27.2
2024-04-26 20:10:59.800 [info] Extensions: Installed extension. ms-vscode-remote.remote-wsl 0.88.0
2024-04-26 20:10:59.800 [info] Extensions: Installed extension. yifei-zhang.cmake-test-tools 0.2.6
2024-04-26 20:10:59.800 [info] Extensions: Installed extension. gitlab.gitlab-workflow 4.9.0
2024-04-26 20:10:59.800 [info] Extensions: Installed extension. ms-vscode.cpptools 1.19.9
2024-04-26 20:10:59.818 [info] Extensions: Migrating last sync user data. Resource does not exist.
2024-04-26 20:10:59.818 [info] Extensions: Last sync data state does not exist.
2024-04-26 20:11:00.347 [info] Extensions: Updated remote extensions. Added: ["github.codespaces","acherkashin.yandex-music-extension","ahmadawais.shades-of-purple","ajshortt.tokyo-hack","alefragnani.bookmarks","arcticicestudio.nord-visual-studio-code","asvetliakov.vscode-neovim","atlassian.atlascode","batisteo.vscode-django","bazelbuild.vscode-bazel","bierner.jsdoc-markdown-highlighting","britany.dark-cyberpunk-aesthetic","clerian.cyberdyne20xx","codestream.codestream","continue.continue","cquery-project.cquery","cschlosser.doxdocgen","davidanson.vscode-markdownlint","deepred.daily-anime","donjayamanne.python-environment-manager","donjayamanne.python-extension-pack","dtsvet.vscode-wasm","eamodio.gitlens","eckertalex.borealis","enkia.tokyo-night","equinusocio.vsc-material-theme","equinusocio.vsc-material-theme-icons","esbenp.prettier-vscode","fisheva.eva-theme","gaborv.flatbuffers","github.vscode-pull-request-github","golang.go","gruntfuggly.calendar","hediet.vscode-drawio","hololove.hololive-en","hoovercj.vscode-power-mode","johnpapa.vscode-peacock","johnpapa.winteriscoming","josetr.cmake-language-support-vscode","jprestidge.theme-material-theme","juanblanco.solidity","kevinrose.vsc-python-indent","leonardssh.vscord","lkytal.pomodoro","lucy.vscode-pink","m417z.regedit","maattdd.gitless","maelvalais.autoconf","malcolmmielle.semantic-highlighting","mark-wiemer.vscode-autohotkey-plus-plus","matepek.vscode-catch2-test-adapter","max-ss.cyberpunk","mechatroner.rainbow-csv","medo64.render-crlf","ms-ceintl.vscode-language-pack-ko","ms-dotnettools.vscode-dotnet-runtime","ms-python.isort","ms-toolsai.jupyter","ms-toolsai.jupyter-keymap","ms-toolsai.jupyter-renderers","ms-toolsai.vscode-jupyter-cell-tags","ms-toolsai.vscode-jupyter-slideshow","ms-vscode.azure-account","ms-vscode.hexeditor","ms-vscode.js-debug-nightly","ms-vscode.live-server","ms-vscode.makefile-tools","ms-vscode.powershell","ms-vscode.vscode-js-profile-flame","ms-vscode.vscode-typescript-next","ms-vsliveshare.vsliveshare","naco-siren.gradle-language","njpwerner.autodocstring","nopjmp.fairyfloss","omkov.vscode-ebnf","platformio.platformio-ide","quicktype.quicktype","redhat.java","redhat.vscode-xml","redhat.vscode-yaml","robbowen.synthwave-vscode","rust-lang.rust-analyzer","samrapdev.outrun","samwillis.python-inline-source","sdras.night-owl","shyykoserhiy.vscode-spotify","slevesque.shader","stackbuild.bazel-stack-vscode","stackbuild.bazel-stack-vscode-cc","tamasfe.even-better-toml","teabyii.ayu","thenikso.github-plus-theme","tkrkt.linenote","tobermory.es6-string-html","tyriar.sort-lines","uloco.theme-bluloco-dark","ustymukhman.matrix-theme","vadimcn.vscode-lldb","vallentin.vscode-bnf","visualstudioexptteam.intellicode-api-usage-examples","visualstudioexptteam.vscodeintellicode","vscjava.vscode-java-debug","vscjava.vscode-java-dependency","vscjava.vscode-java-pack","vscjava.vscode-java-test","vscjava.vscode-maven","webfreak.cute-theme","webrender.synthwave-x-fluoromachine","wholroyd.jinja","xynny.hackerred","yummygum.city-lights-theme","zainchen.json","zenghongtu.vscode-asciiflow2","zxh404.vscode-proto3"]. Updated: ["ms-vscode.cpptools"].
2024-04-26 20:11:00.351 [info] Extensions: Updated last synchronized extensions. Skipped: ["gigacode.gigacode-vscode","ms-vscode-remote.remote-ssh-edit","ms-vscode-remote.remote-containers","ms-vscode-remote.remote-ssh"].
2024-04-26 20:11:00.353 [info] Profiles: No changes found during synchronizing profiles.
2024-04-26 20:11:00.354 [info] Profiles: Updated last synchronized profiles.
2024-04-26 20:11:00.363 [info] Sync done. Took 43712ms
2024-04-26 20:11:00.368 [info] Auto Sync: Triggered by Interval
2024-04-26 20:11:00.369 [info] Sync started.
2024-04-26 20:11:00.543 [info] Settings: No changes found during synchronizing settings.
2024-04-26 20:11:00.545 [info] Keybindings: No changes found during synchronizing keybindings.
2024-04-26 20:11:00.546 [info] Snippets: No changes found during synchronizing snippets.
2024-04-26 20:11:00.547 [info] Tasks: No changes found during synchronizing tasks.
2024-04-26 20:11:01.381 [info] GlobalState: Updated remote ui state. Updated: workbench.explorer.views.state.hidden,workbench.activity.pinnedViewlets2,workbench.panel.pinnedPanels,workbench.welcomePage.walkthroughMetadata,memento/gettingStartedService.
2024-04-26 20:11:01.453 [info] GlobalState: Updated last synchronized ui state
2024-04-26 20:11:03.033 [info] Extensions: Updated remote extensions. Updated: ["ms-vscode.cpptools"].
2024-04-26 20:11:03.042 [info] Extensions: Updated last synchronized extensions. Skipped: ["gigacode.gigacode-vscode","ms-vscode-remote.remote-ssh-edit","ms-vscode-remote.remote-containers","ms-vscode-remote.remote-ssh"].
2024-04-26 20:11:03.101 [info] Profiles: No changes found during synchronizing profiles.
2024-04-26 20:11:03.102 [info] Sync done. Took 2733ms
2024-04-26 20:11:03.447 [info] Auto Sync: Triggered by Activity
2024-04-26 20:11:03.447 [info] Sync started.
2024-04-26 20:11:03.552 [info] Settings: No changes found during synchronizing settings.
2024-04-26 20:11:03.556 [info] Keybindings: No changes found during synchronizing keybindings.
2024-04-26 20:11:03.557 [info] Snippets: No changes found during synchronizing snippets.
2024-04-26 20:11:03.559 [info] Tasks: No changes found during synchronizing tasks.
2024-04-26 20:11:03.564 [info] GlobalState: No changes found during synchronizing ui state.
2024-04-26 20:11:04.420 [info] Extensions: No changes found during synchronizing extensions.
2024-04-26 20:11:04.421 [info] Profiles: No changes found during synchronizing profiles.
2024-04-26 20:11:04.422 [info] Sync done. Took 975ms
2024-04-26 20:11:56.169 [info] Auto Sync: Triggered by Activity
2024-04-26 20:11:56.169 [info] Sync started.
2024-04-26 20:11:56.732 [info] Settings: No changes found during synchronizing settings.
2024-04-26 20:11:56.738 [info] Keybindings: No changes found during synchronizing keybindings.
2024-04-26 20:11:56.739 [info] Snippets: No changes found during synchronizing snippets.
2024-04-26 20:11:56.740 [info] Tasks: No changes found during synchronizing tasks.
2024-04-26 20:11:56.744 [info] GlobalState: No changes found during synchronizing ui state.
2024-04-26 20:12:01.546 [info] Extensions: No changes found during synchronizing extensions.
2024-04-26 20:12:01.548 [info] Profiles: No changes found during synchronizing profiles.
2024-04-26 20:12:01.549 [info] Sync done. Took 5380ms

Can you please share logs during following steps?

Force settings sync

Share the settings sync log after removing the extensions and forcing sync

Open vscode.dev again, log into remote tunnel again

Share the settings sync log when you think vscode.dev is pushing these removed extensions back to cloud

On laptop, extensions are downloaded again

Share the log at this moment - I believe you have already shared here. Sharing it again might give us may be new info.
It will help us understand the root cause. Thanks

This issue has been closed automatically because it needs more information and has not had recent activity. See also our issue reporting guidelines.
Happy Coding!


Fixes planning scene monitor (message_filter)

The message_filter has been changed to a ros subscriber here.  Also there is no similar implementation for registerFailureCallback() on ROS2 message_filter

@ahcorde, can you have a look as well?


Accessing memory of WASM module

I'm trying to implement logging into my Wasm module. However, I've no idea how to actually get access to the memory of the module. For example, if I have an export:
extern {
    fn log_error(text: *mut c_str);
}

And I call it using CString, the host is going to get a memory offset. Is there a function to get direct access to the memory of the module? (I couldn't find it so that's why I'm asking.) Also, how would I determine the length of the string?

You may do the following:
WASM:
extern {
    pub fn log_impl(msg: *const u8, msg_len: u32);
}

fn log(msg: &str) {
    unsafe {
        log_impl(msg.as_ptr(), msg.len() as u32)
    }
}

log("Hello from Wasm");

Runtime:
pub fn read_wasm_string(offset: u32, length: u32, wasm_mem: &[u8]) -> &str {
    core::str::from_utf8(&wasm_mem[offset as usize..offset as usize + length as usize]).expect("read_wasm_cstring failed to parse invalid utf-8 string")
}

impl Externals for Runtime {
    fn invoke_index(
        &mut self,
        index: usize,
        args: RuntimeArgs,
    ) -> Result<Option<RuntimeValue>, Trap> {
    let direct_mem = self.memory.direct_access();
    let wasm_direct_mem = direct_mem.as_ref();

    match index {
         FUNCIDX_LOG => {
                let addr: u32 = args.nth_checked(0)?;
                let length: u32 = args.nth_checked(1)?;

                let string = read_wasm_string(addr, length, &wasm_direct_mem);
                (self.log_routine)(string);
                Ok(None)
            },
    }
}

where memory is an export by name as per wasm modules usually have:
let memory_export = main.export_by_name("memory").ok_or(wasmi::Error::Validation("memory export missing".to_string()))?;
let memory = memory_export.as_memory().ok_or(wasmi::Error::Validation("memory export not a memory type".to_string()))?;

Ah, okay. Thank you!

Runtime seems to not exist, is it gone in newer versions? Is this still possible in newer versions, and how?

Not sure I understand your question but wasmi API has changed significantly since the creation of this issue.
We are now mirroring the well established and battle tested Wasmtime API.
If you want to access the Memory of an instantiated Wasm module you can use this API: https://docs.rs/wasmi/0.29.0/wasmi/struct.Instance.html#method.get_memory


Fix AstExpressionBuilder.java for backport 1456 to 2.x

Description
Fix build for https://github.com/opensearch-project/sql/pull/1508
Issues Resolved
https://github.com/opensearch-project/sql/pull/1508
Check List

[ ] New functionality includes testing.

[ ] All tests pass, including unit test, integration test and doctest

[ ] New functionality has been documented.

[ ] New functionality has javadoc added
[ ] New functionality has user manual doc added

[ ] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Codecov Report

Merging #1515 (9e733f5) into backport/backport-1456-to-2.x (171b5c9) will increase coverage by 0.07%.
The diff coverage is n/a.

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
@@                         Coverage Diff                         @@
##             backport/backport-1456-to-2.x    #1515      +/-   ##
===================================================================
+ Coverage                            98.41%   98.49%   +0.07%
- Complexity                            3727     3926     +199
===================================================================
  Files                                  335      347      +12
  Lines                                 9281     9768     +487
  Branches                               599      645      +46
===================================================================
+ Hits                                  9134     9621     +487
  Misses                                 142      142
  Partials                                 5        5

Flag
Coverage Δ

sql-engine
98.49% <ø> (+0.07%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

...pensearch/sql/sql/parser/AstExpressionBuilder.java
100.00% <ø> (ø)

... and 11 files with indirect coverage changes
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


Enhancement idea: intermittently publish the fish data in accessible format(s)

Rather than having others re-use your scraping code to gather salmon data,
maybe publish a .csv or other convenient format in your repo or a new one in the salish-sea organization,
so that others can just ingest directly?

I just noticed that the new paper by Stewart et al., 2023, used siloed data and didn't provide it for reproducibility, but did include a distillation of Albion data in tabular form. Perhaps it's of interest (e.g. to error-check your scraping script?), but mostly I wanted to highlight it as a non-optimal way of addressing this issue. (Copy/pasting a table from a PDF is annoying when you could have provided an actual data table for programatic convenience as a digital supplement.

@zoe I know you've closed this issue, but I just saw in the Ettinger paper that they published their Albion data in a SRKW phenology Github repo...

...but I don't see the Orcamaster + Lime Kiln longitudinal presence/absence data for SRKWs that they said would be there...

Am I missing something @AileneKane ?
At any rate, Zoe, I continue to take notes on possible additional sources of salmon data in this Acartia Project card related to SRKW-salmon integrations, including the WDFW escapement data that Ailene scraped and provided as a .csv file it seems.

Thank you for pointing this out- I will take a look and add the data file. I was waiting to recieve the appropriate permissions from orcamaater.Sent from my iPhoneOn Apr 13, 2023, at 1:15 PM, Zoe Liu @.***> wrote:
I took a quick look at orcaphen_esr2022.R in Ailene's repo, and it seems that Lime Kilm subset data was read from data/limedat.csv. The file is not currently included in the repo. My guess is that there may be some limitations to where the Orca Master data can be released... Ailene would probably be able to answer this better.
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: @.***>

Thanks, @AileneKane ! And no rush...
We appreciate your excellent publication and use of Github to help lead folks towards more reproducible science.
@liu-zoe you right about Bob Otis longitudinal dataset from Lime Kiln.
The SRKW occurrence data are referenced in
fitsrkwmods_regional.R and look like:
j_dat_assumeSRKW.csv
k_dat_assumeSRKW.csv
l_dat_assumeSRKW.csv

but the methods section suggests there was a processing to obtain this "subset of Orca Master" so the entire procedure may not be reproducible?


Improve flexibility of configuration via env. vars

Currently it's possible to configure searchd via env. vars, but it's limited by only those settings that are hardcoded in manticore.conf, so some new and undocumented settings are not working, e.g. -e searchd_query_log_commands=1. The task is to refactor the feature, so it's possible to pass arbitrary settings via env. vars.

Done in https://github.com/manticoresoftware/docker/pull/56

Reopening since the CI failed https://github.com/manticoresoftware/docker/actions/runs/6947961151/job/18902901514

I just rerun job and all fine now. It happens sometimes. Maybe because cronjob complete at the same second as container runs

It's not working for me:
snikolaev@dev2:~$ docker run  -e searchd_listen='9307:mysql' -e EXTRA=1 --name manticore --rm -d manticoresearch/manticore:dev && echo "Waiting for Manticore docker to start. Consider mapping the data_dir to make it start faster next time" && until docker logs manticore 2>&1 | grep -q "accepting connections"; do sleep 1; echo -n .; done && echo started && docker exec -it manticore bash -c 'cat /etc/manticoresearch/manticore.conf.debug' && docker stop manticore
96cd544291948eb0abf031f5b18e8c84d626f792e819957079d15b864f701566
Waiting for Manticore docker to start. Consider mapping the data_dir to make it start faster next time
...started
searchd {
    listen = 9306:mysql41
    listen = /var/run/mysqld/mysqld.sock:mysql41
    listen = 9308:http
    listen = <IP_ADDRESS>:9312
    listen = <IP_ADDRESS>:9315-9325:replication
    log = /var/log/manticore/searchd.log
    query_log = /var/log/manticore/query.log
    pid_file = /var/run/manticore/searchd.pid
    data_dir = /var/lib/manticore
}
manticore

Done in https://github.com/manticoresoftware/docker/commit/5d426e5ea4c54a13ee78aafd1a374554a48821c2

seems this fix is cause of the inconsistency in the env option of the docker \ docker-compose file that cause the issue  https://github.com/manticoresoftware/manticoresearch/issues/2500
ie the docker-compose.yaml that allows to issue CLUSTER CREATE well with 6.2.12 release
    environment:
      - EXTRA=1
      - searchd_listen=9306:mysql41
    cap_add:
      - IPC_LOCK

in the 6.3.6 release fails with the error message
mysql> create cluster xxx;
ERROR 1064 (42000): cluster 'xxx' is not ready, starting

that is due to searchd_listen now overrides all default port settings and daemon starts as
starting daemon version '6.3.6 593045790@24080214 (columnar 2.3.0 88a01c3@24052206) (secondary 2.3.0 88a01c3@24052206) (knn 2.3.0 88a01c3@24052206)' ...
listening on all interfaces for mysql, port=9307
prereading 0 tables

however at the 6.2.12 release searchd_listen adds new port mappings to the default port settings and daemon starts as
listening on all interfaces for mysql, port=9307
listening on all interfaces for mysql, port=9306
listening on UNIX socket /var/run/mysqld/mysqld.sock
listening on <IP_ADDRESS>:9312 for sphinx and http(s)
listening on all interfaces for sphinx and http(s), port=9308

not sure is this a bag that should be fixed or planned change that just lack of the description in the changelog as now the changelog looks like
:exclamation:Issue #47 Improved flexibility of configuration via environment vars.

not sure is this a bug that should be fixed or planned change that just lack of the description in the changelog as now the changelog looks like

The readme was improved in https://github.com/manticoresoftware/docker/commit/3ebe4bffeab6832c4de412899e8c3b5c500114a7
Closing this issue.


Allow undef parameter passing with nil

Now, when nil is used as a param value, its inspect method is not
called to construct the Puppet parameter value. Instead, the unquoted
string 'undef' is used in the generated resource declaration. So now
let(:params) { { :foo => nil } } leads to the desired resource
declaration class { 'test': foo => undef }.
Puppet undef is nearly equivalent to Ruby nil, and there is no nil
in Puppet, so it makes sense to do this translation.
Before this change, it was impossible to pass undef as a parameter.
Using nil, :undef, or 'undef' all failed for various reasons.
Credit to the discussion at [1] for identifying the issue and proposing
an alternate solution.
Fixes #279.
[1] https://groups.google.com/forum/#!topic/puppet-users/6nL2eROH8is

interestingly, it appears that your tests are failing.

Thanks @igalic . My tests are failing because I named my test class undef, which is an invalid identifier name. I renamed it locally, and the tests pass. I just haven't pushed the fix yet.
Though I'm questioning whether this is the correct approach. I was looking into the code for the generic matchers, and I saw that nil has a different meaning there. When doing any of .with_foo, .with_foo(nil), .with({:foo=>nil}), .with([:foo,:bar]), the expected value of :foo is taken to be nil, but this doesn't mean undef, but means any value. Similarly, when doing any of `.without_foo, .without_foo(nil), .without({:foo=>nil}), .without([:foo,:bar]), the unexpected value of :foois taken to benil, but this doesn't mean not undef``, but means not any value.
Since not passing a parameter is not always semantically equivalent to passing an undef value, we can't just use .without_foo to test for undef being passed as the value for :foo.
So it might make sense to go with the suggestion in https://groups.google.com/forum/#!topic/puppet-users/6nL2eROH8is, and introduce a special Undef class, which can be used both with let(:params) and with .with and .without.
@igalic, what do you think?

I like the idea with a special Undef class. @jmoldow are you still interested in getting this cleaned up and merged?

This was merged as #335 . Thanks to all involved for getting a good solution in!

I think this PR can be closed now.


Devcon

This adds support for validation against multiple public keys for a single conference ID

@micwallace it looks good, but would be nice with a negative test validating that keys from "wrong" conferences cannot be used to verify. I.e. that a key for Devcon 6 cannot be used to validate a signature on a ticket issued for Devcon 5.

Note that the reason the tests fail is not because of the change, but because the smart contract integration tests fails due to obsolescence of Rinkeby


upddated validations.py

Fixes #<issue_number_goes_here>

It's a good idea to open an issue first for discussion.

[ ] Tests pass
[ ] Appropriate changes to README are included in PR

xzczc


pull request for e2e test at 12/19/2016 14:45:19

:white_check_mark: Validation status: passed
For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


[MRG] Disable interactive annotations when orig_time is not None.

Noticed that the drawn annotations don't align right when orig_time is not None. The fix gets quite involved, so I don't think I can get it working for 0.14.

thx @jaeilepp


Add MCVE, MRE, ReprEx

Hi,
Let's add this three values:

MCVE: Minimal Complete Valid Example
MRE: Minimal Reproducible Example
ReprEx: Reproducible Example

https://stackoverflow.com/help/minimal-reproducible-example
https://reprex.tidyverse.org/articles/learn-reprex.html

Hi @aloisdg. I'd like to take this up.

@shahx95 sure let's go :)


Uncaught error in dropInstance onblocked handler

If I try to call dropInstance from one tab while the instance is in use by another tab, the onblocked event is triggered.
                    req.onerror = req.onblocked = function (e) {
                        var db = req.result;
                        if (db) {
                            db.close();
                        }
                        reject(e);
                    };

The handler attempts to close the database before the promise rejects, however the attempt to access request.result throws an error Uncaught DOMException: Failed to read the 'result' property from 'IDBRequest': The request has not finished. at IDBOpenDBRequest.req.onerror.req.onblocked.
It seems either the attempt to close the database should not happen here (if blocked implies something else is currently using the database, I would think you wouldn't want this to close it anyway) or the readyState should be checked first. Not sure if the same problem occurs with the onerror case.

Thanks for the report and the extra digging that you did :+1: I would be more than happy to review a PR with your suggestions, even though tested might be difficult to right for such case.

After rethinking about it, if one tab tries to save to a store and an other one tries to drop that store and gets blocked, we might want to abort the write and unblock the drop. The issue still needs investigation though.

Not sure about the 'correct' behavior, maybe it's situational, but in my case I think I would want to leave it open and reject the drop request if the db is in use. Is there a way to check that the db is not in use before calling dropInstance?
I noticed the docs for readyState mention "Every request starts in the pending state. The state changes to done when the request completes successfully or when an error occurs." So it sounds like the onerror case should work as-is and only the onblocked case would have this problem. For that I would propose to just reject without attempting to close the db. I can make a PR if you want to go with that, otherwise I should probably leave it to someone else to look into implementing the more complex behavior as I'm not very familiar with the indexedDB API.
One other thing I noticed, the e argument of the handler is actually an Event rather than an Error object, so reject warns that it should only be called with an Error. In the case of onerror the error should be accessible from req.error, but for the blocked handler there is no available Error object as far as I can tell.

I made a PR with this change.

@thgreasi Just checking back, any further thoughts on this?

Oh sorry I was blocked on other things all this time.
Will try to check this weekend.

I updated the PR based on what I found in #821.

Do we know anything new about this? Something similar it's happening to me and I think it's gonna get solved with the PR.

Sorry, I sort of gave up on this after continuing to run into errors. Some additional changes may be necessary to accommodate this situation correctly. I tried a couple more attempted fixes, but I think I was still getting errors and at that point I opted to just avoid using dropInstance.
If you're hitting this problem you could try this change and see if it works for you, maybe the errors I hit were just due to other problems with my design.

Hi!
Been looking for an answer to this for months, but I finally rolled up my sleeves and tried to overcome this issue.
I had trouble with a site being open in several tabs, and if one tab used dropInstance() while another tab read or wrote to the database, the database crashed. I couldn't read, write, dropInstance or createInstance. No error messages. Nothing. Only solution was to delete the database in the console.
So I wrote this method:
    function _waitForDatabaseBeingDropped(localforagePromise) {
      return localforage.ready()
        .then(function() {
          return localforagePromise
        });
    },

   Example of Usage
   _waitForDatabaseBeingDropped( localforage.getItem('itemKey') )

No more database crashes.

I confirm that the strange behavior even in a single tab mode.

Uncaught DOMException: Failed to read the 'result' property from 'IDBRequest': The request has not finished. at IDBOpenDBRequest.req.onerror.req.onblocked.
I have them same error also :(

@trongquyvn I'm getting the same error, it is annoying.
@Rickard80 how I'm supposed to pass in params the localforagePromise? What should be this promise the function expects ?

+1
I'm getting the same error. If it helps, happens when calling dropInstance({name:'dbname'})

Is there any solution to handle this issue of blockage when the request for dropping the instance is immediately followed by another request for creating back a new instance having same name as well as storenames? If so can someone provide the solution for the same as we would like to implement IndexedDB using this library


draw_bounding_boxes and save_image  require different input dtype

Hi, when using draw_bounding boxes an image torch.tensor needs to be converted to uint8 dtype because the function expects that dtype. When inserting the image with drawn boxes to save_image, it needs to be converted back to float32 dtype. Can this be changed to easily insert the images to both functions without type conversion?

I think I have no capacity to do that.

Hey @NicolasHug
Can you please re-open this issue? I have lined up a new PR #8276
Thank you!


[W6.2][W13-B2] Ronak Lakhotia

I see you completed all LOs of week 6. Keep up the spirit.

Hi Ronak Lakhotia, I see that you know how to implement polymorphism and JavaFX. As a mandatory LO, I suggest you implement interface, as well as minor yet helpful LOs: abstract, NFT, User Stories and Use Cases.


Changing Parameters

Hello,
First of all, thanks for sharing this tool!
Amazing work!
I am just doing some tests, and it seems that it's not ready for CHANGING parameters?
I dont see any columns for the CHANGING params in ZAFL_LOG and when trying to re-execute an FM with CHANGING param we have a dump.
Thanks again for sharing

just saw #13 ..

closed by #16

Firstly I think the changing is not necessary. But yes you are right, if it is not a optional parameter, the missing parameter will cause a dump. So it must be included.


Fares need to be created with timezone aware start and end dates

https://docs.djangoproject.com/en/1.11/ref/utils/#django.utils.timezone.make_aware

I'm not sure if a date (Fare.start/end) can have a timezone. But we might look into checking how the date that's compared with those dates is generated (currently  iirc it's timezone.now())


Docling models path

#335

Changed how my unit tests mock to avoid a Python 3.10 vs 3.11 incompatibility with patching certain modules. This should get the tests green on the Python 3.10 CI again.

Thanks @bbrowning for adding the quick unit tests 💯

@Mergifyio backport release-v0.5


Training with my dataset using cityscapes pretrained weights.

Hi,
I would like to train ESPNet model with my dataset from Cityscapes pretrained weights that you uploaded on /pretrained/encoder/ folder, not from scratch.
Could you please give some advice how to do this?
Thanks.

You can load the pretrained model and replace the classification layer with the randomly initialized one. Then you should be good to train on your dataet.
Let me know if you have any specific questions.

I can figure it out now thanks.
Do you also have pre-trained weights on the ImageNet?
I would like to try training the network with ImageNet pretrained weights data but I am not sure you have this one. (like you shared it on ESPNetv2)
Thank you.

No, we didn’t train ESPNet on the ImageNet dataset.

Thanks.


kom

This function don't work:
console-command-on-entry:
My worldguard file:
        flags:             console-command-on-entry: ['/toast %username% -t:goal -icon:spyglass &2you enter vip area'], the command is not activating when entering an region it broken on the version 4.2.1, works fine on 4.2.0

The bypass wasn't properly checked in 4.2.0 which was fixed in 4.2.1.

But I am running as console, so it should bypass permissions etc.

You have /rg bypass turned on, which means the flags aren't applied to you.

Ahh I see my bad. Thanks for fast help


add metric for dead_letter_queue

Hello,
first of all, thank you for your project!
The logstash node stats api has an additional metric - dead_letter_queue:
curl -s localhost:9600/_node/stats | jq '. as $in | reduce leaf_paths as $path ({}; . + { ($path | map(tostring) | join(".")): $in | getpath($path) })' | grep dead_letter
  "pipelines.main.dead_letter_queue.queue_size_in_bytes": 15,

Currently this metric is not parsed, but it would be good to monitor it.
This issue mentioned in: #14
This PR adds support for this metric.

Hello. Thank you for PR. I pushed to master squashed version of your patch as 9afc3b65f3a368b5bd85d756e02fa743bf1760f6


Update CSP to latest version

This checklist is used to make sure that common guidelines for a pull request are followed.

For new extensions:

[ ] My extension description/summary conforms to the Extension Summary Guidelines.

About Extension Publish
There is a pipeline to automatically build, upload and publish extension wheels.
Once your PR is merged into master branch, a new PR will be created to update src/index.json automatically.
The precondition is to put your code inside this repo and upgrade the version in the PR but do not modify src/index.json.

connectedk8s

Hi sirireddy12,
Do you think it is necessary to rerun live test for bump version ？

Hi sirireddy12, Do you think it is necessary to rerun live test for bump version ？

H

Hi sirireddy12, Do you think it is necessary to rerun live test for bump version ？

Hi @wangzelin007 , the test cases to test this bump version are different and not included in livetest. I have done the testing manually on my side. We have a tracking item to include more test cases in livetest. Having said that, these test cases include testing our extension on CLI core version <2.30 as well. It is yet to be figured out how to include those cases in livetests sicne azdev framework uses only latest CLI core version


fix error of exercise 13: min -> Float.min

A DCO is needed to submit! Can you fix that?

@yminsky fixed~


[C#] ArrowArrayConcatenator doesn't handle non-zero offsets

Describe the bug, including details regarding any error messages, version, and platform.
Example unit test to reproduce:
[Fact]
public void TestConcatenateSlicedArrays()
{
    Int32Array array0 = new Int32Array.Builder()
        .Append(new []{0, 1, 2})
        .AppendNull()
        .Append(new []{3, 4})
        .Build();
    Int32Array array1 = new Int32Array.Builder()
        .Append(new []{5, 6})
        .AppendNull()
        .Append(new [] {7, 8, 9})
        .Build();

    var concatenated = ArrowArrayConcatenator.Concatenate(new[]
    {
        ArrowArrayFactory.Slice(array0, 2, 3),
        ArrowArrayFactory.Slice(array1, 2, 3),
    });

    var expected = new Int32Array.Builder()
        .Append(2).AppendNull().Append(3)
        .AppendNull().Append(new[] { 7, 8 })
        .Build();

    Assert.Equal(expected, (IReadOnlyList<int?>)concatenated);
}

This fails with:
Assert.Equal() Failure: Collections differ
           ↓ (pos 0)
Expected: [2, null, 3, null, 7, ···]
Actual:   [0, 1, 2, 5, 6, ···]
           ↑ (pos 0)

Component(s)
C#

Issue resolved by pull request 41245
https://github.com/apache/arrow/pull/41245

Can this be backported to a bugfix 16.x release?

The Arrow release process is fairly labor-intensive, so bugfix releases are typically reserved for regressions. This fix is for a longstanding bug and not a regression, so on its own it wouldn't be a candidate for a point release. If it turns out that we need a 16.2 release for some other reason, I'll investigate the possibility of porting this set of fixes alongside it.
The next regular release will be roughly at the beginning of August.


AR/MR mode doesn't work on PICO headsets / Make Hit Test API optional

Current behavior:
AR/MR mode doesn't work on PICO headsets (https://picoxr.com).
Tested in the PICO 4 headset on the latest version of PICO Browser and the latest version of firmware version.
Removing 'hit-test' from requiredFeatures or moving it into optionalFeatures in the
await navigator.xr!.requestSession!('immersive-ar', {
  requiredFeatures: ['hit-test'],
  optionalFeatures: ['dom-overlay', 'light-estimation'],
  domOverlay: this.overlay ? {root: this.overlay} : undefined
});

in the /packages/model-viewer/src/three-components/ARRenderer.ts file generally fixes the problem.
The 3D model can be successfully viewed in AR/MR mode.
The only thing is that dragging, scaling and rotating 3D models using motion controllers or hands, recently added by #4643, doesn't work.
I'm pretty sure it's because of unhandled errors (that can be seen in the browser console) related to the Hit Test API not actually being supported, but being called.
Expected behavior:
AR/MR mode does work on PICO headsets.
Hit Test API is optional, not required feature.
The code uses progressive enhancement, i.e. it checks if the browser supports specific Web APIs, and if so, only then it can use them.
Related issues:
See also #4959.

Yeah, sounds like we need a backup option for hit testing.


[patch] Fix cache bug

Description
description_with_details_and_reasoning
Type of change

[x] Bug fix
[ ] New feature
[ ] Refactoring (no functional changes, no api changes)
[ ] Non-code changes (update documentation, pipeline, etc.)

Flags

[ ] breaks backward compatibility
[ ] requires a documentation update
[ ] has untestable code

Related issue/PR
Delete if not applicable

Fixes #issue
Closes #PR

Checklist

[ ] Followed the guidelines in the CONTRIBUTING document
[ ] Added prefix [major]/[minor]/[patch]/[skip] in the PR title
[ ] Tested and linted the code
[ ] Commented the code
[ ] Made corresponding changes to the documentation
[ ] Confirmed no dropping in test coverage (by Codecov)
[ ] Passed all pipeline checking
[ ] Approved by >1 reviewer

Checklist for maintainer

[ ] Use Squash and merge
[ ] Double-confirm the merge message has prefix [major]/[minor]/[patch]/[skip]
[ ] Delete the branch after merge

Codecov Report

Merging #75 into master will decrease coverage by 0.04%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master      #75      +/-   ##
==========================================
- Coverage   94.53%   94.48%   -0.05%
==========================================
  Files          19       19
  Lines        1263     1270       +7
==========================================
+ Hits         1194     1200       +6
- Misses         47       48       +1
  Partials       22       22

Impacted Files
Coverage Δ

authorizerd.go
92.51% <100.00%> (+0.21%)
:arrow_up:

policy/daemon.go
91.53% <0.00%> (-1.59%)
:arrow_down:

pubkey/daemon.go
94.48% <0.00%> (+1.37%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 908920f...ccdbdd4. Read the comment docs.


Document behavior when cancellation is signaled

My assumption, when passing a cancellation token to .ToListAsync() or other similar APIs, is that the method will throw OperationCanceledException() if cancellation is signalled during the operation. If this is accurate, could the documentation be updated to describe this?
https://docs.microsoft.com/en-us/dotnet/api/microsoft.entityframeworkcore.entityframeworkqueryableextensions.tolistasync?f1url=https%3A%2F%2Fmsdn.microsoft.com%2Fquery%2Fdev16.query%3FappId%3DDev16IDEF1%26l%3DEN-US%26k%3Dk(Microsoft.EntityFrameworkCore.EntityFrameworkQueryableExtensions.ToListAsync``1);k(DevLang-csharp)%26rd%3Dtrue&view=efcore-3.0

@roji Can you find an example of the boilerplate from the BCL that you were mentioning?

I did a quick test and ToListAsync throws TaskCanceledException, which extends OperationCanceledException and is the right behavior (IIRC).
For the exception message, found several in the BCL, I guess we can take our pick:

System.Collections.Concurrent/BlockingCollection`1.xml: If the CancellationToken is canceled.
System.Linq.ParallelEnumerable: The query was canceled with the token passed in through WithCancellation.

There are others. The first one seems sufficient to me...


Regular expressions - range.

Briefly:
I want a particular translation to apply to a particular number
Detail:
Almost every word can have a plural applied to it. This changes the ending. There are usually two variants “Day”, “Days”. In English, usually only an “S” is added at the end of the word. (I'm not very good at English and I don't know all the subtleties yet).
But in Russian there can be several endings depending on the dependent word. “1 день” “2 дня” ”5  дней”
What I want to do:
Let's say in regular expressions there are digits from 1 to 20. I want to make a range of digits to which one of the given translations will be applied.
For example
The translation “A” will be applied to digit 1.
To digits from 2→4 the translation “B” will be applied.
To digits 5→20 the translation “D”.
“А” = 1
デッキあと{{A}}枚必要 同カード{{B}}枚まで=We need another {{A}} card. Stack of cards {{B}}
“В” = 2→4
デッキあと{{A}}枚必要 同カード{{B}}枚まで=Want more {{A}} cardS. Stack of cards {{B}}
“Д” = 5→20
デッキあと{{A}}枚必要 同カード{{B}}枚まで=Need more {{A}} cardS. Stack of cards {{B}}
{{A}} and {{B}} is an included regular expression to the numbers so as not to duplicate the translation. You already know that.
I will duplicate the examples in Russian so that you can notice the difference in the endings.
"А" = 1
デッキあと{{A}}枚必要 同カード{{B}}枚まで=Нужно еще {{A}} картА. Стак карт {{B}}
"В" = 2→4
デッキあと{{A}}枚必要 同カード{{B}}枚まで=Нужно еще {{A}} картЫ. Стак карт {{B}}
"Д" = 5→20
デッキあと{{A}}枚必要 同カード{{B}}枚まで=Нужно еще {{A}} карт. Стак карт {{B}}

You might need to use \ instead of . idk

Those are substitutions, not regular expressions. Make 3 regexes, the last one is r:"^デッキあと([5-9]|1[0-9]|20)枚必要 同カード(\d+)枚まで$"=Need more $1 cards. Stack of cards $2.

So if I want to apply this to all three options it would look like this?
"А"
r:"^デッキあと([1]|1[0-9]|20)枚必要 同カード(\d+)枚まで$"=Нужна ещё $1 карта. Стак карт $2.`
"B"
r:"^デッキあと([2-4]|1[0-9]|20)枚必要 同カード(\d+)枚まで$"=Нужно ещё $1 карты. Стак карт $2.`
"D"
r:"^デッキあと([5-9]|1[0-9]|20)枚必要 同カード(\d+)枚まで$"=Нужно ещё $1 карт. Стак карт $2.`

Detail: Almost every word can have a plural applied to it. This changes the ending. There are usually two variants “Day”, “Days”. In English, usually only an “S” is added at the end of the word. (I'm not very good at English and I don't know all the subtleties yet). But in Russian there can be several endings depending on the dependent word. “1 день” “2 дня” ”5 дней” What I want to do: Let's say in regular expressions there are digits from 1 to 20. I want to make a range of digits to which one of the given translations will be applied.
For example The translation “A” will be applied to digit 1. To digits from 2→4 the translation “B” will be applied. To digits 5→20 the translation “D”. “А” = 1 デッキあと{{A}}枚必要 同カード{{B}}枚まで=We need another {{A}} card. Stack of cards {{B}} “В” = 2→4 デッキあと{{A}}枚必要 同カード{{B}}枚まで=Want more {{A}} cardS. Stack of cards {{B}} “Д” = 5→20 デッキあと{{A}}枚必要 同カード{{B}}枚まで=Need more {{A}} cardS. Stack of cards {{B}}
{{A}} and {{B}} is an included regular expression to the numbers so as not to duplicate the translation. You already know that.
I will duplicate the examples in Russian so that you can notice the difference in the endings. "А" = 1 デッキあと{{A}}枚必要 同カード{{B}}枚まで=Нужно еще {{A}} картА. Стак карт {{B}} "В" = 2→4 デッキあと{{A}}枚必要 同カード{{B}}枚まで=Нужно еще {{A}} картЫ. Стак карт {{B}} "Д" = 5→20 デッキあと{{A}}枚必要 同カード{{B}}枚まで=Нужно еще {{A}} карт. Стак карт {{B}}

Thanks for your help.
I don't understand how to use regular expression validation sites.


Ethiopia (+251) has mobile phone numbers that start with 94

I bought a SIM card recently that was not recognized by libphonenumbers. Its national number starts with 94. Photo of the SIM card attached.

Hi,
Without information from the carrier or another official source, we're likely going to add prefix 944 instead. Please let us know if you have a strong reason to believe the prefix 94 is supported as a whole.

This seems fine. I have encountered three 944 numbers but no other 94x
numbers.
On Thu, Nov 5, 2015 at 8:15 PM, roes<EMAIL_ADDRESS>wrote:

Hi,
Without information from the carrier or another official source, we're
likely going to add prefix 944 instead. Please let us know if you have a
strong reason to believe the prefix 94 is supported as a whole.
—
Reply to this email directly or view it on GitHub
https://github.com/googlei18n/libphonenumber/issues/882#issuecomment-154126470
.

@lincolnq ,
Thanks for your input.
The library has been updated to support the mentioned 944 range.
This change will be live in next release.
Closing this issue. Please re-open if you reply with any questions.

Fixed in release 7.2.3: http://libphonenumber.appspot.com/phonenumberparser?number=0944102228&country=ET


Jf docker scan results to add "Impact Pact" column

Description:
The current Jf docker scan results with an output where "impact Patch" is missing. However, the Xray reports downloaded from Xray and Artifactory UI are visible in the Impact path section. I'm attaching a sample outcome for reference.

Expected OutCome:
An additional column with the Impact path will help the developers point to the right descendant in a particular layer.

Impact path would be really helpful for us as well. Our CI systems run on demand scans prior to pushing the artifact to a repository and due to the design of the on demand scan, normal users do not have access to view the report with impact path.
The JSON report is kind of hard to look at compared to the table view


reduce t2t nmt model size

Hi:
I trained a nmt model use t2t  with vocab size 32000, the nmt model size is about 1.2G, I think it is a little big, so how can i reduce the size of my model with little loss about blue? Any Suggestions would be appreciated, thanks.

I guess this is transformer_big. You can try transformer_base, which may result in lower BLEU. You can also try Afafactor instead of Adam, which should make the model about 60% smaller.

The single_gpu suffix should not affect the model size. In my experience Adafactor gives better results than Adam because I can afford higher batch size, but I need to use Adafactor with learning_rate_schedule=rsqrt_decay or learning_rate_schedule=linear_warmup*rsqrt_decay (depending on T2T version).

thanks @martinpopel , i will have a try.

I guess this is transformer_big. You can try transformer_base, which may result in lower BLEU. You can also try Afafactor instead of Adam, which should make the model about 60% smaller.

@martinpopel Just to clarify, you mean that the model footprint during training (both memory and checkpoint) will be 60%, but not the final exported model for inference (which will be identical).

you mean that the model footprint during training (both memory and checkpoint) will be 60%

Yes.

not the final exported model for inference (which will be identical)

I don't remember if the default export procedure removes the first and second moments (for Adam/Adafactor). If yes, then the final models will have the same size. If not, you could probably remove these moments manually (their names starts with training/). The models won't have identical weights because Adafactor is a different optimizer than Adam, but in my experience the BLEU results are very similar if training with the same batch size (but Adafactor allows you to train with a higher batch size).
Yet another way how to train smaller models without losing much quality (BLEU) is the standard model distillation: translate big monolingual data as well as the source side of your parallel data with the big teacher model. Then use this newly created parallel corpus for training a small student model - you can e.g. decrease the number of layers in the decoder to just two (or replace the Transformer in decoder with some kind of GRU). For details see e.g. Microsoft's excellent work with many other practical tricks.

@martinpopel Excellent. So if the Adafactor is better than Adam, we don't need Adam anymore to be the default in t2t, do we?
Regarding the tricks you mentioned, is there an easy way to quantize the model for inference, for example float -> 8 bit quantization?

So if the Adafactor is better than Adam, we don't need Adam anymore to be the default in t2t, do we?

Adam is good for comparison with prior work and other frameworks, which don't have Adafactor, btw. there is no TF2 Adafactor implementation yet. There maybe tasks where memory is not a problem (either you have large GPUs or larger batch size is not beneficial for a given tasks) or tasks where Adafactor does not work - I don't know; I have tried only MT. Also, there is MultistepAdamOptimizer in T2T (now in two implementations for different T2T+TF versions), which can be used to increase the effective batch size, but there is no MultistepAdafactor.

is there an easy way to quantize the model for inference, for example float -> 8 bit quantization?

This was done in Marian. I am not sure if anything similar is possible in T2T. That said, you can use a T2T model as the teacher and a Marian model as the student.


data generate --model parameter used for local file path and where to point to remote teacher model endpoint (need two separate variables)

Describe the bug

With the addition of context aware chunking: the --model parameter in data generate is used for two competing places that makes it impossible to interact with a remote teacher endpoint anymore. The first location it is used is to point to a local tokenizer file to enable context aware tokenization shown in code path below:
https://github.com/instructlab/instructlab/blob/main/src/instructlab/data/generate_data.py#L93
https://github.com/instructlab/sdg/blob/main/src/instructlab/sdg/utils/chunkers.py#L226
https://github.com/instructlab/sdg/blob/main/src/instructlab/sdg/utils/chunkers.py#L151
https://github.com/instructlab/sdg/blob/main/src/instructlab/sdg/utils/taxonomy.py#L427
The next independent path is it is used to add to the base endpoint-url to properly talk to a remote teacher model endpoint shown in the pipeline context:
https://github.com/instructlab/instructlab/blob/main/src/instructlab/data/generate_data.py#L93
https://github.com/instructlab/sdg/blob/main/src/instructlab/sdg/generate_data.py#L305
https://github.com/instructlab/sdg/blob/main/src/instructlab/sdg/generate_data.py#L378
Therefore when running ilab data generate with the endpoint-url on a pdf document you will always see the following error
[root@ty-sdg-pdftry ~]# /root/bin/ilab.sh data generate --taxonomy-path /root/RBC-Instructlab-Taxonomy --taxonomy-base empty --endpoint-url https://781d2e7c-us-east.lb.appdomain.cloud/v1  --model-family mixtral --sdg-scale-factor 30 --pipeline /root/sdg-config/pipelines/agentic/ --model /instructlab/models/mixtral-8x7b-instruct-v0-1 --output-dir /root/outputdir/ --server-ctx-size 32768
INFO 2024-11-21 03:47:56,419 numexpr.utils:148: Note: NumExpr detected 24 cores but "NUMEXPR_MAX_THREADS" not set, so enforcing safe limit of 16.
INFO 2024-11-21 03:47:56,419 numexpr.utils:161: NumExpr defaulting to 16 threads.
INFO 2024-11-21 03:47:58,664 datasets:59: PyTorch version 2.4.1 available.
INFO 2024-11-21 03:48:01,028 instructlab.data.generate_data:87: Generating synthetic data using '/root/sdg-config/pipelines/agentic/' pipeline, '/instructlab/models/mixtral-8x7b-instruct-v0-1' model, '/root/RBC-Instructlab-Taxonomy' taxonomy, against https://781d2e7c-us-east.lb.appdomain.cloud/v1 server
INFO 2024-11-21 03:48:02,556 instructlab.sdg.utils.taxonomy:147: Processing files...
INFO 2024-11-21 03:48:02,556 instructlab.sdg.utils.taxonomy:153: Pattern 'RBC_ILAB.pdf' matched 1 files.
INFO 2024-11-21 03:48:02,556 instructlab.sdg.utils.taxonomy:157: Processing file: /root/outputdir/documents-2024-11-21T03_48_01/RBC_ILAB.pdf
INFO 2024-11-21 03:48:02,556 instructlab.sdg.utils.taxonomy:172: Loading PDF document from /root/outputdir/documents-2024-11-21T03_48_01/RBC_ILAB.pdf
INFO 2024-11-21 03:48:02,566 instructlab.sdg.utils.taxonomy:182: PDF '/root/outputdir/documents-2024-11-21T03_48_01/RBC_ILAB.pdf' has 283 pages.
INFO 2024-11-21 03:51:00,399 instructlab.sdg.utils.taxonomy:218: Unloaded PDF document: /root/outputdir/documents-2024-11-21T03_48_01/RBC_ILAB.pdf
INFO 2024-11-21 03:51:36,400 instructlab.sdg.generate_data:408: Synthesizing new instructions. If you aren't satisfied with the generated instructions, interrupt training (Ctrl-C) and try adjusting your YAML files. Adding more examples may help.
ERROR 2024-11-21 03:51:38,716 instructlab.sdg.utils.chunkers:397: Failed to load tokenizer as no valid model was not found at /instructlab/models/mixtral-8x7b-instruct-v0-1. Please provide a path to a valid model format. For help on downloading models, run `ilab model download --help`.

There should be a separate variable/cli parameter for passing in the local context aware tokenizer file than that which is used to point to the remote endpoint url holding the teacher model
To Reproduce
Steps to reproduce the behavior:

Run ilab data generate using the --endpoint-url on a taxonomy that has a PDF document in it (example shown above)
It will error saying failed to load tokenizer

Expected behavior

I should be able to run data generate against a remote teacher model endpoint on a taxonomy with a pdf file if I have the local tokenizer files. I should be able to point to the remote teacher model and the local files through separate vars
Screenshots

Device Info (please complete the following information):

Hardware Specs: [e.g. Apple M2 Pro Chip, 16 GB Memory, etc.] L40
OS Version: [e.g. Mac OS 14.4.1, Fedora Linux 40] RHEL AI 1.2
Python Version: [output of python --version] 3.11
InstructLab Version: [output of ilab system info]

Platform:
  sys.version: 3.11.7 (main, Aug 23 2024, 00:00:00) [GCC 11.4.1 20231218 (Red Hat 11.4.1-3)]
  sys.platform: linux
  os.name: posix
  platform.release: 5.14.0-427.37.1.el9_4.x86_64
  platform.machine: x86_64
  platform.node: ty-sdg-pdftry
  platform.python_version: 3.11.7
  os-release.ID: rhel
  os-release.VERSION_ID: 9.4
  os-release.PRETTY_NAME: Red Hat Enterprise Linux 9.4 (Plow)
  memory.total: 117.91 GB
  memory.available: 116.30 GB
  memory.used: 0.58 GB

InstructLab:
  instructlab.version: 0.21.0
  instructlab-dolomite.version: 0.2.0
  instructlab-eval.version: 0.4.1
  instructlab-quantize.version: 0.1.0
  instructlab-schema.version: 0.4.1
  instructlab-sdg.version: 0.6.0
  instructlab-training.version: 0.6.1

Torch:
  torch.version: 2.4.1+cu121
  torch.backends.cpu.capability: AVX512
  torch.version.cuda: 12.1
  torch.version.hip: None
  torch.cuda.available: True
  torch.backends.cuda.is_built: True
  torch.backends.mps.is_built: False
  torch.backends.mps.is_available: False
  torch.cuda.bf16: True
  torch.cuda.current.device: 0
  torch.cuda.0.name: NVIDIA L40S
  torch.cuda.0.free: 44.1 GB
  torch.cuda.0.total: 44.5 GB
  torch.cuda.0.capability: 8.9 (see https://developer.nvidia.com/cuda-gpus#compute)

llama_cpp_python:
  llama_cpp_python.version: 0.2.79
  llama_cpp_python.supports_gpu_offload: True

Additional context

What's the urgency of this issue, to help with prioritization? Were you able to get things going with some workarounds for now, or is this blocking the ability to use InstructLab for your use-case?

This one is non blocking!

But I do believe ultimately this should be a unique controllable variable (we were able to proceed by doing some mount magic between the containerized filesystem and the host.) basically ensuring the proper host path is mounted at the same file path as the remote file mount option.

I agree we need to separate out these variables, and will keep this on the radar to tackle. I'm glad you have a workaround for now, even if it sounds like it was quite a bit of work to implement.

Per usual: super appreciative of everyone's time and attention/help reviewing issues! all of instructlab has an amazing community around it :)


0 device in list

 async ngOnInit() {      this.serviceDiscovery.getNetworkServices('ssdp:all')      .then(devices => console.log(devices))      .catch(error => console.error(error));
result : [ ]
On iOS and Android
???

I'm also trying to get this plugin to work, not sure what I need to do...