Build and test rippled in Github Actions, plus some related fixes and improvements

High Level Overview of Change
The primary focus of this PR is building and testing rippled using Github Actions. Along the way, some other issues came up that either needed to be fixed or were convenient improvements. The changes are divided into several different commits, which can (and probably should) be reviewed independently - they probably could have been independent PRs, but I didn't want to get hung up on the dependencies.
These commits should NOT be squashed down to a single commit for merging, unless there is a consensus otherwise.
Note that I restructured the directories for external dependencies (.nih_c). Until this is merged, if building in an existing repo, you will almost certainly need to move or delete that folder first, then move or delete again when switching to another branch.
Context of Change
Travis CI can be slow and unreliable. Github Actions offers some improvement in those two areas. It's still not as fast as I'd like, but that can potentially be addressed in the future with hosted runners.
The new platform also offers different ways to organize the jobs, and manage caches and artifacts. For example,

The tasks are organized across 7 different mostly-independent workflow files. If a failure occurs in one job, that workflow can be re-run without worrying about dependencies on others.
Every job is organized into steps, which can be expanded and collapsed on github.com, reducing the need to scroll or search through multi-thousand line log files.
Caches are accessed using keys, which are computed based on the CMake config and rippled source code, and are immutable once written. This should mitigate or prevent "corrupt" cache issues, and reduce or eliminate the need to manually delete caches for PRs and branches.
Logs and executables are provided as artifacts. This allows developers to get more information more easily, and debug the executable built by the job in the event of CI-only problems. It also speeds up test-only steps by reusing executables instead of rebuilding them.

Type of Change

[x ] Refactor (non-breaking change that only restructures code)
[x ] Tests (You added tests for code that already exists, or your new feature included in this PR)
[x ] Release

Test Plan
There isn't much to test other than verifying that the CI jobs succeed. Unit tests were added, and some changes were made to non-test code, but they shouldn't affect any run-time functionality.
Future Tasks
As mentioned above, I would eventually like to get these jobs running on better hardware, either some kind of paid-tier resource via Github, or self-hosted runners.

Some data for comparison:
The Github Actions all ran simultaneously. This list indicates how long each of them took.

Linux, gcc-8 release unity: 55m 49s
Linux, clang-8 debug unity: 55m 9s
Linux, clang-8 release unity: 1h 24m 1s
Linux, gcc-8 debug unity: 1h 46m 12s
MacOS: 1h 51m 45s
Linux: 2h 1m 24s
Windows: 3h 0m 54s

All of them succeeded. I expect they'll run faster in the future as caches are seeded.
Travis took a little longer to get started, and when it did, it still took longer to run all the jobs.

Ran for 6 hrs 38 min 59 sec
Total time 22 hrs 9 min 15 sec
And to top it off, two of the tasks failed:
ipv6 (macos)
windows, visual studio, debug

I've restarted those to make sure there is not a real problem, but I doubt it.
I can't make any promises that the Github jobs will never have a spurious failure, or start this quickly when more of us start using it, but these numbers look pretty good.

NIce PR @ximinez !

@thejohnfreeman @manojsdoshi @legleux @xzhaous
I just pushed a bunch of commits that fix more issues, and address most, if not all, of the feedback so far. To make reviewing a tiny bit easier, I didn't rewrite any of the commit history you may have already seen, and I tried to organize the commits logically. If this PR gets approved, I'll squash them down into the single "Build and test rippled in Github Actions" commit, other than the unit test fixes.
Highlights:

The build-action has no parameters
I think I've resolved occasional build failures due to OOM. For now.
I think I've resolved occasional test failures by restricting the number of unittest processes in jobs that tended to fail.
Job names are much cleaner and clearer.
Caches are better organized, especially in Mac and Windows builds that have relatively static dependencies.
Documentation

To answer one lingering question: I left the Travis configuration in for now, so we can run them side-by-side for a while until we're confident that the real-world performance and reliability of Github Actions is at least as good.
 For now. 🤞

Rebased on to 1.8.0-b4. No other changes. I see a couple of jobs have failed. I'll look into that.

(notes: not ready to merge; need @thejohnfreeman to make the Conan changes before we can do this)

@ximinez given conan, what is the best path forward (or not) for this PR?

I've changed this PR to be a draft, indicating that it's not ready for prime time. I would like to integrate it with Conan to build the dependencies, but use the much larger configuration matrix in these workflow files to get more test coverage, but it's not a high priority. (I don't necessarily have to do it if someone else wants to pick up the baton.)


Add linkify-symbolic-links feature

Closes #2276
Test
https://github.com/wmluke/angular-flash/blob/0.1.14/app/components
Additional test links welcome. This one leads to a 404

Dear Quick Response Force members!
Thank you for your work.


Modularize and cleanup traces

Description

Clean up and modularize traces
Add cypress tests for breadcrumbs and table sorting

Issues Resolved
[List any issues this PR will resolve]
Check List

[x] New functionality includes testing.

[x] All tests pass, including unit test, integration test and doctest

[ ] New functionality has been documented.

[ ] New functionality has javadoc added
[ ] New functionality has user manual doc added

[x] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Codecov Report

Merging #601 (33138fa) into main (af54d6a) will increase coverage by 0.04%.
The diff coverage is 68.57%.

@@             Coverage Diff              @@
##               main     #601      +/-   ##
============================================
+ Coverage     53.32%   53.37%   +0.04%
  Complexity      271      271
============================================
  Files           238      239       +1
  Lines          7748     7736      -12
  Branches       1581     1566      -15
============================================
- Hits           4132     4129       -3
+ Misses         3421     3414       -7
+ Partials        195      193       -2

Flag
Coverage Δ

dashboards-observability
46.14% <68.57%> (+0.04%)
:arrow_up:

opensearch-observability
72.33% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

...trace_analytics/components/dashboard/dashboard.tsx
100.00% <ø> (ø)

...s/trace_analytics/components/services/services.tsx
100.00% <ø> (ø)

...nents/trace_analytics/components/traces/traces.tsx
100.00% <ø> (+35.48%)
:arrow_up:

...trace_analytics/components/traces/traces_table.tsx
69.56% <66.66%> (+6.29%)
:arrow_up:

...ace_analytics/components/traces/traces_content.tsx
70.00% <70.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update af54d6a...33138fa. Read the comment docs.


Solution [Danylo Kunchenko]

Good job!
Consider removing empty lines after every action, there is no need for this. Empty lines are usually used in tests to separate some code blocks, e.g. for preparing test data, performing an action, and asserting the result.

gj

GJ

Good Job!:)


undo very slow in unity 2019.3.5f1

when i open any graph and move node position, press ctrl+z.
it's need wait over 1 min then the node recover to origin position.
unity version : 2019.3.5f1

Hello,
I can't repro on my side with 2019.3.6f1. Do you have any modifications on top of the package ? or maybe you are working on a very big graph ?
In any case, can you try to see what's taking time with the Unity Profiler ?

Hello,
I can't repro on my side with 2019.3.6f1. Do you have any modifications on top of the package ? or maybe you are working on a very big graph ?
In any case, can you try to see what's taking time with the Unity Profiler ?

thanks for your reply, i found that it should be caused by other plugins


feat(dependency): Bump react to v15

Current coverage is 99.47%

Merging #20 into master will not affect coverage as of 25877cf

@@            master     #20   diff @@
======================================
  Files           15      15
  Stmts          378     378
  Branches         0       0
  Methods          0       0
======================================
  Hit            376     376
  Partial          0       0
  Missed           2       2

Review entire Coverage Diff as of 25877cf

Powered by Codecov. Updated on successful CI builds.


Make minion tasks pluggable via reflection

Description
Introduces 3 annotations to plugin minion components:

TaskGenerator: For task generators to be plugged on controller side
TaskExecutorFactory: For task executor factories to be plugged on minion side
EventObserverFactory: For event observer factories to be plugged on minion side

Annotated classes will automatically be plugged into the controller and minion.
Release Notes
In order to make classes auto-pluggable, the following interfaces have been changed:

PinotTaskGenerator
PinotTaskExecutorFactory
MinionEventObserverFactory

Customized implementation of these interfaces need to be modified accordingly.

@snleee Please take a look and see if the interface change is okay for the LinkedIn internal tasks


PDFio doesn't handle LC_NUMERIC locale formatting

Unfortunately, setlocale influences how numbers are formatted and scanned/parsed, and a libcups issue has exposed how this affects localized programs using PDFio.
Need to update the file/stream APIs to format and scan numeric values using locale-neutral functions.

[master e882622] Fix locale support (Issue #61)

Adding unit tests, current code isn't working for some reason...

[master b117959] Make sure all output code paths set the locale information (Issue #61)


tests: kernel: mem_protect: stack_random: enable qemu_riscv32

Describe the bug
TL:DR - needed to disable that test only for qemu_riscv32 because it broke CI
Details in #36996
To Reproduce
west build -p always -b qemu_riscv32 -t run tests/kernel/mem_protect/stack_random

It has only failed in CI so far, and I am unable to reproduce the failure locally.
Expected behavior
% west build -p always -b qemu_riscv32 -t run tests/kernel/mem_protect/stack_random
...
*** Booting Zephyr OS build zephyr-v2.6.0-1039-g523764b3fd75  ***
Running test suite stack_pointer_randomness
===================================================================
START - test_stack_pt_randomization
Test Stack pointer randomization
stack pointer changed 13 times out of 64 tests
PASS - test_stack_pt_randomization in 0.5 seconds
===================================================================
Test suite stack_pointer_randomness succeeded
===================================================================
PROJECT EXECUTION SUCCESSFUL

Impact
It made CI fail consistently.
Logs and console output
% west build -p always -b qemu_riscv32 -t run tests/kernel/mem_protect/stack_random
...
*** Booting Zephyr OS build zephyr-v2.6.0-1063-g0106d8f2a391  ***
Running test suite stack_pointer_randomness
===================================================================
START - test_stack_pt_randomization
Test Stack pointer randomization
stack pointer changed 0 times out of 64 tests
Assertion failed at WEST_TOPDIR/zephyr/tests/kernel/mem_protect/stack_random/src/main.c:68: test_stack_pt_randomization: (sp_changed equal to 0)
Stack pointer is not randomized
FAIL - test_stack_pt_randomization in 0.6 seconds
===================================================================
Test suite stack_pointer_randomness failed.
===================================================================
PROJECT EXECUTION FAILED

Environment (please complete the following information):

OS: CI
Toolchain: Zephyr SDK
Commit SHA or Version used: between 84c5a46 and de13034

Additional context

Thanks for reporting this bug. I don't think anyone is currently working on this issue. I will look into it. Could you possibly reassign this to me?

@fkokosinski
Sorry for late. I'm welcome. I can reproduce this problem but there is strange behavior.
If I added printk("%x\n", last_sp); at last of alternate_thread() then this test has passed...
8000623c
8000622c
8000625c
8000624c
8000623c
8000622c
8000625c
8000624c
8000623c
8000622c
8000625c
8000624c
8000623c
stack pointer changed 61 times out of 64 tests
 PASS - test_stack_pt_randomization in 0.15 seconds
===================================================================
Test suite stack_pointer_randomness succeeded
===================================================================
PROJECT EXECUTION SUCCESSFUL
qemu-system-riscv32: terminating on signal 2
ninja: build stopped: interrupted by user.

I want to share current progress:

By debugger analysis (NOT use printk, NOT change the binary): sp is changed for every thread
This test is passed if ...

Children threads call printk()
A main thread calls k_yield() after k_thread_create()
Not use icount option of QEMU

So I guess it's problem of timing of thread switching in special situation.
	/* Start thread */
	for (i = 0; i < THREAD_COUNT; i++) {
		k_thread_create(&alt_thread_data, alt_thread_stack_area,
				STACKSIZE, (k_thread_entry_t)alternate_thread,
				NULL, NULL, NULL, K_HIGHEST_THREAD_PRIO, 0,
				K_NO_WAIT);
		k_yield();    /* Add this line */
	}

# NG (use icount)

qemu-system-riscv32 -nographic -machine virt -bios none -m 256 -net none -pidfile qemu.pid -chardev stdio,id=con,mux=on -serial chardev:con -mon chardev=con,mode=readline \
-icount shift=6,align=off,sleep=off \
-rtc clock=vm -kernel zephyr.elf

# OK (not use icount)

qemu-system-riscv32 -nographic -machine virt -bios none -m 256 -net none -pidfile qemu.pid -chardev stdio,id=con,mux=on -serial chardev:con -mon chardev=con,mode=readline \
-rtc clock=vm -kernel zephyr.elf

Any comments and ideas are welcome.

* Not use icount option of QEMU

I've also found that manipulating the value of icount can force this test to pass. For example, changing the value of icount shift in qemu_riscv32 board config seems to do the trick.
Other than that, I've also noticed that changing the values of the following constants also makes this test pass:

increasing CONFIG_STACK_POINTER_RANDOM to >64 here
changing STACKSIZE here to either >=4096 (max value tested 2096000) or <2048 (min value tested 512)

I've also discovered that the commit that causes this test to fail is the following one:
78c222e56f23b2cfff86f49e8885d74f48110869 is the first bad commit
commit 78c222e56f23b2cfff86f49e8885d74f48110869
Author: Michał Narajowski<EMAIL_ADDRESS>Date:   Wed Jul 14 10:04:37 2021 +0200
    doc: release-notes-2.7: Return value for opcode callback
    Add release notes for Stable API Change in Mesh.
    Signed-off-by: Michał Narajowski<EMAIL_ADDRESS> doc/releases/release-notes-2.7.rst | 2 ++
 1 file changed, 2 insertions(+)

which seemed strange, as this commit only changes the release notes. But what this commit also does, is it increases the length of BUILD_STRING by one character. I've verified that I can make the test pass by forcing the BUILD_STRING to be one character shorter i.e. by:
diff --git a/CMakeLists.txt b/CMakeLists.txt
index 2f8dcbc7ad..7214c0d98a 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -106,7 +106,7 @@ endif()
if(BUILD_VERSION)
  zephyr_compile_definitions(
-    BUILD_VERSION=${BUILD_VERSION}
+    BUILD_VERSION="zephyr-v2.6.0-100-g78c222e56f23"
  )
endif()

The rodata section properties look like this after/before the change:
Section Headers:
 [Nr] Name              Type            Addr     Off    Size   ES Flg Lk Inf Al
 [10] rodata            PROGBITS        80004928 0049b8 000e24 00   A  0   0  4 # fails
 [10] rodata            PROGBITS        80004928 0049b8 000e20 00   A  0   0  4 # passes

Hmm... It's very strange...
If I add this while loop after thread creation loop, this test is going to freeze.
So no context switching is happen.
--- a/tests/kernel/mem_protect/stack_random/src/main.c
+++ b/tests/kernel/mem_protect/stack_random/src/main.c
@@ -60,6 +60,9 @@ void test_stack_pt_randomization(void)
                                K_NO_WAIT);
        }

+       while (changed == 0) {
+               k_yield();
+       }

        printk("stack pointer changed %d times out of %d tests\n",
               changed, THREAD_COUNT);

But... this test is suddenly passed if I add one variable in global name space.
--- a/tests/kernel/mem_protect/stack_random/src/main.c
+++ b/tests/kernel/mem_protect/stack_random/src/main.c
@@ -13,6 +13,7 @@

 void *last_sp = (void *)0xFFFFFFFF;
 volatile unsigned int changed;
+volatile unsigned int foo;

 void alternate_thread(void)
 {
@@ -60,6 +61,10 @@ void test_stack_pt_randomization(void)
                                K_NO_WAIT);
        }

+       while (changed == 0) {
+               k_yield();
+       }
+       foo++;

        printk("stack pointer changed %d times out of %d tests\n",
               changed, THREAD_COUNT);


:pencil2: fix: Package.json name of next app

Description
Typo rename to be consistent and so it doesn't publish to npm
Testing
Explain the quality checks that have been done on the code changes
Additional Information

[ ] I read the contributing docs (if this is your first contribution)

Your ENS/address:

Current dependencies on/for this PR:

main

PR #318

PR #319

PR #320   👈

This comment was auto-generated by Graphite.


Update yafti.yml with way too many options

I added way too much stuff and got a little carried away.  In a way I probably should have thought about storage usage and use cases for this image.  I do not mind if most of these are not accepted, but if you see any here that may be interesting then here are my suggestions.
Other applications I did not add for the following reasons:

Desktop Files Creator (GUI to create .desktop files), felt it was a little out of scope but if anyone wants to look into for Utilities it could be fine?

Audacity, replaced it with Tenacity

Brasero (CD ripping and burning tool), no Flatpak. There are alternative applications on Flathub but I have no idea how well they work.  I also don't know how many people still have CD players in their PCs.

Piper, needs ratbagd.

Peazip, Ark is preinstalled, but there are situations where this might be needed.

PowerISO, proprietary and not many scenarios where you need to mess with ISO files, but there are some reasonable instances where this would come in handy and be easier than any command line tools.

Vivaldi Web Browser, no Flatpak.  Distrobox could work, and I only would suggest this because it is relatively popular.  No reason to go through the hassle of figuring out a container for this when most people would opt for the available web browsers on Flathub.

Due to this being made in-between some major changes to Yafti I'm closing this, but I've merged a majority of these changes in https://github.com/ublue-os/bazzite/commit/abed6f86df14c3c4abb2bb370018f2f1333f84d4
Thanks!


Using provider level layers instead of function level layers

This is to address https://github.com/newrelic/serverless-newrelic-lambda-layers/issues/242

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

@Shereef Thank you for your submission! Would you be able to update it so that it passes npm run lint?

@mrickard It would need https://github.com/DefinitelyTyped/DefinitelyTyped/pull/60209 to be merged and released first then we'd have to update the dependency via npm i -D @types/serverless@latest

@mrickard  any update?

@Shereef I've been on some other projects today, but I'll have a chance to review the updates a bit later.


Convert forward simulation notebooks.

There's two notebooks (wfforward.ipynb and wfcython.ipynb) in the old-content/notebooks directory that need to be converted to jupyterbook format. Hopefully this can be reasonably automatic using jupytext.
The only concern is the environment requirements - what is need to run these notebooks? It would be great if we keep this a no-conda affair.
@molpopgen, would you mind taking a look please?

Also, we want to make sure these pages don't take to long to run.

Also, we want to make sure these pages don't take to long to run.

I doubt they need conda--apt should be fine. The one with cython took quite a while to run, in order to demonstrate correctness.  I have previously suggested that we delete it.

I doubt they need conda--apt should be fine. The one with cython took quite a while to run, in order to demonstrate correctness.  I have previously suggested that we delete it.

Your call on deleting stuff. Demonstrating correctness is beyond the scope of these tutorials now I think.

Your call on deleting stuff. Demonstrating correctness is beyond the scope of these tutorials now I think.

Looking at it now.  It was written back when the access to tskit was via msprime.  It'll take some time to convert.  Also, there's an annoyance--if you convert with jupytext, you can get YAML frontmatter that somehow is incompatible with what is in other md files, causing the whole thing to fail to build.

Looking at it now.  It was written back when the access to tskit was via msprime.  It'll take some time to convert.  Also, there's an annoyance--if you convert with jupytext, you can get YAML frontmatter that somehow is incompatible with what is in other md files, causing the whole thing to fail to build.

I've not bothered modernising the content in the first pass @molpopgen, if that's any help. I've basically just been getting things working, and going to make notes on the outstanding issues so we can follow up in smaller chunks afterwards.

I've not bothered modernising the content in the first pass @molpopgen, if that's any help. I've basically just been getting things working, and going to make notes on the outstanding issues so we can follow up in smaller chunks afterwards.

yeah, I'm going for "working" right now, which it doesn't!

yeah, I'm going for "working" right now, which it doesn't!

I'm going to shorten it considerably.  Looking at it now, I really don't like the length.

I'm going to shorten it considerably.  Looking at it now, I really don't like the length.

This will take a bit more time than I thought.  Teaching, etc.,..

This will take a bit more time than I thought.  Teaching, etc.,..

No, totally off of my radar.  I still vote to delete the Cython example and then minimally port over the other to markdown.

That sounds ideal to me. You mentioned something about shortening though? Is that still relevant?

Yes, IMO a tutorial should strive to be only a "screen or 2 long".  To that end, something like cell 3 modified to do exactly 1 crossover / meiosis in the "pseudo-haploid" style of the C example will be sufficient.  Updating to include periodic simplification would be a good thing, too, I think.

IMO a tutorial should strive to be only a "screen or 2 long"

@molpopgen - I am looking at this with a view to showing a simple forwards simulator for a workshop. I rethought the python code from scratch, and came up with this. I would be interested in how clear or otherwise you think it is (I am guessing it isn't very efficient though). The nice thing about picking parents, rather than genomes, at random is not just that it is more obvious to a biologist, but that we can  relatively easily modify the add_inheritance_paths function below to allow recombination between the two genomes within a parent:
import tskit
import random

def make_genomes(tables, time, individual):
    "Make two sample genomes by adding to the nodes table, returning their IDs"
    return (
        tables.nodes.add_row(tskit.NODE_IS_SAMPLE, time=time, individual=individual),
        tables.nodes.add_row(tskit.NODE_IS_SAMPLE, time=time, individual=individual),
    )

def make_initial_population(tables, time, num_individuals):
    """
    Adds a set of individuals and their associated genomes (nodes) to the tables
    Returns a dictionary of {indivID: (maternal_genomeID, paternal_genomeID), ...}
    """
    population = {}
    for _ in range(num_individuals):
        newID = tables.individuals.add_row()
        population[newID] = make_genomes(tables, time=time, individual=newID)
    return population

def add_inheritance_paths(tables, parent_nodes, child_node):
    """
    Add inheritance paths from parent node IDs (i.e. parent genomes)
    to a child node ID (i.e. a child genome)
    """
    parent_node = random.choice(parent_nodes)  # Simplest option: no recombination
    tables.edges.add_row(
        left=0, right=tables.sequence_length, parent=parent_node, child=child_node)

def simple_diploid_sim(popsize, generations, sequence_length, random_seed=123):
    # This TableCollection, called "tables" is where we will store our genealogy
    tables = tskit.TableCollection(sequence_length)
    tables.time_units = "generations"
    random.seed(random_seed)
    # tskit time is in units ago: we want the final generation to be at time 0
    generations_ago = generations - 1  # start with the oldest generation ago

    parents = make_initial_population(tables, generations_ago, popsize)
    while generations_ago > 0:
        generations_ago = generations_ago - 1
        children = {}  # As for parents, maps each individual ID to two node IDs
        for _ in range(popsize):
            newID = tables.individuals.add_row()
            children[newID] = make_genomes(tables, generations_ago, individual=newID)
            # Pick two individual parent IDs at random with replacement (allows selfing)
            random_parentIDs = random.choices(list(parents.keys()), k=2)
            for child_genome, pID in zip(children[newID], random_parentIDs):
                add_inheritance_paths(tables, parents[pID], child_genome)
        parents = children
    tables.sort()
    return tables.tree_sequence()

ts = simple_diploid_sim(5, generations=15, sequence_length=1000)

I think I'd also go straight to the case with recombination

ISWYM. It is nice to be able to simulate without recombination, so you get a single tree for viz purposes. But I could do that simply by setting recombination_rate to 0.

As much as Python type hints can bother me, I do think they'd help here. :)

I should also say that having make_new_generation as a function that is called each time means it should be easy to both add more generations onto the end of the simulation, and show how to simplify e.g. every 10 generations.

I have worked this up into a (very) draft workbook at https://hyanwong.github.io/workshop-pages/lab?path=ForwardSimulation.ipynb, which has a bit of explanation between the function definitions, etc, and examples of what simplification does. I have not talked about incorporating simplification into the actual simulation, however - perhaps I should do so. I have also not mentioned mutation.
The idea would be to gradually port @molpopgen 's nice text into this notebook somehow, which could perhaps form the nucleus of a new tutorial. But if we want to keep the existing code, that's fine too (as I am wanting to use mine for a separate "workbook" -type file for workshops rather than the standard tskit tutorials.
More comments gratefully accepted. I know it's a bit rough at the moment.

Thinking about this, I wonder if it is a good idea to split it into two tutorials. The first can be something like the workbook I put up, and is a "simple simulator" (but with recombination). The second can contain all the gotchas, details about mutations, repeated simplification (and invariance), recapitation, validation, etc. What do you think @molpopgen ?

It think that the level of detail depends a bit on your target audience.  For static tutorials served up on a website, I think that several short pages is okay.

It think that the level of detail depends a bit on your target audience. For static tutorials served up on a website, I think that several short pages is okay.

Yes, I agree. Are you happy for me to take your wfforward notebook, cur it into two, and graft a modified version of my simulation code into the first part, @molpopgen ? Some comments about the first part of your text below:

A node is used to label the birth time of a lineage.  A node can be described by a tuple, (id, time), where id is unique and time reflects the birth time of that id.

I think we should describe a node as a genome rather than an "event" (e.g. the birth of a lineage). For instance, a coalescent node might not actually mark the exact time of coalescence: rather it is the neareast representative genome we have to that event. See the eARG vs gARG distinction in the ARG preprint.

For the case of the discrete-time Wright-Fisher (WF) model, if $N$ individuals currently exist at time point $t$, then there are $2N$ nodes, $[i,i+2N)$, and the diploids are defined as tuples grouping pairs of adjacent nodes, $D \in [(i,i+1),(i+2,i+3),\dots,(2N-2,2N-1)]$, where each $(i,j)$ pairing is a diploid.

I don't think we need to require diploids to be adjacent nodes any more, do we? Instead we can group them as belonging to the same individual. Indeed, once we have simplified, the internal nodes can no longer be thought of as adjacent diploids anyway (ancestral nodes are unlikely to also have their matching diploid pair in the genealogy)

All of our simulations will follow the parameter scaling laid out by Dick Hudson's ms software,

I have not followed that scaling in my example, preferring e.g. to have a genome of length e.g. 50kb. Using base pairs seems easier for beginners to me, but I can also see the benefits of using 0..1, so am willing to be convinced that the Hudson scaling is better as an introduction.

We also need to define what we mean by time, which is simple for the discrete-time model.  A simulation will start at generation 0,

I find it easier to code time as generations ago (i.e. standard tskit direction), rather than distract the reader by having to reverse the time scale on export. The time-reversal technique has the down-side that the tables are invalid all the way through the simulation, up until the point that the times are reversed. My suggestion is that when we simply count for gen in range(ngens-1, 0, -1): (another option would be start at generation zero but count down, creating negative generation times, and adding ngens to the node times on export). The only issue is when simulations do not have a fixed generation time, or are resumed (in this case, one option is to add ngens to all the existing node (and mutation) times).
I suggest that we combine both counting-down techniques. I.e. we set the initial generation to be time ngens-1 (so that in the simple case, the final generation is at time zero). But once we get to more advanced code, we check the current value of gens and deduct it from the node (and mutation / migration) times, see https://github.com/tskit-dev/tskit/issues/2809

It think that the level of detail depends a bit on your target audience. For static tutorials served up on a website, I think that several short pages is okay.

Yes, I agree. Are you happy for me to take your wfforward notebook, cur it into two, and graft a modified version of my simulation code into the first part, @molpopgen ? Some comments about the first part of your text below:

A node is used to label the birth time of a lineage.  A node can be described by a tuple, (id, time), where id is unique and time reflects the birth time of that id.

I think we should describe a node as a genome rather than an "event" (e.g. the birth of a lineage). For instance, a coalescent node might not actually mark the exact time of coalescence: rather it is the neareast representative genome we have to that event. See the eARG vs gARG distinction in the ARG preprint.

I defined a node intentionally with respect to the birth of a genome.  I think this definition is appropriate in the context.

For the case of the discrete-time Wright-Fisher (WF) model, if N individuals currently exist at time point t, then there are 2N nodes, [i,i+2N), and the diploids are defined as tuples grouping pairs of adjacent nodes, D∈[(i,i+1),(i+2,i+3),…,(2N−2,2N−1)], where each (i,j) pairing is a diploid.

I don't think we need to require diploids to be adjacent nodes any more, do we? Instead we can group them as belonging to the same individual. Indeed, once we have simplified, the internal nodes can no longer be thought of as adjacent diploids anyway (ancestral nodes are unlikely to also have their matching diploid pair in the genealogy)

Here, you need to think about computer architecture.  Even with "individuals", nodes are likely to be kept adjacent for performance reasons.  The "data oriented" paradigm from, e.g. computer games, would have "nodes" by an array within an "individuals" structure.

All of our simulations will follow the parameter scaling laid out by Dick Hudson's ms software,

I have not followed that scaling in my example, preferring e.g. to have a genome of length e.g. 50kb. Using base pairs seems easier for beginners to me, but I can also see the benefits of using 0..1, so am willing to be convinced that the Hudson scaling is better as an introduction.

Either way.

We also need to define what we mean by time, which is simple for the discrete-time model.  A simulation will start at generation 0,

I find it easier to code time as generations ago (i.e. standard tskit direction), rather than distract the reader by having to reverse the time scale on export. The time-reversal technique has the down-side that the tables are invalid all the way through the simulation, up until the point that the times are reversed. My suggestion is that when we simply count for gen in range(ngens-1, 0, -1): (another option would be start at generation zero but count down, creating negative generation times, and adding ngens to the node times on export). The only issue is when simulations do not have a fixed generation time, or are resumed (in this case, one option is to add ngens to all the existing node (and mutation) times).
I suggest that we combine both counting-down techniques. I.e. we set the initial generation to be time ngens-1 (so that in the simple case, the final generation is at time zero). But once we get to more advanced code, we check the current value of gens and deduct it from the node (and mutation / migration) times, see tskit-dev/tskit#2809

Here it is a matter of taste.  I'm not sure that "generations" ago generalizes to a forward simulation.  For example, there could be a termination condition such that the end time is not tskit's "zero".  While that doesn't matter for the data model, it is not necessarily true that the final time is zero. It is, in many cases, more natural for a forward simulation to increment time using positive integers and then do the conversion.

Here, you need to think about computer architecture. Even with "individuals", nodes are likely to be kept adjacent for performance reasons. The "data oriented" paradigm from, e.g. computer games, would have "nodes" by an array within an "individuals" structure.

Yes, I think that we could say that, for efficiency, you might want that. But I don't think we need any more to define a diploid as two successive node IDs. And certainly, two successive node IDs don't have to correspond to a diploid, right? (otherwise all ancestors would have to be diploids too)

I think there's a miscommunication here.  My text refers to what forward simulations generate, which is individuals born with some ploidy.  The text has that audience in mind.  What tskit does with ancestors is an internal detail of tskit.

Maybe it's worth asking a newcomer?

I think it depend on the target audience.  For example, is familiarity with the tskit data model a strong prereq for this tutorial?

I think there's a miscommunication here. My text refers to what forward simulations generate, which is individuals born with some ploidy. The text has that audience in mind. What tskit does with ancestors is an internal detail of tskit.

Ah yes, that is a very good point. I am probably thinking too much in terms of tskit definitions, rather than the (independent) simulation code.

I think there's a miscommunication here. My text refers to what forward simulations generate, which is individuals born with some ploidy. The text has that audience in mind. What tskit does with ancestors is an internal detail of tskit.

Ah yes, that is a very good point. I am probably thinking too much in terms of tskit definitions, rather than the (independent) simulation code.

However, if, at any point in the simulation, we want to remove (or renumber) a node independently of the individual in which it resides, then using adjacent IDs to define an individual becomes problematic. Most of the time we leave the details of removing / reallocating IDs to tskit, but I suppose I could imagine cases where this might be required in the simulation itself?

However, if, at any point in the simulation, we want to remove (or renumber) a node independently of the individual in which it resides, then using adjacent IDs to define an individual becomes problematic. Most of the time we leave the details of removing / reallocating IDs to tskit, but I suppose I could imagine cases where this might be required in the simulation itself?

But again, I am talking about births.   Birth happen at once, "in bulk", adding rows to the node table.  That's consecutive, right?

But again, I am talking about births. Birth happen at once, "in bulk", adding rows to the node table. That's consecutive, right?

Right. One of the confusions is coming from the fact that I am using tskit to allocate (successive) node IDs, rather than resetting IDs to zero each generation. That means, if we are simplifying part-way, there could be an odd number of nodes in the node table (i.e. after simplification), so that individuals start being allocated nodes such as (101,102), (103, 104), which , although consecutive, do not start with even numbers, implying that the lower IDs do not conform to the standard 2-nodes grouping.

Right. One of the confusions is coming from the fact that I am using tskit to allocate (successive) node IDs, rather than resetting internal simulation IDs to zero each generation. That means, if we are simplifying part-way, there could be an odd number of nodes in the node table (i.e. after simplification), so that individuals start being allocated nodes such as (101,102), (103, 104). Although these groupings are consecutive, they do not start with even numbers, implying that the lower IDs do not conform to the standard 2-nodes grouping.

There's no requirement that things start with an even number.  For a diploid, they are simply the next 2 available numbers.

There's no requirement that things start with an even number. For a diploid, they are simply the next 2 available numbers.

Sure, but the implication would then be that nodes exist (or have existed) in the simulation that are not grouped in pairs. Without going into the gory details of dropping single nodes from the ID list, it could potentially be a bit confusing to the reader?

There's no requirement that things start with an even number. For a diploid, they are simply the next 2 available numbers.

Sure, but the implication would then be that nodes exist (or have existed) in the simulation that are not grouped in pairs. Without going into the gory details of dropping single nodes from the ID list, it could potentially be a bit confusing to the reader?

At some point, you have to assume some background in population genetics or else you will go crazy.  This is domain-specific stuff you're talking about.

At some point, you have to assume some background in population genetics or else you will go crazy. This is domain-specific stuff you're talking about.

Sure - anyway, I think it's useful to mention the tskit way of recording individuals (because we can store those in the simulation too), so I'll see if I can put together some sort of wording that details both cases, without going into too much detail for the beginner (wish me luck!).

Oh, now I see the confusion! For me, an individual has nothing to do with the "individuals table" until a final export happens.  Even when keeping ancient samples, it is often cleaner to manage those nodes yourself, remapping them each time you simplify, then deal with the individuals table.

Ah yes. Personally, I think it is helpful for a forwards simulation to record individuals in the individuals table as they are created (rather than generate them on export). Then you also keep all the individuals associated with ancestral non-sample genomes too, which can be helpful. For example, it means you can e.g. store the geographical location of the individual in the table, and hence gain access to the location of ancestral genomes.

That works, but has drawbacks.  For example, having the locations only in the tskit table means that they aren't in, say, an R-tree or some other means of efficiently looking up things in "space".

That works, but has drawbacks. For example, having the locations only in the tskit table means that they aren't in, say, an R-tree or some other means of efficiently looking up things in "space".

Yes, although I'm not saying that they should only be stored in tskit. In practice, you might keep the locations for the parent and the child generations in some efficient structure too, but throw them away when you go to the next generation. Having the locations stored for ancestors is generally useful (and if you wanted to do anything time consuming with those locations, you might well extract the coordinates from the individual table and put them into a more efficient structure.
Either way, I'm pretty sure that it is helpful to store individuals as-we-go, recording any metadata about them as necessary. Individuals associated with non-sample nodes can be discarded later if need be, but they can contain useful information which can be worth storing and accessing from the output tree sequence. I don't think this is easily done by constructing the individuals at the end.

Here's what I have so far (a hybrid of your texts (mostly) and mine):

Forward-time simulation with tskit
This tutorial shows the basics of creating a basic forward-time simulator that stores the evolving genealogy as a tskit tree sequence. We will focus on the case of diploids, in which each individual contains 2 genomes, but the concepts used here generalize to any ploidy, if you are willing to do the book-keeping!
:::{note}
The distinction between an individual and the genomes it contains is an important one. In fact, individuals are not strictly necessary for representing genetic genealogies (it's the genomes which are important), but a simulator needs to account for individuals, at least temporarily.
:::
Definitions
Before we can make any progress, we require a few definitions.
A node represents a genome at a point in time (often we imagine this as the "birth time" of the genome).  It can be described by a tuple, (id, time), where id is a unique integer, and time reflects the birth time of that id. When generating a tree sequence, this will be stored in a row of the node table.
A diploid individual is a group of two nodes. During simulation, a simple and efficient grouping assigns sequential pairs of node IDs to an individual. It can be helpful (but not strictly necessary) to store individuals within the tree sequence as rows of the the individual table (a node can then be assigned to an individual by storing that individual's id in the appropriate row of the node table).
An edge reflects a transmission event between nodes.  An edge is a tuple (Left, Right, Parent, Child) whose meaning is "Parent genome $P$ passed on the genomic interval $[L, R)$ to child genome $C$". In a tree sequence this is stored in a row of the edge table
The time, in the discrete-time Wright-Fisher (WF) model which we will simulate, is measured in integer generations. To match the tskit notion of time, we record time in generations ago: i.e. for a simple simulation of G generations, we start the simulation at generation $G-1$ and count down until we reach generation 0 (the current-day).
The population consists of $N$ diploid individuals ($2N$ nodes) at a particular time $t$. At the start, the population will have no known ancestry, but subsequently, each individual will be formed by choosing (at random) two parent individuals from the population in the previous generation.
Approach
The goal of this tutorial is to work through the book-keeping required to generate edges, nodes, and individuals forwards in time, adding them to the relevant tskit tables. To aid efficiency, we will also see how to "simplify" the tables into the minimal set of nodes and edges that describe the history of the sample. Finally, these tables can be exported into an immutable tree sequence for storing or analysis.

@molpopgen - I suggest that material in your wfforward notebook that deals with "Starting with a prior history" (e.g. matching time units, etc) should be ported into the (already existing) Completing forwards simulations tutorial. Does that seem sensible?

Please just go ahead and do whatever you think is needed. I don't have any attachment to anything these docs and don't need to approve any changes.


Triangle Quest

Solution added to the repo : https://github.com/jarvismayur/Hackerrank-Problem-Solutions/blob/main/Python/Triangle Quest/Triangle_Quest.py

Docs are available here : https://github.com/jarvismayur/Hackerrank-Problem-Solutions/tree/main/Python#triangle-quest


Cut down startup banner

With @clairexen's blessing I touched on the startup banner. I moved the license verbiage from the banner off to a new license command.
This is what it looks like:
$ ./yosys

 /----------------------------------------------------------------------------\
 |                                                                            |
 |  yosys -- Yosys Open SYnthesis Suite                                       |
 |  Copyright (C) 2012 - 2024  Claire Xenia Wolf<EMAIL_ADDRESS>       |
 |  Distributed under an ISC-like license, type "license" to see terms        |
 |                                                                            |
 \----------------------------------------------------------------------------/

 Yosys 0.38+92 (git sha1 173f4b5fb, clang++ 11.1.0 -fPIC -Os)

yosys> license

 /----------------------------------------------------------------------------\
 |                                                                            |
 |  yosys -- Yosys Open SYnthesis Suite                                       |
 |                                                                            |
 |  Copyright (C) 2012 - 2024  Claire Xenia Wolf<EMAIL_ADDRESS>       |
 |                                                                            |
 |  Permission to use, copy, modify, and/or distribute this software for any  |
 |  purpose with or without fee is hereby granted, provided that the above    |
 |  copyright notice and this permission notice appear in all copies.         |
 |                                                                            |
 |  THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES  |
 |  WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF          |
 |  MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR   |
 |  ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES    |
 |  WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN     |
 |  ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF   |
 |  OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.            |
 |                                                                            |
 \----------------------------------------------------------------------------/

yosys>
exit
End of script. Logfile hash: 0e57a725e0, CPU: user 0.04s system 0.02s
Yosys 0.38+92 (git sha1 173f4b5fb, clang++ 11.1.0 -fPIC -Os)
Time spent: 100% 1x license (0 sec)

Maybe shrink it even further, by two almost-empty lines?


Add support for PHPUnit 11

I would like to update the Debian package to be compatible with PHPUnit 11 (in experimental branch currently): https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1070567
I have identified a few of the changes needed:

Add static to test providers.

Change $this-> to self::.
Need to find a replacement for getMockForAbstractClass. Maybe a callback that is called within the test case.

Change setMethods to onlyMethods or addMethods.

There could be additional changes needed, this is just what I found so far.

@reeperbahnause would you mind sending a PR?

I have landed https://github.com/php-lock/lock/commit/22d4b4253b4472659092964e761cbe022df71081 which should cover the majority of LoC changes needed.


MeasureText have only width informations

Hi everybody,
Enjoy with this extensions, thanks a lot
When i try to use _context.MesureTextAsync, only width parameter is present, all other return 0    :-(
just tested with this simple test :
protected override async Task OnAfterRenderAsync(bool firstRender)
{
    this._context = await this._canvasReference.CreateCanvas2DAsync();
    await this._context.SetFillStyleAsync("green");

    await this._context.FillRectAsync(10, 100, 100, 100);

    await this._context.SetFontAsync("48px serif");
    await this._context.StrokeTextAsync("Hello Blazor!!!", 10, 100);

    **var size = await this._context.MeasureTextAsync("48px serif");**

}

Any help is welcome,
thanks,
Noeilnoeil17

Me too on this. Changing fonts, changing font sizes, nothing seems to change the results - all zeros except for the width and that value never changes regardless of the length of text, the font etc.

Hi bigDave1357, thanks for your feedback, for now, only solution is using javascript...
function measuretextincanvas(font, text, mmToPix) {
var canvas = document.getElementById('hiddencanvas');
var ctx = canvas.getContext('2d');
ctx.font = font;
var TextWidth = ctx.measureText(text).width / mmToPix;
var TextHeight = ctx.measureText("H").actualBoundingBoxAscent / mmToPix;
return { TextWidth, TextHeight };
}
Noeilnoeil17


Fix for writable dirs

See https://github.com/capistrano/symfony/issues/38#issuecomment-113264709

Thanks @wideawake


Merging Defaults with CLI

What version of Ajv you are you using?
<EMAIL_ADDRESS>What problem do you want to solve?
I'm actually not sure if I'm missing a setting or just getting confused by the docs, but I was hoping to use the cli and the --use-defaults option to produce the merged JSON output, as is done in scripting.
What do you think is the correct solution to problem?
Based on the documentation indicating this function modified data, I expected the cli to either edit the data file in place, or else output the combined JSON (either would be fine).
The closest thing currently possible appears to be returning a JSON patch which would yield the data complete with defaults, but a) I can't seem to find any CLI tools that apply JSON patches and b) It seems like it would be a bit hacky in any case as the result of the validation is mixed into stdout with the diff.
As for implementing this, the paths of least resistance would seem to be supporting the -o argument  in validations and/or supporting a merged value for the --changes argument. I would also suggest moving status messages about validation to standard error.
Will you be able to implement it?
Probably not due to time constraints and my lack of familiarity with the code, but I suspect this would be fairly trivial as this is already possible via the api.

Correct - currently use-defaults option only takes defaults into account when validating, but the modifying object is rejected.
-o option to print resulting object is ok as the idea. I don’t think I follow the ideas about “merged” for changes.
Re moving errors to stderr is a good suggestion, but it’s a breaking change. Also it’s not necessary for this change - there would be either errors or output - as with compile, when the proposed -o option is used there should be no other output.
PR is welcome.

-o option to print resulting object is ok as the idea. I don’t think I follow the idea about “merged” for changes.

I just meant that instead of printing the patch it could print the data with the patch applied. That might have fit in better with existing functionality.

Re moving errors to stderr is a good suggestion, but it’s a breaking change. Also it’s not necessary for this change - there would be either errors or output (and there will be a different exit code)

I agree it's not necessary if -o is added (at least for my purposes) but currently a correct validation with the changes outputted prints both the patch and x.json is valid to stdout. So if it passes (per the exit code) and you want to do something with the patch, you have to extract it from the status info.

as with compile, when the proposed -o option is used there should be no other output.

Up to you. Normally I'd expect the status to still be printed out (albeit to stderr) unless there was a quiet/-q flag available and set. Not too big of a deal as long as the status isn't written to the file though, as if you're scripting the status message doesn't matter.

PR is welcome.
I'll see if I have a chance. It doesn't seem like too big a change. Someone with more familiarity would be able to do it pretty quickly though, should some such person happen upon this and have a couple of spare minutes.


lib/attrsets: add concatMapAttrs

Description of changes
concatMapAttrs: Map each attribute in the given set and merge them into a new attribute set.
concatMapAttrs ::
  (String -> a -> AttrSet)
  -> AttrSet
  -> AttrSet

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
22.11 Release Notes (or backporting 22.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

I changed mapAttrsMulti to concatMapAttrs and changed filterMapAttrs to a similar definition, also added a test for each

Related PRs that add a recursive aspect

https://github.com/NixOS/nixpkgs/pull/206965
https://github.com/NixOS/nixpkgs/pull/202517
https://github.com/NixOS/nixpkgs/pull/221608


Implement CoreTypeConcrete::Bytes31 return type

Implement the parse result for CoreTypeConcrete::Bytes31
https://github.com/lambdaclass/cairo_native/blob/ddfd28217c10f7202a03f10234f0a19113c46904/src/executor.rs#L786
We should be able to run the bytes31_ret.cairo test
https://github.com/lambdaclass/cairo_native/blob/e89d38ad2f01bbd85c9f9de0b37841ac98661b50/tests/tests/cases.rs#L113

hey @pefontana, I'm down to try this is possible

Hi @raizo07 !
Sorry, I am currently implementing it!
But if you want to take another CoreTypeConcrete return type, you can ask us and implement it!


x/tools/gopls: normalize feature sets of unimported completions and organize imports

What version of Go are you using (go version)?

go version go1.13.4 darwin/amd64

Does this issue reproduce with the latest release?
Can't test on latest release, will update when brew makes go1.13.5 available
What operating system and processor architecture are you using (go env)?
go env Output
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/steelphase/Library/Caches/go-build"
GOENV="/Users/steelphase/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GONOPROXY="go.company.org"
GONOSUMDB="go.company.org"
GOOS="darwin"
GOPATH="/Users/steelphase/Code/Go"
GOPRIVATE="go.company.org"
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/Cellar/go/1.13.4/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.13.4/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/0b/zk7t542s1sj_31mwpwyk3cy81gy8z8/T/go-build062639987=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?
Pasted in some code that referenced a dependency that needed to be imported.
What did you expect to see?
import to be autocompleted for direct dependency go.company.org/org/lib/v4/errors
What did you see instead?
import autocompleted to transitive dependency github.com/pkg/errors
Additional Details
This is the go.mod of the module I'm working in
module go.company.org/org/thing

go 1.13

require (
	go.uber.org/zap v1.13.0
	go.company.org/org/lib/v4 v4.1.0
)

output of go mod why
steelphase@macbook:lib$go mod why github.com/pkg/errors
# github.com/pkg/errors
go.company.org/org/thing/cmd/thing
go.uber.org/zap
go.uber.org/zap.test
github.com/pkg/errors

@gopherbot FeatureRequest

Partly a followup to https://github.com/golang/go/issues/36077 -- we should prefer direct deps more than transitive. Also, maybe look into having imports on save do the same sorting.

Retitling to reflect the remaining work.
Unimported completions are missing the "sibling imports" trick. This would be a nice feature. I'm not sure whether to reuse the implementation from the imports package or rewrite it from scratch. Sharing code would be nice, but gopls already has all the parsing done. Just pass all the sibling files to GetPackageExports?
Organize Imports is missing the sorting by relevance. This should be easy to plug in to addExternalCandidates, as long as we want it to affect goimports. Sibling imports will take precedence, but that's probably appropriate.

...except that addExternalCandidates goes to great lengths to return the first usable candidate it finds for speed reasons. It would be very nice to prioritize the scan in the right order, rather than sorting afterward.

It would be nice if gopls could scan the entire workspace for import aliasing patterns rather than only sibling files.
I also imagine it would be convenient if the aliased unimported packages were addressable by both the alias and the package name since the user may or may not anticipate the alias.

Any movement on this? It's quite an annoying issue.

Please be specific about what you're seeing and why it's annoying.

Sorry, I wasn't under the impression that the issue covered more than one bug. What I'm seeing is that import-organization-on-save adds imports that aren't direct dependencies (in go.mod) or even transitive deps (in go.sum).
For example, assert often pulls in gotest.tools/assert, which is in my local mod cache but nowhere else, instead of github.com/stretchr/testify/assert, which is in go.mod.

Change https://go.dev/cl/623296 mentions this issue: internal/imports: carve out a Source interface for index integration

I've tried the latest gopls/v0.17.0-pre.4, still not resolved.


Hangs when crawling for description

The URL I'm crawling is: https://www.flickr.com/explore/
I stepped through the source and see that it's hanging in crawlDescription() at the crawlCode() call.
I see this in the debugger console: SendDelegateMessage(NSInvocation *): delegate (<CFNotificationCenter 0x6080000aed60 [0x108679df0]>) failed to return after waiting 10 seconds. main run loop mode: kCFRunLoopDefaultMode

Hey @taykay08. Thanks. I'll have a look.

This is weird, here it works.

Ok, thanks for checking. Maybe it's on my end - I'll try to isolate it in a new project.

Ok. Please let me know.

Yikes, thought I had grabbed the latest code, I guess I didn't. Anyway, it seems to be working the same as yours now. Takes about a minute, but happy to see it hit the completion method now. Sorry about that, and thanks for the quick response!


Optimization Ideas For MinHash

Noticed your comment on Hacker News about this repo.  I worked on something similar at a previous job (so I don't have the code the share), but looked pretty deeply into optimizing minhash.  Not sure if it's helpful or if you're already aware of these tricks, but I found them to significantly speed up my minhash algorithm (and they are fun to code :P).

One Permutation Hashing.  Basically use a single hash function in clever way instead of the typical 128.  Made a pretty big different in speed during my testing.
Densified One Permutation Hashing.  A small tweak for handling the "empty bin" problem of one permutation hashing.
Reservoir Sampling.  A trick for using less memory and avoiding dynamic allocations.  This last paper also has a summary of the first two.

Caveat: Seems like you're working on Super Min Hash, which I was never able to fully code myself. I don't know how it compares to the one-hash approach.

Thanks for your suggestions. I may have encountered some of these ideas in papers, but never took a deep look.  Currently, I use MinHash for relatively short text documents (usually 50 - 200 characters), and generating min-hash signatures takes a small fraction of the whole pipeline. For bigger documents hashing can become a bottleneck.
Memory usage is definitely an issue for me, which I mitigate by using smaller hashes (thanks to Rust generics this is easy). I should definitely look at Reservoir Sampling and One Permutation Hashing, (especially if they are fun to code :) )

Hello All,
Just a followup, the most recently densification is called optimal densification: two ways to perform the densification are optimal (RMSE is the same with the original, much slower, many hash function implementation): http://proceedings.mlr.press/v70/shrivastava17a.html or http://proceedings.mlr.press/v115/mai20a.html. Implementation here: https://github.com/jean-pierreBoth/probminhash/blob/master/src/densminhash.rs
@serega I have some interesting idea to combine minhash with graphs to do very large scale search. Let me know if you are interested.
Jianshu


Add learning resource

Hi,
Should we add this: https://blog.bytebytego.com/p/how-to-build-a-smart-chatbot-in-10 ?
It's a paid article but there is a free trial and from what I read it looks well written

Hey thanks for the suggestion. Being accessible and open is the core of this list, so we will need to pass on this article.


login button has no visible feedback, when defunct

Tested with owncloud/ocis:1.0.0-beta8

log in at phoenix web ui
do some uploads
wait 3 minutes for automatic logout
kill the server hard,
restart a fresh server
press the login button.
nothing happens

expected behaviour

users always see a reaction when clicking a button. Here: Show a "tls: bad certificate" message on the gui.

Same as above, but without restarting the server after the crash, (just let it hang)
When clicking on the Login button the browser sends a request, but gets no response.

Expected behaviour: a missing response also causes a message in the gui.

@LukasHirt @kulmann Any ideas?
Seems  more like a general problem.
Difficult if you send a request and get no response, how should the product behave
@pmaier1 Indicator that "something" is going on?


Carthage error: "Not a git repository"

I've tried these variations in my Cartfile:
github "Alamofire/Alamofire" "swift-2.0"
git "https://github.com/Alamofire/Alamofire.git" "swift-2.0"
And I get this error each time when running carthage update:
fatal: Not a git repository (or any of the parent directories): .git
All of my other dependencies can clone and build - any ideas?

Hi @bruceflowers,
It's not an issue with Alamofire but some other issue. Instead of creating a support issue in Github, please open a question on Stack Overflow and tag Alamofire. The Alamofire community is very helpful and supportive. Stack Overflow is the best place to get your question answered quickly and correctly.
Best of luck!

From our Contribution Guidelines
Asking Questions
We don't use GitHub as a support forum. For any usage questions that are not specific to the project itself, please ask on Stack Overflow instead. By doing so, you'll be more likely to quickly solve your problem, and you'll allow anyone else with the same question to find the answer. This also allows maintainers to focus on improving the project for others.


Document Biber's behaviour for handling entries with duplicate keys

When Biber encounters two entries with the same key, it outputs a warning that indicates the second entry is being ignored rather than overriding the first entry:
WARN - Duplicate entry key: 'foo' in file 'test2.bib', skipping ...

As far as I can tell, this behaviour is not described in the documentation.  It would be useful to mention it there so that users don't need to guess or experiment to see what happens.  (Consider the case where you have two .bib files that contain some overlapping entries, but for these overlapping entries, one of these .bib files has better information. There is no way of knowing what order to include the two files in without trial and error.)

It is mentioned in the biber doc that the --help output is a major source of information and there you will find the --noskipduplicates option - does that help?

Well, the exact wording is that --help "is the most useful brief source of usage information" (my emphasis), so I assumed that the PDF documentation would be more comprehensive.  So yes, now that you mention it, the --help output does help, but given what the manual says, I would never have thought it necessary to look there.

Yes, I think you are right. I have altered the biber doc for the next release.


Add overload to improve type inference

Add a function overload for useSWR that improves type inferencing for array keys.
Unfortunately a workaround for up to 4 parameters had to be made to get around missing variadic generics

Thanks @Janpot! Now it seems that TS has launched https://github.com/microsoft/TypeScript/pull/39094 (will be included in 4.0), I think we can wait and solve this elegantly once we can use it.

Any updates on this?

It's already launched in swr! Close as resolved


Feature Request: Support User Defined Functions

As of v0.8.0, duckdb supports a Python Functions API: https://duckdb.org/docs/api/python/function
It would be great to have a supported method for registering user defined functions in dbt-duckdb!
Currently, it's very difficult to register custom functions with dbt-duckdb, because everything runs in a separate thread, and each model run gets it's own connection. (Please correct me if I'm wrong!)
I've been able to proof-of-concept function registration by using dbt-core to drive our project's DBT runs, and wrapping the existing DuckDBConnectionManager.open method. This functionally gives us an on_connection_open hook to register our custom functions (example below).
Obviously this is not ideal :)
Assuming first class UDF support would be a valuable feature, I see at least three API design options:

Expose a generic on_connection_open hook that integrators can wire into. This hook would be passed the new raw DuckDB connection for customization.
Create a new opinionated function registration API that mirrors DuckDB's create_function and remove_function, without exposing the full connection object.
Add a new static "on-disk" version of custom functions definition that will be auto-discovered and registered by dbt-duckdb before model runs, a la the macros/*.sql directory

Thoughts? Happy to help implement if this is a wanted addition.
from dbt.adapters.duckdb.credentials import DuckDBCredentials
from dbt.adapters.duckdb.environments import Environment
from duckdb import DuckDBPyConnection
from duckdb.typing import VARCHAR

def custom_function() -> str:
    return "HELLO DUCKDB"

def register() -> None:
    def initialize_db(cls: Environment, credentials: DuckDBCredentials) -> DuckDBPyConnection:
        connection = old_initialize_db(credentials)
        connection.create_function("custom_function", custom_function, [], VARCHAR)
        return cast(DuckDBPyConnection, connection)

    old_initialize_db = Environment.initialize_db
    Environment.initialize_db = initialize_db  # type: ignore

Hey @amiller-gh, pleasure to meet you and thanks for raising this issue. I agree that this is something that we need to provide good support for in dbt-duckdb.
The need for this UDF support came up for me last week when I was taking another crack at #22 to create a mode that would allow dbt-duckdb to emulate another dbt adapter (in my case, dbt-snowflake ) by using sqlglot to transpile any Snowflake-specific SQL into DuckDB-friendly SQL, and then using custom types and UDFs to handle the rest of the functionality from Snowflake that sqlglot does not (can not? should not?) translate into DuckDB. I just pushed my work here onto a branch: https://github.com/jwills/dbt-duckdb/compare/jwills_snowflake_compat?expand=1 (it's not ready yet and there are other problems with this approach that may lead me to take a different tack, but it at least demonstrates most of the major touch points in the code that need to be changed in order to be able to make this work.)
Leaving that stuff aside, the place that I think we should add the support here would be in the initialize_db method defined here: https://github.com/jwills/dbt-duckdb/blob/master/dbt/adapters/duckdb/environments/init.py#L53 The good news is that we only need to set this up once per connection; when you run multiple threads in dbt-duckdb, each thread uses a "cursor" (which in DuckDB is just another instance of the same PyConnection class that is returned from duckdb.connect) that is derived from the parent PyConnection instance returned by duckdb.connect and inherits most of it's settings, including any attached fsspec filesystems and additional databases. The python UDFs get inherited as well, so we only need to declare them once, inside of that initialize_db function, for them to be picked up by all of the threads dbt uses. (I verified this in my snowflake compatibility testing and had a conversation about it in the python channel of the DuckDB Discord.)
So given that, the question is just how we should express the place that the dbt-duckdb code should look for the UDFs to create on the connection object?
We could re-use the DuckDBCredentials class to indicate e.g. a list of modules that have a well-known function defined on them (maybe something like register_udfs(conn: PyConnection) )-- which sounds closest to the macros/*.sql option you described above.
Or we could go all out with a plugin system ala pluggy for defining and registering the UDFs in the modules to include (which would be sort of a hybrid of option 1 and 2 you mentioned?)
I think that one prereq for this is to bite-the-bullet and support the ability to add a local directory inside of the dbt-duckdb project as a component of sys.path by default (maybe modules, as a sibling of models and macros? But that might be annoying for tab-completion purposes) to solve the common problem of having some code that you want to re-use across several of your python models but packaging it up into a custom python module that you need to distribute/install separately from your dbt project is too much of a PITA.)

Ah very cool re: initialize_db and the PyConnection inheritance – just updated my monkeypatch to use that instead!
Agree with you on the need to bite-the-bullet and add local directory support of some kind. I've actually already extended my monkepatched function registration POC to include super simple automatic function file discovery!
A programmatic API is very helpful for module maintainers, or people like me who manage their own DBT runner by calling dbt-core directly, but for your average project maintainer who is doing everything through the CLI, the programmatic API doesn't actually help much. Any implementation that would expose the programmatic API to CLI users, very quickly scope-creeps to becomes that on-disk implementation! May as well start there from the get-go.
(Conveniently, any API that's file-system based will also work for most programmatic integration requirements as well :)
I apologize in advance: my background is in the Javascript tooling ecosystem, so that's where most of my design inspiration will come from. But, I do really like the decorators-as-a-hook API model that tools like pluggy bring – it actually works similarly to some other libraries I maintain in the JS world. Actually, the combination of decorators, lifecycle hooks, and file-system based discovery reminds me a lot of Ember.js' addon system – it's super extensible, easy to integrate with, and scales nicely. It's also easy to add guardrails to keep users from shooting themselves in the foot.
If you do end up going with the the on-disk, decorator API route, you can actually do some pretty interesting stuff to keep the feature well-scoped, without coding the library into a corner. E.g:
# file: /dbt/duckdb/custom-function.py

from dbt.adapters.duckdb import hooks
from duckdb.typing import BIGINT
from duckdb import DuckDBPyConnection

# Functions decorated with hooks.create_function will be auto-discovered by dbt-duckb at runtime.
# The hook API footprint mirrors the native duckdb create_function, allowing full configuration.
# Optional function type arguments can be provided if mypy is not used, just like the library.
@hooks.create_function('my-custom-func', [BIGINT], BIGINT, exception_handling='return_null')
def custom-function(config, args):
    return 'HELLO DUCKDB'

# With a decorator library, you can start by rolling out a single hook: create_function.
# As additional use cases come up, you can easily extend the hooks library as needed.
@hooks.after_connect()
def custom-function(connection: DuckDBPyConnection): DuckDBPyConnection
    return connection

Alternatively, a whole lot of tooling in the frontend build ecosystem is actually moving toward a "code-as-config" model, which may apply nicely here. (E.g: Rollup, Vite, Broccoli.js, Parcel, etc). This means they basically just say "screw it" to trying to interpret what features a user might want out of a config, and instead just give them a single well-named script file to let them do whatever they want with the pipeline. Now, there's normally some amount of additional add-on / middleware API layered into here to keep the wheels on, but for the most part they forgo auto-discovery or over complicated APIs in the name of simplicity and flexibility. It's much simpler to maintain, but a whole lot harder to limit the user's control! In this case, that may mean an api more like:
# file: /duckdb.config.py

def config(connection: DuckDBPyConnection):
    # User can do whatever the heck they want here.
    # Maybe instead of only passing the connection, you pass Environment, or dbt, or whatever.
    return connection

So, I suppose I have three questions to add to the growing API design pile that came from my rambling above:

Do you have any concerns with exposing the raw duckdb connection (or more) to consumers? Technically, the goal here is to allow registration of UDFs, and a whole host of shenanigans can happen if you just expose the entire connection to make that happen. I personally find it easier to open API access up later than it is to roll back an over-permissive API change. Can totally see the argument to the contrary though – I know as a user I'd love carte-blanche access to the connection, even if I break it ;)

If you go forward with the auto-discovered, on-disk API, do you want to claim a namespace that the main DBT project will never use under sys.path for all dbt-duckdb specific components? I could totally see DBT releasing a feature called modules or functions down the line, and stomping all over that folder name in sys.path. It may be better to have everything in a folder called {sys.path}/duckdb, or even have the folder be a sibling to sys.path, to future-proof the feature-set.

How do you imagine these functions will interact with DBT packages? There are a lot of namespacing and execution order issues to figure out there...  I think there's certainly value in first releasing a version of the hooks system that only discovers hooks in the local project, but whatever the design is will likely need to take package scope names into account right?

@amiller-gh thank you for this, and apologies that it took me a couple of days to digest it and come back to this issue with some thoughts/responses to your questions.

I am temperamentally much more inclined to give users direct access to/control over as much of the logic as possible, so a configuration-as-code approach is the one that is most appealing to me. I also think it's the most practical; as the DuckDB folks keep adding new functionality at a tremendous pace (including, duh, user-defined functions) it's the easiest way for the project to keep up with the needs of the most advanced users w/o needlessly overcomplicating the lives of folks who just want to get work done.

I share your concerns about dbt coming along and stomping on whatever on-disk auto-discovery system I come up with, and (related to 3) I'm worried about the complexity that having multiple modules wanting to do modifications of the DuckDBPyConnection in an undefined order. As a concrete example, I'm thinking really hard about updating PR #22 to add a Snowflake emulation mode to dbt-duckdb, but doing that well is likely going to require defining a whole bunch of UDFs that implement functions from Snowflake that aren't transpiled by sqlglot. So immediately, I could be in a situation where the emulation mode would want to add a bunch of UDFs and the user could want to add a bunch of (potentially conflicting) UDFs, and we would want to have a clear way to indicate the relative priority of the implementations in a way that the user can control. Again, opting for config-as-code in item 1 helps solve part of the problem here, but I think we're going to need some general way -- likely in the profiles.yml config-- to specify a list of modules that need to be loaded at startup time.

There are several places where I think plugins makes sense in the context of dbt-duckdb in addition to the connection-type events, including:
a. The file formats we support for external models (currently CSV and Parquet, but it was recently made evident that you can use the new spatial stuff to read and write Excel files)
b. The ability to work with sources/seeds that require a little bit of code execution in order to work correctly, as I started working on with the plugins constructs in the 1.5.x line of dbt-duckdb but haven't really advertised/documented yet b/c I'm not sure I have it right yet.

In my ideal world, there would be a unified way to handle all of these different plugin-like constructs inside of dbt-duckdb that had the same basic setup for discoverability and implementation.

Came up with a relatively painless revamp of the plugin setup I created for loading external data and generalized it to support configuring the main DuckDBPyConnection; I updated the test_plugins with an example of it in action that I think will work for my purposes (and hopefully yours too!) -- please take a look and let me know what you think!


Keyboard manager should be able to distinguish similar keys

Description of the new feature / enhancement
Currently, Numpad Enter vs Enter are treated as though they are the same key, and can't be mapped separately.
Similarly, there doesn't seem to be any visual distinction made in the interface for Numpad . vs ., Numpad - vs -, and Numpad / vs /.
Also for Japanese keyboards, there is no visual distinction between the \ _ ＿ ろ key, to the left of right shift, and the ￥ \ | ー key, to the left of backspace. Honestly it's a miracle that this one is working at all, as Windows has typically mismanaged this key. I recommend changing the ￥ \ | ー one (scancode 7D I think) to display as ￥ and leaving \ as is.
Scenario when this would be used?
When I want to use the extra keys on my keyboard for something, and also when I'm remapping my keys and want to see what I did.
Supporting information
No response

we detect vkeys, if another app sends inputs and registers after us, we will see the bubble through. We currently don't know who sent what.
/dup #14120 and #14055 are the R&D items to see if we can't support this scenario easily.  based on that, we'll create an actual work item


Add argv information for VHPI

Get argc/argv information from vhpiTool class. Copy data to sim_info
struct and free after gpi_embed_init() call.
Enables use of cocotb.plusargs when using VHPI.
I've only tested this on Aldec Active-HDL.

Enabled VHDL for test_plusargs.
Had to update Makefile.aldec because VSIMSA ignores plusargs passed on the command line.
Moved the plusargs to runsim.tcl asim command.
VSIMSA output:
VSIMSA: Warning: Unknown option `+foo=bar', ignored.
VSIMSA: Warning: Unknown option `+test1', ignored.
VSIMSA: Warning: Unknown option `+test2', ignored.
VSIMSA: Warning: Unknown option `+options=fubar', ignored.

VHDL/Verilog/EDIF/SystemC Simulator build 10.1.3088.5434

(c) 1997-2014 Aldec, Inc. All rights reserved.

The VPI standard returns a nice s_vpi_vlog_info struct with a char **argv argument already available, and it gets passed to gpi_embed_init() easily.
VHPI requires using vhpi_iterator() and vhpi_scan() to iterate through each member of argv, so I copy them locally to a char **argv array to have the correct structure to pass to gpi_embed_init().
I could probably just set the argv pointer to the return of vhpi_get_str() rather than do a full copy of the string. I'll try that.

I cleaned the function up a lot, removed the gpi_copy_name() calls, and also made the test fail if +foo=bar is not passed into the plusargs.

Thanks for the update, merged this!


Add command to restart stopped containers and stop running containers

Add commands for users that are not familiar with docker to start a stopped container as well as stop a running container.
issue #68

Good idea!
It would be worth offering all three start, stop & restart commands, in my opinion.
I was expecting them and had to resort to native Docker commands instead 🙂

Could we change the command names? This could just be in my brain but...
To me enable means to start or switch on and disable means to stop or switch off. Take out thinks of them as to require or remove. Are we working more along the lines of composer?
takeout require mysql
takeout remove mysql

Requiring and removing sits better in my head since now I know to start/stop the containers in my docker dashboard or using the docker command line. Or as this PR introduces:
takeout start|stop mysql or takeout start|stop TO--mysql--<tag>

@jonsugar You're thinking  a little more like a Docker user here. We intentionally called them enable and disable to distance these commands from the idea of being like Composer, where you install it but it can be running or not running, because we couldn't think of a single compelling use case for why someone would need to stop a container, rather than just removing it.
However.. in the comments a case has been made! Removing a container is just fine, but booting the same container with the exact same settings later is harder if you have to remove it instead of potentially stopping it... at which point it now makes sense again to consider differentiating  starting and stopping vs adding and removing.
So, now that we're doing that, there's a bit more a Composer-like-ness, but we're also starting it when we add it and stopping when we remove it.
This was what I was thinking about last week:
takeout add
takeout stop
takeout start
takeout remove
The biggest issue with those is that running them with no parameters feels a bit off--takeout stop sort of feels like it should stop takeout, but it's not going to. But I think I'm OK with that.

All this said, if we limit this PR to start and stop, we can hopefully just merge it in and then consider renaming the other commands.
The biggest question here is, what's the most appropriate syntax to use for passing a command into start or stop? This PR shows container ID; Jon's suggest is the volume name.
I personally think the volume name is pretty clumsy so I'm inclined to go for container ID.
Any thoughts?

I would use the container ID since that is displayed in the takeout list command. Optionally, leaving the container ID off of the command provides you with a list of takeout containers that are running or stopped for the user to chose from.

I'm all for this new command set 👍
takeout add
takeout remove
takeout start
takeout stop

It makes much more sense than the actual, whether or not one is familiar with Docker!
That was the main pain point encountered during testing with a developer friend.
Thanks!

Thanks!
I'm going to add a separate issue about writing tests for this; I want to merge it but I also want to get test coverage.


WIP: use correct maximum work group size for OpenCL kernels

On macOS (Sierra / 10.12 in my case, but I guess it's more general) the maximum work group size of an OpenCL kernel depends not just on the hardware, but on the compiled kernel as well. The initial commit below shows how to get this information from the compiled pyopencl.Kernel object, i.e.:
print(func.get_work_group_info(cl.kernel_work_group_info.WORK_GROUP_SIZE, cl.get_platforms()[0].get_devices()[0]))

where func is the Kernel object.
For instance, on my laptop (Using: Intel(R) Core(TM) i5-6267U CPU @ 2.90GHz) this tells us the following maximum work group sizes for some of the examples:

vector_add.py: 128
convolution.py: 1
convolution_correct.py: 128
matmul.py: 1
reduction.py: 1
sepconv.py: 1
stencil.py: 128
vector_add_codegen.py: 128

The vector_add.py example is indeed how we found this bug, since this crashed above 128.
Fix
Before the kernel is compiled, there should be a check on this size. However, it can only be obtained from a compiled kernel...
One possible way around this is perhaps to first compile the kernel for a work group size of 1 and get the information from that. This could be (de)activated with a flag or, alternatively, it could be run only after compilation fails in a try-except.
Thoughts, suggestions?

Hi Patrick, a workgroup size of 1 for these kernels does not make any sense. Have you tried running the kernels with sizes larger than these numbers?

For completeness, this is what I get when running vector_add.py (similar for the other tests):
$ python vector_add.py
Using: Intel(R) Core(TM) i5-6267U CPU @ 2.90GHz
block_size_x=128, time=7.9663522
Error while benchmarking: vector_add_192
Error while compiling or benchmarking, see source files: /Users/pbos/sw/src/git/kernel_tuner/examples/opencl/temp_02idhtc6.c
Traceback (most recent call last):
 -- STUFF --
pyopencl.cffi_cl.LogicError: clEnqueueNDRangeKernel failed: INVALID_WORK_GROUP_SIZE

@benvanwerkhoven: as far as I understand, these numbers are maximum work group sizes. I run the tests as they are, so the work group sizes that are requested are just those from the test scripts.

Btw, I got the idea for using func.get_work_group_info from https://lists.tiker.net/pipermail/pyopencl/2010-April/000309.html

And here is another person with the same problem (but no general solution in this case): http://pyopencl.tiker.narkive.com/ejM5UnCG/benchmark-py-os-x-cannot-do-256-workers-on-cpu

Note that the pyopencl wiki is outdated on this issue (I assume he doesn't test on macOS either): https://wiki.tiker.net/OpenCLOddities
As noted in the narkive thread above, the 1 work item limit has now been partially lifted, depending on kernel.

I think that the real problem is that you are running OpenCL code intended for GPUs on your CPU. We will have a look at it tomorrow.


utils package was created

Hi Esteban,
Today I create a new directory.

Hi Fede,
Oks, wait me few minutes.. and I review it


<EMAIL_ADDRESS>decompress error

The workaround is documented but are there any plans to remove the unnecessary dependency on admin privileges?


Save suite action serialize csv profile properties incorrectly (#612)

EmptyCell, MissingCell and FirstRowHeader are included in NBi.Xml.Settings.SettingsXml.ShouldSerializeCsvProfile(). Test cases are included.

@Seddryck: The pipeline seems broken at the moment. This and my other pending PR both failed the checks. It's not my changes - I promise. :-)

I know it. I already worked on fixing the pipeline but still a few things to tune.


Cannot enter values in the Allowed object IDs: box after createing a Azure API for FHIR server

Hi,
I have a azure free subscription account and I have created a Azure API for FHIR. But even after creating the "Allowed object IDs" box is grey in color and is disabled. I am not able to enter any object Id inside it.
I tried assigning the role FHIR Data Reader to the user that is my account here. But even after that the box does not get enabled. Can you please help me out.
Attaching the screenshot.
Regards,
Pulak Kumar Swain

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 5ee09f0a-cf58-5592-29d8-7c86113eb304
Version Independent ID: 87f2d2f4-7f4d-1e50-c3ee-1e73f34be216
Content: Find identity object IDs for authentication - Azure API for FHIR
Content Source: articles/healthcare-apis/find-identity-object-ids.md
Service: healthcare-apis
Sub-service: fhir
GitHub Login: @hansenms
Microsoft Alias: mihansen

The box will not get enabled. You should just use the access control to assign roles. Box will only be enabled if you use an external tenant.

@pulakkumar599 following up on this to see if you have made it work?

@pulakkumar599 If you get a chance, please let us know if this is still an issue. I am going to move forward and proceed to close this thread since we have not heard back from you but if there are further questions regarding this matter, please comment and we will gladly continue the discussion.

Hi guys,

The box will not get enabled. You should just use the access control to assign roles. Box will only be enabled if you use an external tenant.

Hi there, Could you please explain a bit about this. Even i am also facing the same issue. What role should i assign in order to get the "Allowed object IDs" field enabled ?

@ronyv you need to point the Authority to different tenant/subscription then Allowed Object IDs will get enabled automatically, otherwise use Access Control (IAM) to assign appropriate role assignments to users or service principals.


Update hardcoded url to parameter

The error was giving an incorrect path and is hardcoded to a version. This should be the same parameter used to attempt the download.
Really small change.

Type of Change

[ ] Bug fix (non-breaking change, fixes #)

Great patch, thanks so much for the fix @slimrichard ! 🌮


Fix type mismatch on is_master result

Fixes a monitoring bug found by @saghm while building the import/export tool -- the server and topology information was not being updated, because the IsMasterResult would return ok as a FloatingPoint instead of an I64 like the automated tests.

Merging; super small fix.


Under Maintenance

Hey there, I know in the GitHub mainpage it said that WabbaJack would be updated if this happens but it's been a bit over two weeks and I was hoping to install this, is there a way to bypass or override the WabbaJack "under Maintenance" gate?

My apologies @darkmaniac7
This has now been fixed as of v1.5.0


add CBAM attention mechanism block within Resnet backbone

Hello , I would like to thank you all for such a great work.
I am using mmdetection now since a round 1 year. and i like this environment and platform.
I would like to as about adding the general attention mechanism block (CBAM ) within my algorithm or backbone to improve the detection.
I used the spatial attention mechanism represented by empirical_attention within my config , and i  use the instruction plugins
I am still wounding how can I add the CBAM in my model, can you please help me in that?
i already have some resnet_CBAM model ready but in tensorflow. keras , what shall i do to add such block easily within mmdetection either backbone or neck structure.
looking forward to your answer.

I found this model of resnet_cbam model ready in pytorch
https://github.com/luuuyi/CBAM.PyTorch/blob/master/model/resnet_cbam.py
what will be the way to add it in my config and use it within mmdetection library ?

Maybe this issue can give you some advice.


Compilation error involving interp and sort.

This does not compile, but strangely, removing the useless line about Z makes it compile!
#pythran export test()
import numpy as np

def A(X, factor):
    Z = X #<- remove this useless line also makes it compile.
    W = np.interp(X, X, X)
    U = np.sort(factor * W) #< factor causes the issue here.
    return U

def test():
    A(np.random.random(10), 1)

Dude, thank you SO MUCH!


Is delta_s lower bound in dp_st_graph.cc  correct?

Hello,
in dp_st_graph.cc

*next_lowest_row = point.index.s() - static_cast(delta_s_lower_bound / unit_s_);

I was wondering why you substract  and do not add the static casted value.
I think s can only increasewhen driving forward and the performance of the dp_st_speed algorithm would be better if the
number of reachable nodes is smaller.
Greetings

It is fixed in #4921


Require path resolution, Resolves #3

This will add the feature as outlined in issue #3.

Awesome, thanks @TatumCreative! Sorry for the delay, it's been a crazy week :grimacing:
p.s. possible I've already said this, but let me know if you'd like me to add you to the stackgl org! No obligations, but certainly useful :)

No problem! Thanks for the merge. It's nice to get to know the internals of glslify a bit better. Also I'd love to get added to the stackgl org. Part of my reasoning with making glam is to have a path for using the ecosystem and contributing back.

No worries! It's great having Glam to fill that gap in stack.gl's user-friendliness, and having that feed back through into the ecosystem. Surprised we hadn't added you already — done! p.s. congrats again on the new gig :grin:


feat: Idle

ok

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-9.4%) to 47.913%

Totals

Change from base Build<PHONE_NUMBER>:
-9.4%

Covered Lines:
482

Relevant Lines:
1006

💛  - Coveralls


feat(Parsers) Implement Parsers

Current coverage is 85.13% (diff: 80.00%)

Merging #234 into master will decrease coverage by 0.01%

@@             master       #234   diff @@
==========================================
  Files            17         17
  Lines           875        881     +6
  Methods         199        201     +2
  Messages          0          0
  Branches        273        275     +2
==========================================
+ Hits            745        750     +5
- Misses          130        131     +1
  Partials          0          0

Powered by Codecov. Last update e776b48...335de44


Remove the insist require

Remove the insist require as it might not be used, so it's something the user of this gem should require and include whenever s/he wants to use this lib.

LGTM (its not even in the gemspec! only worked because devutils has always been a part of others that did include it as a dependency)

LGTM


Adding eraser tool

Issue Title: [Feature Request]: Add erasing feature in the draw section
closes #404
Type of change ☑
What sort of change have you made:

[x] New feature (non-breaking change which adds functionality)

Checklist: ☑

[x] My code follows the guidelines of this project.
[x] I have performed a self-review of my own code.
[x] I have made corresponding changes to the documentation.

@AnshikaAg03 The eraser is not working, do add a recording.

@AnshikaAg03 Any updates?

Hello @singodiyashubham87  is this issue of eraser tool solved, or will you please assign me the issue, or else am I supposed to raise a new issue for the same eraser functionality and work on that.