[CT-1215] [Bug] support indirect strings in ref and source for Python models

Is this a new bug in dbt-core?

[X] I believe this is a new bug in dbt-core
[X] I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior
EDIT: see comment below. this does not fail due to iterables/lists/dictionaries, but rather us statically analyzing the code and being unable to indirectly use strings via variables. thus an even simpler reproduction:
def model(dbt, session):

    model_name = "orders"

    return dbt.ref(model_name)

Take debug1.py and debug2.py, built off of jaffle_shop, as simple examples:
def model(dbt, session):

    model_names = ["orders", "customers"]
    models = []

    for model_name in model_names:
        models.extend(dbt.ref(model_name))

    return models[0]

def model(dbt, session):

    models = {
        "orders": None,
        "customers": None
    }

    for model_name in models:
        models[model_name] = dbt.ref(model_name)

    return models["orders"]

I would expect both of these examples to work, but they fail with parsing error below
Expected Behavior
above works
Steps To Reproduce
gh repo clone dbt-labs/jaffle_shop
cd jaffle_shop
dbt seed
dbt run
code debug1.py debug2.py

copy code above into those files
dbt run -s debug1
dbt run -s debug2

Relevant log output
@lostmygithubaccount ➜ .../learn-dbt-py/demos/jaffle_shops/snowpark (main ✗) $ dbt run -s debug1
16:32:45  Running with dbt=1.3.0-b2
16:32:45  Unable to do partial parsing because profile has changed
16:32:46  Encountered an error:
Parsing Error in model debug1 (models/debug1.py)
  malformed node or string on line 8: <ast.Name object at 0x7f262340ba60>
16:32:46  Traceback (most recent call last):
  File "/workspaces/codyspace/dbt-core/core/dbt/parser/models.py", line 88, in _safe_eval
    return ast.literal_eval(node)
  File "/usr/local/lib/python3.10/ast.py", line 108, in literal_eval
    return _convert(node_or_string)
  File "/usr/local/lib/python3.10/ast.py", line 107, in _convert
    return _convert_signed_num(node)
  File "/usr/local/lib/python3.10/ast.py", line 81, in _convert_signed_num
    return _convert_num(node)
  File "/usr/local/lib/python3.10/ast.py", line 72, in _convert_num
    _raise_malformed_node(node)
  File "/usr/local/lib/python3.10/ast.py", line 69, in _raise_malformed_node
    raise ValueError(msg + f': {node!r}')
ValueError: malformed node or string on line 8: <ast.Name object at 0x7f262340ba60>

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/workspaces/codyspace/dbt-core/core/dbt/main.py", line 129, in main
    results, succeeded = handle_and_check(args)
  File "/workspaces/codyspace/dbt-core/core/dbt/main.py", line 191, in handle_and_check
    task, res = run_from_args(parsed)
  File "/workspaces/codyspace/dbt-core/core/dbt/main.py", line 238, in run_from_args
    results = task.run()
  File "/workspaces/codyspace/dbt-core/core/dbt/task/runnable.py", line 453, in run
    self._runtime_initialize()
  File "/workspaces/codyspace/dbt-core/core/dbt/task/runnable.py", line 161, in _runtime_initialize
    super()._runtime_initialize()
  File "/workspaces/codyspace/dbt-core/core/dbt/task/runnable.py", line 94, in _runtime_initialize
    self.load_manifest()
  File "/workspaces/codyspace/dbt-core/core/dbt/task/runnable.py", line 81, in load_manifest
    self.manifest = ManifestLoader.get_full_manifest(self.config)
  File "/workspaces/codyspace/dbt-core/core/dbt/parser/manifest.py", line 220, in get_full_manifest
    manifest = loader.load()
  File "/workspaces/codyspace/dbt-core/core/dbt/parser/manifest.py", line 350, in load
    self.parse_project(
  File "/workspaces/codyspace/dbt-core/core/dbt/parser/manifest.py", line 475, in parse_project
    parser.parse_file(block)
  File "/workspaces/codyspace/dbt-core/core/dbt/parser/base.py", line 414, in parse_file
    self.parse_node(file_block)
  File "/workspaces/codyspace/dbt-core/core/dbt/parser/base.py", line 388, in parse_node
    self.render_update(node, config)
  File "/workspaces/codyspace/dbt-core/core/dbt/parser/models.py", line 226, in render_update
    self.parse_python_model(node, config, context)
  File "/workspaces/codyspace/dbt-core/core/dbt/parser/models.py", line 207, in parse_python_model
    dbtParser.visit(tree)
  File "/usr/local/lib/python3.10/ast.py", line 410, in visit
    return visitor(node)
  File "/usr/local/lib/python3.10/ast.py", line 418, in generic_visit
    self.visit(item)
  File "/usr/local/lib/python3.10/ast.py", line 410, in visit
    return visitor(node)
  File "/usr/local/lib/python3.10/ast.py", line 418, in generic_visit
    self.visit(item)
  File "/usr/local/lib/python3.10/ast.py", line 410, in visit
    return visitor(node)
  File "/usr/local/lib/python3.10/ast.py", line 418, in generic_visit
    self.visit(item)
  File "/usr/local/lib/python3.10/ast.py", line 410, in visit
    return visitor(node)
  File "/usr/local/lib/python3.10/ast.py", line 420, in generic_visit
    self.visit(value)
  File "/usr/local/lib/python3.10/ast.py", line 410, in visit
    return visitor(node)
  File "/workspaces/codyspace/dbt-core/core/dbt/parser/models.py", line 129, in visit_Call
    self.visit_Call(obj)
  File "/workspaces/codyspace/dbt-core/core/dbt/parser/models.py", line 122, in visit_Call
    args, kwargs = self._get_call_literals(node)
  File "/workspaces/codyspace/dbt-core/core/dbt/parser/models.py", line 104, in _get_call_literals
    rendered = self._safe_eval(arg)
  File "/workspaces/codyspace/dbt-core/core/dbt/parser/models.py", line 91, in _safe_eval
    raise ParsingException(msg, node=self.dbt_node) from exc
dbt.exceptions.ParsingException: Parsing Error in model debug1 (models/debug1.py)
  malformed node or string on line 8: <ast.Name object at 0x7f262340ba60>

@lostmygithubaccount ➜ .../learn-dbt-py/demos/jaffle_shops/snowpark (main ✗) $ dbt run -s debug2
16:35:55  Running with dbt=1.3.0-b2
16:35:55  Unable to do partial parsing because profile has changed
16:35:56  Encountered an error:
Parsing Error in model debug2 (models/debug2.py)
  malformed node or string on line 9: <ast.Name object at 0x7f2f0ce29780>
16:35:56  Traceback (most recent call last):
  File "/workspaces/codyspace/dbt-core/core/dbt/parser/models.py", line 88, in _safe_eval
    return ast.literal_eval(node)
  File "/usr/local/lib/python3.10/ast.py", line 108, in literal_eval
    return _convert(node_or_string)
  File "/usr/local/lib/python3.10/ast.py", line 107, in _convert
    return _convert_signed_num(node)
  File "/usr/local/lib/python3.10/ast.py", line 81, in _convert_signed_num
    return _convert_num(node)
  File "/usr/local/lib/python3.10/ast.py", line 72, in _convert_num
    _raise_malformed_node(node)
  File "/usr/local/lib/python3.10/ast.py", line 69, in _raise_malformed_node
    raise ValueError(msg + f': {node!r}')
ValueError: malformed node or string on line 9: <ast.Name object at 0x7f2f0ce29780>

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/workspaces/codyspace/dbt-core/core/dbt/main.py", line 129, in main
    results, succeeded = handle_and_check(args)
  File "/workspaces/codyspace/dbt-core/core/dbt/main.py", line 191, in handle_and_check
    task, res = run_from_args(parsed)
  File "/workspaces/codyspace/dbt-core/core/dbt/main.py", line 238, in run_from_args
    results = task.run()
  File "/workspaces/codyspace/dbt-core/core/dbt/task/runnable.py", line 453, in run
    self._runtime_initialize()
  File "/workspaces/codyspace/dbt-core/core/dbt/task/runnable.py", line 161, in _runtime_initialize
    super()._runtime_initialize()
  File "/workspaces/codyspace/dbt-core/core/dbt/task/runnable.py", line 94, in _runtime_initialize
    self.load_manifest()
  File "/workspaces/codyspace/dbt-core/core/dbt/task/runnable.py", line 81, in load_manifest
    self.manifest = ManifestLoader.get_full_manifest(self.config)
  File "/workspaces/codyspace/dbt-core/core/dbt/parser/manifest.py", line 220, in get_full_manifest
    manifest = loader.load()
  File "/workspaces/codyspace/dbt-core/core/dbt/parser/manifest.py", line 350, in load
    self.parse_project(
  File "/workspaces/codyspace/dbt-core/core/dbt/parser/manifest.py", line 475, in parse_project
    parser.parse_file(block)
  File "/workspaces/codyspace/dbt-core/core/dbt/parser/base.py", line 414, in parse_file
    self.parse_node(file_block)
  File "/workspaces/codyspace/dbt-core/core/dbt/parser/base.py", line 388, in parse_node
    self.render_update(node, config)
  File "/workspaces/codyspace/dbt-core/core/dbt/parser/models.py", line 226, in render_update
    self.parse_python_model(node, config, context)
  File "/workspaces/codyspace/dbt-core/core/dbt/parser/models.py", line 207, in parse_python_model
    dbtParser.visit(tree)
  File "/usr/local/lib/python3.10/ast.py", line 410, in visit
    return visitor(node)
  File "/usr/local/lib/python3.10/ast.py", line 418, in generic_visit
    self.visit(item)
  File "/usr/local/lib/python3.10/ast.py", line 410, in visit
    return visitor(node)
  File "/usr/local/lib/python3.10/ast.py", line 418, in generic_visit
    self.visit(item)
  File "/usr/local/lib/python3.10/ast.py", line 410, in visit
    return visitor(node)
  File "/usr/local/lib/python3.10/ast.py", line 418, in generic_visit
    self.visit(item)
  File "/usr/local/lib/python3.10/ast.py", line 410, in visit
    return visitor(node)
  File "/usr/local/lib/python3.10/ast.py", line 420, in generic_visit
    self.visit(value)
  File "/usr/local/lib/python3.10/ast.py", line 410, in visit
    return visitor(node)
  File "/workspaces/codyspace/dbt-core/core/dbt/parser/models.py", line 122, in visit_Call
    args, kwargs = self._get_call_literals(node)
  File "/workspaces/codyspace/dbt-core/core/dbt/parser/models.py", line 104, in _get_call_literals
    rendered = self._safe_eval(arg)
  File "/workspaces/codyspace/dbt-core/core/dbt/parser/models.py", line 91, in _safe_eval
    raise ParsingException(msg, node=self.dbt_node) from exc
dbt.exceptions.ParsingException: Parsing Error in model debug2 (models/debug2.py)
  malformed node or string on line 9: <ast.Name object at 0x7f2f0ce29780>

Environment
- OS: linux
- Python: 3.10
- dbt: off of main branch (1.3)

Which database adapter are you using with dbt?
snowflake
Additional Context
No response

this is a bit complicated -- even this would fail:
def model(dbt, session):

    model_name = "orders"

    return dbt.ref(model_name)

intermediate step could be more clear error message

We raised better error message to tell folks not do this, but still want to leave this open to revisit and see if other better options we would like to provide in the future

Interested in this, is there a plan to support dynamic values in the future?

Hi @ChenyuLInx @lostmygithubaccount , I am currently developing on DBT/Databricks and is facing this issue.
Context:
My Databricks catalog is designed in a way where my client companies are all cleanly separated by schemas (schema names are the companies' identifiers and each company has the exact same table structure). This means that I require being able to run DBT source with dynamic source_name arguments. (so that the DBT run can reference/transform the correct client company's schema in Databricks).
I would like to understand a little more on why this feature is not supported/encouraged.

Do we have an update on this? This feature can add a lot flexbility to many python data processes. E.g. allowing for f-string input can be great enough. Or is there any workaround currently for Python models?

Interested in this, is there a plan to support dynamic values in the future?


config.js in index.html not found (fixed with #20 but bug persists)

While loading in Chrome (and IE) both report missing config.js. The link is there but there's no file with that name in the repo. I was able to find a simple, easy fix, and sent in a PR #20 but both Chrome and IE do not authenticate correctly. Chrome brings up the dialog box and tries to contact ADN, after that process I get this error message
Error
Please contact the website that sent you here and let them know that there is a problem with the Redirect URI.
IE11 does not bring up the dialog box to login, in fact IE11 just keeps spinning. Once I know Chrome works correctly I'll take a closer look at IE11 settings, but that shouldn't matter since IE11 works fine with nice when hosted at https://nice.social/

Did you put your key and redirect (and based on 15E081 site name too) into config.js? It's deliberately not in the repo, the idea you copy config.sample.js and rename it, add your secret keys and it's in .gitignore so it won't be committed.
Unless I'm misinterpreting your problem.

This appears to be a configuration issue. Issue #23 will include a more interactive solution for this problem.
Mark: do let me know if Matt and I are misunderstanding the issue. So long as a person creates the config.js file and puts it in the web root, the site should load properly, even with dummy data.


git12 initial code

Signed-off-by: David Radley<EMAIL_ADDRESS>

please don’t merge yet. the sonatype team are having a look


Is the regular onBind() ever called now we overrode ..withPayload()?

If not remove the unneeded one, add a few tests.
cc @Teovald , thanks for bringing my attention to this question :)

https://github.com/Genius/groupie/pull/32

you are welcome :)


Test/release 1 14 live preprod us east2 (do not merge)

Allure Report
allure-report-publisher generated test report!

processReports: ❌ test report for b76ab825

passed
failed
skipped
flaky
total
result

Total
30
2
0
0
32
❌


Use SKV Partial Update in TPC-C Benchmark

In the tpcc benchmark for SKV, change full writes into partial writes where possible. The changes need to be made in src/k2/cmd/tpcc/transactions.h. If you need a reference on what the transactions are supposed to do you can look at the specification: http://www.tpc.org/tpc_documents_current_versions/pdf/tpc-c_v5.11.0.pdf
You can use test/integration/test_k23si_tpcc.sh locally test if the changes work. For performance testing, I recently updated the scripts and README in the cluster directory. You will need to modify the configuration to work on 3 machines, with fewer cores used and fewer TPC-C warehouses used. You don't need to do performance testing for this task but getting setup for performance testing will be very useful soon.

TPCC transactions during locally tpcc test has got some problems while using the newest committed chogori version (#64 ). Data load phase can be done successfully, but during tpcc transaction phase, no writeRow can be done successfully.
[0001:00:58:14.118.506]-tpcc_client-(0) [INFO] [/home/fengyazhi/chogori-platform/src/k2/cmd/tpcc/tpcc_client.cpp:300 @operator()]completedTxns=0(0 per sec)
[0001:00:58:14.118.550]-tpcc_client-(0) [INFO] [/home/fengyazhi/chogori-platform/src/k2/cmd/tpcc/tpcc_client.cpp:301 @operator()]read ops 4161.5 per sec)
[0001:00:58:14.118.557]-tpcc_client-(0) [INFO] [/home/fengyazhi/chogori-platform/src/k2/cmd/tpcc/tpcc_client.cpp:302 @operator()]write ops 0.3 per sec)
[0001:00:58:14.118.563]-tpcc_client-(0) [INFO] [/home/fengyazhi/chogori-platform/src/k2/cmd/tpcc/tpcc_client.cpp:121 @operator()]Done with benchmark
[0001:00:58:14.118.587]-tpcc_client-(0) [INFO] [/home/fengyazhi/chogori-platform/src/k2/appbase/Appbase.cpp:136 @operator()]graceful stop user applets

The second newest version (#57 )  does not have this problem. So I change full writes into partial writes based on the #57 version for now. And I will refresh the code if this problem is fixed and then make a pull request.

Hi Yazhi, sorry about that. I found and fixed the problem and it is in PR now. I will let you know when it is merged.

Also, when you are done with this task or it is in PR you can work on https://github.com/futurewei-cloud/chogori-platform/issues/58 Let me know if you have any questions.

The TPCC bug is fixed now.

Today, I've been working on some cooperative projects issues and I've had a few meetings. So I could not make a pull request. I'll do it as soon as possible.


How to block nuisance pop-ups?

I would really like to know how to permanently disable these pop-ups that are bothering me, and I would also like to know who allows these things to pop up constantly.

This issue is caused by an extension, please file it with the repository (or contact) the extension has linked in its overview in VS Code or the marketplace for VS Code. See also our issue reporting guidelines. If you don't know which extension is causing the problem, you can run Help: Start extension bisect from the command palette (F1) to help identify the problem extension.
Happy Coding!


Make plugin work under Linux / Mono

... mono implementation of SerialPort have limitations - work around them if platform is not windows.

This is done


Permissions on Sendinblue API

Hello,
Would it be possible to create a Sendinblue API and then set permissions on it?
For example, I want to create a Sendinblue API but I want to disable the following abilities:

Export\import of emails
Delete operations

What do you think about these additions?
Thank you.

Hi @webaddicto
Thanks for contacting us.
This facility is not available with SendinBlue APIs.
Regards
SendinBlue Team


Consider communication statistics while choosing peers in pool

It would be nice to accumulate response statistics (latency, speed, success rate) in client pool and try to load-balance request stream, when possible. Exposing such stats as metrics at the uppr level would also be a very useful feature.

Dup #283


Texture handle is buggy in WASM builds

When compiling to WASM I see different behaviour in texture indexing, it seems that at index 0 we have the last texture from texture map (last square from tiles.png). When compiling to linux binary I do not see this issue, I can index all the textures from map.
It is possible to reproduce this behaviour running the basic example:

Do cargo run --example basic (assuming you're on linux) - you will see a lot of green tiles (first square from tiles.png)
Do cargo run --target wasm32-unknown-unknown --example basic and go to http://<IP_ADDRESS>:1334/ - you will see a lot of white tiles (last square from tiles.png)

For the second point to work you need to have:
In ~/.cargo/config.toml:
[target.wasm32-unknown-unknown]
runner = "wasm-server-runner"

And you need to install wasm-server-runner with cargo install wasm-server-runner.
If you try to change index of the tile to something more then 0 then it will not display correctly under WASM (just gray empty space will be shown) but will behave correctly on linux build.

I'm not sure if this is an bevy_ecs_tilemap bug or just an bevy bug. It seems to me that the problem lies in asset handling and asset_server.load("tiles.png"); method but I'm not sure how to prove it.

Ok, sorry for this, I've found out that I missed that I need to enable atlas feature. With it everything works fine.


Port #4383 to develop with tests

Fix #4383

Coverage remained the same at 90.122% when pulling 4b7d1887a7163071f2f1e25d565ddda2918b4aa5 on yakky:merge/4383 into f3807c99ca4a1d77d6c137916bdcae09fbb6c8b7 on divio:develop.

Coverage remained the same at 90.122% when pulling 4b7d1887a7163071f2f1e25d565ddda2918b4aa5 on yakky:merge/4383 into f3807c99ca4a1d77d6c137916bdcae09fbb6c8b7 on divio:develop.


Build error on macos

I cloned the head of master branch and tried to build 'products/platforms/darwin' on macos.
I get the following warning and error:
Renjis-MacBook-Pro:build_wamr renji$ make
Scanning dependencies of target libiwasm
[  1%] Building C object CMakeFiles/libiwasm.dir/Users/renji/tmp/wasm-micro-runtime/core/shared/platform/darwin/bh_assert.c.o
[  3%] Building C object CMakeFiles/libiwasm.dir/Users/renji/tmp/wasm-micro-runtime/core/shared/platform/darwin/bh_definition.c.o
/Users/renji/tmp/wasm-micro-runtime/core/shared/platform/darwin/bh_definition.c:8:9: warning: 'RSIZE_MAX' macro redefined
      [-Wmacro-redefined]
#define RSIZE_MAX 0x7FFFFFFF
        ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/stdint.h:176:9: note:
      previous definition is here
#define RSIZE_MAX         (SIZE_MAX >> 1)
        ^
1 warning generated.
[  5%] Building C object CMakeFiles/libiwasm.dir/Users/renji/tmp/wasm-micro-runtime/core/shared/platform/darwin/bh_platform.c.o
/Users/renji/tmp/wasm-micro-runtime/core/shared/platform/darwin/bh_platform.c:142:34: error: use of undeclared identifier
      'MADV_HUGEPAGE'
        if (!madvise(addr, size, MADV_HUGEPAGE))
                                 ^
1 error generated.
make[2]: *** [CMakeFiles/libiwasm.dir/Users/renji/tmp/wasm-micro-runtime/core/shared/platform/darwin/bh_platform.c.o] Error 1
make[1]: *** [CMakeFiles/libiwasm.dir/all] Error 2
make: *** [all] Error 2

CMAKE files were generated as follows:
Renjis-MacBook-Pro:build_wamr renji$ cmake ../wasm-micro-runtime/product-mini/platforms/darwin/
-- The C compiler identification is AppleClang <IP_ADDRESS>00033
-- The CXX compiler identification is AppleClang <IP_ADDRESS>00033
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Build Configurations:
     Build as target X86_64
     CMAKE_BUILD_TYPE Release
     WAMR Interpreter enabled
     WAMR AOT enabled
     WAMR JIT disbled
     Libc builtin enabled
     Libc WASI disbled
-- The ASM compiler identification is Clang
-- Found assembler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/renji/tmp/build_wamr

CLANG version is:
Renjis-MacBook-Pro:build_wamr renji$ clang --version
Apple clang version 11.0.0 (clang-1<IP_ADDRESS>)
Target: x86_64-apple-darwin19.3.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

OS is:
macOS Catalina, version 10.15.3

@renjipanicker @jondong by any chance, could you help to identify the root cause and fix it? We don't have MacOS development environment at this moment.

Done, pull request submitted.

Awesome! @renjipanicker


ARROW-15895: [R] R docs version switcher disappears & reappears with back button on Chrome

I haven't tested this yet as I'm not entirely sure how to.

Steps to replicate:

Open docs (arrow.apache.org/docs/r/)
Navigate to version 5 via the dropdown
Press back - dropdown should be there and displaying version 5 (this is wrong)

If my fix works, after it is applied, once the user clicks back, the latest version should be selected.

The fix works on Safari (tested with local override in the web inspector).


CASMINST-5557: Linting

Description
Just linting and one command correction (fixing a grep command that would not work as written). No content changes.
Checklist Before Merging

[X] If I added any command snippets, the steps they belong to follow the prompt conventions (see example).
[X] If I added a new directory, I also updated .github/CODEOWNERS with the corresponding team in Cray-HPE.
[X] My commits or Pull-Request Title contain my JIRA information, or I do not have a JIRA.

/backport release/1.3


Liberium v2 development

⚠ Discontinued - no longer maintatined ⚠
This project is no longer maintained.
I (@panda885), the author and maintainer of this plugin, am no longer interested in creating plugins for Minecraft and therefore have no motivation to maintain this plugin.
If you are interested in this plugin and would like to develop it further, feel free to can contact me.
What's this?
Here is all the information about the v2 development.
The information here is work in progress, please use the Liberium v2 Project to show available issues.

Discontinued.


Design fixes for new @ UI

Various design fixes.
Test plan

CI

I should get a proper Code Search icon in there instead of the SG logo in color, but left it out of this PR

very good, thanks! adopted in my PR


syscall.Ptrace* always start in new thread, causing the tracee untraceable

What version of Go are you using (go version)?

$ go version
go version go1.13.8 linux/amd64

Does this issue reproduce with the latest release?
Yes
What operating system and processor architecture are you using (go env)?
go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/jm33/.cache/go-build"
GOENV="/home/jm33/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/jm33/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/lib/go-1.13"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go-1.13/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build432196136=/tmp/go-build -gno-record-gcc-switches"

What did you do?
// attach to a running process
err = syscall.PtraceAttach(pid)
if err != nil {
	return fmt.Errorf("ptrace attach: %v", err)
}

// read registers
regs := &syscall.PtraceRegs{}
err = syscall.PtraceGetRegs(pid, regs)
if err != nil {
	return fmt.Errorf("my pid is %d, reading regs from %d: %v", os.Getpid(), pid, err)
}

What did you expect to see?
Registers are successfully returned
What did you see instead?
my pid is 4018, reading regs from 4028: no such process from syscall.PtraceGetRegs(pid, regs)
The issue

The process I was trying to attach to is attachable, as syscall.PtraceAttach(pid) returns no error
Checking the /proc/$(pidof tracee)/status, found a TracerPid matching a non-existent process, I think it's a thread (which has already exited) of my go program
Putting runtime.LockOSThread() at the beginning of my function doesn't solve this issue, TracerPid still doesn't match my current go process, which it should

Adding:
runtime.LockOSThread()
defer runtime.UnlockOSThread()

right before syscall.PtraceAttach(pid) (instead of at the top of calling function) seems to fix this issue

Adding:
runtime.LockOSThread()
defer runtime.UnlockOSThread()

right before syscall.PtraceAttach(pid) (instead of at the top of calling function) seems to fix this issue

Adding:
runtime.LockOSThread()
defer runtime.UnlockOSThread()

right before syscall.PtraceAttach(pid) (instead of at the top of calling function) seems to fix this issue

Adding:
runtime.LockOSThread()
defer runtime.UnlockOSThread()

right before syscall.PtraceAttach(pid) (instead of at the top of calling function) seems to fix this issue


fix KPDB builds on main

build fails on main due to

lack of BUILD_ENGINE_SERVER envar because it's needed in build_env_setup.sh (build log)
an invalid corrections filename (build log)

successful build on this branch

@damonmcc confirming this was successfully built before I proceed with QA

@damonmcc confirming ztl build was successful before I proceed with QA.

@caseysmithpgh yup! the build succeeded overnight here

@damonmcc Mornin! I'm currently doing qa of /db-zoningtaxlots/draft/main/. I noticed a lot-based issue, which led me to check the source_data_versions.csv. Seems like the dtm used for this build is dated 12/4 but the dtm uploaded to edm-publishing/datasets/dof_dtm_tax_lot_polygons was 11/24 version. I believe this is the version that should be used in the ztl build, is something afoot with source data? cc @fvankrieken

Our "version" is the date that the ztl dataloading step was run. I actually have an issue around this - #396 - figure that our specific versions should align with yours, not be the date of archival

@fvankrieken Noted, thanks. Crisis avoided, it was sparked an issue in my qa environment, no source data concerns for dtm.


ignore a 3rd party libray in cpplint

Description
Need this to be merged so my other PR #12637 can pass CI lint.

@mxnet-label-bot [pr-awaiting-review]

Moving the ctc_include to 3rdparty as @szha suggested. Closing this PR.


Fill cell with hex code color

Hi, is possible to fill a Cell with an hex code color?

You can use Color.decode for that:
TextCell.builder()
  .text("...")
  .textColor(Color.decode("#ff00ff"))
  .backgroundColor(Color.decode("#111111"))
  .build()`

thanks again, is there another way to contact you without opening issues?

Hey, I suggest opening a question on stackoverflow tagging it with pdfbox and maybe mentioning easytable in the title or so. Then others can answer as well and the solution can be found easily, too.


Set up primary window and get examples working

Adds a new system to Bevy's render app to set up the primary window. As-is, this will need more since it's setting up the pipeline every time it's called, but it gets Egui rendering again.
I also removed the node edge with ui_pass_driver since it doesn't exist anymore. It seems to work for me, but I can't make any guarantees on that.

I figured I should work on this more, so now the pipeline's only set up once and all the examples are working again.
Getting the ctx_mut function working again will need more substantial API changes. Here's how I'm working around it in my own project:

#[derive(SystemParam)]
pub(crate) struct EguiHelper<'w, 's> {
    egui: ResMut<'w, EguiContext>,
    window: Query<'w, 's, Entity, With<PrimaryWindow>>,
}

impl<'w, 's> EguiHelper<'w, 's> {
    pub fn ctx(&mut self) -> &egui::Context {
        self.egui
            .ctx_for_window_mut(self.window.get_single().unwrap())
    }
}

@Zyllian Sorry I totally missed this PR. From what I can tell the recent updates to the PR supersede this? (making egui::Context a component)
I did see your note about removing ui_pass_driver and agree.

@DGriffin91 No worries! Feel free to close this if you don't need it.


Project status

Hi all,
First of all, thanks for kopf, it's refreshing to see Python can be considered as legitimate as Go in this arena.
I wondered however, what's the status of the project? I see a 0.11 version so I can assume it's still in progress, but I also see a few opened enhanchements that I feel will help the overall reliability of running kopf. So, what I mean is, what is the status of the project in general? Is there a roadmap for this year by any chance?
Knowing this could help making a decision towards using it now in production or not :)
Cheers,

@Lawouach Thanks for your feedback ;-)
The project is fully alive, but still in the proof-of-concept stage (0.x versions). What is missing for the 1.0 release is:

Tests. A lot of tests. #13 — I would say 80% is done (merged & in my fork).
Silent spy-handlers (#30) and built-in resources support.
And some less critical issues from milestone 1.0.

I work hard on that. The framework was published ~20.03.19, the initial estimation was 2 months — by ~20.05.19 (due). But I was quite occupied with other tasks in the past month, so the new estimation of 1.0 is maybe one more month later — by ~30.06.19. If we are lucky, maybe by mid-June.
The roadmap beyond 1.0 release (i.e. coverage and stability) is not defined, any ideas are welcome (e.g. as issues). I guess, I will think of extending Kopf even further: to handle the validations via AdmissionControllers (#55), which requires some internal web-server running for hooks (a new way of interacting with the cluster), and basically all issues not in the milestones yet — to improve the ease of use and to expand to scope of application.
The decision will be done based on the feedback from the early adopters, if any. If you have any specific concerns for the usage of the framework — feel free to share, they will be addressed.
Regarding the long-term, I have really big plans on Kopf to bring the huge & diverse world of Python to the growing world of K8s — K8s ecosystem can benefit from this. But let's not look too far (yet). One step at a time.

Thank you @nolar for the clear and honest answer :)
I think knowing a 1.0 could be released in the next few weeks/months is great by me. It reassures me.
I'll be starting working using kopf more often now and will report as I see problems .
Cheers,

+1 great work!


'RECURSIVE' gets lost when 'withRecursive' is used after a 'with'

Hey,
please look at: playground link
In the compiled query, the CTE "ShoulBeRecursiveButIsnt" should have the RECURSIVE keyword, but doesn't. If you swap the CTE declarations, the keyword is correctly there.
My guess is the problem is somewhere here: https://github.com/kysely-org/kysely/blob/d9e06ab6568de976af675df4794a49e73fe648aa/src/query-creator.ts#L483
but right now I'm unable to check it out and try it for myself.

The whole WITH statement is either recursive or not. That's not a per CTE option. Kysely API is a bit suboptimal for this since currently the first CTE determines the recursiveness of the whole WITH statement. I don't know how we could improve this without complicating the API.

Ohh, I see, thank you!


support for Font-Awesome

Hi,
As I'm already using font awesome for icons (and I think the majority are as well), I'm think to make use of the already built css style for my promise-buttons.  I tried to configure the template as the following:
.config(function (angularPromiseButtonsProvider)
        {
            angularPromiseButtonsProvider.extendConfig({
                spinnerTpl: '<i class="fa fa-spinner fa-pulse"></i>'
            });
        });

The issue is that the icon keeps displayed and spin initially.
I expect the template is going to be shown only when the promise is fired/starts (but not initially and before the promise is run).
Dose it make sense? Is something missing?

Hey there! Thanks for posting. If you post a plunkr I can have a look at it, but I don't think that this is really related to angular-promise-buttons as it is just an css issue.

Hi!
Please refer to this plunkr for better understanding.

You have to set the icon to not display via css and then display it by targeting it via .is-loading parent selector.

I know that but I mean if this can be controlled inside the directive.
Straightforward, can the directive display spinnerTpl only when the promise is fired?
(something like: ng-if="isLoading")

No.


Develop

@tig
new version 0.2.1  is under way, most important changes

changed return value enable() functions to bool
made index(stream) public
added stream(index)
added free() == free slots
minor refactor / optimizations

none of these should break existing code


Remove Localize calls for persistent damage

Will want to do a follow up to add persistent damage proper to a lot of these, but the effects for tiger stance, daemon form, insect form, and monstrosity form all used notes for the persistent damage still


Does not work with Webpack

I'm using typescript with webpack, and when instantiating Dexie, the export that's used is relationships.default.
import Dexie from 'dexie';
import relationships from 'dexie-relationships'

export class AppDatabase extends Dexie {
    constructor(){
        super('AppDatabase', {addons: [relationships]});
    }
}

After compilation and packing,  {addons: [relationships]} turns into what is essentially  {addons: [relationships.default]}.

Is this an error with the module, or an export?

The code is bundled using rollupjs as an umd module, treating default export differently than how webpack interprets it (rollup default exports are compatible with the var module = require('module')  style) . We had the same problem with Dexie and solved it there by setting defaultExport.default = defaultExport. This made it work no matter using webpack or commonjs require. Should solve it the same way here.


importHelpers true but no lib dependency?

It seems importHelpers was added to tsconfig.json but no dependency on tslib was added to package.json?
This seems like a natural dependency else the build will fail.

Thanks @lwallent for your help :)


Build(deps-dev): Bump @testing-library/user-event from 14.2.0 to 14.2.1 (#3656)

Pull request opened by the merge tool on behalf of #3656

Merged in 8daf18236a51493a0011e568b900e5456aa9e34f


Handle in-progress scrapes more gracefully on config reload

What did you do?
Reload (HUP) prometheus server to add new rules.  In-flight scrapes get canceled with "context canceled".  Thus making the up metric bounce to 0.
What did you expect to see?
Reload is graceful, and waits for in-flight scrapes to complete or timeout before continuing with reload.
What did you see instead? Under which circumstances?
The Prometheus server stops scheduling new scrapes, waits for the in-flight scrapes to complete or timeout, then reload the config.
Environment

System information:
insert output of uname -srm here

Prometheus version:
insert output of prometheus -version here

Alertmanager version:
insert output of alertmanager -version here (if relevant to the issue)

Prometheus configuration file:

insert configuration here

Alertmanager configuration file:

insert configuration here (if relevant to the issue)

Logs:

insert Prometheus and Alertmanager logs relevant to the issue here

Another possible option would be to have a reload-rules action that doesn't update the targets, but only reloads the rule evaluation.

The Prometheus server stops scheduling new scrapes, waits for the in-flight scrapes to complete or timeout, then reload the config.

Stopping all processing for potentially minutes isn't something we should do imho.

Another possible option would be to have a reload-rules action that doesn't update the targets, but only reloads the rule evaluation.

Doesn't need a separate action but just a validation whether the configuration changed or not. Sounds useful in general. Want to take a stab at implementing that?

We've had it in the past where a HUP was used to work around various SD stuckness, I'm not sure we should lose that.

Closing in favour of #2756


Restructure logic in resolve_test_paths to better handle file globs.

Resolves #5052
How Has This Been Tested?
I ran tests on pascal and my Windows 10 machine, both from command-line and script, using various forms of passing test names:
"tests/plots/*.py", "tests/plots/mesh.py tests/hybrid/*.py", "tests/plots/*.py tests/hybrid/*.py"
Checklist:

[X] My code follows the style guidelines of this project
[X] I have performed a self-review of my own code
[X] I have commented my code, particularly in hard-to-understand areas
[X] New and existing unit tests pass locally with my changes
[X] I have assigned reviewers (see VisIt's PR procedures for more information).

@cyrush can you please review?


bot responding to all threads, I want it to respond only to the thread bot was mentioned

bot responding to all threads, I want it to respond only to the thread bot was mentioned
Reproducible in:
mvn dependency:tree | grep com.slack.api
gradle dependencies | grep com.slack.api
java -version
sw_vers && uname -v # or `ver`

The Slack SDK version
attached actual GitHub codebase url above
Java Runtime version
java version "13" 2019-09-17
OS info
(Paste the output of sw_vers && uname -v on macOS/Linux or ver on Windows OS)
ProductName: macOS
ProductVersion: 13.3.1
ProductVersionExtra: (a)
BuildVersion: 22E772610a
Darwin Kernel Version 22.4.0: Mon Mar 6 21:00:17 PST 2023; root:xnu-8796.101.5~3/RELEASE_X86_64
Steps to reproduce:
(Share the commands to run, source code, and project settings (e.g., pom.xml/build.gradle))

created a thread with bot mentioned
created a regular thread
please check screenshots

Expected result:
respond to only Messages in the thread the bot was mentioned @MyBot create
Actual result:
responds to all thread
Requirements
Please make sure if this topic is specific to this SDK. For general questions/issues about Slack API platform or its server-side, could you submit questions at https://my.slack.com/help/requests/new instead. :bow:
Please read the Contributing guidelines and Code of Conduct before creating this issue or pull request. By submitting, you agree to those rules.

below is the sample code that executes for all the threads, I tried to use an event.getBotId() but it is null for regular thread so I cant avoid threads other than MyBot mentioned thread.
public void messageEvent() {
	app.event(MessageEvent.class, (payload, ctx) -> {
		MessageEvent event = payload.getEvent();
		// Listen only thread replies, if ThreadTs is present it is a reply
		if (event.getThreadTs() != null) {
			// my code executes for all thread
		}
		return ctx.ack();
	});
}

https://github.com/mallik30/actionbot/blob/f0efe1788322a58760523a7820bd429ecfa60de2/src/main/java/com/arjun/slack/api/ActionBotManager.java#L37

Hi @mallik30, thanks for asking the question!
As you've already done, checking the existence of thread_ts in a message is the right way to know if the message is posted in a thread. To know whether your app's bot user is mentioned in the thread, the only feasible way is to call conversations.replies API method and find a string part like <@{your bot's user ID here}> in their text properties. To know your bot's user ID, you can call either auth.test API or bots.info API.
I hope this helps.

hi @seratch thank you


develop-orderbook-generator

Merge orderbook-generator into develop

@talbaneth it seems my previous PR was merged to master (my mistake, probably when creating the PR).
As it is a standalone tool this should have no negative effect I can think of.
Please merge it to develop.


enum fields different behavior in a python versions

Initial Checks

[X] I confirm that I'm using Pydantic V2

Description
is that expected behavior&
Example Code
class LoggerLevels(str, enum):
    INFO = 'INFO'

class LoggingSettings(BaseModel):
    """Logging settings"""

    log_level: LoggerLevels = LoggerLevels.INFO
    model_config = ConfigDict(use_enum_values=True)

settings = LoggingSettings()

# in a python 3.11.9
# settings.log_level -> LoggerLevels.INFO a enum class

# in a python 3.12.5
# settings.log_level -> LoggerLevels.INFO.value a enum value

Python, Pydantic & OS Version
debian, ubuntu 22.04
3.11.9, 3.12.5
pydantic = "^2.4.2"
pydantic-settings = "^2.5.2"
pydantic-core = "^2.23.4"

I can't reproduce: I get <LoggerLevels.INFO: 'INFO'> in both versions. Note that you'll want to use validate_default so that use_enum_values applies when not providing any value for log_level.


Paging doesn't work on OS X

I've just installed latest version from github. Paging doesn't work. If you press enter at "--More--", you just get a blank line.

Hello there @bjohas, thanks for filing this issue.
Yeah the first --More-- displayed is a bug, but if you pressed Enter the second time it should paginate. Gonna fix this.

For me, I just get several --More--, i.e. I see
(e.g. first page of files)
---More--- (press enter)
---More--- (press enter)
---More--- (press enter)
---More--- (press enter)

Thanks!

@bjohas please get the latest from master.

Thanks!


APEX-268 #resolve

@davidyan74 please merge. Please note the change is attributed to jenkins

@vrozov @davidyan74 I have addressed the review comments. However I don't think introducing local variables should be part of fixing checkstyle violations.
If this is more about fixing read-ability of code (not just style violations) then I don't think that I can take up this task.


Records of requests for surveillance data

Someone has requested help on a data project:
Submission notes
I was able to get a dataset from the New Orleans Police Department re their requests for surveillance data sent to the state's fusion center for 2018-2022. Anyone else have any datasets that shed light on when/for what type of incidents police make requests for surveillance data?
Location
Any, for comparison
How to help
Leave a comment if you'd like to help. Do you have ideas, questions, or need any more information to get started?

For context: https://github.com/ayyubibrahimi/eos
Relate: https://atlasofsurveillance.org/
Closing as too open-ended. Data source identification is happening here: https://github.com/Police-Data-Accessibility-Project/data-source-identification


No error when the same virtual host is used 2 or more times

Currently if I wrongly add the same virtualhost 2 times the last wins.
Is there any way to avoid the overwrite and raise an error in the logs?

Not sure I understand. If you add the same virtual host to different containers, they'll be load balanced.

Face palm.. thank you and sorry


Mirrors should not migrate refs/pull and then the relative commits will be gone.

Description
No response
Gitea Version
main
Can you reproduce the bug on the Gitea demo site?
Yes
Log Gist
No response
Screenshots
No response
Git Version
No response
Operating System
No response
How are you running Gitea?
Some filters should be applied in https://github.com/go-gitea/gitea/blob/main/services/mirror/mirror_pull.go#L225-L235 to not sync refs/pull for git mirror.
Database
None

Hi @lunny
It looks like this still happens, in spite of https://github.com/go-gitea/gitea/blob/846f6187168c807e1353d46d5d2260bf077b43cd/services/mirror/mirror_pull.go#L478-L480

Is there a known work-around ?

Thanks ! I'll test it when it's released


Fix simple pricing computation

Remove the truncate on transaction update.

The issue seems to also impact all computed price in the DB, so a task similar to RecomputeAllTransactionsConsumptionsTask shall be added and run to recompute all prices in DB.


the pastrel variant results in out of bounds saturation value

The husl pastrel variant outputs a out of bounds saturation value.
> husl.p.fromRGB(0, 0, 0.5)
[<PHONE_NUMBER>1817733,<PHONE_NUMBER>442829, 8.646890129976928 ]

Thank you

Actually, the problem here is that RGB (0, 0, 0.5) is not in the HUSLp gamut, it's too saturated. I'm guessing you want to use HUSL rather than HUSLp if you're converting RGB to it. I'm going to add a note to the README. Thanks for the report!

Got it, thanks!


Check to see if we can down-coerce BigDecimal when converting to CValue

[ch73]

@djspiewak a couple integration tests failed legit in lwc_local.

@alissapajer I'm glad they failed! We had a bug in trunc that was hidden by the fact that we never ever produced DoubleColumn in lwc_local.


Crazy ideas: table structure

Here are a few thoughts I had while implementing table features in pandoc readers and writers. None of the suggestions have been thought through, so take the below with a grain of salt.

Promote RowHeadColumns from TableBody to the full table. The number of row heads will typically be constant for all table bodies; it is also relevant for, and should also apply to, the table head and foot.

Use grid-based data structure. Going from a grid structure to a list of rows and cells seems much simpler than the other way around. Most writers need to compute the table grid, so it might as well be the main structure. This could provide additional type-level guarantees and make it easier to access cells column-wise, e.g. when checking for the most frequent cell alignment in a column.
A possible data structure would be Array from package array. It is already a transitive dependency of pandoc-types (through deepseq) and described in the Haskell 2010 report.

HTML limits rowspan to a max value of 65534 (= 2¹⁶ - 2), and it would be reasonable to adopt this limit. RowSpan could be then be a newtype wrapper for Word16.

The colspan attribute is limited to a max value of 1000 in HTML. Like for rowspan, this seems like a reasonable limit. ColSpan could also wrap Word16.

Cc: @despresc

I'm not very familiar with the use of multi-body tables, but the examples I've seen do normally have constant-width row heads. So RowHeadColumns could be made to apply to the entire table. We could also enforce the boundaries of the row head in the table head and foot, since normally the overall head and foot of the table won't have cells that stretch across that boundary. It still shouldn't apply to the intermediate heads, as I discussed here.

Having the span types be a wrapper around Word16 is fine, though it will make working with them slightly more inconvenient, since the other table dimensions are still Int-based. It seems unlikely that anyone will want tables with spans larger than the HTML limits, so those exact limits could be enforced by the table builder too.

At the moment all the writers assume that they may encounter any possible Table, including ones that aren't normalized, since every Table is considered valid internally. If that's still a desirable assumption, then from perspective of the writers the extra cached information in Table itself would be useless: they couldn't trust it to be consistent. In that case having the grid table construction in a module in pandoc would be better.
The alternative is to give the Table type clear invariants that must not be violated, making normalized tables the only valid tables internally. If we did that, then we could cache as much information as we like in Table and have the writers simply trust that that information is accurate. This would save pandoc from having to normalize tables twice, at the cost of us having to be very careful not to construct ill-formed tables internally. The FromJSON and Peekable instances would need to normalize the tables that they read, for instance.
Other thoughts:

If the representation were switched to something like Array, then it would be convenient for the readers to have a function that constructs such a table from the current representation. Otherwise they would have to calculate the exact grid representation (including cell coordinates and occupied grid positions) themselves. That function and its inverse would probably also be the easiest way of ensuring a grid table is well-formed, by converting it to the current representation, normalizing it, then converting it back.

Traversals of tables might also be a bit slower, since they have to construct entire new arrays when old ones are modified. I'm not familiar with the performance characteristics of Array-like structures, so this might be avoidable, or might not be a problem in practice.

On that topic, we could get rid of the double-normalization that currently happens by simply removing that behaviour from the table builder. The defensive normalization that the writers now do (through toLegacyTable and AnnotatedTable.toTable) would then ensure that the writer output is well-formed, which is the point of the normalization that's done now. That would mean that pandoc and external filters would encounter odd-looking internal tables more frequently than they do now, if that's a concern.

From a filter perspective, I like a lot the power of the new tables but also dread a bit the complexity for those implementing the filters.
For instance, it feels a bit inconsistent that, within a TableBody, you use the RowHeadColumns integers to indicate headers on the left but have to create a separate list of rows to indicate headers at the top.
When walking over a tree, the structure of a table is also quite different from those of other elements. With most elements, you just traverse the children nodes directly, and each element only has one type of children. However, with tables, you need to traverse the main tree (with multiple TableBody elements) and also need to traverse the head and the foot of the table. Maybe it would be simpler to say that a Table can contain three types of elements, TableHead, TableBody, and TableFoot, with the constraint that TableHead must be the first element and TableFoot must be the last element?
With these two changes (adding a HeadRows to TableBody, and moving TableHead TableBody and TableFoot to the same list), a Table would be much easier to iterate, as every table would just have three types of children, which in turn will have lists of Rows, which in turn just have Cells.
(I'm not necessarily saying we should go that way, just thinking aloud in line with the title of this issue :) )

I'll think out loud for a moment, with the caveat that I don't know much about the characteristics in Haskell programming and am thinking more in the perspective of pandoc filter writer, basically mainly from my pet project pantable written in Python.
First, the new table AST is a great thing to happen. It is long overdue. But at the same time it is a very disruptive change. I think any filter involved in manipulating pandoc tables are broken by this. In the case of pantable, I took advantage of this change to rewrite it completely as its new features really worth the efforts.
But what I'm saying is may be people writing filter for pandoc table are still playing catch-up to this new AST, a yet another unnecessary change may be unwelcomed.
Eg the proposed RowHeadColumns would make it becoming more restrictive. Not that the restriction is that restricting in practice, but it is still a regression. (But I understand the point about head and foot.)
About the grid data structure, I think the current AST structure that's more HTML like is fine. I think having a grid structure as an alternative would be helpful. But I think replacing it by that would be an unnecessary change. In fact, in the work in progress pantable, the internal structure is exactly a grid and it basically provide a way you can jump losslessly between it (grid) and panflute AST (basically pandoc like AST).
So what I'm saying is probably you should have both, and lazily convert between both on demand within Haskell. (This is speaking from some OOP thinking if that makes sense... Eg if the reader already lay things on a grid, then it has a grid representation already, if the writer requires a grid, then no conversion, else convert and perhaps normalized once here...) For JSON filter, may be it is good to have one single representation only for consistency, and let the filter framework do our own grid and normalization (which is what I'm doing in pantable). Or may be you can allow both and let the filter user choose (Eg an arg from pandoc to control this.)

Just catching up on this thread.
I suppose that since we're already in a period of churn with the table types, further changes  might be less disruptive now than if we waited til everything worked again.  Especially if the changes made it easier to process tables.
I'm curious what the grid representation of tables would look like in a Haskell type?
The suggestion to treat row headers like column headers (using an Int) seems a great one.  @despresc - are there reasons not to do it this way?

Do you mean indicating the number of rows in the head of a TableBody with an Int? It seemed like separating the head rows from the body rows would be clearer, since they're different kinds of rows. It's like representing a pair of lists as ([a], [a]) versus (Int, [a]). I can't think of other reasons, so if it's clearer or easier to work with in practice then the Int way is fine.

I'm not sure how much easier it will make things. We should think about that before making a change such as this (which would require a lot of changes throughout the code base). Maybe @sergiocorreia could elaborate on that. But it's hard to see a good rationale for using two lists for row headers but using an Int for column headers.

Do you mean indicating the number of rows in the head of a TableBody with an Int?

Yes! Some arguments in favor are:

TableHead, TableBody, and TableFoot would then be very similar to each other, with the only difference being two extra properties in TableBody (num_head_columns, num_head_rows, or however we name them).

This then reduces the mental overhead of manipulating tables. For instance, converting a TableFoot to a TableBody would be trivial.

Headers within a TableBody are rare enough that most users would then be able to ignore them unless they need to use them, and even then they would just need to specify an integer.

Even if they are different types of rows, in most document types it's not the rows that are treated differently but the cells (e.g. in HTML they are all <tr> objects). This would also simplify writers as there would only be one way to deal with header column/rows (think of the HTML writer, where header rows and columns are saved as <th> instead of <td>).

On the other hand, this might make it a bit harder to create filters that apply to table body heads. Right now, in panflute this would involve:
def action(e, doc):
    if isinstance(e, TableRow) and isinstance(e.parent, TableBody) and e.location=='head':
        pass # do something

With the change, the code would be:
def action(e, doc):
    if isinstance(e, TableRow) and isinstance(e.parent, TableBody) and (e.index < e.parent.num_head_rows):
        pass # do something

Just trying to talk about the few things that is difficult to handle in the current AST, with the mindset of implementing a one-one correspondence of the pandoc table AST to a grid structure in pantable. (Below I used a terminology to distinguishing things and it would be great if there's an official terminology associated with them. Whatever is in double-quote is my made-up terminology. ICA is (Id, class, attr). I called the current AST "tree-like representation" and the proposed one "grid-like representation".)

"row-block" related: this gives the most pain

in rows, we have head, body, foot, and within body, we have another head. So there's 4 kinds of "row-blocks", head, "head of body", "body of body", foot.
only head, body, foot has "row-blocks ICA", i.e. only 3 kinds of "row-blocks" are discerned here, not "head of body", "body of body"
only body has RowHeadColumns

align: there are 2 places for allignment, in the "spec" where the column alignment is with the col-width. And in the cell.
caption: short-caption only accept Inline and caption accepts Block.
a lot of different places for ICA, per cell, per row, per row-block, per table
row and columns are fundamentally asymetric here. For example if you transpose the body, if RowHeadColumns are different between different bodies, it is unsure what its transpose should be. (The same can be said if there's more than a head and a "body of body" however, so a moot point.)

From my experience of implementing a grid representation of the pandoc table AST in Python, while the above things are annoying, it becomes easy once the grid structure is set in-place. I hold the view that there's no best representation here. (Especially with span, because without span and tree representation is the grid representation.) The default tree representation in pandoc is more suitable if the input/output formats are also tree like, such as HTML and LaTeX. (My guess is that the HTML and LaTeX writers first supporting table span is not a coincidence, but because it is a more natural translation.)
That's why I recommend you have 2 representations, a grid one and a tree one. For some intput/output, if the grid one is easier to work with, put it in a grid representation. The grid representation and the tree representation should be one-one correspondance so there's no problem in translating back and forth.
And for performance, I think it is debatable if a grid-only approach would be faster. I would expect for some input-output pairs such as the HTML the tree representation is more natural, and hence fastest to work with them alone. Also I am not sure translating the document into a grid structure is (significantly) faster than put it in a tree than a grid.
About RowHeadColumns:

even if there's no RowHeadColumns per body, a body cannot be easily converted to a head or foot, because a body has a concept of "head of body" and "body of body".
I'd argue if you want to redefine where are the "row-blocks", it is much easier to put it on a grid first and work from there. At least in the case of pantable it is trivial to do that.
replacing RowHeadColumns per body to one single no. is more restrictive. It is hard to convince why that is a necessary change and has no obvious advantage.

Just made aware of this discussion by an allusion in pandoc-discuss...

I think we actually should discuss this on pandoc-discuss first, especially change like this is not about adding a feature, but how best to represent table. Signal-to-noise might be lower in pandoc-discuss, but since this change is going to affect a lot of people with no obvious gain it might be better discussed there first. At the very least we can have a community-wise consensus on the last change that is needed regarding the table.

In fact, I'd go a bit further and say they could be viewed of the same TableBody kind

In pantable, by putting the cells on a grid, I don't distinguish the head, "head of body", "body of body", foot in the table itself. I keep a list of int like this: [1, 0, 9, 0], which indicates there's 1 row of head, 0 row of "head of body", 9 rows of "body of body", 0 row of foot. It will get longer as there's multiple bodies. This approach does have its quirks as inferring which "row-block" a row belongs, or the "n-th body" it belongs to, will be harder. But they are doable (just crunching numbers.)

wouldn't it be sufficient that pandoc write the "coordinates" in cells' attributes in the current scheme?

Going back and forth between table and cells will be difficult here. To find a cell you need to walk the whole table in this case. e.g. normalizing tables probably need to do this. So a grid representation is still advantageous in some operations.

One of the premise in this issue is that "Going from a grid structure to a list of rows and cells seems much simpler than the other way around." But I think you probably just pushed "going from 'tree-representation' to 'grid-representation'" earlier in the stage of reader. In cases that the reading format is also "tree-like", such as HTML/LaTeX, may be there's no gain?

we actually should discuss this on pandoc-discuss first

ook, was just surprised about the news. I'll keep the 'noise' to a minimum 😉.

Going back and forth between table and cells will be difficult here. To find a cell you need to walk the whole table in this case.

True, was not thinking about going "back and forth", only one way.
In any case... a quick look at AnnotatedTable show that it seems to offer some kind of grid mapping (and more), at least if I interpret the code comments correctly! I should have checked before my initial suggestion.

I don't mean that, I mean the topic of this issue might better be discussed in pandoc-discuss first, I don't mean your comment in particular. I meant if we discussed there the signal to noise might be lower but it probably can get more attention from people that would be affected by this kind of change most. It's a bit unusual, but AST change as big as this is unusual. And if we have to change it a 2nd time then we better discuss it more and nailed it.

Here is

I'm curious what the grid representation of tables would look like in a Haskell type?

This is roughly what I imagine:
-- | A grid cell contains either a real table cell, or is the
-- continuation of a column or row-spanning cell. In the latter case,
-- the index of the continued cell is provided.
data GridCell
  = ContentCell Attr Alignment RowSpan ColSpan [Block]
  | ContinuationCell RowIndex ColIndex

newtype RowIndex = RowIndex Int deriving (Ix)
newtype ColIndex = ColIndex Int deriving (Ix)

-- | Cells are placed on a grid. Row attributes are
data CellGrid = CellGrid
  { cellArray :: Array (RowIndex,ColIndex) GridCell
  , rowAttrs  :: Array Int Attr
  }

data GridTable = GridTable
  { tableAttr     :: Attr
  , tableCaption  :: Caption
  , tableColSpecs :: Array ColIndex ColSpec
  , tableHead     :: CellGrid
  , tableBodies   :: [CellGrid]
  , tableFoot     :: CellGrid
  , tableRowHeads :: Int
  }

FWIW, the grid representation would be really useful to have for docx tables, where rowspans are handled by adding "merge cells" in subsequent rows.

Can you remind me: is the idea to change the main table type to a grid representation, or to add a GridTable type and conversion functions from/to the Table we have now?  (In the latter case, these wouldn't necessarily need to go in pandoc-types; they could go in pandoc itself.)

The original idea was to update the main table type, but it's probably too late for that now. Maybe it's just my way of thinking, but I realized that I find grids the most natural way to think about tables.
I may try to write something akin to a GridTable for the docx writer, or maybe modify the AnnotatedTable, and we can see where that leads us.

I wouldn't rule out changing the table type again, if the grid model is much easier to deal with.  But that's going to be a lot of work.  Maybe a good first step would be to implement the type in an auxiliary model in pandoc, together with conversion functions.

Grid model has its own problems but I agree it seems easier to work with. I agree with @jgm in the direction of

create a grid-table in addition to the current tree-table
has lossless functions converting between them (lossless in valid data only)
then in the long run see which one you uses more, and if needed, change the pandoc-type to grid-table and rely on those lossless functions for those that needs tree-table

this probably can be done after a long time since the new table AST, but it'd be great if a tool such as ast-migrator can deal with this lossless conversion in the filter level. (Or better yet, a flag that change how pandoc writes the AST, i.e. you have both AST representation of the same thing that can be losslessly converted to one another. e.g. the filter can call pandoc to convert from what it receives to the AST that it understands.)

I have got to admit that I lost a good bit of enthusiasm for grids after using them for the docx writer. I still think they are the right data structure for that specific case, but I forgot just how wrong it feels to use unsafe operations like array indexing in Haskell. We'd have to provide a number of good helper functions to safely use arrays throughout the code base.

We should completely avoid unsafe partial functions like (!!) in the code -- though it's easy enough to create safe wrappers as is done in the safe package.

Smaller point that came to mind while I was implementing column width reading in the docx reader: would it be better to have the column widths as Rational rather than Double? At least in the docx reader I was creating them by division, and it looked a bit odd having the full doubles in the native. I guess any rounding errors are going to be quite small, so maybe it doesn't matter?

I wish I'd used Rational rather than Double from the beginning.  That is indeed a better choice.  But maybe not worth fixing now, until the next major revision of pandoc-types.

I agree rational is a good choice. But what would that becomes in JSON AST? A float or specially formatted string?
[pantable (pandoc table ast in Python) allows rational ratio optionally. Also the auto width calculated from say a markdown table is already rational.]

I suppose the JSON representation could just be a string, yes.  That might make it a bit more difficult for consumers of the AST other than pandoc.


Rendering particle using BatExemple.

Description of the issue:
When trying to spawn particles with the BatExemple an issue occurred and only one instance of the entity renders the particle.
Here some picture of the issue, we can see each entity produces particles when they are isolate but when looked at the same time only one produce particles.

Environment information:

Geckolib: 4.2.2
Neoforge: 47.14.70
Minecraft: 1.20.1

How to reproduce:

Spawn 2 BatEntity one on the left and the other one on the right. The particles should work fine at this moment.
Leave the world and join it again.
If you look at each entity one by one it should work fine, but if you look at the two at the same time only one bat does particle.

One easy fix:
An easy fix I have found is to use a HashMap where the keys are the entities' id and the values are their tickCount.
private HashMap<Integer, Integer> currentTickMap = new HashMap<>();

@Override
    public void renderFinal(PoseStack poseStack, BatEntity animatable, BakedGeoModel model, MultiBufferSource bufferSource, VertexConsumer buffer, float partialTick, int packedLight, int packedOverlay, float red, float green, float blue, float alpha) {
        if (!currentTickMap.containsKey(animatable.getId()) || currentTickMap.get(animatable.getId()) != animatable.tickCount) {
            currentTickMap.put(animatable.getId(), animatable.tickCount);

			// Find the earbone and use it as the point of reference
			this.model.getBone("leftear").ifPresent(ear -> {
				RandomSource rand = animatable.getRandom();
				Vector3d earPos = ear.getWorldPosition();

				animatable.getCommandSenderWorld().addParticle(ParticleTypes.PORTAL,
						earPos.x(),
						earPos.y(),
						earPos.z(),
						rand.nextDouble() - 0.5D,
						-rand.nextDouble(),
						rand.nextDouble() - 0.5D);
			});
        }

        super.renderFinal(poseStack, animatable, model, bufferSource, buffer, partialTick, packedLight, packedOverlay, red, green, blue, alpha);
    }

Yeah it's a known issue, and I don't care to go crazy with it cus it's just an example meant for showing off worldspace positioning


improve make bench

Uses hyperfine to compare between cairo-run, mlir JIT and native with clang.

Description of the pull request changes and motivation.
Checklist

[ ] Linked to Github Issue
[ ] Unit tests added
[ ] Integration tests added.
[ ] This change requires new documentation.

[ ] Documentation has been added/updated.

Codecov Report

Merging #170 (35e5dc7) into main (37bfaf1) will increase coverage by 1.05%.
The diff coverage is 0.00%.

:exclamation: Current head 35e5dc7 differs from pull request most recent head fb46124. Consider uploading reports for the commit fb46124 to get more accurate results

@@            Coverage Diff             @@
##             main     #170      +/-   ##
==========================================
+ Coverage   87.75%   88.81%   +1.05%
==========================================
  Files          70       70
  Lines       11532    11395     -137
==========================================
  Hits        10120    10120
+ Misses       1412     1275     -137

Impacted Files
Coverage Δ

cli/src/main.rs
0.00% <0.00%> (ø)

sierra2mlir/src/compiler/mod.rs
96.66% <ø> (+59.00%)
:arrow_up:

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


⚠ Bofrost.at has degraded performance

In 12c8141, Bofrost.at (https://www.bofrost.at) experienced degraded performance:

HTTP code: 200
Response time: 1333 ms

Resolved: Bofrost.at performance has improved in 4b510ef after 8 minutes.


Installation Guide 404 (README.md)

Not adding a commit for this since the 404 should probably be fixed on the documentation site.
Bug report
What I did:
Clicked "Install Backpack on Laravel 5.5"
What I expected to happen:
Taken to installation guide.
What happened:
Taken to 404 doc page.
What I've already tried to fix it:
n/a
Backpack, Laravel, PHP, DB version:
n/a

Updated! Thank you @bachhuberdesign . Go here. The process is the same as L5.6 - the current version supports both 5.5 and 5.6. But my recommendation - update your Laravel to 5.6. You're going to have to do it anyway - better now than later.
Cheers!


obsolete MONDO:0009148

closes #432

I followed up again

Ada said OMIM will not merge these terms. I'll close this PR.


line 16: wmctrl: command not found

I've encountered this bug a few times already and glad that it's not just me. I tried your script on my Ubuntu but I'm getting the following error:
line 16: wmctrl: command not found

hi, you can fix that by installing the wmctrl package in ubuntu:
sudo apt-get install wmctrl

I thought I'd better comment here before opening a new issue.
When running
sudo apt-get install wmctl

I get the error
E: Unable to locate package wmctl

you have a typo there - package name is wmctrl, you are missing the r

Doh. Thanks for pointing that out. I'll remove my comments as to not dilute this thread.
Thanks again!


Interface bits/updates

Interface cleanup.
Build cleanup/add submodules/rebased builds.
readme updates (more on this next PR after I've decided whether/how to split the package)

No functional changes.
@markhibberd

:+1: on :green_apple:

👍


Fullstack Feature: Candidate Results Preview using WebSockets

User story
Who: As a user
What: I want to see temporary results prior to deadline calculated using the current candidate ordering
Why: so that Users can get an estimate of the results before deadline
Acceptance criteria
User should be able to click a button to see temporary results.
Results should get calculated using the users current ordering of the candidates.
Additionally the results screen should update as users reorder their candidates.
Development
This requires several implementations on both the front-end and backend.
Front-end:

A button to direct the user to the results preview screen
A preview screen should update without the need for user interaction as other users reorder their candidates.
Websocket library to communicate with backend.

Backend:

Websocket technology must be implemented to receive candidate updates and emit temporary results to all connected users.

Inspiration :
https://deepinder.me/creating-a-real-time-chat-application-with-react-hooks-socket-io-and-nodejs

So by elimination, I'll take this one

So by elimination, I'll take this one

Maybe its better to get Milestone 1 and 2 done first


I have a problem with the translation of blocks in the missions, I would like the items to be read in Spanish from the mission book. for example the Stone is taken from the minecraft system and is in English.

Where else have you looked for answers to this question?
Share your question with as much detail as possible?
If applicable, which class or method is this in reference to?

Thanks for your question. Unfortunately, this isn't possible as translations can only occur in client-side chat. Books cannot be supported at this time.


GithubBrowserSample : cross usage of DTO in all layers

In the sample "GithubBrowserSample" , the data transfer objects like Repo and User are used across the data, view and presentation layer. This does violate the clean architecture principle of separation of concerns.

Dup of #388


Update metadata dump

Adds in a mapping file for seqr family id : seqr family Guid, to the metadata dump, for both the exome and genome seqr projects. Adds the mapping file as family_guid_map.json to the metadata zip.
family_guid_map.json:
"exome": {
  "FAM001": "exome_GUID_001",
  "FAM002": "exome_GUID_002"
  },
"genome" : {
  "FAM001": "genome_GUID_001",
  "FAM002": "genome_GUID_002",
  "FAM004": "genome_GUID_004",
}

The json inputs are generated from the AIP script:
https://github.com/populationgenomics/automated-interpretation-pipeline/blob/main/helpers/process_seqr_metadata.py

@cassimons this PR was about adding a file to the metadata dump which mapped the seqr family GUIDs to each sequencing group ID.
I've just made another minor change to retrieve the latest DatasetVCF by filtering the analyses by analysis.meta.stage == "DatasetVCF" - this will now return the latest analysis from the DatasetVCF stage. From testing this looks to work good.
I'm going to merge this now and get the outputs ready before sharing with Timo.

Great thanks!


[Demo] Add gantt chart to visual-vocabulary

Description
Issue -> https://github.com/mckinsey/vizro/issues/709
Added Gantt chart example to the visual vocabulary and updated README
Screenshot

Notice

[x] I acknowledge and agree that, by checking this box and clicking "Submit Pull Request":

I submit this contribution under the Apache 2.0 license and represent that I am entitled to do so on behalf of myself, my employer, or relevant third parties, as applicable.
I certify that (a) this contribution is my original creation and / or (b) to the extent it is not my original creation, I am authorized to submit this contribution on behalf of the original creator(s) or their licensees.
I certify that the use of this contribution as authorized by the Apache 2.0 license does not violate the intellectual property rights of anyone else.
I have not referenced individuals, products or companies in any commits, directly or indirectly.
I have not added data or restricted code in any commits, directly or indirectly.

@jazwu - do let me know if you prefer to finish this PR off on your own or if you want me to help you finish this off :)
If you want me to finish this off, you would have to provide me write access to your forked repository: https://github.com/jazwu/vizro
For this you need to go to your forked repository -> settings -> collaborators and teams --> add people
Then grant me write access :)

@jazwu - do let me know if you prefer to finish this PR off on your own or if you want me to help you finish this off :)
If you want me to finish this off, you would have to provide me write access to your forked repository: https://github.com/jazwu/vizro
For this you need to go to your forked repository -> settings -> collaborators and teams --> add people
Then grant me write access :)

Hi! Thanks for the comments! I would like to finish my part. Sorry for the late response because I was busy with GHC from 8th to 11th.


feat: Allow building on Darwin for both arm64 and amd64 architectures

Prerequisite checklist

[x] Read the contribution guidelines regarding submitting new changes to the project;
[x] Tested your changes against relevant architectures and platforms;
[x] Ran make fmt on your commit series before opening this PR;
[ ] Updated relevant documentation.

Description of changes
Create darwin specific equivalents of _linux code.
If possible, merge common parts into _unix suffixed files. Move others into _linux.go and _darwin.go files.
Augment goreleaser's ytt templates to produce two additional targets:

darwin amd64
darwin arm64

Make it build a binary runnable on the host MacOS machine via make kraft (both with DOCKER variable set and not set)
GitHub-Fixes: #30

Hi @jake-ciolek, thanks a lot for this work. I've turned #266 into an epic since it looks like we need several things to get this all tied up. Can you please remove the GitHub-Fixes from the commit message when you make your updates?

@nderjung Thanks. I've modified the commit message so it only fixes #30

@nderjung I've added the run-related changes you mentioned.


Rework native classes

Currently each (list and listiterator) native class is an explicit value. But there should instead just be a "native_class" value.

https://github.com/JanEricNitschke/generic-lang/pull/36


[sourcekit] Prevent race between getInitializationInfo and requests

Request handling happens on a global queue, so we need to use a barrier_async on the same queue to ensure the initialization finishes before requests are handled.

@swift-ci please smoke test

@swift-ci please smoke test


Add docker based test to ensure Dragonboat can be used without RocksDB

Add docker based test to ensure LevelDB based raft storage can be used without RocksDB.

done.


fdk.12.rc1

Current gem release (0.0.11) is not consistent with the fdk-ruby repo.  Mainly because the FDK interface in the repo uses named parameters.
This PR creates a release candidate Gem with the interface updated.

Looks good - although be aware that we're planning a fevamp of the FDK contract right now


Always retain metadata.finalizers and metadata.annotations

Now that overriding via jsonpatch is supported, it is possible to restrict modification of the annotations and finalizers fields to overrides. Modifications to these collection fields can be made using the add or remove jsonpatch operations to ensure that a given item exists (or does not exist) with less chance of conflicting with local controllers.
kind: FederatedDeployment
...
spec:
  ...
  overrides:
    - clusterName: cluster1
      clusterOverrides:
        # Ensure the annotation "foo: bar" exists
        - path: "/metadata/annotations"
          op: "add"
          value:
            foo: bar
        # Ensure an annotation with key "foo" does not exist
        - path: "/metadata/annotations/foo"
          op: "remove"

To support this change, it was necessary to move the application of overrides from before retention of local values to after so  that overrides would be applied to the result of retention.
This change suggests implementing selector-based overrides to simplify applying an annotation across multiple clusters.
Fixes #982

@marun please rebase.

Rebased

Rebased and ready for review

@font I've replaced references to 'local controller' with 'controllers in member clusters'.  Does that make more sense?

Rebased.

Rebased

/lgtm
Thanks @marun

Hi! I think annotations still should propagate for non controller-managed resources, e.g. Ingress or Service because in most cases these resources should be consistent across the entire deployment. If you want to override it for some clusters, it still may be done via jsonpatch. Otherwise, you are forced to add the same annotations for all member clusters which is not very convenient.


Sample/one-pager: genesis config

Explaining the Genesis Config in greater detail:
How to set up things like (and what they mean):

total issuance
existential deposit
transfer fee
vesting
creation fee
etc.

How to create new storage variables for genesis, how to set up genesis in testing, when to use add_extra_genesis, etc.

We have a small part of the last point (How to create new storage variables for genesis) in the GenesisConfig Type doc - https://docs.substrate.dev/docs/genesisconfig-struct
Should be considered when writing this doc.

Some of this content is covered in https://www.shawntabrizi.com/substrate-collectables-workshop/#/5/testing-genesis

And a helpful example from nicole https://github.com/substrate-developer-hub/substrate-collectables-workshop/pull/100#discussion_r287309170

This tidbit came across substrate technical chat.
Guillaume
I would expect to show in the doc, it is added here
https://github.com/paritytech/substrate/blob/master/srml/support/procedural/src/storage/transformation.rs#L437

and defined here
https://github.com/paritytech/substrate/blob/master/srml/support/procedural/src/storage/transformation.rs#L416
if you do cargo doc don't you have this PhantomData field (I'm actually not sure it is, but I thought so)
bedeho / Joystream
Now it says

"
193 |         _generisi_phantom_data: Default::default()
    |         ^^^^^^^^^^^^^^^^^^^^^^ help: a field with a similar name exists: `_genesis_phantom_data`
"
Guillaume
it is just a phantom data to deal with the generic argument T. If you are sure that you genesis config don't need this extra field you can bypass our best effort assumption using another definition in decl_storage which is used here for example:
https://github.com/paritytech/substrate/blob/master/srml/balances/src/lib.rs#L334
yes _genesis_phantom_data not _genesisi_phantom_data
bedeho / Joystream
oh, I didnt see the typo. Ok, that actually worked.
Guillaume
cool :-)
bedeho / Joystream
but I don't understand why this was needed here in particular?
I have another genesisconfig type where I dont need to do this
what is the distinction?
This is the only configurable field for this gensisconfig where your fix had an effect

pub ForumUserById get(forum_user_by_id) config(): map T::AccountId => Member<T::AccountId>;
Guillaume
we might lack doc here indeed but I would expect this field to be shown in doc.

decl_storage generate struct GenesisConfig<T: Trait> {..} this genericity is used later for some stuff.
But if the field of your genesis config doesn't make use of T like

struct GenesisConfig<T: Trait> {
forum_by_id: u64,
}
Then rust can't compiles because T is not used in the struct.

In our macro we try to distinguish with a best effort method when it used and when it is not. if it seems not to be used we add a phantomdata field to capture it

I'm a bit surprise by this definiton as our macro should have seen T::AccountId and thus see that T was used by genesis config.

Also if you are sure T is used (if it compiles) then you can add an extra field in decl_storage which is extra_genesis_skip_phantom_data_field; look at srml/balances/src/lib.rs
this bypass our assumption

https://crates.parity.io/srml_support_procedural/macro.decl_storage.html#genesisconfig

Started a draft at https://github.com/substrate-developer-hub/substrate-developer-hub.github.io/pull/154 but got stuck earlier than expected on a compile issue related to the _genesis_phantom_data issue discussed above.

You just need to give it this value and initialize it to Default::default().
On master this is fixed, but we can not backport as this breaks some code.

Thanks for that tip @bkchr !
#154 is merged now, so I'm closing this.

not very clear in this tutorial on how to set up the genesis configuration. it will be useful to add some references.
https://substrate.dev/substrate-verifiable-credentials/#/docs/securing-issuing

@lerajk Are you currently stuck on this? If you share your code I can help you get unstuck. If you can say what wasn't clear or have some specific tips to fix that tutorial, you could open an issue over here https://github.com/substrate-developer-hub/substrate-verifiable-credentials/issues


Add comment why os:chmod tests might fail

Fixes #1748

Thanks for bringing the issue to my attention.
The PR has some issues:

The change to the actual test code is unnecessary - it seems to be based on the assumption that subsequence test cases share the same directory, so you'll need to ensure that the directory names are different. But that's not the case: when using evaltest.TestWithSetup the setup is run for every individual test, so each test gets run in its own temp directory.
The comment you added also seems wrong - this has nothing to do with how tmpfs is mounted, this is just how mkdir behaves on FreeBSD, as explained in https://github.com/golang/go/issues/19596.

We should also simply fix the issue - as shown in https://go-review.googlesource.com/c/go/+/41430/5/src/cmd/go/internal/work/build_test.go, doing chgrp on the parent of the directory that one wishes to call chmod on does the trick.

The problem on my FreeBSD VM was because when I installed the OS it didn't put me in the "wheel" group. And the /tmp mount point was owned by the "wheel" group with the "sticky" attribute. I fixed it by adding my account to that group. Doing chgrp on /tmp is not a viable solution to this failure mode. The comment my change introduced was meant to only provide a hint to a future developer who might wonder why this unit test was failing.
The functional change to the unit test was only meant to make it obvious that the two tests operated on different directories. There are quite a few unit tests which do the same thing because it doesn't require the reader to recognize that TestWithSetup runs each test in a distinct temp dir and thus reduces the cognitive load when interpreting the test code.

I see your fix changes the ownership of the created temp directory rather than the parent of the created temp dir. I misunderstood your previous comment as implying the ownership of /tmp should be changed. Still, the comment in your fix is not very clear about the nature of the "Work around a quirk of FreeBSD's mkdir". I removed the if runtime.GOOS == "freebsd" { statement and the unit test passed on every platform I use for testing these things (macOS, Windows, FreeBSD, and several Linux distros). In other words, there is no reason to predicate the os.Chown on the OS being FreeBSD, AFAICT. The chown is a nop on non FreeBSD platforms.

Sure, done in 464cfb64c64a9f61fbb1fe48ec43078860492960.


fix: reverted migrate script rename

What kind of change does this PR introduce?
Bug fix
What is the new behavior?
Revert renaming of the migrate-call.ts file to migrate.ts restoring it to its original
This is because the CLI is hard coding the script name HERE
Fixes #515

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 79.331%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
12948

Relevant Lines:
16187

💛  - Coveralls

Hey @fenos, how do I get the supabase CLI to pull the latest docker image to get this fix when running supabase db pull?

Hey @fenos, how do I get the supabase CLI to pull the latest docker image to get this fix when running supabase db pull?

Never mind, I just deleted the .temp directory inside my supabase folder and reran and it pulled the right image this time.


RFC2136 provider dont' woking

I've a questin how to use the rfc2136 provider. My test zone (a bind server, wich is only internal reacheable) is aditocloud.de. So I've configured the zone to allow update with a key.
The log of bind dns show okay
03-Dec-2018 08:31:29.975 client @0x55e7a286d0a0 <IP_ADDRESS>#54032/key adito-de-trans (aditocloud.de): transfer of 'aditocloud.de/IN': AXFR started: TSIG adito-de-trans (serial<PHONE_NUMBER>)
03-Dec-2018 08:31:29.977 client @0x55e7a286d0a0 <IP_ADDRESS>#54032/key adito-de-trans (aditocloud.de): transfer of 'aditocloud.de/IN': AXFR ended

Also I've added a txt entry in aditocloud zone file:
$TTL 600
@      IN     SOA    a.ns     hostmaster (
<PHONE_NUMBER>                                        ; serial
                     14400                                             ; refresh after 4 hours
                     3600                                              ; retry after 1 hour
                     604800                                            ; expire after 1 week
                     180   )                                           ; minimum TTL of 3 minutes
       IN     CAA    0       issue   "letsencrypt.org"                 ;
       IN     NS     a.ns                                              ;
       IN     NS     b.ns                                              ;
       IN     NS     c.ns                                              ;
       IN     NS     d.ns                                              ;
       IN     MX     10      smtp                                      ;
       IN     MX     20      mail                                      ;
       IN     A      <IP_ADDRESS>                                     ; must be an a-record
       IN     TXT    "heritage=external-dns,external-dns/owner=extdns"

And here the log from external dns:
...
time="2018-12-03T08:30:19Z" level=debug msg="Skipping endpoint ntp.aditocloud.de 600 IN CNAME static.<IP_ADDRESS>.gsh.aditocloud.de map[] because owner id does not match, found: \"\", required: \"extdns\""
time="2018-12-03T08:30:19Z" level=debug msg="Skipping endpoint servicecloud.aditocloud.de 600 IN CNAME heku.aditocloud.de map[] because owner id does not match, found: \"\", required: \"extdns\""
time="2018-12-03T08:30:19Z" level=debug msg="Skipping endpoint gok.demo.aditocloud.de 600 IN CNAME heku.aditocloud.de map[] because owner id does not match, found: \"\", required: \"extdns\""
time="2018-12-03T08:30:19Z" level=debug msg=ApplyChanges
time="2018-12-03T08:30:19Z" level=debug msg="RemoveRecord.ep=aditocloud.de 600 IN A <IP_ADDRESS> map[]"
time="2018-12-03T08:30:19Z" level=debug msg="Adding RR: aditocloud.de 0 A <IP_ADDRESS>"
time="2018-12-03T08:30:19Z" level=debug msg=SendMessage
time="2018-12-03T08:30:19Z" level=debug msg=SendMessage.success
time="2018-12-03T08:30:19Z" level=debug msg="RemoveRecord.ep=aditocloud.de 0 IN TXT \"heritage=external-dns,external-dns/owner=extdns\" map[]"
time="2018-12-03T08:30:19Z" level=debug msg="Adding RR: aditocloud.de 0 TXT <IP_ADDRESS>"
time="2018-12-03T08:30:19Z" level=debug msg=SendMessage
time="2018-12-03T08:30:19Z" level=debug msg=SendMessage.success
...

The configuration of external dns:
        - --source=ingress
        - --source=service
        - --log-level=debug
        - --provider=rfc2136
        - --rfc2136-host=<IP_ADDRESS>
        - --rfc2136-port=53
        - --rfc2136-zone=aditocloud.de
        - --rfc2136-tsig-secret=YbxMEkQ0PpD7d25X4LL8n4On+bAEm7q7EFiX5UVftVr+G9XniHyDFJ861Ojfa0kyVuy0UIcOR0/wgJw==
        - --rfc2136-tsig-secret-alg=hmac-sha512
        - --rfc2136-tsig-keyname=adito-de-trans
        - --rfc2136-tsig-axfr
        - --registry=txt
        - --txt-owner-id=extdns
...

Ingress testconfiguration:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: external-dns-test-ing.aditocloud.de
  namespace: control
  annotations:
    nginx.ingress.kubernetes.io/force-ssl-redirect: 'true'
    external-dns.alpha.kubernetes.io/hostname: test2.aditocloud.de
    kubernetes.io/tls-acme: "true"
spec:
  tls:
  - hosts:
    - k8s-exmp.aditosoftware.local
    secretName: aditosoftware-default
  rules:
  - host: k8s-exmp.aditosoftware.local
    http:
      paths:
      - path: /
        backend:
          serviceName: k8s-exmp-aditosoftware-local-http
          servicePort: 8080
      - path: /jclient
        backend:
          serviceName: k8s-exmp-aditosoftware-local-http
          servicePort: 7779

When I show the logs, I think the configuration between bind and exernal dns is okay.
But I think my configuration of external dns and maybe the zone file is wrong.
Can someone help me with this error!?

It seems like you created a TXT record with the extdns keyword in it for aditocloud.de. I'm not an expert in DNS but I think the problem is the following line in your bind config:
       IN     TXT    "heritage=external-dns,external-dns/owner=extdns"

you don't need to specify this in your bind configuration.
Therefore this record is managed by external dns and since there is no service in your cluster which claims this DNS name it will be deleted by external dns.


[system-probe] Handle/reduce stat cookie collisions

What does this PR do?
Mitigates stat cookie "collisions" by rehashing a 64 bit cookie in userspace for connections. A stat cookie collision is when the cookie value for two connections is the same, but the byte key (essentially the connection tuple) is different. A cookie is supposed to uniquely identify a connection, but due it being 32 bits in ebpf (due to stack constraints), there is an insignificant amount of collisions when the connection count/churn is high.
This change rehashes the cookie to a 64 bit value in userspace. This essentially eliminates cookie collisions in test environments. Additionally, if a collision does occur, we try to handle it as best as we can, picking the "newest" connection, and removing any stored state for that cookie.
~6% cpu increase:

no memory change:

Motivation

Additional Notes

Possible Drawbacks / Trade-offs

Describe how to test/QA your changes

Reviewer's Checklist

[x] If known, an appropriate milestone has been selected; otherwise the Triage milestone is set.
[ ] Use the major_change label if your change either has a major impact on the code base, is impacting multiple teams or is changing important well-established internals of the Agent. This label will be use during QA to make sure each team pay extra attention to the changed behavior. For any customer facing change use a releasenote.
[x] A release note has been added or the changelog/no-changelog label has been applied.
[ ] Changed code has automated tests for its functionality.
[x] Adequate QA/testing plan information is provided if the qa/skip-qa label is not applied.
[x] At least one team/.. label has been applied, indicating the team(s) that should QA this change.
[ ] If applicable, docs team has been notified or an issue has been opened on the documentation repo.
[ ] If applicable, the need-change/operator and need-change/helm labels have been applied.
[ ] If applicable, the k8s/<min-version> label, indicating the lowest Kubernetes version compatible with this feature.
[ ] If applicable, the config template has been updated.

Seems like the memory dropped by ~10%, how that can happen from the change?

+1 to @guyarb on the performance impact being further reduced if possible.

there is an insignificant amount of collisions when the connection count/churn is high.

Did you mean to say significant instead of insignificant here?
Do we have any graphs to highlight the impact on collisions before and after this change, and can you provide some more detail on the consequence that collisions for stat cookies has?

+1 to @guyarb on the performance impact being further reduced if possible.

there is an insignificant amount of collisions when the connection count/churn is high.

Did you mean to say significant instead of insignificant here?
Do we have any graphs to highlight the impact on collisions before and after this change, and can you provide some more detail on the consequence that collisions for stat cookies has?

Yes, sorry, significant.

Can we do anything to make the cookie generation less conflict prone in the system-probe? One thought is to use a monotonic counter tracked in eBPF.
The CPU/memory impact seems okay, but the  complexity added by having two potential values for cookie makes code harder to follow.

Can we do anything to make the cookie generation less conflict prone ~in the system-probe~ at generation time? One thought is to use a monotonic counter tracked in eBPF.
The CPU/memory impact seems okay, but the complexity added by having two potential values for cookie makes code harder to follow.

The problem is the size, 32 bits is just too short. There is already a sort of counter used to generate the hash in ebpf in the form of the current time.


[decomp] sync-info 1

Used by various game objects

Pull Request Test Coverage Report for Build 736762070

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.02%) to 69.018%

Totals

Change from base Build 712063599:
0.02%

Covered Lines:
32985

Relevant Lines:
47792

💛  - Coveralls