Support for debugging services

Enhance the plugin to allow debugging of services. See https://github.com/creek-service/creek-system-test/issues/22 for details.

There is a link in creek-system-test to this issue. This needs to be replaced with a link to the documentation on how to use the plugin to debug.


Failed to validate spending during block preparation

i did get stuck here
2023-02-04 15:02:18 WARNING Current Block: 87387 / 94382 - e719a5391199aaa1bb1e58d647317200cc94045db5575430c74779e10ee2d3c0
2023-02-04 14:29:24 WARNING Preparing block 2023-02-04 14:29:24 ERROR Failed to validate spending during block preparation: ndo334bb05d47f014c402a1c393685cf341f7fa28e8c67168 spending more than owned in a single transaction 2023-02-04 14:29:24 ERROR Block preparation failed due to: ndo334bb05d47f014c402a1c393685cf341f7fa28e8c67168 spending more than owned in a single transaction 2023-02-04 14:29:24 WARNING Block production skipped due to: ndo334bb05d47f014c402a1c393685cf341f7fa28e8c67168 spending more than owned in a single transaction
force sync and resync did not help

Do you have an issue with the "too many files open" ?

2023-02-05 16:34:16 INFO Working with 37b5e1f9da113895cf1cc0acad537150dd610bc8efd793f81be582f56ad4c0a5 from a pool of 1 2023-02-05 16:34:16 INFO <IP_ADDRESS> knows block e719a5391199aaa1bb1e58d647317200cc94045db5575430c74779e10ee2d3c0 2023-02-05 16:34:16 INFO Period: 0 2023-02-05 16:34:16 INFO Block Hash Agreement: 68% 2023-02-05 16:34:16 INFO Transaction Hash Agreement: 100% 2023-02-05 16:34:16 INFO Block Producer Agreement: 21% 2023-02-05 16:34:16 DEBUG Transactions: 0tp/0tb/0ub 2023-02-05 16:34:16 DEBUG Linked Peers: 24 2023-02-05 16:34:16 DEBUG Block Producers: 299 2023-02-05 16:34:16 WARNING Emergency Mode: True 2023-02-05 16:34:16 WARNING Current Block: 87387 / 95914 - e719a5391199aaa1bb1e58d647317200cc94045db5575430c74779e10ee2d3c0 2023-02-05 16:34:16 WARNING Seconds since last target: 511635 / 60 2023-02-05 16:34:16 WARNING Unreachable: 27 2023-02-05 16:34:16 WARNING Forced sync: None 2023-02-05 16:34:16 INFO Loop durations: Core: 23; Consensus: 0; Peers: 28 2023-02-05 16:34:16 INFO Open files: 1 2023-02-05 16:34:17 WARNING Preparing block 2023-02-05 16:34:17 ERROR Failed to validate spending during block preparation: ndo334bb05d47f014c402a1c393685cf341f7fa28e8c67168 spending more than owned in a single transaction 2023-02-05 16:34:17 ERROR Block preparation failed due to: ndo334bb05d47f014c402a1c393685cf341f7fa28e8c67168 spending more than owned in a single transaction 2023-02-05 16:34:17 WARNING Block production skipped due to: ndo334bb05d47f014c402a1c393685cf341f7fa28e8c67168 spending more than owned in a single transaction 2023-02-05 16:34:18 INFO Maximum reached cascade depth: 1 2023-02-05 16:34:18 INFO Working with 37b5e1f9da113895cf1cc0acad537150dd610bc8efd793f81be582f56ad4c0a5 from a pool of 1
nope

Newer version will not help with the state of the database

with second resync/purge it did pass 87387 but now i get too many files opend error

Yes, this leads to database corruption. You need to increase maximum allowed open files limit.

Please follow this guide to permanently increase number of open files:
https://github.com/bismuthfoundation/Bismuth-FAQ/blob/master/Install/Linux.md#limits


[API] add release_counter to Repo API

add release_counter to GET /repos/{owner}/{repo} responce
@lunny https://github.com/go-gitea/gitea/pull/9202#issuecomment-559820035: done ;)

Codecov Report

Merging #9214 into master will decrease coverage by <.01%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master    #9214      +/-   ##
==========================================
- Coverage   41.02%   41.01%   -0.01%
==========================================
  Files         553      553
  Lines       72141    72142       +1
==========================================
- Hits        29593    29592       -1
- Misses      38816    38820       +4
+ Partials     3732     3730       -2

Impacted Files
Coverage Δ

modules/structs/repo.go
36.36% <ø> (ø)
:arrow_up:

models/repo.go
48.27% <100%> (+0.02%)
:arrow_up:

models/unit.go
62.16% <0%> (-5.41%)
:arrow_down:

models/repo_list.go
73.14% <0%> (-0.93%)
:arrow_down:

routers/repo/view.go
38.59% <0%> (-0.88%)
:arrow_down:

models/error.go
33.26% <0%> (+0.59%)
:arrow_up:

models/pull_list.go
59.85% <0%> (+2.11%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8f8c250...e8896a3. Read the comment docs.

@axifive do you like too look at this PR too? - would be nice (but you dont have to)


Reduce cardinality of pending_workloads metric

What type of PR is this?
/kind feature
What this PR does / why we need it:
Remove the queue label from the pending_workloads metric.
As opposed to ClusterQueues, Queues are more likely to be unbounded by administrators, which could lead to cardinality explosion.
OTOH, administrators are more likely to care about metrics per ClusterQueue. Users, who might be the owners of Queues, might not even have access to metrics. They can still check the Queue status.
Which issue(s) this PR fixes:
Ref https://github.com/kubernetes-sigs/kueue/pull/291#discussion_r916296493
Special notes for your reviewer:

/lgtm


Use debug! logging instead of info! for verbose logging

This issue has not been discussed, but I believe it simple enough that a discussion wasn't needed.  The info! log is meant for high level logs that are relevant to every invocation. The logs in this PR are only relevant when debugging so they should use the debug! logging macro.
I'm not sure who should look at this, but perhaps @peterhuene can take a look.

@sunfishcode no it doesn't seem like there's good docs on when to use which log. From what I've seen out there

info is high-level logs that you'd want to show on every normal run of the program
warn is something went wrong but it's not fatal and should just be checked out when you have a chance
error an error occured. It's not enough to crash the program but it's something that should be looked into ASAP
*debug any info that would be relevant to see the flow of the executing program
trace very verbose logs. Generally not helpful for program wide debugging but useful when inspecting a small segment of the program.

It looks like CI failed to build because ld couldn't link, but it gave no discernible diagnostic message, just returned an exit code of 1. Hopefully just a transient failure.

That always happens for me when the disk is full.

The failure is that CI is running out of disk space and we have some PRs up that will hopefully help mitigate the problem shortly.

@peterhuene This should be good to go now. Sorry for the delay!

@rylev I'm cleaning up old PRs and came upon this; are you still interested in this change? (Lots of conflicts now but it seems it wouldn't be too hard to redo the change in general) Sorry this got dropped on the floor beforehand!

@cfallin no worries! I'll close this. The user can always filter the logs fairly easily if it's too verbose.


Does this work with async and await?

No examples in the documentation with async and await. Does it work?
Many thanks

@ianengelbrecht why shouldn't it? If the library returns the Promise you can use await on it.

Yes, works fine with async/await


Install Keycloak using operator pattern on K8s cluster.

Updates:

Keycloak recommends using OLM
OLM installed successfully, but unable to access OLM dashboard
Installed nginx Ingress Controller, pending configuration for accessing OLM and Keycloak

Keycloak deployed using operator. Accessible over http. Need to enable https and provide SSL termination at Nginx controller.

Keycloak deployed using operator. enabled https and provided SSL termination at Nginx controller, enabled external database support for Keycloak, need to enable SSL termination for external database communication.

Keycloak deployed using operator. provided SSL termination at Nginx controller and  for external database communication.
For external database communication different types of SSL settings can be done, that comes with different level of security for database communication. Completed testing with SSLMODE allow, in which Keycloak generates self sign certificates for db communication (Refer to the PostgreSQL documentation link here) https://www.postgresql.org/docs/current/libpq-ssl.html


MathJax does not render properly in Jupyter Notebook with Sagemath kernel.

I'm running a Jupyter Notebook with a SageMath kernel. By default, I use %display latex to have LaTeX-style outputs from my computations, which is rendered by MathJax.
When I use the notebook in a browser window, everything works fine. But when I use it in VSCode, then it does not render automatically: when I run a cell, I see no output, and I need to manually select any other LaTeX-style output (I can get some by executing any %%latex cell), manually select any math renderer, and then all the other outputs that were not showing before magically appear. I need to do this every time I run a cell. Is there any solution?
Environment data

VS Code version: 1.51
Extension version: v2020.10.332292344
OS and version: Windows 10 Education
Python version: Python 3.7 in Sagemath 9.1

Expected behaviour
Render math automatically after running a cell.
Actual behaviour
Math is not rendered unless the renderer is manually reloaded.
Steps to reproduce:

Install a SageMath kernel.
Open a new Jupyter Notebook.
Run %display latex.
Run 1+1.

To render the math, run any %%latex cell.

I repro this problem @sashalr. Thanks for filing an issue.
This seems like a problem with the nteract Transforms that we are using to render the output from the 1+1 code cell. This is not a kernel communication problem--we are getting an execute_result update from the kernel in response to executing a cell containing 1+1, but the output to display has the text/html mimetype. In our UI we then route this to the appropriate nteract Transforms to render the html, and perhaps the Transform that we're using doesn't support LaTeX for text/html mimetypes.
Subsequently when running the cell containing %%latex, the data that comes back from the kernel has the text/latex mimetype, which we again hand off to the appropriate nteract Transform--this apparently causes the first cell to also rerender as LaTeX, though I'd need to debug this more to figure out why.
I believe our team is hoping to rework our rendering support, particularly as we move to the new native notebooks experience in VS Code. We'll discuss prioritization of a fix for this issue at our weekly triage.


Add a node (+ sample flow) for the scene classifier  model

Model URL: https://developer.ibm.com/exchanges/models/all/max-scene-classifier/

closed by #36


v0.9.0-rc1 issues

Please post any issues you have with<EMAIL_ADDRESS>here. It will not be installed as "latest" automatically yet. You will need to install it explicitly:
npm install -g vows@next
npm install vows@next

Or put the explicit version in your package.json:
{
  // other properties
  "devDependencies": {
    "vows": "0.9.0-rc1"
  }
}

the goal is to release 0.9.0 before the end of next week so please let us know if you have any problems with the release. There are some pretty big changes in there which are documented in the changelog. This is an important step to shipping 1.0.0 with some big wins so we appreciate your help!

@silkentrance these have been fixed. Thanks for your contributions!

Since all the issues reported with the 0.9.0 release have been fixed here and there have been further  comments on it in over a eyar, could it be released now?
0.9.0 would fix a deprecation warning that's currently happening when using vows -v with 0.8.1 run with Node.js 4.4.2. The warning is:
  util.print is deprecated. Use console.log instead.

This issue has already been fixed in latest code on Github.

We ended up going in a different direction, so I'm closing this omnibus issue


Discord splash command support

Pls make this bot for v13 discord.js

To be honest I can't upgrade to the new version. Fivem is giving an error. If anyone can do this, I'll update the codes, no problem


webfont.js 1.6.24 is not available on ajax.googleapis.com

At the moment of writing, the latest version of webfont.js in the repo is 1.6.24.
However on https://ajax.googleapis.com, it seems like the highest available version is 1.6.16 (  https://ajax.googleapis.com/ajax/libs/webfont/1.6.16/webfont.js
)
If I change the version to 1.6.17 or above, a 404 occurs.
Did I use an incorrect url? If not, what is the release schedule of the js file to the hosting site?

Does your Web Font Loader link tag come before your style.css link tag?
The following tag worked for me locally:
<script src="https://ajax.googleapis.com/ajax/libs/webfont/1.6.24/webfont.js"></script>

As an option, does this tag work?
<script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/webfont/1.6.24/webfontloader.js"></script>

This is how I load Roboto font:
<script src="https://ajax.googleapis.com/ajax/libs/webfont/1.6.24/webfont.js"></script>
<script>
  WebFont.load({
    google: {
      families: ['Roboto']
    }
  });
</script>
<link href="style.css" rel="stylesheet">

Google CDN does not get all releases for webfont loader (the current latest 1.6.16 for google cdn). See https://developers.google.com/speed/libraries/#web-font-loader for versions they are hosting. If you want any version, @nick3499 suggestion on cloudflare is the alternate approach


read replica not needed since we have multi-az enabled.

closes #61

@killerwails I added multi_az only in production and staging regions (us-west-2 and us-east-1). I feel like an additional var is just clutter. The same reason why we don't have a var for rds port.

logical. ty!

:rocket: Looks good, thanks @noqcks


QnAMaker trace/call shows up in log after intent was recognized correctly.

Setup a bot with Orchestrator recoginzer (could be LUIS) and QnAMaker. Notice how QnAMaker trace shows up after intent was correctly recognized. See images:

@scheyal do you have a question about this? the behavior seems correct.


Rework bin codacycoverage to use symfony/console

rework by use symfony/console
add support git-commit option
add support base-url option
small fixes in CodacyApiClient

@kirillkomlev very good work! I am OK with merging this after the minor issues (see my comments) are resolved.


SINGA-491 Code Cleaning with the Reference of LGTM Analysis Result

Since LGTM has been applied for our code analysis (see SINGA-484), I have cleaned up many of the obvious issues alerted by LGTM.
The compile result is okay as follows:
ubuntu@ip-172-31-39-137:~/incubator-singa/build$ rm -rf *
ubuntu@ip-172-31-39-137:~/incubator-singa/build$ cmake -D CMAKE_PREFIX_PATH="/usr/local/cuda/lib64;/usr/local/                          cuda/" -DENABLE_TEST=OFF -DUSE_CUDA=ON -DUSE_PYTHON3=ON -DUSE_MKLDNN=ON -DUSE_MODULES=OFF -DUSE_DIST=ON ..
-- The C compiler identification is GNU 5.4.0
-- The CXX compiler identification is GNU 5.4.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
-- Found Protobuf: /usr/local/lib/libprotobuf.so;-lpthread (found suitable version "3.0.0", minimum required i                          s "3.0")
-- Found CBLAS: /usr/local/include
-- Found GLOG: /usr/include
-- Found cuda_v10.0
-- Found CUDNN: /usr/local/cuda/include
-- Found Cudnn_7401 at /usr/local/cuda/include /usr/local/cuda/lib64/libcudnn.so
-- Found PythonInterp: /usr/bin/python3 (found suitable version "3.5.2", minimum required is "3")
-- Found PythonLibs: /usr/lib/x86_64-linux-gnu/libpython3.5m.so (found suitable version "3.5.2", minimum requi                          red is "3")
-- Found SWIG: /usr/local/bin/swig (found suitable version "3.0.12", minimum required is "3.0.10")
-- Found MKLDNN at /usr/local/include
-- Found MPI at /home/ubuntu/mpich-3.3/build/include
-- Found MPI lib at /home/ubuntu/mpich-3.3/build/lib/libmpi.so
-- Found all lib at /usr/local/lib/libprotobuf.so;/usr/local/lib/libopenblas.so;/usr/lib/x86_64-linux-gnu/libg                          log.so;/usr/local/cuda/lib64/libcudnn.so;/usr/local/cuda/lib64/libcudart.so;/usr/local/cuda/lib64/libcurand.so                          ;/usr/local/cuda/lib64/libcublas.so;/home/ubuntu/incubator-singa/build/lib/libcnmem.a;/usr/local/lib/libmkldnn                          .so;/home/ubuntu/mpich-3.3/build/lib/libmpi.so;/home/ubuntu/mpich-3.3/build/lib/libmpicxx.so
-- Found NCCL at /usr/local/cuda/include
-- Found NCCL lib at /usr/local/cuda/lib/libnccl.so
-- Configuring done
-- Generating done
-- Build files have been written to: /home/ubuntu/incubator-singa/build
ubuntu@ip-172-31-39-137:~/incubator-singa/build$ make -j4
Scanning dependencies of target cnmem
Scanning dependencies of target copy_protobuf
[  1%] Running C++ protocol buffer compiler on /home/ubuntu/incubator-singa/src/proto/model.proto
[  2%] Creating directories for 'cnmem'
[  3%] Running C++ protocol buffer compiler on /home/ubuntu/incubator-singa/src/proto/caffe.proto
[  4%] Running C++ protocol buffer compiler on /home/ubuntu/incubator-singa/src/proto/core.proto
[libprotobuf WARNING google/protobuf/compiler/parser.cc:547] No syntax specified for the proto file: core.prot                          o. Please use 'syntax = "proto2";' or 'syntax = "proto3";' to specify a syntax version. (Defaulted to proto2 s                          yntax.)
[libprotobuf WARNING google/protobuf/compiler/parser.cc:547] No syntax specified for the proto file: model.pro                          to. Please use 'syntax = "proto2";' or 'syntax = "proto3";' to specify a syntax version. (Defaulted to proto2                           syntax.)
[  5%] Running C++ protocol buffer compiler on /home/ubuntu/incubator-singa/src/proto/io.proto
[libprotobuf WARNING google/protobuf/compiler/parser.cc:547] No syntax specified for the proto file: io.proto.                           Please use 'syntax = "proto2";' or 'syntax = "proto3";' to specify a syntax version. (Defaulted to proto2 syn                          tax.)
[  6%] Performing download step (git clone) for 'cnmem'
Cloning into 'cnmem'...
[  7%] Copying Protobuf headers
[  7%] Built target copy_protobuf
[  8%] Building NVCC (Device) object src/CMakeFiles/cuda_compile_1.dir/core/tensor/cuda_compile_1_generated_ma                          th_kernel.cu.o
Scanning dependencies of target singa_objects
[  9%] Building CXX object src/CMakeFiles/singa_objects.dir/caffe.pb.cc.o
[ 10%] Building CXX object src/CMakeFiles/singa_objects.dir/io.pb.cc.o
[ 11%] Building CXX object src/CMakeFiles/singa_objects.dir/core.pb.cc.o
Already on 'master'
Your branch is up-to-date with 'origin/master'.
[ 12%] No patch step for 'cnmem'
[ 13%] Performing update step for 'cnmem'
Current branch master is up to date.
[ 14%] Performing configure step for 'cnmem'
-- The C compiler identification is GNU 5.4.0
-- The CXX compiler identification is GNU 5.4.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
[ 15%] Building CXX object src/CMakeFiles/singa_objects.dir/model.pb.cc.o
[ 16%] Building CXX object src/CMakeFiles/singa_objects.dir/utils/channel.cc.o
-- Detecting CXX compile features - done
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
-- Configuring done
-- Generating done
-- Build files have been written to: /home/ubuntu/incubator-singa/build/cnmem-prefix/src/cnmem-build
[ 17%] Performing build step for 'cnmem'
Scanning dependencies of target cnmem
[ 50%] Building CXX object CMakeFiles/cnmem.dir/src/cnmem.cpp.o
[100%] Linking CXX static library libcnmem.a
[100%] Built target cnmem
[ 18%] Performing install step for 'cnmem'
[100%] Built target cnmem
Install the project...
                           Please use 'syntax = "proto2";' or 'syntax = "proto3";' to specify a syntax version. (Defaulted to proto2 syn                          tax.)
[ 94%] Linking CXX shared library ../lib/libsinga.so
[libprotobuf WARNING google/protobuf/compiler/parser.cc:547] No syntax specified for the proto file: core.prot                          o. Please use 'syntax = "proto2";' or 'syntax = "proto3";' to specify a syntax version. (Defaulted to proto2 s                          yntax.)
[ 95%] Running Python protocol buffer compiler on /home/ubuntu/incubator-singa/src/proto/model.proto
[libprotobuf WARNING google/protobuf/compiler/parser.cc:547] No syntax specified for the proto file: model.pro                          to. Please use 'syntax = "proto2";' or 'syntax = "proto3";' to specify a syntax version. (Defaulted to proto2                           syntax.)
Scanning dependencies of target _singa_wrap
[ 96%] Building CXX object python/CMakeFiles/_singa_wrap.dir/__/src/api/singa_wrap.cxx.o
[ 97%] Built target singa
[ 98%] Linking CXX shared library singa/_singa_wrap.so
[100%] Built target _singa_wrap

The test results are okay as follows:
ubuntu@ip-172-31-39-137:~/incubator-singa/examples/autograd$ python3 mnist_cnn.py
Starting Epoch 0:
Training loss = 582.689453, training accuracy = 0.794207
Evaluation accuracy = 0.937200, Elapsed Time = 4.268453s
Starting Epoch 1:
Training loss = 231.800217, training accuracy = 0.922559
Evaluation accuracy = 0.950120, Elapsed Time = 4.180309s
Starting Epoch 2:
Training loss = 168.152725, training accuracy = 0.943103
Evaluation accuracy = 0.966346, Elapsed Time = 4.185745s
Starting Epoch 3:
Training loss = 135.532410, training accuracy = 0.954259
Evaluation accuracy = 0.977464, Elapsed Time = 4.186186s
Starting Epoch 4:
Training loss = 116.884033, training accuracy = 0.960312
Evaluation accuracy = 0.972256, Elapsed Time = 4.202717s
Starting Epoch 5:
Training loss = 104.868248, training accuracy = 0.965815
Evaluation accuracy = 0.976462, Elapsed Time = 4.200955s
Starting Epoch 6:
Training loss = 95.433990, training accuracy = 0.967766
Evaluation accuracy = 0.980869, Elapsed Time = 4.190672s
Starting Epoch 7:
Training loss = 87.900131, training accuracy = 0.970418
Evaluation accuracy = 0.983273, Elapsed Time = 4.217619s
Starting Epoch 8:
Training loss = 81.392876, training accuracy = 0.973102
Evaluation accuracy = 0.985978, Elapsed Time = 4.247715s
Starting Epoch 9:
Training loss = 77.320335, training accuracy = 0.973969
Evaluation accuracy = 0.980869, Elapsed Time = 4.236908s

ubuntu@ip-172-31-39-137:~/incubator-singa/test/python$ python3 test_operation.py
...............................................................................WARNING: Logging before InitGoo                          gleLogging() is written to STDERR
I0920 09:42:13.534116  5992 tensor_math_cpp.h:146] not equal stride
I0920 09:42:13.534477  5992 tensor_math_cpp.h:146] not equal stride
.I0920 09:42:13.535104  5992 tensor_math_cpp.h:146] not equal stride
I0920 09:42:13.535485  5992 tensor_math_cpp.h:146] not equal stride
....
----------------------------------------------------------------------
Ran 84 tests in 0.963s

OK

ubuntu@ip-172-31-39-137:~/incubator-singa/examples/autograd$ python3 resnet.py
Start intialization............
100%|███████████████████████████████████████████████████████████████████████| 100/100 [01:27<00:00,  1.14it/s]
Throughput = 36.70721025063381 per second
Total=0.871763334274292, forward=0.27190722942352297, softmax=0.002759065628051758, backward=0.597097039222717                          3, sgd=0.018022277355194093

Thank you.
In future we should not accept only the compile output alone, we should also run the unit tests and make sure they pass with no errors. This must be done on all supported platforms, not only on the developer machine.
This is why the Travis continuous integration needs to be improved to automate this task which is very hard to be done manually.

Thanks so much for your comments. It is a great idea concerning the CI.


Needs to actually attach the version number to export zip

This version screen needs to attach the requested version to the filename too. not just the internal manifest.json

This is what is desired


Redisson fails to recognize slave that become a master due to master down in Redis Cluster

Hi Niko, seems like problem really exists with redisson driver for cluster.
Here is our configuration:
aa19c3c2a927384b6a4673539cd7dcdb2efa33e8 <IP_ADDRESS>:7001 slave d20242223349a4d51b30da40dad09f0dbc9a2cc8 0<PHONE_NUMBER>219 1 connected d20242223349a4d51b30da40dad09f0dbc9a2cc8 <IP_ADDRESS>:7000 myself,master - 0 0 1 connected 0-5461 ba12c571516b130a9973fdc95ba8cebbfaf5ac7e <IP_ADDRESS>:7000 master - 0<PHONE_NUMBER>222 3 connected 5462-10922 4a21be5ffb25369bf4243918c34467103fa9c8a4 <IP_ADDRESS>:7001 slave ba12c571516b130a9973fdc95ba8cebbfaf5ac7e 0<PHONE_NUMBER>222 3 connected f6b4e5a94409ca397731165acad90f2193154c62 <IP_ADDRESS>:7000 master - 0<PHONE_NUMBER>422 0 connected 10923-16383 6a7c1e8510d29aed17dc1e0ca7f0e6a7278b4de5 <IP_ADDRESS>:7001 slave f6b4e5a94409ca397731165acad90f2193154c62 0<PHONE_NUMBER>721 5 connected
I'm shutting down Redis instances on <IP_ADDRESS>:  ba12c571516b130a9973fdc95ba8cebbfaf5ac7e and aa19c3c2a927384b6a4673539cd7dcdb2efa33e8
When I did this Redis cluster reorganized to following:
aa19c3c2a927384b6a4673539cd7dcdb2efa33e8 <IP_ADDRESS>:7001 slave,fail d20242223349a4d51b30da40dad09f0dbc9a2cc8<PHONE_NUMBER>304<PHONE_NUMBER>700 1 disconnected d20242223349a4d51b30da40dad09f0dbc9a2cc8 <IP_ADDRESS>:7000 myself,master - 0 0 1 connected 0-5461 ba12c571516b130a9973fdc95ba8cebbfaf5ac7e <IP_ADDRESS>:7000 master,fail -<PHONE_NUMBER>204<PHONE_NUMBER>202 3 disconnected 4a21be5ffb25369bf4243918c34467103fa9c8a4 <IP_ADDRESS>:7001 master - 0<PHONE_NUMBER>791 6 connected 5462-10922 f6b4e5a94409ca397731165acad90f2193154c62 <IP_ADDRESS>:7000 master - 0<PHONE_NUMBER>787 0 connected 10923-16383 6a7c1e8510d29aed17dc1e0ca7f0e6a7278b4de5 <IP_ADDRESS>:7001 slave f6b4e5a94409ca397731165acad90f2193154c62 0<PHONE_NUMBER>293 5 connected
As you can see cluster is ok, but requests are failing for some reason.
If I restart tomcat without touching anything on Redis requests are not failing.
Very strange.
This is the link to the redisson log: https://www.dropbox.com/s/pluyzxtxpd0u23z/redisson.log?dl=0
Please assist,
Thank you!

As I see you have shutdown all nodes for slot range [0-5461], so any objects bounded to this range will be unavailable

I didn't review your log before, sorry. It seems like it's a real bug. Will fix it

Could you please update to build from "master" and try it due to bug with slots migration. Then please share your log

Sorry, didn't understand your request.
What exactly you want me to do?

Yes

Fixed. Please try

This is the log of exactly the same scenario, but with last redisson-2.2.12 (seems like before your fix :)
Please see: https://www.dropbox.com/s/7vm1g028vhfz2an/redisson_2.2.12.log?dl=0

I checked again same scenario after your fix with same results, falling requests.
JMeter output:
summary +      1 in     0s =   35.7/s Avg:    28 Min:    28 Max:    28 Err:     0 (0.00%) Active: 3 Started: 3 Finished: 0 summary +    102 in   7.2s =   14.2/s Avg:   178 Min:     3 Max:  4445 Err:     4 (3.92%) Active: 10 Started: 10 Finished: 0 summary =    103 in   7.2s =   14.3/s Avg:   177 Min:     3 Max:  4445 Err:     4 (3.88%) summary +    118 in    10s =   11.9/s Avg:   523 Min:     3 Max:  4405 Err:    14 (11.86%) Active: 17 Started: 17 Finished: 0 summary =    221 in  13.2s =   16.7/s Avg:   361 Min:     3 Max:  4445 Err:    18 (8.14%) summary +    123 in  10.4s =   11.9/s Avg:   894 Min:     2 Max:  4417 Err:    25 (20.33%) Active: 25 Started: 25 Finished: 0 summary =    344 in  19.2s =   17.9/s Avg:   552 Min:     2 Max:  4445 Err:    43 (12.50%) summary +    136 in    10s =   13.6/s Avg:  1067 Min:     2 Max:  4408 Err:    33 (24.26%) Active: 32 Started: 32 Finished: 0 summary =    480 in    25s =   19.2/s Avg:   698 Min:     2 Max:  4445 Err:    76 (15.83%)
Attaching redisson log(after the fix): https://www.dropbox.com/s/eyqllb66qz49w6b/redisson_2.2.12_after_fix.log?dl=0

It's strange I don't see "cluster nodes state from" since 20 Apr 2016 14:55:56 from your log. Cluster state update method should try next cluster node to get cluster state. But in your case something goes wrong.

we also have Connection pool exhausted! error all the time, may be these are related?
20 Apr 2016 14:57:12  ERROR - An error occurred when opening a session. Error: Connection pool exhausted! All connections are busy. Increase connection pool size.
20 Apr 2016 14:57:12  ERROR - An error occurred when opening a session. Error: Connection pool exhausted! All connections are busy. Increase connection pool size.
20 Apr 2016 14:57:13  ERROR - An error occurred when opening a session. Error: Connection pool exhausted! All connections are busy. Increase connection pool size.
20 Apr 2016 14:57:13  ERROR - An error occurred when opening a session. Error: Connection pool exhausted! All connections are busy. Increase connection pool size.
20 Apr 2016 14:57:13  ERROR - An error occurred when opening a session. Error: Connection pool exhausted! All connections are busy. Increase connection pool size.
20 Apr 2016 14:57:14  ERROR - An error occurred when opening a session. Error: Connection pool exhausted! All connections are busy. Increase connection pool size.
20 Apr 2016 14:57:14  ERROR - An error occurred when opening a session. Error: Connection pool exhausted! All connections are busy. Increase connection pool size.
20 Apr 2016 14:57:14  ERROR - An error occurred when opening a session. Error: Connection pool exhausted! All connections are busy. Increase connection pool size

I'll take a look at this tomorrow

Important to note that problem exists only if I shutdown redis instances on <IP_ADDRESS>
When I shutdown redis instances on <IP_ADDRESS> redisson works well, even despite that I can see in the log Connection pool exhausted! error.

Please try again

I did same scenario on different environments and the problem always occurred if I'm shutting down redis instances on the middle machine.
Like on this picture: https://www.dropbox.com/s/nei0va82j2hzymo/redis-cluster-middle-host-down.jpg?dl=0

+1 for this bug. I get it on stage every time trying to check redis sentinel configuration.
Jedis works nice (application uses both libraries).

@ybonda so nothing has changed at all with new fix?

Not tested yet.
Will test it in 10 min

Will wait for your results

Yes Niko!
This fixes the issue.
I did a test not only for the middle host, but also check it on all possible scenarios and it worked.
Great job man!
Thanks a lot!

@ybonda  At your service!

@thelamon could you please test it too?

@mrniko Thanks for the service!Could test it next Monday only (

@thelamon OK I'll close it then. If you will have any problems please post a comment


Wait for DB container to be extracted

Currently ETAJ does not wait for the script language container contained in the Exasol database (< version 8) to be extracted.
This can take ~ 5 minutes, and hence ETC offers method waitUntilContainerStarted() in ExasolContainer line L530, but ETAJ does not call this method.
This led to problems while working on KCE issue #72.
Acceptance Criteria
ETAJ, ExasolTestcontainerTestSetup line 25 does not specify any services, e.g. UDF, BucketFS, ...
Therefore ETC would not wait for SLC to be extracted even when asked for.
Proposal:

enhance ETAJ to require all services by default and call method waitUntilContainerStarted() unconditionally.

Note: As mentioned by @tkilias Docker Containers for Exasol DB version 8 and higher are probably not affected as these are using a container format with SLC contents already extracted, not requiring additional extraction.

When looking at @pj-spoelders' log files we identified that SLC had already been extracted:
Log file /exa/logs/cored/bucketfsd.*.log contained a line matching regular expression "ScriptLanguages.*extracted$" as exasol-testcontainers / UdfContainerWaitStrategy searches for.
So currently we assume a different issue and propose to

setup script logging
run the SQL statements again, e.g. with an SQL client such as DBeaver
inspect output of nc

@ckunki is this ticket still valid? I read your last comment as a "no".

I propose to add .withRequiredServices(ExasolService.BUCKETFS, ExasolService.JDBC, ExasolService.UDF) when starting the container. This ensures that it waits until the SLC is extracted.


Should modules update automatically?

Force is set to $True by default, but modules aren't updating automatically. Should the action update to the latest version of a module automatically or am I expecting a behaviour that's not as intended?
I use the action in a few workflows (e.g. update-json.yml); however, unless I specify a module version, the modules don't update to newer versions of a module.

You are expecting a behavior that is not intended, but could be a very cool feature. I'll see what I can do 👍🏼

Thanks for the quick reply. I've started using psmodulecache in my workflows, because I think it's more elegant than using a script. I am using the action in a few workflows that uses a specific module that updates often, hence looking to get modules to update.
In the meantime, I've added Update-Module to my workflows.

I also have this need for the following scenario:
Before uploading my modifications I use a local build on my computer, this build is reused in a remote CI.
In this case, if you don't specify a version when installing the necessary modules, you can have different versions locally and in the VM.
The build can therefore fail following breaking changes in the modules, in these cases I used a semi-automatic update locally. Since the CI always uses the latest version.
(this way of doing things is also useful to me if I work in an environment without internet access).
If I use this scenario with PSModuleCache, I will encounter the same problem.
A possible solution:
1- Add a boolean 'Updatable' parameter to the Action.
The value 'True' indicates to check for updates for the requested modules. We add in the cache key the cache type, here 'Updatable' (can be useful in Github CLI).
The value 'False' does not trigger any updates of the requested modules. We add in the cache key the cache type, here 'Immutable'.
2 - Enrich the syntax of the 'modules-to-cache' parameter in order to allow:
-to use a fixed version (required), which already exists,
-to use a module without specifying its version or its update (regardless of the version [the one in the cache or in the repository]), which already exists,
-to use a module by specifying its update (always use the latest version), which does not exist.
The workflow does not need to know the version number, but to know for which modules it forces an update at each execution.
For the latter case, the '::' pattern can be used without indicating a version number. Its presence indicates the triggering of a search for a new version and the creation of a new cache key if necessary.
(To date there is no naming convention for module names, so you can find any character there except those prohibited by the targeted FileSystem. Here it does not interfere because you create a key name not a file name)
Key examples :
For :
module-to-cache='PSFramework,Pester,dbatools:1.1.0,DTW.PS.FileSystem'
Updateble=false
The key is 'Windows-v4.5-Immutable-powershell-pwsh-PSFramework-Pester-dbatools:1.1.0--DTW.PS.FileSystem'
For :
module-to-cache='PSFramework,Pester::,dbatools:1.1.0,DTW.PS.FileSystem'
Updateable=true
The key is 'Windows-v4.5-Updatable-powershell-pwsh-PSFramework-Pester:5.3.3-dbatools:1.1.0-DTW.PS.FileSystem'
Note that 'allow-prerelease' will influence the version fetched.
Example https://www.powershellgallery.com/packages/PnP.PowerShell/1.11.0
For this module we get a new version every day.
We must add consistency checks:

This also includes managing multiple repositories :

Example of a result:

This issue has been resolved 😊  thank you again Laurent!


Please update to 1.21.3 (crashes game when try to load on that version)

@adamviola Please update the mod to 1.21.3

@mrjasonn https://github.com/Merphyx/BoostedBrightness-1.21.x


Add additional permissions to operator in chart

Summary

Follow up of #13

Checklist
For Helm Chart changes

[x] Categorize the PR by setting a good title and adding one of the labels:
bug, enhancement, documentation, change, breaking, dependency
as they show up in the changelog
[x] PR contains the label area:chart
[x] PR contains the chart label, e.g. chart:appcat-service-s3
[x] Variables are documented in the values.yaml using the format required by Helm-Docs.
[x] Chart Version bumped if immediate release after merging is planned
[x] I have run make chart-docs
[x] Link this PR to related code release or other issues.

PR is based on #13 , should be rebased after merging


Also set GitBuildVersionSimple cloud variable

For use cases that require just a simple semver, also set the
GitBuildVersionSimple cloud build variable from BuildVersionSimple.

Thanks for contributing.


Upgrade altool to notarytool

Apple no longer supports using altool for notarization so the pipelines are currently failing. We need to update the pipeline to switch over to notary tool so that we can continue to build main.

Gon is also deprecated so we'll need to rip that out and switch over to running everything manually. This covers some of it. This covers the migration to notarytool.

Fixed by #254


Update drop to be consistent with gpt3 paper

Also has changes for coqa to be consistent with gpt3 paper

and triviaqa data

Also added storycloze data (need to implement download since there is no current public download link)


Making forecasts with dynamic features

I have successfully used run_eval.py on a few of the long sequence forecasting data set, and the results look promising. From previous posts, I can see how a univariate sequence can be applied using the model.generate().
Now I'm wondering how we could make forecasts on a multivariate customized data with specified dynamic features and visualize the results or output a pandas dataframe? Any suggestion is appreciated.

Thank you for your interest in our work! Currently, Time-MoE supports only univariate time-series forecasting and does not accommodate forecasting with dynamic features. However, if you wish to forecast a multivariate time series, we recommend converting it into multiple univariate time series and then using Time-MoE for forecasting.
If you have any questions or need further assistance, please feel free to contact us.

Thanks for your input. When I tried using the code below, the prediction results seem much worse than using run_eval.py script. I assume it's because the data loader module in evaluation actually used all variates with channel independence, right? How can we integrate this when visualizing the results?
`
path = "./dataset/ETT-small/ETTh1.csv"
df = pd.read_csv(path)
model = AutoModelForCausalLM.from_pretrained(
"./model/",
device_map="cpu",
trust_remote_code=True,
)
seqs = torch.tensor(df["OT"].values).float().unsqueeze(0)
max_val = seqs.max(dim=-1, keepdim=True)[0]
normed_seqs = seqs / max_val
forecast
output = model.generate(normed_seqs, max_new_tokens=prediction_length)
normed_predictions = output[:, -prediction_length:]
predictions = normed_predictions * max_val
`

These benchmark datasets use standard normalization. The mean and standard deviation are calculated for each channel using the training split of the dataset and then applied to the test split. For more details, please refer to the process described here: GitHub link. You can use this as a guide for your implementation.
If you have any questions, please feel free to contact us.


Keep tsd definitions inside the repo ?

I see a couple of advantages to do so:

No more error with Travis (or less at least)
Already referenced ./tools/typings/tsd/**/*.d.ts in tsconfig.json files

right now if we just clone, install, start ... tsd definition are missing in tsconfig

Travis is failing because of GitHub rate limit, which can be easily solved as described in #176
If the point is keep this repo as barebone as possible, then I wouldn't include these typings. And imho it's good that repo contains only things that we have coded, not 3rd party libraries.
But generally I think there is no "wrong" or "right" answer for this.

My main concern here is

right now if we just clone, install, start ... tsd definition are missing in tsconfig

@ludohenin The problem that the typings are missing in tsconfig.js could be solved with adding the following:
{
    "filesGlob": [
        "!./tools/typings/tsd/**/*.d.ts"
    ]
}

I don't know how other IDE's treat the tsconfig.json, but atom-typescript doesn't mark e.g. any express imports as errors (tools/utils/server.ts) If I exclude tsd-typings in filesGlob. Would this be a valid solution for one of your arguments?
IMHO including the tsd typings in git is really dirty project design. It adds unnecessary files to the repository, because their versions are already defined in tsd.json. One wouldn't include node_modules, or bower_components either.
I think doing this just because of the git rate limit and travis is not worth the cons.


Release version 0.46.0

Follow on to https://github.com/sqlparser-rs/sqlparser-rs/issues/1162
Filing a ticket here to track the work of a release (mostly so others can follow along)
Target Release Date: Approximately 2024-05-12
Release instructions: https://github.com/sqlparser-rs/sqlparser-rs/blob/main/docs/releasing.md
Steps:

[ ] Make a PR to update CHANGELOG.md:
[ ] run cargo release
[ ] verify the new release is on crates.io
[ ] Make ticket for next release:

Per @jmhain and @iffyio  let's try and get https://github.com/sqlparser-rs/sqlparser-rs/pull/1239 in as well (see https://github.com/sqlparser-rs/sqlparser-rs/pull/1239#issuecomment-2087617934)

Got a new release https://crates.io/crates/sqlparser 🎉
Next version tracked https://github.com/sqlparser-rs/sqlparser-rs/issues/1251


[3.2] Add MP3 import and playback support (Backport)

This is a 3.2 backport for mp3 playback and import support. Tested and working with mono and stereo files.

Thanks!


Feature support java11

What is the purpose of the change
aa
Brief changelog
aa
Verifying this change
aa
Follow this checklist to help us incorporate your contribution quickly and easily. Notice, it would be helpful if you could finish the following 5 checklist(the last one is not necessary)before request the community to review your PR.

[x] Make sure there is a Github issue filed for the change (usually before you start working on it). Trivial changes like typos do not require a Github issue. Your pull request should address just this issue, without pulling in other changes - one PR resolves one issue.
[x] Format the pull request title like [ISSUE #123] Fix UnknownException when host config not exist. Each commit in the pull request should have a meaningful subject line and body.
[x] Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
[x] Write necessary unit-test(over 80% coverage) to verify your logic correction, more mock a little better when cross module dependency exist. If the new feature or significant change is committed, please remember to add integration-test in test module.
[x] Run mvn -B clean apache-rat:check findbugs:findbugs checkstyle:checkstyle to make sure basic checks pass. Run mvn clean install -DskipITs to make sure unit-test pass. Run mvn clean test-compile failsafe:integration-test  to make sure integration-test pass.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

Coverage increased (+0.03%) to 51.333% when pulling ed94f118094c24dda15fd91a12ff8e4820d41fa7 on feature_support_java11 into 9f95a972e10e0681bc3f2d00e9957aa212e897b5 on master.

Codecov Report

Merging #2408 (ed94f11) into master (9f95a97) will decrease coverage by 0.10%.
The diff coverage is 66.66%.

@@             Coverage Diff              @@
##             master    #2408      +/-   ##
============================================
- Coverage     45.69%   45.59%   -0.11%
+ Complexity     4270     4264       -6
============================================
  Files           546      546
  Lines         35860    35860
  Branches       4763     4763
============================================
- Hits          16387    16351      -36
- Misses        17429    17456      +27
- Partials       2044     2053       +9

Impacted Files
Coverage Δ
Complexity Δ

.../java/org/apache/rocketmq/acl/common/AclUtils.java
83.12% <ø> (ø)
66.00 <0.00> (ø)

...mq/client/impl/producer/DefaultMQProducerImpl.java
36.21% <0.00%> (ø)
69.00 <0.00> (ø)

...cketmq/acl/plain/RemoteAddressStrategyFactory.java
91.20% <100.00%> (ø)
14.00 <0.00> (ø)

...ketmq/client/impl/consumer/PullMessageService.java
71.11% <0.00%> (-8.89%)
8.00% <0.00%> (-2.00%)

...pache/rocketmq/test/listener/AbstractListener.java
66.66% <0.00%> (-7.41%)
9.00% <0.00%> (-1.00%)

...org/apache/rocketmq/common/stats/StatsItemSet.java
44.77% <0.00%> (-5.98%)
17.00% <0.00%> (-2.00%)

...in/java/org/apache/rocketmq/test/util/MQAdmin.java
38.88% <0.00%> (-5.56%)
7.00% <0.00%> (-1.00%)

...ain/java/org/apache/rocketmq/test/util/MQWait.java
43.24% <0.00%> (-5.41%)
3.00% <0.00%> (ø%)

...ava/org/apache/rocketmq/test/util/VerifyUtils.java
46.26% <0.00%> (-2.99%)
14.00% <0.00%> (-1.00%)

...nt/impl/consumer/ConsumeMessageOrderlyService.java
42.59% <0.00%> (-2.89%)
16.00% <0.00%> (-3.00%)

... and 12 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9f95a97...ed94f11. Read the comment docs.


Investigate moving dependency from extinction to dust_extinction

Do we want to moe to dust_extiction?

Ping, @jchiang87

all places (not tests or docs) with extinction:
https://github.com/sncosmo/sncosmo/blob/f2a7d6736b48d62788c4dfaf4c7ad68e980adc78/sncosmo/models.py#L1287
https://github.com/sncosmo/sncosmo/blob/f2a7d6736b48d62788c4dfaf4c7ad68e980adc78/sncosmo/models.py#L1997
https://github.com/sncosmo/sncosmo/blob/f2a7d6736b48d62788c4dfaf4c7ad68e980adc78/sncosmo/models.py#L2013
https://github.com/sncosmo/sncosmo/blob/f2a7d6736b48d62788c4dfaf4c7ad68e980adc78/sncosmo/models.py#L2027
https://github.com/sncosmo/sncosmo/blob/f2a7d6736b48d62788c4dfaf4c7ad68e980adc78/sncosmo/models.py#L2032

So, I got curious and compared the implementations of F04, F99, and CCM89 in extinction and dust_extinction.
CCM89 is the same within rounding error, and F99 differs by <0.2%. F04 appears to be something different than extinction's fm07, differing by >5%, even though the comments in dust_extinction seem to imply it was based on the 2007 paper.
Model evaluation is >30x faster in extinction than in dust_extinction, though I'm not sure that's a bottleneck worth worrying about.
Notebook here: https://gist.github.com/jvansanten/952a323b0f0b54acfc84076a2c2c5984
Also, extinction.apply(mag, flux) is literally just flux * 10**-0.4*mag

Hi all, apologies for the radio silence. I just left a note about this issue on the related extinction github PR. would the sncosmo maintainers be interested in moving extinction into the sncosmo organization and taking over maintenance?
Regarding a move to dust_extinction, I do see the motivation for moving to dependency that actually has a maintainer! The original motivation for the extinction package was that the extinction calculation was, at some point, a significant contribution to the overall runtime of a light curve fit. It's probably not significant now, but with 30x slower extinction calculation it might be. It's probably worth some benchmarking before doing such a move, but of course it's ultimately up to the current maintainers. ❤

I am willing to have extinction moved and maintain it along side SNCosmo. That 30x speed hit might be too much for us to move to the more generally used package.

Thanks @benjaminrose! I've transferred the repo. Do you have a PyPI account? I should add you as a maintainer or owner of sncosmo and extinction there.

@kbarbary I am Ben_Rose on PyPI.

Added you


Fix nick changes wrongly reported

Before
After

⚠ Note that this fix will only work after #1771 gets merged. The bug this PR fixes is due to user.nick being set too early and msg.from being modified after instantiating msg.

I wonder if we want to do something to link old nicks to new nicks though? I.e. in astorije2 is now known as astorije3 if we could link astorije2 to astorije3

@MaxLeiter I'd rather not, you'd have to link further nick changes, and if user rejoins the channel the whole thing falls apart.


WRK-7 and WRK-8 supporting changes

Shift the context around control sessions and heartbeats to the underlying protocols.
This is to help tackle WRK-8 as part of the list of simplifications.

Reviewed on the call on the 6th December 2022.


Create robo-delivery-11-12 final code

created final code for project

TODO:

[ ] rename lesson-final-code/robo-delivery-11-12.md to lesson-final-code/robo-delivery-11-12-code.md to match existing style
[ ] add lesson-final-code/robo-delivery-11-12-code.md to pxt.json under files with the other tutorial docs


Clean up TODO item - serialize without buffer

Hi,
trying to get my feet wet and understand your code as I may need to work with orientdb in the near future and would like to help in what little way I can. I noticed this todo in your code and thought you might like a small patch to avoid the Buffer allocation. Hopefully this was what you were thinking about doing. If not then no harm.
Change log:
Use encoding/binary to directly serialize 10 bytes rather than
allocating a Buffer object.

add type checking to incoming Link and return an error if coercion invalid

Hi yanfali,
Thanks for the contribution.  This looks great!
Sorry for the delay, I just got back from a family emergency "vacation" where I didn't have internet access for a week.  I've made a number of changes in the last week that I will push tomorrow, but this section should merge fine.  I'll pull this in now and then push my other changes later today.


New settings for custom kwargs

Overview

Added settings for render_mode and title_format. These can be passed to show() through settings.
the settings have default values (defined in constants.py)
they were added to SettingsStore
Added derived identifiers for building title
Added utility to create list of active filter expressions
Added utility to remove units in parenthesis (for title - axis labels not affected)
Adds generated title to kwargs, works on all charts
In the process of testing, found and fixed:

wordcloud didn't pass title to imshow
histogram was missing y (comes into play with orientation / histfunc etc)
pie and scatter_matrix missing color
scatter_matrix missing symbol and size

even with default title, it can be overwritten with custom kwargs
it is also possible to add to default title by using {title} in the title custom kwarg (ie. Chart 6: {title}

Note, default title format is:
"{name}: {title_columns}{title_dimensions}{names}{title_y}{title_z}{over_by}{title_x} {selection}
"\
"{groupings}{filters} {title_trendline}"

Definition of the fields (also in utility.py comment):
Besides all dragger selections (x, y, z, color etc), and all custom kwargs, extra template variables are:

          date: current datetime
          filters: active filter expressions, comma delimited
          title_x: x minus units and with log scale if selected
          title_y: y minus units and with log scale if selected
          title_z: z minus units and with log scale if selected
          title_dimensions: dimensions list minus units
          title_columns: columns list minus units
          title_trendline: trendline description
          vs: when doing a title with x vs y, use {x}{vs}{y}
          over_by: when doing a title y over x, use {y}{ober_by}{x}. Preferred. for distributions, will use "by"
          name: dataframe name
          total: total number of observations
          subset: observations with active filters applied
          selection: ready to use string representing {subset} observations of {total}
          groupings: groupings tied to Legend and not on legend: marker_symbol, line_group, size etc

The title can also contain a few select html tags, like , , ,
 etc (whatever plotly supports)
Testing
render_mode
Passed render_mode = 'auto', 'svg', 'webgl' and not passing it at all, worked as intended. svg was a few times slower than webgl/auto on a large set as expected, but at least that allows use of pandasgui without webgl.
Title
Histogram
filter applied:

testing orientation:

Scatter
size not in legend, but in subtitle:

Lowess trendlines (requires statsmodels to be installed and the use of the trendline custom kwargs set to lowess:

OLS trendlines (requires statsmodels to be installed and the use of the trendline custom kwargs set to ols:

Line
This will have to be revisited once line and bar have groupby as optional, title is not totally accurate as it is not reflecting masked observations and the average.
Using log scale on Y:

Regular scale:

Bar
See comment above on groupby for lines (for bar, in one case, average, in the other would be a count)
This is the current result with groupby:

With groupby disabled it would have looked something like this:

Box
Works fine with dark theme:

Title above has additional text, entered through custom kawrgs:

While on the subject of dark theme, trying a different plot type:

Violin
[edited] Violin has similar title to box plot

Scatter 3D
most basic 3d plot:

3d with log scale on Y, filter applied and using size:

Heatmap
Heatmap with X/Y:

Heatmap with X/Y/Z default aggregation (note, this is sum for heatmap):

Heatmap with X/Y/Z specified histfunc of max:

Contour
Contour with X/Y:

Contour with X/Y/Z default aggregation (note, this is count for contour):

Heatmap with X/Y/Z specified histfunc of max:

Pie
Standard pie chart:

Pie chart with custom kwarg:

Scatter matrix
Multiple dimensions (with units removed in title), no filters, origin / model year in legend (color, symbol), size in subtitle, all possible elements in play:

Wordcloud
With filter applied:

Closes
This closes #101
This closes #102
END OF LINE

Nice work thanks


some temporary text on login / signup pages

Until these are enabled - can we put a message explaining the current state of affairs?

Done. I want to add a link or duplicate of the "register your interest" form once that's sorted.
@andybak let me know if the text is ok.

Wording is fine. Thanks


arch/risc-v/src/mpfs/mpfs_ddr.c: Correct erroneous register addresses…

… in DDR training
Writes to MPFS_CFG_DDR_SGMII_PHY_EXPERT_DFI_STATUS_OVERRIDE register were not done properly. Use correct address for writes.

This is still pending testing on real HW, will do that tomorrow

Good catch!


Documentation / Confiuration Options

The v1 Chart had a table with all the configuration options.  See here
I couldn't find the same documentation for this chart.
Is there one?

@DanielHabenicht
Thanks for your question.
The list of available options is available here

That's all?
The list for v1.x was way more extensive. But 2.x seems young so it's ok that there is not as much functionality as in v1.
As soon as I find out how to configure let's encrypt with traefik and helm I will contribute :)

@DanielHabenicht yes that's all and we haven't plan to add more things on it.
With fields available in the values.yaml you can configure all Traefik features.
We have decided to have another approach to have something more generic and simplest to maintain in the long term.


KeyError when running task from another queue

You have multiple queues created. e.g.
queue1 = AtLeastOnceQueue(tasks={'task1': task1})
queue2 = AtLeastOnceQueue(tasks={'task2': task2})

Once woken, workers will work through all the tasks on the main job table.
But this worker's queue may not have a task which matches one in its available task list. You'll get a KeyError here:
https://github.com/gavinwahl/django-postgres-queue/blob/master/dpq/queue.py#L23
Instead of raising KeyError, the worker should do something useful. e.g.

Ignore the task, it obviously was meant for someone else
Not fetch that task in the first place, since it cannot complete it
Raise a specific error that could be caught e.g. DpqInvalidTask

I will make a PR with a test case highlighting the problem, and I think option 1 or 2 above is the best, i'll implement one of them.

I think if this happens the most likely cause is a programming error that should be reported immediately and not ignored. For example adding a new task type without restarting your worker. I'd want to get an error as soon as possible.

My PR has a test case showing the error...


[#7758] Update for reactor netty 1.x

backport #7758 #7761

Codecov Report

Merging #7772 (4dfe3dc) into 2.2.x (4bc0962) will decrease coverage by 0.04%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##            2.2.x    #7772      +/-   ##
==========================================
- Coverage   38.87%   38.82%   -0.05%
==========================================
  Files        3487     3488       +1
  Lines       99907    99920      +13
  Branches    12284    12286       +2
==========================================
- Hits        38834    38796      -38
- Misses      58070    58120      +50
- Partials     3003     3004       +1

Impacted Files
Coverage Δ

...point/plugin/reactor/netty/ReactorNettyPlugin.java
0.00% <0.00%> (ø)

...lOperationsOnInboundCompleteMethodInterceptor.java
0.00% <0.00%> (ø)

...r/context/storage/AsyncQueueingUriStatStorage.java
73.52% <0.00%> (-16.18%)
:arrow_down:

...rp/pinpoint/rpc/control/ControlMessageDecoder.java
64.28% <0.00%> (-14.29%)
:arrow_down:

...p/pinpoint/rpc/stream/StreamChannelRepository.java
75.00% <0.00%> (-10.00%)
:arrow_down:

...pc/server/LoggingServerMessageListenerFactory.java
90.00% <0.00%> (-10.00%)
:arrow_down:

.../pinpoint/rpc/client/PinpointClientHandshaker.java
58.74% <0.00%> (-7.00%)
:arrow_down:

...orp/pinpoint/profiler/sender/NioUDPDataSender.java
72.00% <0.00%> (-4.00%)
:arrow_down:

...rc/main/java/com/navercorp/pinpoint/test/Item.java
70.00% <0.00%> (-3.34%)
:arrow_down:

...tor/metric/datasource/DefaultDataSourceMetric.java
75.00% <0.00%> (-3.13%)
:arrow_down:

... and 6 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4bc0962...4dfe3dc. Read the comment docs.


PTP synchronization related query

Hi,
I am testing the Qbv scheduling and running the PTP synchronization using the below command.
ptp4l -i eth0 -p /dev/ptp0 -m -l 7 -t 1000
But I am getting a huge delay most of the time.
So, I want to set the NXP LS1021ATSN device as a master clock and my two BeagleBone Black as slaves. The setup of my experiment is as follows.
"I have two BeagleBone Black (BBB) as two hosts connected to the ETH3 and ETH5 ports of the NXP LS1021ATSN device."
So, could you please guide me with how to set the NXP LS1021ATSN as the master clock.
Thanks in advance for your time.

Do the Beaglebone Blacks support hardware timestamping?
ethtool -T eth0

Also, ptp4l implements the best master clock selection algorithm. Which device was selected as master clock can be inferred from each device's ptp4l log.
To force the LS1021A-TSN board as master, you can use the following ptp4l option on the 2 Beaglebones (taken from the manual):
-s     Enable the slaveOnly mode.

Also I am not clear which command you are running on which board. You said ptp4l -i eth0 but if you are running this command on LS1021A-TSN, it should be ptp4l -i eth2 since that is the interface that goes through the switch and towards ETH3 and ETH5.


Move all components next js

Added all components and fixed layout css

@Lovekr Is this meant for us? If so please provide more context on the PR.


cortextool: init at version 0.1.4

Motivation for this change
Add cortextool, a CLI for interacting with Cortex instances.
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[ ] NixOS
[ ] macOS
[x] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[x] Determined the impact on package closure size (by running nix path-info -S before and after)
[x] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

aarch64 is failing. Please restrict this package to linux/darwin. doc :point_right: https://nixos.org/nixpkgs/manual/#chap-meta

Is it possible this is a generalized problem with the go packaging rather than a platform specific issue? Both aarch64-linux and x86_64-linux fail with a complaint about the +incompatible suffix introduced in go 1.13
<EMAIL_ADDRESS>invalid version: +incompatible suffix not allowed: module contains a go.mod file, so semantic import versioning is required

I was able to build this package by apply this PR to nixpkgs back when it used go 1.12 (not 1.13 or 1.14). So it looks like the go.mod in the upstream sources is incompatible with more recent versions of go.

The upstream project uses -mod=vendor in conjunction with go build, not go mod. See https://github.com/grafana/cortextool/blob/master/Makefile
It's a bit beyond me at the moment to figure out how (or even if) to shim this into the buildGoModule machinery.

nixpkgs/pkgs/development/go-modules/generic/default.nix:3:1 called without required argument 'vendorSha256'

Please fix the eval error.

Closing due to inactivity from author.


Shouldn't a routing constraint constructor use DI ?

I have a routing constraint that limites entity names to db tables (mapped in EF-Core).
To do this, I have this:
[Microsoft.AspNetCore.Mvc.HttpGet("API/{entity:BlueMineTable}/Remove/{id:int}")]
public JsonpResult RemoveEntity(string entity, int? id)
{ ... }

 services.AddMvc();

            services.Configure<Microsoft.AspNetCore.Routing.RouteOptions>(options =>

                options.ConstraintMap.Add("BlueMineTable"
                    //, typeof(RouteConstraints.EntitiesRouteConstraint<BlueMine.Db.BlueMineContext>)
                    ,typeof(RouteConstraints.EntitiesRouteConstraint
                        <BlueMine.Db.BlueMineContext,
                            BlueMine.Db.GenericEntityFramworkRepository<BlueMine.Db.BlueMineContext>,
                            BlueMine.Db.BlueMineRepository>)
                    )
            );

Now, the problem is, I can't get the EntityContext via DI in the constructor of the constraint.
Therefore, I need to get it via httpContext.RequestServices, which I find a bit odd, as this doesn't need to be done on a per-request basis.
So I have to always check on every Match if the hashtable, has been initialized, and if not, initialize it.
This frankly is suboptimal, IMHO.
For reference:

namespace BlueMine.RouteConstraints
{

    public class EntitiesRouteConstraint<TDbContext, TGenericRepository, TRepository>
        : Microsoft.AspNetCore.Routing.IRouteConstraint
        // we need TDbContext to define TGenericRepository
        where TDbContext: Microsoft.EntityFrameworkCore.DbContext
        // we need TGenericRepository to call our function
        where TGenericRepository: BlueMine.Db.GenericEntityFramworkRepository<TDbContext>
        where TRepository: TGenericRepository // we need TRepository for DI resolution
        // TODO: Define interface for repository so EF can be dropped as dependency
    {

        private static object s_lock;
        private static System.Collections.Generic.HashSet<string> s_tables;

        static EntitiesRouteConstraint()
        {
            s_lock = new object();
        }

        public EntitiesRouteConstraint()
        { }

        // https://stackoverflow.com/questions/45667126/how-to-get-table-name-of-mapped-entity-in-entity-framework-core
        public void InitializeHashSet(System.IServiceProvider services)
        {
            lock (s_lock)
            {
                if (s_tables == null)
                {
                    TGenericRepository bmc = (TGenericRepository)
                        services.GetService(typeof(TRepository));

                    if (bmc == null)
                    {
                        // throw new System.NullReferenceException(nameof(bmc));
                        return;
                    } // End if (bmc == null)

                    System.Collections.Generic.HashSet<string> hs =
                        new System.Collections.Generic.HashSet<string>(
                        System.StringComparer.InvariantCultureIgnoreCase
                    );

                    foreach (string table in bmc.ListTableNames())
                    {
                        hs.Add(table);
                    }

                    s_tables = hs;
                } // End if (s_tables == null)

            } // End lock (s_lock)

        } // End Sub InitializeHashSet

        bool Microsoft.AspNetCore.Routing.IRouteConstraint.Match(
              Microsoft.AspNetCore.Http.HttpContext httpContext
            , Microsoft.AspNetCore.Routing.IRouter route
            , string routeKey
            , Microsoft.AspNetCore.Routing.RouteValueDictionary values
            , Microsoft.AspNetCore.Routing.RouteDirection routeDirection)
        {
            if (routeDirection != Microsoft.AspNetCore.Routing.RouteDirection.IncomingRequest)
                return false;

            if (s_tables == null)
            {
                InitializeHashSet(httpContext.RequestServices);
            } // End if (s_tables == null)

            // object value = values[routeKey];

            object value;
            if (values.TryGetValue(routeKey, out value) && value != null)
            {
                return s_tables.Contains(value.ToString());
            }

            return false;
        } // End Function IRouteConstraint.Match

    } // End Function EntitiesRouteConstraint<T>

}

@pranavkm can you please look into this, thanks!

@rynowak is more familiar with this area, in particular if adding this feature would affect the dispatcher changes.

DbContext is scoped, route constraints have application lifetime.
I would suggest resolving the DbContext per-request. When you constraint gets invoked you are provided the current HttpContext, which should allow you to resolve per-request services like the DbContext.
You can find an example of a contraint that resolves services here:
https://github.com/aspnet/Mvc/blob/dev/src/Microsoft.AspNetCore.Mvc.Core/Routing/KnownRouteValueConstraint.cs

Per request ?
Why I would want to resolve the db-context on a per-request basis ?
What for ? Multi-tenant application with single-tenant database based on the host-header ?
Makes no sense IMHO, and introduces way to much latency, plus needlessly puts mindless load on the database.
When I fetch the tables in the application, I do this only once, and then it's read-only during application-lifetime.  Why would I ever want a different db-context depending on the request ?
(well, maybe in some obscure edge-cases - in which case I could still do it via httprequest.geet)
I certainly wouldn't want to create a new hash-table for every request - again latency.
The case of the action descriptor is fundamentally different - as it isn't constant per request.
But such logic makes little sense on the DB level.
The ifs and the static could be avoided if it had the db-context in the constructor, in my case.
Or just pass an already-constructed singleton to ConstraintMap.

Factory would do the job, too.
But that would make the first request slow.
Is that really necessary ?
Can't it just read/cache all it needs on application startup, and then use that information already at the first request ? So the first request will be fast, too ?
Say, I update an application at 2 o'clock in the morning, when nobody is using it.
At 5 o'clock the first users will start using it.
Do the requests at 5 really need to be slow ?
It had 3 hours to start up...  plenty of time for anything - ANYTHING.
Do I really want to write a script that issues HTTP requests to warm-up ?
Do I really never forget to call that script after updating ?
Do I want to have to call a script every time after updating ?
Do I want to have to remember to call a script every time after updating ?
Will I have to call N startup-scripts for N routes ?
Sounds like baaaaaad to me.

Then it's fine ;)

I'm going to close this as done.
2.2 Preview 2 will be out next month and will support injected services on constraints. It would be great if you could try the preview and test whether it does what you want. If it doesn't we can reopen this issue.

Just tested.
Works as expected - all good in my 10 minutes test, as far as I am concerned.
How does it determine which constructor to use ?
e.g.
// services.AddSingleton<RouteConstraints.GenericRepository>(new RouteConstraints.GenericRepository());

using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;

namespace TestNetCore22.RouteConstraints
{

    public class GenericRepository
    {

        public System.Collections.Generic.List<string> ListTableNames()
        {
            System.Collections.Generic.List<string> ls =
                new System.Collections.Generic.List<string>();

            ls.Add("abc");
            ls.Add("def");

            return ls;
        }

    }

    public class EntitiesRouteConstraint
        : Microsoft.AspNetCore.Routing.IRouteConstraint
    {

        protected System.Collections.Generic.HashSet<string> m_tables;

        public EntitiesRouteConstraint()
        {
            System.Console.WriteLine("hello world");
        }

        public EntitiesRouteConstraint(GenericRepository repo)
        {
            System.Collections.Generic.HashSet<string> hs =
                       new System.Collections.Generic.HashSet<string>(
                       System.StringComparer.InvariantCultureIgnoreCase
                   );

            foreach (string table in repo.ListTableNames())
            {
                hs.Add(table);
            }

            this.m_tables = hs;
        }

        public EntitiesRouteConstraint(GenericRepository repo, string test)
        {
            System.Console.WriteLine(test);
        }

        bool Microsoft.AspNetCore.Routing.IRouteConstraint.Match(
              Microsoft.AspNetCore.Http.HttpContext httpContext
            , Microsoft.AspNetCore.Routing.IRouter route
            , string routeKey
            , Microsoft.AspNetCore.Routing.RouteValueDictionary values
            , Microsoft.AspNetCore.Routing.RouteDirection routeDirection)
        {
            if (routeDirection != Microsoft.AspNetCore.Routing.RouteDirection.IncomingRequest)
                return false;

            // object value = values[routeKey];
            object value;
            if (values.TryGetValue(routeKey, out value) && value != null)
            {
                return this.m_tables.Contains(value.ToString());
            }

            return false;
        } // End Function IRouteConstraint.Match

    } // End Function EntitiesRouteConstraint<T>

}

https://github.com/aspnet/Routing/blob/release/2.2/src/Microsoft.AspNetCore.Routing/Internal/ParameterPolicyActivator.cs#L103-L139
Matches on inline args and then most number of arguments to get from DI.

Note that constraints have an UNBOUNDED lifetime. So if you try to access things like transient disposable services or a scoped service, expect some explosions.

Nice warning/catch - but that much should be obvious.
But then again, who knows what all those services really do.

Lots of folks expect that everything works everywhere 😁 I do my best now to make it clear what's supported, because if it accidentally works, then someone will use it.


Developer settings part confusing

the placement of the paragraph "This setting applies to streaming and PUSH datasets. If disabled, users will not be allowed send data to streaming and PUSH datasets using the API with a resource key." makes it confusing to the readers. Does this statement only apply to the tenant settings: Block ResourceKey Authentication (disabled by default)*
or does this statement apply to all the developer tenant settings?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 85be11e7-5a03-a1b1-605a-d49b4215a8aa
Version Independent ID: 0877169a-673f-1fe9-45b4-dec6b9e9fe26
Content: Tenant settings guidance - Power BI
Content Source: powerbi-docs/guidance/admin-tenant-settings.md
Service: powerbi
Sub-service: powerbi-resource
GitHub Login: @paulinbar
Microsoft Alias: painbar

Thanks for your comment, @jopebeni. This issue has been assigned to the doc author for review.

Thanks for your question, @jopebeni.
That statement only applies to Block ResourceKey Authentication.
I updated the page and clarified the issue.


simplify stdlib to make MintedValue a quick convert to value.

Also added testing methods to make use of MintedValue in unit tests.
This approach mainly focuses on converting MintedValue to Value. Thus forcing the user to deal with the knowledge of making a small quick conversion. Less type confusion as well???

I like the idea. The majority of testing I have done is either a value of zero or some token. My question about this implementation would be is it worth it to just have a public mint_value.to_value and mint_value.from_value instead?  Then we can do things like from_value(zero()) or when values with multiple token mints or burns like from_value(value_with_ref_and_nft).

Hm where did all the mint specific functions go? Aesthetically I'm not exactly pleased with prefixing or suffixing mint to distinguish between the methods. Can't we just add a from_assets method to minted_value?

Note: we will most likely want an equivalent from_value function so we can still construct MintedValue from a sanitized Value (for example, to construct transactions for testing).


[Bug] Ref keys spawn in docs & SICC cases

Blacklist these by default.

Didn't manage to implement a blacklist for this in time as moving the mod to DI & startup async took a bit longer than expected, intend to do it for one of the updates in 3.10 instead.

Feature added in dev


Add route scope annotation

Like described on Stackoverflow the route scope is missing in this example.
https://stackoverflow.com/questions/58091745/redirect-to-controller-throws-unfortunately-something-went-wrong/58092066#58092066

thanks @mnaczenski :+1: :tada: :blue_heart:


Add translation for ReplicaSet in ID localization

This address #15341

/retest

Should be okay for now. Looking forward to your review! :slightly_smiling_face:
@irvifa @girikuncoro

Nice work.. thanks..
/lgtm
/approve

Nice work.. thanks..
/lgtm
/approve


DuckDB Reciever

This feature implements support for DuckDB as a supported sink for pgwatch3 to run OLAP wokloads on the collected metrics

To be continued, marking this as low priority


How 2 fix this problem??????

Describe your problem here.
When i Reopen the VSC,it show me problems:TYPE not found : 'everythings'   ,but i can still build my game.how 2 fix it.

Are you modding a build from source or with Lua?
Source
What is your build target?
Windows x64
Did you edit anything in this build? If so, mention or summarize your changes.
No

its stupid, it always shows it

click that bell

Wait I might know why since it used to happen to me too Did you install any extensions for haxe in the visual studio code extensions manager?

i just install extension "haxe"

Wait I might know why since it used to happen to me too Did you install any extensions for haxe in the visual studio code extensions manager?

i just install extension "haxe"

You sure that is nothing else?
Try to delete all extensions and install just the Haxe one
Just that not even other Haxe's extensions

等等，我可能知道为什么，因为它曾经也发生在我身上 您是否在Visual Studio代码扩展管理器中安装了Haxe的任何扩展？

我只是安装扩展“Haxe”

你确定没有别的？尝试删除所有扩展并仅安装Haxe扩展 只是甚至没有其他Haxe的扩展

不，它不起作用

那么它只是 vsc 错误 你是如何安装库的 使用 Hmm 方法？

use cmd 'haxelib install'

Contributor

Wait I might know why since it used to happen to me too Did you install any extensions for haxe in the visual studio code extensions manager?

i just install extension "haxe"

You sure that is nothing else? Try to delete all extensions and install just the Haxe one Just that not even other Haxe's extensions

Nope,it did't work

Then it's just vsc fault How did you install the libraries Using the Hmm method?

Use CMD

Contributor

Wait I might know why since it used to happen to me too Did you install any extensions for haxe in the visual studio code extensions manager?

i just install extension "haxe"

You sure that is nothing else? Try to delete all extensions and install just the Haxe one Just that not even other Haxe's extensions

Nope,it did't work

Then it's just vsc fault How did you install the libraries Using the Hmm method?

Use CMD

Yeah but did you do stiff like running hmm in the cmd?

Contributor

Wait I might know why since it used to happen to me too Did you install any extensions for haxe in the visual studio code extensions manager?

i just install extension "haxe"

You sure that is nothing else? Try to delete all extensions and install just the Haxe one Just that not even other Haxe's extensions

Nope,it did't work

Then it's just vsc fault How did you install the libraries Using the Hmm method?

Use CMD

Yeah but did you do stuff like running hmm in the cmd?

no

no

Then this is fr odd
But dw since you can build the game, it's not a big deal


how to correct list of many long reads files for Proovread?

Hello:
After I got ~100 .fq long reads files from SeqChunker step, I wonder how to add this list of files into the program? for example, we can use -s 1.fq -s 2.fq -s 3.fq  for many high quality short reads, how to deal with ~100 long reads .fq files.
In the proovread.cfg file, it says (line37-38):

LIST of Pacbio read files to correct. FASTA or FASTQ format.

'long-reads' => [],
However, I cannot figure out how to use this option.
I tried:

'long-reads' => [pb-001.fq],[pb-002.fq]

'long-reads' => [pb-001.fq],
'long-reads' => [pb-002.fq],

'long-reads' => [*.fq],

ALL of them are FAILED. Any suggestions? Thanks

The correct syntax would be:
'long-reads' => ["pb-001.fq", "pb-002.fq"],

However, if you want to run all of your reads in one go, there is no need to split them in the first place. You can just run proovread on the unsplit file directly. The advantage of splitting is, that you can submit a proovread job for every chunk independently to a computing cluster. This allows you, in particular, to adjust runtimes to given queue limits, etc, or to do test runs on a subset of the data. If your data set is small, i.e. not a large eukaryotic genome, and you have a computer with +10 cores and +50 GB RAM, then you should be fine running all at once.

Hi @thackl , Thanks so much for your response. It saves me a lot of time to testing different syntax.
My PacBio data is ~12G in .fq, the genome is around 120 Mb, I'll run Proovread in our computer cluster in which I am allowed to use 12 threads, 64 CPU and 504 G of memory.
So what's suggestion, e.g.: how many chunk? or I can run all my 12G data at once?
Again, your suggestion will save me a lot of testing time, so I am looking forward to getting your suggestions. Thanks so much.

That is a rather small genome. 12G sounds like you have multiple PacBio
cells, I would run one instance of proovread per pacbio cell (or some
500MB to 1G chunks). Max out the CPUs. Submit the job with 64 CPUs and
set proovread --thread to 64. Make sure to set --coverage to the
coverage of your Illumina data.
On 19.08.2016 11:26, wsuplantpathology wrote:

Hi @thackl https://github.com/thackl , Thanks so much for your
response. It saves me a lot of time to testing different syntax.
My PacBio data is ~12G in .fq, the genome is around 120 Mb, I'll run
Proovread in our computer cluster in which I am allowed to use 12
threads, 64 CPU and 504 G of memory.
So what's suggestion, e.g.: how many chunk? or I can run all my 12G data
at once?
Again, your suggestion will save me a lot of testing time, so I am
looking forward to getting your suggestions. Thanks so much.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/BioInf-Wuerzburg/proovread/issues/74#issuecomment-241049613,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AI7ej8ghnLKlNG7-pZRhP-xvkNlTzHccks5qhcswgaJpZM4JoHj1.

Thanks so much. I'll try different chunks and test your suggestion. Thanks for such a nice program.


Update navigation bar

This PR adds the headings of the Table of Content. The final layout will be updated soon.

Closing in favour of #7