Cannot compile Intel-mllib of master branch.

We try to compile Intel-mllib of master branch and meet the error like:

The maven command is "mvn clean package  -DskipTests  -Pspark-3.1.1" and we've successfully installed oneapi and oneccl.

@xwu99 @zhixingheyi-tian Please help to track the issue. Thanks!

@xwu99 Do you have any instructions about this issue?

Is it related to #77? Are you able to to build before?
Are you able to build for 3.0.x profiles?

@haojinIntel
Would you paste the specific UT

I've paste the whole message below:
[root@bdpe-sky2 mllib-dal]#     mvn clean package    -Pspark-3.1.1
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building OAP Project OAP MLlib 1.1.0
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ oap-mllib ---
[INFO] Deleting /home/oap/Beaver/package/source_code/oap-tools/oap-mllib/mllib-dal/target
[INFO]
[INFO] --- build-helper-maven-plugin:3.2.0:add-source (add-source) @ oap-mllib ---
[INFO] Source directory: /home/oap/Beaver/package/source_code/oap-tools/oap-mllib/mllib-dal/src/spark-3.1.1/main/java added.
[INFO] Source directory: /home/oap/Beaver/package/source_code/oap-tools/oap-mllib/mllib-dal/src/spark-3.1.1/main/scala added.
[INFO]
[INFO] --- build-helper-maven-plugin:3.2.0:add-test-source (add-test-source) @ oap-mllib ---
[INFO] Test Source directory: /home/oap/Beaver/package/source_code/oap-tools/oap-mllib/mllib-dal/src/spark-3.1.1/test/scala added.
[INFO]
[INFO] --- maven-resources-plugin:3.0.2:resources (default-resources) @ oap-mllib ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 3 resources
[INFO] Copying 1 resource
[INFO] Copying 2 resources
[INFO] Copying 0 resource
[INFO]
[INFO] --- scala-maven-plugin:4.4.0:add-source (scala-compile-first) @ oap-mllib ---
[INFO] Add Source directory: /home/oap/Beaver/package/source_code/oap-tools/oap-mllib/mllib-dal/src/main/scala
[INFO] Add Test Source directory: /home/oap/Beaver/package/source_code/oap-tools/oap-mllib/mllib-dal/src/test/scala
[INFO]
[INFO] --- scala-maven-plugin:4.4.0:compile (scala-compile-first) @ oap-mllib ---
[INFO] Using incremental compilation using Mixed compile order
[INFO] Compiler bridge file: /root/.sbt/1.0/zinc/org.scala-sbt/org.scala-sbt-compiler-bridge_2.12-1.3.5-bin_2.12.10__52.0-1.3.5_20200330T000931.jar
[INFO] Compiling 14 Scala sources and 9 Java sources to /home/oap/Beaver/package/source_code/oap-tools/oap-mllib/mllib-dal/target/classes ...
[INFO] Done compiling.
[INFO] compile in 12.4 s
[INFO]
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ oap-mllib ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 9 source files to /home/oap/Beaver/package/source_code/oap-tools/oap-mllib/mllib-dal/target/classes
[INFO]
[INFO] --- maven-antrun-plugin:1.8:run (default) @ oap-mllib ---
[INFO] Executing tasks
build-natives:
[echo] Building native code
[exec] Cleaning up
[exec] rm -rf ./OneCCL.o ./OneDAL.o ./service.o ./error_handling.o ./KMeansDALImpl.o ./PCADALImpl.o ./ALSDALImpl.o ./ALSShuffle.o ./NaiveBayesDALImpl.o ./LinearRegressionDALImpl.o ../../../target/libMLlibDAL.so
[exec]
[exec] Building file: OneCCL.cpp
[exec] clang++ -g -Wall -Wno-deprecated-declarations -fPIC -std=c++11 -I /opt/Beaver/jdk/include -I /opt/Beaver/jdk/include/linux -I /tmp/oneCCL/build/_install/include -I /opt/intel/oneapi/dal/2021.2.0/include -I ./javah -I ./ -c -o "OneCCL.o" "OneCCL.cpp"
[exec] Building file: OneDAL.cpp
[exec] Building file: service.cpp
[exec] clang++ -g -Wall -Wno-deprecated-declarations -fPIC -std=c++11 -I /opt/Beaver/jdk/include -I /opt/Beaver/jdk/include/linux -I /tmp/oneCCL/build/_install/include -I /opt/intel/oneapi/dal/2021.2.0/include -I ./javah -I ./ -c -o "OneDAL.o" "OneDAL.cpp"
[exec] clang++ -g -Wall -Wno-deprecated-declarations -fPIC -std=c++11 -I /opt/Beaver/jdk/include -I /opt/Beaver/jdk/include/linux -I /tmp/oneCCL/build/_install/include -I /opt/intel/oneapi/dal/2021.2.0/include -I ./javah -I ./ -c -o "service.o" "service.cpp"
[exec] Building file: error_handling.cpp
[exec] clang++ -g -Wall -Wno-deprecated-declarations -fPIC -std=c++11 -I /opt/Beaver/jdk/include -I /opt/Beaver/jdk/include/linux -I /tmp/oneCCL/build/_install/include -I /opt/intel/oneapi/dal/2021.2.0/include -I ./javah -I ./ -c -o "error_handling.o" "error_handling.cpp"
[exec] Building file: KMeansDALImpl.cpp
[exec] Building file: PCADALImpl.cpp
[exec] clang++ -g -Wall -Wno-deprecated-declarations -fPIC -std=c++11 -I /opt/Beaver/jdk/include -I /opt/Beaver/jdk/include/linux -I /tmp/oneCCL/build/_install/include -I /opt/intel/oneapi/dal/2021.2.0/include -I ./javah -I ./ -c -o "KMeansDALImpl.o" "KMeansDALImpl.cpp"
[exec] clang++ -g -Wall -Wno-deprecated-declarations -fPIC -std=c++11 -I /opt/Beaver/jdk/include -I /opt/Beaver/jdk/include/linux -I /tmp/oneCCL/build/_install/include -I /opt/intel/oneapi/dal/2021.2.0/include -I ./javah -I ./ -c -o "PCADALImpl.o" "PCADALImpl.cpp"
[exec] Building file: ALSDALImpl.cpp
[exec] clang++ -g -Wall -Wno-deprecated-declarations -fPIC -std=c++11 -I /opt/Beaver/jdk/include -I /opt/Beaver/jdk/include/linux -I /tmp/oneCCL/build/_install/include -I /opt/intel/oneapi/dal/2021.2.0/include -I ./javah -I ./ -c -o "ALSDALImpl.o" "ALSDALImpl.cpp"
[exec] Building file: ALSShuffle.cpp
[exec] clang++ -g -Wall -Wno-deprecated-declarations -fPIC -std=c++11 -I /opt/Beaver/jdk/include -I /opt/Beaver/jdk/include/linux -I /tmp/oneCCL/build/_install/include -I /opt/intel/oneapi/dal/2021.2.0/include -I ./javah -I ./ -c -o "ALSShuffle.o" "ALSShuffle.cpp"
[exec] Building file: NaiveBayesDALImpl.cpp
[exec] clang++ -g -Wall -Wno-deprecated-declarations -fPIC -std=c++11 -I /opt/Beaver/jdk/include -I /opt/Beaver/jdk/include/linux -I /tmp/oneCCL/build/_install/include -I /opt/intel/oneapi/dal/2021.2.0/include -I ./javah -I ./ -c -o "NaiveBayesDALImpl.o" "NaiveBayesDALImpl.cpp"
[exec] Building file: LinearRegressionDALImpl.cpp
[exec] clang++ -g -Wall -Wno-deprecated-declarations -fPIC -std=c++11 -I /opt/Beaver/jdk/include -I /opt/Beaver/jdk/include/linux -I /tmp/oneCCL/build/_install/include -I /opt/intel/oneapi/dal/2021.2.0/include -I ./javah -I ./ -c -o "LinearRegressionDALImpl.o" "LinearRegressionDALImpl.cpp"
[exec] Finished building: error_handling.cpp
[exec]
[exec] Finished building: ALSShuffle.cpp
[exec]
[exec] Finished building: OneCCL.cpp
[exec]
[exec] Finished building: OneDAL.cpp
[exec]
[exec] Finished building: service.cpp
[exec]
[exec] Finished building: PCADALImpl.cpp
[exec]
[exec] Finished building: NaiveBayesDALImpl.cpp
[exec]
[exec] Finished building: KMeansDALImpl.cpp
[exec]
[exec] Finished building: LinearRegressionDALImpl.cpp
[exec]
[exec] Finished building: ALSDALImpl.cpp
[exec]
[exec] Building target: ../../../target/libMLlibDAL.so
[exec] Invoking: Linker
[exec] clang++ -g -Wall -Wno-deprecated-declarations -fPIC -std=c++11 -shared -o ../../../target/libMLlibDAL.so ./OneCCL.o ./OneDAL.o ./service.o ./error_handling.o ./KMeansDALImpl.o ./PCADALImpl.o ./ALSDALImpl.o ./ALSShuffle.o ./NaiveBayesDALImpl.o ./LinearRegressionDALImpl.o -L/tmp/oneCCL/build/_install/lib -lccl -L/opt/intel/oneapi/dal/2021.2.0/lib/intel64 -l:libdaal_core.a -l:libdaal_thread.a -L/opt/intel/oneapi/tbb/2021.2.0/env/../lib/intel64/gcc4.8 -ltbb -ltbbmalloc
[exec] Finished building target: ../../../target/libMLlibDAL.so
[exec]
[INFO] Executed tasks
[INFO]
[INFO] --- maven-resources-plugin:3.0.2:testResources (default-testResources) @ oap-mllib ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 3 resources
[INFO] Copying 1 resource
[INFO] Copying 2 resources
[INFO] Copying 1 resource
[INFO]
[INFO] --- scala-maven-plugin:4.4.0:testCompile (scala-test-compile) @ oap-mllib ---
[INFO] Using incremental compilation using Mixed compile order
[INFO] Compiler bridge file: /root/.sbt/1.0/zinc/org.scala-sbt/org.scala-sbt-compiler-bridge_2.12-1.3.5-bin_2.12.10__52.0-1.3.5_20200330T000931.jar
[INFO] Compiling 5 Scala sources to /home/oap/Beaver/package/source_code/oap-tools/oap-mllib/mllib-dal/target/test-classes ...
[ERROR] /home/oap/Beaver/package/source_code/oap-tools/oap-mllib/mllib-dal/src/test/scala/org/apache/spark/ml/recommendation/MLlibALSSuite.scala:1005: value cleanShuffleDependencies is not a member of object org.apache.spark.ml.recommendation.ALS
[ERROR] one error found
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 30.261 s
[INFO] Finished at: 2021-07-02T14:38:25+08:00
[INFO] Final Memory: 165M/3951M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal net.alchim31.maven:scala-maven-plugin:4.4.0:testCompile (scala-test-compile) on project oap-mllib: Execution scala-test-compile of goal net.alchim31.maven:scala-maven-plugin:4.4.0:testCompile failed. CompileFailed -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginExecutionException

It shoud be fixed with #88


MSID: 97615 Version: 2 DOI: 10.1101/2<IP_ADDRESS>3574

MSID: 97615
Version: 2
Preprint DOI: https://doi.org/10.1101/2<IP_ADDRESS>3574
Step 1. Awaiting reviews
Editorial to post reviews via hypothesis
Useful links:

DocMap: https://data-hub-api.elifesciences.org/enhanced-preprints/docmaps/v2/by-publisher/elife/get-by-manuscript-id?manuscript_id=97615
New model tracking: https://docs.google.com/spreadsheets/d/1_fHaoOy7hjyocptKtVJRijeNpUY4hBS7Ck_aVmx6ZJk/
Reviews on sciety: https://sciety.org/articles/activity/10.1101/2<IP_ADDRESS>3574

For trouble shooting (e.g. no Docmaps available):

DocMap issue addressing: https://miro.com/app/board/uXjVNCwK6EI=/
Explore DataHub DocMaps API: https://lookerstudio.google.com/reporting/4c2f0368-babb-4beb-b5b3-497e7e7b0f08/page/ejphD
Unmatched submissions and preprints: https://lookerstudio.google.com/u/0/reporting/9f86204f-3bf7-477c-9b18-5c5ef141bf69/page/p_gxi57ha93c
Unmatched manuscripts spreadsheet: https://docs.google.com/spreadsheets/d/15QcK8w-ssB7109RQEDtFpJPZ0J5HTGxoHa_2TtpMBbg/edit#gid=1336081641

Step 2. Preview reviewed preprint
Production QC content ahead of publication
Instructions:

QC preview: https://prod--epp.elifesciences.org/previews/97615v2
Update ticket with any problems (add blocked label)
When QC OK, add QC OK label to ticket and add publication date and time to https://docs.google.com/spreadsheets/d/1amAlKvdLcaDp5W8Z8g77NmkwbMF5n_u89ArSqPMO8jg
Move card to next column
(At end of the day post link in #enhanced-preprint and ask for PDF to be generated)

Useful links:

Preprint DOI :  https://doi.org/10.1101/2<IP_ADDRESS>3574
Confirm reviews returned by EPP: https://prod--epp.elifesciences.org/api/reviewed-preprints/97615/v2/reviews
To update the MECA path in the docmap: https://docs.google.com/spreadsheets/d/1mctCQuNFBjSn97Lihy7_vBO6z7-N-oqyLv4clyi6zHg

Step 3: Awaiting search reindex
This step adds the reviewed preprint to the homepage: https://elifesciences.org
The search reindex is triggered once an hour. We need the reviewed preprint to be indexed as the search application serves the journal homepage.
Useful links:

Jenkins pipeline to reindex search can be triggered sooner or monitored here: https://alfred.elifesciences.org/job/process/job/process-reindex-reviewed-preprints/

Step 4: Published! PDF requested
Waiting for PDF to be generated
Useful links:

PDF tracking: https://docs.google.com/spreadsheets/d/106_XeDjmuBae7gexOTNzg60lapeqjl2aRn9DzupGyS8/

Step 5: Introduce PDF to data folder and git repo
Upload PDF to relevent folder in git repo https://github.com/elifesciences/enhanced-preprints-data/
Step 6: Done!

[ ] Kettle is on!

temporal still running

I wanted to see if the elife-bot messages could be in colour, here's one quick example, just for fun and a test - @fred-atherden and others:
- error: [journal-ref-article-title] This journal reference (id c25) has no article-title element.
- error: [journal-ref-article-title] This journal reference (id c28) has no article-title element.
! warning: [journal-ref-text-content] This journal reference (id c11) has untagged textual content - ), jeb188532.. Is it tagged correctly?
! warning: [journal-ref-text-content] This journal reference (id c13) has untagged textual content - ), jeb193201.. Is it tagged correctly?
! warning: [journal-ref-text-content] This journal reference (id c50) has untagged textual content - , M. (. Is it tagged correctly?
+ info: [article-version-flag] This is preprint version 1.3.

@gnott, very cool! 😎

@rkcook312, here's the error message from temporal (no colours like above I'm afraid 😞 ):
"payloads": [
        {
          "details": [
            {
              "message": "\"value\" does not match any of the allowed types",
              "path": [],
              "type": "alternatives.match",
              "context": {
                "message": "\"article.references[3].identifiers[0].name\" is required. \"url\" is required",
                "details": [
                  {
                    "message": "\"article.references[3].identifiers[0].name\" is required",
                    "path": [
                      "article",
                      "references",
                      3,
                      "identifiers",
                      0,
                      "name"
                    ],
                    "type": "any.required",
                    "context": {
                      "label": "article.references[3].identifiers[0].name",
                      "key": "name"
                    }
                  },
                  {
                    "message": "\"url\" is required",
                    "path": [
                      "url"
                    ],
                    "type": "any.required",
                    "context": {
                      "label": "url",
                      "key": "url"
                    }
                  }
                ]
              }
            }
          ]
        }
      ]

Doesn't look like there's an issue with ref c4 in the XMl as far as I can tell though 🤔

Ah it's complaining about the v1, not the v2!
In the v1 we have:
<ref id="c4"><mixed-citation publication-type="journal"><string-name><surname>Dacke</surname>, <given-names>M.</given-names></string-name>, <string-name><surname>Nilsson</surname>, <given-names>DE.</given-names></string-name>, <string-name><surname>Warrant</surname>, <given-names>E.</given-names></string-name> <etal>et al.</etal> <article-title>Built-in polarizers form part of a compass organ in spiders</article-title>. <source>Nature</source>, <volume>401</volume>, <fpage>470</fpage>–<lpage>473</lpage> (<year>1999</year>). <pub-id>10.1038/46773</pub-id></mixed-citation></ref>

(The pub-id element is missing the attribute pub-id-type="doi").

I've updated the v1 and re-run the temporal workflow. I think it should be good to check now @rkcook312


support Docusaurus TypeScript

see https://docusaurus.io/docs/typescript-support

Pls, assign this to me. I will start the work at the right time.


WSL2 is better behaved that old "bash on windows" WSL

Working through this with WSL2 and various terminal and keyboard related behaviors just work right now. One specific instance, chapter 5 in the save as section:
"(Note: If you’re using Bash on Windows, you will have to press Escape 3 times to get one Escape keypress to register in our program, because the read() calls in editorReadKey() that look for an escape sequence never time out.)"
Works fine with a single keypress of ESC on my configuration. Suggest changing wording about Bash on Windows to "Bash on Windows prior to WSL2" or similar. Great job on the whole thing though. Appreciated.

Much to my chagrin - WSL2 is better behaved than my actual linux distro.
I was happily plucking away at the chapters on a windows machine, then at step 44 I switched to a linux machine, pulled down my repo, and... suddenly ctrl-q doesn't work! I have go go all the way back to step 32 to get a version of the program that will actually exit without me terminating bash


Supporting 0 to 1 range colors

When passing colors to shaders, traditionally color channel values are typically given in the range 0 to 1 rather than 0 to 255
Is it possible to hint to tweakpane we want 0 to 1 color values rather than 0 to 255?

Would be really great !

For example, with three.js you might want to do
// material.color is a THREE.Color object, which is a simple object with 3 properties, r, g, b defined between 0 and 1
pane.addInput(material, 'color');

https://threejs.org/docs/#api/en/math/Color.r
A solution might look like
pane.addInput(material, 'color', { colorType: 'float' });

Where the default 0 to 255 would have colorType byte, which is where the 255 / 0xFF comes from – generally as display technologies improve we're slowly moving beyond 32 bit colors

Thank you for the details. Added float color support in #411:
const params = {foobar: {r: 0, g: 0, b: 0}};

// [0-255]
pane.addInput(params, 'foobar');

// [0-255]
pane.addInput(params, 'foobar', {
  color: {type: 'int'},
});

// [0-1]
pane.addInput(params, 'foobar', {
  color: {type: 'float'},
});

Please wait for the next release.

Released the latest version 3.1.0.


Extending Documentation for Durable Functions with sections for Visual Studio Code and Azure Portal

Test Commit for creating editing space.

@gled4er : Thanks for your contribution to the Azure documentation! The author, @cgillum, has been notified to review your proposed change.


WireMock for C/C++ with a Testcontainers module

Proposal
I have recently created a new Testcontainers for C/C++ to support my talk on using WireMock for embedded projects. It would be niceto keep expanding this module, because there are many C and C++ projects that would benefit from API mocking embedded into the tests. Contributions are welcome!
If there are any existing implementations, prototypes, blogs or etc, let us know!
N/A
Reference any relevant documentation, other materials or issues/pull requests that can be used for inspiration.

https://github.com/testcontainers/testcontainers-c

Hi Oleg, I will be to contribute, but i need little guidance to start, I am beginner in open source contribution. lease let me know how I can assist.
Thanks

Hi @Khamaria221306. You can find some good first issues in the Testcontainers for C/C++ repo

Thanks Oleg


Logging indicates all images weren't downloaded

Describe the bug
When downloading a profile (_marina.mari), the log never gets to the total amount. i.e. after running today the last output says [277/280]. When looking at how many metadata files were downloaded, only 278 (1 extra for profile pic) were downloaded. No other error messages are logged.
To Reproduce
Steps to reproduce the behavior:
instaloader _marina.mari
Expected behavior
A clear and concise description of what you expected to happen (if not obvious).
Last output should either be [277/277] or [280/280] (I'm unsure how many posts are actually there or what is causing this offset)
Error messages and tracebacks
If applicable, add error messages and tracebacks to help explain your problem.
No error messages in logs.
Instaloader version
instaloader --version output.
4.3.2
Additional context
Add any other context about the problem here.
(please also see https://instaloader.github.io/contributing.html for how to report a bug)

This seems to be an error on Instagram's side. The web interface also shows only 277 posts while the count variable states 280.


[Issue-2518] Fix issue related to detect account type

PR for #2518

🚀 Deployed on https://pr-2519--sw-web-runner.netlify.app


Add pom.xml sample and README

Current coverage is 61.31% (diff: 100%)

Merging #115 into master will not change coverage

@@             master       #115   diff @@
==========================================
  Files            47         47
  Lines          2182       2182
  Methods           0          0
  Messages          0          0
  Branches        325        325
==========================================
  Hits           1338       1338
  Misses          765        765
  Partials         79         79

Powered by Codecov. Last update e93377f...cde0dcf

PTAL

Removed google from pom.xml

LGTM


Use bytearray in streaming writes

This is much faster for large buffer sizes

Thanks!

Actually this seems slightly slower and causes the tests to fail due to the typing.
old:
C:\p
(blobfile) λ python misc\blobfile_dev\write_speed.py
large write elapsed 5.145035266876221
small write elapsed 1.20097017288208

new:
(blobfile) λ python misc\blobfile_dev\write_speed.py
large write elapsed 5.4869225025177
small write elapsed 1.2590796947479248

Am I testing it wrong? Here is the program I used:
import time
from blobfile._common import BaseStreamingWriteFile

class TestFile(BaseStreamingWriteFile):
    def _upload_chunk(self, chunk: bytes, finalize: bool) -> None:
        pass

def main():
    N = 10
    b = b"x" * 2**30
    f = TestFile(conf=None, chunk_size=33)
    start = time.time()
    for i in range(N):
        f.write(b)
    end = time.time()
    print(f"large write elapsed {end - start}")

    M = 1_000_000
    f = TestFile(conf=None, chunk_size=33)
    b = b"x" * 2**10
    start = time.time()
    for i in range(M):
        f.write(b)
    end = time.time()
    print(f"small write elapsed {end - start}")

if __name__ == "__main__":
    main()

Your program looks right to me, although when I ran it it was kind of noisy. Anyway, it certainly doesn't look better / shouldn't be better since upload_buf isn't actually called in a loop (quite literally now, since a2cc9de). tldr; revert


E2E test to fetch signature from authenticated registry.

Description
Add a test to verify if the policy fetch is able to fetch the Sigstore data from an authenticated registry.
Fix https://github.com/kubewarden/policy-fetcher/issues/181
Test
make test

Mac e2e fail on testcontainers :/.

Github Mac runner image does not have Docker installed by default. :tired_face:

@kubewarden/kubewarden-developers I've tried to install docker in the MacOS to run the added test on it as well. But I've run into an issue on QEMU/Colima starting. Which was consuming some time. Thus, I've decided to disable the test on Mac and leave this issue for later.

@kubewarden/kubewarden-developers I'm aware of the windows failing tests. I'll take a look on it next week.


Python SDK 1.4.1 release

Added wheel to the project

Hi @rnagpal, I'm your friendly neighborhood Azure Pull Request Bot (You can call me AZPRBOT). Thanks for your contribution!

In order for us to evaluate and accept your PR, we ask that you sign a contribution license agreement. It's all electronic and will take just minutes. I promise there's no faxing. https://cla.azure.com.

TTYL, AZPRBOT;


Revert "Remove freetype from windows playbook (#766)"

This reverts commit 0628f935dccbc37a2ffdde135ec5222ce43b1828.
@jdekonin @karianna this PR reverts #766.
Is it ok if we remove freetype from main.yml but leave the role definition?

@jdekonin I've pushed another commit that removes  Freetype from roles in main.yml.

Discussed at the TSC - philosophy is that Adopt should prioritize its own build farm but should always make its infrastructure as code open to extension.  This seems like a good compromise, thanks, everyone.


Unclear that log file is circular

In the old experfwiz script, Circular was added to the name of the file. This is helpful to make sure that the script was run correctly with circular mode option selected

Filed name is now updated with _Circular if circular is enabled

Filed name is now updated with _Circular if circular is enabled


[evo] add scenario controllers, computes, ceph and no l3_agent

@nhicher what's the point of "no l3_agent" (if this is the only difference from another existing scenario) ?
I mean, I see the point in not using an l3 agent by not declaring an external public network, but isn't it sufficient to just not configure it ?

I will try both solution to validate it's functionnal with l3 not configured, I will create a PR later.
Thanks.
Nico


IE8 Modal windows createElementNS

Hi,
I've just discovered your plugin, it's really usefull thank you !
However I got a problem with IE8 when you launch a modal, it throws an error about createElementNS.
The problem is just when you call a modal, tooltips and notices are working well.

Any idea ?

0.4.0 is out, IE8 is not supported anymore


make msvc release build much faster

cl.exe for some reason spends unreasonable amount of time optimizing df_window_update_and_render function. We can use pragma to skip optimizations for this function.
This change improves build.bat release raddbg msvc time from ~50 seconds to ~7 seconds for me.
Alternatively we could split this function into few smaller ones to find out what triggers this slowness and leave only smaller amount of code under this pragma.

Improvement is visible also in Github actions.
Before this commit: 2m 3s - https://github.com/mmozeiko/raddebugger/actions/runs/7736121105/job/21092939284
With this commit: 1m 8s - https://github.com/mmozeiko/raddebugger/actions/runs/7736109789/job/21092905958


Flex Defaults

Column direction should have cross axis set to "stretch" by default

Made stretch the default for all directions


is aspectjrt-1.7.3.jar necessary?

Hello, is aspectjrt-1.7.3.jar necessary? Can you delete it?

I will find a way to see if we can do without


cached file cannot be found in the next hit

My workflow will cache llvm/lib and llvm/bin , and it succeeded at the first time:
https://github.com/hellodword/MyApplication/actions/runs/119157969
But the cached files cannot be found at the next time (but the cache key can be hitted):
https://github.com/hellodword/MyApplication/actions/runs/119660612
I tried a lot to find out the reason:

[x] I deleted the llvm/bin folder so the whole size of llvm is 290MB, it failed;
[x] With another workflow I cached a same folder structure, and use dd to create files so that the folder size can reach the same size (290MB), it succeeded;

Is it a bug or my usage problem? Any idea about this? Thanks.

:wave: Looking at the Cache multiple paths step in the second link, I see it says Cache Size: ~0 MBs (22 B).  It sounds like it's not finding the path to cache, creating an empty tar.  Looking at your workflow file, I suspect this is because you're creating that directory under the GITHUB_WORKSPACE (mkdir ${GITHUB_WORKSPACE}/ollvm.out), which is different from the path you are caching (ollvm.out).
Please note that after fixing the path, you will also want to change the key so you create a new cache instead of keep hitting the old cache.

👋 Looking at the Cache multiple paths step in the second link, I see it says Cache Size: ~0 MBs (22 B). It sounds like it's not finding the path to cache, creating an empty tar. Looking at your workflow file, I suspect this is because you're creating that directory under the GITHUB_WORKSPACE (mkdir ${GITHUB_WORKSPACE}/ollvm.out), which is different from the path you are caching (ollvm.out).

I thought ${GITHUB_WORKSPACE} is the base dir of the path in action/cache@v2.
I tried in this: testcache/actions

@hellodword I met issues on cache when I try github actions too, the glob pattern in the path is likely the reason of missing cache files.
try to add debug flag ACTIONS_STEP_DEBUG in your repository secret to debug your steps, you should see the logs at step Post Cache multiple paths

##[debug]Search path '/home/runner/work/testcache/testcache'
##[debug]Matched: XXX

more information about debug
https://github.com/actions/toolkit/blob/master/docs/action-debugging.md#step-debug-logs

@hellodword I met issues on cache when I try github actions too, the glob pattern in the path is likely the reason of missing cache files.
try to add debug flag ACTIONS_STEP_DEBUG in your repository secret to debug your steps, you should see the logs at step Post Cache multiple paths

##[debug]Search path '/home/runner/work/testcache/testcache'
##[debug]Matched: XXX

more information about debug
https://github.com/actions/toolkit/blob/master/docs/action-debugging.md#step-debug-logs

Thanks!
Added ACTIONS_STEP_DEBUG here: MyApplication/actions
And change the ollvm.out to ollvm in order to exclude some pattern error with the dot.
But it still failed.
I cannot find the reason of deferent results between MyApplication/actions and testcache/actions.

@hellodword I met issues on cache when I try github actions too, the glob pattern in the path is likely the reason of missing cache files.
try to add debug flag ACTIONS_STEP_DEBUG in your repository secret to debug your steps, you should see the logs at step Post Cache multiple paths

##[debug]Search path '/home/runner/work/testcache/testcache'
##[debug]Matched: XXX

more information about debug
https://github.com/actions/toolkit/blob/master/docs/action-debugging.md#step-debug-logs

Thanks!
Added ACTIONS_STEP_DEBUG here: MyApplication/actions
And change the ollvm.out to ollvm in order to exclude some pattern error with the dot.
But it still failed.
I cannot find the reason of deferent results between MyApplication/actions and testcache/actions.

I noticed that your cache path could not be resolved at step Post Cache multiple paths.
Your path seems correct, but you could try using path: ./ollvm/*. path is a little bit tricky, for example, I would like to cache whole workspace with path ./, however, it doesnt work as expected. I should use ./* and it works fine. see glob patterns

##[debug]Search path '/home/runner/work/MyApplication/MyApplication/ollvm'
##[debug]Cache Paths:
##[debug][]

I noticed that your cache path could not be resolved at step Post Cache multiple paths.
Your path seems correct, but you could try using path: ./ollvm/*. path is a little bit tricky, for example, I would like to cache whole workspace with path ./, however, it doesnt work as expected. I should use ./* and it works fine. see glob patterns

I tried both ./ollvm/* and ./ollvm/** and it also could not be resolved
U're so right, it's too tricky😭

I will try this
node_modules
*/*/node_modules
**/yarn.lock

Hope **/ollvm will work...

@hellodword Any luck?  Do you mind if I fork your repo to test this?

just overlooked at step Releasing using Hub, there are branch switching action Switched to a new branch 'master' and your ollvm folder was likely be stashed and hence, the later step  Post Cache multiple paths could not find any patterns matched. btw, the path ollvm is correct in your case.

@hellodword Any luck? Do you mind if I fork your repo to test this?

No luck it still failed 😭
And no not mind at all.

just overlooked at step Releasing using Hub, there are branch switching action Switched to a new branch 'master' and your ollvm folder was likely be stashed and hence, the later step  Post Cache multiple paths could not find any patterns matched. btw, the path ollvm is correct in your case.

Yes I think this is the reason! I will open a issue to this action.
Thanks for being a big help!

Thanks @etnet-raymondlin he is right! This is the reason.
Not the issue with actions/cache, sorry @dhadka

@hellodword @etnet-raymondlin Thanks!  I appreciate both of you taking the time to diagnose this issue and report back.


Plan creation wizard rework

Several users / unis have reported confusion over the plan creation wizard. It's not always intuitive to know how to get institutional templates. Perhaps we need to adjust the layout of this wizard so institutional templates are immediately visible beneath the org box (once selected) and it shows as an either/or to select a uni template or a funder one?
Dutch users have asked to discuss this at the next user group as they have a slightly different position emerging. ZonMw and NWO are allowing researchers to use uni templates when they are approved. This means DMPonline will need to change the logic for this use case to present relevant options. We can configure it as an option that's switched off for others.
Thoughts from US examples welcome @mariapraetzellis

We've also had requests to allow a one-click plan creation type feature for organizational templates.

The main preference to come out of the user group was having basic filters for orgs and funders which then bring up a dropdown list of all possible options. Admin users would like to configure the names and order of that list to align with local policy e.g. use the TU Delft template

relates to: https://github.com/DMPRoadmap/roadmap/issues/2835

Adding a recent comment from a user:

When creating a DMP, selecting a template is gated by the primary funding organization. It might be nice to have an option to use an alternative organization's template (e.g., the DOT template is not accessible if I list FAA as my primary research organization).

Not sure whether we discuss the same here @mariapraetzellis (might be) but we got suggestion that when a plan is created and the funder is selected the funder template will appear in the list of templates the researcher can choose. rather than automatically be presented with the funder template. In Netherlands the funders often approve institutional templates, and this is quite clear from their naming conventions but the institutional template would not appear the second funder template is selected/ Therefore having a list of templates where researchers could select one would be better way to go forward

Linking another set of issues that should be considered as part of this work: #3088 #2886

I have started on this. I see it is quite complex, so may take some time to understand functionality and then change it. As I need to understand current functionality to understand what may be feasible.

Adding a recent comment from a user:

When creating a DMP, selecting a template is gated by the primary funding organization. It might be nice to have an option to use an alternative organization's template (e.g., the DOT template is not accessible if I list FAA as my primary research organization).

@mariapraetzellis a question regarding the user's suggestion. So in the quote cited is the user wanting to see all publicly available templates from other orgs too?

@mariapraetzellis a question regarding the user's suggestion. So in the quote cited is the user wanting to see all publicly available templates from other orgs too?

Yes, this is one of the issues we will need to address, some funders allow institutional templates to be used, so while the current approach is helpful to narrow it down to the right template, it can be too restrictive in cases. The initial mock-up suggests something like a few recommended templates but also the option to use "other" and search for more. We probably need to narrow things down to exactly what we want where, but this is the general challenge.

@pherterich  All such input is helpful as I study and play around with the code.

It may be worth having @benjaminfaure show us their plan creation workflow in more detail. They have a custom approach that I believe is more flexible than the baseline Roadmap approach

@briri can you arrange it?

@johnpinto1 and @pherterich, I'm excited to start working on this feature. A larger Roadmap team meeting to discuss approaches to implementing this would be helpful. I will propose some times via Slack.

Just a note that we should be sure to make it easy for a user to generate a plan based on the generic/default template.

@briri I think I have done the changes you suggested. Please double check https://github.com/DMPRoadmap/roadmap/commit/c645381aa213621640e07c9cdcd33dd2b23e8fd0.

@briri  I have updated the code https://github.com/DMPRoadmap/roadmap/commit/fc5ac7e6f85db40f2d19e7b373365f39099fff4e . So the required asterisk appears beside template selection with associated tooltip and screen reader read only text.

Shouldn't there be a place for administrative data? I mean: "primary research organisation" and "funder" look in the wizard like fields that do not have anything to do the selection of a template. A different layout would make that more clear:
Choose your template:
by primary research organisation
by funder
Some of our users are confused that about these fields..

Or:
Choose your template
--Select your primary research organisation-- [ ] No research organisation associated ..
--Select your funder--[ ] No funder associated..
--Select your template--
All of them would have to be select boxes, and each of them is activated
by the select box above (or by checking the box next to it).
Every select box should have an empty value (with instructions),
so one must select a value, even if it is just one.
This way one knows what template was selected

@nicolasfranck Thanks will incorporate one of suggestion when I revisit code.

We have run into problems with users not selecting an appropriate template because they were trying to be as administratively correct as possible in the 'create plan' page, while they should actually be selecting the most appropriate template. This is a serious risk because this also means that they do not fill out the privacy list of questions (which is an institutional customisation to the template) we want them to answer.
We could think of 3 distinct functionalities where organisations and funders can be selected/entered in the DMP application

Purpose 1: org to which the DMP belongs in db: important for visibility, org admins etc. This should not be modifiable
Purpose 2: select a suitable template with tailored guidance

This is what you do in 'create plan' page and you want to make it as fool-proof as possible that users select the correct template with the instructions you want them to have. And you also want to make sure there is no confusion: this merely serves to select a template and guidance, not to have the most complete administrative information (see purpose 3)

Purpose 3: administrative details about your project regarding organisation and funder

This is not limited to available templates and guidances, there can be many more affiliations or funding sources. Just as is now the case for the funder information on the 'project details': this does not influence the funder template as entered in 'create plan', nor the available customisations eg question specific guidance.
A field similar to Funder could be added to Project Details to indicate the Research Organisation (or even call this 'affiliation' to not use the same terminology as 'organisation' in the application).
In an ideal world, this information is standardized by selecting from RoR supplemented by an 'Other' field in which free text can be entered

Shouldn't there be a place for administrative data? I mean: "primary research organisation" and "funder" look in the wizard like fields that do not have anything to do the selection of a template. A different layout would make that more clear:
Choose your template:
by primary research organisation
by funder
Some of our users are confused that about these fields..

@nicolasfranck I looked at your suggestion and realised that: this is not what the DCC & UC3 asked for. They want to the user to choose any template, with a default template being initially selected.

We have run into problems with users not selecting an appropriate template because they were trying to be as administratively correct as possible in the 'create plan' page, while they should actually be selecting the most appropriate template. This is a serious risk because this also means that they do not fill out the privacy list of questions (which is an institutional customisation to the template) we want them to answer.
We could think of 3 distinct functionalities where organisations and funders can be selected/entered in the DMP application
* **Purpose 1**: org to which the DMP belongs in db: important for visibility, org admins etc. This should not be modifiable

* **Purpose 2**: select a suitable template with tailored guidance

  * This is what you do in 'create plan' page and you want to make it as fool-proof as possible that users select the correct template with the instructions you want them to have. And you also want to make sure there is no confusion: this merely serves to select a template and guidance, not to have the most complete administrative information (see purpose 3)

* **Purpose 3**: administrative details about your project regarding organisation and funder

  * This is not limited to available templates and guidances, there can be many more affiliations or funding sources. Just as is now the case for the funder information on the 'project details': this does not influence the funder template as entered in 'create plan', nor the available customisations eg question specific guidance.
  * A field similar to Funder could be added to Project Details to indicate the Research Organisation (or even call this 'affiliation' to not use the same terminology as 'organisation' in the application).
  * In an ideal world, this information is standardized by selecting from RoR supplemented by an 'Other' field in which free text can be entered

@laurastandaert as I pointed out to @nicolasfranck it looks like the requirements suggested by DCC & UC3 are different from your suggestions. I think we need @mariapraetzellis , @dsisu and @briri to come up with reconciles their requirement with your suggestions. I know of one of them being currently on holiday.

@briri
@dsisu
@gjacob24
@laurastandaert
@mariapraetzellis
@martaribeiro
@nicolasfranck
We have deployed code for the current iteration of the Plan wizard to
https://dmponline-dev.dcc.ac.uk/
(Please note because of expired certificate on site you will get warnings which you should ignore to create account and login.)
Then go to "Create Plan" page (this is the current page). On the page there is a button "Try New Plan Creation Wizard" which will take you to the new functionality for you to critique for forthcoming meeting.
Warning to ignore:

Current "Create Plan" page with button button "Try New Plan Creation Wizard" to link to new functionality

New Plan Creation page

@pengyin-shan @benjaminfaure see John's note above about the new create plan page.

@pengyin-shan @benjaminfaure see John's note above about the new create plan page.

Great! unfortunately I cannot log in because I don't know the name and password...but the flow looks good to me.
Just FYI that DMP Assistant does not use Funder in workflow, so we always commented out the part that shows 'Select the primary funding organization'. We allow users to input anything in the 'funder' field when creating a new plan. This field will be a 'unmanaged' organization that saved to database without any other information

@johnpinto1 I'm wondering if we need to add a typeahead in the Template field. DMPTool has many (+40) templates shared between all organizations, and some orgs have numerous institutional templates on top of this. So, a user will be presented with a long list of templates to scroll through. Adding a filter or typeahead could be a way to find the template they want quickly.

@pengyin-shan You just create a new email account on server to login. Good luck!

@johnpinto1 Do you think it would be possible to organise the dropdown content?
I was thinking in something like:
Which DMP template would you like to use?

Templates from Primary Research Organisation
Template 1
Template 2
etc

Templates from Funders
Template 1
Template 2
etc

All other templates
Template 1
Template 2
etc

Will see I can incorporated both  suggestions by @martaribeiro & @mariapraetzellis

@briri I have made the changes @mariapraetzellis requested using your suggestion of HTML5 datalist in commit https://github.com/DigitalCurationCentre/roadmap/commit/ebed027f9a626557b4fabb19f079d6bf48c15ba5
This will be deployed to Dev by @martaribeiro tomorrow. I will update.

@briri I have made the changes @mariapraetzellis requested using your suggestion of HTML5 datalist in commit https://github.com/DMPRoadmap/roadmap/commit/adce18a61178c79f5e96b95c79b2d1dbf9a394b3
@martaribeiro has deployed the changes.
Styling of the Template dropdown requires styling. I found I will need to use a gem datalist-css as otherwise datalist style depends on browser.
@mariapraetzellis I hope this is what you requested

@briri
@dsisu
@gjacob24
@laurastandaert
@mariapraetzellis
@martaribeiro
@nicolasfranck
@pengyin-shan
I have updated the code which is deployed on the Dev server
https://dmponline-dev.dcc.ac.uk/
Changes:

Templates now can be searched in dropdown with typeahead.
Templates in dropdown have been split into three categories:

Templates from Primary Research Organisations
Templates from Funders
Other Templates

Note I am using an HTML5 datalist for the Template dropdown. It cannot be easily styled and is browser dependent. I will talk to @briri about alternatives when he is back from his holidays.

@laurastandaert found a bug in the New Plan Creation. You will get the following error if try to click on New Plan Creation button if you a new user or you are not an Org Admin. Let me know if you get error and I will temporarily make you Org Admin until I fix issue.

I have a fix for bug above. So new users won't need to be made Org admins by me. It will be deployed this afternoon. So bear with us if you get error above.

Fix now deployed. So all users should be able to create plans using New Plan page.

The plan creation branch will be temporarily unavailable until next week, as we need to do some testing on the roadmap one. We will let you know as soon as it is up again.

Adding a link to the ideas document shared by @nicolasfranck and Laura Standaert at UGent.

A Swedish user has come up with some further suggestions:

As for the plan creation wizard, there are some suggestions that might be worthwhile to consider.

Why not change the current text on the front page of the plan creation wizard and replace “No research organisation associated with this plan or my research organisation is not listed” with a sentence like “I would like to add a research organisation that is not listed here”? In a similar way, one could replace “No funder associated with this plan or my funder is not listed” with a sentence like “I would like to add a funder that is not listed here”. That would probably sound a bit more welcoming and would likely be no big deal to change.

As for research funders in Sweden, there is a long list of different funders that not only includes the Swedish Research Council and FORMAS but also many large and small private foundations. It is probably difficult to include them all, but it would not be difficult to invite a person who would like to write a data management in DMPonline to add their own funder and research organisation.

That applies also to the page about “Project details”. There is no need to show researchers a red flag when they enter a research funder that is not on the default list of DMPonline. Instead a of a red message “The name you entered was not one of the listed suggestions!” it might perhaps be possible to write something like “You entered a funder that currently is not on the default list, which may be perfectly fine.”

People in the UK are known for being very polite and why should DMPonline not speak the same kind of language? To add one more point about the page for “Project details”, it would likely be greatly appreciated, if there was a field that would allow people to add the research organisation that is responsible for the research project because this a very important information about a research project.

To add two more points that would make life a bit easier for people who use DMPonline, it would be very helpful, if the section about contributors would appear in the printed version of a data management plan because it would save us the effort to include this part as an own section into our template. In addition, we believe that it would make life easier for everybody, if the option “project detail cover sheet” would be preselected in the section “Download settings”. Not everybody looks at this point and as a result it can happen that people send in data management plans with no administrative information about their project. At least the second suggestion is likely a minor point that could be easily changed.

@briri @benjaminfaure @dsisu@gjacob24 @laurastandaert @mariapraetzellis @martaribeiro @nicolasfranck
@pengyin-shan
A reworking of the Plan creation wizard based on a spec suggested by the folk at Ghent @laurastandaert  and @nicolasfranck. With coding help from @nicolasfranck to get me started as I was lost trying to implement the spec. I hope I have done it justice. All omissions are solely my responsibility. It is a start and a work in progress. The deployed code is at a new URL https://dmponline-test.dcc.ac.uk/ . Note you might need to register again or for first time. There is a certificate issue on server so ignore warnings to proceed.

Seems very good to me, but I'm just a programmer ;-)
Anyone else?

@nicolasfranck Thanks. It appears our have contradictory requirements so we still have to sell this or change it to suit one group or the other.

@johnpinto1 sorry for taking so long, I was waiting for the end users to answer (to no avail). Do you have a list of the contradicting requirements/expectations of the end users?

@nicolasfranck We have not created a list. But at an User meeting we had in November some folk liked the change, others said giving users a choice of templates will confuse some researchers.

Also see issue #2835


is not a valid Absinthe.Schema with wrapped plugs in 1.5.1

I've wrapped the Absinthe.Graphiql.Plug so I can forward to it twice... Once with authentication, and another time for unauthenticated introspection queries:
defmodule IdoWeb.GraphQL.Introspection do
  @moduledoc """
  Wrapper around GraphiQL plug so we can re-use it in another route.
  (Phoenix prohibits forwarding to the same plug more than once)
  https://github.com/graphql-elixir/plug_graphql/issues/7
  https://github.com/phoenixframework/phoenix/pull/1419
  """
  use Plug.Builder

  plug(Absinthe.Plug.GraphiQL)
end

  # router.ex

  # GraphQL
  scope "/" do
    pipe_through([:api, :protected])

    forward("/graphql", Absinthe.Plug)

    if Mix.env() == :dev do
      forward("/graphiql", Absinthe.Plug.GraphiQL)
    end
  end

  if Mix.env() == :dev do
    # forward("/introspection", IdoWeb.GraphQL.Introspection)
  end

This worked great until 1.5.1, but now I'm getting this:
== Compilation error in file lib/ido_web/graphql_introspection_plug.ex ==
** (ArgumentError) IdoWeb.Schema is not a valid `Absinthe.Schema`

The change in 1.5.1 is the addition of this check:
  defp valid_schema_module?(module) do
    is_atom(module) &&
      Code.ensure_loaded?(module) &&
      Absinthe.Schema in Keyword.get(module.__info__(:attributes), :behaviour, [])
  end

I don't get this error with forward("/graphql", Absinthe.Plug) or forward("/graphiql", Absinthe.Plug.GraphiQL).  Is there some other change I need to make for 1.5.1 for this wrapping to continue working?

Hey @dfalling the correct way to wrap a plug is to delegate to the plug module.
defmodule IdoWeb.GraphQL.Introspection do
  @behaviour Plug
  defdelegate init(opts), to: Absinthe.Plug.GraphiQL
  defdelegate call(conn, opts), to: Absinthe.Plug.GraphiQL
end

I'm not 100% sure how your version is giving you your error though, are you eliding the schema option from your code samples here?

Ah, thanks!  Switching to defdelegate fixed it.
I was excluding the schema from my plugs- it's configured in my config.exs and from the error message I could see it was at least getting picked up.


Major upgrade of gem to use the latest Ruby, Faraday, Activesupport, and Sinatra

What does this PR do?

Upgrade Ruby to Version 3.3.5
Upgrade Faraday to Version 2
Removes faraday_middleware as it is no longer supported
Updates the Connection builder to not use the calls specific to faraday_middleware
Upgrade activesupport to 7.2
Upgrade Sinatra to version 3
Update change log

Thanks for this. I no longer use Pocket, but I'm happy to merge this once it passes. I'm noticing a CI failure relating to standard. If you run bundle exec rake you should see it.

@andyw8 Ive made the suggested changes

Looks like the changelog checker action job isn't working but I'll merge anyway.

https://rubygems.org/gems/pocket-ruby/versions/0.6.0

Thank you 🙏🏻


add GetScreenAspectRatio() function

add TEN.Util.GetScreenAspectRatio() lua function.
Corrected PlaySoundEffect() function.
Add Moveable:GetEndFrame() lua function to retrieve "End frame" of object's active animation

pr looks good to me :) can be merged.


500 TypeError [ERR_INVALID_ARG_TYPE]: The "path" argument must be of type string. Received undefined

am I having this error, any idea what it might be?
Connect
500 TypeError [ERR_INVALID_ARG_TYPE]: The "path" argument must be of type string. Received undefined
at validateString (internal/validators.js:120:11)
at resolve (path.js:980:7)
at View.lookup (/var/www/strider/apps/strider/node_modules/express/lib/view.js:114:15)
at new View (/var/www/strider/apps/strider/node_modules/express/lib/view.js:94:20)
at Function.render (/var/www/strider/apps/strider/node_modules/express/lib/application.js:570:12)
at ServerResponse.render (/var/www/strider/apps/strider/node_modules/express/lib/response.js:1012:7)
at exports.emberIndex (/var/www/strider/apps/strider/dist-lib/routes/index.js:60:13)
at Layer.handle [as handle_request] (/var/www/strider/apps/strider/node_modules/express/lib/router/layer.js:95:5)
at next (/var/www/strider/apps/strider/node_modules/express/lib/router/route.js:137:13)
at Route.dispatch (/var/www/strider/apps/strider/node_modules/express/lib/router/route.js:112:3)
at Layer.handle [as handle_request] (/var/www/strider/apps/strider/node_modules/express/lib/router/layer.js:95:5)
at /var/www/strider/apps/strider/node_modules/express/lib/router/index.js:281:22
at param (/var/www/strider/apps/strider/node_modules/express/lib/router/index.js:354:14)
at param (/var/www/strider/apps/strider/node_modules/express/lib/router/index.js:365:14)
at Function.process_params (/var/www/strider/apps/strider/node_modules/express/lib/router/index.js:410:3)
at next (/var/www/strider/apps/strider/node_modules/express/lib/router/index.js:275:10)
at slashes (/var/www/strider/apps/strider/node_modules/connect-slashes/lib/connect-slashes.js:81:9)
at Layer.handle [as handle_request] (/var/www/strider/apps/strider/node_modules/express/lib/router/layer.js:95:5)
at trim_prefix (/var/www/strider/apps/strider/node_modules/express/lib/router/index.js:317:13)
at /var/www/strider/apps/strider/node_modules/express/lib/router/index.js:284:7
at Function.process_params (/var/www/strider/apps/strider/node_modules/express/lib/router/index.js:335:12)
at next (/var/www/strider/apps/strider/node_modules/express/lib/router/index.js:275:10)
at /var/www/strider/apps/strider/dist-lib/app.js:226:9
at Layer.handle [as handle_request] (/var/www/strider/apps/strider/node_modules/express/lib/router/layer.js:95:5)
at trim_prefix (/var/www/strider/apps/strider/node_modules/express/lib/router/index.js:317:13)
at /var/www/strider/apps/strider/node_modules/express/lib/router/index.js:284:7
at Function.process_params (/var/www/strider/apps/strider/node_modules/express/lib/router/index.js:335:12)
at next (/var/www/strider/apps/strider/node_modules/express/lib/router/index.js:275:10)
I tried everything, but I can't find the solution.

Try 2.4.11
I'll push a fix tonight.

This is still an issue starting from v2.4.7 to strider_v2.4.13
There is no v2.4.11 release

v2.4.19 should work now.

Releases are on npm or via tags on github.


[Event Request] Cod 5763 "Whse.-Post Shipment" PrintDocuments()

Describe the request
Could you please add a new event in Codeunit 5763 "Whse.-Post Shipment" PrintDocuments()?
    local procedure PrintDocuments()
    var
        IsHandled: Boolean;
    begin
        // new publisher ...
        IsHandled := false;
        OnBeforePrintDocuments(SalesInvHeader, SalesShptHeader, PurchCrMemHeader, ReturnShptHeader, TransShptHeader, ServiceInvHeader, ServiceShptHeader, IsHandled);
        if IsHandled then
            exit;
        // ... new publisher

        SalesInvHeader.MarkedOnly(true);
        if not SalesInvHeader.IsEmpty() then
            SalesInvHeader.PrintRecords(false);`

Publisher
    [IntegrationEvent(false, false)]
    local procedure OnBeforePrintDocuments(
                    var SalesInvoiceHeader: Record "Sales Invoice Header";
                    var SalesShipmentHeader: Record "Sales Shipment Header";
                    var PurchCrMemoHdr: Record "Purch. Cr. Memo Hdr.";
                    var ReturnShipmentHeader: Record "Return Shipment Header";
                    var TransferShipmentHeader: Record "Transfer Shipment Header";
                    var ServiceInvoiceHeader: Record "Service Invoice Header";
                    var ServiceShipmentHeader: Record "Service Shipment Header";
                    var IsHandled: Boolean)
    begin
    end;

many thanks
stefan
Additional context
This would allow us to implement alternative printing logic.Internal work item: AB#493433

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. Please do not reply to this, as we do not monitor closed issues. If you have follow-up questions or requests, please create a new issue where you reference this one.
Build ID: 14518.


Unable to deploy. Always prompting Access Denied

sls s3deploy
Serverless Error ---------------------------------------
Access Denied
Get Support --------------------------------------------
Docs:          docs.serverless.com
Bugs:          github.com/serverless/serverless/issues
Issues:        forum.serverless.com
Your Environment Information ---------------------------
OS:                     win32
Node Version:           8.10.0
Serverless Version:     1.40.0

I tried adding the recommended IAM policy but still I have the isue. :(

@fredmararac Our teams ran into this issue too. The issue is that the acl default in this plugin is public-read and your bucket probably has S3 block public access turned on which prevents the default public-read acl from being set. See: https://github.com/funkybob/serverless-s3-deploy#acl and specify an acl: private

I've just published v0.9.0 which defaults to acl: private, thanks to kindly provided PRs.
Hope this helps. If not, please re-open this ticket.


Deprication Warning Use Post Request in the future

Hello,
I am seeing this warning continuously even though i have used Post Request method
Post Entry
    [Arguments]    ${ALIAS}    ${uri}    ${data}
    [Documentation]     Create an instance of a yang list
    ${headers}=    Create Dictionary    Content-Type=application/json
    ${resp}=    Post Request    ${ALIAS}    ${uri}    data=${data}    headers=${headers}
    Run Keyword And Return If    ${resp.status_code} != 200    Append Failed Log    ${resp.status_code}
    Return From Keyword    ${True}    ${resp.json()}

Could you please suggest me did i made any mistake here?
07:59:28.111 	INFO 	Deprication Warning  Use Post Request in the future

Thanks
Malli

Try installing the latest version of robotframework-requests package and try.

Thanks its worked with
robotframework-requests-0.3.9

-Malli


Python pandas pyarrow

Some general (very basic) clean up in the arrow code, made while I was reading it
Reorganized some of the pythonpkg files+folder layout
Did some basic work to separate Numpy from Pandas, for future work where we might want to add support for another backend (pyarrow)
Added support for converting pyarrow backed dataframes (by just calling pyarrow.lib.Table.from_pandas(df))
Extended our tests for Pandas, adding two proxies:
NumpyPandas just wraps the general pandas module with no extra logic for non-pyarrow columns.
ArrowPandas creates a pyarrow-backed dataframe instead (from the pandas.DataFrame constructor)
Both of these also proxy the testing.assert_frame_equal method, in the pyarrow case we currently convert pyarrow backed dataframes back to numpy-backed ones - because we can't produce a pyarrow backed dataframe yet.
Once that support is added though, this should hopefully only require changes there, and then all tests that use ArrowPandas should test whether we can properly roundtrip arrowpandas -> duckdb ->arrowpandas

TODO: make it compile and run on pandas 1.5.3


Update dotnet-core-cli-v2.md - remove duplicates on workingDirectory

description:

@Meir017 Good catch! The doc tooling imports these rules from the task source, and in this case, the property has rules, and is part of a group that has the same rules (but in a different order), and so they get duplicated.
https://github.com/microsoft/azure-pipelines-tasks/blob/0824d2516739b634154e211583db0dbcb960ede3/Tasks/DotNetCoreCLIV2/task.json#L500C1-L508C7
    {
      "name": "workingDirectory",
      "type": "filePath",
      "label": "Working directory",
      "helpMarkDown": "Current working directory where the script is run. Empty is the root of the repo (build) or artifacts (release), which is $(System.DefaultWorkingDirectory). The project search pattern is **NOT** relative to working directory.",
      "required": "false",
      "groupName": "generalAdvanced",
      "visibleRule": "command != restore && command != push && command != pack"
    },

And then in the referenced group:
https://github.com/microsoft/azure-pipelines-tasks/blob/0824d2516739b634154e211583db0dbcb960ede3/Tasks/DotNetCoreCLIV2/task.json#L62
    {
      "name": "generalAdvanced",
      "displayName": "Advanced",
      "isExpanded": false,
      "visibleRule": "command != pack && command != push && command != restore"
    }

I am merging this PR, and making the fix in the doc tooling to screen out this duplication. Thank you for letting us know!


flow: Annotate some more exports.

Toward #4907.
These are the four items I'd flagged as "tricky." Here, I follow Greg's suggestions for handling them at https://github.com/zulip/zulip-mobile/issues/4907#issuecomment-885992786. As it turns out, some of them are not all that tricky! 🙂

Thanks for the review! Revision pushed.

Thanks! Looks good -- merging.


runtime: unboundedly expensive equality testing

What version of Go are you using (go version)?
playground (1.21, i think?)
Does this issue reproduce with the latest release?
presumably
What operating system and processor architecture are you using (go env)?
N/A, should happen everywhere
What did you do?
https://go.dev/play/p/CcJuPywA867
summary: nest a handful of objects each of which is a struct{a, b, c, d any}, so that each points four times to the next. compare the "top" one. at N=16 or so, this begins taking longer to compare than the playground will let it run.
What did you expect to see?
I don't know.
What did you see instead?
Timeouts.
On the one hand, this is clearly You're Holding It Wrong. Don't Do That Then. Etcetera.
But I think in nearly all cases, I've been well-served by my belief that comparing go objects would have reasonably bounded runtime, because after all, anything that would cause them to explode or whatever would end up with pointer comparisons, and pointer comparisons come out equal-or-unequal quickly and easily. But interfaces provide a way to have a thing which sort of partakes in pointer-ish semantics, but doesn't actually fully copy things. If you weren't using interfaces for this, you'd actually have to populate all 4^16 items, or you'd have to use pointers (and then the comparisons would be fast). Interfaces let us bypass that perfectly good intuition and have copies of an arbitrarily-large nested tree that don't actually require copying it, but also don't enjoy the "we can compare these quickly" behavior of pointers. So each tier of this structure copies the next tier's four-interfaces header, but doesn't dereference the next tier's objects, so you can assemble the structure in time that's pretty much just linear with tree depth, but reading it is exponential.
On the one hand, I don't see a way to exploit this through user input to, say, encoding/json or anything like that. On the other hand, it feels like the way in which interface types are allowing us to, by copying a few words, create a thing that will act like it contains megabytes of data when you do == on it, is... unusual? Distinctive?
Obviously this is extremely susceptible to a small cache. On the other hand, the expense of building that cache for most comparisons would be punitive to every other go program in the world, pretty much. I don't have a good idea how to address this, or even whether it's worth addressing, but it sure is weird.

We do code-generate comparison functions, so we could add caches in just the cases that matter.
We'd only need a cache for types with >1 interface field.
It would be nice if we could detect that interfaces are equal if their data pointers are equal. I think that would mostly be correct, except for the pesky NaN. We could add a bit to types to mark them as reflexive (we already do that for map keys), and then data pointer equality implies interface equality. (Not sure if any of this paragraph would help OP's issue though, as different pointers doesn't imply values are different.)

Yeah, in this case, i think, the data pointers aren't equal? because i think each interface in a given struct is going to be a new copy of the next tier's struct, copied off and allocated and shoved into the heap for the T2I conversion. So we actually have four copies of each of these structs in memory, at different addresses, but which happen to have identical contents. Hmm. Although I guess we're not actually comparing those; we're never checking that x.a and y.b are equal, we're always comparing x.a to x.a, so they do in fact have identical pointers in that case.
I think that if you want to synthesize one of these that doesn't involve same-pointers, I think you end up needing to actually spend the space and time to build the data, at which point I think it's much more clearly Your Own Fault and you can reasonably be expected to anticipate that comparing two multi-gigabyte data structures is actually that expensive.

I've convinced myself this could actually happen by poking at the edges of "what if you used interface wrapping like this as a way to get quasi-immutable behavior for tree structures". consider type bintree struct { left, right any; val int }
you can make a binary tree of these, and later modifications to a node won't affect anything that got a copy of that node, and this is true recursively.
but say you do a slightly fancier thing, with  type node interface{} type btree {     first, last node     children [8]node     childCount int     val int }
Now a node with a single child will in fact end up comparing that child three times if you compare it to itself, and so on... So I think you could actually hit this in code that someone might write for reasons other than just to cause problems on purpose.

So I've been wrestling with why this case confuses my intuition so much, and I think... In general, the cost of comparison closely mirrors the cost of copying, except that for header-like things, the cost of comparison can be unrelated to the cost of copying -- but it'll still be comparable to the cost of making the thing the first time. Nested interfaces allow you to make a thing which goes exponential. Only it's actually integral-of-exponential; it's 1 + b^1 + b^2 + [...] + b^N, rather than just b^N. But for cases where all the objects are different objects, or there's only one interface per level, b is effectively 1, so it's still just linear time comparison from a constant-time assignment.
So, when you create each of these objects, you are allocating four copies of the next-tier-down object... But each of those copies is only paying for the cost of copying the headers, but the resulting comparison will have to pay for the cost of comparing the things the headers point to. So we end up with four copies of each of the structs being made to store in interfaces, but end up with 4^N comparisons to run.
I can't come up with a way to get similar behavior except through interfaces. I looked briefly at gob after someone suggested it in gopherslack performance chat, and I think it might be possible to synthesize a similar case there, but also gob explicitly disclaims security when handling untrusted inputs. I can't find a way to trigger this from encoding/json.


BET East

Source
https://www.tvpassport.com/tv-listings/stations/bet-eastern-feed/323
Channels
BETEast.us
Notes
Adding to tvpassport please? it was added to another already

Added in #1776


Evaluation metric (acc vs. acc_norm) for lm-evaluation-harness tasks

Hi, thank you very much for generously open-sourcing your excellent work.
I've run the evaluation code you kindly shared and obtained the below results. I have a question regarding the metric for each task. Could you please clarify which one between acc or acc_norm [ref] was used for the PIQA, HellaSwag, ARC-e, ARC-c, and OBQA tasks? Thanks for taking the time to check this inquiry.
20%-pruned -> post-trained LLaMA from scripts/llama_prune.sh

Reproduced with LLM-Pruner's code based on lm-evaluation-harness 4d21ab6b27

Task
BoolQ
PIQA
HellaSwag
WinoGrande
ARC-e
ARC-c
OBQA

LLM-Pruner paper
66.79
77.58
68.48
64.96
64.06
37.88
39.00

Reproduced: acc
65.20
77.15
53.19
63.69
64.77
36.26
28.80

Reproduced: acc_norm
n/a
76.93
68.63
n/a
52.27
36.95
40.40

Original LLaMA-7B

Reproduced with LLM-Pruner's code based on lm-evaluation-harness 4d21ab6b27

Task
BoolQ
PIQA
HellaSwag
WinoGrande
ARC-e
ARC-c
OBQA

LLaMA paper
76.5
79.8
76.1
70.1
72.8
47.6
57.2

LLM-Pruner paper
73.18
78.35
72.99
67.01
67.45
41.38
42.40

Reproduced (LLM-Pruner code): acc
73.06
78.35
56.42
67.01
67.34
38.14
28.20

Reproduced (LLM-Pruner): acc_norm
n/a
77.37
72.99
n/a
52.48
41.38
42.40

Reproduced* with different commit hash, lm-evaluation-harness 3326c547a7

Task
BoolQ
PIQA
HellaSwag
WinoGrande
ARC-e
ARC-c
OBQA

Reproduced* (Different hash): acc
75.05
78.67
56.93
69.93
75.29
41.89
34.60

Reproduced* (Different): acc_norm
n/a
79.16
76.22
n/a
72.85
44.71
44.40

Note

underline score: reported metrics in the LLM-Pruner paper
The scores reported in LLM-Pruner paper are fully reproducible using this repo, and the lm-evaluation-harness version affects the scores because of recent updates.
[Table 1 of LLM-Pruner Paper] The evaluation is performed under different prompts, which is lower than the official results in the LLaMA paper.

Hi. I used these keys:
select_key = {
'boolq': 'acc',
'hellaswag': 'acc_norm',
'arc_easy': 'acc',
'piqa': 'acc',
'arc_challenge': 'acc_norm',
'winogrande': 'acc',
'openbookqa': 'acc_norm',
}
Besides, I saw the difference in the performance of the original LLaMA-7B. I double-checked the code and used the evaluation code in my repo to re-evaluate the performance of LLaMA-7B, and I got a very similar performance (on ARC-easy, 67.38 vs. 67.45). Does the performance of LLaMA-7B you listed above use the lm-evaluation-harness (a previous commit of lm-evaluation-harness) in my repo? Since the lm-evaluation-harness has changed a lot in these months, some results are not consistent.

Thank you for your response and clarification!
I apologize for the missing commit hash regarding the LLaMA-7B results. Upon your indication, I realized that I used a different hash for lm-evaluation-harness compared to your code.

Since the lm-evaluation-harness has changed a lot in these months, some results are not consistent.

Thanks for pointing out this point. To ensure clarity, I've updated the table above and will soon provide results using your evaluation code. Thanks again for your assistance.

Hi, I've added the results using your repo, which are fully reproducible. I also made an explicit note to avoid any confusion. Big thanks for your time and help!

Note

underline score: reported metrics in the LLM-Pruner paper
The scores reported in LLM-Pruner paper are fully reproducible using this repo, and the lm-evaluation-harness version affects the scores because of recent updates.
[Table 1 of LLM-Pruner Paper] The evaluation is performed under different prompts, which is lower than the official results in the LLaMA paper.

Hi😄 . Thank you very much for the detailed notes and the experimental results you contributed based on the new version of the lm-evaluation-harness!


Issue being used as a nested addon

app.import is not a function
TypeError: app.import is not a function
    at Class.module.exports.included (/Users/nmcclay/git/light-table/node_modules/ember-light-table/node_modules/ember-element-resize-detector/index.js:20:17)
    at /Users/nmcclay/git/light-table/node_modules/ember-cli/lib/models/addon.js:244:32

I encountered this issue trying to upgrade ember-light-table and noticed this problem.
    if (!process.env.EMBER_CLI_FASTBOOT) {
      app.import('vendor/element-resize-detector/element-resize-detector.js');
    }

Able to reproduce with ember 2.6 and a new addon project.

This issue seems directly related to this discussion about nested addons: https://github.com/ember-cli/ember-cli/issues/3718
Moving ember-light-table to a peerDependency seemed to resolve this issue... so I'm moving on.

@nmcclay I have the same issue but I'm contributing for this addon. I cannot add ember-light-table as peerDependency. Is there a solution?


statistics/testDist01.py ERROR - numParticles <= 0:

C:\Users\Win\src\radiasoft\radtrack>python ./test/statistics/testDist01.py
Traceback (most recent call last):
  File "./test/statistics/testDist01.py", line 18, in <module>
    myDist = dist6d.RbDistribution6D(myArray6D)
  File "C:\Users\Win\src\radiasoft\radtrack\RadTrack\bunch\RbDistribution6D.py",
line 25, in __init__
    self.phaseSpace6D = ps.RbPhaseSpace6D(numParticles)
  File "C:\Users\Win\src\radiasoft\radtrack\RadTrack\bunch\RbPhaseSpace6D.py",
line 13, in __init__
    self.setNumParticles(numParticles)
  File "C:\Users\Win\src\radiasoft\radtrack\RadTrack\bunch\RbPhaseSpace6D.py",
line 29, in setNumParticles
    raise Exception(message)
Exception: ERROR - numParticles <= 0:
<RadTrack.bunch.RbPhaseSpace6D.RbPhaseSpace6D instance at 0x02C953C8>

This issue has been overtaken by events.
This non-test has been moved into the examples directory.


Update checks.js - Cross OS Compatible return handling

I had issues running this in mac, it turns out it doesn't handle returns the same way. the change replaces the two types of return (/r and /r/n) with a pipe which isn't commonly used in text, this in turn is used to split the file contents so this should work in windows and other OSs. I can't check windows today.

Tested on windows, worked without a problem. Merged, thanks for the patch.


⚠ Burdi - Selfservice has degraded performance

In 54782b7, Burdi - Selfservice (https://manage.burdi.com.au) experienced degraded performance:

HTTP code: 200
Response time: 9141 ms

Resolved: Burdi - Selfservice performance has improved in 6c20233.


Model-Server: License and formatting

This PR is about adding a plugin (spotless) for:

Add the License Header for the Apache License v2.0 to all Java source files under the Model-Server project
Formatting the code

The code can be reformatted at any time with a command (specified in the Readme).
I also removed the "Generated by MPS" comment. Besides that all the changes to Java code are the result of running the plugin (no intended change in behavior)

The header says

Licensed to the Apache Software Foundation (ASF)

We are not related to them. This is an example of the correct header:
https://github.com/JetBrains/MPS/blob/master/core/model/source/jetbrains/mps/model/ModelDeleteHelper.java#L4-L14

My bad, I took the wrong snippet.
Also, I read the copyright notice is not strictly needed, but if you want I can insert it.


Reset eye lerp on parent map change

Shouldn't affect escape shuttles but if we're mapping it will just reset it to our new relative entity.
:cl:

tweak: Eye lerping will reset on map changes.

Gonna be a future sloth problem if it ever comes up anyway.


quic: remove duplicate code in QUIC encodeData(), encodeMetadata() and encodeTrailers() implementations

quic: remove duplicate code in QUIC encodeData(), encodeMetadata() and encodeTrailers() implementations
Add a QuicSpdyStream& member to EnvoyQuicStream so that the encodeData(), encodeMetadata() and encodeTrailers() implementation can be moved to EnvoyQuicStream and out of the subclasses.
Risk Level: Low
Testing: Existing
Docs Changes: N/A
Release Notes: N/A

/assign @danzh2010

@danzh2010 friendly ping

This is a good movement! Thanks for doing it!
Just one question, is it possible to also move part of encodeHeaders(), i.e. ScopedWatermarkBufferUpdater, byte tracker and local_end_stream_ to EnvoyQuicStream?

Discussed offline, this can be done as a follow up.

/retest


KeyError: 'gilded'

Traceback (most recent call last):
  File "/home/red/redarc_docker/redarc/scripts/load_sub.py", line 33, in <module>
    gilded = sub_dict['gilded']
KeyError: 'gilded'

When entering AskReddit_submissions

Pushshift data isn't normalized and there are inconsistencies sometimes.  Working on a fix

https://github.com/Yakabuff/redarc/commit/5da45f2abcc2130bf4cb93e009cac149dbd2caa9
This commit should fix it.  Let me know if that works for you

@ohhdemgirls

Prefect, thanks. Seems to be working, going to take awhile to get everything imported mind you :rofl:

Spoke too soon, same file failed differently toward the end...
Identifier inserted succesfully x9zhzi
====================
ERROR: A string literal cannot contain NUL (0x00) characters.

(also spelling typo in your code :stuck_out_tongue_closed_eyes: )

Working on a fix.  I'm also considering removing the SQL transactions; it's not ideal to have to reinsert every submission/comment after an error especially for very large dump files

it's not ideal to have to reinsert every submission/comment after an error especially for very large dump files

True, once I'm able to get everything into the DB it's going to take some tuning for it to be lean but I've already provisioned all nvme storage and a terabyte of ram for it so we should be able to host a static historical version of what PS had available on it's API before people started wanting things removed and reddit nuked it!
There's going to be more edge cases in this data I'm sure so I'm very glad you're able to put in some time wrangling it, get on discord if you want to knock heads on this. Realizing who you are I've been a fan of your previous works :ok_hand:

I completely revamped the scripts. It now inserts everything and just simply logs the problematic rows.

I also added .replace("\u0000", "") to handle null characters in user submitted fields like selftext and title

@ohhdemgirls

real    383m8.352s
Worked that time, but now lets work on speeding up ingest if possible :yum:

real    383m8.352s
Worked that time, but now lets work on speeding up ingest if possible :yum:

It's the print statements that's making it slow.  Btw, did you check the log file for any warnings/errors


Performance limitations

Hi there, what are the performance limits on this? We are hitting "Reached maximum number of concurrent runs per pipeline" running from Blob Storage to Azure SQL Server

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: b7ee5166-ff40-f4ed-ce07-5312f935e388
Version Independent ID: 68f01b43-e3b2-003b-24cb-446a08aa0c1b
Content: Create event-based triggers in Azure Data Factory
Content Source: articles/data-factory/how-to-create-event-trigger.md
Service: data-factory
GitHub Login: @douglaslMS
Microsoft Alias: douglasl

@timdbowden Azure Data Factory limits are documented here: Data Factory limits. Maximum concurrent runs per pipeline is currently 100.

Does this mean that the event based trigger is not suite to instances where
we are receiving 100+ blob per second?
On Mon, Oct 1, 2018 at 6:18 PM Douglas Laudenschlager <
<EMAIL_ADDRESS>wrote:

@timdbowden https://github.com/timdbowden Azure Data Factory limits are
documented here: Data Factory limits
https://docs.microsoft.com/en-us/azure/azure-subscription-service-limits#data-factory-limits.
Maximum concurrent runs per pipeline is currently 100.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/MicrosoftDocs/azure-docs/issues/15909#issuecomment-425989504,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AFPB77ULt6VoikiPGG_VTwJ29A0GHkfLks5ugk5LgaJpZM4XCPIC
.

Sent question to Data Factory team.

#in-progress

The default Data Factory concurrent pipeline runs limit is set to 100. If you have an event trigger that creates events faster than your pipeline can process than you have a couple of options. One, and the method we recommend, is to change your trigger configuration so that it fires less frequently. Instead of having the trigger fire for each blob created and/or deleted you could instead use a control file that signals when a "batch" of files should be processed. If however, this is not feasible for your scenario and you need to process each file individually, than the only other alternative is to request a pipeline concurrency limit increase from the product team.

#please-close


Keep empty files empty

__init__.py is the most prominent example, usually is only put on
a folder to turn it into a package, but no code is added there.

Coverage remained the same at 100.0% when pulling 0e4615188d30e8c80286bee0e98c8e8fbecdd703 on gforcada:emtpy-files into dadcc0086d1b6a428b32fe0115510d0a14fe67cc on tk0miya:master.

@tk0miya could you provide some feedback on this pull request?

@gforcada Sorry for late response.
This PR is LGTM.
I will merge it and release new version. (but I have no time now...)

@tk0miya I could do the release myself if you give me the rights for it both on PyPI (gilforcada) and here at github (@gforcada)

Sorry for late response. Finally, I released it as v1.1.0.
Thank you for contribution.


#3260 - CAS Integration 3A - Create new Supplier and Site - Refactor CAS authentication

As agreed in the previous PR, the CAS authentication is now encapsulated inside the CAS API service and the auth token no longer needs to be passed to every method.
The current CAS token expiration is set to 1 hour and the token will be considered "about to expire" one minute before it.

Thanks for making this refactor to simplify the CAS logon call. Please have a look at the comments.


Change default bind address to localhost

Pull Request
Related issue
Fixes #2782
What does this PR do?

Change the default bind address to localhost so that it can be accessed with IPv6

PR checklist
Please check if your PR fulfills the following requirements:

[x] Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
[x] Have you read the contributing guidelines?
[x] Have you made sure that the title is accurate and descriptive of the changes?

Thank you so much for contributing to Meilisearch!

Thanks @Fall1ngStar 🚀

bors try

Hi 👋
Thanks for the review ! Looks like I didn't ran the test in release mode, so that error slipped under my radar 😅
Should I make the default_http_addr function public in options.rs and import it in segment_analytics.rs ? Or should I just use the raw value in the latter ?


Update reference files

Include ZZ-Zgs mixture check for the ZZ final states in gluon fusion

This PR adds the ZZ-Zgs mixture checks on gg MCFM vs JHUGen.
Bottom line: There is agreement.

Please approve so that we can merge.

(Note: There are slightly different MCFM vs JHUGen ratios when flavor=4, which corresponds to Z1->qq summed, but that is because summation is performed without K factors in MCFM and with in JHUGen. This is known. Exclusive Z1->dd, Z->uu agrees using flavor=5.)

Sorry, but I have no reason to split this PR into two. The update to references were pending for the JHUGen release, and they do not change how MELA operates. I would suggest to approve this PR immediately. A discussion of changes to the JHUGen repository should happen there (and it is a trivial matter that does not even need a discussion tbh). In addition, matters related to policy should be outside of Git discussions, I agree, but matters related to code review should be on Git; Git is precisely designed to optimize it, and it is common for discussion to happen on this platform.


new hook - allRequestsFinished

One hook right after all updates finished.
I tried using requestFinished but sometimes it didn't fired after the last request.
(it would be also usefull for plugins whose updates after each requestFinished)

Is this still necessary, or do the finer grained hooks now available serve the function of this request?


reformat template

ty tox
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Closed in favor of https://github.com/aws/aws-parallelcluster/pull/975


feat:Enhance model

This PR will allow customization on models.

Thanks @naimsolong !
would you mind to update the docs too?

Hey @fabio-ivona
Done update the docs 👍🏻

@naimsolong looks good to me! thanks for your contribution!


Set/GetThreadContext

Please read the remarks about Set/GetThreadContext
https://learn.microsoft.com/en-us/windows/win32/api/processthreadsapi/nf-processthreadsapi-setthreadcontext
https://learn.microsoft.com/en-us/windows/win32/api/processthreadsapi/nf-processthreadsapi-getthreadcontext
You have to Suspend the thread before using the above functions, and resume them after.

You cannot get a valid context for a running thread. Use the SuspendThread function to suspend the thread before calling GetThreadContext.

Do not try to set the context for a running thread; the results are unpredictable. Use the SuspendThread function to suspend the thread before calling SetThreadContext.

If you call GetThreadContext for the current thread, the function returns successfully; however, the context returned is not valid.

The code in test.cpp is complete trash. No one would use it like that.
Better use try catch/except for your code if you want exception handling.
For use in an debugger or for hooking purposes your library is not suitable.
In general not really a clean implementation for a library and it is incomplete. No adding and handling of exception handlers. If breakpoint is hit you should check which debug register was hit, Dr6 contains this info.

Sorry, my reply was a little delayed. I know what the docs say about using a separate thread, but it was not possible to do in my scenario (too slow), and it turned out that just doing it on the same thread worked.
As for implementation being a "complete trash", I should mention that I used it exactly like it is. And it worked for me, but I can admit that I was not under debugger, I needed a crash to be generated.
I agree that the library is not perfect. If you have more specific suggestions, PRs are welcome.


chore(deps): update vitest

#660  👈 (View in Graphite)
main

This stack of pull requests is managed by Graphite. Learn more about stacking.


Add connectivity diagram

Coverage increased (+0.4%) to 58.449% when pulling b1b19d18a583e9aab3d11e5949ba3b6913c75ca0 on gabrielburnworth:master into 46a348dd7f10fed88b1bb573668733b243a54227 on FarmBot:master.


Has the pipeline function been removed from node-redis?

Description
Hi,
I remember using the pipeline function in node-redis to batch commands and execute them together. Is this function now deprecated? Has it been replaced with automatic pipelining within the client?
Here's an example of how I've been using the pipeline function:
const redis = require("redis");
const client = redis.createClient();

client.on("error", (err) => {
  console.error("Error:", err);
});

function usePipelining() {
  // Create a pipeline within the client
  const pipeline = client.pipeline();

  pipeline.set("key1", "value1");
  pipeline.set("key2", "value2");
  pipeline.set("key3", "value3");
  pipeline.get("key1");
  pipeline.get("key2");
  pipeline.get("key3");

  // Execute the pipeline and handle results
  pipeline.exec((err, results) => {
    if (err) {
      if (err instanceof redis.RedisError) {
        console.error("Redis Error:", err);
      } else {
        console.error("Pipeline Execution Error:", err);
      }
      return;
    }

    console.log("Pipeline results:");
    results.forEach((result, index) => {
      if (result === null) {
        console.log(`Key ${index + 1} doesn't exist`);
      } else {
        console.log(result);
      }
    });
    client.quit();
  });
}

usePipelining();

Can someone clarify if the pipeline function is still the recommended way to batch commands, or should we rely on automatic pipelining features in newer versions of node-redis?
Environment Details:
Node.js version: v22.2.0
node-redis version: v4.6.15
Redis Server Version: v7.0.15
Operating System: Ubuntu 24.04

@sjpotter

commands that are written on the same "tick" are auto pipelined, so you can just:
const replies = await Promise.all([
  client.get('a'),
  client.get('b')
]);

you can also execute a multi as a pipeline (without the "MULTI" and "EXEC" wrappers):
const replies = await client.multi()
  .get('a')
  .get('b')
  .execAsPipeline();

Hi @leibale
Does client.pipeline() or client.batch() was deprecated ?

https://itsvinayak.hashnode.dev/redis-pipelining-in-nodejs

@itsvinayak sorry for the late reply, but:
"Similar to exec, this command executes the queued commands as a pipeline instead of a transaction." - correct.
"Non-atomicity: execAsPipeline doesn't guarantee atomicity. It sends commands to the server in a single network round trip but doesn't ensure that the operations are executed as a single atomic block." - not 100% correct, "It sends commands to the server in a single network round trip" - it'll be executed in "one round trip" as long as the socket "can handle it" (it will follow the "drain" events).
"Performance: It can be faster for bulk operations since it reduces the number of network round trips but lacks atomic guarantees." - with auto pipelining on the performance of these 2 should be similar:
await client.multi()
  .get('a')
  .get('b')
  .execAsPipeline();

await Promise.all([
  client.get('a'),
  client.get('b')
]);

TBH, the only reason to use "pipeline" is:

when sometimes you execute it as a multi

const mutli = client.multi()
  .ping();

if (Math.random() > 0.5) {
  await multi
    .set('a', 'b')
    .exec();
} else {
  await multi.execAsPipeline();
}

when you are reusing the same "pipeline" multiple times

const incrPipeline = client.multi()
  .incr('a')
  .incr('b');

await Promise.all([
  incrPipeline.execAsPipeline(),
  incrPipeline.execAsPipeline()
]);

Thank you for providing clarification, @leibale


Use action Verb

I accomplished X

Reliable, Relevant, Remarkable

• Assess client needs
• Facilitate problem-solving solutions
• Implement strategy
• Design product specifications
• Develop campaigns
• Conduct analysis
• Write expert documents
• Manage development
• Consult with investors
• Deliver tailor-made solutions
• Revamp supplier trainings
• Analyze revenues


Dont working dataset download links

Can you check the download links please. Not working any of them . Thanks.

should now work again.


Support for Shadow of the Demon Lord

Hello! I've used this module a bunch with other systems, but can understand why it doesn't inherently work out of the box for Shadow of the Demon Lord.
SotDL tracks "Damage" instead of Health, meaning a character that is healthy has a value of 0 and a character that is dead is one who's damage is equal to their max health.
Unfortunately the fractions don't seem to work in a way to allow that.
Thanks for everything!

Are you sure? It should work with this:
Hit Points Attribute Data Path: actor.system.characteristics.health
Fraction Equation set to the 2nd option, (Max - Cur)/Max.

You could be right. Maybe it was just a matter of not updating properly, in which case I am dreadfully sorry. Let me give it a try.

Well, thanks for the quick assist and pointing out my error. Thank you for your time!


Use on own audio stream

@SaoYear
Thank you for your work!
I want to be able to detect a unique audio cue in my video stream that captures audio. Assuming that i have trained a fine-tuned model, how can i go about using it in a causal manner?

Hi! Thanks for the interest and sorry for the late response!
The CRNN model for SED is causal while the ATST-SED is not. ATST-SED composes of a pretrained ATST model, which is based on the Transformer model, and since the self-attention modules in ATST is NOT a "masked" version. The ATST cannot be used in an online manner.
As for obtaining the audio cue, if you want to capture the audio cue in an offline way, you can refer to this issue.
Hope these help and please contact me if you have any other problem.
Nian

@SaoYear
Thank you for the clarification.

The CRNN model for SED is causal

what model are you referring? is it a different repo?

Sorry,
the CRNN means the baseline model of the DCASE challenge 2021-2023, you can find it here.
This CRNN is also included in the ATST-SED architecture, which is why I mentioned it.


Change database I/O for records to strings instead of arrays

This PR changes the mechanism for transferring records to and from the database to use multivalue delimited strings instead of arrays.  There are a few underlying reasons for this change:

The UniBasic code to convert the dynamic array to a UDO Array is very complicated and challenging to debug and modify.  The equivalent TypeScript code is significantly easier to understand, extend, and debug.
This should end up as a performance improvement.  Currently a lot of work is done on the DB backend to turn the dynamic array into a UDO array, only to then serialize it into JSON which then also incurs its own performance penalty on large data sets.  The DB server is not scalable in the same way that a TypeScript consumer might be without vertically scaling the DB server and a TypeScript consumer can likely be horizontally scaled.  I believe avoiding the work of converting to an array, then serializing the array for transit and serializing it back into an array will improve things significantly.
By handling multivalue strings directly, Document instances can be constructed directly from a multivalue string.  This will allow for multivalue strings, regardless of how they have been acquired, to be converted into a Document.  Therefore it becomes possible to get data outside of the DB connection created through this library and still apply a schema to it and construct an object to work with.  Essentially this increases the flexibility of the library for future use cases that might not require database connections.

The main changes in this PR are to adjust the formatDocument partial and the save database feature to emit and receive multivalue delimited strings.  Additionally, the getServerInfo database feature has been adjusted to emit the delimiter marks in use on the database server so the strings can be appropriately split regardless of which delimiters are being used by the database server.  Defaults are applied prior to connection, but will be overwritten once the server connection has been established.
The Model constructor has been adjusted to accept a delimited string instead of an array and the Document class has been augmented with several static methods to create Document instances via specific factory methods instead of via the constructor.  The constructor has been marked as protected to encourage consumers to use the factory methods instead of the constructor.  One of those factory methods will allow for instantiating a Document via a multivalue string for the future use case indicated in point 3 above.  Several internal changes were made to accommodate for these changes to the constructors.
Finally, because the instantiation of Model instances has been changed to use a string instead of an array, the use case where an array could be provided to the Model constructor to act as a template has been removed and is a breaking change.  To ease consumers ability to navigate this change, a new non-enumerable property _originalRecordString has been added to the Model instance which will retain the string that the model was instantiated from.  If there is a desire to provide a template for a new Model instance, this string can then be passed in to the Model constructor and it will act as the baseline for the data on newly created instances.

I've converted this to a draft as there's some tweaks to how the delimiters are being ascertained I'd like to make.