Add support for non compiling scripting language

What problem does this solve or what need does it fill?
Currently Bevy require to have on computer rustc compiler to be able compile and use game which user created.
What solution would you like?
Adding new interpreted language will allow to run games without needing to compile anything.
Such language is really great to create fuzzers or tests
Engines like Godot or Unreal Engine have their own scripting languages GDScript and Unreal Verse(still WIP) which doesn't require compiling and works on almost any computer out of box.
Other engines use e.g. Python or Lua but this greatly increase its size since additional software must be present inside game engine to properly parse and execute code written in this languages

Adding an first party scripting language is officially a non-goal. However, there are plans for making it possible to tightly integrate scripting languages in third party modules.

Rhai is an interesting one, but probably not ideal scripting language. I myself will try to use it to program separate systems in runtime (mostly for prototyping purposes, but maybe also for "games about programming") I didn't tried Rhai+Bevy yet, but Rhai alone seems nice.

Currently Bevy require to have on computer rustc compiler to be able compile and use game which user created.

This is not completely true, you can distribute a binary of your game that you compiled, and then anyone can play it without having rustc.
That said, it really depends on what you want to be able to script:

Calling a user defined script with a predefined interface from a system defined in rust should be easy. For example, in a system with a Query<(&ComponentA, &mut ComponentB)>, pass every value of ComponentA to a script then modify ComponentB according to what the script return.
having a script being able to run any query or execute commands is quite harder and not very possible for now

Adding an first party scripting language is officially a non-goal. However, there are plans for making it possible to tightly integrate scripting languages in third party modules.

Can I ask why this is the case? I assume you see the value in a scripting language. I understand if your motivation is to not force bevy users to use any specific language.
Some research around using wasm could be the answer here. Our friends over at SWC are currently developing a plugin system using wasm (Wamer I believe).
This could be a good way to provide a first party solution for scripting. Maybe even a first party language since it would be as easy to integrate any language that supports wasm.

Can I ask why this is the case?

Scripting languages create a strict barrier between the engine and game code, which can be undesirable.
On another note, wasm is definitely considered as a possiblity for modding and scripting.

Just thought i would mention a few use-cases in support of this issue.

expose a dev-tools like console in game to change variables or whatever.
separate game logic and engine code.
allow modders to get into your game and extend it. a plugin system more or less.
let non-programmer team mates to adjust the game as they see fit easily. lua/python for example are much easier for them to adjust compared to a rust codebase.
extend bevy's plugin ecosystem.

for example, with godot, they have gd-native to write libraries to load at runtime or compile time from gdscript. they can afford that because C has a stable ABI.
meanwhile, bevy uses rust features heavily like Traits and most of them are not ABI Safe, so exposing bevy to any scripting lang would be a pain. bevy is just not designed with the use case of FFI.
godot doesn't really need to expose a "modding" interface. games can just load godot scenes as well as gdscripts from random folders at runtime. mods are as easy and safe as putting into a folder.
any kind of runtime modding support with Bevy is much much harder.
having first class support for scripting langauge is a huge benefit for being "accessible" to a lot of people. modders, devs and users.

+1 @coderedart
I think personally for me, the biggest argument for scripting is the ability to create a moddable game, I can't really see a way to achieve that without exposing some sort of sandboxed interpreted language.

https://github.com/bevyengine/bevy/issues/4474 is going to help move this forward.

You can probably implement interpreted language support for user generated content with the help of something like boa. I don't think an interpreted language should become part of the engine. I have used (jint)[https://github.com/sebastienros/jint] in Unity before, it took ~5 C# files to implement a a little framework where people could use JavaScript to interact with a tiny API. Admittedly, the project would grow much more if the API actually did things other than updating object positions, but I would leave this out of the engine

I agree the language shouldn't be part of the engine, but facilities for enabling one should be provided

There's an up-and-coming third party crate for scripting here: https://github.com/makspll/bevy_mod_scripting


Only partial import & no notes?

My highlights are only partially imported (about 50 books with highlights) and notes are not visible at all? Still very happy that I now have a direct way and interested in where the plugin goes from here!

Hello!
Unfortunately, notes were not in my mind while I was developing this plugin. I could add this feature request in the TODOs.
About highlights, can you provide more info?
For example

I can't see any highlights from a book or I can only see 70% of them
Books I couldn't extract highlight from are in PDF format

Thanks

Yes please add it to TODO if possible,
I can see all the highlights from individual books, but not all the books with highlights. None are in PDF, it is all Epub for me (or in some cases the kobo proprietary epub format

So, something goes wrong with some book "types" ...
At this point, you could check by yourself why some books are not considered during the extraction ( the whole process revolves around this query )
I can help you from here if you need support in doing this.
( I didn't want to inspect your sqlite DB beacuse I don't want you to share it publicly... )

I will play around with the query. If i Need help I will ask. Thanks again for the plugin,

You are welcome to my DB if it helps you, Is there a non public way to get it to you?
I see nothing out of the ordinary in the query, but I am not well versed in SQL, it has been a long time since I did anything with that.

Unfortunately there is no way to send direct messages to Github users, the only way to communicate is through projects.
There is something wrong with my query because I built it looking only on my specific database...
My goal was to extract highlights along with some metadata ( for example chapter title ) that are not easy to link. What I mean is that I couldn't find a better pattern than the one I used to build the query.
Connection between an highlight and a book is made by joining Bookmark and content tables. If there is no match between key columns i used to make this join, some books may be filtered out.
Other tools, such as this one, might help you to extract all highlights, but without metadata I was mentioning before. Can you please check if using this tool you could extract all your highlights?

I am not sure the issue is in the query per se, but then again there should not be a difference in the DB structure between two kobo's to account for the difference in output.

Yes I agree with you, DB model should be the same among devices.
Okay, the only way left to understand what's happening is to inspect data... Without sharing the whole sqlite file, could be ok if you provide only

content table
Bookmark table

Before doing this, check if no sensible data goes in your outputs. Remember that you're doing this at your own risk, I don't take any responsibility.

SELECT T2.ContentID as ID, BookTitle, Title as ChapterTitle, Text as Highlight, T2.DateCreated, T2.DateModified, VolumeIndex, StartContainerPath, EndContainerPath, SubChapters
FROM (SELECT *, group_concat(Title, '-') as SubChapters FROM content GROUP BY ChapterIDBookmarked ORDER BY VolumeIndex) as T1 INNER JOIN Bookmark as T2 ON T1.ChapterIDBookmarked = T2.ContentID
GROUP BY T2.BookmarkID
ORDER BY ChapterIDBookmarked, ChapterProgress

@bitwiseops I think the IDs used to join the 2 tables should be content.ContentId = Bookmark.VolumeId.
SELECT T2.VolumeID as ID, BookTitle, Title as ChapterTitle, Text as Highlight, T2.DateCreated, T2.DateModified, VolumeIndex, StartContainerPath, EndContainerPath, SubChapters
FROM (SELECT *, group_concat(Title, '-') as SubChapters FROM content GROUP BY ContentId ORDER BY VolumeIndex) as T1 INNER JOIN Bookmark as T2 ON T1.ContentId = T2.VolumeID
GROUP BY T2.BookmarkID
ORDER BY T1.ContentId, ChapterProgress

With this query, I was able to retrieve more highlights.
Retrieving the note is a matter of adding Annotation into SELECT.

General update: this is on my list to try and implement in my fork :)

The partial import issue has been fixed in 1.2.0!

Notes export is now included in 1.3.0.
@bitwiseops if you could please close this one as well 🙏🏾


chore: upgrade github workers to run on latest ubuntu

Related Issues/PRs
GH Announcement: https://github.com/actions/virtual-environments/issues/6002

#xxx
What changes are proposed in this pull request?
Briefly describe the changes included in this Pull Request.
How is this patch tested?

[ ] I have written tests (not required for typo or doc fix) and confirmed the proposed feature/bug-fix/change works.

Does this PR change any dependencies?

[ ] No. You can skip this section.
[ ] Yes. Make sure the dependencies are resolved correctly, and list changes here.

Does this PR add a new feature? If so, have you added samples on website?

[ ] No. You can skip this section.
[ ] Yes. Make sure you have added samples following below steps.

Find the corresponding markdown file for your new feature in website/docs/documentation folder.
Make sure you choose the correct class estimators/transformers and namespace.
Follow the pattern in markdown file and add another section for your new API, including pyspark, scala (and .NET potentially) samples.
Make sure the DocTable points to correct API link.
Navigate to website folder, and run yarn run start to make sure the website renders correctly.
Don't forget to add <!--pytest-codeblocks:cont--> before each python code blocks to enable auto-tests for python samples.
Make sure the WebsiteSamplesTests job pass in the pipeline.

AB#1918126

/azp run

Codecov Report

Merging #1603 (743aaea) into master (28a35b4) will decrease coverage by 8.62%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #1603      +/-   ##
==========================================
- Coverage   83.73%   75.10%   -8.63%
==========================================
  Files         310      310
  Lines       15823    15823
  Branches      752      752
==========================================
- Hits        13249    11884    -1365
- Misses       2574     3939    +1365

Impacted Files
Coverage Δ

...ft/azure/synapse/ml/explainers/TextExplainer.scala
0.00% <0.00%> (-100.00%)
:arrow_down:

...icrosoft/azure/synapse/ml/explainers/Sampler.scala
7.54% <0.00%> (-92.46%)
:arrow_down:

...soft/azure/synapse/ml/cognitive/AudioStreams.scala
0.00% <0.00%> (-87.88%)
:arrow_down:

...oft/azure/synapse/ml/explainers/FeatureStats.scala
0.00% <0.00%> (-87.50%)
:arrow_down:

...soft/azure/synapse/ml/explainers/TabularLIME.scala
11.76% <0.00%> (-83.83%)
:arrow_down:

...osoft/azure/synapse/ml/explainers/VectorSHAP.scala
11.76% <0.00%> (-82.36%)
:arrow_down:

...synapse/ml/cognitive/TextAnalyticsSDKSchemas.scala
0.00% <0.00%> (-81.20%)
:arrow_down:

...soft/azure/synapse/ml/explainers/TabularSHAP.scala
10.81% <0.00%> (-81.09%)
:arrow_down:

...crosoft/azure/synapse/ml/explainers/TextSHAP.scala
16.66% <0.00%> (-80.00%)
:arrow_down:

...crosoft/azure/synapse/ml/explainers/TextLIME.scala
17.24% <0.00%> (-79.32%)
:arrow_down:

... and 33 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

/azp run

/azp run

/azp run

/azp run

/azp run

Closing as Serena has added this


Fixing Liquid File for a Play button

Describe the bug
I told him to fix this play video issue..

  {% schema %}
  {
    "name": "Video Reels",
    "settings": [],
    "max_blocks": 10,
    "blocks": [
      {
        "type": "video",
        "name": "Video Reel",
        "settings": [
          {
            "type": "video",
            "id": "video",
            "label": "Video"
          },
          {
            "type": "text",
            "id": "title",
            "label": "Title",
            "default": "Video Title"
          },
          {
            "type": "textarea",
            "id": "description",
            "label": "Description",
            "default": "Video description goes here"
          }
        ]
      }
    ],
    "presets": [
      {
        "name": "Video Reels",
        "blocks": [
          {
            "type": "video"
          }
        ]
      }
    ]
  }
  {% endschema %}

        {% for block in section.blocks %}

{{ block.settings.title }}
{{ block.settings.description }}

        {% endfor %}

  <div class="reel-controls">
    <button class="reel-button" onclick="prevReel()">←</button>
    <button class="reel-button" onclick="nextReel()">→</button>
  </div>
</div>

✕

For programming/project advice, please visit discord.gg/stackblitz. Github issues is for feature requests and bugs in the bolt application, but is not the right place for help on bugs in your project unfortunately. Best of luck on your app!


Exclude ingress rule for keycloak when identity.keycloak.enabled is false

Which problem does the PR fix?
This pull requests addresses issue #530.
What's in this PR?
Added a if/else to prevent an ingress rule from being added for Keycloak when identity.keycloak.enabled is false
Checklist
Please make sure to follow our Contributing Guide.

Before opening the PR:

[x] There is no other open pull request for the same update/change.
[x] The commits follow our Commit Guidelines. I had to install go 😉
[x] Tests for charts are added (if needed).
[x] The main Helm chart and sub-chart are updated (if needed).

After opening the PR:

[x] Did you sign our CLA (Contributor License Agreement)? It will show once you open the PR.
[x] Did all checks/tests pass in the PR?

To avoid a bigger refactoring now, I've fixed it in #549


Can't use an output-only stream as request content

Hi eidheim,
In client_http.h, ClientBase::request function is declared as:
std::shared_ptr<Response> request(const std::string& request_type, const std::string& path, std::ostream& content,
const std::map<std::string, std::string>& header=std::map<std::string, std::string>())

But if we actually use an output-only stream like std::ostringstream for content, the request would fail because content.rdbuf() would return an empty buffer.
You can examine the behavior by changing the following line in http_examples.cpp:
stringstream ss(json);

to
ostringstream ss(json);

Running the modified program will make the main thread hang at the subsequent client request call.
My solution: change content's declared type to std::iostream&, enforcing it to be a input-output stream.
Also, please consider adding some overloads of the request function to support other content types, such as boost::string_ref.
Thanks.

Splendid, I'll fix this Tomorrow hopefully.
Have never actually used boost::string_ref, but I'll have a look at it (trying as far as possible to stick to std::). When you say overloads to the request function, you mean add string type template for the first two parameters? Or add one declaration specifically for boost::string_ref? Not sure one would notice the optimisation from using boost::string_ref instead of const std::string& here though.

Or can just as well template the first two arguments, as backward compability will be kept if I am not mistaken.

I mean we can use boost::string_ref for the third argument, to avoid unnecessary copying of the post content in some situations, it may provide some performance benefits when the size of content is large.
For example, I want to post some Json data generated from Qt's QJsonDocument, now I have to write:
QByteArray req_content = myJsonDocument.toJson();
std::stringstream req_body_stream;
req_body_stream.write(req_content.constData(), req_content.length()); // copying content
auto response = client.request("POST", req_path, req_body_stream, headers);

If we have an overload of boost::string_ref conent, above code can be changed into
QByteArray req_content = myJsonDocument.toJson();
boost::string_ref req_content_ref(req_content.constData(), req_content.length()); // no copying
auto response = client.request("POST", req_path, req_content_ref, headers);

boost::string_ref have a similar interface to std::string, internally it has only 2 data members: a const char * buffer pointer and a length, so it can be constructed and copied efficiently.

The first issue is fixed: content parameter in Client::request is now of type std::iostream (https://github.com/eidheim/Simple-Web-Server/commit/99204ec73dd59d310f55e0c38e5e898479455fa8)


Drop transforemrs testing of pre EAP 6.2, drop old transformers code

Windows Build 894 is now running using a merge of 9d7d23f073e8bccea87f0b4803c0dafa9738307f

Core - Full Integration Build 472 is now running using a merge of 9d7d23f073e8bccea87f0b4803c0dafa9738307f

Linux Build 1204 is now running using a merge of 9d7d23f073e8bccea87f0b4803c0dafa9738307f

Windows Build 894 outcome was SUCCESS using a merge of 9d7d23f073e8bccea87f0b4803c0dafa9738307f
Summary: Tests passed: 3014, ignored: 68 Build time: 0:18:30

Linux Build 1204 outcome was SUCCESS using a merge of 9d7d23f073e8bccea87f0b4803c0dafa9738307f
Summary: Tests passed: 3014, ignored: 68 Build time: 0:18:53

Core - Full Integration Build 472 outcome was FAILURE using a merge of 9d7d23f073e8bccea87f0b4803c0dafa9738307f
Summary: Tests failed: 1 (1 new), passed: 2824, ignored: 412 Build time: 0:47:40
Build problems:
Failed tests detected
Failed tests
org.jboss.as.test.integration.ws.wsse.signencrypt.EJBSignEncryptMultipleClientsTestCase.encryptedAndSignedRequestFromAlice: javax.xml.ws.WebServiceException: Could not send Message.
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:152)

Looks good other than needing a rebase!

Windows Build 936 is now running using a merge of 3d74fb7fe4b5810df8fce97a7d8ff2c81873333d

Linux Build 1250 is now running using a merge of 3d74fb7fe4b5810df8fce97a7d8ff2c81873333d

Core - Full Integration Build 514 is now running using a merge of 3d74fb7fe4b5810df8fce97a7d8ff2c81873333d

Windows Build 936 outcome was SUCCESS using a merge of 3d74fb7fe4b5810df8fce97a7d8ff2c81873333d
Summary: Tests passed: 3040, ignored: 68 Build time: 0:18:58

Linux Build 1250 outcome was SUCCESS using a merge of 3d74fb7fe4b5810df8fce97a7d8ff2c81873333d
Summary: Tests passed: 3040, ignored: 68 Build time: 0:22:43

Core - Full Integration Build 514 outcome was FAILURE using a merge of 3d74fb7fe4b5810df8fce97a7d8ff2c81873333d
Summary: Tests failed: 1 (1 new), passed: 2825, ignored: 414 Build time: 0:49:11
Build problems:
Failed tests detected
Failed tests
org.jboss.as.test.integration.ejb.stateful.passivation.PassivationTestCase.testPassivationMaxSize: java.lang.reflect.UndeclaredThrowableException: null
	at com.sun.proxy.$Proxy62.runTestMethod(Unknown Source)
	at org.jboss.arquillian.protocol.jmx.JMXMethodExecutor.invoke(JMXMethodExecutor.java:77)

Argh!  The legacy-test revert introduced the need for another rebase!

retest this please

Windows Build 940 is now running using a merge of 3d74fb7fe4b5810df8fce97a7d8ff2c81873333d

Core - Full Integration Build 518 is now running using a merge of 3d74fb7fe4b5810df8fce97a7d8ff2c81873333d

Linux Build 1255 is now running using a merge of 3d74fb7fe4b5810df8fce97a7d8ff2c81873333d

Windows Build 940 outcome was SUCCESS using a merge of 3d74fb7fe4b5810df8fce97a7d8ff2c81873333d
Summary: Tests passed: 3044, ignored: 68 Build time: 0:18:44

Linux Build 1255 outcome was SUCCESS using a merge of 3d74fb7fe4b5810df8fce97a7d8ff2c81873333d
Summary: Tests passed: 3044, ignored: 68 Build time: 0:22:20

Core - Full Integration Build 518 outcome was FAILURE using a merge of 3d74fb7fe4b5810df8fce97a7d8ff2c81873333d
Summary: Tests failed: 29 (29 new), passed: 2722, ignored: 414 Build time: 1:07:23
Build problems:
Failed tests detected
Failed tests
org.jboss.as.test.integration.ws.wsse.signencrypt.EJBSignEncryptMultipleClientsTestCase.encryptedAndSignedRequestFromAlice: javax.xml.ws.WebServiceException: Could not send Message.
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:152)

org.jboss.as.test.manualmode.ws.WSAttributesChangesTestCase.testWsdlHostChanges: java.lang.AssertionError: null
	at org.junit.Assert.fail(Assert.java:86)
	at org.junit.Assert.assertTrue(Assert.java:41)

org.jboss.as.test.manualmode.ws.WSAttributesChangesTestCase.testWsdlPathRewriteRuleChanges: java.lang.AssertionError: null
	at org.junit.Assert.fail(Assert.java:86)
	at org.junit.Assert.assertTrue(Assert.java:41)

org.jboss.as.test.manualmode.ws.WSAttributesChangesTestCase.testWsdlPortChanges: java.lang.AssertionError: null
	at org.junit.Assert.fail(Assert.java:86)
	at org.junit.Assert.assertTrue(Assert.java:41)

org.jboss.as.test.manualmode.ws.WSAttributesChangesTestCase.testWsdlUriSchemeChanges: java.lang.AssertionError: null
	at org.junit.Assert.fail(Assert.java:86)
	at org.junit.Assert.assertTrue(Assert.java:41)

org.wildfly.test.integration.security.picketlink.federation.SAMLMetadataTestCase.testFederationWithGlobalLogout: com.meterware.httpunit.HttpInternalErrorException: Error on HTTP request: 500 Internal Error [http://[::1]:8080/sp-metadata1/index.jsp]
	at com.meterware.httpunit.WebClient.validateHeaders(WebClient.java:619)
	at com.meterware.httpunit.WebClient.updateClient(WebClient.java:500)

org.wildfly.test.integration.security.picketlink.federation.SAMLMetadataTestCase.testFederationWithLocalLogout: com.meterware.httpunit.HttpInternalErrorException: Error on HTTP request: 500 Internal Error [http://[::1]:8080/sp-metadata1/]
	at com.meterware.httpunit.WebClient.validateHeaders(WebClient.java:619)
	at com.meterware.httpunit.WebClient.updateClient(WebClient.java:500)

org.wildfly.test.integration.security.picketlink.federation.SAMLPostBindingTestCase.testFederationWithGlobalLogout: com.meterware.httpunit.HttpInternalErrorException: Error on HTTP request: 500 Internal Error [http://[::1]:8080/sp-post1/index.jsp]
	at com.meterware.httpunit.WebClient.validateHeaders(WebClient.java:619)
	at com.meterware.httpunit.WebClient.updateClient(WebClient.java:500)

org.wildfly.test.integration.security.picketlink.federation.SAMLPostBindingTestCase.testFederationWithLocalLogout: com.meterware.httpunit.HttpInternalErrorException: Error on HTTP request: 500 Internal Error [http://[::1]:8080/sp-post1/]
	at com.meterware.httpunit.WebClient.validateHeaders(WebClient.java:619)
	at com.meterware.httpunit.WebClient.updateClient(WebClient.java:500)

org.wildfly.test.integration.security.picketlink.federation.RestoreOriginalRequestTestCase.testPostOriginalRequest: com.meterware.httpunit.HttpInternalErrorException: Error on HTTP request: 500 Internal Error [http://[::1]:8080/sp-post2/savedRequest/savedRequest.html]
	at com.meterware.httpunit.WebClient.validateHeaders(WebClient.java:619)
	at com.meterware.httpunit.WebClient.updateClient(WebClient.java:500)

org.wildfly.test.integration.security.picketlink.federation.RestoreOriginalRequestTestCase.testPostOriginalRequestWithParams: com.meterware.httpunit.HttpInternalErrorException: Error on HTTP request: 500 Internal Error [http://[::1]:8080/sp-post2/savedRequest/savedRequest.jsp?SAVED_PARAM=Param+was+saved.]
	at com.meterware.httpunit.WebClient.validateHeaders(WebClient.java:619)
	at com.meterware.httpunit.WebClient.updateClient(WebClient.java:500)

##### there are 19 more failed tests, see build details

Core - Full Integration Build 520 is now running using a merge of fc8d57f6b4f28c714cf4ab4267e87e356e0b2813

Windows Build 942 is now running using a merge of fc8d57f6b4f28c714cf4ab4267e87e356e0b2813

Linux Build 1257 is now running using a merge of fc8d57f6b4f28c714cf4ab4267e87e356e0b2813

Windows Build 942 outcome was FAILURE using a merge of fc8d57f6b4f28c714cf4ab4267e87e356e0b2813
Summary: Execution timeout (new); tests failed: 5 (5 new), passed: 3010, ignored: 68; org.wildfly.core:wildfly-core-testsuite-domain Build time: 0:45:06
Build problems:
Failed tests detected
Execution timeout
Process exited with code 1
Failed tests
org.jboss.as.test.integration.respawn.RespawnHttpTestCase.testStartKilledServer: java.lang.AssertionError: null
	at org.junit.Assert.fail(Assert.java:86)
	at org.junit.Assert.assertTrue(Assert.java:41)

org.jboss.as.test.integration.respawn.RespawnHttpTestCase.testReloadHcButNotServersWithFailedServer: java.lang.AssertionError: Did not have all running processes [Process{id=5448, process=respawn-two}, Process{id=3424, process=Host Controller}]
	at org.junit.Assert.fail(Assert.java:88)
	at org.jboss.as.test.integration.respawn.RespawnHttpTestCase.waitForProcesses(RespawnHttpTestCase.java:518)

org.jboss.as.test.integration.respawn.RespawnHttpTestCase.testDomainRespawn: java.lang.AssertionError: Did not have all running processes [Process{id=5448, process=respawn-two}, Process{id=3424, process=Host Controller}]
	at org.junit.Assert.fail(Assert.java:88)
	at org.jboss.as.test.integration.respawn.RespawnHttpTestCase.waitForProcesses(RespawnHttpTestCase.java:518)

org.jboss.as.test.integration.respawn.RespawnHttpTestCase.testReloadHc: java.lang.AssertionError: Did not have all running processes [Process{id=5448, process=respawn-two}, Process{id=3424, process=Host Controller}]
	at org.junit.Assert.fail(Assert.java:88)
	at org.jboss.as.test.integration.respawn.RespawnHttpTestCase.waitForProcesses(RespawnHttpTestCase.java:518)

org.jboss.as.test.integration.respawn.RespawnHttpTestCase.testReloadHcButNotServers: java.lang.AssertionError: Did not have all running processes [Process{id=5448, process=respawn-two}, Process{id=3424, process=Host Controller}]
	at org.junit.Assert.fail(Assert.java:88)
	at org.jboss.as.test.integration.respawn.RespawnHttpTestCase.waitForProcesses(RespawnHttpTestCase.java:518)

Linux Build 1257 outcome was FAILURE using a merge of fc8d57f6b4f28c714cf4ab4267e87e356e0b2813
Summary: Execution timeout (new); tests passed: 2701, ignored: 53; org.wildfly.core:wildfly-core-testsuite-domain Build time: 0:45:07
Build problems:
Execution timeout
Process exited with code 137

Core - Full Integration Build 520 outcome was FAILURE using a merge of fc8d57f6b4f28c714cf4ab4267e87e356e0b2813
Summary: Tests failed: 1, passed: 2825, ignored: 414 Build time: 0:48:30
Build problems:
Failed tests detected
Failed tests
org.jboss.as.test.integration.ws.wsse.signencrypt.EJBSignEncryptMultipleClientsTestCase.encryptedAndSignedRequestFromAlice: javax.xml.ws.WebServiceException: Could not send Message.
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:152)

retest this please

Core - Full Integration Build 523 is now running using a merge of fc8d57f6b4f28c714cf4ab4267e87e356e0b2813

Linux Build 1260 is now running using a merge of fc8d57f6b4f28c714cf4ab4267e87e356e0b2813

Windows Build 945 is now running using a merge of fc8d57f6b4f28c714cf4ab4267e87e356e0b2813

Linux Build 1260 outcome was SUCCESS using a merge of fc8d57f6b4f28c714cf4ab4267e87e356e0b2813
Summary: Tests passed: 3045, ignored: 68 Build time: 0:19:27

Windows Build 945 outcome was FAILURE using a merge of fc8d57f6b4f28c714cf4ab4267e87e356e0b2813
Summary: Execution timeout; tests failed: 5, passed: 3010, ignored: 68; org.wildfly.core:wildfly-core-testsuite-domain Build time: 0:45:05
Build problems:
Failed tests detected
Execution timeout
Process exited with code 1
Failed tests
org.jboss.as.test.integration.respawn.RespawnHttpTestCase.testStartKilledServer: java.lang.AssertionError: null
	at org.junit.Assert.fail(Assert.java:86)
	at org.junit.Assert.assertTrue(Assert.java:41)

org.jboss.as.test.integration.respawn.RespawnHttpTestCase.testReloadHcButNotServersWithFailedServer: java.lang.AssertionError: Did not have all running processes [Process{id=6020, process=respawn-two}, Process{id=3940, process=Host Controller}]
	at org.junit.Assert.fail(Assert.java:88)
	at org.jboss.as.test.integration.respawn.RespawnHttpTestCase.waitForProcesses(RespawnHttpTestCase.java:518)

org.jboss.as.test.integration.respawn.RespawnHttpTestCase.testDomainRespawn: java.lang.AssertionError: Did not have all running processes [Process{id=6020, process=respawn-two}, Process{id=3940, process=Host Controller}]
	at org.junit.Assert.fail(Assert.java:88)
	at org.jboss.as.test.integration.respawn.RespawnHttpTestCase.waitForProcesses(RespawnHttpTestCase.java:518)

org.jboss.as.test.integration.respawn.RespawnHttpTestCase.testReloadHc: java.lang.AssertionError: Did not have all running processes [Process{id=6020, process=respawn-two}, Process{id=3940, process=Host Controller}]
	at org.junit.Assert.fail(Assert.java:88)
	at org.jboss.as.test.integration.respawn.RespawnHttpTestCase.waitForProcesses(RespawnHttpTestCase.java:518)

org.jboss.as.test.integration.respawn.RespawnHttpTestCase.testReloadHcButNotServers: java.lang.AssertionError: Did not have all running processes [Process{id=6020, process=respawn-two}, Process{id=3940, process=Host Controller}]
	at org.junit.Assert.fail(Assert.java:88)
	at org.jboss.as.test.integration.respawn.RespawnHttpTestCase.waitForProcesses(RespawnHttpTestCase.java:518)

Core - Full Integration Build 523 outcome was SUCCESS using a merge of fc8d57f6b4f28c714cf4ab4267e87e356e0b2813
Summary: Tests passed: 2826, ignored: 414 Build time: 0:47:58

retest this please

Linux Build 1261 is now running using a merge of fc8d57f6b4f28c714cf4ab4267e87e356e0b2813

Core - Full Integration Build 524 is now running using a merge of fc8d57f6b4f28c714cf4ab4267e87e356e0b2813

Windows Build 946 is now running using a merge of fc8d57f6b4f28c714cf4ab4267e87e356e0b2813

Linux Build 1261 outcome was SUCCESS using a merge of fc8d57f6b4f28c714cf4ab4267e87e356e0b2813
Summary: Tests passed: 3045, ignored: 68 Build time: 0:18:47

Windows Build 946 outcome was SUCCESS using a merge of fc8d57f6b4f28c714cf4ab4267e87e356e0b2813
Summary: Tests passed: 3045, ignored: 68 Build time: 0:19:15

Core - Full Integration Build 524 outcome was SUCCESS using a merge of fc8d57f6b4f28c714cf4ab4267e87e356e0b2813
Summary: Tests passed: 2826, ignored: 414 Build time: 0:50:12


mosquitto: 1.6.12 -> 2.0.7

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://repology.org/metapackage/mosquitto/versions.
meta.description for mosquitto is: "An open source MQTT v3.1/3.1.1 broker"
meta.homepage for mosquitto is: "https://mosquitto.org/"
meta.changelog for mosquitto is: ""
Updates performed

Version update

To inspect upstream changes

Compare changes on GitHub

Impact

Checks done (click to expand)

built on NixOS

The tests defined in passthru.tests, if any, passed

0 of 0 passed binary check by having a zero exit code.

0 of 0 passed binary check by having the new version present in output.

found 2.0.7 with grep in /nix/store/l1kcwsvangfps8qjkb00mhpsznndl6pd-mosquitto-2.0.7

directory tree listing: https://gist.github.com/e8b7f63af89cd68840c3012fa506fda8

du listing: https://gist.github.com/db405a35618a818b0359abadd025b79f

Rebuild report (if merged into master) (click to expand)

1768 total rebuild path(s)

574 package rebuild(s)

562 x86_64-linux rebuild(s)
480 i686-linux rebuild(s)
465 x86_64-darwin rebuild(s)
261 aarch64-linux rebuild(s)

First fifty rebuilds by attrpath
androidStudioPackages.beta
androidStudioPackages.canary
androidStudioPackages.dev
apache-airflow
aws-adfs
aws-google-auth
aws-mfa
aws-sam-cli
awscli
awsebcli
awslogs
azure-cli
barman
bash-my-aws
buildbot
buildbot-full
buildbot-ui
calibre
cdesktopenv
certbot-full
checkov
cloud-custodian
collectd
collectd-data
cq-editor
credstash
csvs-to-sqlite
deja-dup
domoticz
duplicity
duply
dvc-with-remotes
elasticsearch-curator
far2l
fcft
fcppt
foot
fuzzel
gImageReader
gerbera
git-remote-codecommit
graph-cli
home-assistant
horizon-eda
hound
iceshelf
krename
lexicon
libebml
liblinphone

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/l1kcwsvangfps8qjkb00mhpsznndl6pd-mosquitto-2.0.7 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A mosquitto https://github.com/r-ryantm/nixpkgs/archive/5b7e208b6b23728d4bbd828bc88de31fc6bfb467.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/l1kcwsvangfps8qjkb00mhpsznndl6pd-mosquitto-2.0.7
ls -la /nix/store/l1kcwsvangfps8qjkb00mhpsznndl6pd-mosquitto-2.0.7/bin

Pre-merge build results
NixPkgs review skipped

Maintainer pings
cc @peterhoeg for testing.

Beware future reviewers: I tried to run nixpkgs-review pr 112672 on this but it took too long so I had to abort.

Infos regarding the migration from 1.x to 2.x: https://mosquitto.org/documentation/migrating-to-2-0/

@GrahamcOfBorg eval
Something about the rebuild count is way off.

Result of nixpkgs-review pr 112672 run on x86_64-linux 1

4 packages built:

collectd
collectd-data
domoticz
mosquitto

/rebase master

This is a semi-automatic executed nixpkgs-review with nixpkgs-review-checks extension. It is checked by a human on a best effort basis and does not build all packages (e.g. lumo, tensorflow or pytorch).
If you have any questions or problems please reach out to SuperSandro2000 on IRC.
Result of nixpkgs-review pr 112672 run on x86_64-linux 1

4 packages built:

collectd
collectd-data
domoticz
mosquitto

The following issues got detected with the above build packages.
Please fix at least the ones listed with your changed packages:

collectd:

warning: unclear-gpl
gpl2 is a deprecated license, check if project uses gpl2Plus or gpl2Only and change meta.license accordingly.
Near pkgs/tools/system/collectd/default.nix:47:5:
   |
47 |     license = licenses.gpl2;
   |     ^

See: https://github.com/jtojnar/nixpkgs-hammering/blob/master/explanations/unclear-gpl.md
warning: build-tools-in-build-inputs
libtool is a build tool so it likely goes to nativeBuildInputs, not buildInputs.
Near pkgs/tools/system/collectd/default.nix:20:3:
   |
20 |   buildInputs = [
   |   ^

See: https://github.com/jtojnar/nixpkgs-hammering/blob/master/explanations/build-tools-in-build-inputs.md
warning: unused-argument
Unused argument: fetchpatch.
Near pkgs/tools/system/collectd/default.nix:1:26:
  |
1 | { lib, stdenv, fetchurl, fetchpatch, darwin, callPackage
  |                          ^

Our NixOS module is not really affected by the change of mosquitto to be non-anonymous by default because this has been the case already. We need to write a changelog that certificates have to be readable by the mosquitto user now, which was not the case before.

Superseded by #110863.


OD weight for pretraining missmatch

when I use the shell train_RLIP_ParSeDA_v2_mixed_vgcocoo365_swinL.sh to reproduce the pretraining process, I set the --pretrained to the OD weight I downloaded swin_large_cocoo365_bs64_lr141_drop_path0.5_dp0_mqs_lft_dab_deformable_detr_plus_iterative_bbox_refinement_36eps_converted.pth.
However, there are many missing_keys. I wonder which checkpoint is the correct one for this shell.

Hi, actually there should be some mismatches since there are extra components introduced on top of detection parameters. If you use RLIP parameters for HOI fine-tuning, then there should not be any mismatch.


Waiting for the code!

The paper is very Good, wating for the code!

Thanks for your interest in our work. We plan to release the code in the next few weeks.

Really good job. Hope you can release the code. I want to apply it to my research.

Really good job. Hope you can release the code. I want to apply it to my research.

Thanks for your interest in this work. We are currently processing the code. It will be released in the next few days.


Draft: Unknown fields

A proof-of-concept implementation allowing users to opt in for preserving unknown fields.  There is more cleanup to do on this implementation, but if the approach is promising, it would address the issue for adding support for unknown fields.
This PR adds a prost-examples crate which illustrates the generate_unknown_fields build option.

You can try it out by running the test suite in prost-examples.

Hi sorry for the delay, I think the discussion around uknown fields/extensions etc is being discussed here https://github.com/tokio-rs/prost/issues/674. Since these are quite big changes to prost I'd like to go through a proper design process.

Thanks for getting back to me.  A full design process for something like this makes sense.  Indeed, this draft PR is mostly given in the spirit of starting a conversation like that.
I should note that the particular problem I am interested in resolving involves unknown fields as such, rather than extensions or custom options, though I can see how the way one deals with one of these problems might inform the other.
I manage a couple of tonic services such that service A sends a message to service C, proxied through service B (there are some additional details about the proxying which make it difficult for service B to just pass the encoded protobuf through to C without inspecting it and even changing it).  I would like to be able to add a field to my message and update services A and C to include that field, without having to update service B as well.  This new field is not meant to be available for third-party extensions though; it's just part of the new/future versions of my API.
Anyway, thanks again for your response.  Please let me know how best I can help with this feature, the design discussion, etc.  Thanks!

Hey, I just wanted to follow up on this draft PR. Is the feature desirable? Is the general approach (adding a catch-all list of unknown fields, keeping their values as raw bytes) reasonable? Would it be worth me attempting to refine the PR with an eye toward getting it merged? Thanks!

Hello, Just wanted to let you know that I'm very interested in this feature too, we have multiple processes which write in the same local DB (KV) where those processes can have different version of the schema.

@jasonahills Yeah, I think they are all related and the solution we have for extensions should cover your use case as well. I don't have much time to push forward on actually writing this feature but if someone wants to champion the design/implementation I would be more than happy to mentor.


Is it possible to deploy ACS locally on windows 1o

Is it possible to install AKS locally on my machine and deploy the ACS locally

@VenkateshSrini . No. AKS  is a managed Kubernetes service in Azure. You won't be able to install it on a PC.  The same applies to ACS as well. However both of these will be available in Azure Stack (The Operating System for Azure, available for on-prem use). Either way AKS instances will exist in Azure/Azure stack; not on Windows 10.   You can however install the "Kubectl" tools on Windows 10 and manage your AKS instance :).

@asubmani thanks for the detailed explanation.
@VenkateshSrini will close this issue for now, please re-open if you still have further questions.


Change GroupAffine/GroupProjective to maintain invariants

Currently the GroupAffine and GroupProjective structs do not maintain either of these invariants:

the point is on the curve;
the point is in the prime-order subgroup;

The GroupAffine struct provides methods to check these, but GroupProjective does not.  The API could be improved by:

[ ] removing the invariant check methods;
[ ] adding the invariant checks to the new() method and making it fallible;
[ ] adding immutable field accessors fn x(&self) -> &F, fn y(&self) -> &F, etc;
[ ] making the coordinate fields private.

This ensures that any users of the structures outside of the module subtree can't violate these invariants.

Please consider using the zkcrypto/group and zkcrypto/ff crates (the latter probably without the "derive" feature), if Arkworks' group APIs are going to change. These crates are pretty stable now (despite the 0.9 version number) and used in production for Zcash and the bls12_381 crate, so interoperability with them would be very useful, and they're quite minimal dependencies.

If this happens, it's worth retaining some new_unchecked interface for points not in the prime order subgroup:  You can avoid doublings like ((..((c + a_1) + a_2)..) + a_n) - c with c independent.  I've always seen this trick with DDH yielding independence of c, but I've heard people argue for also placingc in another coset, and maybe that's useful sometimes for reducing constraint count at the cost of fragility elsewhere.  This seems orthogonal to everything else discussed here.

@hdevalence That's why I only suggested switching to it in the case that Arkworks' APIs were going to change in order to make essentially the same PrimeGroup/CofactorGroup distinction that the group crate makes — which had been proposed.

Hmm I don't think we intend to have separate traits for PrimeGroup and CofactorGroup; the discussion in this issue is about changing APIs on certain generic structs (twisted_edwards_extended::Group{Affine,Projective} and short_weierstrass_jacobian::Group{Affine,Projective})


cmd/bosun: (wip/poc) expose append in templates

now that go1.11 supports in block assignment

notification test {
    actionBody = myActionBody
    post = http://localhost:2345
    contentType = application/json
}

alert test {
    template = test
    $seriesA = series("host=server01", epoch(), 1)
    $seriesB = series("host=server02", epoch(), 2)
    warn = avg(merge($seriesA, $seriesB))
    warnNotification = test
}

template test {
    subject = ``
    body = ``
    myActionBody = `
       {{- $s := makeSlice -}}
       {{- range $x := .States -}}
            {{- $s = append $s (index $x.Events 0).Time -}}
       {{- end -}}
       {{- $s | json }}
    `
}

Mixed Type Matrix example:
template test {
    subject = ``
    body = ``
    myActionBody = `
       {{- $s := makeSlice -}}
       {{- range $x := .States -}}
            {{- range $a := $x.Actions }}
                {{- $s = append $s (makeSlice $a.Time $a.User $a.Message $a.Type) -}}
            {{- end -}}
       {{- end -}}
       {{- $s | json }}
    `
}

[
  [
    "2018-09-26T13:53:14.346164162Z",
    "Kyle Brandt",
    "I hope this helps with testing.",
    "Note"
  ],
  [
    "2018-09-26T13:53:14.34616435Z",
    "Craig Peterson",
    "Me Too!",
    "Note"
  ]
]

Image for the UI commit:


overflow detected, setting loss scale to: 1024.0 but model is sitll being trained.

Hi,
When I train RoBERTa, I have many the warnings like,
2020-08-01 12:39:16 | INFO | fairseq.trainer | NOTE: overflow detected, setting loss scale to: 1024.0
2020-08-01 12:39:39 | INFO | train_inner | epoch 001:   3615 / 6547 loss=0.439, nll_loss=0.001, accuracy=87.9, wps=82565, ups=2.66, wpb=31075.5, bsz=64, num_updates=3600, lr=9e-06, gnorm=3.952, loss_scale=1399, train_wall=38, wall=1353
Is it fine?
My command is
WARMUP_UPDATES=4000      # 6 percent of the number of updates
LR=1e-05                # Peak LR for polynomial LR scheduler.
HEAD_NAME=fariseq_head_pre     # Custom name for the classification head.
NUM_CLASSES=2           # Number of classes for the classification task.
MAX_SENTENCES=8         # Batch size.
ROBERTA_PATH=fairseq/roberta.base/model.pt
SAVE_DIR=roberta_class_base_pre
fairseq-train fairseq/fairseq_data-bin
--restore-file $ROBERTA_PATH
--max-positions 512
--max-sentences $MAX_SENTENCES
--max-tokens 4400
--task sentence_prediction
--reset-optimizer --reset-dataloader --reset-meters
--required-batch-size-multiple 1
--init-token 0 --separator-token 2
--arch roberta_base
--criterion sentence_prediction
--classification-head-name $HEAD_NAME
--num-classes $NUM_CLASSES
--dropout 0.1 --attention-dropout 0.1
--weight-decay 0.1 --optimizer adam --adam-betas "(0.9, 0.98)" --adam-eps 1e-06
--clip-norm 0.0
--lr-scheduler polynomial_decay --lr $LR --warmup-updates $WARMUP_UPDATES
--fp16 --fp16-init-scale 4 --threshold-loss-scale 1 --fp16-scale-window 128
--max-epoch 40
--best-checkpoint-metric accuracy --maximize-best-checkpoint-metric
--shorten-method "truncate"
--find-unused-parameters
--update-freq 4
--save-dir $SAVE_DIR

Yes, that's expected when training with mixed precision (--fp16). It's just a notice, you can ignore it as long as you don't see the scale get too small... any scale >= 0.1 is probably fine.

Yes, that's expected when training with mixed precision (--fp16). It's just a notice, you can ignore it as long as you don't see the scale get too small... any scale >= 0.1 is probably fine.

Thanks for the answer. I got the same warning in training. However, What is actually happening when it detects an overflow?


Reflect changes to Rapid Reviews Infectious Diseases group

Rapid Reviews Covid-19 group has evolved to become Rapid Reviews Infectious Diseases.
As such we will need to reflect these changes in Sciety.
Our domain model does not currently support renaming groups, so we need to consider how to approach this. The group have said that they do not link to Sciety from their own pages and are happy for any URLs to be updated
Current URL slug  https://sciety.org/groups/rapid-reviews-covid-19/
Updated URL slug https://sciety.org/groups/rapid-reviews-infectious-diseases/
Brief Description:
RR\ID (Rapid Reviews\Infectious Diseases) is an open-access overlay journal that accelerates peer review of important infectious disease-related research preprints.
Who we are summary:
RR\ID (Rapid Reviews\Infectious Diseases) is an open-access overlay journal that accelerates peer review of important infectious disease-related research preprints. We are an evolution of Rapid Reviews: COVID-19 (RR\C19), which won the 2022 PROSE Award for Innovation in Journal Publishing for introducing a new peer review model facilitating rapid publishing of COVID-19-related research preprints. RR\ID expands beyond COVID-19 to advance our understanding of infectious diseases, leaning on a similar “curate, review, publish” model. RR\ID aims to prevent the dissemination of false/misleading scientific information and accelerate the validation and diffusion of robust findings. RR\ID aims to increase the application of science for the common good, responding to infectious disease challenges throughout the world.
Logo

New homepage seems to be https://rrid.mitpress.mit.edu/guidelines, that is where the existing one redirects.


[Libp2p] Add Request Response for Data to networking layer

Closes #2647
This PR:
Goal of this PR is to add a request response mechanism to the libp2p layer and exposed it via ConnectedNetwork.  The tricky part of this was figuring out a way for the network layer and consensus to interact nicely.
My first pass at this I wanted the network layer to completely contain the tasks for doing requesting and responding.  I thought maybe we could inject some consensus info into the network layer however I didn't see a way to make this work because the ConnectedNetwork interface is completely unaware of consensus.  It might make sense to keep the responding to requests internal by injecting the consensus state map (wrapped in Arc<RwLock>), I think that's just a small interface change.  It also might be possible to have the task which requests data also live in the libp2p network layer.  This one would require the network layer either getting events from Hotshot, or validating messages on it's own.  I didn't see a great path forward for either.
This PR does the following:

Adds to the Message structure a new data messages for requesting data and sending responses to those requests
Adds 2 new functions to ConnectedNetwork for requesting data, and for receiving requests to respond to.
Adds new behaviour to libp2p stack.  This is the standard request response behaviour, with no retries
Adds handling of all the events from the new behaviour

One thing to note is that request_data returns results directly to the caller.  It's not fire and forget.  This is done so it's easy to implement a retry mechanism where we iterate through different peers
This PR does not:
We don't yet use the request response.  On top of this we can build 2 new modules.  One will a new task that handles incoming requests for data by looking for the it in our consensus data and then responding.  The other will be a task which requests missing data based on the consensus messages we are receiving.
Key places to review:
The new behaviour
The flow of events through the libp2p stack
The interface additions to ConnectedNetwork.

The changes look good, but some general questions (since I'm a bit out of date with the plan for VID share dissemination):

How will nodes know which VID share to request and when? Will we do something similar to the polling model we have currently?
Won't we be sending direct messages with the share data to all VID participants? Would we save bandwidth by switching to this request/response model?

Good questions, The request response plan is detailed here: https://www.notion.so/espressosys/Libp2p-Fallback-cea28d2cf806466da0699a658ed312eb.
You can roughly how the requests will work in this new draft PR: https://github.com/EspressoSystems/HotShot/pull/2698.
To more specifically answer both questsions:

Nodes will spawn a request task just like other network tasks, this task will receive QuorumProposalValidated events (a new event emitted after QC verification on new proposals).  Then the task will wait some delay before sending the request.  Delay will be a configuration option.  If the data is available in our state at after the delay we'll cancel sending the request.  Same thing for trying other peers, we will stop if the data is in our state.  So we don't need any special injection of consensus events.
After this is merged we actually want to remove the direct sending of shares to everyone.  Combined with a tuned delay and CDN it should be the case that the majority of times no VID shares are sent over request response.  In the cases where many nodes do need to ask for the shares the load will be spread to the committee (nodes will request the leader or random committee member) which should help our scaling by 1 order of magnitude (assuming 10 nodes in a DA committee).

Side note:  The DA committee will have to calculate the VID shares from the block data for this to work.


How to use router.NotFound() in this template?

Hi!
I try to use router.NotFound(redirectToIndex()) in this template and ozzo-routing newer calls my redirectToIndex()
router := routing.New()

	router.To("GET,HEAD", "/ping", func(c *routing.Context) error {
		c.Abort()  // skip all other middlewares/handlers
		return c.Write("OK " + app.Version)
	})

	router.NotFound(redirectToIndex())

	router.Use(
		app.Init(logger),
		slash.Remover(http.StatusMovedPermanently),
	)

	rg := router.Group("/v1")
...

...
func redirectToIndex() routing.Handler {
	return func(c *routing.Context) error {
		http.Redirect(c.Response, c.Request, "http://dev.in-frame.com/a2/" + c.Request.RequestURI,
			http.StatusMovedPermanently)
		return nil
	}
}

What I do wrong?

I found a little problem.
I use serve static files in my code
logger.Info("Start Serving static files on " + app.Config.StaticPath)

	router.Get("/a2", file.Content("ui/index.html"))

	router.Get("/*", file.Server(file.PathMap{
		"/a2/": "./ui/",
		"/data/img/": app.Config.ImagePath,
		"/": app.Config.StaticPath,
	}))

	router.NotFound(redirectToIndex())

And this function send me a 404 error when file is not located on file system:
		if file, err = dir.Open(path); err != nil {
			if options.CatchAllFile != "" {
				return serveFile(c, dir, options.CatchAllFile)
			}
			return routing.NewHTTPError(http.StatusNotFound, err.Error())
		}
		defer file.Close

Can we check if is a path of Angular 2 not a file, then redirect it to our index.html?
Thanks!

I do not know why router.NotFound() It does not work if serve static files... (

router.NotFound only handles the situation when there is no matching route. In your case you do have a matching route (/*). Therefore NotFound won't be called. You may want to take a look at app/init.go file and modify the code that handles errors.

Thanks, but where it located? I see nothing..
https://github.com/qiangxue/golang-restful-starter-kit/blob/master/app/init.go#L20

Or you mean about https://github.com/qiangxue/golang-restful-starter-kit/blob/master/app/init.go#L64

It's this line: https://github.com/qiangxue/golang-restful-starter-kit/blob/master/app/init.go#L29
If you take a look at fault.Recovery, you may see how it handles errors. In your case, you may specially handle an error that you want to do redirection instead of normal error return.

If I understand you correctly me need to write own fault.Recovery function?

And one more question. Is it correct for Angular 2 in html5 mode?
http.Redirect(c.Response, c.Request, "index.html",	http.StatusOK)

Thanks!

Yes, write your own recovery function - it's not very complicated.
Sorry, I don't know about Angular2. Also, this starter kit is mainly meant for REST APIs. I'm not sure how you should make redirection work in Angular2 - should you return a redirection header or some JSON response that you handle on the client side to initiate redirection?

If I have correctly understood the problem, we just need to send our index.html and statusOk (not redirect)
http://stackoverflow.com/questions/31415052/angular-2-0-router-not-working-on-reloading-the-browser
As it is easiest to do in this case?

Maybe we just need to add this (optional) functionality to ozzo-routing?

I think this is application specific, although you could write a middleware (handler) so that the same code can be reused more easily.

Well this is how you look. I think it useful for all single page applications.))

Yes, it could be useful. If you can come up with an implementation, I'll be glad to take a look and see if it's generic enough to be included in the core package.

I think we may simply add next option:
if file, err = dir.Open(path); err != nil {
			if options.CatchAllFile != "" {
				return serveFile(c, dir, options.CatchAllFile)
			}
                        if options.EntryFile != "" {
                               http.ServeFile(c.Response, c.Request, options.EntryFile)
                        } else {
                               return routing.NewHTTPError(http.StatusNotFound, err.Error())
                        }
		}
		defer file.Close()

What is EntryFile?

http://stackoverflow.com/questions/31415052/angular-2-0-router-not-working-on-reloading-the-browser
EntryFile is the path to the file that we send, if not found in the file system.
The name is taken by analogy.
--entry-file=PATH - serve this file in place of missing files (useful for single page apps)
https://www.npmjs.com/package/live-server

What's the difference between this and CatchAllFile?

What is CatchAllFile?

options.CatchAllFile

Ohh! I miss this option)
Affect three sleepless nights)

So did it work for you now?

BTW, where do you deploy your application to? If to AWS, you may consider using S3 to distribute static resource files (js, images, etc.).

Thank you very much! I looked at options.IndexFile string and not on options.CatchAllFile :)))
...
logger.Info("Start Serving static files on " + app.Config.StaticPath)

	indexFile := "ui/index.html"

	router.Get("/a2", file.Content(indexFile))

	var serverOptions file.ServerOptions

	serverOptions.CatchAllFile = indexFile
	router.Get("/*", file.Server(file.PathMap{
		"/a2/": "./ui/",
		"/data/img/": app.Config.ImagePath,
		"/": app.Config.StaticPath,
		},
		serverOptions,
	))

	return router

Thats worked fine!

BTW, where do you deploy your application to? If to AWS, you may consider using S3 to distribute static resource files (js, images, etc.).

At this moment I use my own server on hc.ru, but on future it's a good idea!
We create a social network for the Russian film industry, and we will have a lot of content, including photos and videos.

Cool!

Qiang, add details about options.CatchAllFile to ozzo-routing doc please. This will save a lot of time for other people. )


Fix memory leak in slang.dll

When we run "slangc.exe", _CrtDumpMemoryLeaks() reports the memory leaks on Windows platform.
The memory leak has been reported for a while but it has been ignored.
It is because the messages are printed to the "debug output", which is visible only if you ran slangc.exe from VisualStudio.
Memory leak is never a good thing and we should fix it.
We should also setup CI to prevent it from happening in the first place.

When the change #4210 is merged, the issue can be reproduced by just running slangc.exe.
Here is an example,
$ build/Debug/bin/slangc.exe
Detected memory leaks!
Dumping objects ->
{1319712} normal block at 0x0000021ACE6BE010, 48 bytes long.
 Data: <`\&             > 60 5C 26 C4 1A 02 00 00 00 00 00 00 00 00 00 00
{3949} normal block at 0x0000021AC4265C60, 48 bytes long.
 Data: <          k     > 00 00 00 00 00 00 00 00 10 E0 6B CE 1A 02 00 00
{3948} normal block at 0x0000021AC425F420, 32 bytes long.
 Data: <                > 00 00 00 00 00 00 00 00 00 10 00 00 00 00 00 00
{3947} normal block at 0x0000021AC4201C50, 1024 bytes long.

 Data: <X a             > 58 D1 61 F5 FA 7F 00 00 01 00 00 00 00 00 00 00
{234} normal block at 0x0000021AC41DBED0, 1024 bytes long.
 Data: < 7              > 80 37 92 F5 FA 7F 00 00 D4 A6 A7 F5 FA 7F 00 00
{215} normal block at 0x0000021AC41DBA90, 1024 bytes long.
 Data: <                > 10 AC 91 F5 FA 7F 00 00 C0 BE A7 F5 FA 7F 00 00
{200} normal block at 0x0000021AC41DB650, 1024 bytes long.
 Data: <                > FC A6 91 F5 FA 7F 00 00 B4 A6 A7 F5 FA 7F 00 00
{190} normal block at 0x0000021AC41DB210, 1024 bytes long.
 Data: <                > E0 A1 91 F5 FA 7F 00 00 D0 A6 A7 F5 FA 7F 00 00
{169} normal block at 0x0000021AC41DADD0, 1024 bytes long.
 Data: <                > 90 9B 91 F5 FA 7F 00 00 B4 A6 A7 F5 FA 7F 00 00
{168} normal block at 0x0000021AC41DAB80, 528 bytes long.
 Data: <                > 00 00 00 00 00 00 00 00 90 AB 1D C4 1A 02 00 00
{163} normal block at 0x0000021AC41BF030, 128 bytes long.
 Data: <                > 00 00 00 00 00 00 00 FC FF 00 00 00 00 00 00 00
{161} normal block at 0x0000021AC41BF7B0, 128 bytes long.
 Data: <                > FC 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Object dump complete.

The message above shows 13 objects were still using memory when slangc.exe was about to exit.

We can use _crtBreakAlloc to track down where those allocations are made. I suspsect they may just be the global dictionary used in our profiler.

Valgrind gives significantly more useful output that this also

I think ASAN will be a good method to check this type of problems and Venki was able to use it.
@venkataram-nv , can you describe how you enable it?
We should also consider running ASAN on our CI if it isn't too slow.

We can enable ASAN through the CMake option SLANG_ENABLE_ASAN. A problem that I had with this though, was that in cmake/CompilerFlags.cmake it would ignore the -fsanitize=address flag because it supposedly was not supported by the compiler. What I did to fix this was to skip the check_cxx_compiler_flag check on line 29. Interestingly even with ASAN there are no leaks reported, and I believe its because ASAN cannot track leaks with global variables.


[Bug][Move compiler] Panic:  'ICE invalid use of tmp local' at state.rs:315

🐛 Bug
This program causes a panick in move-build:
script {
fun main() {
    let x = 5;
    let x_ref;
    let y_ref;
    x_ref = &x;
    y_ref = x_ref;
    while(true)
        x_ref = &8;
    assert(*x_ref < 2, 42);
    y_ref;
}
}

with
thread 'main' panicked at 'ICE invalid use of tmp local tmp%#17', language/move-lang/src/cfgir/borrows/state.rs:315:36

Side note: this is a pretty minimal test case, every line in the above program is required for the crash.
To reproduce
Shell command:
cat <<EOF > /tmp/crash.move && move-build /tmp/crash.move
script {
fun main() {
    let x = 5;
    let x_ref;
    let y_ref;
    x_ref = &x;
    y_ref = x_ref;
    while(true)
        x_ref = &8;
    assert(*x_ref < 2, 42);
    y_ref;
}
}
EOF

Stack trace/error message

click to expand trace
stack backtrace:
   0: rust_begin_unwind
             at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4/library/std/src/panicking.rs:483
   1: std::panicking::begin_panic_fmt
             at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4/library/std/src/panicking.rs:437
   2: move_lang::cfgir::borrows::state::BorrowState::check_use_borrowed_by::{{closure}}
             at /usr0/home/rijnard/diem-MASTER-confirm/language/move-lang/src/cfgir/borrows/state.rs:315

Expected Behavior
No panick/crash, probably want to raise a friendlier compile error message?
System information
Please complete the following information:

Diem Version: master @ 4130cb0e068d01faa769ec09961fe2b97e6b43c1
rustc 1.48.0 (7eac88abb 2020-11-16)
Ubuntu 18.04.5 LTS

Additional context
Found as part of a compiler fuzzing project that mutates programs: https://github.com/agroce/afl-compiler-fuzzer and https://github.com/comby-tools/comby-mutation-server.

This one is a bit of a doozy. Really I don't think we should error in this case (the assertion fails for trying to generate a message in this case). But it is unclear if it is easy to completely fix it by a deeper change in the borrow graph

cc @shazqadeer @sblackshear


chore: Fix Wagtail Preview

Related ticket(s):

<- Paste ticket URL here ->

About these changes
<- Add human-readable description here ->
How to check these changes
<- Where possible, provide guidance to help your reviewer ->
Before assigning to reviewer, please make sure you have

[ ] Checked things thoroughly myself before handing over to reviewer.
[ ] Included the ticket number in the PR title to help us keep track of changes.
[ ] Ensured that PR includes only commits relevant to the ticket.
[ ] Waited for all CI jobs to pass before requesting a review.
[ ] Added/updated tests and documentation where relevant.

For Reviewer
Once PR is merged, check and arrange to deploy on platform-sh.

This is resolved by updating Hostname in admin.site settings with the provisioned URL. Therefore PR is not required.


Fix invalid multiline string output in REPL

REPL inputs are being parsed as PlainLine. This PR formats PlainLine string literals containing \n as triple-quoted block strings.
This works (as far as I can see) because the regular code formatting will parse "\n" as a Line containing EscapedChar, so the source code formatting will not be impacted by this change.
Closes #3793.

Thanks for this contribution!


Merge fixes

You did it @lopezton-anthem!
Thank you for signing the Anthem Opensource Software CLA.


Add a skeletal PPP

File name (i.e., in the repo): docs/team/github_username_in_lower_case.md e.g., docs/team/goodcoder123.md
Follow the example in the AddressBook-Level3

Fix by #15, #20, #25, #33 and #35


Homepage can't see the list of files when used on Android, and it appears completely white

Details
Homepage can't see the list of files when used on Android, and it appears completely white
Steps to reproduce
No response
Debug information
No response
Operating systems
No response
Checklist

[X] I updated to the latest version of the plugin.

Unfortunately I don't have an Android device to test this on, but this sounds similar to #48, which is a core Obsidian issue. I recommend using a separate workspace on mobile as a workaround.


Feature: lsblk command add more useful flags

lsblk command add more useful flags like -O (outputs all available columns), -o (output select columns), -i (use ascii characters for tree formatting), -l (displays output in list format), -n (no heading)

Hi @HariniJeyaraman this has now been updated, but feel free to make any improvements and submit a PR. I will be more than happy to review it and merge it.


Fixed migration relavant issues and added some limitations to CI

With this PR, The issues related to migration to moveit organization is solved.

Benchmarks inside CI uses the docker image created by github actions. This dockerfile always clone from main branch. When we merge this PR, i hope this CI will work.


Change way to add global styles

I found this interesting situation, that if i add global styles with @import (for example import normalize.css, or google fonts) through LayoutContainer style file (as it is done in boilerplate), it doesn't work. Get an error:

site/node_modules/statinamic/lib/static/index.js:13
throw error;
^
TypeError: Cannot read property 'layout' of undefined
at Layout.render (index.js:25:24)
at [object Object].ReactCompositeComponentMixin._renderValidatedComponentWithoutOwnerOrContext (site/node_modules/react/lib/ReactCompositeComponent.js:678:34)
at [object Object].ReactCompositeComponentMixin._renderValidatedComponent (site/node_modules/react/lib/ReactCompositeComponent.js:698:32)
at [object Object].ReactCompositeComponentMixin.performInitialMount (site/node_modules/react/lib/ReactCompositeComponent.js:283:30)
at [object Object].ReactCompositeComponentMixin.mountComponent (site/node_modules/react/lib/ReactCompositeComponent.js:236:21)
at Object.ReactReconciler.mountComponent (site/node_modules/react/lib/ReactReconciler.js:39:35)
at [object Object].ReactCompositeComponentMixin.performInitialMount (site/node_modules/react/lib/ReactCompositeComponent.js:289:34)
at [object Object].ReactCompositeComponentMixin.mountComponent (site/node_modules/react/lib/ReactCompositeComponent.js:236:21)
at Object.ReactReconciler.mountComponent (site/node_modules/react/lib/ReactReconciler.js:39:35)
at [object Object].ReactCompositeComponentMixin.performInitialMount (site/node_modules/react/lib/ReactCompositeComponent.js:289:34)

But if i add this global styles file through scripts/index-client.js it work properly.
I spent a few hours on this shit, thinking why it does not work :D

I think the error is obvious. You removed the declaration of styles but you didn't remove it usage.
See
https://github.com/MoOx/statinamic/blob/eb4e94ff4d4843a258a8f2fbda7087c173b66c28/boilerplate/web_modules/LayoutContainer/index.js#L25
And https://github.com/MoOx/statinamic/blob/eb4e94ff4d4843a258a8f2fbda7087c173b66c28/boilerplate/web_modules/LayoutContainer/index.js#L33

Heh, yes. My fault :)


Update Development guide: Installation instructions

Hey! 👋
Just leaving a quick issue to say that I think the installation instructions in the development guide could have a line to say clone/fork the repo first. As it reads right now, it sounds like the pip command is the only thing a user needs to install scona.
Thanks!
Update 1
Users diving straight into the pip command may see this error message:
Could not detect requirement name for 'git+https://github.com/WhitakerLab/scona.git', please specify one with #egg=your_package_name

Perhaps the command needs to be?:
pip install -e git+https://github.com/WhitakerLab/scona.git#egg=scona

Update 2
Confirmed! pip command should be as described above in Update 1 ☝Package installs without cloning/forking 🎉

This will also need updating in the Getting Started section of the README

Why I think the #egg addition is needed
The lovely developers of this package probably haven't noticed this bug in your documentation before as you've probably all got local copies of the repo which contains a setup.py file with your dependencies and builds the egg info. Without a local copy (that is to say, a git clone or fork), pip doesn't know where to get this info from and so asks you to provide it in the command line.
(At least, I think this is what's going on. I might be wrong!)

Yes if I do the following, pip doesn't complain:
git clone https://github.com/WhitakerLab/scona.git
cd scona
pip install git+https://github.com/WhitakerLab/scona.git

So I think it's important to clarify whether or not users need to clone the repo first.


How to extract encoded features from an adversarial autoencoder (AAE)

Hello Wiseodd,
The source code of GAN and its variations provided here are very helpful for those who are interested in applying GAN in bioinformatics.
My question is how to extract encoded features from an AAE (e.g. aae_pytorch.py in the VAE folder). After reading the codes in aae_pytorch.py, I guess that the following codes can be used to extracted the encoded features of training set of MNIST data.
X = Variable(torch.from_numpy(mnist.train.images))
z_fake = Q(X)
np.savetxt('encoded_feature_MNIST.txt', z_fake.data.numpy())

Am I correct?
Thanks,
Xiangchun

Hi Xiangchun. Yes, that's correct.
On Mar 9, 2017 18:04, "Xiangchun Li"<EMAIL_ADDRESS>wrote:

Hello Wiseodd,
The source code of GAN and its variations provided here are very helpful
for those who are interested in applying GAN in bioinformatics.
My question is how to extract encoded features from an AAE (e.g.
aae_pytorch.py in the VAE folder). After reading the codes in
aae_pytorch.py, I guess that the following codes can be used to extracted
the encoded features of training set of MNIST data.
X = Variable(torch.from_numpy(mnist.train.images))
z_fake = Q(X)
np.savetxt('encoded_feature_MNIST.txt', z_fake.data.numpy())
Am I correct?
Thanks,
Xiangchun
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/wiseodd/generative-models/issues/6, or mute the
thread
https://github.com/notifications/unsubscribe-auth/ACMLklRBkm0Ju2pgz-fHVmdhpDORQf93ks5rj9y9gaJpZM4MX8ER
.

Hi Wiseodd,
Thanks for your reply. Just another follow-up discussion.  When applying AAE to my data (consisted of 272 samples and each sample has 10386 number of features), I observed that the D_loss and recon_loss are consistently decreasing whereas the G_loss is increasing.
Any idea on:

How to determine whether an AAE has been trained good enough
How to balance D_loss, G_loss and recon_loss simultaneously

Any comments and suggestions are appreciated.
Best,
Xiangchun

In AAE, reconstruction loss is well understood (it's just cross entropy or L2), and expected to decrease.
However, for GAN loss (D_loss and G_loss), there is no correlation between the quality of samples and the value of the loss. So, visual inspection is unfortunately still the best way to monitor the training.
To my knowledge, only recently there is a work that shows a meaningful loss function for GAN (WGAN paper https://arxiv.org/abs/1701.07875).

Thanks for your suggestion.


offline use in pyspark2.3.3

I  use this project on my work server spark cluster which version is 2.3.0 and python version is 3.6.1 . I must install this offline , but your introduction is an online setup .what should I do with offline install on offline use.
I install spark-knn0.3.0 on my cluster class path, and move your /python/pyspark-knn on my code.
use  /python/test.py to test .
my console info

Traceback (most recent call last):
File "", line 2, in
File "/usr/hdp/current/spark2-client/python/pyspark/init.py", line 105, in wrapper
return func(self, **kwargs)
File "/home/wsk/ttrms_local/src/jobs/holiday_forecast/knn/classification.py", line 19, in init
"org.apache.spark.ml.classification.KNNClassifier", self.uid)
File "/usr/hdp/current/spark2-client/python/pyspark/ml/wrapper.py", line 63, in _new_java_obj
return java_obj(*java_args)
File "/opt/anaconda2/envs/py36/lib/python3.6/site-packages/py4j/java_gateway.py", line 1428, in call
answer, self._gateway_client, None, self._fqn)
File "/usr/hdp/current/spark2-client/python/pyspark/sql/utils.py", line 63, in deco
return f(*a, **kw)
File "/opt/anaconda2/envs/py36/lib/python3.6/site-packages/py4j/protocol.py", line 320, in get_return_value
format(target_id, ".", name), value)
py4j.protocol.Py4JJavaError: An error occurred while calling None.org.apache.spark.ml.classification.KNNClassifier.
: java.lang.NoSuchMethodError: org.apache.spark.ml.param.shared.HasInputCols.$init$(Lorg/apache/spark/ml/param/shared/HasInputCols;)V
at org.apache.spark.ml.classification.KNNClassifier.(KNNClassifier.scala:23)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:247)
at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
at py4j.Gateway.invoke(Gateway.java:238)
at py4j.commands.ConstructorCommand.invokeConstructor(ConstructorCommand.java:80)
at py4j.commands.ConstructorCommand.execute(ConstructorCommand.java:69)
at py4j.GatewayConnection.run(GatewayConnection.java:214)
at java.lang.Thread.run(Thread.java:745)
Thank you very much!

@SummersKing  did you find how to solve it ?

@SummersKing did you find how to solve it ?

i just give up


CI: disable ioctl test

Codecov Report

Merging #2655 (85eada2) into main (bdc4178) will decrease coverage by 0.31%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main    #2655      +/-   ##
==========================================
- Coverage   59.90%   59.59%   -0.32%
==========================================
  Files         138      138
  Lines       23480    23482       +2
==========================================
- Hits        14065    13993      -72
- Misses       7591     7662      +71
- Partials     1824     1827       +3

Impacted Files
Coverage Δ

pkg/vfs/vfs_unix.go
76.23% <0.00%> (-15.21%)
:arrow_down:

pkg/vfs/reader.go
74.78% <0.00%> (-4.21%)
:arrow_down:

pkg/fuse/fuse.go
76.64% <0.00%> (-2.92%)
:arrow_down:

pkg/vfs/writer.go
76.49% <0.00%> (-1.20%)
:arrow_down:

pkg/fs/fs.go
75.73% <0.00%> (-0.85%)
:arrow_down:

pkg/meta/redis.go
70.14% <0.00%> (-0.18%)
:arrow_down:

pkg/meta/sql.go
64.29% <0.00%> (-0.10%)
:arrow_down:

pkg/vfs/handle.go
85.56% <0.00%> (ø)

pkg/vfs/vfs.go
76.44% <0.00%> (+0.03%)
:arrow_up:

pkg/chunk/disk_cache.go
79.70% <0.00%> (+0.49%)
:arrow_up:

... and 1 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


POC decorators @pipeline and @step

As a Data Scientist
I want to develop a POC
So that I know how the steps and pipelines will be implemented before the refactoring
Acceptance Criteria:

[x] a step decorator or class should available
[x] a pipeline decorator/class should be available
[x] pipelines/steps should be able to track their states (FAILED, RUNNING, SKIPPED, etc)
[x] a step should be able to retrieve its own logs
[x] a step should be able to time its own duration
[x] steps should be able to be forced running when the pipeline has failed
[x] the step should be able to send its own logging to picsellia
[x] the pipeline should be able to register its steps before running
[x] Decide with @n7-Thomas and @hicham how to resolve pipeline scan and context injection
[x] Apply the changes that were discussed

Definition of Done:

[x] All acceptance criteria are met.

Need a PR


Remove unnecessary headers

Change suggested by @cgmb. Should address SWDEV-340918.

This only affects cuda, which is passing, so I'm going to go ahead and force the merge.


Empty afiles

Allow the processing of sondes that have a broken or empty afile.
This also removes sondes that don't have any information about altitude from the pipeline between level 2.

(if you want to test this, those cases happen on HALO-20240827 and HALO-20240821)


support search in SICP JS

I add search functionality to the current NavigationBarSicp component.
need to work together with my modification for SICP backend which generate searchDataFile
run yarn json to generate searchjson Data before running yarn tryjson in the SICP backend, and run yarn start at the frontend

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

32 of 180 (17.78%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.2%) to 34.606%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

src/commons/navigationBar/subcomponents/SicpNavigationBar.tsx
32
180
17.78%

Totals

Change from base Build<PHONE_NUMBER>:
-0.2%

Covered Lines:
4823

Relevant Lines:
12990

💛  - Coveralls

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Warning: This coverage report may be inaccurate.
This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

For more information on this, see Tracking coverage changes with pull request builds.
To avoid this issue with future PRs, see these Recommended CI Configurations.
For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

32 of 180 (17.78%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.2%) to 34.606%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

src/commons/navigationBar/subcomponents/SicpNavigationBar.tsx
32
180
17.78%

Totals

Change from base Build<PHONE_NUMBER>:
-0.2%

Covered Lines:
4823

Relevant Lines:
12990

💛  - Coveralls


Updated wildcard filter for global recordset search with reverseFQDN

Changes in this pull request:

Updated wildcard filter for global recordset search with reverseFQDN using recordset_data table

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.1 out of 2 committers have signed the CLA.:white_check_mark: Jay07GIT:x: jvelku299jvelku299 seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.


Add LCD autoscroll

Awesome @joppi! Lcd module is one of the most incomplete parts of Nanpy (drawChar api would be great to  have).. I'll test and merge this  pr in the weekend, no worries for tests failing, I think to remove 2.6 support in the next version or pin a particular version of pyserial ..

I intend to use nanpy in a project where I may need the custom char api. I'll look into it then.

It would be really great to have the custom char api, @joppi , thanks a lot! :smile:

@joppi I've also added the clear method and the official example for autoscroll rewritten in python https://github.com/nanpy/nanpy/blob/4e293daad5af96ee16d93058dda45bae733be9ad/nanpy/examples/lcd_autoscroll.py thanks again! :smile:

Well in the end you really did more work on this one ;) thanks

@joppi nah I just added an example :smile: can I assign this to you, then? https://github.com/nanpy/nanpy/issues/63

@astagi sure, I should have time to look at it this week.

sounds great, thank you @joppi ! :smiley:


Editing a service name makes it impossible to schedule against the new service name

Rancher Version: 1.1.0-dev3.
Docker Version: 1.10.3.
OS and where are the hosts located? (cloud, bare metal, etc): aws.
Setup Details: (single node rancher vs. HA rancher, internal DB vs. external DB) single node rancher.
Environment Type: (Cattle/Kubernetes/Swarm/Mesos): cattle.
Steps to Reproduce:

Add two hosts, one with label "foo=1" and the other with "bar=1" for example.
Add a load balancer with the name "lb" to any stack and set a scheduling policy for hosts with label "foo=1".
Now, let's say you made a mistake in the last step and you actually wanted "bar=1". So, clone the load balancer and name it "lb-new".
Delete the old load balancer (named "lb").
Rename the new load balancer from "lb-new" to "lb".
Add another service "redis" (for example) and set a scheduling policy for a host that has a service with name "lb".

Results:
The redis service will never start because it won't be able to find the service. This is because the load balancer's container is still named "stack_lb_new_1".
Expected:
In my opinion, I expected one of the following to happen:

Cloning a load balancer (or any other service) should also rename the underlying containers to match the new service name.
The io.rancher.scheduler.affinity:container_label scheduling policy should look up the current service name instead of relying of the container's name.
Allow us to change all of the settings of a load balancer without having to clone it (scheduling policies, stickiness, etc.)

In the API, we support upgrade of load balancers, but it hasn't been exposed in the UI, which would solve  the exact issues that you are facing.  https://github.com/rancher/rancher/issues/3161
The scheduler actually schedules against the labels of the container, so I could see that there is an issue where if you rename a service, you would not be able to schedule against the new service name. We aren't able to change the labels on the container as docker container
I'm going to edit this issue to address the scheduling issue.
Steps to Reproduce:

Create service called foo
Edit service to be called foo1
Try and schedule a new service against the service name foo1.

Results: This will fail as the containers of foo1 will still have the label io.rancher.stack_service.name=<stackname>/foo instead of the new label io.rancher.stack_service.name=<stackName>/foo1

@deniseschannon has this been resolved with the upgrade LB feature?

Version - master 1/7
This is actually not specific to load balancer or to the scheduler.  The main issue here is when the service name is renamed, that does not reflect in the already started container.  The label for the service is not changed so nothing can find it.  Probably shouldn't be able to edit the service name, it should be part of the upgrade so container can be restarted with proper name and service label.

Version - 2.0 master 8/5
Editing the name of Services and Containers has been removed.


Update the lockfile for removing mixpanel

Eh, figured it was worth it regardless of the incoming PR

Decided to keep Mixpanel, so closing. Thanks for the help @orta !


Implement Candidate for Arc<str>

Should've already done it in #721

See #810

Thanks @gwenn ! 👍🏿


Add lerp pathing to voronoi project

We want to introduce splines in future. cubic lerp is a good place to start

Added with #31


Incorrect hash output when decompiling Cold War LUAs

When decompiling a Cold War LUA, the hashes are read and shortened incorrectly. The first issue is that the high and low bytes are read the wrong way round, as seen in LuaJitFunctionBOCW.cs:
            {
                var lo = Reader.ReadULEB128();
                var hi = Reader.ReadULEB128();

when it should be:
            {
                var hi = Reader.ReadULEB128();
                var lo = Reader.ReadULEB128();

The second issue is that hashes are shortened one byte too short, as seen in LuaJitFunctionBOCW.cs and Constant.cs where there are only 15 Fs:
var hash = ((hi << 32) | lo) & 0xFFFFFFFFFFFFFFF;
case ValueType.Hash:
                    return $"0x{Hash & 0xFFFFFFFFFFFFFFF:X}";

when it should be:
var hash = ((hi << 32) | lo) & 0xFFFFFFFFFFFFFFFF;
case ValueType.Hash:
                    return $"0x{Hash & 0xFFFFFFFFFFFFFFFF:X}";

Once those fixes have been applied, hashes now show up correctly. The DVAR "lobbysearchpingband"'s hash is 3BD37C49236A4C70 and the decompiled output went from Dvar[0x36A4C703BD37C49] to Dvar[0x3BD37C49236A4C70].
So far I have only tested Cold War LUA files, and the issue might also happen to other game LUA files.

Thanks for letting me know about this. The 15 F's is done with a reason so I'd rather keep it like this. For assetnames they sometimes add extra bits on top of the hash so in order to combat that we get rid of that one F, this is done by other tools like Greyhound aswell.


About UI Authentication

How to manage user access， I see that the UI does not require login. My requirement is that only admin can view it.   how to do it?  thanks.

Hi, @bonnlee2024. You have two options:

Put Netdata behind NGINX (or any other web server/proxy). Also, see https://learn.netdata.cloud/docs/netdata-agent/securing-netdata-agents/.
Use Netdata Cloud.


Fiddler Not Working

installed the latest GUI but whenever i start the program it said still connecting. any advice on how to make this work?

Maybe the game and the version of Fidder are not compatible.
If I'm not mistaken, there is no backwards compatibility with Fidder.
Also, it seems to me that I often had to start Fidder after the game side was fully launched and the title was displayed before it would connect properly.
Try it.


Enhance PinotHelixResourceManager.updateInstance() to preserve custom fields

When using rest API to update the instance config, preserve the custom fields. Some example custom fields are: fault domain, resource info etc.
Also add support to custom broker hostname in PinotQueryResource

Codecov Report

Merging #7098 (057985d) into master (1561414) will decrease coverage by 31.79%.
The diff coverage is 9.09%.

@@              Coverage Diff              @@
##             master    #7098       +/-   ##
=============================================
- Coverage     73.49%   41.69%   -31.80%
+ Complexity       91        7       -84
=============================================
  Files          1492     1492
  Lines         73422    73451       +29
  Branches      10574    10580        +6
=============================================
- Hits          53959    30629    -23330
- Misses        15946    40251    +24305
+ Partials       3517     2571      -946

Flag
Coverage Δ

integration
41.69% <9.09%> (-0.06%)
:arrow_down:

unittests
?

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

...pache/pinot/common/utils/config/InstanceUtils.java
0.00% <0.00%> (-75.87%)
:arrow_down:

...t/controller/api/resources/PinotQueryResource.java
0.00% <0.00%> (ø)

...ntroller/helix/core/PinotHelixResourceManager.java
37.50% <40.00%> (-24.97%)
:arrow_down:

...c/main/java/org/apache/pinot/common/tier/Tier.java
0.00% <0.00%> (-100.00%)
:arrow_down:

...ava/org/apache/pinot/spi/data/MetricFieldSpec.java
0.00% <0.00%> (-100.00%)
:arrow_down:

...va/org/apache/pinot/spi/utils/BigDecimalUtils.java
0.00% <0.00%> (-100.00%)
:arrow_down:

...java/org/apache/pinot/common/tier/TierFactory.java
0.00% <0.00%> (-100.00%)
:arrow_down:

...a/org/apache/pinot/spi/config/table/TableType.java
0.00% <0.00%> (-100.00%)
:arrow_down:

...a/org/apache/pinot/spi/data/DateTimeFieldSpec.java
0.00% <0.00%> (-100.00%)
:arrow_down:

.../org/apache/pinot/spi/data/DimensionFieldSpec.java
0.00% <0.00%> (-100.00%)
:arrow_down:

... and 953 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1561414...057985d. Read the comment docs.


[Static Dumper] [Docs] Add compiler pass in kernel

On a fresh install of Symfony, after require static dumper, and importing it's config, I get this error :
In DefinitionErrorExceptionPass.php line 54:

  Cannot autowire service "Symplify\SymfonyStaticDumper\ControllerWithDataProviderMatcher": argument "$controllerWithDataProviders" of method "__construct()" is type-hinted "array", you should configure its value explicitly.

When adding the compilerPass AutowireArrayParameterCompilerPass, it's fixed.
I think it should be mentionned :+1:

Good point, thanks!


Fix wrong krnl.load when broadcasting with all prefix 1s

The lowering of binary ops wrongly generates krnl.load in case of broadcasting, which is reported by issues #2154, #2150, #2137.
The issue is that the lowering failed to detect broadcasting in this case:
%0 = "onnx.Add"(%arg0, %arg1) : (tensor<1x1x5xf32>, tensor<5xf32>) -> tensor<1x1x5xf32>

While computing shape, tensor<5xf32> is prepended with 1x1, which turns out to be similar to the first input tensor<1x1x5>, then hasNoBroadcast function thinks they are the same and there is no broadcasting.
This patch fixes the issue by keeping the original shape during shape inference, and using the original shape to check broadcasting.

I confirmed that Issue#2154 was solved by this PR. Thanks!

@negiyas Did you test using the docker image or did you build onnx-mlir yourself? I use the docker image....so not sure if maybe that is why we have different outcomes. I can also test the model you used to see what I get back.

But our hasNoBroadcast has different meaning. What does hasNoBroadcast really mean to us?

It should be consistent with numpy's spec finally I think.

It should be consistent with numpy's spec finally I think.

The bumpy's spec says when the two inputs are multiple directional broadcastable, namely the requirement for the inputs.  Look at the first condition, for example, The tensors all have exactly the same shape.. I do think we want hasNoBroadcast() returns true for this case.
we do not to check braodcastable if the model is assumed to be correct. What we need from hasNoBroadcast() is related to how we generate the load. For example, we may implement the load without hasNoBroacast(). The loop can be generated as the current implementation. The affine map from iteration vector is to 1) mask out the none existing dimensions and 2) replace the iteration variable with constant 1 if the size of that dimension is 1. For the example the Roberta error, for tensor<768xf32>, the map will be #map<(d0, d1, d2)[] = (d2)>'. For case of tensor<2x3x4xf32> and tensor<2x1x4xf32>, the map for the tensors is  #map<(d0, d1, d2)[] = (d0, d1, d2)> and '#map<(d0, d1, d2)[] = (d0, 1, d2)>' Again, I think we need to clearly define the meaning ofhasNoBroadcast()while they are allmultiple directional broadcastable`.

The tensors all have exactly the same shape.. I do think we want hasNoBroadcast() returns true for this case.

Do you mean the current hasNoBroadcast() returns false for this case? and it is a bug and it should return true.
@AlexandreEichenberger any thought? I think you wrote the function for SIMD.

I am modifying this very code to get partial SIMD execution. What is the pattern that causes the issue? This one?
  %0 = "onnx.Add"(%arg0, %arg1) : (tensor<1x1x5xf32>, tensor<5xf32>) -> tensor<1x1x5xf32>

What is the pattern that causes the issue? This one?
%0 = "onnx.Add"(%arg0, %arg1) : (tensor<1x1x5xf32>, tensor<5xf32>) -> tensor<1x1x5xf32>

Yes, it is.

From a code pattern perspective, 1x1x5 and 5 have no broadcasting, in the sense that there are 5 values loaded for the first argument, and 5 loaded for the second, and each one is paired with each other.
While simd code will not be generated for such a short sequence, say it was 1x1x1024 and 1024, we should and we will.
Please @negiyas give me the test case you want me to investigate (e.g. the original benchmark with the specific commands to force arguments to 1) and I will make sure they work.

The original test case is Roberta. I think a simple test case should be OK.

From a code pattern perspective, 1x1x5 and 5 have no broadcasting, in the sense that there are 5 values loaded for the first argument, and 5 loaded for the second, and each one is paired with each other.
While simd code will not be generated for such a short sequence, say it was 1x1x1024 and 1024, we should and we will.

The error is related to how to generate the reference. As I said in my previous comments, the meaning of hasNoBroadcast is not clear. From the discussion, I found that hasNoBroadcast serves two purposes in our code: 1) check the reference is contiguous for SIMD, and 2) how to generate the reference.  They are not the same.  I made comments on how to generate reference in my previous comments.

@AlexandreEichenberger this is the test case (also in the lit test of this patch):
func.func @add_1_1(%arg0 : tensor<1x1x5xf32>, %arg1: tensor<5xf32>) -> tensor<1x1x5xf32> {
 %0 = "onnx.Add"(%arg0, %arg1) : (tensor<1x1x5xf32>, tensor<5xf32>) -> tensor<1x1x5xf32>
 return %0 : tensor<1x1x5xf32>
}

When you run onnx-mlir-opt --convert-onnx-to-krnl --convert-krnl-to-affine, you will get error with the main branch.

Totally agree. They are not the same, and I think this hasNoBroadcast is now mainly used for SIMD but it is also passed to shapeHelper.getAccessExprs to compute reference indices, which causes confusion if its semantics is vague.

Agreed, they need to work in sink. Fixing both of them at this time. The solution I am after is a bit more complicated yet as it handles partial simd, as I am collapsing subsets of inner loops and their memory accesses.
To give you a taste, ?x?x4x5 and ?x4x5 can have a collapse of inner loop of 2, to become ?x?x20" and ?x20where we handle possible broadcast in, respectively, the outer 2 and 1 dims. But?x?x4x5and1becomes?and1` as the single scalar is splatted with any (regardless of how many) dynamic dimensions...

The original test case is Roberta. I think a simple test case should be OK.

I was testing using Roberta-base-11 model but this also impacts resnet_50v1. @tungld and @AlexandreEichenberger please see this issue as well: https://github.com/onnx/onnx-mlir/issues/1571. Issue 1571 has been open since August of last year with the exact same error message.

#2161 fixes the issues listed here; please double check that it works for you as well.

@AlexandreEichenberger could you add the lit test here into #2161 to make sure it works?
By the way, in #2161, hasNoBroadcasting is still confusing, I don't see how it solves the semantics issue @chentong319 mentioned here. For example, hasNoBroadcasting still returns true for the case %0 = "onnx.Add"(%arg0, %arg1) : (tensor<1x1x5xf32>, tensor<5xf32>) -> tensor<1x1x5xf32>, doesn't it?

#2161 fixes the issues listed here; please double check that it works for you as well.

Okay I can test it now

Using code in PR #2161
func.func @add_1_1(%arg0 : tensor<1x1x5xf32>, %arg1: tensor<5xf32>) -> tensor<1x1x5xf32> {
 %0 = "onnx.Add"(%arg0, %arg1) : (tensor<1x1x5xf32>, tensor<5xf32>) -> tensor<1x1x5xf32>
 return %0 : tensor<1x1x5xf32>
}

get lowered to
  func.func @add_1_1(%arg0: memref<1x1x5xf32>, %arg1: memref<5xf32>) -> memref<1x1x5xf32> attributes {llvm.emit_c_interface} {
    %alloc = memref.alloc() {alignment = 16 : i64} : memref<1x1x5xf32>
    affine.for %arg2 = 0 to 1 {
      affine.for %arg3 = 0 to 1 {
        affine.for %arg4 = 0 to 5 {
          %0 = affine.load %arg0[0, 0, %arg4] : memref<1x1x5xf32>
          %1 = affine.load %arg1[%arg4] : memref<5xf32>
          %2 = arith.addf %0, %1 : f32
          affine.store %2, %alloc[%arg2, %arg3, %arg4] : memref<1x1x5xf32>
        }
      }
    }
    return %alloc : memref<1x1x5xf32>
  }

you see the

3 dim load:  %0 = affine.load %arg0[0, 0, %arg4] : memref<1x1x5xf32> where 0 are directly inserted (new pattern)
1 dim load: %1 = affine.load %arg1[%arg4] : memref<5xf32>

I think that is exactly what is expected. Same example run in SIMD too.
There are several lit tests checking for this pattern in the new PR too.

I also need to set the record straight about broadcasting and hasNoBroadcast() function.
To recap, here is what numpy say about broadcasting:

###General Broadcasting Rules
When operating on two arrays, NumPy compares their shapes element-wise. It starts with the trailing (i.e. rightmost) dimension and works its way left. Two dimensions are compatible when

they are equal, or
one of them is 1.

If these conditions are not met, a ValueError: operands could not be broadcast together exception is thrown, indicating that the arrays have incompatible shapes.
Input arrays do not need to have the same number of dimensions. The resulting array will have the same number of dimensions as the input array with the greatest number of dimensions, where the size of each dimension is the largest size of the corresponding dimension among the input arrays. Note that missing dimensions are assumed to have size one.
https://numpy.org/doc/stable/user/basics.broadcasting.html#general-broadcasting-rules
ONNX itself fully agrees with NumPy

In ONNX, a set of tensors are multidirectional broadcastable to the same shape if one of the following is true:

The tensors all have exactly the same shape.
The tensors all have the same number of dimensions and the length of each dimensions is either a common length or 1.
The tensors that have too few dimensions can have their shapes prepended with a dimension of length 1 to satisfy property 2.

https://github.com/onnx/onnx/blob/main/docs/Broadcasting.md
See rule 3, which will prepend 1 before trying to satisfy rules 1 or 2 for dimensions that are too short.
The hasNoBroadcast() is perfectly in line with both definitions.
Incidentally, the bug came from the getAccessFunction which took a shortcut when hasNoBroadcast() was true. The new PR #2161 fixes that (and some more).

I think all confusion comes from the term broadcasting and broadcastable. We think there is broadcasting when we broadcast a single value to multiple values. It means tensor<3x5xf32> and tensor<3x5xf32 is not broadcasting.
But ONNX defines broadcastable:
In ONNX, a set of tensors are multidirectional broadcastable to the same shape if one of the following is true:

The tensors all have exactly the same shape.
The tensors all have the same number of dimensions and the length of each dimensions is either a common length or 1.
The tensors that have too few dimensions can have their shapes prepended with a dimension of length 1 to satisfy property 2.

Looking Rule 1, tensor<3x5xf32 and tensor<3x5xf32> are broadcastable but for use there is no broadcasting.

Closing this since #2161 can fix the model.

This should have been fixed by #2161.