Variable DLC causes "ValueError: Short data" in message.decode()

Thanks for the awesome tool!
Certain messages have variable DLCs based on how much information is available, which differ from the DLC in the DBC file. It looks like this property is not handled in the db.decode() function. Everything is fine if the actual DLC is higher than the DBC DLC, but if the actual DLC is lower, the below error trace happens (tested with DBC-DLC: 2, actual-DLC: 1, also with DBC-DLC: 22, actual-DLC: 8).
Ideally, the return key-value dict would only include the signals that are decode-able from the actual data, but filling the unavailable data with their maximum decodable value to represent an error would also work if all the keys need to be present.
...
File "/usr/local/lib/python3.7/dist-packages/cantools/database/can/database.py", line 469, in decode_message
return message.decode(data, decode_choices, scaling)
File "/usr/local/lib/python3.7/dist-packages/cantools/database/can/message.py", line 853, in decode
return self._decode(self._codecs, data, decode_choices, scaling)
File "/usr/local/lib/python3.7/dist-packages/cantools/database/can/message.py", line 736, in _decode
scaling)
File "/usr/local/lib/python3.7/dist-packages/cantools/database/utils.py", line 107, in decode_data
unpacked = formats.big_endian.unpack(bytes(data))
ValueError: Short data.

@frysyz - Are you able to add a test case for this?
For example, there's already a test for DLC of 0. Note a simple DBC file was added here:
https://github.com/cantools/cantools/blob/c70aebf898c4d4dcbd95ebe57710d2ab1a43f46a/tests/files/dbc/message-dlc-zero.dbc#L1-L12
And then a test was added here, asserting the encoding and decoding works as expected:
https://github.com/cantools/cantools/blob/c70aebf898c4d4dcbd95ebe57710d2ab1a43f46a/tests/test_database.py#L5504-L5508
If you can add a test with a specific DBC file and desired functionality of decode, that helps ensure any fix covers your use case.
I'm not sure how this should be fixed, I think you would need to update this function:
https://github.com/cantools/cantools/blob/c70aebf898c4d4dcbd95ebe57710d2ab1a43f46a/cantools/database/utils.py#L119
This could be a significant change from what I can tell, but would love someone who is more familiar with the bistruct module and how this part of cantools works to opine.

This could be a significant change from what I can tell, but would love someone who is more familiar with the bistruct module and how this part of cantools works to opine.

I don't think that variable-length DLCs are intended with normal signal frames (in contrast, container frames only make sense with a variable DLC), but I agree that having a "non-strict decoding" feature would be quite useful because sometimes ECUs might save a few bytes in certain scenarios (e.g. if multiplexer frames are involved). I'll have a look, but it might take a while...

@andlaus One example where this happens is in J1939, if an ECU has no active trouble codes, it can send a regular 8 byte frame of the diagnostic message. But if there are multiple trouble codes, it will send the diagnostic message with >8 bytes in multiple frames, using J1939 transport protocol. Since the DBC file usually covers the >8 byte message, the db.decode() function crashes when it sees the single frame version of the message.
@samc1213 Info below. Running the test cases, I get the same short data error only when the DBC length is higher than the actual data. For a janky workaround, I have been checking the length of the message and padding out the raw bytes to match the DBC length before decoding.
db.get_message_by_name('TestMessage1').length
This produces fake data rather than the desired behavior of only returning the decode-able bytes. I'll have a closer look at the internal functions to see if there's a cleaner way to do it.
DBC:
VERSION ""
BS_:
BU_: Test

BO_<PHONE_NUMBER> TestMessage1: 3 Test
 SG_ Signal1 : 0|8@1+ (1,0) [0|0] ""  Test
 SG_ Signal2 : 8|8@1+ (1,0) [0|0] ""  Test
 SG_ Signal3 : 16|8@1+ (1,0) [0|0] ""  Test

BO_<PHONE_NUMBER> TestMessage2: 10 Test
 SG_ Signal4 : 0|15@1+ (1,0) [0|0] ""  Test
 SG_ Signal5 : 15|15@1+ (1,0) [0|0] ""  Test
 SG_ Signal6 : 30|15@1+ (1,0) [0|0] ""  Test
 SG_ Signal7 : 45|15@1+ (1,0) [0|0] ""  Test
 SG_ Signal8 : 60|15@1+ (1,0) [0|0] ""  Test

BA_DEF_ BO_  "VFrameFormat" INT 0 3;
BA_DEF_DEF_  "VFrameFormat" 3;

New test cases:
    def test_decode_dlc_variable(self):
        db = cantools.database.load_file('tests/files/dbc/variable_dlc.dbc')
        decoded_message1_full = {
            'Signal1': 1,
            'Signal2': 2,
            'Signal3': 3,
        }
        decoded_message1_limited = {
            'Signal1': 1,
            'Signal2': 2,
        }
        decoded_message2_full = {
            'Signal4': 4,
            'Signal5': 5,
            'Signal6': 6,
            'Signal7': 7,
            'Signal8': 8,
        }
        decoded_message2_limited = {
            'Signal4': 4,
            'Signal5': 5,
            'Signal6': 6,
            'Signal7': 7,
        }
        #message 1 (dlc 3 match, dbc lower, dbc higher)
        self.assertEqual(db.decode_message('TestMessage1', b'\x01\x02\x03'), decoded_message1_full)
        self.assertEqual(db.decode_message('TestMessage1', b'\x01\x02\x03\x04'), decoded_message1_full)
        self.assertEqual(db.decode_message('TestMessage1', b'\x01\x02'), decoded_message1_limited)
        #message 2 (dlc 10 match, dbc lower, dbc higher)
        self.assertEqual(db.decode_message('TestMessage2', b'\x04\x80\x02\x80\x01\xe0\x00\x80\x00\xf8'), decoded_message2_full)
        self.assertEqual(db.decode_message('TestMessage2', b'\x04\x80\x02\x80\x01\xe0\x00\x80\x00\xf8\x01'), decoded_message2_full)
        self.assertEqual(db.decode_message('TestMessage2', b'\x04\x80\x02\x80\x01\xe0\x00\x80\x00'), decoded_message2_limited)

@frysyz I think the transport layer of the J1939 device you are interacting with should always use a DLC of 8 and pad with 0xff to fill it up. If you do not have the SAE J1939/21 document, I can recommend this Autosar spec for details.
That said, cantools does not support J1939. If you require more complex protocol decoders, there might be alternatives available elsewhere.
But maybe @andlaus proposal would be sufficient to allow for some "DM1 light" support?

@andlaus @samc1213 Please let me know if I should create a pull request for the new test cases and dbc. Also let me know if the workaround I posted above is too janky, I can spend some more time looking for something cleaner.

I hope to get the bitstruct/bitstruct.c stuff required for this feature ready by the end of next week. I suppose your code above is a bit too hacky to include something like it in the master branch, but I suppose that I can possibly make a unit test  based on it.

@andlaus thanks again for looking into this. Feel free to use the DBC and test cases I posted above and let me know if I can help.


Make SESE loop tests to be independent of basic block numbering.

The SESE loop tests are brittle as they refer to basic block numbers.  The basic block numbering is not deterministic (as observed in a  recent CI). Changing tests to be independent of basic block numbers.

@swift-ci please test tensorflow


Damapper command fails

Hi,
thank you for dentist, it is an intriguing tool.
However, it is a bit annoying for me. I keep getting errors like the one below, without any useful information.
damapper: Command Failed:
              LAmerge  -a reference.reads.439<EMAIL_ADDRESS>
Or
LAmerge: Did not write all records to reference.reads.426 (8035)

damapper: Command Failed:
              LAmerge  -a reference.reads.426<EMAIL_ADDRESS>
I wondered if the tmp dir was too small, so I set that to a shared folder that can contain multiple terabytes, so I doubt that is the issue anymore.
Some partitions go through fine, however, it is only a handful before one throws an error.
Is there a way to get more useful debugging information? So that I can know how the commands fail and can address that issue?
Thank you.
Sincerely,
Ole

Hi Ole,
I am aware of this bug but have not found a solution, yet. Also, the author of damapper/daligner is informed about the bug.
I encountered this particular bug only recently, so I am positive it can be resolved by a certain combination of versions of damapper and LAmgere (contained in daligner package). I will probably find a quick fix until the end of this week and keep you updated.

So, the bug is introduced by a new feature in LAsort that removes duplicates automatically while sorting. Damapper will probaly need some adjustments.
For the time being, I resorted to using daligner as of 2020-01-15.

This should be documented in the dependencies section of the README.

Documentation added. I will keep this issue open as a reminder that this might get a proper fix sometime in the future.

Great,
thank you Arne.
Ole

In the dentist readme, daligner >=2020-07-27. I have daligner/2020.07.27 and dentist is still failing on:
damapper: Command Failed:
LAmerge  -a XX.raw_subreads.308<EMAIL_ADDRESS>Should I just use daligner version 2020-01-15?

@cassondranewman Sorry, for the late answer. Yes, please try the version 2020-01-15 of daligner.


Update to the spec for row-level deletes (WIP)

This PR tracks updates to the Iceberg spec for v2.

The changes LGTM. +1


[mesheryctl] system config eks fails to find aws cli in path

See mesheryctl Command Tracker for current status of commands.
Current Behavior
mesheryctl fails to find aws cli in path even though it is present.
Desired Behavior
mesheryctl system config eks should connect meshery successfully with the cluster.
Possible causes
This is very hacky and executes the command using sh which might not be having the aws cli on path and thus causing this error.
Screenshots/Logs

Environment

Host OS: WSL2
Meshery Server Version: stable-latest
Meshery Client Version: v0.5.49
Platform: EKS

Contributor Resources

mesheryctl Contributing Guide
Beginner's guide to contributing to Meshery and mesheryctl
mesheryctl Command Tracker
Meshery CLI Commands and Documentation
🙋🏾🙋🏼 Questions: Layer5 Discussion Forum and Layer5 Community Slack

Hi :), So like @DelusionalOptimist mentioned, are we looking to interact with the cloud services within mesheryctl using the SDKs for Go instead of running the shell commands? If yes, I could work on modifying the related code here and here to use the SDKs instead.

Thank you @krithikvaidya for showing interest here and also pinging me in slack.
Since you've told that you're willing to do it even after knowing about the prerequisites (an EKS), I'll assign this to you. :+1:

Thanks for assigning me. I'm having some issues with my MFA on AWS, so I'll get started once that's resolved :)


[P4_Symbolic] Clean up typos, unused code, and Buildifier warnings. Write P4-Symbolic IR golden test output to files instead of standard output. Change EvaluateSaiPipeline to depend on the generic parser instead of SAI hardcoded parser

Keyword Check:
~/sonic-buildimage/src/sonic-p4rt/sonic-pins$ ~/tools/keyword_checks.sh .
Keyword check Passed.
Build Result:
/sonic/src/sonic-p4rt/sonic-pins$ bazel build $BAZEL_BUILD_OPTS ...
INFO: Analyzed 637 targets (1 packages loaded, 30 targets configured).
INFO: Found 637 targets...
INFO: Elapsed time: 1.113s, Critical Path: 0.01s
INFO: 1 process: 1 internal.
INFO: Build completed successfully, 1 total action
Test Result:
/sonic/src/sonic-p4rt/sonic-pins$ bazel test $BAZEL_BUILD_OPTS --cache_test_results=no ...
INFO: Analyzed 637 targets (0 packages loaded, 7 targets configured).
INFO: Found 424 targets and 213 test targets...
INFO: Elapsed time: 185.218s, Critical Path: 118.50s
INFO: 267 processes: 322 linux-sandbox, 18 local.
INFO: Build completed successfully, 267 total actions
//dvaas:port_id_map_test                                                 PASSED in 1.0s
//dvaas:test_run_validation_golden_test                                  PASSED in 0.1s
//dvaas:test_run_validation_test                                         PASSED in 0.6s
//dvaas:test_run_validation_test_runner                                  PASSED in 0.0s
//dvaas:test_vector_stats_diff_test                                      PASSED in 0.1s
//dvaas:test_vector_stats_test                                           PASSED in 0.1s
//dvaas:test_vector_test                                                 PASSED in 0.7s
//dvaas:user_provided_packet_test_vector_diff_test                       PASSED in 0.1s
//dvaas:user_provided_packet_test_vector_test                            PASSED in 0.1s
//gutil:collections_test                                                 PASSED in 0.5s
//gutil:io_test                                                          PASSED in 0.6s
//gutil:proto_matchers_test                                              PASSED in 0.7s
//gutil:proto_ordering_test                                              PASSED in 0.6s
//gutil:proto_test                                                       PASSED in 0.6s
//p4rt_app/tests:resource_limits_test                                    PASSED in 2.9s
//p4rt_app/tests:response_path_test                                      PASSED in 6.5s
//p4rt_app/tests:role_test                                               PASSED in 1.6s
//p4rt_app/tests:state_verification_test                                 PASSED in 3.7s
//p4rt_app/tests:vrf_table_test                                          PASSED in 2.8s
//p4rt_app/tests/lib:app_db_entry_builder_test                           PASSED in 0.0s
//p4rt_app/utils:event_data_tracker_test                                 PASSED in 0.0s
//p4rt_app/utils:table_utility_test                                      PASSED in 0.7s
//sai_p4/instantiations/google:clos_stage_test                           PASSED in 0.6s
//sai_p4/instantiations/google:fabric_border_router_p4info_up_to_date_test PASSED in 0.0s
//sai_p4/instantiations/google:middleblock_p4info_up_to_date_test        PASSED in 0.0s
//sai_p4/instantiations/google:sai_nonstandard_platforms_build_test      PASSED in 0.0s
//sai_p4/instantiations/google:sai_nonstandard_platforms_cc_test         PASSED in 0.6s
//sai_p4/instantiations/google:sai_p4info_fetcher_test                   PASSED in 0.8s
//sai_p4/instantiations/google:sai_p4info_test                           PASSED in 1.2s
//sai_p4/instantiations/google:sai_pd_proto_test                         PASSED in 0.1s
//sai_p4/instantiations/google:sai_pd_util_test                          PASSED in 0.6s
//sai_p4/instantiations/google:tor_p4info_up_to_date_test                PASSED in 0.0s
//sai_p4/instantiations/google:union_p4info_up_to_date_test              PASSED in 0.1s
//sai_p4/instantiations/google:wbb_p4info_up_to_date_test                PASSED in 0.0s
//sai_p4/instantiations/google/test_tools:table_entry_generator_helper_test PASSED in 2.6s
//sai_p4/instantiations/google/test_tools:test_entries_test              PASSED in 1.0s
//sai_p4/instantiations/google/tests:p4_fuzzer_integration_test          PASSED in 3.6s
//sai_p4/tools:p4info_tools_test                                         PASSED in 2.0s
//sai_p4/tools:packetio_tools_test                                       PASSED in 0.8s
//tests:thinkit_gnmi_interface_util_tests                                PASSED in 1.1s
//tests/lib:p4info_helper_test                                           PASSED in 0.9s
//tests/lib:p4rt_fixed_table_programming_helper_test                     PASSED in 0.7s
//tests/lib:packet_generator_test                                        PASSED in 27.3s
//tests/lib:switch_test_setup_helpers_golden_test                        PASSED in 0.2s
//tests/lib:switch_test_setup_helpers_golden_test_runner                 PASSED in 0.1s
//tests/qos:gnmi_parsers_test                                            PASSED in 0.0s
//tests/qos:gnmi_parsers_test_runner                                     PASSED in 0.0s
//thinkit:bazel_test_environment_test                                    PASSED in 0.5s
//thinkit:generic_testbed_test                                           PASSED in 0.9s
//thinkit:mock_control_device_test                                       PASSED in 0.6s
//thinkit:mock_generic_testbed_test                                      PASSED in 0.7s
//thinkit:mock_mirror_testbed_test                                       PASSED in 0.7s
//thinkit:mock_ssh_client_test                                           PASSED in 0.0s
//thinkit:mock_switch_test                                               PASSED in 0.7s
//thinkit:mock_test_environment_test                                     PASSED in 0.1s
//thinkit:switch_test                                                    PASSED in 0.7s
//sai_p4/instantiations/google/tests:p4_constraints_integration_test     PASSED in 0.8s
Stats over 5 runs: max = 0.8s, min = 0.7s, avg = 0.8s, dev = 0.0s
//sai_p4/instantiations/google/test_tools:table_entry_generator_test     PASSED in 43.5s
Stats over 50 runs: max = 43.5s, min = 0.8s, avg = 4.6s, dev = 11.4s
Executed 213 out of 213 tests: 213 tests pass.
INFO: Build completed successfully, 267 total actions

/azp run

Co-authored-by: Srikishen Pondicherry Shanmugam<EMAIL_ADDRESS>

/azp run

/azp run

/azp run

@kishanps Kindly review the PR
Thanks

/azp run


Logging Log4J

A Factory to logging with Log4J.

There are a number changes here unrelated to logging.  Can you clean those up?  From what I can tell so far this looks fine but there's a lot of cruft to dig through.

Rather than adding support for log4j directly, why not use the slf4j/log4j bridge?  This is what the java driver does, too, and it's been working well for quite some time.

Given this PR's age and the existence of a workaround, I'm closing this for now.  It can be revisited should something else come up.


Create figma prototype for minimally functional proposal module (M2)

In order to complete Milestone 2 for Catalyst Voices, we must satisfy the following item:
“Deploy minimally functional proposal module to testnet for refinement over course of project”
As a user, I need to be able to view & edit the proposal template, so that I can submit my ideas for a chance to win funding. Once I have created a proposal, I expect to see it in the overall Proposal List, so that others can find my idea & provide feedback.
Screen1:

Basic site header
Empty list of proposals
“Add proposal” button

Screen2:

Proposal editor header
Clickable anchors on left navigation rail
Editable fields in the main working space

Title | String | 60char limit
Problem | String | 140char limit
Solution | String | 140char limit

Save button

Screen3:
Same as screen1
Proposal list contains the one just created
Acceptance criteria:
Screens created, reviewed, and handed off for development

Tasks to refine prototype after review with team:

Simplify splash to only the final panel with “impactful projects" header
Remove anchors & toggle from left navigation menu
Remove metadata section
When change status to publish, then return user to proposal list

Problem | String | 140char limit
Solution | String | 140char limit
Character limit is 200 for both.

Initial prototype completed & ready to pick up for future iterations on the flutter app. Will not be developed as part of Milestone 2, due to lack of flutter engineer capacity. Instead, M2 will be simplified to only include the splash screen as described in this ticket: https://github.com/input-output-hk/catalyst-voices/issues/274
@nielskijf @minikin @stevenj


12494: add new 'meta' section to ODK XML

to fix ODK Central importing:

tested and verified working after this change.
per the ODK XForms docs, this section was missing:
<model>
  <instance>
      <data id="asdf" orx:version="123">
          <question1/>
          <question2/>
+          <orx:meta>
+            <orx:instanceID/>
+          </orx:meta>
      </data>
  </instance>
</model>

note: other meta options include timeStart/timeEnd (automatic timestamps when the form entry was started/ended), deviceID, etc. if we wish to use them in the future.

Looks great to me, thanks Kevin!


role requires gtar on host

centos minimal install does not have gtar, so the following error is produced:
TASK [cloudalchemy.coredns : Unpack coredns binary] *******************************************************************************************************************************************************
fatal: [coredns -> localhost]: FAILED! => {"changed": false, "msg": "Failed to find handler for "/var/root/.ansible/tmp/ansible-tmp-1559537922.9-262484430401977/source". Make sure the required command to extract the file is installed. Command "/usr/bin/tar" detected as tar type bsd. GNU tar required. Command "/usr/bin/unzip" could not handle archive."}

Ansible host was macOS, which was missing gnu-tar.  'brew install gnu-tar' allowed things to install ok.


Regular problems with MDShare timeouts

I was trying to run through the notebooks again this morning from a fresh repo clone. I am having problems with md-share timeouts -- I have tried repeatedly the last 30 mins. I mentioned a similar problem some weeks back to @cwehmeyer when I first started looking at the notebooks. Are we confident that this solution will be robust in the wild?
TimeoutError: [Errno 60] Operation timed out

It is as robust as our IT departments servers are. So this is at least questionable. We could host the data elsewhere? Then we should decide upon a provider. Github LFS seems costly. Probably Amazon S3 is suited best for this. But we can also be brave and believe that issues of this kind are only temporary and rare.

We can also try a ZeDaT-hosted userpage for this.

I bet this is the same webserver.

No it is not. I'm good with changing that (at the end of the day, when more important stuff has been done).

It's a bit less convenient since we don't have a group account...

I do not know a solution to this without some compromises. Moving the data to a proprietary platform fx. github or figshare. Maybe dryad is an option? https://datadryad.org/ (open, pay to play)

In my opinion we could for now just rely on our server infrastructure. Servers in general are never up 100% - never.

Still, we should make mdshare more stable. I know that no server is available all the time, but I experienced problems on no less than three days in this week alone.

This seriously need to be resolved. The current solution is NOT OK in my opinion. There will countless emails coming our way if we do not do something about this now. FAO @franknoe

So is the issue the server availability or are there issues with mdshare itself?
How much data are we talking about? Any reason not to put this on a robust location like figshare and pulling things with mdshare from there? I believe the Pande people have also used figshare for some time.
Improving the IT situation is high up on my list, but this is not going to be a fast solution.

On 09.07.2018 23:07, Frank Noe wrote:

So is the issue the server availability or are there issues with mdshare itself?
It is about the server availability.
How much data are we talking about? Any reason not to put this on a robust location like figshare and pulling things with mdshare from there? I believe the Pande people have also used figshare for some time.
couple of hundreds of MB, so not that much.
Improving the IT situation is high up on my list, but this is not going to be a fast solution.

A third party option would be best I'd guess. However this will cause a
lot of traffic even for the small data set size.

The fastest fix would be to

hard-code some content, i.e., the tutorial file catalogue into mdshare. That way, we can bypass the need for an online (filename pattern) lookup after we already downloaded the data.
mirror the files on another ZeDaT server under a private user account.

I suggest to have two locations, the default location (e.g. FU) and the fallfack location (e.g. figshare). But we should then check availability of the first choice fast (i.e. not a super long timeout).
If we need to buy space somewhere tell me the place and the cost. A few 100 MB can't be very expensive.

On 09.07.2018 23:13, Frank Noe wrote:

I suggest to have two locations, the default location (e.g. FU) and the fallfack location (e.g. figshare). But we should then check availability of the first choice fast (i.e. not a super long timeout).
If we need to buy space somewhere tell me the place and the cost. A few 100 MB can't be very expensive.

The costs are caused by the traffic not the tiny storage capacity. I
like the idea of the fall back solution.

I think there are many options: I like the idea of Dryad, non-profit, research focused, citable DOI for the data. The only problem as I see it is that the data is static once uploaded, and additional data has to be separately submitted and paid for (ball park 100usd/submission as far as I can see). The most important aspect would be to have a platform which would work with the MDShare library as-is, so no major code changes need to be done. Does LiveCommsJ allow for supplementary information of this size? This could also be an option.

https://refubium.fu-berlin.de/

I think this is the solution we should go for. Who wants to register an
account/profile?

We can login with our zedat account.
Publishing data is allowed for FU members but it seems to come with a bit of overhead. I also have not yet explored how we can access parts of a dataset and there are no published datasets (only manuscripts) yet which I could use as an example.
What about a github repository for data only?


Modules

Logically organize code through the use of modules. A module is simply a file of Ghost code. Module's can define functions, classes, and variables to be shared and re-used in other files.
Modules are loaded through an import statement.
import MyModule

Started on this with #56


[Docs] Add page title

Closes #3225.

Nice! :tada:
Docs have really come a long way over the past month or so

I believe that we should update the package.json too.

I believe that we should update the package.json too.

Yup, I just tried this and ran into this problem :+1:

I just tried it though (installing react-title-component myself) and it works and looks good. :+1:
@mbrookes I think putting the Page subject before Material-UI is a good approach in terms of text getting cut off if the tab width is too small :+1:
When we make the package.json fix I think this is good to go :+1:

I believe that we should update the package.json

Ah, sorry, that was a commit with Title added to a single component that I  git reset --hard to. Forgot it was in there. I'll add it back.


ng2-inline-editor/dist/index"' has no exported member 'InlineEditorModule'

I'm using latest angular 4.1.0 as 5/5/2017.
I'm experiencing an error:
"ng2-inline-editor": "0.1.1-alpha.3"
Is anyone else having the same issue?

issue fixed after upgrading to "0.1.1-alpha.19"


.pyd files from Python

Did you try maybe a combine .dll and .pyd files as both in .exe?
Want to support us here with Your solution? :  https://github.com/pyinstaller/pyinstaller/issues/8572 ?

Never tried, but it looks interesting.
There's still no solution for in-memory execution?

Still not.. poor, right? : )

I've found PyOxidizer that seems to have features you're looking for.

It's very problematic solution - I know this. It require whole Rust, and producing big chains/elements (going in to GB's, eg. 2GB).
Soo.. pyinstaller need management from files to ram only.


XFA form saved using pdf.js cannot be re-opened with pdf.js - "Warning: XFA Foreground documents are not supported"

Attach (recommended) or Link to PDF file here:
CIE-XFA-work.pdf
Configuration:

Web browser and its version: chromium 90
Operating system and its version: Debian 11
PDF.js version: master@891f21fba6db64cd602c1a9a51826d7b9cd06af0
Is a browser extension: no

Steps to reproduce the problem:

Open the attached form in pdf.js
Fill out a field
Save/Download the form
Re-open the form

What is the expected behavior? (add screenshot)
The form should render again, displaying the field with filled in value.
What went wrong? (add screenshot)
The form only renders the first time. Saving it with at least one field filled and re-opening the saved PDF fails with
Warning: XFA Foreground documents are not supported
Additional info
Saving/Downloading the form without filling a field does not produce the error.
I have also tested other PDFs (e.g. canadian-xfa-example.pdf) that do not exhibit this problem.
From what I can see in the code, it would appear that saving the form somehow turns it from being pureXfa to losing that characteristic.
https://github.com/mozilla/pdf.js/blob/c68dc03be685a5f2de5c2e99595f9bc747ffaa34/web/app.js#L1572

The issue here is that saving it somehow corrupts the XFA data, causing even e.g. Adobe Reader to refuse to open because of XML errors. In PDF.js, when opening the re-saved document, the following warning is printed in the console:
Warning: XFA - Invalid utf-8 string.

Indeed, I initially thought that the UTF-8 warning would even appear when opening it for the first time (without any fields filled).
Note that I can open the version filled and saved by pdf.js correctly in masterpdfeditor though.

The saved xml contains some tag where the names have an accentued characters (é and à).
If I remove them from the pdf, everything is fine in pdf.js or acrobat.

The serialized xml is a js string (utf-16) and we must encode it into utf-8 before saving:
https://github.com/mozilla/pdf.js/blob/891f21fba6db64cd602c1a9a51826d7b9cd06af0/src/core/xfa/data.js#L78
@Snuffleupagus, I think this function:
https://github.com/mozilla/pdf.js/blob/891f21fba6db64cd602c1a9a51826d7b9cd06af0/src/shared/util.js#L1017
should be called utf8StringToString and the following one stringToUTF8String or am I wrong ?


ModuleNotFoundError: No module named 'cv2' in PyCharm

Hi everyone, I get the following error in PyCharm Community 2021.1:
 import cv2
ModuleNotFoundError: No module named 'cv2'

Terminal inside PyCharm:

I'm using the latest version of opencv-python on Linux Pop!_OS 20.04, Python 3.8.
The package is installed under a virtual environment (venv) in PyCharm.
I tried some solution on various forums, but I have not solved yet.

Hi @alex27riva,
Probably, your pip is not connected with your venv. Pip is used from the system, not from venv.
Could you try to use pip as a python module?
Try this:
python -m pip install opencv-python
python -m pip list | grep "opencv"
python -c 'import cv2; print(cv2.__version__);

cd to the venv path and do what @VictorBebnev suggests using a command line. You'll probably also get a prompt to update pip. This workaround has fixed multiple issues I had with module installation in pycharm.

@alex27riva Have you tried the solution?

@alex27riva I close the issue as there is no response. Please reopen the ticket if the solution does not work and you have more details.

tema ne zakrita


Fix the compareTo method for PulsarResourceDescription

Describe the bug
A clear and concise description of what the bug is.
To Reproduce
Steps to reproduce the behavior:

Go to '...'
Click on '....'
Scroll down to '....'
See error

Expected behavior
A clear and concise description of what you expected to happen.
need to return zero if two resourceDescription match exactly
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):

OS: [e.g. iOS]

Additional context
Add any other context about the problem here.
I will fix for this。

The issue had no activity for 30 days, mark with Stale label.


Game automatically disconnecting/logging out on runelite in macOS Sequoia

Type
Crash or hang
Operating System
MacOS
OS Version
15.0
Bug description
After upgrading to Sequoia on 2 separate devices, the game is constantly automatically logging out, doing temporary hiccup disconnects (disconnects then reconnects before logging out entirely) and is entirely unstable. Game worked 100% fluidly until the upgrade.
This has been attempted on 3 separate types of wifi in 3 locations.
Also attempted to remove and reinstall, removed all cache repeatedly, and kept timers on to determine how long game lasted.
Even something as benign as fishing in catherby couldn't be handled consistently for more than a minute with disconnects. Similar issues were found in the grand exchange, Kourend, Fossil Island (it didn't seem to be affected by anything server-side).
OSRS default client seems fine.
SPECS:
M3 Pro 14" Macbook Pro
18GB of RAM
Screenshots or videos
No response
RuneLite version
Runelite: <IP_ADDRESS>
Launcher: 1.2.0
Logs
client.log
launcher.log

Reinstalled macOS on both devices and it seems to have worked just fine

Reinstalled macOS on both devices and seems to be working just fine now

No way reinstalling macOS is the fix? That's something I cannot do on my laptop. Any other suggestions?

Sorry, it incidentally only fixes it temporarily. It's worse than that, it seems it's a Sequoia issue--to the point whole universities and companies are preventing people from upgrading:
https://techcrunch.com/2024/09/19/apples-new-macos-sequoia-update-is-breaking-some-cybersecurity-tools/
https://www.forbes.com/sites/ariannajohnson/2024/09/20/macos-sequoia-update-why-some-organizations-are-advising-against-updating-devices/
https://discussions.apple.com/thread/255761318?sortBy=rank
People are having insane internet issues and connectivity ones, particularly those that have any security tools installed, such as a VPN or other things. I think until this is resolved on Apple's end this will continue. Even those without some tools are having issues.


[Client]: Change responsive sidebar behaviour

https://www.figma.com/design/mVEbXXryqv2oBxMcNg8yjC/Anvil-AI?node-id=1055-7955&node-type=canvas&t=KKR5baVZ1kFpPacV-0

Closing as done.


Fixing hostTags nil in heade.tmpl

What does this PR do?
If hostTags is empty, the whole template will be broken.
Also, the Collector section had too much space:
before:
    hostname provider: aws
    unused hostname providers:
      configuration/environment: hostname is empty
      gce: unable to retrieve hostname from GCE: status code 404 trying to GET http://<IP_ADDRESS>/computeMetadata/v1/instance/hostname

=========
Collector
=========

  Running Checks
  ==============

    consul (1.9.1)
    --------------

after:
    hostname provider: aws
    unused hostname providers:
      configuration/environment: hostname is empty
      gce: unable to retrieve hostname from GCE: status code 404 trying to GET http://<IP_ADDRESS>/computeMetadata/v1/instance/hostname

=========
Collector
=========

  Running Checks
  ==============

    consul (1.9.1)
    --------------

For the hostTags:
When present:
    socket-fqdn: ip-10-128-35-134.ec2.internal.
    socket-hostname: ip-10-128-35-134
    host tags:
      datacenter:us1.staging.dog
      env:staging
      site:datad0g.com
    hostname provider: aws

(no change)
When missing, before:
root@datadog-cluster-agentXXXqdj9:/# agent status
Getting the status from the agent.
template: header.tmpl:59:13: executing "header.tmpl" at <len .hostTags>: error calling len: len of untyped nil
[...]
    socket-hostname: datadog-clXCXX2qdj9=========
Collector
=========

After:
 Hostnames
  =========
    host_aliases: [gke-charly-default-pool-XCXXX]
    hostname: gke-charly-default-pool-XXXX
    socket-fqdn: datadog-cluster-agent-CXXXX
    socket-hostname: datadog-cluster-agentXXX
    hostname provider: gce
    unused hostname providers:
      configuration/environment: hostname is empty

  Leader Election
  ===============
    Leader Election Status:  Running
[...]

Motivation
What inspired you to submit this pull request?
Additional Notes
Anything else we should know when reviewing?

Updated the Description of the PR @truthbk - Thanks for the quick 👀


Other backup type

Create format .backup

I've just added a few options, I think you might want to use ENABLE_CUSTOM_BACKUPS=yes.


[ React Migration ] Create SOS Report page

Hi there,
This issue is open to anyone who wants to contribute and is specific to the "reactdev" branch, where we are currently working on migrating the project to React.
The request is to create the SOS Report page, ensuring it follows the same pattern and design as the main project. It's important to maintain consistency with the project's structure. Additionally, if possible, we would like tests to be written using vitest in order to achieve 100% test coverage.
If you have any questions or need further clarification, feel free to ask. We appreciate your willingness to help with this task.

I would like to work on this issue.
Please assign this issue to me.

@Vidip-Ghosh Thank you! You have been assigned to the issue.

@ramonbs
Can you please add GSSOC'23 label to it??

@ramonbs
Almost some work has been done on this. Like if I want to push these changes after completing, how can I do since I created a new react project>


ImportError: No module named certificates

Hi, we are getting No module named certificates when using this lib.
Our script is currently running on an ubuntu box with python Python 3.8.0.
Distributor ID: Ubuntu
Description:    Ubuntu 20.04.1 LTS
Release:        20.04
Codename:       focal

To Reproduce

python3 -m venv .venv
source .venv/bin/activate
python
import censys.certificates <-- Error ModuleNotFoundError: No module named 'censys.certificates'

pip show censys
Name: censys
Version: 2.0.0
Summary: An easy-to-use and lightweight API wrapper for Censys APIs (censys.io).
Home-page: UNKNOWN
Author: Censys Team
Author-email<EMAIL_ADDRESS>License: Apache License, Version 2.0
Location: /home/tony/expanse/.venv/lib/python3.8/site-packages
Requires: requests, backoff
Required-by:

We also check previous issues related to import errors.
Our python script is called censys_subdomain.py and we also removed pyc files as suggested here
We are following the guide provided in your docs at https://censys.io/api.

Hi @blueteamzone we are working on updating our docs on the new search site search.censys.io. In the mean time the most up to date documentation can be found on ReadTheDocs. I have linked to an example import statement. Many of the pats have changed as we have upgraded to a namespace package for more advanced functionality. If you have any more questions please feel free to reach out or create a discussion post.

Hi @blueteamzone I just updated the examples on our site. For more examples you can also see our examples. For now I am going to close this issue.


brink remonvent sky 300

i have brink renovent sky 300. can change only modes not fan speed. dont see filter message.
shows like this :

Hey,
Can you turn on and off fan? Is this the only entity you see?
Because you can control fan speed with fan entity, when you would click on it

I just installed it for the renovent sky 300 and it works.
Example card, with percentages 0,33,66&100:
type: vertical-stack
cards:

type: horizontal-stack
cards:

type: custom:button-card
entity: select.homemodule_operating_mode
name: Mode
show_state: true
styles:
name:
- font-size: 12px
- font-weight: 600
state:
- font-size: 12px
- font-weight: 300
icon:
- width: 50px
- height: 50px
- border-radius: 100px
- padding: 5px
- margin: 5px
- opacity: 0.7
type: custom:button-card
entity: binary_sensor.homemodule_status_filter_message
theme: waves
name: Filter
show_state: true
styles:
name:
- font-size: 12px
- font-weight: 600
state:
- font-size: 12px
- font-weight: 300
icon:
- width: 50px
- height: 50px
- border-radius: 100px
- padding: 5px
- margin: 5px
- opacity: 0.7

type: custom:mushroom-fan-card
entity: fan.homemodule_ventilation_mode
show_percentage_control: true
show_oscillate_control: false
icon_animation: true
fill_container: false


HEC-1426 CTUTR attempts blocking functionality

Details in ticket -> https://jira.tools.tax.service.gov.uk/browse/HEC-1426

@AndyHWChung @priyankadhiman3006  I've made the following changes based on comments from yesterday

replaced CtutrAttempts.lastUpdated with blockedUntil that gets populated only when the max allowed attempts is reached. Once this field is populated, the value in ctutr attempts field is irrelevant.
instead of injecting AppConfig into CtutrAttemptsServiceImpl, I've taken Andy's suggestion and injected Configuration instead allowing us to revert to the original simpler state of AppConfig
replaced CompanyHECSession.ctutrAttempts with crnBlocked boolean field. We need a way to understand when a CRN is blocked within the JourneyService so this information needs to be in the session. But a simple boolean indicating whether or not the CRN in the answers is blocked should be sufficient.

Let me know your thoughts.

I think there is a bug in there, go through company journey, enter ctutr wrong two times,
enter it correct 3rd time. go to CYA page. click on change link against ctutr. it still allow you to enter wrong ctutr three more times.
As discussed earlier, We are planning to remove that change link against ctutr  from CYA page, @AndyHWChung should we consider this as bug or ignore it


fix(eraser): erase data in static connector before to share new data

:tada: This PR is included in version 1.8.2 :tada:
The release is available on:

GitHub release
crate.io packages (1 packages published)

Your semantic-release bot :package::rocket:


docs: add v0.3.7 release notes to the CHANGELOG

Description
Add the v0.3.7 release notes to the CHANGELOG.
References

Review Checklist

[ ] I have clicked on "allow edits by maintainers".
[ ] I have added documentation for new/changed functionality in this PR or in a PR to openfga.dev [Provide a link to any relevant PRs in the references section above]
[ ] The correct base branch is being used, if not main
[ ] I have added tests to validate that the change in functionality is working as expected

Codecov Report
Base: 84.22% // Head: 84.24% // Increases project coverage by +0.01% :tada:

Coverage data is based on head (cee131f) compared to base (a383e03).
Patch has no changes to coverable lines.

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #567      +/-   ##
==========================================
+ Coverage   84.22%   84.24%   +0.01%
==========================================
  Files          86       86
  Lines       13472    13472
==========================================
+ Hits        11347    11349       +2
+ Misses       1745     1743       -2
  Partials      380      380

Impacted Files
Coverage Δ

cmd/run/run.go
61.63% <0.00%> (+0.40%)
:arrow_up:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


[HW]

Name: Rimon
Email<EMAIL_ADDRESS>Attach the homework step I screenshots below:

Attach the homework step II screenshots below:

Hi, well done! Enjoy your badge and see you on the third episode of the series this week!
p.s. Spirited Away is my favorite by Miyazaki.

Hi i think the badge is not yet issued as i could not find the link nor is it showing in my badgr account.
p.s. Spirited away and grave of the fireflies are my favourite too :D

Hey Rimon, great job! Congrats! Here is your badge! https://api.badgr.io/public/assertions/-L4NcDFsS7KxE1sicQ6LMA


support of or-functions

Probably not a clean code, I am learning both rust and rego languages ...
I tried to add support of mutiple definitions of a function

@eric-therond Thanks for the contribution. Can you sign off you commit?

The PR looks good to me. Are there other tests that you can think of?

@anakrish I signed-off the commit
I'am ok with the tests for now

@eric-therond Kindly address suggestions from @mingweishih

@eric-therond Can you squash you commits?
I will then merge the PR and then investigate fixing the return values in the case you pointed out.
Thanks for your contribution!


gather-class-name-change

Gather.js

Renamed class name from 'App' to 'Gather'

This issue was fixed in chebi97's pull request


update VisualStudio.gitignore file : ignore visual studio 2015 Roslyn…

… ignored Compiler engine temporary files

As of VS15 CTP6 these temporary files now live under the .vs directory, which we've already added (see line 26).


How can I access to the calibre deck?

Hi,
I have sent the request on the website https://asap.asu.edu/. And I haven't received any replies yet.
Anyone knows whom to reach out and I could request the calibre deck file?
Thank you.
Best regards,
Waxpple

Hi Waxpple,
Sorry for the late reply. Your email went to my spam folder. Can you please
email Prof. Lawrence Clark @.***) with subject line that
indicates that it is for Calibre deck?
On Wed, Mar 23, 2022 at 1:05 AM Waxpple @.***> wrote:

Hi,
I have sent the request on the website https://asap.asu.edu/. And I
haven't received any replies yet.
Anyone knows whom to reach out and I could request the calibre deck file?
Thank you.
Best regards,
Waxpple
—
Reply to this email directly, view it on GitHub
https://urldefense.com/v3/__https://github.com/vvashish/asap7_git/issues/2__;!!IKRxdwAv5BmarQ!J9vF-Ud2HOahmJeY0VWlOQat50hcsN0pGRkeLJsmg7QCzQV3szSZr3VE176vlx0Q8wUIBQ$,
or unsubscribe
https://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/AGRRK7SP5VU6F4KPLGP62KDVBLGEVANCNFSM5RNGGQUA__;!!IKRxdwAv5BmarQ!J9vF-Ud2HOahmJeY0VWlOQat50hcsN0pGRkeLJsmg7QCzQV3szSZr3VE176vlx1oXnM_OA$
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

--
Regards,
Vinay Vashishtha

Hi,
I am going to use asap7.lylvs in the calibre deck file. And I have also sent the request on the website https://asap.asu.edu/, but haven't received any replies. Could you please send this file to me? Thank you very much.
Best regards,
haifengchen

Hi,
I am going to use asap7.lylvs in the calibre deck file. And I have also sent the request on the website https://asap.asu.edu/, but haven't received any replies. Could you please send this file to me? My email address is<EMAIL_ADDRESS>Thank you very much.
Best regards,
haifengchen


Unable to use truemail for ruby 2.3.1

I am trying to use truemail in the project with ruby version 2.3.1,
error: Bundler could not find compatible versions for gem "ruby":
In Gemfile:
ruby
truemail (= 1.0.0) was resolved to 1.0.0, which depends on
ruby (>= 2.5.0)
Is there any version of truemail that supports ruby 2.3.1?

@bestwebua any updates on supporting ruby 2.3? I am happy to help in any way.

Hi!
The hardcoded ruby version to 2.5 is for some specific reason?
I have a bunch of rails apps running with ruby 2.4 that could benefit from using this gem and I was considering forking just to drop the version and see if it works.


Query algebrizer initial work

Ready for review, @ncalexan. @victorporof, if you have time that would also be great.

I've addressed most of the comments; will land with those and then file issues or address the rest.

5af7082165061f530dd77f14a13196230cce5c34


Unable to attach to pid

Commands I tried to far:
a) echo "hi" ; sleep 200; echo "hi"
b) ping "foo.com"
c) irssi
Version: 0.5 on x64.
Tested on zsh + bash.
Steps to reproduce
$ ctrl+z                     # Suspend the current process
(I also tried to use bg here)
(optional: $ jobs -l      # get the PID )
$ disown "%1"           # Detach irssi from the shell

start a new terminal:
$ reptyr  <PID>  # Get back the process

Unfortunately I get the following error:
[+] Checking for problematic process group members...
[+] Allocated scratch page: 7f011f20d000
[+] Looking up fds for tty in child.
[+] Resolved child tty: 880a
[+] Found an alias for the tty: 0
[+] Found an alias for the tty: 1
[+] Found an alias for the tty: 2
[-] Unable to open the tty in the child.
Unable to attach to pid 15021: Permission denied

echo 0 > /proc/sys/kernel/yama/ptrace_scope not work in a docker container
Seems ptrace and strace not work due to security features
https://github.com/moby/moby/issues/20064
https://github.com/moby/moby/issues/21051


SMTP errors after configurtion

Home Assistant release with the issue: 0.72.0
Operating environment (Hass.io/Docker/Windows/etc.): RP3 - regular instaltion
Component/platform: smtp
Description of problem:
Errors during restarting HA after settings smtp for google, smtp componet is not working
Problem-relevant configuration.yaml entries and (fill out even if it seems unimportant):
name: sendemail
platform: smtp
server: smtp.gmail.com
port: 587
timeout: 15
sender<EMAIL_ADDRESS>starttls: true
username<EMAIL_ADDRESS>password: password
recipient:
-<EMAIL_ADDRESS>sender_name: Home Assistant

Traceback (if applicable):
2018-06-25 19:10:10 ERROR (MainThread) [homeassistant.components.notify] Error setting up platform smtp
Traceback (most recent call last):
File “/srv/homeassistant/lib/python3.6/site-packages/homeassistant/components/notify/init.py”, line 95, in async_setup_platform
platform.get_service, hass, p_config, discovery_info)
File “/usr/local/lib/python3.6/concurrent/futures/thread.py”, line 56, in run
result = self.fn(*self.args, **self.kwargs)
File “/srv/homeassistant/lib/python3.6/site-packages/homeassistant/components/notify/smtp.py”, line 75, in get_service
if mail_service.connection_is_valid():
File “/srv/homeassistant/lib/python3.6/site-packages/homeassistant/components/notify/smtp.py”, line 120, in connection_is_valid
server = self.connect()
File “/srv/homeassistant/lib/python3.6/site-packages/homeassistant/components/notify/smtp.py”, line 106, in connect
self._server, self._port, timeout=self._timeout)
File “/usr/local/lib/python3.6/smtplib.py”, line 251, in init
(code, msg) = self.connect(host, port)
File “/usr/local/lib/python3.6/smtplib.py”, line 336, in connect
self.sock = self._get_socket(host, port, self.timeout)
File “/usr/local/lib/python3.6/smtplib.py”, line 307, in _get_socket
self.source_address)
File “/usr/local/lib/python3.6/socket.py”, line 724, in create_connection
raise err
File “/usr/local/lib/python3.6/socket.py”, line 713, in create_connection
sock.connect(sa)
OSError: [Errno 99] Cannot assign requested address

Additional information:
DNS:
nslookup smtp.gmail.com
Server: <IP_ADDRESS>
Address: <IP_ADDRESS>#53
Non-authoritative answer:
smtp.gmail.com canonical name = gmail-smtp-msa.l.google.com.
Name: gmail-smtp-msa.l.google.com
Address: <IP_ADDRESS>
Name: gmail-smtp-msa.l.google.com
Address: <IP_ADDRESS>
** Allow less secure apps at google=TRUE

This is not a bug!
For anyone else encontering problem with SMTP in case it was related to the firewall.
After installing firestarter and stopping the firewall the problem seems to be gone.
Still needs to verify which port should be opened (587?)


BUG: Dots disappear/reappear in Clusters

iPhone model: 13 mini
iOS version: 17.6.1
App version: 1.2.1
When playing Clusters, the dots on the pre-identified blocks with only one neighbor of the same color sometimes disappear after you place a block of the wrong color next to them. It seems that the logic that marks the (dotted) block as invalid & draws a red border on the dotted block is incorrectly clearing out the dot when the invalid state is resolved.
This is probably an upstream issue, but I haven't built/played the x-sheep puzzles so I can't say for sure.
Steps to reproduce (should be pretty reliable):

Start a small game of Clusters. I use size 5x5.
Pick a red block with a dot on it, ideally in a corner.
Make all of the boxes adjacent to the red dot block blue via single-tapping. The red block should get a red "invalid" border.
Tap one of the blue boxes to make it red. The dot on the red center block disappears.

This also works with blue boxes, although it seems slightly less consistent than with red boxes.

Thanks - I can indeed reproduce this with your steps!
Similar to #11 this is likely something I'll need to fix upstream - I'll take a look!

Issue fixed in 1.3 and released to app store. Closing as complete!


Multiple inheritance in the native component is ignored with RnWinRT.

A native component I'm trying to reuse for React Native migration has multiple inheritance. A simplified code is like the following.
public interface INative1
{
    void method1();
}
public interface INative2: public INative1
{
    void method2();
}
public class Native: public INative2 {}

With this setup, only method2() is projected to JavaScript, and method1() is unavailable. The following code is not traversing up the inheritance chain but only looking through the direct parents. ☹
react-native-winrt-main\rnwinrt\rnwinrt\react\strings\base.cpp

projected_object_instance::projected_object_instance(const winrt::IInspectable& instance) : m_instance(instance)
{
    auto iids = winrt::get_interfaces(m_instance); <<<< Getting direct parents.
    for (auto&& iid : iids)
    {
        if (auto iface = find_interface(iid))
        {
            m_interfaces.push_back(iface); <<<< Not pushing back the ancestors of the direct parent interface.
        }
    }
}

For a quick reference on behavior and functionality, interfaces have two important functions in WinRT that are relevant here: QueryInterface and GetIids. QueryInterface is how you get a pointer to a particular interface that a type supports, and GetIids is how a type "advertises" which interfaces it supports. In general, these two sets of interfaces - which Interfaces/IIDs QueryInterface supports and returns success/a valid pointer back for, as well as which IIDs are included in the list returned from GetIids - should be the same (there's an exception where interfaces can support an interface, but not advertise it; these are typically referred to as "cloaked" interfaces, but this is not really relevant to this discussion). WinRT does not have proper interface inheritance. Therefore, although your idl defines INative2 as deriving from INative1, it really doesn't and instead only derives from IInspectable. Therefore, in order to call functions on the INative1 interface, consuming code would need to call QueryInterface with INative1's IID, get back a pointer to the interface, and then call the function.
RnWinRT doesn't really deal with types in this way - the developer does not indicate which interface it wishes to invoke a particular function on. Instead, it relies on the GetIids function to get a list of supported interfaces on a per-object basis. Therefore, if you are seeing that you are able to invoke functions on INative1 in non-JavaScript code (i.e. the code calls QueryInterface, which succeeds), but are unable to invoke the same functions on the same objects using RnWinRT (i.e. GetIids does not advertise the interface), then there's a bug in your component. The type's QueryInterface method responds to both INative1 as well as INative2, whereas its GetIids only advertises INative2.
That all said, interface requirements (which may "appear" to be like inheritance in idl definitions) do persist into the metadata (and I'm now slightly curious how Chakra handled this situation). So, in theory, we could modify each interface definition to have an additional array of required interfaces and then merge those into the m_interfaces array when constructing an object instance. I'm not a fan of that, however. This means more data, which means more binary bloat, and more runtime overhead to either de-duplicate interfaces on construction of an object interface, or to just add them all and then redundantly enumerate the same interfaces when looking for functions/properties. It seems much simpler and much more desirable for types to just behave as they are supposed to and advertise all interfaces that they support.


Add Existing Module From Another Tab Relocates All Current Tab Modules to Same Pane

Description of bug
When adding an existing module from another tab to the current tab, all of the current tab's modules move to the pane selected for the copied module.
Steps to reproduce

Create a new tab.
Add some new test modules to a couple of different panes.
Add an existing module from another page to the current page, making sure to uncheck the "Make a copy" box.
After the module is added, notice that ALL existing modules in different panes have been relocated to the pane of the copied module.

Current result
All existing modules move to the target pane of the copied module.
Expected result
Existing modules should remain in the same pane they were before adding the existing module
Affected version

[x] 9.2.1
[x] 9.2.2

Affected browser

[ ] Chrome
[x] Firefox
[ ] Safari
[ ] Internet Explorer
[ ] Edge

Can anyone else confirm or acknowledge? This issue cripples building out sites with even only a couple of modules...
I had to create my own quick and dirty razor module to work around this...
BIGideas.CopyExistingModule.zip

please verify that the issue still persists in DNN 9.2.2, thank you

Thank you for the follow-up, Sebastian. Yes, I can confirm that the issue persists in DNN v9.2.2

Thanks for the heads up, I'll try to reproduce.
Unfortunately, there is no "severity" level. like MS added to Win10 issues recently

I can reproduce on Evoq Engage 9.2.0.

I can reproduce on a site upgraded from 9.1.1 to 9.2.2

And I can confirm it does not happen on a clean 9.2.2 install too.

This also is happening on new install:
PRODUCT: DNN Platform
VERSION: 9.2.0
BROWSER:
Google Chrome
Version 70.0.3538.77 (Official Build) (64-bit)

@ohine @mitchelsellers I think it would be important to make sure this is taken care of for 9.3.0, can one of you guys add this to the milestone?

Maybe related to #2163 , just writing this link here to check on the other issue when this one is resolved.
It also happens on localized sites that use module references much more often that other users, it is a nightmare right now! I will try to take a look into the cause of this soon, if you have any clue of what changed, let me know...

I am also experiencing this error and may not be able to wait until it is fixed by the community though I lack the knowledge to fix it myself. Could someone please recommend a course of action to resolve this issue with a workaround. For example, would some type of downgrade be possible? If so, is there a guide somewhere on how to downgrade DNN?

@britchson there is no easy way to downgrade unfortunatelly, it is always a good idea to take a database backup and files backup before any upgrade for situations like this. That being said, this issue was tagged for a fix in Dnn 9.3.0 I am currently working on other parts of the next release but if nobody submits a pull request for this before I finish, it is definitely on my todo list. It is a big issue for me that have many localized sites.

@valadas Thank you for your response. Unfortunately, the site was created at 9.2.1, so the issue has always existed. I'll wait then.

Oddly enough, I am no longer able to reproduce, not sure why, tried enabling localization, different themes, etc. Not sure what makes it happen and not happen...

So far, I have one site that was upgraded from 9.1.1 to 9.2.2 with localization and another site that was built on directly with localization also, those two have the issue.
Trying to reproduce the same scenario 9.1.1>9.2.2 with localization and 9.2.2 new with localization and I can't make the issue happen again for some reason.
I tried with and without localization, with and without a module visible on all pages, on SQL 2012 up to 2017, with various cache and minification settings, with Chrome and Firefox, with and without ssl. If you guys get more ideas, let me know...

I've been experiencing this issue consistently on a clean 9.3.2 Platform install. I wasn't sure whether it was in issue with the skin or not, but it is exactly as described above.

@djfenske, the issue was fixed in a subsequent release, and no longer exists. I recommend you start with DNN v.9.4.1 at https://github.com/dnnsoftware/Dnn.Platform/releases

Thanks, Phil - that's good news. I may try an upgrade of this particular site to see if that corrects the issue for me;  what prompted me to look this up was that I didn't see any mention of it in the release notes for 9.4.1, and then discovered this still listed as an Open issue.

Awesome, I remember there was something near this issue merged in 9.4.x somewhere, I only had this very intermitently on a site or two (don't even remember which now). In the spirit of cleaning the backlog I am closing this issue because we have one positive confirmation of it being fixed.
If there are still issues with this using 9.4.1, please comment and I will be glad to re-open

Hi. I confirm the bug still exists in 9.4.1. Isn't it possible to just disable search of relative modules when the added module is linked to another existing module? Because most of the time, when adding an existing module as a ref, it will have a lower ID than new modules on the current page.

Here is the solution I've applied to work around this problem.
In ModuleDialog.js, I changed the setModuleId function as follow (I save in cooke and clear from cookie the value of _callFromExistingModule):
    setModuleId: function (moduleId) {
        if (moduleId <= 0) {
            this._removeCookie('CEM_NewModuleId');
            this._removeCookie('CEM_ExistingModule');
        } else {
            this._setCookie('CEM_NewModuleId', moduleId);
            this._setCookie('CEM_ExistingModule', this._callFromExistingModule);
        }
    },

In ContentEditor.js, I modified the window load event like this (I load the new cookie and add its value to apply method of moduleDialog):
        var cookieModuleId = moduleDialog._getCookie('CEM_CallbackData');
        var existingModule = moduleDialog._getCookie('CEM_ExistingModule');
        if (!existingModule) existingModule = false;

        if (cookieModuleId && cookieModuleId.indexOf('module-') > -1) {
            var moduleId = cookieModuleId.substr(7);

            var module = $('div.DnnModule-' + moduleId);
            var moduleManager = module.parent().data('dnnModuleManager');

            moduleDialog.apply(moduleManager, existingModule);
            moduleDialog._processModuleForDrag(module);
            $('#moduleActions-' + moduleId).addClass('floating');
            moduleDialog._removeCookie('CEM_CallbackData');
        } ...

As the value of callFromExistingModule is restored in ModuleDialog, function _processModuleForDrag ignores relative modules when the added module is an existing one.

I still get this issue in DNN9.4.1 ??

I will look into this.

thx Peter. Let me know if you need any help.

In the meantime, you can apply the fix above. It's just 2 js files to modify.

OK, thx, it's not a problem for me at the moment.  I was just confirming your findings on 9.4.1.

I have a client that has this issue, but on one of my test installs (9.3.2) I cannot reproduce it using these steps:
https://github.com/dnnsoftware/Dnn.Platform/issues/2289#issuecomment-456783898

https://github.com/dnnsoftware/Dnn.Platform/issues/3309
Here are the step I can reproduce in 9.4.1

Can you guys check 2 things please for triage:

Is this still reproducible on 9.4.2 (or 9.4.3 RC)
Is this issue and #2289 the same ? If so we could close one of them as a duplicate...

Just upgraded a test site. I can confirm that the problem still exists on 8.4.2.
Yes. Issue 3309 can be closed. It's the same bug.

Thank you.

I see this issue exists in 09.05.00 as well. I had thought it was fixed a couple releases ago?

We did not receive a PR with the proposed changes, @jacoch can you submit a pull request for your changes please ?

Well it was just a work around I used on my side. I thought it would be fixed by devs in another way. But sure, I can open a pull request.

Well it was just a work around I used on my side. I thought it would be fixed by devs in another way. But sure, I can open a pull request.

that would be really nice :-)


BUG stacking get_params bugfix for estimator_list=[]

Previously, stacking estimators with estimator_list=[] threw an IndexError when get_params(deep=True) was called. This PR fixes that bug.

We can block the release on this.

May as well, this is a pretty straightforward fix.


Randomly get current identifier as completion

To reproduce:

Type namespace
Type randomly. Every once in a while, the current text will be recommended as a completion. I've gotten this to reproduce with const also.

May be a vscode bug. @mjbvz Any idea how this could happen?

Moved the issue to vscode since tsserver correctly returns no completion info.


Generate Ruby DLP

@swcloud @neozwu ping
@lukesneeringer any idea why CI is not running?

@lukesneeringer any idea why CI is not running?

I turned off building on forks by accident. I turned it back on this morning.

@swcloud regenerated.


[release/1.6 backport] update runc binary to v1.1.13

backport https://github.com/containerd/containerd/pull/10334

full diff: https://github.com/opencontainers/runc/compare/v1.1.12...v1.1.13
Release notes:

If building with Go 1.22.x, make sure to use 1.22.4 or a later version.

Support go 1.22.4+.

runc list: fix race with runc delete.

Fix set nofile rlimit error.

libct/cg/fs: fix setting rt_period vs rt_runtime.

Fix a debug msg for user ns in nsexec.

script/*: fix gpg usage wrt keyboxd.

CI fixes and misc backports.

Fix codespell warnings.

Silence security false positives from golang/net.

libcontainer: allow containers to make apps think fips is enabled/disabled for testing.

allow overriding VERSION value in Makefile.

Vagrantfile.fedora: bump Fedora to 39.

ci/cirrus: rm centos stream 8.

(cherry picked from commit cb99d33cd9a62ae57adc0a1fe51941a345d60e24)

@dmcgowan looks like we need the Vagrant changes in the 1.6 branch as well 😞

Yep, backporting now

This one needs https://github.com/containerd/containerd/pull/10339 to make CI go green

rebased nowa that https://github.com/containerd/containerd/pull/10339 was merged

All green now, @dmcgowan


[TEVA-3236] Fix missing translation bug

Jira ticket URL
https://dfedigital.atlassian.net/browse/TEVA-3236

Curious as to why this didn't break any specs 😕


The "Hidden" checkbox in the "Add game" prompt is confusing

I understood afterwards that it was for non-game files such as BIOS files, but the info is kinda buried in the Github Readme page, and I remembered it only because I read about it on the Discord server.
I think that since it's not a game, its upload form should not be here.
I suggest to add an "Emulator settings" button somewhere in the emulator's menu itself. For example for PS1 http://gameros.local/platforms/ps1

The "Hidden" option is not necessarily meant for BIOS files. I use it all the time for ROM files.
It is meant as a way of managing a larger collection of ROMs. Sometimes you do not want all your games visible in Steam.
But instead of deleting and re-adding them, you can simply mark them as "Hidden".

Jump to...
Home
HTML Tutorial
Database Tutorial
MySQL Tutorial


Image creation error

Hello, I'm a VFX compositor.
Thank you so much for providing such good information.
I've finished installing the plug-in, and I'm trying to use the inpaint function, but as a result, the image has not been created.
May I ask you about the solution?

Hi @jdaezone , thanks for using the tool, I appreciate your feedback!
It's difficult to figure out the issue, could you please send some screenshots from the NukeDiffusion Terminal? It would be useful to check the information.
Just to make sure, is it working for the other workflows? (img2img/txt2img)

The image above is an inpainting test image, but when I tested it again, there was an error related to the mask.

The above image is a test image related to txt2img.

The above image is a test image related to img2img.

In conclusion, the image is created like this, so what is the solution?

Thanks for providing the screenshots!
Here are some questions...
1- When you opened this Issue, you mentioned that it was not working for the Inpainting workflow only. However, from these provided screenshots, it seems like the other workflows are not working either. Is that correct?;
2- Could you check the _input folder and see if Nuke rendered your input images? This is because the Inpainting workflow will render your inputs from the NukeDiffusion node (this is just to check if it is rendering them or not);
3- Could you try a different Checkpoint? Maybe using an SD model instead of SDXL;
4- Lastly, just to make sure, did you wait for the whole setup installation process until you see the "The NukeDiffusion setup has been completed!"? I know this looks obvious, but some people didn't wait for it, and some dependencies were not installed properly;

1,3. Yes, it doesn't work in all workflows (txt2img, img2img, Inpainting, SDXL/SD)
2. No additional images have been rendered in the input folder.
4. I re-installed Pytorch and Nuke Diffusion. Is it related to the image not being created? Please check the image below..

*Pytorch

*install_or_update_dependencies

Thanks for providing the answers!
Well, honestly, it's a bit weird because from your screenshots everything seems to be correct.
The only thing I could say is that I think you are using the older version of NukeDiffusion. I uploaded the v01.1 last month, and looking at your screenshots I see a different folder structure.
In the updated version, we don't have the cuda folder and the Python built-in anymore. I highly recommend that you download it and install it again (https://github.com/danilodelucio/NukeDiffusion).
If you do that, make sure to have a Python version 3.8-3.11 installed into your system. If you need help with any of these steps don't hesitate to ask!
I'm sorry for taking your time doing all this process, I'm trying my best. 🙁

Thank you so much for letting me know about the solution :)
I installed a new version and retested it, but the same error repeats...
Should I proceed with the test on another PC?

No worries! Yeah if possible it would be great to test it on another computer, because I don't know why it is not working for you (really sorry for that).
For all the information you provided, it does not look to be the tool I made, maybe it's a conflict with some Stable Diffusion dependencies.
If you installed the new version, could you confirm if the input images were rendered inside the _input folder? Also, could you confirm which version of Python you are using? (if you can send screenshots would be better)

After rendering the image (input), I proceeded to create the image, but the same problem occurred as before.
Python version is using version 3.11.9 as shown in the terminal...!


Bugfix for interdomain examples

Signed-off-by: Mikhail Avramenko<EMAIL_ADDRESS>Issue
needed to fix https://github.com/networkservicemesh/integration-interdomain-k8s/pull/80/checks?check_run_id=3817629543

@denis-tingaikin ok, done


enhancement: display job summaries when done

https://github.com/orgs/community/discussions/27649#discussioncomment-5258640 explains how to get the job summary for a finished job.  Watchgha could fetch those and show them in the final display.
https://github.com/<owner>/<repo>/actions/runs/<run>/jobs/<job>/summary_raw

There's no indication in the API that a summary exists.  The summary_raw endpoint returns 404 if there isn't.

Ick: the program has to be logged in to github.com, which is different than having a token for the API.


Installing USBDK may make the system unstable

This was reported on the GIMX forum:
https://forum.gimx.fr/viewtopic.php?f=5&t=1229
The PC is a Dell Latitude E5530 laptop, and the OS is Windows 7 64bit.
Issue first happened with USBDK 1.0.4. Unplugging the laptop from the docking station solved the issue. I think the docking station is model XX6F0, which interestingly has USB 3.0 ports, and most probably a USB 3.0 HUB.
After installing USBDK 1.0.6 the system became unstable again, and unplugging from the docking station doesn't help anymore.

Hi @matlo,
While I cannot reproduce this exact problem I have reasons to suspect that it was fixed by commit 4a0354fdcd2d9ba342199180c16a853534c52527 at version 1.0.7
Please ask the problem reporter to check if the issue still exists on version 1.0.7 (just released). In case it does please ask him to collect binary trace as described at https://github.com/daynix/UsbDk/blob/master/Documentation/Tracing.txt (the latest code base contains fixes for the binary tracing collection scripts problems observed on previous release).
Thanks,
Dmitry

Hi,
I've asked the user to test USBDK 1.0.7. Thanks!

Closing this issue for now. Please re-open in case this problem still exists.

Hi
I have tried to install USBDK for GIMX but it wants to restart my pc and after that all USB components like USB 3.0 GIGABIT Network adapter or USB 2.0 Hub stops work.
I can use GIMX without this USB components but I need them for work. So I had idea. I can install USBDK before I start GIMX and than I can uninstall USBDK for work - but I cannot uninstall it! When I try to uninstall USBDK it shows just "preparing to uninstall" but nothing happen. Second problem is I cannot turn off my pc. It still shows "Turning off" and nothing happen. I must kill power for turn off pc and than start in recovery mode and restore my computer to the old compilation.
Thank all for any idea.
Stu - (Dell Ispiron 7710 + Windows 7 pro x64)

Which UsbDk version are you using?

I have tried v1.00-11 at last but I had same problem with version included in GIMX.

Hi,
The latest GIMX installer provides USBDK 1.0.7.

Please collect binary trace for UsbDk 1.00-11 install and uninstall at described here:
https://github.com/daynix/UsbDk/blob/master/Documentation/Tracing.txt#L52

UsbDkTrace.zip
Here you are. Let me know it is what you need (I hope so).

UsbDkTrace.zip
Here is second file. Sorry the first I did in USBDK Master, is it all what you need?

So I found a solution for me. It isn't regular solution but it is usable.
My solution: If I tried to uninstall the USBDK library by the Controll Panel in Windows it freeze every time. I have just reed some posts and now I know that I can use the UsbDkController to install and uninstall the USBDK. I have created two .bat files to install and uninstall the USBDK and now I can do it every time if I need use the GIMX. If you find a regular solution for me let me know.
Tank you very much. :-)

Unfortunately the log is empty. Please make sure you did everything according to the manual and there were no errors during the process.

Ups. What I did wrong? Where I have to place the TOOLS directory?

Doesn't matter. Just make sure there are no errors in script log.

And the script is running when you reproduce the problem.

The UsbDK have to be instaled if I run the UsbDkTraceToFile.bat ?

No. You may run the script first and then install/uninstall UsbDk.

Also you may try to build UsbDk from master and collect the text log:
https://github.com/daynix/UsbDk/blob/master/Documentation/Tracing.txt#L21

Problem was between chair and screen. :-)

Trace.zip
There are two files. One with problematic USB HUB and one without it. But the USBDK required restart PC. Have I to restart it?

Yes, you have to restart. Does this happen on reboot only? What do you see in device manager after reboot? Do you see usb host controllers marked by exclamation sign? What is the error shown in device properties for the Usb host controller?

Ok, now log is not empty but my PDB does not match. What version is this?

I don't see the devices (in device manager) immediately after I install USBDK and I see them immediately after I uninstall USBDK. I don't know the version of USBDK. It's the version included in GIMX 5.3. Should I install USBDK 1.0.11?

Windows freezes in boot with USB devices plugged in after I install the USBDK and restart PC. I must unplug them for start PC.

Yes, please collect the same log with UsbDk 1.0.11

Here is log with UsbDk 1.0.11 with restart but I cannot uninstall it now by UsbDkController. :-(
I must restore my PC now.
UsbDkTrace_Install_with_restart.zip

Thanks, I'll take a look.

Unfortunately, the symbols of 1.0.11 were lost and we can decode the log only partially (we can see only 2 connected USB devices: BlueTooth and Camera)
If the problematic system is still available and the problem can be reproduced:

before installing UsbDk please use the UsbView tool ( https://drive.google.com/open?id=0B9gp3tZmgMhdR25XLVl5VnJMdlU ): run it, then use File-Save as txt and attach resulting file
From administartor command prompt run attached batch file to collect logging info of install of UsbDk 1.0.19 (rename it ro .BAT)
UsbDkLogman.txt
Try installing UsbDk 1.0.19 ( https://github.com/daynix/UsbDk/releases/tag/v1.00-19 ), let us know how it works. Stop the recording after attempt to install, zip and attach the recorded log.

Thanks in advance

Closing for now. Please reopen in case problem still exists.


badsongs.txt says that this song has an unsupported instrument even though I didn't change it in the Instrument category when exporting

When I exported POLYBIUS GB SPEEDRUN (Glitchless 100% WR in 0:03:57) to Clone Hero, it didn't show up, so I looked for the badsongs.txt file, to which it said there was an unsupported instrument in it. Thing was, I never changed the instrument, it was just Guitar by default and I never change it. So what happened? (For context, I'm on MSCE <IP_ADDRESS>, am on PTB v<IP_ADDRESS>75 though I doubt that's why it's happening, and even tried resetting my chart and reexporting it, but to no avail.)

Update: I went to MSCE <IP_ADDRESS> and when I exported it, it was fixed.


Kitchen Sink: Build A Calendar App

Build a calendar application with event creation and reminders that you'll add to the kitchen sink. Feel free to use any existing projects of yours or start one from scratch.
For more information about the Kitchen sink; check out:

https://sink.million.dev/
https://github.com/aidenybai/million/blob/main/packages/kitchen-sink/README.md

N.B: Upon PR review, your usage of Million.js is a plus

hey @tobySolutions, can I work on this issue?

hey @tobySolutions, Please assign to me?

Hey there @rjangid9080, @ATREAY requested first, so maybe we should let him have this one.

Hey @tobySolutions, I've added a calendar app. Could you please take a look when you have a moment?

Hey @ATREAY! Sure thing, gonna check it out.

Issue fixed by #642


Propagate timeout to manual redirects

We had an issue with a fetch where the server was generating an infinite number of redirects. In this case even having a timeout set, the fetch where never throwing but stuck in the loop. With this PR we are checking if there is a timeout in the options for a fetch and in a positive case we track the time spent for the fetch and modify the timeout according to it when making the recursion.

@javivelasco can you add tests and coverage analysis?


Include homebrew

Added instructions to use the homebrew package manager on OS X. There is an official formula which is also relatively uptodate: https://github.com/Homebrew/homebrew/blob/master/Library/Formula/kubernetes-cli.rb

Thank you!


The future of this project

Hello again 👋,
I'm curious in regards to your plans for typst.js @jcbhmr. From my understanding, currently it's only a wrapper around the typst binary distributed in npm. Are there any plans to go any further for e.g JS bindings? Like what is done by the typst.ts project. Or do your goals end here?

Yes, I have been thinking along those lines.
There are a few ideas for how this typst package could be used:

To auto-install the typst binary when a user runs npm install with dependencies having typst: "^1.0.0" in it.
To provide functions like compile() that do exec("typst", args) similar to how other CLI JS-API wrapper works (Docker, Pulumi, tar, apt, etc.)
To provide a WASM/browser build of Typst so that you can run typst.compile("my typst text") or something idk in your browser

All of these can be part of the same package.
The first one is relatively easy. Hence, it's the first one that I did.
The second is just tedious JS -> arg mapping.
and third is much tougher.
I've attempted and succeeded in compiling the typst-cli package to WASIX and getting it to run on my local PC with wasmer run typst.wasm https://github.com/jcbhmr/typst/actions but I'm still having trouble getting it to work with the Wasmer WASIX JavaScript browser API https://wasmerio.github.io/wasmer-js/
so that lead me to search "typst javascript" and you're absolutely right i did find typst.ts.
I've been trudging through the absolute monster of a codebase (there's like 10 rust crates and 10 npm packages in it lol) https://github.com/Myriad-Dreamin/typst.ts project to try and figure out how it was done! i dont need all that complexity; just enough to mirror the exposed cli API. particularly typst compile.
typst compile: needs all the compiler infra. very tough.
typst watch: idk if relevant in programmatic context? maybe. basically typst compile on loop tho.
typst query: very tough. needs compiler infra plus metadata serialization. ugh.
typst fonts: ezpz just use existing web fonts API in browser! https://developer.mozilla.org/en-US/docs/Web/API/CSS_Font_Loading_API https://developer.mozilla.org/en-US/docs/Web/API/Local_Font_Access_API
typst update: not relevant.
typst help: ezpz just print help text.
(this might be more for my benefit than yours)
so far  my understanding is this:

there's the main @myriaddreamin/typst.ts which depends on @myriaddreamin/typst-ts-web-compiler. That's the main juicy part
@myriaddreamin/typst-ts-web-compiler does the actual typst -> SVG magic
in packages/compiler/src/lib.rs there's a compile() function that takes in a path (ok weird; how are files emulated) and a format "vector" to provide some binary data (probably the SVG source text as utf8? idk)
then it uses some internal compiler magic (some complex indirection and dep injection 😨)
then that delegates to typst_ts_svg_exporter::SvgModuleExporter::default()
that's actually from the typst-ts-svg-exporter rust crate under core/exporter/svg/src/frontend/flat.rs
idk what magic lies in there
then that gets somehow export_module()-ed to bytes from the SvgDocument type
and tada! not complex at all /s

so to answer your question, yes, I have thought about it and yes I would like to add such functionality but I'm still a complete novice in WASM Rust world lol. the Node.js exec("typst", ...) stuff; that I can do. the wasm browser stuff; that's more of an experiment and I'll see how it goes.
any input or ideas or help or code or education or resources or wasm magic is much appreciated lol. ❤

Sounds great,
I'm curious to if you thought about merging step 1 and step 2 with the use of node-rs to bind to the typst-lib for cli/lib usage. For a more cohesive, direct approach without the headache of managing Installations. Not sure if typst-ts does this, but their exposure of a new typst-ts cli might lead me to believe it to be the case.
However, I have to ask if you've evaluated the need for typst.js in the first place with the existence of typst.ts. Existence can be always justified if there is meaning consider LaTeX js eco with various different implementations. But undesired fragmentation is not ideal.
I should mention that on a first glance, I have an upcoming project for typst that has a web-stage however I don't plan to use typst.ts due to certain concerns I've noted whilst exploring the project, might not be the case if they're addressed, but most.likely not.
For shipping typst to the web, we can discuss this later. I have a general idea regards to how it could be implemented, but we'll leave that for later.

I'm curious to if you thought about merging step 1 and step 2 with the use of (napi-rs/deno-ops) to bind to the typst crates for cli/lib usage. To allow for a more cohesive, direct approach without the headache of managing Installations.

Since napi-rs effectively builds platform-specific mylib.node files (one for Windows x64, one for Linux x64, etc.) you'd run into the same "distributing a big fat binary" problem again. Their solutions is to, guess what, do #3 🤣
so effectively: instead of one typst binary for each platform, you'd end up with:

recreating bindings for the cli interface through https://napi.rs to make sure arguments, env, etc. get forwarded to the main() rust function
figuring out how to publish all those fat typst.node things (they're actually just dll/so renamed .node)
now maintaining a weird nodejs-only binding in addition to the wasm browser binding

So I think that's a lot of work for not-so-much gain. Using exec("typst", ["compile",...]) is already what https://www.npmjs.com/package/docker-cli-js, https://github.com/minhhh/node-uncompress, etc. do: they wrap some fancy exec() calls in nice JS functions.
on another note lol: pulumi has a similar "distribute binaries as npm package" issue that might be related: https://github.com/pulumi/pulumi/issues/11744

However, I have to ask if you've evaluated the need for typst.js in the first place with the existence of typst.ts. To avoid unnecessary fragmentation of the community so early on. Existence can be always justified if you e.g have a different approach to the implementation consider LaTeX JS eco with the various different implementations.

"hey; what if i used wasix instead of trying to wrangle with wrappers?" <-- who knows how far that will get.
as of now the browser API is indeterminate. i have no idea. it could not exist! or it could be great! or it could just re-export typst.ts. or it could combine with typst.ts. who knows?!?
right now im just having a silly time trying to wrangle wasm and learn more about wasm/rust magic. typst seemed like a great project to do that since it compiles relatively cleanly to wasix using cargo wasix build with minimal mods.
so far I've:

gotten it to build reliably
published it to https://wasmer.io/typst-community/typst
redownloaded the raw .wasm file and done wasmer run ./typst.wasm --help and had it work
had my computer run out of memory when i do wasmer run ./typst.wasm compile test.typ
run the ./typst.wasm in the browser only to have the browser wasix runtime run out of memory and exit code 1. at least i think that's what it did. it was rather opaque and indeterminate.

point being dont worry about fragmentation. right now typst.js https://npm.im/typst is just a redist installable version of the plain old typst binary. that's ok for now.
bottom line: my next step is to wrap those cli commands in pretty js/ts typed functions so you get autocomplete for them lol
dont worry about browser wasm; im having fun experimenting with that. for now, use typst.ts as its the clear "already works" in-browser typst magic that works.

note to self: i was pointed to https://github.com/napi-rs/napi-rs/issues/796
wasm support seems to be on the horizon but not here until napi-rs v3? https://github.com/napi-rs/napi-rs/issues/1493

Since napi-rs effectively builds platform-specific mylib.node files (one for Windows x64, one for Linux x64, etc.) you'd run into the same "distributing a big fat binary" problem again. Their solutions is to, guess what, do #3 🤣

There's a huge difference between interfacing with a binary and library. Yes, you are distributing platform-specific software in all cases, however you miss the point of going the route of a library. When you wrap a cli you're essentially restricted with the interface, this is fine if of course the purpose is simple, you can just go with this unpleasant way of simply wrapping the cli. However, if you need granularity over the interface and reach over the underlying interface/functions which are not exposed, that is not the case. You'll have to either manually replicate these features in javascript, which is not going to end well, so you'll end up just wrapping the library anyways with napi.
Ah and did I mention that if you want to ship WASM you'll need to do it anyways 🙂? napi-rs supports targeting wasm-bindgen, so you can avoid rewriting the bindings. And It's partially supported by the bun runtime, and there is work on going to get it to work in deno.

recreating bindings for the cli interface through napi.rs to make sure arguments, env, etc. get forwarded to the main() rust function

You have to do it anyways with WASM, and mind my previous comments.

figuring out how to publish all those fat typst.node things (they're actually just dll/so renamed .node)
now maintaining a weird nodejs-only binding in addition to the wasm browser binding

Fat? You know that you'll have fine-grain control over the underlying library when wrapping it right? Compared to a fixed binary output that you have zero control over and is literally always going to be larger..

So I think that's a lot of work for not-so-much gain. Using exec("typst", ["compile",...]) is already what npmjs.com/package/docker-cli-js, minhhh/node-uncompress, etc. do: they wrap some fancy exec() calls in nice JS functions.

Think again

youre right a direct nodejs <=> rust binding would be more powerful. in the future, that might be what happens! and the idea to avoid binding a typst.node and a typst.exe by just using a main.js => typst.node for the binary instead of a separate typst.exe is a good one.
but im not there yet. ill get there. still working on it. right now it works great with the cli wrapper! s othat's where it's at now. now that I've achieved getting it to have mirror with all the typst --help commands available as js typst.*() functions lol i feel good moving to next challenge! 👍

@jcbhmr @huwaireb web compiler runs in browser so it introduces extra overhead to understand. I created a typst.node yesterday, which does simpler binding to typst.

For benefit of a node binding rather than a binary shim to typst-cli, there is at least an advantage: you can run compiler with font loading once and comemo cache. Please see https://github.com/typst/typst/issues/2738.

Load font once. For each exec('typst'), there would introduce a 80ms (windows default fonts) ~ 100ms (user installing fonts) overhead to load font. With node binding you can load it once and run multiple compiler actions.

in my PC I install some heavy fonts that brings time to font loading to 400ms.

Share comemo cache across compiler actions. typst is fast, but become 10x slower if we disable its comemo cache. If we run actions on similar files, we can gain benefit from comemo cache.

in packages/compiler/src/lib.rs there's a compile() function that takes in a path ...

It is done in by a hierarchical access model, indexing data by a standard unix path.
https://github.com/Myriad-Dreamin/typst.ts/blob/main/compiler/src/vfs/mod.rs#L125
type VfsAccessModel<M> = CachedAccessModel<OverlayAccessModel<NotifyAccessModel<M>>, Source>;

/// Create a new `Vfs` harnessing over the given `access_model` specific for
/// [`crate::world::CompilerWorld`]. With vfs, we can minimize the
/// implementation overhead for [`AccessModel`] trait.
pub struct Vfs<M: AccessModel + Sized> {
    ...

    /// The wrapped access model.
    access_model: VfsAccessModel<M>,

  ...
}

Interpreting CachedAccessModel<OverlayAccessModel<NotifyAccessModel<M>>, Source>.

NotifyAccessModel is for system notifying a vfs that some physical files are changed.

In browser, "system" is some remote http service.
In operating system, "system" is a physical file system.

OverlayAccessModel is for you shadowing memory content above NotifyAccessModel correctly. There are some synchronization disaster detected long time ago, and I have solved it for typst-preview and get well-tested by preview users.
CachedAccessModel is internally for compiler storing cache within a single compilation.


Error in simulation - Planner Fails

Hi,

I tried to launch the example. I launched the simulation with "roslaunch autonomous_flight dynamic_navigation.launch" and I when give first goal point, the quadrotor navigates in RVIZ navigates but not in gazebo and when it reaches that point, I tried to give another goal point but all I see is error and no plan is generated. I'm attaching the video of the same which can give you clear idea.

https://drive.google.com/file/d/1x-P6u9bIwuj1XQTzKm9oJJ6tOLY6lQoQ/view?usp=sharing
Please help me resolve this and when I try to give navigate to another point post first one.

And also I am trying to integrate the planner with my quadrotor and all I figured is /autonomous_flight/target_state will give the desired state of the robot but I'm confused on which topics I need to give the odom and depth_points.

Current simulation have any restriction to set the goal point? If so, how to expand and which file have that config.

I think the link is not valid

Please check this link
https://drive.google.com/file/d/1x-P6u9bIwuj1XQTzKm9oJJ6tOLY6lQoQ/view?usp=sharing

it seems your simulation is super laggy. You have to run gazebo with a graphic card

Yes. When switched my simulation to other system, it worked. Thanks!


fix up electron and nw.js docs

Should resolve https://github.com/nodegit/nodegit/issues/905, as well as comparable fixes for nw.js

Only failure was a network failure, and this is a docs only change.


Update snyk job to handle npm deps

Fixes the broken build on main. From slack:

the issue is that snyk is looking at the package-lock.json file during a golang job, which is kinda weird. There are "forced" reconciliations that happen on install to avoid vulnerabilities that Snyk doesn't pick up because it doesn't do the install.

This PR removes the offending UI stuff from a job that scans golang files.

Use Github/NPM scanning for UI code


Feature : Refine Linting and Formatting Configuration

Updated ESLint and Prettier configs for better code consistency. Includes refined rules and pre-commit hook adjustments to automatically fix format and lint issues, making code contributions smoother.
Changes

Updated .eslintrc with additional rules for TypeScript, improving code quality and consistency.
Modified the .prettierrc file to refine formatting rules, ensuring that automatic code formatting aligns with our project's standards.
Enhanced the pre-commit hooks in .husky/pre-commit to run both formatting and linting fixes automatically, reducing the likelihood of committing code that doesn't meet our guidelines.

Impact

Contributors will experience fewer discrepancies when pushing code, as the pre-commit hooks now more thoroughly ensure code adheres to our standards.
These updates aim to streamline the development workflow by reducing manual linting and formatting interventions required from contributors.
By enforcing a consistent coding style, we improve the readability and maintainability of our codebase.

Testing

Introduced intentional linting/formatting violations to verify that the pre-commit hooks automatically correct them or block the commit when auto-fixes are not possible.
Confirmed that the updated configurations do not conflict with our current codebase by successfully committing changes after the updates.

Please review the changes and provide any feedback you might have!  fixed #201

@TanmayDhobale I also created a similar PR although I believe it solves a different problem at the moment. Your PR is valuable in that it includes improved formating, linting, and Husky pre-commit configs.
But this PR still has issues with ESLint overwriting rules from Prettier when I tested this PR locally.
Could you also verify by trying format:fix and then lint:fix to see if ESLint is overriding any Prettier changes? See videos here to see what I'm specifically talking about.
I think best case would be to first merge #244 to fix conflicting style rules, and then bring over all of your additions to linting, formatting, and Husky.
PS: Also, I think it's best if you exclude the actual linting/formatting changes to files - and have that in a separate PR as this PR currently has a lot of file changes.

@aryanprince yea sure , i am closing this pr now thanks


Dead loop of counting the number of possible activations per refresh interval

Hi,
I try to run Blacksmith on my machine with an i7-8700 CPU, Ubuntu 18 with Linux version 5.4.0-84-generic, and 1x16G Hynix/Hyundai HMA82GU6DJR8N-VK DRAM.
However, the program is stuck at the function DramAnalyzer::count_acts_per_trefi(). Based on my understanding, it's due to a hard-coded value 3.0 in https://github.com/comsec-group/blacksmith/blob/0d775c95569950b09e8a503b15af258473dff9d6/src/Memory/DramAnalyzer.cpp#L165
Could you please explain why 3.0 is chosen and how to determine this value in other machines?

I am not the author of blacksmith but also had some issues with this on some machines.
To my understanding this is a "gut instinct" boundary for the standard derivation of the measurements. The idea here is that we want to sample long enough for the sampled values to be fairly regular/stable.
To improve the timing accuracy you can try to pin blacksmith to an isolated CPU core.
Otherwise I would suggest to dump the gathered timings to see where things go wrong (i.e. if there are some heavy outliers). On some machines tweaking the timing function worked for me.

Hi @its-luca ,
Thanks for your prompt reply.

Otherwise I would suggest to dump the gathered timings to see where things go wrong (i.e. if there are some heavy outliers). On some machines tweaking the timing function worked for me.

Could you please give a detail on how to tweak it? Just change the value 3.0 to other values based on the gathered timings?

For me, changing the actual time measurement code a little improved the stability of the measurements on some machines. I'd have to look up specifics.
You can ofc. also increase the threshold for the std. However, if your measurement is too noisy it will be of little use. I cannot give you a specific upper boundary here.
Another error source could be the selection of the two addresses used in the experiment. They have be in the same bank and are sampled in https://github.com/comsec-group/blacksmith/blob/0d775c95569950b09e8a503b15af258473dff9d6/src/Memory/DramAnalyzer.cpp#L6
Did you adjust the timing threshold in https://github.com/comsec-group/blacksmith/blob/0d775c95569950b09e8a503b15af258473dff9d6/include/GlobalDefines.hpp#L50  to your machine?
For debugging, I would again dump the timing values of this experiment, plot a histogram and check that it resembles Figure 3 in the drama paper (i.e. one large heap and a smaller one. The actual timing values might vary).