Center shape according to logic Bullet applies

Bullet will center the heightmap data you supply to it. Bit counter intuitive but so be it.
This PR adjusts our debug shape to match and adds a remark in the help pages about this behaviour.
This could be improved by somehow offsetting the shape in bullet, giving it a different transform, but that is a larger structural change.
Fixes #27998

I've changed this back to WIP, see #27998 for the discussion on improving this logic even further.

Ok, I've added my changes based on the discussion in #27998.
It works really well however it has one fundamental flaw. Doing so implements a fix specific to Bullets behavior into the physics system. I believe it's the wrong place however fixing this in the bullet module feels a lot more difficult to handle (I could be wrong).
So I am leaving this in WIP for further discussion.

Ok, I managed to make the adjustment in the bullet glue code. Any suggestions on making it prettier are welcome but it works like a charm.
Someone should really make an editor gizmo for this. I have no idea how to do this (yet).

I looks ok to me, but I didn't try it.
The only two things are:

are you sure that when you change the height map data you reload the position of the shape in the btCollisionObject?
I think that call the function get_transform instead of get_adjusted_transform is better

however nice work

@AndreaCatania I need to reload the position because a change in the heightmap data requires us to adjust the transform Bullet uses.
If the lowest height in my heightmap is -100, and the highest is 100, then bullet will position the collision shape as we expect, nicely centered on our transform.
If I change the lowest height in my heightmap to -50 and keep the highest as 100, I do not expect my heightmap to move, but bullet will recenter it and suddenly my heightmap behaves asif the lowest value is -75 and the height is 75.
What I do in this case is adjust the transform where we've placed the collision shape by moving it upwards by 25 units.
Hence the name, get_adjusted_transform

Yep sure, but for the other shapes doesn't make sense, for this reason I think that would be more appropriate get_transform.
While review the code the question that I asked to myself was: Why the shape need an adjusted transform with this change?
Then I opened the function to answer it, and I got it. But since this is true only for one case I think that is more appropriate name it get_transform.

Thanks!

Then I opened the function to answer it, and I got it. But since this is true only for one case I think that is more appropriate name it get_transform.

Woops, just merged while you were posting this. If you agree on renaming it, it can be done in another commit.

I realize just now that this breaks compatibility in my plugin. Now I have to add an if to check if it's Godot 3.2 or later to change the transform differently.

As I understand it that's an expected compatibility breakage, but it would be good to have a confirmation from @BastiaanOlij or @AndreaCatania. Should also be mentioned in the changelog (@Calinou).

@akien-mga that is my position indeed. Before we introduced the collision shape the heightmap could only be used directly on the physics engine so its unlikely a lot of people used it extensive enough and I think the previous behavior is just plain confusing. With the collision shape added in 3.2 and this behavior changed in 3.2 I think we can get away with the breakage. Probably @Zylann plugin is the only thing really effected by this.
I understand why Bullet does what it does, but it would have been easy to solve upstream, instead they chose to have the collision shape centered and thus move which to the end users is unintuitive and hard to predict.
For us to properly deal with this so the heights you set in the heightmap are the actually heights at which the collision shape is positioned instead of it moving around on you as your minimum and maximum height values change, seems legit to me :)
The way we solved it means that Bullet is still happy as the collision shape is still centered in its bounding box, but we reposition it so the end user isn't tripped up by this and its totally transparent.


Why rwkv6 has a similar GPU memory occupancy on lower resolution images  with ViT methods?

I try to train the RWKV6 block on 256x256 images. However, I found there is almost no GPU reduction relative to ViT. So what is the advantage of Vision-RWKV6 in this setting?

Hi, thanks for your attention to VRWKV!
The reduction of GPU memory of VRWKV/VRWKV6 on low resolution is minimal (vrwkv may even cost more VRAMs or have a slower speed, as the original attention mechanism is optimized over several versions in common DL Frameworks). VRWKV will show its advantages in higher-resolution scenarios. In low-resolution cases, we show the VRWKV has comparable performance to ViT(so that it has the potential to replace ViT in most tasks currently and shows its advantages in high resolutions).


Fix TLS issue with OpenStack auth (Designate)

It appears as though the certificate options for OpenStack were only being configured for requests to Designate, and not for the initial authentication requests to Keystone. This change resolves that issue.

I signed it

Hi @njuettner, I didn't add or remove any functionality from the Designate provider, and it looks like only the implementation of the provider interface is tested. A test here would just assert that the OpenStack client was correctly initialized -- should I add that?

@njuettner I added a test, but keep running into timeouts with verify-gometalinter.sh in CI. Are you familiar with that issue?

Sometimes there's a timeout. I just triggered it again.

/lgtm

/approve


About training simi-supervised model

Thanks for your code! I have been successful in running 'train.py'. But when running the 'train_semi_supervised.py' ,I got some trouble.

when I running Function 'backward_D', I can not get the value 'self.real_z_mean'. Then I change the code as following:
if self.real_z_mean is None:
pred_real = self.netD(self.real.detach())
else:
pred_real = self.netD(**{'x': self.real.detach(), 'z': self.real_z_mean.detach()})
Then, I got another trouble.
in line 332: AttributeError: 'Discriminator' object has no attribute 'infer_img'. Then I change the code as following:
self.netD.infer_img.blocks[0][0].conv1.bias.grad==0
but the gradients of D are nan
finally, I change the 'Discriminator' to 'BiDiscriminator', Because 'BiDiscriminator' has attribute 'infer_img'. But It dose not work.
what should I do?
Thank you!

Hi, thanks for you interest!
We will look into this, we have a fix coming soon anyway.

I hope this commit helps, please re-open of not


publish<EMAIL_ADDRESS>

Requested by: @AbhiPrasad
Quick links:

View changes
View check runs

Assign the accepted label to this issue to approve the release.
Targets

[ ] aws-lambda-layer
[ ] gcs
[ ] github
[ ] npm
[ ] registry

Targets marked with a checkbox have already been executed.  Administrators can manually tick a checkbox to force craft to skip it.

hi!, is the publishing action working fine? I see it has been running for 4hrs and it has been stuck in the "Publishing using craft" step for 3h 33m already 🤔

Yeah it’s strange, we’ll take a look!

2nd times a charm 🚀

good news, thank you!


Script attempted to access unexisting global variable 'bit'

Version: node_redis v2.6.0-1 and Redis server v=2.8.17 sha=00000000:0 malloc=libc bits=64 build=454d996953bc053b
Platform: This is happening on my Macbook during development and on Ubuntu 14 in our staging enviornment
Description: We are using bull as a redis queue. When adding a job to the queue (settlementGameQueue.add({game: game}, {delay: delay});), we get the following error:

Unhandled rejection Error: ERR Error running script (call to f_1f46a25dd9f50b1331d5c86b90ba4dce1e15cd8f): @enable_strict_lua:14: user_script:3: Script attempted to access unexisting global variable 'bit'
    at JavascriptReplyParser.parseResult (/Users/aaronroberson/Sites/gsn/draft4-server-api/node_modules/redis-parser/lib/javascript.js:90:16)
    at JavascriptReplyParser.tryParsing (/Users/aaronroberson/Sites/gsn/draft4-server-api/node_modules/redis-parser/lib/javascript.js:117:21)
    at JavascriptReplyParser.run (/Users/aaronroberson/Sites/gsn/draft4-server-api/node_modules/redis-parser/lib/javascript.js:131:22)
    at JavascriptReplyParser.execute (/Users/aaronroberson/Sites/gsn/draft4-server-api/node_modules/redis-parser/lib/javascript.js:112:10)
    at Socket.<anonymous> (/Users/aaronroberson/Sites/gsn/draft4-server-api/node_modules/bull/node_modules/redis/index.js:223:27)
    at emitOne (events.js:77:13)
    at Socket.emit (events.js:169:7)
    at readableAddChunk (_stream_readable.js:146:16)
    at Socket.Readable.push (_stream_readable.js:110:10)
    at TCP.onread (net.js:523:20)

I installed hiredis and I still get the same error, but hiredis is in the stacktrace. I also downgraded to node_redis 2.5.3:
    at Error (native)
    at HiredisReplyParser.parseData (/Users/aaronroberson/Sites/gsn/draft4-server-api/node_modules/redis-parser/lib/hiredis.js:13:28)
    at HiredisReplyParser.execute (/Users/aaronroberson/Sites/gsn/draft4-server-api/node_modules/redis-parser/lib/hiredis.js:24:22)
    at Socket.<anonymous> (/Users/aaronroberson/Sites/gsn/draft4-server-api/node_modules/redis/index.js:223:27)
    at emitOne (events.js:77:13)
    at Socket.emit (events.js:169:7)
    at readableAddChunk (_stream_readable.js:146:16)
    at Socket.Readable.push (_stream_readable.js:110:10)
    at TCP.onread (net.js:523:20)

Is this a local redis server? Could you try another server and see if you have the issue, looks like your redis server is loading certain libs, specifically bit which is used here: https://github.com/OptimalBits/bull/blob/c61e3ff88277f33ebec30c060588f85b31ff49c9/lib/scripts.js#L90

The error is thrown by Redis itself and the stacktrace is just not helpful currently. Seems like bull tries to use a variable that is not set at execution time.
So it's not a error in node_redis and I suggest to open a new issue in bull for it.

@Salakar On my local MacBook I have redis-server running and I'm connecting to it all on the same machine. However, I'm getting the exact same error on our staging server which connects to a sperate box running redis.
@BridgeAR Looking at the code that @Salakar linked to, it does, in fact, look like bull is referencing a variable which doesn't exist. I will open a ticket there as you've suggested.
Thank you both for the quick reply!


s3-remote-state.principals should not have empty list as default.

Default value means the caller can skip that variable, but skipping that value ends up invalid policy document passing to AWS and terraform stuck on creating the policy.
The default value should be gone to inform the caller that he must pass a value.

I have https://github.com/fpco/terraform-aws-foundation/pull/297 for this issue

This issue has been completed now with https://github.com/fpco/terraform-aws-foundation/pull/297


JUnit tests

Implement useful tests and integrate them into Jenkins.
Or:
Delete JUnit dependencies from build.gradle.

---
Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/5757214-junit-tests?utm_campaign=plugin&utm_content=tracker%2F715506&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F715506&utm_medium=issues&utm_source=github).

Closing as completed long ago :-)


OTA update for board without u-boot

Hi,
this is not an issue but maybe getting more info about possibilities and others to share ;). I saw that for u-boot based board like nano mender + other OTA can be used without an issue. But what about boards like NX which don't contain u-boot in boot chain? I had an idea to modify cboot to mimic u-boot functionality (store bootcount + bootlimit) and then swupdate can be used. Other opinion was to start u-boot from cboot but this seems not straightforward (maybe even impossible). Are there any other documents, tips where to look? Thanks a lot.

@nandra @madisongh has added support for cboot based update on mender in the zeus branch.  See the full history of changes for zeus at https://github.com/mendersoftware/meta-mender-community/pull/147/commits
The way he's done it for mender is to make the state scripts use NVIDIA's redundant boot and nv_update_engine described in NVIDIA boot redundancy documentation and make the mender fw_printenv and fw_setenv calls do nothing other than set a flag /var/lib/mender/upgrade_available.  He can explain it a lot better but these are a few pointers.
Also FYI I'm working on a mender port to dunfell how see this link for work in progress which will target cboot platforms first since there's additional work for u-boot support in l4t-r32.4.2

There are some drawbacks to relying entirely on NVIDIA's redundant boot support - there are some cases where it will simply stop trying and switch to recovery mode, which isn't desirable on devices that run unattended.  I created some tools to deal with that.

@dwalkes @madisongh thanks a lot for pointers. I'll give a try.

@madisongh would be possible to cherrypick nx module support for zeus branch? I checked dunfell and there cherrypick 3 commits whihc added support for nx emmc but build is failing. Do I need also some other commits? Thanks.

It's a bit more involved than that, since support for NX didn't come until L4T R32.4.2. The dunfell-l4t-r32.4.2 branch should have everything needed, and I build both nx-devkit and nx-emmc with my test distro that uses that branch.

@madisongh ok thanks. I would like to try mender for nx (not sure when dunfell land in meta-mender as zeus is not yet ready :))

@dwalkes has something in the works, I believe - see this test-distro issue.

@nandra @madisongh

see this test-distro issue

Correct, the branches here work with jetson-tx2-cboot as long as you don't care about the test-distro code changes mentioned there.  I haven't tried Xavier NX yet but help with that is welcome.  I also hope to have an update on test-distro proposal in https://github.com/madisongh/test-distro/issues/5 within the next week.

@dwalkes I've cloned test distro and hopefuly setup everything be when start building core-image-minimal I got:
--- Starting bitbake server pid 17410 at 2020-06-15 19:53:50.574587 ---
ERROR: ParseError in configuration INHERITs: Could not inherit file classes/mender-full.bbclass

and also in build I don't have bblayers.conf? I just source ./setup-env Thanks.

@nandra

I've cloned test distro

Presumably you mean my hacked together mender branch test-distro repo at https://github.com/Trellis-Logic/test-distro/tree/dunfell-mender-l4t-r32.4.2 correct?
Not sure if this was obvious but you'll need to also
git checkout --track origin/dunfell-mender-l4t-r32.4.2
git submodule update --init

Then
source setup-env

Then follow instructions here to clear security options in local.conf unless you've setup signing/encryption.
We have some hacks for jetson-tx2 cboot we can share that can get you running Mender successfully on that platform if you like.  We'll try to get those out later today.
All this will be cleaned up with https://github.com/madisongh/test-distro/issues/5 but this should get you going for now.

Also I haven't tried core-image-minimal yet, the only thing we've got working at the moment is core-image-base with jetson-tx2_cboot

@dwalkes  I performed steps you mentioned  and still have same issue when running:
MACHINE=jetson-xavier-nx-devkit-emmc nice bitbake core-image-base

Maybe it's not prepared yet for this machine? Thanks.

@nandra

Maybe it's not prepared yet for this machine? Thanks.

Correct, jetson-tx2-cboot is the only build we've tried thus far.

@dwalkes ok let me try to look and check what is going on ;). Thanks.

The jetson-tx2-cboot mender image now works at https://github.com/Trellis-Logic/test-distro/tree/dunfell-mender-l4t-r32.4.2 without signing turned on, see README instructions

@dwalkes I've tried added support for jetson-nx emmc and I have core-image-minimal build working but I have few qestions:

for tx2 you created new machine which inherit original one + add mender stuff on top (I did same for nx). Is that right?
in recipe bootfiles there is eks.img Can I reuse it also fro nx?

My dummy modifications was only to add (copy'n'paste) in tegra-bootfiles_32.4.2.bbappend recipe + added compatible machine to bootfiles. Thanks a lot.

@nandra

I've tried added support for jetson-nx emmc and I have core-image-minimal build working

Great!

for tx2 you created new machine which inherit original one + add mender stuff on top (I did same for nx). Is that right?

I actually didn't create it, as with most of the stuff in this test branch it came from Matt's zeus changes, but yes, the jetson-tx2-cboot machine was originally added with mender specific cboot support.  Based on the comment here it looks like this was just to make it easier to support both cboot and uboot options for TX2, which supports both.  So I don't think it would be necessary to make a new machine for Xavier NX which appears to only support cboot.

in recipe bootfiles there is eks.img Can I reuse it also fro nx?

This is related to secure boot encryption support.  You can see the commit here where it was added and comments from Matt here which is where I'd start if you need to support encryption.  When you use my security options hack it won't be used so I don't think it will matter.  It would be better to remove the dependency I suppose and condition the bootfiles logic which references it on whether signing is supported.  I expect this is something which will be cleaned up in https://github.com/madisongh/test-distro/issues/5 using secureboot MACHINEOVERRIDES as discussed here

@dwalkes I was too fast with reply that something works ;). I build master branch not dunfell-mender. When building later one I'm getting some signing issue here. kernel-bup-payload.bb:do_deploy task fails with this trace. Ideas? I have SECURITY_OPTIONS="" setup in local.conf. Thanks.

@nandra
It looks like the first error is here:
| [   2.9429 ] File DATAFILE open failed
| [   3.5520 ]
| Error: Return value 19

Which I think means there's a problem with the sed commands here not getting which I haven't tried to trace.
There's another issue here;
| File /home/jenkins/test-distro/build/tmp/work/jetson_xavier_nx_devkit_emmc-td-linux/kernel-bup-payload/1.0-r0/bup-payload/signed/xusb_sil_rel_fw does not exist
...
|   File "/home/jenkins/test-distro/build/tmp/work/jetson_xavier_nx_devkit_emmc-td-linux/kernel-bup-payload/1.0-r0/bup-payload/BUP_generator.py", line 266, in fill_image
|     binary_handle = open(entry_info[0], 'rb')
| TypeError: expected str, bytes or os.PathLike object, not NoneType
|
| FAILURE: xusb_only_payload not created

I don't have my head deep wrapped around the signing bup logic enough to be able to give a suggestion here off the cuff.  I'd probably poke around in the run script to see where xusb_sil_rel_fw is getting referenced.
You can watch this branch for some work in progress changes related to mender support but nothing related to Xavier NX yet.

I had the same error yesterday.
The xusb_only_payload failure is due to DATAFILE not being found and stopping the script. So trip the  tags from your own partitions as @dwalkes metioned and the bup signing will succeed.

@lfdmn you mean drop DATAFILE partition? I'm building mender image and would like to use data partition.
Can you please be more specific how do you fix it? Thanks.
I looked at image_types_mender_tegra.bbclass but I'm not sure I understand this line:
sed -i -e"1a sed -i -e'/DATAFILE/d' ./flash.xml.in" $outfile

Also I searched work directory but cannot find any reference of generate_bup_payload.sh script which should be updated. Any pointers?

That's a sed command to insert (1a means "append after line 1") an extra sed command to remove the DATAFILE tag from the flash.xml.in file.  The script being changed is either generate_bup_payload.sh when creating a normal tegraflash package, or doflash.sh when generating a BUP payload.  Take a look at image_types_tegra.bbclass in meta-tegra to see the functions the _appends in image_types_mender_tegra.bbclass apply to.

The kernel-bup-payload error should be fixed up by https://github.com/BoulderAI/meta-mender-community/commit/78b441c333d05ea758563c4c956363caedbd81e2 .
See here for the work-in-progress -- jetson-tx2 (only virtual/bootloader=cboot for now) builds and works and jetson-xavier at least builds now.
I haven't looked at all at xavier-nx yet, but if it shares the same flash layout as jetson-xavier, the meta-mender-tegra layer will handle it the same, and it might just work. Let me know ;)

I haven't looked at all at xavier-nx yet, but if it shares the same flash layout as jetson-xavier, the meta-mender-tegra layer will handle it the same, and it might just work. Let me know ;)

This is NVIDIA, so of course it's different... on both the dev kit and eMMC flavors, the bootloader partitions are in SPI flash, with the kernel/kernel-dtb/etc. residing on SDcard/eMMC.  That said, since u-boot isn't involved, it might just work.

Hope springs eternal, I guess.
If needed, an additional override for the flash layout on the NX can be added here: https://github.com/BoulderAI/meta-mender-community/tree/dunfell/meta-mender-tegra/recipes-bsp/tegra-binaries/mender-custom-flash-layout
Here's the diff vs the stock nvidia one for t194. Probably you can just add an override for the nx spi/emmc or spi/sd machine with similar changes:
--- flash_t194_sdmmc.xml	2020-04-08 18:26:25.000000000 -0700
+++ flash_mender.xml	2020-07-02 14:22:50.959162900 -0700
@@ -212,7 +212,6 @@
--- flash_t194_sdmmc.xml	2020-04-08 18:26:25.000000000 -0700
+++ flash_mender.xml	2020-07-02 14:25:49.252317203 -0700
@@ -212,7 +212,6 @@
             <allocation_attribute> 0x8 </allocation_attribute>
             <align_boundary> 4096 </align_boundary>
             <percent_reserved> 0 </percent_reserved>
-            <unique_guid> APPUUID </unique_guid>
             <filename> APPFILE </filename>
             <description> **Required.** Contains the rootfs. This partition must be defined after
               `primary_gpt` so it can be accessed as the fixed known special device
@@ -630,6 +629,16 @@
             <description> **Optional.** Reserved for future use by the recovery filesystem;
               removable. </description>
         </partition>
+        <partition name="APP_b" type="data">
+            <allocation_policy> sequential </allocation_policy>
+            <filesystem_type> basic </filesystem_type>
+            <size> APPSIZE </size>
+            <file_system_attribute> 0 </file_system_attribute>
+            <allocation_attribute> 0x8 </allocation_attribute>
+            <align_boundary> 4096 </align_boundary>
+            <percent_reserved> 0 </percent_reserved>
+            <filename> APPFILE </filename>
+        </partition>
         <partition name="UDA" type="data">
             <allocation_policy> sequential </allocation_policy>
             <filesystem_type> basic </filesystem_type>
@@ -637,6 +646,7 @@
             <allocation_attribute> 0x808 </allocation_attribute>
             <percent_reserved> 0 </percent_reserved>
             <align_boundary> 4096 </align_boundary>
+            <filename> DATAFILE </filename>
             <description> **Required.** Automatically takes all remaining space on the device except space
               occupied by `secondary_gpt`. Allocation attribute must be set to 0x808. This
               partition may be mounted and used to store user data. </description>

I gave it a quick try, and after some hackery I was able to get it to work on my Xavier NX dev kit (the SDcard one).

I created a new mender flash layout file by applying the diffs above to the stock flash_l4t_t194_spi_sd_p3668.xml file.
I commented out the MENDER_STORAGE_TOTAL_SIZE_MB setting in testdistro-mender.conf - there is no EMMC_SIZE setting for the SDcard-based Jetsons, so this was causing parsing errors.
I added these settings to tegra-mender-setup.bbclass:

MENDER_DATA_PART_NUMBER_jetson-xavier-nx-devkit = "12"
MENDER_ROOTFS_PART_B_NUMBER_jetson-xavier-nx-devkit = "11"

I set IMAGE_OVERHEAD_FACTOR to "1.0" so the partition sizes wouldn't be too large for the total SDcard space.

The big problem appears to be with the NVIDIA flashing tools, which were crapping out on the additions to the flash layout file - looks like tegraparser_v2 in particular uses mmap() to map in the files, causing it to get ENOMEM errors and/or segfaults when there is more than one multi-gigabyte partition that needs mapping.  I worked around it by removing the filenames for APP_b and UDA from the XML file and then writing those partitions manually after running dosdcard.sh.
Once I got past those obstacles, I got a bootable image on the SDcard, could register it with my hosted Mender setup, and could use Mender to update.
If I get a chance, I'll see if I can get it working with my eMMC Xavier NX.  I suspect, though, that the tegraparser_v2 problem will prevent me from making it work, since I won't have a way to flash the extra partitions.

The big problem appears to be with the NVIDIA flashing tools, which were crapping out on the additions to the flash layout file

Turns out this is not a new thing, just something I hadn't run into before.  After several experiments I think I've hit on the right way to deal with this... and the good news is that it simplifies data partition handling.  See #377 for more info.

I was also able to get a Mender-enabled image built for the eMMC-based Xavier NX module, after a couple of further tweaks:

I put back the MENDER_STORAGE_TOTAL_SIZE_MB setting, adjusting it to be:

MENDER_STORAGE_TOTAL_SIZE_MB ?= "${@tegra_mender_calc_total_size(d) - int(d.getVar('TEGRA_FLASH_PARTITIONS_MB'))}"

I change the part number settings to be:

MENDER_DATA_PART_NUMBER_xavier-nx = "12"
MENDER_ROOTFS_PART_B_NUMBER_xavier-nx = "11"

since the same numbering is used on both the SDcard and the eMMC for Xavier NX (thanks to the use of SPI flash for booting on both).

That's great! Thanks for taking at look at those, Matt. I'll start incorporating these changes to the dunfell branch we've got going for meta-mender-community this weekend, as time permits, unless you've already got a working branch. At some point we'll consolidate all this into one downstream since mendersoftware/meta-mender-community is still on zeus.

At some point we'll consolidate all this into one downstream since mendersoftware/meta-mender-community is still on zeus.

Sounds good.  I've merged #377 after running some test builds and flashing operations across the platforms. With those changes in place, you should be able to remove all of the tegraflash_XXX functions/appends from image_types_mender_tegra.bbclass and just set
IMAGE_TEGRAFLASH_DATA ?= "${IMGDEPLOYDIR}/${IMAGE_LINK_NAME}.dataimg"

If i'd like to work with mender on the xavier nx is my best present path to use OE4T/meta-mender-community and the zeus branch? It seems this is still a work in progress but it appears it's far enough along to try out?

Mender support is now complete for all machines in the current dunfell-l4t-r32.4.3 and master branches. Have a look at the tegrademo-mender distro here: https://github.com/OE4T/tegra-demo-distro

Thanks! I've never used this demo project repo before but after following the instructions it's building so greatly appreciate the work.

Mender support is now complete for all machines in the current dunfell-l4t-r32.4.3 and master branches. Have a look at the tegrademo-mender distro here: https://github.com/OE4T/tegra-demo-distro

tegrademo-mender ? i dont see a branch or such, i see how to build it, but nothing explains much on how to get a mender build out of it. Nothing in the wiki.......  i can build demo-oimage-full but dont see mender is included. nothing in the conf/local.conf file for it

Mender support is provided when your DISTRO is set to tegrademo-mender. When you run setup-env, you can pass the --distro tegrademo-mender option, or set DISTRO to tegrademo-mender in the $BUILDDIR/conf/local.conf

Mender support is provided when your DISTRO is set to tegrademo-mender. When you run setup-env, you can pass the --distro tegrademo-mender option, or set DISTRO to tegrademo-mender in the $BUILDDIR/conf/local.conf

great, now i have a clue.... much appreciated!

okay now what did i miss!
. ./setup-env --machine jetson-xavier-nx-devkit  --distro tegrademo-mender
NVIDIA_DEVNET_MIRROR location not found; some recipes may not be available
Shell environment set up for builds.
You can now run 'bitbake '
Targets for building Tegra demo images with test applications:
core-image-minimal  - minimally bootable image (no demo apps, no graphics)
demo-image-base     - basic image with no graphics
demo-image-egl      - basic image with DRM/EGL, no window manager
demo-image-sato     - X11 image with 'sato' UI
demo-image-weston   - Wayland with Weston compositor
demo-image-full     - X11/sato UI plus docker, openCV, multimedia API samples
dingo@gitlab:~/tegra-demo-distro/build$ bitbake demo-image-full
ERROR: ParseError in configuration INHERITs: Could not inherit file classes/mender-full.bbclass

nevermind... missing layers in bblayer.conf ... fixed

FYI I submitted a PR at https://github.com/mendersoftware/meta-mender-community/pull/175 to update documentation in meta-mender-community, and started rework on mender hub pages.

This isn't really an issue, but I'm looking for more info on meta-tegrademo.
Is it best to keep the demo layer and then build on top of it with my own layer?
Or, is it recommended method to re-work the demo layer into my own layer?
BTW, thanks again for adding in mender support to the xavier nx. What a time saver.

Is it best to keep the demo layer and then build on top of it with my own layer?
Or, is it recommended method to re-work the demo layer into my own layer?

We had this same discussion in https://github.com/madisongh/test-distro/issues/5#issuecomment-687850353 and you can see where we ended up based on Kurt's suggestion here.
I actually haven't had a chance to dig in to the final implementation myself yet but it's on my list to put some documentation together about this.  I think we could either use an update to the README or a new wiki page on tegra-demo-distro about this.

Thanks @dwalkes that was a very useful read.

I think the original question here has been answered, so am going to close.


Convenience constructors for Interpolations.jl

I find the construction methods for Interpolations.jl to be almost impenetrable.  It would be really nice to have some convenience constructors to make things easier to use.  Coupled with the () access, I think this makes Interpolations.jl feasible for teaching.
I can't figure out whether we would usually want https://github.com/JuliaMath/Interpolations.jl#gridded-interpolation or https://github.com/JuliaMath/Interpolations.jl#scaled-bsplines ?
The big scenarios I can think of to make easy in 1 dimension are:

Linear interpolation on a regular grid
Linear interpolation on an irregular grid
Cubic spline interpolation on a regular grid
Cubic spline interpolation on an irregular grid.
Look into the same for 2 or more dimensions

I think it would make sense to figure out:

[x] Document the canonical way to do each of them in the current Interpolations.jl

Where possible, we can discuss reasonable default parameters

[x] Do we need to get Cubic spline interpolation working in the gridded setup in order to support cubic splines on an irregular grid?
[x] See what sort of convenience constructor would make sense so these are easier to use.

A much saner interface for construction (if not calling) is https://github.com/kbarbary/Dierckx.jl

Note that I found a PR in progress for #48 which we might be able to help with.

Saw this: https://discourse.julialang.org/t/julia-version-of-matlab-function-interp1/8540  Not sure it is the best idea to do it that way, but gives you a sense of the usage patters.

I made the following tutorial file that lists all possible scenarios and their solutions: https://github.com/chiyahn/notes/blob/master/interpolations/interpolation-scenarios.md including some suggestions for convenient constructors as well!
For cubic splines (or other nonlinear splines) on irregular grids, I can think of the following two solutions as I explained in the markdown file:

Construct interpolations using a recursive algorithm: Slow, but trivial. I tried implementation a bit by myself but it seems like integration with the current code is not trivial and requires substantial changes.

ref: https://github.com/floswald/ApproXD.jl

Construct gridded interpolation algorithm dedicated for cubic splines: Fast, but requires substantial amount of verification.

ref: https://github.com/ubcecon/computing_and_datascience/issues/48 (PR you mentioned)

Great, thanks!  But I didn't see the convenience constructors?
Also, see the final comments on #26 to get it ready to merge.

Sorry, now I see the constructors.  I don't love the interp1 approach especially much, since I am not sure how well it supports uniform vs. irregular grids, andCubic(Line()) is still pretty ugly.  Also, I am not 100% sure that it needs to be written to be written specific to one dimension, since we can use Julia dispatching.
One thing that could help is that we can dispatch based on the StepRangeLen type, i.e.
julia> typeof(1.0:0.1:2.0)
StepRangeLen{Float64,Base.TwicePrecision{Float64},Base.TwicePrecision{Float64}}

julia> typeof(1:10)
UnitRange{Int64}

Is that a way to have a single interface support both irregular and regular grids?  Maybe the convenience constructor should have the name of the interpolation method rather than the dimension?  That is,
f(x) = sin(x) / 2
xs = linspace(1.0, 2.0 ,10) #Note, this does not return a vector!
xs_irregular = collect(xs); #Now this is an an AbstractVector  and no longer a StepRangeLen

 #This knows it is a regular grid since xs is StepRangeLen or a StepRange
LinearInterp(xs, f.(xs))

 #This could use the fact that typeof(1:10) is a UnitRange to realize that rescaling is not needed?
LinearInterp(1:10, f.(1:10))

#Cubic Spline knowing that xs is a StepRangeLen
CubicSplineInterp(xs, f.(xs)) #This knows it is a regular grid since xs is StepRangeLen

#Now, this uses the irregular grids, since it dispatches on the AbstractVector rather than the UnitRange or StepRangeLen?
LinearInterp(xs_irregular , f.(xs_irregular ))

#This isn't functional yet..., but could be added later.
CubicSplineInterp(xs_irregular , f.(xs_irregular ))

If things moved to multiple dimensional interpolation, then I think that this pattern for convenience constructors would work well, we would just need to dispatch on what xs looks like for a uniform grid.
Note that with this, we could write a 1D LinearInterp for the non-uniform case that completely moves out (and deprecates) https://github.com/QuantEcon/QuantEcon.jl/blob/master/src/interp.jl

I see your point! I was trying to imitate what MATLAB users are going to be more familiar with, but I agree with you that writing a constructor based on degrees (rather than dimension) is a more reasonable choice. The best way to handle it would be to create overloading constructors that are dependent on types of parameters so that (at least for linear case) it returns a scaled interpolation instance if the grids are regular and gridded one if they are not.
function LinearInterp(xs::AbstractArray,ys)
    itp = interpolate((xs,), ys, Gridded(Linear()))
    return(itp)
end

function LinearInterp(xs::Range,ys)
    itp = interpolate(ys, BSpline(Linear()), OnGrid())
    return(scale(itp, xs))
end

The following unit tests passed:
# test regular grid cases
xs = linspace(xmin,xmax,N)
ys = f(xs)
itp = LinearInterp(xs, ys)

# unit tests
@test itp[xs[1]] == f(xs[1])
@test itp[xs[2]] == f(xs[2])
@test itp[(xs[1] + xs[2]) / 2] == (f(xs[1]) + f(xs[2])) / 2

# test irregular grid cases
xs = [x^2 for x = 1:xmax]
ys = f(xs)
itp = LinearInterp(xs, ys)

# unit tests
@test itp[xs[1]] == f(xs[1])
@test itp[xs[2]] == f(xs[2])
@test itp[(xs[1] + xs[2]) / 2] == (f(xs[1]) + f(xs[2])) / 2

I will look into multi-dimensional cases tomorrow as soon as the survey conference (😭) is finally done.

Great!
Though before multidimensional cases, I would try to finish https://github.com/JuliaMath/Interpolations.jl/pull/206#issuecomment-402281931 so that we can tell people to merge your PR.
A few small things on this, which seems extremely close:

The test failed for me for my f because of the exact floating point comparison, but  I think you meant to use ≈ instead
Probably worth adding LinearInterp(1:10, f.(1:10)) to the testsuite (i.e., a UnitRange) but it seems to work.
This approach seems to work well with Cubic Splines as well.  That is, I think it is as simple as

function CubicSplineInterp(xs::Range,ys)
    itp = interpolate(ys, BSpline(Cubic(Line())), OnGrid())
    return(scale(itp, xs))
end

If an irregular grid version was eventually added, then CubicSplineInterp(xs::AbstractArray, ys) would be added.
What are your thoughts about extrapolation in the convenience constructor?

Mine is that it should be avoided so there are no silent errors from misuse (and that people should use the full interface if they like).  I have already made that mistake in testing
I had trouble figuring it out, though...

I did a basic profiling with using BenchmarkTools and @btime.  For a simple lookup, it seems like itp4 takes 72ns with extrapolation and 92ns with no extrapolation.  I think this is a reasonable price to pay for safety with a convenience constructor, but we can get other peoples feedback

For multidimensional cases, it turns out I have to change the order of parameters for constructors from (ranges, values) to (values, ranges...) to allow arbitrary dimensions for ranges. Gridded interpolations (for irregular grids) work in the following version:

LinearInterp(vs, ranges::AbstractArray...) = interpolate(ranges, vs, Gridded(Linear()))

But for scaled interpolations (for regular grids), there's a bit of problem; the following constructor seems to be a trivial choice:

LinearInterp(vs, ranges::Range...) = scale(interpolate(vs, BSpline(Linear()), OnGrid()), ranges)

but running the following lines returns an error:
xs = 1:.1:10
ys = 1:.5:10
f(x, y) = log(x+y)
A = [f(x,y) for x in xs, y in ys]
LinearInterp(A, xs, ys)

The reason is that when xs and ys are passed as parameters for ranges, they are going to be provided as a single Tuple instance while scale does not have any overloading constructor with tuples. In fact, using implementation of scale, the following version works:
LinearInterp(vs, ranges::Range...) = ScaledInterpolation(interpolate(vs, BSpline(Linear()), OnGrid()), ranges)

On the other hand, notice that this bypasses check functions (for dimension compatibility, etc.) in scale; I am not sure why maintainers decided to make these two different versions (for safety?), but it seems like the only difference between ScaledInterpolation and scale is coming from these check functions. So we can proceed with one of the followings:

Add these check functions in constructors manually; this is the simplest and safest way to go, but makes the constructor look complicated and there's a possibility of being inconsistent with check functions in scale when there are changes in future
Stick with the version above; this makes the code neat and consistent with the other scripts in the package, but might be unsafe as users might not be aware of what the errors are about.
I personally prefer the first option, but I want to hear your opinions too. 😃

I agree that it should be avoided. Calling extrapolation is relatively easy too.

I agree that it is a reasonable price to pay. My understanding is that the value of having a safe & convenient constructor outweighs its computational cost as long as it is reasonable (92ns is) since it is relatively too small compared to the computation time needed for interpolation evaluations anyway.

Interesting.  So that vs in LinearInterp(vs, ranges::Range...) would be on the grid, with the convention that the ranges.... represent the cartesian product.  Are you sure it isn't possible to do LinearInterp with tuples and splat?  That is something like (the untested)
LinearInterp(ranges, vs) = ScaledInterpolation(interpolate(vs, BSpline(Linear()), OnGrid()), ranges...)
xs = 1:.1:10
ys = 1:.5:10
f(x, y) = log(x+y)
A = [f(x,y) for x in xs, y in ys]
LinearInterp((xs, ys), A)

I think it is reasonable to bypass the checks for simplicity.  I like clear code, and the whole thing is kind of unsafe.

Another, fancier, way to do it is to look into ways to represent cartesian products with operator overloading.  We can see if http://juliaapproximation.github.io/ApproxFun.jl/latest/usage/domains.html does anything interesting...  For example, instead of a tuple it might help to be explicit.
f(x, y) = log(x+y)
A = [f(x,y) for x in xs, y in ys]
LinearInterp(ProductDomain(xs, ys), A)
#(or with overloading)
LinearInterp(xs ⊗, ys, A)

Check out https://github.com/econtoolkit/Expectations.jl/issues/9 for some discussion of the generic, abstract types for ranges.... this will matter when it comes to the interface to call with uniform grid methods.

Oh, I see your point; in fact the code you provided works by changing ScaledInterpolation to scale, which takes splat on the second parameter:
LinearInterp(ranges, vs) = scale(interpolate(vs, BSpline(Linear()), OnGrid()), ranges...)

but the challenge here is that Tuple is not a UnionAll type so that when we want to specify the type Range the following line is not going to work with 2-dimensional tuples:
LinearInterp(ranges::Tuple{T}, vs) where T <: Range = print(typeof(ranges))

in fact, one has to define it alternatively as LinearInterp(ranges::Tuple{T,T}, vs) where T <: Range = print(typeof(ranges)), specifying the dimension as well, which is not desirable.
We can, for instance, alternatively take Array instances that are UnionAll as an alternative choice for grids, but it is not trivial on how we should handle the cases when the length for grids in the first and second dimension differ. Looking into other ways to represent cartesian products will be a good idea as well, but after all I think having the parameter for grids located after the value matrices as I suggested still makes sense, except 1D cases (as it is trivial to have it in the order of (x, y)).

On 0.6,
julia> LinearInterp(ranges::NTuple{N,T}, vs) where {N,T <: Range} = print(typeof(ranges))
LinearInterp (generic function with 1 method)

julia> LinearInterp((1:10,2:5), 1)
Tuple{UnitRange{Int64},UnitRange{Int64}}

julia> LinearInterp((1:0.1:10,2:0.1:5), 1)
Tuple{StepRangeLen{Float64,Base.TwicePrecision{Float64},Base.TwicePrecision{Float64}},StepRangeLen{Float64,Base.TwicePrecision{Float64},Base.TwicePrecision{Float64}}}

Still not 100% sure of the right approach.  Regardless of which argument comes first, my feeling is that the ranges really represents some coherent "thing" and that variable argument parameters to the function do not capture that.  Having it as a single object allows us to change behavior later to be more in line with the spirit of http://juliaapproximation.github.io/ApproxFun.jl/latest/usage/domains.html
If what I suggested works, maybe we can get a tuple based approach and then ask for people's feedback.

No problem, I'll add in throwing extrapolations as well and work on the version for cubic spline interpolation! Meanwhile, note that the current version supports both univariate and multivariate cases; unit tests I created support both cases.
And I agree -- I prefer longer names too.

I think this is the right strategy, and lets not overthink the interface.  These were intended to patch up Interpolations.jl to a point where it is usable for our needs, but ultimately it might make sense to build on new interpolations libraries.  Tough to know when the timeline for that new library is feasible, though.

FYI, here are unit tests I have run (included in the PR as well): https://github.com/chiyahn/notes/blob/master/interpolations/multivariate-constructor.ipynb

Great!
The last thing I see is that I think we need a 1D version of the interfaces... the whole tuple of one element thing is ugly.
Maybe add in somethingn like
LinearInterpolation(range::T, vs) where {T <: Range} = LinearInterpolation((range,), vs)

That way you can just go LinearInterpolation(xs, A)

After that, why don't

I think this can be closed now -- the PR (https://github.com/JuliaMath/Interpolations.jl/pull/214) has been merged! 🎉 There are a few changes in README.md for quickstart guide that have to be done (https://github.com/JuliaMath/Interpolations.jl/pull/216), but implementation is now basically complete.

Great!


Use folder-hash instead of hash-files.

Closes #44.
The deprecation warning originated in package hash-files. It was the simplest one to use, but hasn't been maintained since 2016. I switched to folder-hash (https://www.npmjs.com/package/folder-hash), with latest release from January, which seems to be the most popular one now.

Would you care to tag the latest main as v2.1 and then move the v2 tag to that new commit as well?
Remember that it's important to minimize the time the repo is missing a v2 tag, because any actions that use the tag and run in that time will fail.
For best results, I recommend:

make the v2.1 tag locally and push it
delete the v2 tag locally
make the new `v2 tag locally
git push origin <tag-name> --delete && git push origin <tag-name> to replace the v2 tag as quickly as possible

I trust you to do this. If for any reason you'd prefer not to, just let me know and I'll take care of it.

Done! Thank you for the detailed instruction, it was very useful ;)

Happy to help!


Metadata for select form field shows first list item instead of actual value

When providing multiple options for a select dropdown metadata form field the value shown on the metadata view is always the first item in the option list and not the actual value stored as the frontmatter attribute.
This happened regardless of "placeholder" attribute set, in multiple repositories I tested this, and both when specifying an options list above the prose settings, eg
languages: &languages
  - name: 'English'
    value: 'en'
  - name: 'French'
    value: 'fr'

and referring to it in the prose metadata config with
      - name: "lang"
        field:
          element: "select"
          label: "Select language"
          options: *languages

as well when specifying the options straight in the prose config, like
      - name: "lang"
        field:
          element: "select"
          label: "Select language"
          options:
            - name: 'English'
              value: 'en'
            - name: 'French'
              value: 'fr'

(OS Windows 7 Professional, Browser Chrome 49)

Dupe of #882

Also, the placeholder property does nothing. This should either be removed from the documentation, or made to work.