Archive delete tag doesn't work per Microsoft support?

Email in our archive mailboxes is not being deleted after 4 yrs, even though we have an archive delete tag in our policy, have manually run the ManagedFolderAssistant multiple times, and don't have any in-place holds. Opened a case with Microsoft Support and am being told that email in an archive mailbox will never get deleted even if the tag is applied. Shouldn't this info be included in this page discussing retention policy?
[Enter feedback here]

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 81e9c1f9-d22d-8c83-25bb-637f3121ef33
Version Independent ID: b6c1f57d-978f-c959-ef0a-00ef1b9d9cab
Content: Customize an archive and deletion policy (MRM) for mailboxes - Microsoft Purview (compliance)
Content Source: microsoft-365/compliance/set-up-an-archive-and-deletion-policy-for-mailboxes.md
Service: o365-seccomp
GitHub Login: @cabailey
Microsoft Alias: cabailey

@SeaGT  This isn't my understanding ("email in an archive mailbox will never get deleted even if the tag is applied") & also checked with engineering.  Please ask your support engineer to contact me.

This is not a confirmed doc issue, closing.  If updates are identified after the support engineer contacts me, I'll process that internally.

@SeaGT  We have documentation how to move emails to the archive mailbox (https://learn.microsoft.com/microsoft-365/compliance/set-up-an-archive-and-deletion-policy-for-mailboxes) using MRM retention tags and we have documentation how to delete emails after a specified time using M365 retention (https://learn.microsoft.com/microsoft-365/compliance/create-retention-policies&tabs=other-retention#create-and-configure-a-retention-policy).
I don't have confirmation from the engineering team that owns the features that the retention tag will prevent deletion (my understanding of what you're saying).  I can't update the documentation from anecdotal reports, because it could be a bug that needs fixing (or misconfiguration or missed prereqs).  I will look into this if the support engineer who gave you this statement contacts me, but I have yet to hear from anybody to get the details.
I also haven't heard of users not being able to delete emails from their auto-expanding archive mailboxes. Do you have a support ticket for this separate issue?  We document users can't delete the folders, and the Recover Deleted Items feature isn't supported, but we specifically document "Users can delete items in an auto-expanded storage area": https://learn.microsoft.com/microsoft-365/compliance/autoexpanding-archiving#outlook-requirements-for-accessing-items-in-an-auto-expanded-archive

Hi Carol,
Thanks for following up.
I agree that you’ve got those 2 on articles retention (I’ve been through them multiple times😉), what I’m asking for is an additional paragraph in there that tells the reader that the archive mailbox will NOT have items deleted from it unless you use one of these options:

use a Purview deletion policy (the solution I used)
create Outlook rules and run them on the Archive mailbox
use the Outlook AutoArchive function
create a personal tag and apply that to the archive mailbox (this is untested – it was suggested by the support engineer Jameson but not confirmed / Jameson is CC’d on this email)
A paragraph along these lines would save so much time and stress for your users.

For the auto-expanding archive mailbox issue, I was reading this:
If an archive mailbox has at least one auto-expanded storage area, users can't delete a folder from the archive mailbox or from the auxiliary archive. In other words, after an auto-expanded storage area has been provisioned, they can't delete any folders in the archive.
Learn about auto-expanding archiving - Microsoft Purview (compliance) | Microsoft Learnhttps://learn.microsoft.com/en-us/microsoft-365/compliance/autoexpanding-archiving?view=o365-worldwide#outlook-requirements-for-accessing-items-in-an-auto-expanded-archive
You’re correct about deleting emails (I think, but I haven’t play with the AEAM feature yet).  In the same Learn article, it mentions search being trickier in AEAM and that some folder numbers may be incorrect – so we’re not going to try AEAM until Version 2 is out.
Again, thanks for your time.
Greg
<EMAIL_ADDRESS>Desk:<PHONE_NUMBER> | IT Support:<PHONE_NUMBER>
“A pessimist sees the difficulty in every opportunity, an optimist sees the opportunity in every difficulty” Winston Churchill
From: Carol Bailey @.>
Sent: Monday, March 20, 2023 12:20 PM
To: MicrosoftDocs/microsoft-365-docs @.>
Cc: Greg Trapp @.>; Mention @.>
Subject: Re: [MicrosoftDocs/microsoft-365-docs] Archive delete tag doesn't work per Microsoft support? (Issue #11259)
@SeaGThttps://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FSeaGT&data=05|01|gregt%40preceptwine.com|2ae81ad7942946b3dd5908db297826ff|dbec7e540e5b462e97d88a247cc839c0|0|0|638149368252144268|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000|||&sdata=GCPwVPvtQUuvkA2LoxmKhfoiAE5dbKJyjQrC4QMmpXY%3D&reserved=0 We have documentation how to move emails to the archive mailbox (https://learn.microsoft.com/microsoft-365/compliance/set-up-an-archive-and-deletion-policy-for-mailboxeshttps://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Flearn.microsoft.com%2Fmicrosoft-365%2Fcompliance%2Fset-up-an-archive-and-deletion-policy-for-mailboxes&data=05|01|gregt%40preceptwine.com|2ae81ad7942946b3dd5908db297826ff|dbec7e540e5b462e97d88a247cc839c0|0|0|638149368252144268|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000|||&sdata=aw7R3lMJuyicjK2aczgafhBioNCAChxReqdjpL%2Bz2Wo%3D&reserved=0) using MRM retention tags and we have documentation how to delete emails after a specified time using M365 retention (https://learn.microsoft.com/microsoft-365/compliance/create-retention-policies&tabs=other-retention#create-and-configure-a-retention-policyhttps://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Flearn.microsoft.com%2Fmicrosoft-365%2Fcompliance%2Fcreate-retention-policies%26tabs%3Dother-retention%23create-and-configure-a-retention-policy&data=05|01|gregt%40preceptwine.com|2ae81ad7942946b3dd5908db297826ff|dbec7e540e5b462e97d88a247cc839c0|0|0|638149368252144268|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000|||&sdata=oS0NwEHhteGjud3cbSwY8yQO0oLGBIpLG6jMP0SpZVg%3D&reserved=0).
I don't have confirmation from the engineering team that owns the features that the retention tag will prevent deletion (my understanding of what you're saying). I can't update the documentation from anecdotal reports, because it could be a bug that needs fixing (or misconfiguration or missed prereqs). I will look into this if the support engineer who gave you this statement contacts me, but I have yet to hear from anybody to get the details.
I also haven't heard of users not being able to delete emails from their auto-expanding archive mailboxes. Do you have a support ticket for this separate issue? We document users can't delete the folders, and the Recover Deleted Items feature isn't supported, but we specifically document "Users can delete items in an auto-expanded storage area": https://learn.microsoft.com/microsoft-365/compliance/autoexpanding-archiving#outlook-requirements-for-accessing-items-in-an-auto-expanded-archivehttps://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Flearn.microsoft.com%2Fmicrosoft-365%2Fcompliance%2Fautoexpanding-archiving%23outlook-requirements-for-accessing-items-in-an-auto-expanded-archive&data=05|01|gregt%40preceptwine.com|2ae81ad7942946b3dd5908db297826ff|dbec7e540e5b462e97d88a247cc839c0|0|0|638149368252144268|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000|||&sdata=51Lw7LV%2FrdTj%2FOwqttAO%2BypbB6EY3Pg74Uxxk%2Fd%2BqSo%3D&reserved=0
—
Reply to this email directly, view it on GitHubhttps://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FMicrosoftDocs%2Fmicrosoft-365-docs%2Fissues%2F11259%23issuecomment-1476806724&data=05|01|gregt%40preceptwine.com|2ae81ad7942946b3dd5908db297826ff|dbec7e540e5b462e97d88a247cc839c0|0|0|638149368252144268|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000|||&sdata=LcTcSszbJ1DdDhgXhXMDrA1X%2FbgVblRypzsQwRupFF8%3D&reserved=0, or unsubscribehttps://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAFZCZXHSX2LWTF66X45QIDTW5CUXNANCNFSM6AAAAAAVS2HTR4&data=05|01|gregt%40preceptwine.com|2ae81ad7942946b3dd5908db297826ff|dbec7e540e5b462e97d88a247cc839c0|0|0|638149368252144268|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000|||&sdata=KJ9L2fQUJni6Fs4lyNmfhe%2B%2FNJ2iiFEZX2FwDnPhlNU%3D&reserved=0.
You are receiving this because you were mentioned.Message ID<EMAIL_ADDRESS>

@SeaGT I do not see the need for a doc update, here. Right at the beginning of the article we cover this very point:
"To retain and delete emails, we recommend you use Microsoft 365 retention policies and retention labels] rather than the older messaging records management (MRM) from Exchange Online. However, a valid reason to still use this older feature is to automatically move emails from a user's primary mailbox to their archive mailbox. You might also need to use MRM to apply retention and deletion settings to specific folders in the mailbox, rather than the entire mailbox.
Use this article as an example deployment for these two valid reasons to use MRM retention policies and retention tags. For all other retention and deletion scenarios, use Microsoft 365 retention policies and retention labels."
Although other methods might work to delete the emails, we always recommend M365 retention policies/labels & keep MRM tags for just the 2 purposes documented.  This guidance is consistently repeated in other places, for example https://learn.microsoft.com/en-us/microsoft-365/compliance/retention?view=o365-worldwide#use-retention-policies-and-retention-labels-instead-of-older-features and the Note at the top of the MRM documentation such as the one on https://learn.microsoft.com/en-us/microsoft-365/compliance/retention?view=o365-worldwide#use-retention-policies-and-retention-labels-instead-of-older-features


Better staging

expected behavior
When staging files with tools4zettelkasten stage --fully the program will only change those  files that do not yet confirm to a standard file name. The Program will change the core filename  and add the ordering name and the identifier if necessary.
observed behavior
When staging files with tools4zettelkasten stage --fully the process will overwrite filenames that have been updated already. Thus the identifier gets changed which might result in dangling references during staging.

Fixed


update to Kubernetes 1.12.3

Towards https://github.com/giantswarm/giantswarm/issues/4709.

As you can see build fails with:
# github.com/giantswarm/helmclient/vendor/k8s.io/helm/cmd/helm/installer
vendor/k8s.io/helm/cmd/helm/installer/uninstall.go:55:15: undefined: kubectl.ReaperFor

This is effectively blocking whole<EMAIL_ADDRESS>libs upgrade. See https://github.com/giantswarm/giantswarm/issues/4809. I'll try to create a fork.


Old Graviton lens consumption in Ray Receivers

I noticed the recipe calls for 0.25 per minute instead of 0.1
They reduced the lens cost a few updates back.

Thanks for pointing this out! I should be able to get this fixed easily.


TextBounds update after parent scale

Hello.
How to set correct text bounds after sprite scale?  Example  , https://phaser.io/sandbox/edit/OvGUDREg, button with label . Text bounds not working after scale parent container.

I'm not sure. textBounds works by altering pivot. https://codepen.io/samme/pen/OWYYEM
My first guess would be
text2.setTextBounds(0, 0, button2.texture.frame.width, button2.texture.frame.height);

Not working.
Updated example https://phaser.io/sandbox/edit/onWWwRUZ
If add text to scaled sprite and set corect textBounds.


Bottom Toolbar setting missing on tablet

Perhaps by design, but on my tablet (but no on my phone) the option to move the toolbar to the bottom is missing in settings
Samsung Tab S7
Android 12
98.0.4758.106

Th8s setting 8s only for phones as 9n tablets the tabbed toolbar is use, not the mobile toolbar. If it's a feature request you should change the bug label to feature request

This setting is only for phones as on tablets the tabbed toolbar is used, not the mobile toolbar. If it's a feature request you should change the bug label to feature request

Thanks for the info! I cannot change it to a feature request though


Category improvements

This PR makes a handful of changes based on requests and feedback from #866 and #904. I've also made a PR over at https://github.com/TypeStrong/typedoc-default-themes/pull/61 to make categories more integrated into the theme.
Thanks to @rbuckton for the suggestions. I used some of the ideas from your implementation in your custom plugin.
Group Categories
Categories can now be used within groups or as lump categories. Lump categories mean if you have a method and property you want together, you can do so. The option --categorizeByGroup controls this. By default, reflections will be categorized by group (within properties, methods, etc).
Default Category
Uncategorized reflections will be placed in the default category. If everything falls into the default category, categories aren't used. The option --defaultCategory controls this. By default, the default category will be 'Other'.
Category Ordering
Categories can be ordered however the user wants. Using an array, the user can specify the order categories are displayed on their pages. The option --categoryOrder controls this. By default, they will be ordered alphabetically. An entry of '*' indicates where categories not in this list will be rendered, defaulting to the end.

Hi,
When will this merge be released?


WIP:BUG-119222 : Support for registering CM based cluster

Created FlowManger to initialize and execute flow
Created a Client for Cloudera manager
abstracted the flow between 2 cluster through BaseOperationsManager

Closing this pull request. Will raise a fresh pull request


Global place: Command terminated by signal 11

Describe the bug
This is from the MegaBoom design. Very slightly changed initial conditions and global place is now crashing.

untar https://drive.google.com/file/d/1ZHu7RxqYa_1FMVhqgps9Uxr3LPzZJA0l/view?usp=sharing
./run-me-ChipTop-asap7-flat.sh

OpenROAD v2.0-11865-g6471c0995
This program is licensed under the BSD-3 license. See the LICENSE file for details.
Components of this program may be licensed under more restrictive licenses which must be honored.
global_placement -density 0.25 -pad_left 2 -pad_right 2 -routability_driven -timing_driven
[INFO GPL-0002] DBU: 1000
[INFO GPL-0003] SiteSize: 54 270
[INFO GPL-0004] CoreAreaLxLy: 2052 2160
[INFO GPL-0005] CoreAreaUxUy: 1497960 1497960

[deleted]
[NesterovSolve] Iter: 400 overflow: 0.400787 HPWL:<PHONE_NUMBER>2
[NesterovSolve] Iter: 410 overflow: 0.353168 HPWL:<PHONE_NUMBER>4
[NesterovSolve] Iter: 420 overflow: 0.305717 HPWL:<PHONE_NUMBER>3
[INFO GPL-0100] worst slack -3.76e-07
[INFO GPL-0103] Weighted 174499 nets.

[INFO GPL-0075] Routability numCall: 1 inflationIterCnt: 1 bloatIterCnt: 0
./run-me-ChipTop-asap7-flat.sh: line 7: 1596838 Segmentation fault      (core dumped) openroad -no_init ${SCRIPTS_DIR}/global_place.tcl

Expected Behavior
No crash
Environment
OpenROAD v2.0-11865-g6471c0995

To Reproduce
See above
Relevant log output
No response
Screenshots
No response
Additional Context
Will try to whittle down...
openroad -python ~/OpenROAD-flow-scripts/tools/OpenROAD/etc/deltaDebug.py --start 16 --persistence 3 --use_stdout --error_string "Segmentation fault" --base_db_path bazel-bin/results/asap7/ChipTop/flat/3_2_place_iop.odb --step ./run-me-ChipTop-asap7-flat.sh

@eder-matheus It might be in routability, its hard to tell from the log.  Please take a look

@eder-matheus I have a deltaDebug.py running, it will hopefully reduce the reproduction time from hours to minutes... Stay tuned...
$ openroad -python ~/OpenROAD-flow-scripts/tools/OpenROAD/etc/deltaDebug.py --persistence 3 --use_stdout --error_string "Segmentation fault" --base_db_path bazel-bin/results/asap7/ChipTop/flat/3_2_place_iop.odb --step ./run-me-ChipTop-asap7-flat.sh
OpenROAD v2.0-11865-g6471c0995
This program is licensed under the BSD-3 license. See the LICENSE file for details.
Components of this program may be licensed under more restrictive licenses which must be honored.
[WARNING ORD-0039] .openroad ignored with -python
Backing up original base file.
Performing a step with the original input file to calculate timeout.
Error Code found: Segmentation fault
Step 1, Insts level debugging, Insts 1907862, Nets 1750112, cut elements 953931, timeout 108 minutes
[C#]
...

I have it running in the debugger now as well as trying to whittle down the test case. Whittling down the test case has prove very, very tough(90 minutes for one iteration and it has been running for days...).
It hasn't crashed in the debugger yet, it is running for a long time, but I thought I would attach a screenshot with a stacktrace.

[INFO GPL-0075] Routability numCall: 1 inflationIterCnt: 1 bloatIterCnt: 0
[no further output before the Segfault]

Suspending while it is running here, gives me this stack trace.

@eder-matheus 9 hours later in debugger...
What can I check here?

Something is off here. From what I see, the function mazeRouteMSMD is called, but it should not run inside routability-driven global placement, as we don't want to run the overflow iterations there. I will check the code and try running the test case you provided to see if my changes can fix it.

Wondering around the code... Is this right? Can't this overflow, in theory?
https://github.com/The-OpenROAD-Project/OpenROAD/blob/c1fc5f730a59874a1b9735be82d267da8417c6e7/src/grt/src/fastroute/src/utility.cpp#L67

@eder-matheus Thoery: the large clock tree in MegaBoom has enough pins that some static limit is overflowing. This would make it impossible to whittle down the example.

@oharboe This makes sense. I'll try to get rid of these static limits. I thought I'd removed all of them in the past, but we have a few ones hanging in the code.

@oharboe This makes sense. I'll try to get rid of these static limits. I thought I'd removed all of them in the past, but we have a few ones hanging in the code.

Very good. I'm glad this issue is unstuck now at least ...
The whittling down is not going to work.
#4571 is a "success", lots of failures in fast tests...

This is a good start! Thanks for the help debugging it. I will start the work to remove the static limits today and hopefully will have the fix completed soon.

@oharboe With this PR https://github.com/The-OpenROAD-Project/OpenROAD/pull/4591, I could complete the placement stage in the test case you provided. I search the code for other large arrays and turn them all into vectors, so GRT should be safe now regarding out-of-stack issues.

@eder-matheus Please go ahead and merge. If there are further fixes needed, we can do those in later PRs.


Interactive Chart API not returning Data for hour or minute requests.

The following url returns empty data from the API:
http://dev.markitondemand.com/MODApis/Api/v2/InteractiveChart/json?parameters={"DataInterval"%3A+5%2C+"Elements"%3A+[{"Symbol"%3A+"aapl"%2C+"Type"%3A+"price"%2C+"Params"%3A+["ohlc"]}]%2C+"Normalized"%3A+false%2C+"DataPeriod"%3A+"Minute"%2C+"EndOffsetDays"%3A+100%2C+"NumberOfDays"%3A+20}
This is the corresping code for the request (using Python requests):
    elements = [{
    'Symbol': 'aapl',
    'Type':'price',
    'Params': ["ohlc"]
     }]

    int_chart_json = {
        'EndOffsetDays': 100,
        'NumberOfDays': 20,
        'Normalized': False,
        'DataPeriod' : 'Minute',
        'DataInterval': 5,
        'Elements': elements
        }
my_params = {'parameters': json.dumps(int_chart_json)}

When I change DataPeriod to 'Day' and 'DataPeriod' to '0', it works as expceted.
Am I missing something?

We are currently limited to providing end of day quotes for free.  I think originally we were allowed to distribute interday from BATS but that is no longer the case.


Update builtin.c

is this correct?
( and i didn't find "logo_builtin.h" file, where is it? )

I just updated it myself.

Would you like to show the config file you use?

Thanks for your update, and yes SURE, of course i'd very appreciate to share my Config-File

here i copied for you
{
"logo": {
    "source": "arch",
    "color": {"1": "red", "2": "yellow"}
    },
"display": {
    "color": {"separator": "blue", "output": "white"},
    "separator": " | "
    },
"modules": [
    {"type": "kernel", "key": " /\\rch Linux", "keyColor": "magenta"},
    {"type": "custom", "format": ">-----------<+>----------------------------------<", "outputColor": "blue"},
    {"type": "uptime", "key": "   Uptime   ", "keyColor": "green"},
    {"type": "shell", "key": "   Shell    ", "keyColor": "green"},
    {"type": "terminal", "key": "   Terminal ", "keyColor": "green"},
    {"type": "terminalfont", "key": "   Font     ", "keyColor": "green"},
    {"type": "packages", "key": "   Packages ", "keyColor": "green"},
    {"type": "localip", "key": "   Local IP ", "keyColor": "green"},
    {"type": "custom", "format": ">-----------<+>----------------------------------<", "outputColor": "blue"},
    {"type": "display", "key": "   Display  ", "keyColor": "cyan"},
    {"type": "cpu", "key": "   CPU      ", "keyColor": "cyan"},
    {"type": "gpu", "key": "   GPU      ", "keyColor": "cyan"},
    {"type": "memory", "key": "   RAM      ", "keyColor": "cyan"},
    {"type": "swap", "key": "   SWAP     ", "keyColor": "cyan"},
    {"type": "disk", "key": "   Disk     ", "keyColor": "cyan"},
    {"type": "battery", "key": "   Battery  ", "keyColor": "cyan"},
    {"type": "custom", "format": ">-----------<+>----------------------------------<", "outputColor": "blue"},
    "break",
    {"type":"colors", "paddingLeft": 9}
    ]
}

Did a little cleanup
https://github.com/fastfetch-cli/fastfetch/blob/dev/presets/examples/22.jsonc

Whoa, that's Amazing
And very Thanks for you selected my-config as official-example
( i'am so Happy that i may see the change sooner in next Arch Linux package update )

i see it, i just updated my Arch Linux now, and So thanks to you that i can directly use arch3 as official-builtin-logo right now


Hypixel uses multiple IPs for their proxies

https://www.whatsmydns.net/#A/mc.hypixel.net
Hypixel uses many IP addresses for their proxies, not just one, so swapped to a "startsWith" to allow for all 255 addresses Hypixel could use.

Hey, thanks for the suggestion. We're currently working on a beta version, and we switched to a fully universal detection method: https://github.com/Semx11/Autotip/blob/autotip-2.1/src/main/java/me/semx11/autotip/event/EventClientConnection.java#L41-L68
If we kept the same detection method I would've accepted your PR, but we are using a different method now. Thank you for the suggestion though!


Unload module during module load test

Addresses #43 .  This doesn't fix the underlying issue, but it makes safeload work again.

i guess this one is redundant now. Closing


Core Rename transfer to to write to on responses.

Rename transferBody/ValueTo to writeBody/ValueTo

API change check
APIView has identified API level changes in this PR and created following API reviews.
azure-core


Roll engine 26b30a464684..d2703b4eeb70 (2 commits)

https://github.com/flutter/engine/compare/26b30a464684...d2703b4eeb70
git log 26b30a464684e7cbc1c76a98587154b9edb0c7a9..d2703b4eeb70b8c16c38662d3c97e5030f2306a4 --no-merges --oneline
d2703b4ee Roll src/third_party/skia a1ab899e9e3f..2444c0e9aa23 (2 commits) (flutter/engine#8739)
0f47ab665 Roll src/third_party/skia 46d0f9aad1e6..a1ab899e9e3f (2 commits) (flutter/engine#8737)
The AutoRoll server is located here: https://autoroll.skia.org/r/flutter-engine-flutter-autoroll
Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md
If the roll is causing failures, please contact the current sheriff (liyuqian@google.com), and stop
the roller if necessary.

Trybots failed. These were the failed builds: https://cirrus-ci.com/task/6093368802148352

Commit queue failed; closing this roll.


Configuring External Auth Provider: Add warning regarding local user mapping

Internal Reference: SURE-7301

When a local auth provider admin configures an external auth provider an external auth provider user is requested and authenticated
The provided user is linked to a new admin principal
We need to make it clear this happens and why
Covers all external auth providers

Current docs https://ranchermanager.docs.rancher.com/how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/authentication-config

We should add a banner to the provide config page, with similar text to the docs - i.e.
The account used to enable the external provider will be granted admin permissions. If you use a test account or non-admin account, that account will still be granted admin-level permissions. See External Authentication Configuration and Principal Users to understand why.

External Authentication Configuration and Principal Users should be a link to https://ranchermanager.docs.rancher.com/v2.8/how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/authentication-config#external-authentication-configuration-and-principal-users and should open in a new tab - note v2.8 should be replaced with the correct docsBase url as we do for other docs links.

QA suspects that this is easily automatible.

We already have a banner on the bottom of the page for all the auth providers:
associatedWarning: 'Note: The {provider} user you authenticate as will be associated as an alternate way to login to the {vendor} user you are currently logged in as <code>{username}</code>; all the global permissions, project, and cluster role bindings of this {vendor} user will also apply to the {provider} user.'

As discussed with @nwmac the new message will replace the old one, and should go on top of the page as a warning(yellow) banner.

@nwmac the old message was shown only when the authentication provider was NOT enabled, should the new message follow the same logic, or should it always be displayed?
Also the new message uses a past tense ("The account USED to enable ..."), that might sound incorrect if you're just about to enable the auth provider.

Yeah, I think when only not enabled is fine.
I think the grammar is okay - maybe if it was 'The account that is used' is clearler

Checking the test it seems it's looking for the element existence. An is visible test could complement the coverage. What's your opinion @yonasberhe23

good call out @izaac. can you add this check @momesgin?

good call out @izaac. can you add this check @momesgin? once that's done i think we can move this to Done

sure, I'll work on it


CLI Templates

What is the new topic you would like to added to the OpenShift cookbook?
This topic should handle several oc get -o ...  snippets.
If this is to help you solve a specific problem, can you describe that problem?
The problem is that the output of oc get -o wide|name is not very easy to handle in further processes/pipes. Because of that a snippet collection with some suggestion how to handle jsonpath or golang text template would be very helpful.

Duplicate of https://github.com/openshift-evangelists/openshift-cookbook/issues/3


Intermittent ansible bug resolving temporary directory

failed to resolve remote temporary directory from ansible-tmp-1462528250.89-203048679532712: `( umask 22 && mkdir -p \"` echo $HOME/.ansible/tmp/ansible-tmp-1462528250.89-203048679532712 `\" && echo \"` echo $HOME/.ansible/tmp/ansible-tmp-1462528250.89-203048679532712 `\" )` returned empty string"}

Tracking this upstream - https://github.com/ansible/ansible/issues/13876

Doing an ansible upgrade to 2.0.2 results in this error -

TASK [Wait for port 22 to become available from local server.] *****************
fatal: [kube-worker-0]: FAILED! => {"failed": true, "msg": "'ansible_port' is undefined"}


Conversation history - limit the number of previous messages included in context

Within the Azure OpenAI Studio, you can configure the number of "past messages included" within a conversation.

I've been looking through both the python (/app.py) and typescript (/frontend/src/api/api.ts, frontend/src/pages/chat/Chat.tsx) code, but I can't seem to find any configuration in this sample application that limits the number of previous chat messages within a conversation that are sent to the model as context.
Are all previous messages of a conversation sent into the model, up to the input token limit? And if/when that limit is reached, how does the application handle that?
Thank you!

Hi @RylandDeGregory, yes, all previous messages of the conversation are sent into the model. When the limit is reached, the API will return an error which will surface in the app. Labeling this as an enhancement, it would be a good feature to add a configuration setting for this.

Thank you, @sarah-widder! I appreciate the response and agree that it would be a great enhancement.

A simple but good implementation for this might be to count the tokens of the context messages before they are sent, and when the threshold of e.g. 8000 is exceeded, the oldest user and assistant messages are deleted (but the system messages are kept). As some messages are shorter and some longer, it is difficult to find a good number of messages to keep in context.
Tiktoken from Openai can be used for the token count with the cl100k_base for gpt-4 and gpt-3.5-turbo models.  [https://github.com/openai/tiktoken]


Support passing signals to island props

I think it would be really cool if you could wire together two islands using a Preact signal (but that signal is not global). To make this work you need to be able to pass signals to island props. For that to work we need to be able to serialize and deserialize signals that are passed into props.
Usage example:
export default IndexPage() {
  const cart1 = useSignal([]);
  const cart2 = useSignal([]);
  return (

Cart 1

      <Cart cart={cart1} />
      <AddToCart cart={cart1} product="Lemon" />

Cart 2

      <Cart cart={cart2} />
      <AddToCart cart={cart2} product="Lemon" />

  );
}

To make this work there needs to be one hard constraints on the passed signals: the signal's value must contain a JSON serializable.
For an initial pass there would likely be the additional constraint that the passed signal must be passed directly as a prop (ie that it can not be contained within a data structure like an object or array).
The way this would work is that during the server-side render we'd find the island props that look like signals, and serialize them into the serialized props as an opaque pointer corresponding to the signal identity. We'd then additionally serialize an structure containing mappings from signal identity (opaque pointer) to signal value. On the client we can then use this data to reconstruct live signals with the right values and pass them in the right island prop slots by cross referencing the island identity using the opaque pointer.

https://user-images.githubusercontent.com/7829205/189522395-70747397-3ea4-4ed6-bc82-9d5e45f35061.mov


Unexpected behavior for cloned read-only IonStructs when shared across multiple threads

Version Impacted
This issue exists in 1.10.5 but was not in 1.9.X.  I am not sure which is the first effected version.
Current Behavior
Accessing members of a cloned, read-only struct with more than 5 members from multiple threads is non-deterministic.
Expected Behavior
Accessing members of the cloned read-only struct should always return the value.
Example
The following test fails where I'd expect it to pass.
@Test
public void readOnlyIonStructMultithreadedTest() {
    String ionStr = "{a:1,b:2,c:3,d:4,e:5,f:6}";
    //If the stuct has 5 or fewer members, the test will pass
    //String ionStr = "{a:1,b:2,c:3,d:4,e:5}";

    IonStruct ionValue = (IonStruct)systemA.singleValue(ionStr);
    ionValue.makeReadOnly();

    for (int i=0; i<100; i++) {

        final int iteration = i;
        IonStruct clone = ionValue.clone(); //Removing the clone and just using ionValue causes the test to pass
        clone.makeReadOnly();
        //Adding the below line causes the test to pass
        //assertNotNull(clone.get("a"));

        List<CompletableFuture<Void>> waiting = new ArrayList<>();
        for (int j = 0; j < 4; j++) {
            waiting.add(CompletableFuture.runAsync(() -> {
                for (int k = 0; k <= 100; k++) {
                    assertNotNull(clone.get("a"));
                }
            }));
        }
        waiting.forEach(cf -> cf.join());
    }
}

Additional Context
Removing ionValue.clone() and just passing around ionValue makes the issue disappear.
The issue disappears if the struct has 5 or fewer members.
The issue also disappears if a member of the struct is accessed before the struct is passed to another thread.

Thanks for bringing this to our attention. It looks like this was introduced in https://github.com/amazon-ion/ion-java/commit/559ad54bc356b448c88d46e160c322d115397d20.
The build_field_map() assigns an empty map to the _fields_map field, and then populates it with values. This can result in a race condition where the fields map is active (i.e. not null) before it is actually populated, and attempting to look up any value may fail. We'll need to look into this more to determine the solution.

Another possible solution that would probably meet user expectations around concurrency would be to initialize the _fields_map when makeReadOnly() is called so that there's no internal mutability remaining in the IonStruct.

That would be my preference, since the JavaDoc states after makeReadOnly() the object is immutable and safe to share across threads.


Unify the Chinese translation of “namespace”

There are currently two Chinese translations of namespace: "名字空间" and "命名空间".

I propose to standardize the translation to "命名空间",  because "命名空间" is more widely used in books, blog posts and other Chinese publications.

@tengqm

/language zh
/kind cleanup

'namespace' prefer to '名字空间' cc @tengqm

There has been some discussion about this before, and @RifeWang you can see the following links:

https://github.com/kubernetes/website/issues/28459
https://github.com/kubernetes/website/issues/37694
https://github.com/kubernetes/website/pull/37665


Remove public_network_access_enabled from lifecycle block => Fixes deprecation warning during plan.

Describe your changes
The fix to remove public_network_access_enabled in 8.0.0 was incomplete. Leaving the property in the lifecycle {} block still causes the warning.
Issue number
#526
Checklist before requesting a review

[X ] The pr title can be used to describe what this pr did in CHANGELOG.md file
[X ] I have executed pre-commit on my machine
[X ] I have passed pr-check on my machine

Thanks for your cooperation!

Thanks @JaredDarling for opening this pr! We left this public_network_access_enabled in ignore_change list since we've added it to the resource block before, we want to avoid potential configuration drift by doing this, sorry about the warning, we'd like to remove all deprecated arguments when AzureRM 4.0 has been released. Thanks for your contribution!

Hello. It is intentional to keep that value in the lifecycle block.
To avoid problems with existing terraform states that contain public_network_access_enabled.

I'm closing this pr since it's by design. Please feel free to reopen it if you have different thought @JaredDarling . Thanks for your contribution.


Docker images for 0.8.3 are missing

Hi,
I looks like the tags for 0.8.3 are missing on the docker hub. Are they available somewhere else?

Ah, I should've probably opened an issue instead of this: https://discourse.drone.io/t/drone-0-8-3-docker-image/1177/3

Looks like the tag 0.8.2 is in reality 0.8.3 now: https://beta.drone.io/drone/drone/1356/11

@bradrydzewski maybe you should use the new auto tag :)


Switching to live overhaul

Context
The "Switching to live" section needed a lot of work and updating.
Changes proposed in this pull request
Changes and corrections to the "Switching to live" section.
Guidance to review
I've pushed these changes to the test environment at https://govukpay-docs-test.cloudapps.digital/switching_to_live/ so reviewers can see it how it will appear in situ.
I will leave this PR open for at least a couple of days for the GOV.UK Pay team to have the opportunity to review it and let me know of any corrections and clarifications etc. I should also note that any and all feedback is welcome. @heathd and @katiebates have already reviewed separately, as have @karlchillmaid and @mpwoods at different points.
Please note that the GDS tech docs tool still has some limitations despite recent significant improvements: for large pieces of content like this it is still not perfect, but very few tech docs tools are.
Please also note that even if further clarifications etc. are needed after this is approved/published etc. it will be easy to iteratively do so.

@heathd I don't see why this isn't building via CI 🤔

Aside from my comment here FAO @katiebates https://github.com/alphagov/pay-tech-docs/pull/152/files#r215992298 I'm happy with this now.

"When you sign up for GOV.UK Pay, the team will contact you with specific contact details." This refers to the emergency contact details that Pay gives, but it's not super clear that's what it's about. Maybe something like "When you sign up for GOV.UK Pay, the team will give you emergency contact details" would be a little clearer?

Do we want to add a note about checking card types enabled, checking email settings (for emails to be on/off, and there'll be future options), billing address settings (when we add them)? When we talk through with a service the switch to live process, we often talk them through at least card types. But also it should also be pretty-self explanatory in self-service, and has no dependencies on other suppliers that a service would need to know about (which is the case for some of the 3DS configurations).. so maybe it's fine without?

In the "Request a live GOV.UK Pay account" section, we could also ask services to also supply us with the service name and what type of PSP account. We handle this at the moment by basically being in touch with the service, but it would be a bit simpler to have them send it to us. This is the info we currently need when setting up a live account: https://pay-team-manual.cloudapps.digital/support/creating-gateway-accounts/#creating-a-gateway-account

"Test your configuration" section for Smartpay + ePDQ describes testing using payment links; "How to check Worldpay settings" is the new section we wrote, which describes using API or payment links.

@katiebates

and 3. No problem, will change shortly

We could perhaps add this in latterly: might be easier to review in a different pull request?

Ah this is my oversight, Worldpay has diverged a little. I'll change Smartpay and ePDQ to match Worldpay sections shortly

@katiebates I've added information which I think covers your points 1, 3, and 4. You should see this now at https://govukpay-docs-test.cloudapps.digital/switching_to_live/#switching-to-live
Point 2 I think may be more work: perhaps we could pick that up after the 24th?


Add config for API Server

Can you add a configuration for the API Server address? So that we can use some third-party OpenAI API proxy services.

Yes, that would be very helpful. Thank you.

And nice to have the choice of models, such as the cheaper GPT-3.5 or the advanced GPT-4 Turbo. It allows users to select the model that best fits their needs.

Yes I'll support this as well

You can change the model and API server in options page now.


[BUG]: RLS Policies not applied with push but applied with migrate

Report hasn't been filed before.

[x] I have verified that the bug I'm about to report hasn't been filed before.

What version of drizzle-orm are you using?
0.36.0
What version of drizzle-kit are you using?
0.27.2
Other packages
No response
What is the undesired behavior?
Using drizzle-kit push does not apply the RLS policy SQL statements correctly. However, using drizzle-kit generate and drizzle-kit migrate does apply these policies as expected.

Command
Supabase Evidence

After running drizzle-kit push

After running drizzle-kit generate and drizzle-kit migrate

What are the steps to reproduce it?

I'm using Supabase with Postgres v<IP_ADDRESS>
I don't think this bug pertains to a driver, see below.
I don't think this bug is related to the runtime, see below.
I'm working on a monorepo, but I isolated this issue in a fresh Supabase project and database, and on a separate repo.

Minimal reproducible example
package.json
{
  "name": "minimal-reproducible-example",
  "dependencies": {
    "drizzle-kit": "^0.27.2",
    "drizzle-orm": "^0.36.0",
    "pg": "^8.13.1"
  }
}

index.ts
import { sql } from "drizzle-orm";
import { authenticatedRole } from "drizzle-orm/supabase";
import { pgPolicy, pgTable, uuid } from "drizzle-orm/pg-core";

export const table = pgTable(
  "table",
  {
    id: uuid().unique().defaultRandom().primaryKey(),
  },
  (_) => [
    pgPolicy("policy", {
      as: "permissive",
      to: authenticatedRole,
      for: "all",
      using: sql`true`,
      withCheck: sql`true`,
    }),
  ],
);

drizzle.config.ts
import { type Config } from "drizzle-kit";

export default {
  schema: "./src/schema/index.ts",
  dialect: "postgresql",
  out: "./src/migrations",
  strict: true,
  entities: {
    roles: {
      provider: "supabase",
    },
  },
  migrations: {
    prefix: "supabase",
    table: "journal",
    schema: "drizzle",
  },
  dbCredentials: {
    url: HARDCODED_DATABASE_URL,
  },
  casing: "snake_case",
} satisfies Config;

What I've tried to fix it
1. Use the link API.
index.ts
import { sql } from "drizzle-orm";
import { authenticatedRole } from "drizzle-orm/supabase";
import { pgPolicy, pgTable, uuid } from "drizzle-orm/pg-core";

export const table = pgTable("table", {
  id: uuid().unique().defaultRandom().primaryKey(),
}).enableRLS();

export const policy = pgPolicy("policy", {
  as: "permissive",
  to: authenticatedRole,
  for: "all",
  using: sql`true`,
  withCheck: sql`true`,
}).link(table);

2. Use different runtimes:

Node v23.1.0, npm v10.9.0
Deno v2.0.5
Bun v1.1.34
pnpm v9.12.3

3. Use a different driver:
package.json
{
  "name": "minimal-reproducible-example",
  "dependencies": {
    "drizzle-kit": "^0.27.2",
    "drizzle-orm": "^0.36.0",
    "postgres": "^3.4.5"
  }
}

4. Remove the entities property in drizzle.config.ts:
drizzle.config.ts
import { type Config } from "drizzle-kit";

export default {
  schema: "./src/schema/index.ts",
  dialect: "postgresql",
  out: "./src/migrations",
  strict: true,
  migrations: {
    prefix: "supabase",
    table: "journal",
    schema: "drizzle",
  },
  dbCredentials: {
    url: HARDCODED_DATABASE_URL,
  },
  casing: "snake_case",
} satisfies Config;

5. Upgrade Postgres version:

Postgres v<IP_ADDRESS>

What is the desired result?
I expected drizzle-kit push to apply all Row-Level Security (RLS) policies correctly, similar to the behavior of drizzle-kit generate and drizzle-kit migrate. After reviewing the RLS documentation, the drizzle-kit push documentation and the release notes, I found no indications of limitations or differences in how drizzle-kit push handles RLS policies.
Additional info
push output
❯ npx drizzle-kit push
No config path provided, using default 'drizzle.config.ts'
Reading config file '/minimal-reproducible-example/drizzle.config.ts'
Using 'pg' driver for database querying
[✓] Pulling schema from database...
[✓] Changes applied

generate output
❯ npx drizzle-kit generate
(node:69550) ExperimentalWarning: CommonJS module /opt/homebrew/lib/node_modules/npm/node_modules/debug/src/node.js is loading ES Module /opt/homebrew/lib/node_modules/npm/node_modules/supports-color/index.js using require().
Support for loading ES Module in require() is an experimental feature and might change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
No config path provided, using default 'drizzle.config.ts'
Reading config file '/minimal-reproducible-example/drizzle.config.ts'
1 tables
table 1 columns 0 indexes 0 fks

[✓] Your SQL migration file ➜ src/migrations/20241106180834_colossal_turbo.sql 🚀

migrate output
❯ npx drizzle-kit migrate
(node:69740) ExperimentalWarning: CommonJS module /opt/homebrew/lib/node_modules/npm/node_modules/debug/src/node.js is loading ES Module /opt/homebrew/lib/node_modules/npm/node_modules/supports-color/index.js using require().
Support for loading ES Module in require() is an experimental feature and might change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
No config path provided, using default 'drizzle.config.ts'
Reading config file '/minimal-reproducible-example/drizzle.config.ts'
Using 'pg' driver for database querying

Can replicate the same issue, was driving me insane thinking I was misunderstadning something lol

I am experiencing the same issue, thinking I was doing something wrong after running push, all drizzle does was to create an empty policy.
ALTER TABLE "categories" ENABLE ROW LEVEL SECURITY;
CREATE POLICY "users-to-own-category" ON "categories" AS PERMISSIVE FOR ALL TO "authenticated";

It might be a problem since I want drizzle to automatically handles all the migrations for me without generating any SQLs. Right now I guess I jsut have to generate & migrate the db until the fixes come.

I'm experiencing the same issue where the using clause is empty.
I'm also unable to fix the issue on Supabase's end, every time I save the value of the using clause the value is again empty when opening the policy again. This does not happen when I create a new policy in Supabase directly.

Also ran into this issue, resorting to migrations in the meanwhile.
The Supabase issue is due to a bug in their UI, for some reason they ignore the form submission if the initial using/check statement is null. See https://github.com/supabase/supabase/issues/28215#issuecomment-2525347116
Until it's fixed, you can update the policy with their SQL editor, or with a database explorer e.g dbeaver

I've got a minimal pnpm patch that does enough for me here. There is a minor issue with push always running an ALTER POLICY if the policy already exists.
Happy to do a PR to actually fix this, but the code & a few tests imply that this is intentional.


Persian translation of title template, headings, labels

Summary
Add Persian (fa) translations of title template, headings, labels, etc for locale code fa.
Potential Contributors
See Contributing a Translation section of the README for step-by-step instructions.
Please don't edit the README with the new locale code, despite the Pull Request template requesting that documentation is updated. Changes to the README automatically go live in the GitHub Marketplace listing for the action. I'll take care of updating README with the new locale code in the list of locales when I create the release that contains it to ensure that documentation is consistent with latest release.

Hey there! I speak Persian and I'd love to contribute to this repo.

@AshkanArabim great. I just assigned you.


cli: require absolute type with --ignore-existing

Fixes https://github.com/kcp-dev/kcp/issues/1532

Unrelated sharded flake.


Right click scale functionality for deployments and stateful-sets

It would be nice if I could scale my deployments and stateful-sets right from UI.
I propose adding "right-click" menu item that would say scale and after clicking it and entering a number a scale for that resource would trigger.
We have a lot of similar stuff for PODs like port-forward and terminal but no scaling :(

Good point.  We have the Scale command on the command palette, but it dates from before the explorer, and never got added to the right-click.  Sorry for overlooking that and thanks for flagging it!


ssa-rewrite and loop-unswitch leads to code the validator rejects

May be related to #2214 and/or #2208, but the validator error different.
2334.zip
"command" in the attachment gives the command necessary to repro the bug.
Found using GraphicsFuzz.

Compact-ids invalidates the constant manager.  However, it is not possible to reset the constant manager.  This causes ccp to fail.

Duplicate of #￼2245


Idea: Better, more Rusty, MouseData

Currently, MouseData feels very "javascripty", probably because it's a straight copy-paste from the web event.
For Dioxus, we can improve this a bit by grouping data logically and providing the appropriate types. With something like euclid, we can provide coordinates that the users can easily do math on (e.g. difference between points), and statically ensure they don't accidentally mix different coordinate spaces.
Also, it seems that currently, mouse coordinates are i32; they should be f64, since that is how MDN defines them.
This PR

Add nice and rusty accessor methods for MouseData with correct types
Deprecates the fields, since the accessors are a much better way to use them
Simple usage example in mouse_event (formerly coordinates)
Rename offset -> element coordinates, because offset is weird and ambiguous

look how beautiful it is:

Benefits

Type safety & ergonomics
Hides some internal details
Easier to navigate structs and docs

Todo

[x] The now deprecated fields are still used throughout the codebase. Remove these.

[x] Provide a way to construct MouseData without using the deprecated fields. Even with the restructuring, MouseData still logically consists of 7 components. I would love to completely hide the internal representation from users, but I'm not sure how we can provide a constructor that doesn't look like a monstrosity, especially since Rust doesn't have named arguments. The builder pattern? urgh. Or maybe a Coordinates struct with 4 public fields for each of the coordinates and then we can have a decent 4-argument constructor?

Discussion

[x] Is there an existing crate we can use for structs like a set of modifier keys? This would give better interoperability and we wouldn't reinvent the wheel.

[x] I brought in a new dependency, euclid, hopefully that's ok? I'm just using it for euclid::Point2D to avoid reinventing the wheel, but maybe it can be used for other stuff later.

[x] I'm tempted to provide the held modifier keys and mouse buttons as enumset::EnumSet. It would bring in another dependency, but then it would be easy and efficient to perform various operations on them (union, intersection, etc.).

While reading docs, I found out that button is only "reliable" for events involving mouse up or mouse down. This is bound to cause bugs. I'm guessing it would be best to keep track of the event type and return an Option that is None for irrelevant events.

~Maybe we shouldn't deprecate but instead note that it will be deprecated in the future?~

Future work
Maybe some of the other structs can be rustified too. PointerData for example. No promises from my part but I might if this is seen as a Good Change

I think there's a lot of value in Dioxus becoming its own "platform" where we rely on some of our own types instead of the types given to us in JS land - especially with the ongoing work in TUI and Native. I see this as a "Good Change". Plus, the 0.3 release is revamping quite a few details in the elements/events/component land. A lot of this stuff isn't majorly breaking, but is still breaking. I see you deprecated the fields which is a good start.
A few thoughts:

Bringing in euclid is fine.
I don't know about modifier keys. This tends to be a bespoke thing. I know almost all TUI libraries implement their own. I'd prefer to use a popular Rust crate than our own bespoke impl.
Deprecating is fine since the fields are still available. We can just leave it deprecated for a while.
EnumSet is good. I personally value interoperability over dependency count. The Rust compiler is always getting faster, so invest in the ecosystem where you can.

I like where this PR is going! Thanks for taking this on!

Cool, thanks for the feedback!

I don't know about modifier keys. This tends to be a bespoke thing. I know almost all TUI libraries implement their own. I'd prefer to use a popular Rust crate than our own bespoke impl.

I found keyboard_types, which seems good and is the most popular of its kind. Didn't find anything for mouse buttons though, so using enumset for that.
A minor drawback of enumset is that we can't use it to represent custom unknown mouse buttons, but this is probably fine – it's unlikely anyone will need a 6th or 7th mouse button in their app. I've made it so that these get converted into an ::Unknown variant – sadly we can't put the unknown code inside if we want to use it in a bit set.
I have now updated the TUI code and the deprecated fields are no longer used (except internally).
I also modified the TUI hover example for testing the mouse coordinates.
I noticed that, when handling mouse move events, TUI gets laggy. Not sure why.

(merged #401 into this one for a bugfix, and to prevent failing tests)

yay compiler bug. only on windows/mac though. something about "no MIR available for DefId"
similar issue already recently discussed here https://github.com/rust-lang/rust/issues/84455

I noticed that, when handling mouse move events, TUI gets laggy. Not sure why.

All events are triggering a redraw. Mouse move happens the most, so it is most noticeable. #402 should fix it.

#402 should fix it if it only conditionally updates the state.

Awesome, much smoother now!

what do we do about the internal compiler errors?

regarding compiler error:
i was caused by this as i found after some investigation. to avoid this, i renamed the module, avoiding conflict with the input element. probably not a good idea to have that conflict anyway.

Thank you! :) It'll be good to have the conveniences of Euclid for canvas-y type operations.


US 1583733 Add missing language IDs - 17

US 1583733 - Fix CATS P1 issues by adding missing language IDs to code blocks.

Used "output" for output displayed by the MDA debug assistant.
Used "text" for registry values.

The code block that I just removed the id could perhaps be in a table format,

Good suggestion about the table. I almost did that the first time around.


Upgrade dependencies

Nothing about the business code, all about the tests :-/

@davidbgk should we start a dev branch with this? ATM I'm not confident in merging this in master if master is to be shippable in production at all times.

@abulte Good question. I would say let's merge that one and deploy to verify that dependencies update doesn't break anything. Then iterate with feature branches that might be merged directly on master? Or do we want a dev env to make some live tests prior to push to prod?

@davidbgk we'll need a dev env pretty soon if we want to test the web hook logic and the udata integration against the dev version of udata, don't you think?

Logging is broken, I'm looking into it.

Logbook got a lot less fun since 0.10.0 - you now have to configure a default handler, more boilerplate... Since what we have works and this should not be a critical component, I reverted to Logbook 0.9.1.

Since what we have works and this should not be a critical component, I reverted to Logbook 0.9.1.

Alright, we may want to remove Logbook completely and switch to logging if it's too cumbersome.

we'll need a dev env pretty soon if we want to test the web hook logic and the udata integration against the dev version of udata, don't you think?

Sure, @odtvince can you provide us that?

@davidbgk I got the latest version of Logbook to work w/ minimal additional boilerplate. I also created a dev branch from master to merge this. If it's all good for you I'll merge.

Looks good to me, let's get this merged! 👍


Verbose output for restore and get commands

Hi,
I had same trouble with godep because my ssh keys was not correct. godep was not able to show my that i could not get access to the repository rather. That's why i added verbose flag to the get and restore commands. I hope this helps someone else.
Regards
Bernd

Any feedback on this?
Would be very useful to have a verbose option as it is easy to get into ssh access and other permission errors with private repos.

please merge

I don't like the output of restore in verbose mode, but it's what get's output by go's tools/vcs stuff. We could clean that up later as I think this is helpful as is, modulo a few changes (which I'm just doing).

Thanks! merged


[BUG] startAt fails when querying secondary indices

Summary:
The docs provide the example of startAt:
const response1 = await UserManager.query().partitionKey('partitionKey').eq('1').run();
const response2 = await UserManager.query().partitionKey('partitionKey').eq('1').startAt(response1.lastKey).run();

However, if partitionKey is not a the primary partition key of the table, this fails.  This is because convertPrimaryKeyToAttributeValues is used to convert LastEvaluatedKey from the dynamo format to a TS object, but it only populates the primary partitionKey and sortKey of the table, not the global secondary index that you are querying. This results in a Dynamo error:
ValidationException: Exclusive Start Key must have same size as table's key schema

since Dynamo is expecting the primary partition + sort pair AND the secondary index partition + sort pair in the ExclusiveStartKey field.
Other:

[X ] I have read through the Dynamode documentation before posting this issue
[X] I have searched through the GitHub issues (including closed issues) and pull requests to ensure this issue has not already been raised before
[X] I have searched the internet to ensure this issue hasn't been raised or answered before

Thanks for finding this @quinnromanek, I'll post a PR this week.

A fix is up in the PR above, I still need to add some new tests to ensure everything else works well 😄
Meanwhile try installing 1.0.1-rc.0 and let me know if it works well for you.

@quinnromanek Let me know if it works well for you now 😄


[Discussion] Do we need to rectify SecurityMgr? (Sensitive Data handling)

I personally think that sensitive data is returned by an access to passPhraseJWTPath. What do you think? @tdrozdovsky
https://github.com/lf-edge/edge-home-orchestration-go/blob/22ce49b4b76ae78dfb75e14ef64399e5805c7d02/src/controller/securemgr/authenticator/authenticator.go#L70
This might be the same potential security risk as follow.
https://github.com/lf-edge/edge-home-orchestration-go/blob/22ce49b4b76ae78dfb75e14ef64399e5805c7d02/src/controller/securemgr/authenticator/authenticator.go#L84

@tdrozdovsky Plus, could you rectify if err is used in our platform?
https://github.com/lf-edge/edge-home-orchestration-go/blob/22ce49b4b76ae78dfb75e14ef64399e5805c7d02/src/orchestrationapi/orchestration_api.go#L180

@tdrozdovsky This is due to my ongoing analysis with the tool lgtm as follow.
https://lgtm.com/projects/g/lf-edge/edge-home-orchestration-go/alerts/?mode=list

I personally think that sensitive data is returned by an access to passPhraseJWTPath. What do you think? @tdrozdovsky
https://github.com/lf-edge/edge-home-orchestration-go/blob/22ce49b4b76ae78dfb75e14ef64399e5805c7d02/src/controller/securemgr/authenticator/authenticator.go#L70
This might be the same potential security risk as follow.
https://github.com/lf-edge/edge-home-orchestration-go/blob/22ce49b4b76ae78dfb75e14ef64399e5805c7d02/src/controller/securemgr/authenticator/authenticator.go#L84

Good point, I know and remember this security issue.
These only informs about a failed attempt to create the passPhraseJWTFilePath file.
But of course, storing such information in files (passPhrase, edge-orchestration.key, etc) is a security risk.
I think in the future this should be solved with secure storage or with access control system such as: SeLinux, SMACK, etc.
Thank you for reminder

@tdrozdovsky Thank you for accepting my suggestion. I have just assigned this issue to you. We are looking forward to seeing your another valuable contribution soon!

@tdrozdovsky Plus, could you rectify if err is used in our platform?
https://github.com/lf-edge/edge-home-orchestration-go/blob/22ce49b4b76ae78dfb75e14ef64399e5805c7d02/src/orchestrationapi/orchestration_api.go#L180

I researched using err variable. Error handling is done below.
if len(deviceScores) <= 0 {
		return errorResp
} else if deviceScores[0].score == scoringmgr.INVALID_SCORE {
	return errorResp
}

Additional processing will make the code more complex.
Therefore, I suggest to ignore the err variable.
	deviceResources[i].score, _ = orcheEngine.GetScoreWithResource(dev.resource)

@tdrozdovsky Plus, could you rectify if err is used in our platform?
https://github.com/lf-edge/edge-home-orchestration-go/blob/22ce49b4b76ae78dfb75e14ef64399e5805c7d02/src/orchestrationapi/orchestration_api.go#L180

I researched using err variable. Error handling is done below.
if len(deviceScores) <= 0 {
		return errorResp
} else if deviceScores[0].score == scoringmgr.INVALID_SCORE {
	return errorResp
}

Additional processing will make the code more complex.
Therefore, I suggest to ignore the err variable.
	deviceResources[i].score, _ = orcheEngine.GetScoreWithResource(dev.resource)

@tdrozdovsky It is reasonable. ^^ Could you suggest the regarding PR for this?


Small fixes to docs.md

resoves -> resolves
Per-remote environment variables start with RCLONE_CONFIG_.

Thank you :-)

BTW your git commit id is messed up Ubuntu<EMAIL_ADDRESS>

On Mon, Oct 16, 2017 at 10:58 PM, Nick Craig-Wood<EMAIL_ADDRESS>wrote:

BTW your git commit id is messed up Ubuntu<EMAIL_ADDRESS>
Oops, sorry about that. Next time I'll commit on my normal workstation
rather than on a test VM (or make sure I have correct Git configuration
anyway).
Best regards,
Simon.


API does not give Tether balance for any address

I am using below code to get token balance for an address:
`full_node = 'https://api.trongrid.io'
solidity_node = 'https://api.trongrid.io'
event_server = 'https://api.trongrid.io'
tron = Tron(full_node=full_node,
solidity_node=solidity_node,
event_server=event_server)
tron.private_key = 'my private key'
#tron.default_address = 'TQR27jXR4dHdDtjE63MnnUQvecK97rop2y'
account_info = tron.trx.get_account('TNaRAoLUyYEV2uF7GUrzSjRQTU8v5ZJ5VR')
print(account_info)
print(pd.DataFrame(account_info["asset"]))`
it does provide a dataframe of assets but tether balance is not there. And checked that this address has tether balance.
Pls help me out. thnx

You can send request to URL = 'https://apilist.tronscan.org/api/account?address='.
This URL response with all TRC20 tokens that the account has.


Simplify "x >= 0 != x != 0" to "x<=0"

Subject: https://github.com/rfalke/decompiler-subjects/tree/master/from_holdec/dmi/cfg/ia32_elf/subject.exe
Output
int32_t intermediate_11_loop_with_two_breaks(int32_t a1) {
    if (a1 >= 0 != a1 != 0) {
        // 0x8048825
        return 0;
    }
...

if (a1 <=0)  would be much eaiser to understand.

Thank you for your suggestion. I agree that this is nuts and I'll take a look if we can detect this and substitute for equal expression.
On the other hand, I think that this exotic expression is if (a1 != 1).

e1: a1 >= 0: operator >= has highest precedence: e1 is 0 if a1 is negative and 0 otherwise.
e1 != a1 != 0: operator != is evaluated from left to right => (e1 != a1) != 0
e2: e1 != a1 can be zero only when e1 equals a1. e1 can be only {0,1}. For 0 it is 1 and for 1 it is 0.
e2 != 0 is true only when a1 is not 1.

Well in the original source code (https://github.com/rfalke/decompiler-subjects/blob/master/from_holdec/dmi/cfg/source.c#L137) it is:
int intermediate_11_loop_with_two_breaks(int arg)
{
	while (arg > 0) {
		puts("head");
		if (time(0) == arg) {
			break;
		}
		puts("tail");
		arg--;
	}
	return 0;
}

So I hope that x >= 0 != x != 0 is x<=0 because otherwise the decompiler would have a bug.


Multi Access

Does Multi Acces work in this version?
Thx!
J.
In [3]: MyWindow.set_multi_access(True)
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[3], line 1
----> 1 MyWindow.set_multi_access(True)

AttributeError: 'window' object has no attribute 'set_multi_access'

Sorry, I see it was discussed...


I'd like to be able to add/remove Tags from multiple Objects

currently you can only enter the Tags screen for single Objects

Potentially we would need a different tags screen for tagging multiple nodes as we cannot display a list of current tags on the node as there are multiple nodes in play.
So if we have the text input field, the list of known tags and then Add, Remove and Cancel btns.  Add would add the selected/input tag(s) to all selected nodes if not already present.  Remove would remove the selected/input tag(s) from all selected nodes if present, silently ignoring nodes that do not have the tags to be removed.

Fixed in > 7.2-beta.79


Report for Oberon compiler on x86_64 GNU/Linux

I have checked Oberon compiler.
Hello World with LLVM backend is working fine.
Linux Mint 21.2 Cinnamon
Linux laptop 5.15.0-91-generic #101-Ubuntu SMP Tue Nov 14 13:30:08 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
cmake (3.22.1-1ubuntu<IP_ADDRESS>)
libboost-all-dev (<IP_ADDRESS>ubuntu7)
llvm (1:14.0-55~exp2)
clang (1:14.0-55~exp2)

sudo apt install cmake llvm libboost-all-dev clang
cmake src
make
cp test/oberon/HelloWorld.Mod ./liboberon/
mv oberon-lang ./liboberon/
cd liboberon/
./oberon-lang --filetype ll Out.Mod
./oberon-lang --filetype ll HelloWorld.Mod
clang -o hello.elf Out.ll HelloWorld.ll
./hello.elf
Hoi Niklaus!

Many thanks for this report! I will close the issue, since your report is the "opposite" of an issue! 😎
Nevertheless, let me point out a couple of things that might be of interest to you.

Before merging code into the main branch, I typically test it on macOS (clang toolchain), Ubuntu (GNU toolchain), and Windows (MSVC toolchain), both on x64 and arm64. Since this is a toy/teaching project, I don't expect all functionality to work on all platforms perfectly, but there shouldn't be any major show-stoppers.
There are utility Makefiles in the the liboberon and the oberon/test directory that help with compiling and linking the (beginnings of an) Oberon library and individual modules. The Makefiles assume the OS-toolchaing pairings mentioned in the previous point.
In the long-term, it is the plan to replace these Makefiles by toolchain classes, following the example of the clang frontend.
As you will notice, my Oberon compiler is far from perfect. There are many ways how people can help. Of course, I am always grateful for pull requests with new or improved functionality for the compiler itself. For example, I'm currently working on properly integrating the JIT functionality that has been sketched in one of the open pull requests.
Another way to contribute to oberon-lang is by submitting test modules, ideally ones that do not (but should) compile. The test modules I currently use are all written by me or ChatGPT. Therefore, if I didn't think of something when I wrote the compiler, I surely didn't think about it when I wrote the test module. 🤓


fish shell support

prototyping for supporting fish shell.
reference https://github.com/sbtenv/sbtenv/issues/38

thx!

@all-contributors please add @ikuo-suyama for code


Questions about radius

First of all, thank you so much for providing such wonderful work.
I have two questions.
First, what is sphere_init_radius in neus? I wonder what exactly this role does, and how it relates to radius.
The neus implementation here only provides a square bounding box (as using radius). Why would not provide a rectangular bounding box? Are there any issues with that version of implementation?
Thanks!

Hi! We could use a rectangular bounding box instead of a square one :) The modification should be minor. I'll try to implement this when I have time.
sphere_init_radius corresponds to the sphere initialization widely used in SDF-based methods. We initialize the SDF to be a sphere for better convergence, and sphere_init_radius is the radius of this sphere relative to radius (i.e., setting sphere_init_radius=1.0 should have an initialized sphere in the size of radius).

@bennyguo
Thank you for your quick reply!
I have an another question related with "box artifacts".
Like this issue, There are box artifacts when we train a scene without any mask.
Do you know any idea about removing this?
Thanks.

Are you training on data with monochromatic background (like NeRF-Synthetic) or on real-captured data (like MipNeRF-360)? You may refer to the "Floater problem" section here and see if it helps.

@bennyguo
I'm training on CO3D dataset which is real video frames dataset so the background is not a solid color.
But thank you for your kind reply!

Are you capturing this from forward-facing viewpoints? There are two methods that can be used in the latest version:
https://github.com/bennyguo/instant-nsr-pl/blob/48145751245641b5004068060dab1dbb864857ca/datasets/colmap.py#L30-L43

dataset.center_est_method=camera: regard the center of all cameras as the scene center
dataset.center_est_method=lookat: assumes the cameras are looking at an object and calculates an approximate look-at point as the scene center
I believe neither method is very suitable for forward-facing captures. Maybe I should add another option that takes the center of the sparse point cloud to the scene center? Is it possible to provide the data you used for testing? Thanks!

@bennyguo
The problem above has nothing to do with center estimation methods(camera, lookat, original get_center).
This is due to a new pose normalization process.
I changed the code below to the old version and the problem was solved.
Latest code
    # rotation and translation
    Rc = torch.stack([x, y, z], dim=1)
    tc = center.reshape(3, 1)
    R, t = Rc.T, -Rc.T @ tc
    poses_homo = torch.cat([poses, torch.as_tensor([[[0.,0.,0.,1.]]]).expand(poses.shape[0], -1, -1)], dim=1)
    inv_trans = torch.cat([torch.cat([R, t], dim=1), torch.as_tensor([[0.,0.,0.,1.]])], dim=0)
    poses_norm = (inv_trans @ poses_homo)[:,:3] # (N_images, 4, 4)

    # scaling
    scale = poses_norm[...,3].norm(p=2, dim=-1).min()
    poses_norm[...,3] /= scale

    # apply the transformation to the point cloud
    pts = (inv_trans @ torch.cat([pts, torch.ones_like(pts[:,0:1])], dim=-1)[...,None])[:,:3,0]
    pts = pts / scale

Previous code
    # rotation
    Rc = torch.stack([x, y, z], dim=1)
    R = Rc.T
    poses_homo = torch.cat([poses, torch.as_tensor([[[0.,0.,0.,1.]]]).expand(poses.shape[0], -1, -1)], dim=1)
    inv_trans = torch.cat([torch.cat([R, torch.as_tensor([[0.,0.,0.]]).T], dim=1), torch.as_tensor([[0.,0.,0.,1.]])], dim=0)
    poses = (inv_trans @ poses_homo)[:,:3]
    pts = (inv_trans @ torch.cat([pts, torch.ones_like(pts[:,0:1])], dim=-1)[...,None])[:,:3,0]

    # translation and scaling
    poses_min, poses_max = poses[...,3].min(0)[0], poses[...,3].max(0)[0]
    pts_fg = pts[(poses_min[0] < pts[:,0]) & (pts[:,0] < poses_max[0]) & (poses_min[1] < pts[:,1]) & (pts[:,1] < poses_max[1])]
    center = get_center(pts_fg)
    tc = center.reshape(3, 1)
    t = -tc
    poses_homo = torch.cat([poses, torch.as_tensor([[[0.,0.,0.,1.]]]).expand(poses.shape[0], -1, -1)], dim=1)
    inv_trans = torch.cat([torch.cat([torch.eye(3), t], dim=1), torch.as_tensor([[0.,0.,0.,1.]])], dim=0)
    poses = (inv_trans @ poses_homo)[:,:3]
    scale = poses[...,3].norm(p=2, dim=-1).min()
    poses[...,3] /= scale
    pts = (inv_trans @ torch.cat([pts, torch.ones_like(pts[:,0:1])], dim=-1)[...,None])[:,:3,0]
    pts = pts / scale

Why did you change the normalization code to the current version?
In particular, I wonder why you changed t=-tc to t=-Rc.T @ tc.
In my humble opinion, the translation vector obtained through get_center is a translation vector in world space, not camera space.
Using the old version of the scaling code, I can get the following result for a tight bound!!:

Thank you so much for solving the box artifacts issue!

Both transformations are correct:

In the current version, [R|t] performs first rotation then translation, but we need to first translate by -tc and then rotate by Rc.T. Therefore we have Rx + t = Rc.T(x - tc), leading to t = -Rc.T @ tc.
In the previous version, t is re-calculated as the center of points that are bounded by all cameras after rotation, so it doesn't need to be transformed as above.

I thought center_est_method=camera in the current version (treating the center of all camera positions as the scene center, not using point locations) approximately achieves the same effect as the old version, but it seems better to give another option such as center_est_method=points that does the same as the previous version :)

@bennyguo
Oh, you are right.
The only problem with the current version of camera pose is that the return value here should actually be poses_norm, not poses.

@TLB-MISS Thanks for pointing this out!
I think my issue could be closed now.


info on CKA_VALUE on RSA PublicKey

Hi Jariq,
i trying to read public key value from an RSA key pair generated, but i have always CKA_VALUE empty. Instead if i try to generate an EC key pair, i can read public key value.
could be an HSM settings or i miss something ?
Should i consider only CKA_MODULUS as public key value ?
thanks

You will need to read both CKA_MODULUS and CKA_PUBLIC_EXPONENT and most likely also encode them in SubjectPublicKeyInfo or similar structure.
For example see https://github.com/Pkcs11Admin/Pkcs11Admin/blob/master/src/Pkcs11Admin/Pkcs11Slot.cs#L1036

@armando-basile is there any update on this? Was provided link helpful?

Closing because of 14+ days without a response.


Module busted.runner not found for KONG_VERSION=2.5.0

Exactly the same as in https://github.com/Kong/kong-pongo/issues/94 but for version 2.5.0:
Creating kong-pongo_kong_run ... done

Stopping after installing dependencies for kong-plugin-myplugin 0.1.0-1

Kong version: 2.5.0
ERROR: bin/busted:58: module 'busted.runner' not found:No LuaRocks module found for busted.runner
        no field package.preload['busted.runner']
        no file '/usr/local/openresty/site/lualib/busted/runner.ljbc'
        no file '/usr/local/openresty/site/lualib/busted/runner/init.ljbc'
        no file '/usr/local/openresty/lualib/busted/runner.ljbc'
        no file '/usr/local/openresty/lualib/busted/runner/init.ljbc'
        no file '/usr/local/openresty/site/lualib/busted/runner.lua'
        no file '/usr/local/openresty/site/lualib/busted/runner/init.lua'
        no file '/usr/local/openresty/lualib/busted/runner.lua'
        no file '/usr/local/openresty/lualib/busted/runner/init.lua'
        no file '/kong-plugin/busted/runner.lua'
        no file '/kong-plugin/busted/runner/init.lua'
        no file './busted/runner.lua'
        no file '/usr/local/openresty/luajit/share/luajit-2.1.0-beta3/busted/runner.lua'
        no file '/usr/local/share/lua/5.1/busted/runner.lua'
        no file '/usr/local/share/lua/5.1/busted/runner/init.lua'
        no file '/usr/local/openresty/luajit/share/lua/5.1/busted/runner.lua'
        no file '/usr/local/openresty/luajit/share/lua/5.1/busted/runner/init.lua'
        no file '/root/.luarocks/share/lua/5.1/busted/runner.lua'
        no file '/root/.luarocks/share/lua/5.1/busted/runner/init.lua'
        no file '/usr/local/openresty/site/lualib/busted/runner.so'
        no file '/usr/local/openresty/lualib/busted/runner.so'
        no file './busted/runner.so'
        no file '/usr/local/lib/lua/5.1/busted/runner.so'
        no file '/usr/local/openresty/luajit/lib/lua/5.1/busted/runner.so'
        no file '/usr/local/lib/lua/5.1/loadall.so'
        no file '/root/.luarocks/lib/lua/5.1/busted/runner.so'
        no file '/usr/local/openresty/site/lualib/busted.so'
        no file '/usr/local/openresty/lualib/busted.so'
        no file './busted.so'
        no file '/usr/local/lib/lua/5.1/busted.so'
        no file '/usr/local/openresty/luajit/lib/lua/5.1/busted.so'
        no file '/usr/local/lib/lua/5.1/loadall.so'
        no file '/root/.luarocks/lib/lua/5.1/busted.so'
stack traceback:
        bin/busted:58: in function 'file_gen'
        init_worker_by_lua:47: in function <init_worker_by_lua:45>
        [C]: in function 'xpcall'
        init_worker_by_lua:54: in function <init_worker_by_lua:52>
ERROR: 1

Workaround so far:
KONG_VERSION="2.4.x" pongo run

It works for me locally? What environment are you using? OS, Docker version, etc

macOS Big Sur 11.2.3 with
% docker --version
Docker version 20.10.7, build f0df350

Not sure what else to think about, my understanding was that pongo run does everything in docker, so my env shouldn't matter that much?

there's a difference between "should" and "reality" unfortunately.
If you clear the cached images and rebuild it, are there any errors in the build log?

@dzmitry-kankalovich any update? anything in the build log/output?

@Tieske sorry, I've just moved forward with the workaround, since 2.5.0 is not essential for what I need.

typically this is an image build failure, see #241


Unclear example in Get-ADKeyCredential.md documentation

The documentation for the Get-ADKeyCredential command, example 6:
PS C:\> $certificateSubject =<EMAIL_ADDRESS>It is unclear where that magic UUIDs came from - Are they the user object GUID, or the Directory's, or something else?
It would be very helpful any clarification on that.

Well, it really IS UNCLEAR unclear where those magic UUIDs came from, as there is no official documentation for it. Luckily, the value of the Subject has actually no effect on the functionality, it only appears in DC logs.
Nevertheless, the first UUID seems to be random and the second one appears to be a constant. So I typically generate it using the following code:
$targetAccount = Get-ADUser -Identity john
$certificateSubject = '{0}/{1}/login.windows.net/383a3889-5bc9-47a3-846c-2b70f0b7fe0e/{2}' -f $targetAccount.SID,(New-Guid),$targetAccount.UserPrincipalName


First plot of the job end after 20 minutes

Hi!
When i put 3 plot in paralell in the same job, the first end after 20 minutes, that generate one new plot (only 3 in phase 1 whit my 1tb nvme) and all continuos fine.
Just the first plot have this issue, the others plots goes to the end. Thx!

@rjsears 's project is not about plotting at all. it's only about moving already plotted plots to your farm-machine. dont ask here for anything about plotting. thank you.


4 files limit

great job m8!
It works just fine but it won't decrypt more than 4 files in a row.
I used command line with windows .exe:
LyndaDecryptor /D "K:\Linda Courses\ldc_dl_courses\417082"
and it returns:
K:\Lynda_v1.2>LyndaDecryptor /D "K:\Linda Courses\ldc_dl_courses\417082"
[ARGS] Changing mode to Folder decryption!
[START] Init Decryptor...
[START] Decryptor successful initalized!
[DEC] Decrypting file 438808_f2caf0365368d407180e5bd354c0f945.lynda...
[DEC] Decrypting file 438807_83218ac34c1834c26781fe4bde918ee4.lynda...
[DEC] File decryption completed: K:\Linda Courses\ldc_dl_courses\417082\438808_f
2caf0365368d407180e5bd354c0f945.mp4
[DEC] Decrypting file 438809_b37c2c183d5a4009e0a54324a407a534.lynda...
Press any key to exit the program...
[DEC] File decryption completed: K:\Linda Courses\ldc_dl_courses\417082\438809_b
37c2c183d5a4009e0a54324a407a534.mp4
[DEC] File decryption completed: K:\Linda Courses\ldc_dl_courses\417082\438807_8
3218ac34c1834c26781fe4bde918ee4.mp4
[DEC] Decrypting file 438811_7515ffcc2fe807400fcb273c96ae8cc6.lynda...
[DEC] File decryption completed: K:\Linda Courses\ldc_dl_courses\417082\438811_7
515ffcc2fe807400fcb273c96ae8cc6.mp4
then it is hanging, cursor is blinking.
on click it aborts

also, you hardcoded
"video2brain Desktop App"
while it is actually:
"Lynda.com Desktop App"
C:\Users<user>\AppData\Local\lynda.com\Lynda.com Desktop App
which also leads to an error while trying to use /DB option
K:\Lynda_v1.2>LyndaDecryptor /D "K:\Linda Courses\ldc_dl_courses\417082" /db
[ARGS] Changing mode to Folder decryption!
[START] Init Decryptor...
[START] Decryptor successful initalized!
[DB] Creating db connection...
[START] Error occured: Could not find a part of the path 'C:\Users\1\AppData\Loc
al\lynda.com\video2brain Desktop App'.

I'm also having this issue and still haven't found a solution

@h4ck-rOOt, Thnx for creating this!
I'm having the same problem, in Directory decryption mode it gets through only the first four files then it just hangs. When I try to encode any one of the remaining files using File decryption mode, it immediately exits with the error:
[START] Error occured: Index was outside the bounds of the array.
...followed by the usage instructions.

@plutonian2016 thx man
The app was developed to decrypt files downloaded by Video2Brain Desktop App, so i personally don't own a Lynda.com Account or even using the Lynda.com Desktop App.
Feel free to edit the path in Program.cs or combine them to support Lynda.com Desktop App.  I'm going to support both versions, now that i know the path to this app.
So theres a technique called semaphore i've used to program this app and it seems this is the culprit of all error hanging after 4 files. This issue needs more investigation from my side.
Maybe you can debug this behavior in VS2015?

@DaveLampton just hang in, im going to investigate the strange behavior next week...

The binaries available here  have the 4 file limit, happened to me too. But when I compiled the project from Visual Studio, the compiled one worked just fine. Probably it was a bug when the binaries available here on github were compiled?

@mdomnita Can you post or link me to a working version? If that's okay with you. I cant really compile the build I dont have vs 2015, only vs 2008

There is one in one of the other topics saw it earlier. I can only give you
one in the afternoon (6 hrs from now or so).
On Nov 10, 2016 10:45 AM, "Kobi"<EMAIL_ADDRESS>wrote:

@mdomnita Can you post or link me to a working version? If that's okay
with you. I cant really compile the build I dont have vs 2015, only vs 2008
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

@mdomnita Yah, saw that one. Tried that one but it didn't worked for me,

Then I will build one in the afternoon/evening and post it. I only have all stuff at home and I don't have access to it until then.

@mdomnita Thanks for that! I will be patiently waiting.

https://goo.gl/iKiGM2 try this. Just unpack, go to the folder and run
LyndaDecryptor.exe /D

@mdomnita Tried this on one of the courses I have. It now works!! Thank you! Hope @h4ck-rOOt will make this to work with the master and hopefully will be able to convert .captions to .srt

dude you the best. what not make a gui for it then u would be the master , im not sure how u did it but as far as i can say you are the master

thanks @mdomnita you the real mvp

After the code rewrite this issue should be fixed.


TEST-#2951: add ASV config for omnisci backend

Signed-off-by: Anatoly Myachev<EMAIL_ADDRESS>
What do these changes do?

Add separate configuration for omnisci backend - asv_bench/asv.conf.omnisci.json, since the configuration name is different from the default (asv.conf.json) - we should explicitly specify it through the --config parameter
Add trigger of omnisci execution via frame._execute
Add new ASV version which is needed to be able to create an conda environment from yml file
Benchmarking only working subset of all operations

[x] commit message follows format outlined here
[x] passes flake8 modin
[x] passes black --check modin
[x] signed commit with git commit -s
[x] Resolves #2951
[ ] tests added and passing

It is necessary to find out why some functions work during testing, but not during benchmarking.
For example, concat, append, merge and so on.

append benchmark failed by timeout if run via ASV

Need:

add data import trigger into OmniSci;
revise dimensions of test data;

Interesting:

ASV builds a conda environment correctly only when it is launched from some conda env (at least from base).
Benchmarks running on OmniSci engine cause a timeout error if --launch-method=spawn is used; works correctly with --launch-method=forkserver

Should be good when CI finally passes.

CI is green (except codecov) :)

@devin-petersohn requested changes as well. Devin, are you fine with current version of the PR?

Merge conflicts fixed. @devin-petersohn ready for review.


Unitary learning tutorial with qgrad

Opening this with notebook here since I can't import qgrad_qutip in the examples repo for now. Turns out JAX's grad is not merely a one-liner to be used with cost functions just like that.
I write the JAX-comptabile cost function
def cost(params, inputs, outputs):
    r"""Calculates the cost/error on the whole training datat set.

    Args:
    ----
        params: parameters :math:`\t` and :math:`\tau` in :math:`U^{\dagger} U(\vec{t},\vec{\tau})`
        inputs: input kets :math:`|\psi_{l}>` in the dataset
        outputs: output kets :math:`U(\vec{t}, \vec{\tau})*|ket\_input>` in the dataset

    Returns:
    -------
        cost (float): cost (evaluated on the entire dataset) of parametrizing
                     :math:`\tau` in :math:`U^{\dagger} U(\vec{t},\vec{\tau})` with `params`
    """
    loss = 0.0
    for k in range(train_len):
        pred = jnp.dot(make_unitary(N, params), inputs[k]) #prediction wth parametrized unitary
        loss += jnp.absolute(jnp.real(jnp.dot(outputs[k].conjugate().T, pred)))
        # TODO check real and abs in loss above since
        # it's not in the original paper
    loss = 1 - (1 / train_len) * loss
    return loss[0][0]

but it does not work with grad and throws the following error:

ValueError: Reverse-mode differentiation does not work for lax.while_loop or lax.fori_loop. Try using lax.scan instead

One way hack this, as is also suggested in this issue is to forcefully trigger the scan function by setting the condition in line 203 (in jax/lax/lax_control_flow.py). This does not help either.
Turns out that if the cost function, or the function one wishes to differentiate in general, contains loops, JAX unrolls the loops in a 'functional' way and asks to follow this. And so I write cost function as:
def cost_jax(params, inputs, outputs):
    with loops.Scope() as s:
        s.loss = 0.0
        s.params = params
        s.inputs = inputs
        s.outputs = outputs
        s.N = N
        s.train_len = train_len
        for k in range(train_len):
            s.pred = jnp.dot(make_unitary(s.N, s.params), s.inputs[k])
            s.loss += jnp.absolute(jnp.real(jnp.dot(s.outputs[k].conjugate().T, s.pred)))
        s.loss = 1 - (1 / s.train_len) * s.loss
        return s.loss[0][0]

This evaluates the value of the cost function without any bugs, but grad still gives the same error.
For all the other tutorials that we used grad for, they (luckily!) did not require involve loops since it was just some parametrized operator acting on one state and fidelity with the evolved state was returned. Here, we evaluate loss on whole data set that requires summing over the loss for each point (or a few points in case of SGD), which requires loops. One way might be to call a helper function within cost, say predict to run a for loop in that predict. Another way would be to vectorize operations maybe?
Thoughts?

I suppose we discussed this last time. Any updates on whether this works with a predict function where you will have the loop through the unitaries and get the predicted output vector? This is similar to running a many-hidden-layer neural network and should not be an issue. Also, in any neural network training, you will calculate the cost function as a sum of the individual losses (eg., mean squared loss) for many training examples. Check how that would work with jax since you will essentially be looping over all the training examples. Good luck.

I had tried writing a separate function earlier on. A loss function like this
def loss(params, inputs, outputs):
    loss_val = 0.0
    unit = make_unitary(N, params)
    for k in range(train_len):
        pred = jnp.dot(unit, inputs[k]) #prediction wth parametrized unitary
        loss_val += jnp.absolute(jnp.real(jnp.dot(outputs[k].conjugate().T, pred)))
    return loss_val

And then the cost
def cost(params, inputs, outputs):
    r"""Calculates the cost on the whole training dataset.

    Args:
    ----
        params: parameters :math:`\t` and :math:`\tau` in :math:`U^{\dagger} U(\vec{t},\vec{\tau})`
        inputs: input kets :math:`|\psi_{l}>` in the dataset
        outputs: output kets :math:`U(\vec{t}, \vec{\tau})*|ket\_input>` in the dataset

    Returns:
    -------
        cost (float): cost (evaluated on the entire dataset) of parametrizing
                     :math:`\tau` in :math:`U^{\dagger} U(\vec{t},\vec{\tau})` with `params`
    """
    '''
    loss = 0.0
    for k in range(train_len):
        pred = jnp.dot(make_unitary(N, params), inputs[k])
        loss += jnp.absolute(jnp.real(jnp.dot(outputs[k].conjugate().T, pred)))
        # TODO check real and abs in loss above since
        # it's not in the original paper
    '''

    loss_val = 1 - (1 / train_len) * loss(params, inputs, outputs)
    return loss_val[0][0]

It still returned the same Error

ValueError: Reverse-mode differentiation does not work for lax.while_loop or lax.fori_loop. Try using lax.scan instead

A user kindly suggested (on JAx's repo here) what may the source of error here. And that is something we have discussed earlier as well. It is the fact that the make_unitary function
def make_unitary(N, params):
    r"""Retruns a paramterized unitary matrix.

    : math:: \begin{equation}\label{decomp}
                U(\vec{t}, \vec{\tau}) = e^{-iB\tau_{N}}e^{-iAt_{N}} ... e^{-iB\tau_{1}}e^{-iAt_{1}}
             \end{equation}

    Args:
    ----
        N (int): Size of the parameter vectors, :math:`\tau` and :math:`\t`
        params (:obj:`np.ndarray`): parameter vector of size :math:`2 * N` where the first half parameters are
                                   :math:`\vec{t}` params and the second half encodes \vec{\tau}) parameters.

    Returns:
        unitary (:obj:`np.ndarray`): numpy array representation of paramterized unitary matrix
    """
    unitary = jnp.eye(d)
    params = jnp.array(params)
    for i in range (N):
        unitary = jnp.matmul(jnp.matmul(expm(-1j * B * params[i + N][0]),
                                        expm(-1j * A * params[i][0])), unitary)

    return unitary

exponentiates a complex matrix, B by expm(B) and JAX's  grad(expm(B)) expm still a WIP as Matt says here.

Actually, the forward mode diff of expm is merged on https://github.com/google/jax/pull/2062, but the reverse mode is still a WIP https://github.com/google/jax/issues/3447

Ah I see. The construction of the N x N unitary is done using expm(-iH) where H is Hermitian. In case of the 2x2 unitary you notice that is has a parameterisation in terms of sines and cosines where you do not require the matrix exponential. We must be able to find alternative ways to construct N x N unitaries which do not involve ´expm´ similar to what we found for the Displace operator. Consider this from the supplementary part of the paper Deep learning with coherent nanophotonic circuits by Shen et. al, 2017:

If you take this decomposition, you will be able to parameterise your N x N unitary with just sines and cosines of the 2**N theta parameters. This should then allow us to circumvent the expm issue. If you are familiar with recursive methods then there is also this construction: A recursive parametrization of unitary matrices (C Jarlskog 2005) which decomposes the N x N unitary using a diagonal matrix similar to the Displace function.

@araza6 is this PR review ready?

Not yet, there is the non-autodifferentiable unitary matrix issue in works

The fidelity values are exploding. This is possibly due to the fact that the parametrized unitary generator does not really give a unitary that preserves ket norm, thereby exploding fidelity values -- apparently monotonically. I make a 2 x 2 unitary using the scheme shown in this paper in equation 19.
When I test the function, it appears that the matrix generated is indeed not a unitary. To reproduce,
import jax.numpy as jnp
from jax import ops

from qgrad.qgrad_qutip import dag

def rot(theta):
    return jnp.array([
        [jnp.cos(theta), jnp.sin(theta)],
        [-jnp.sin(theta), jnp.cos(theta)],
    ])

def make_unitary_by_rot(alphas, betas, theta):
    alpha_mat = beta_mat = jnp.zeros((2, 2))
    # Fill diagonals of the alpha matrix
    alpha_mat = ops.index_update(alpha_mat, ops.index[0, 0], alphas[0])
    alpha_mat = ops.index_update(alpha_mat, ops.index[1, 1], alphas[1])
    # Fill diagonals of the beta matrix
    beta_mat = ops.index_update(beta_mat, ops.index[0, 0], betas[0])
    beta_mat = ops.index_update(beta_mat, ops.index[1, 1], betas[1])

    tmp = jnp.dot(alpha_mat, rot(theta))
    return jnp.dot(tmp, beta_mat)

# Test here
alphas = jnp.array([0.1, 0.2])
betas = jnp.array([0.3, 0.4])
theta = jnp.array(0.5)
U = make_unitary_by_rot(alphas, betas, theta)
alphas = jnp.array([0.1, 0.2])
betas = jnp.array([0.3, 0.4])
theta = jnp.array(0.5)
U = make_unitary_by_rot(alphas, betas, theta)
print("U_UD",jnp.dot(U, dag(U)))
print("UD_U",jnp.dot(dag(U), U))

This returns
U_UD [[0.00106089 0.00058903]
 [0.00058903 0.00575642]]
UD_U [[ 0.00152059 -0.00151465]
 [-0.00151465  0.00529673]]

This is ready for review. Two points:

Learning sometimes changes dramatically with different random datasets
Although the evolved kets have a very high density, the crude unitary matrices are not very much the same
cc: @quantshah

The random seed is not available for scipy's unitary group, through which we make our target unitary
from scipy.stats import unitary_group
tar_unitr = jnp.asarray(unitary_group.rvs(N))

So the results in the notebook will not be 100 % reproducible in that the user might have to try different runs to get a good convergence