[RPi5] Migrate Option from external SSD to NVMe base

On recover with a fresh sd card if a Pi5 and a NVMe and an external SSD with the name "BLOCKCHAIN" it could offer an automated data migration.
This could be used to migrate from an old external SSD used on a RPi4 or legacy RPi5 setup to an Pironi style NVMe base.
Or when moving from an 1TB NVMe to a 2TB NVMe later by connecting the old NVMe thru a cheap USB adapter like this one:

Can somebody test if on the RaspberryPi5 with the Pimoroni NVMe Base an additional connected USB SSD (the one we used before or such an USB NVMe adapter) can still copy lots of dummy data correctly without drops in power?

I am in the fortunate position of owning two Pi 5s, allowing me to use one for testing purposes. I will receive a Pineberry Pi HatDrive Top tomorrow. A few weeks later, I'm expecting the Pimoroni NVMe Base. I believe their performance will be comparable.
My initial step involves creating a copy of my current productive USB SSD. Following that, on the test device, I plan to install the latest 1.11.pre dev version on the NVMe. Then, using an SD card loaded with the most recent Pi OS, I'll proceed to copy the blockchain onto the NVMe partition. f the synchronization has already started, I delete the blocks that have already been synchronized.
I intend to replicate the blockchain multiple times, utilizing various tools and settings such as cp, rsync, and dd, to compare their outcomes. My anticipation is that cp will offer the fastest performance, rsync the most reliability, and dd the slowest speed.
I am very much open to suggestions regarding this setup.

First off, copying the productive USB SSD onto the test USB SSD with rsync has already worked without any problems. Following are the results:
sent 732,594,025,718 bytes  received 721,296 bytes  295,103,624.17 bytes/sec
total size is 732,412,654,771  speedup is 1.00
real	41m21.883s
user	0m1.202s
sys	0m2.873s

Copying the blockchain over USB or LAN is unfortunately not possible without temporary cooling eg. using a fan.

After making some adjustments, chatGPT suggests the following:
#!/bin/bash

# NVMe Temperature Monitoring and Blockchain Copy Script for Raspberry Pi
# This script copies the Bitcoin blockchain data from one drive to another,
# while monitoring the temperature of the NVMe SSD. The source files will
# not be deleted after copying.

# Set the source and destination directories
SOURCE="/path/to/source/blockchain" # Replace with the path to the source directory
DEST="/path/to/destination/blockchain" # Replace with the path to the destination directory

# Set the NVMe device. Example: /dev/nvme0n1
NVME_DEVICE="/dev/nvme0n1" # Replace with your NVMe device

# Set the maximum allowed temperature (in Celsius)
MAX_TEMP=50

# Create a temporary file to keep track of copied files
COPIED_FILES="/tmp/copied_files.txt"
> "$COPIED_FILES"

# Main loop
while true; do
    # Read the temperature of the NVMe SSD
    TEMP=$(sudo smartctl -A $NVME_DEVICE | grep -i "Temperature:" | awk '{print $10}')

    # Check if the temperature is within the acceptable range
    if [ "$TEMP" -le "$MAX_TEMP" ]; then
        # Find new files to copy, excluding those already copied
        find $SOURCE -type f | grep -vFf "$COPIED_FILES" | head -n 100 > "/tmp/to_copy.txt"

        # Check if there are new files to copy
        if [ ! -s "/tmp/to_copy.txt" ]; then
            echo "Copying complete"
            break
        fi

        # Copy the files
        rsync -av --progress --files-from="/tmp/to_copy.txt" "$SOURCE" "$DEST"

        # Add copied files to the list of copied files
        cat "/tmp/to_copy.txt" >> "$COPIED_FILES"
    else
        # Pause if the temperature is too high
        echo "Pausing due to high temperature ($TEMP°C)"
        sleep 60 # Wait for 60 seconds
    fi
done

# Cleanup
rm "/tmp/to_copy.txt" "$COPIED_FILES"

# End of script

Looks fine :-)

Overheating is really an issue. Here's an optimized script. I recommend not sorting the files and also consider using an external fan for cooling the NVMe SSD.
#!/bin/bash

# NVMe Temperature Monitoring and Blockchain Copy Script for Raspberry Pi
# This script copies the Bitcoin blockchain data from one drive to another,
# while monitoring the temperature of the NVMe SSD. The source files will
# not be deleted after copying.

# Set the source and destination directories
SOURCE="/mnt/source/bitcoin/blocks" # Replace with the path to the source directory
DEST="/mnt/target/test"             # Replace with the path to the destination directory

# Set the NVMe device. Example: /dev/nvme0n1
NVME_DEVICE="/dev/nvme0n1"          # Replace with your NVMe device

# Set the maximum allowed temperature (in Celsius)
MAX_TEMP=50

# Temporary files
TO_COPY_FILE="/tmp/to_copy.txt"
COPIED_FILES="/tmp/copied_files.txt"

# Delete temporary files if they exist
[ -f "$TO_COPY_FILE" ] && rm "$TO_COPY_FILE"
[ -f "$COPIED_FILES" ] && rm "$COPIED_FILES"

# Create a new empty file for tracking copied files
> "$COPIED_FILES"

# Main loop
while true; do
    # Read the temperature of the NVMe SSD
    TEMP=$(sudo smartctl -A $NVME_DEVICE | grep "Temperature:" | awk '{print $2}')

    # Check if the temperature is within the acceptable range
    if [ "$TEMP" -le "$MAX_TEMP" ]; then
        # Find new files to copy, excluding those already copied
		cd $SOURCE
        find . -type f | sed 's|^\./||'| grep -xvFf "$COPIED_FILES" | head -n 50 > "$TO_COPY_FILE"

        # Check if there are new files to copy
        if [ ! -s "$TO_COPY_FILE" ]; then
            echo "Copying complete"
            break
        fi

        # Copy the files with absolute paths
        rsync -av --progress --files-from="$TO_COPY_FILE" "$SOURCE/" "$DEST/"

        # Add copied files to the list of copied files
        cat "$TO_COPY_FILE" >> "$COPIED_FILES"
    else
        # Count the remaining files
        cd $SOURCE
        REMAINING_FILES=$(find . -type f | sed 's|^\./||'| grep -xvFf "$COPIED_FILES" | wc -l)

        # Pause if the temperature is too high
        echo "Pausing due to high temperature ($TEMP°C). $REMAINING_FILES files remaining to copy."
        sleep 60 # Wait for 60 seconds
    fi
done

# Cleanup
rm "$TO_COPY_FILE" "$COPIED_FILES"

# End of script

What do you think about displaying the NVME temperature alongside the CPU temperature on the dashboard?

What do you think about displaying the NVME temperature alongside the CPU temperature on the dashboard?
The heat problem should only occur when transferring large amounts of data, eg. the blockchain. In later operation, the NVMe hardly has anything to do.


calling functions with different signatures by name within a wrapper, using Parameters<T> to validate arguments

Bug Report
I explained the issue/question in this stack overflow ticket:
https://stackoverflow.com/questions/66226231/typescript-parameters-when-calling-method-by-name
The code below is not valid, since typescript is mixing method signatures together in _method.
I wanted to ask if it is by design and if so why?
🔎 Search Terms

call function by name
Parameters

It's actually really hard to find anything like this issue.
🕗 Version & Regression Information
I think it has always been the case
⏯ Playground Link
Playground link with relevant code
💻 Code
interface SomeObject {
  fn1: (params: { a: string, b: object }) => void,
  fn2: (params: { c: number }) => number,
  fn3: () => number
}

type Methods = keyof SomeObject;

let someObject: SomeObject = {
  fn1: ({ a, b }) => console.log(a, b),
  fn2: ({ c }) => c,
  fn3: () => 1
}

const fn = <T extends Methods>({ method, parameters }: {
  method: T,
  parameters: Parameters<SomeObject[T]>
}) => {
  const _method = someObject[method]

  _method(parameters[0]);
}

fn({ method: 'fn1', parameters: [{a:'str', b:{}}] });

🙁 Actual behavior
Argument of type '{ a: string; b: object; } | { c: number; } | undefined' is not assignable to parameter of type '{ a: string; b: object; } & { c: number; }'.
  Type 'undefined' is not assignable to type '{ a: string; b: object; } & { c: number; }'.
    Type 'undefined' is not assignable to type '{ a: string; b: object; }'

because _method is an intersection of all possible arguments:
const _method: (params: {
    a: string;
    b: object;
} & {
    c: number;
}) => number | void

thus making fn({ method, parameters }) notation impossible to use.
🙂 Expected behavior
I expect typescript to at least deduce a union type of all parameters so that I would be able to enforce the parameters type that I need.

Relevant: #30581 and also #13995 / #27808 / #33014
Technically speaking T could be specified with the full union and therefore it is quite possible to call fn<Methods>() with mismatching method and parameters properties.  Realistically even if you can lock it down to only allow valid calls, this kind of higher-order analysis of generics is just not something the compiler is good at.  As far as I know, you'll just need to use a type assertion or something equivalent:
function call<T extends (...args: any[]) => any>(f: T, args: Parameters<T>): ReturnType<T> {
  return f(...args)
}

const fn = <T extends Methods>({ method, parameters }: {
  method: T,
  parameters: Parameters<SomeObject[T]>
}) => {
  const _method = someObject[method]
  call(_method, parameters);
};

Playground link

Thank you for the response and the linked tickets!
It's actually surprising how, as a casual TS user, one would think that this kind of "basic" stuff is possible, while it's not.
That's what happens to those who haven't really studied how TS works (me, for example :) )

@jcalz thank you for the response. I'm closing the issue.


How to force the generation of the swagger spec from code to turn []byte into string

I have the following scenario:

I need to create a swagger spec file from code
My models are being generated from avro schemas through gogen-avro
I would like fields that are []byte to be expressed in the swagger spec as string as that's what the Java swagger client generator expects for byte arrays

The model looks like this:
// swagger:model
type EventsActorTaskEvent struct {
		Metadata *EventsMetadataDescriptor
		NotificationDataBinary []byte
}

What I'm getting at the moment is the following:
      NotificationDataBinary:
        items:
          format: uint8
          type: integer
        type: array

but the correct notation should be:
      NotificationDataBinary:
        type: string
        format: byte

I saw this issue that is similar: https://github.com/go-swagger/go-swagger/issues/2404
But I cannot change the type of the field as it's being generated and I have no control over that.
Is there any other way to hint go-swagger about what the field should be translated to?

@tanis2000  have you try the following hint : // swagger:type: string ?

@fredbi that did work like a charm. Thanks so much. I'm closing this issue


Pre-warning reservation protocol

Hi Paul!
I think the best way to do this would be to add another reservation protocol rule. It could be called like "pre_message" or something and could hold the text of the message (including link) you want to display. Then it would show at the top of the reservation form, right above 'Event Name':
Inline image 1
In general, this would be used for things we want to warn folks about before they take the time to fill out the form, as opposed to the guidelines, which are for informing folks about the rules of the road once they've decided to make a reservation.
On Tue, Mar 21, 2017 at 8:27 AM, Paul Conahan<EMAIL_ADDRESS>wrote:
Hi Tom,
As part of our CHOPS guidelines, we ask for people to fill out a form for larger events which may require further process and approval from our CH operations (CHOPS) committee.
The form link is here:
https://goo.gl/forms/uHmVAae9e32Nm0Zz1
We?d like folks to at least consider when an event will require more process, so perhaps there could be some text near the ?Create Reservation? button that says ?Is CHOPS approval needed for your event?  See the approval form for more info.? (the ?approval form? text could be a hyperlink).
Realistically, only events involving the kitchen and dining room would trigger CHOPS approval, since the primary factor is the size of the event.  That said, if it is simpler to have the form link appear whenever a room is reserved, that would be OK as well.
As always, we are appreciative of the ever evolving and improving Gather system and we do not ask for changes lightly, so please use whatever is the easiest method.  If you?d rather consider this for a future update to Gather, that works as well.  Having this form available on the reservation system is not a burning issue, but one that we see will be useful in the long run since many folks, especially newbies, don?t know about the procedure for approving large CH events.
Paul
Originally created by Tom Smyth at 2017-03-22 02:21, migrated from redmine-#6120

Tags set to reservations

Priority changed from ? to 2

By Tom Smyth at 2017-03-22 06:22

Priority changed from 2 to 1

Moving to immediate because Sunward pays us money!
By Tom Smyth at 2017-06-11 18:06

Estimate set to 1.00 h

By Tom Smyth at 2017-06-09 11:26

Status changed from New to Closed

Version set to 4.3

By Tom Smyth at 2017-06-18 15:57


Unexpected behaviour resolving modules when requiring files from a loader

Is there a best practice way to require a file from within a loader, so that the current (application) configuration is honoured?
I have a situation where I would like to allow a query param on the loader to require a configuration file. The loader would then use this configuration file before parsing or returning the output file.
Additional module lookups that have been added to resolve don't seem to be used when resolving dependencies of the configuration file that is required by the loader.
If the require within the loader is a part of the returned string, I believe it works as expected - however for my needs I must require the extra file before returning the parsed resource.
See the link below to reproduce the issue.
https://github.com/at0g/webpack-loader-issue-example
#1033
https://github.com/at0g/nunjucks-loader/issues/3

I can see the related issues were closed with conclusions so I'm closing this as well. If still relevant, feel free to re-open the question at Stack Overflow.


Accessing $namespace without argument returns undefined

Present:
User.$namespace() returns 'entities/users/undefined'
Desired behavior:
User.$namespace() returns 'entities/users' or 'entities/users/'
Workaround:
User.$namespace('')
Personal use case: I persist to non relational storage. Much of the data is dynamic to the point of each entity containing a roadmap to it's appropriate vuex module.

@geraldbiggs Thank you for the feedback! Yap, that is how it should be. We should improve this one. 👍


Make RichTextField implement TextField

This will allow RichTextField to be used anywhere a TextField can.

Sorry, my local dartanalyzer failed me, didn't reveal the errors. Closing until concrete implementations are in place.


Authenticated user

Using the Facebook oath2 settings I've managed to connect and authenticate with Facebook, but every time I authenticate the same user is added to the DB (using NeDB), with the same id, _id and facebookId — over and over.
As if it's not checking if the user exists.
The app was generated with the feathers generate command, and authentication and the like added using the feathers book documentation.

This is the normal behavior of NeDB.  Here's an example: https://github.com/louischatriot/nedb/issues/410.  See the section on persistence for how to clean it up.  I originally programmed the adapter to automatically compact every 5 seconds, but I'm not sure what it is setup to do right now.

Please open an issue for the NeDB adapter.  This isn't a problem with the auth plugin.

And the issue appears to be that the datafile isn't getting compacted often enough.


Ionic v2 should have a way to control hardware backButton

Short description of the problem:
In ionic v1, we can use following code to prevent backButton from closing app:
$ionicPlatform.registerBackButtonAction(function (e) {
//Prevent the hardware backButton from closing the app
e.preventDefault();
return false;
});
In v2, there is no way to do this, return false doesn't prevent from closing.
platform.backButton.subscribe(function(){
return false;
})
Which Ionic Version? 1.x or 2.x
2.x

For ionic 2 how can I get that functionality? (hardware back buttton for android)

+1

@GaneshPediredla it is now inplemented in the core (I think since beta5), but no way to override it. In my case I want to minimize the app instead of exiting the app when the root of the stack is reached.

I want give user a confirmation toast "Press back button again to exit app"

Yeah, previously this was working if you just listened to document.addEventListener('backbutton'), but now this is simply not working at all.  This needs to be fixed as currently there's no way to intercept the behaviour at all to customise it. Very important.

I can work on a PR for this feature, but I'm trying to determine if it would be better to allow overriding the default behavior by passing a replacement function, or just disable the default close behavior via a config value?  Any recommendations?

I think being able to override the function is more desirable.

Hello all! This will be in the next release (beta.9) with the new registerBackButtonAction method. Thanks for using Ionic everyone!

Please reopen this issue as it is not working in beta10 even tried document.addEventListener('backbutton')

I have a weird issue, the event listener works if I debug the app ionic run android -c but without -c id doesn't seem to work

It will only work if the user navigates to a second page. Not working at root page when the user has just opened the app. I am running the app using the command: ionic cordova run android -l -c

I believe I'm having the same issue as Eonio. registerBackButtonAction() is reached, but the Back event occurs regardless of the code I have. My console.log can be seen, but my confirmExit() dialog does not appear - the app just minimises, regardless of the page I'm on (root or otherwise).
I'm testing with 'ionic cordova run android -cl', but the release build does the same; this was working fine, up until we upgraded from Ionic 3.4.0 to 3.7.0, so something seems to have gone awry, unless I've missed something.


iOS 13.0 17A577 iPhone 11 Pro not jailbreaking

Describe the bug
Tried multiple times too jailbreak with different version of Uncover. Getting just an immediate crash after pressing jailbreak =(
To Reproduce
Steps to reproduce the behavior:
Just installed uncover & pressed the jailbreak button. Then crash.
Device (please complete the following information):

iOS Version: iOS 13 Stable Build 17A577
iOS Device: iPhone 11 Pro
unc0ver Version: 4.0.3

Place an "x" between the brackets if true:

[ ] this is a bug others will be able to reproduce
[X] this issue is present with all tweaks uninstalled(except for default packages) or disabled
[X] this issue is present after a rootfs restore
[X] this issue is present on the latest version of unc0ver

Logs
If applicable, add logs or error messages here.

iphone 11pro ios 13.2.2 - 4.0.3 - not work too(((

i had same problem with all versions 4.0.1-4.0.3 on 11pro with 13.1.1 . Turn plane mode and power off . After power on open unc0ver but dont push the JB button! you must wait some time 2-5 min and then push the button. After that phone reboots after exploiting kernel and after it boots repeat steps again , open app , wait some time and then push the button. As I wrote you must repeat it again and again and again ...

@isemris thanks for the advice but it doesn't work.. Tried it multiple times / even setting a timer on my Alexa to make sure I waited 4 minutes lol.. nothing. Just the same thing over / over again.

I have jailbreak the iphone correctly !!!
I had to restore the parameters and the data
and then proceed to the jailbreak which has been successfully completed

https://twitter.com/pwn20wnd/status/1230595794429890560?s=21
https://twitter.com/razmashat/status/1230959939885109253?s=21
This is a known issue and will likely be resolved in the next couple days.


[CT-45] [Feature] Expose Snowflake query id in case of dbt test failure

Is there an existing feature request for this?

[X] I have searched the existing issues

Describe the Feature
When we get a dbt test failure, I often look through the debug logs to find the exact query which was run.  Having the dbt_internal_test query with failures, should_warn, and should_error are an amazing first step, but the issue is that when I rerun the query, it often passes.  If the debug logs (or the friendly logs) contained the sfqid, then I could easily modify the query to use snowflake's timetravel feature, so I can recreate the failing test and debug it further.
Right now, I see the following info is logged:
022-01-14 00:14:37.419997 (Thread-39): 00:14:37  On test.[redacted].c630bf19e2: /* {"app": "dbt", "dbt_version": "1.0.1", "profile_name": "user", "target_name": "default", "node_id": "[redacted]"} */

Related issue: https://github.com/dbt-labs/dbt-core/pull/2358
Describe alternatives you've considered
Right now, I look through the snowflake query history to find the sfqid, which works but is a bit tedious.
Who will this benefit?
Analytics engineers running dbt tests being able to track down issues faster.  Speed is critical because the standard edition of snowflake only allows for 1 day of timetravel, so you need to debug it quickly.
The easier it is to track down intermittent test issues the better, so that people are encouraged to write more.
Are you interested in contributing this feature?
Sure!  I have a bit of experience with python (mostly ruby/javascript), but could figure it out if I know:

people think this is a good idea (so it's not a waste of everyone's time)
a point in the right direction

Anything else?
I'm loving dbt!  Thanks for all of the hard work.

@jtcohen6 thank you for your answer above. I upgraded to dbt-snowflake 1.2.0 (same as dbt) and have access to query_id under adapter_response of result object but I get null query id when a query fails and it shows query id only for successful model run. Is that expected ?

@smitthakkar1 you're right! That's because adapter_response is actually never populated for failing queries. We can take one of two approaches here:

Do here as dbt-bigquery does, and wrap query submissions in a custom exception handler that logs out the job_id before raising the exception
Make changes within dbt-core to enable this pattern. Either by logging like above, or change the way that we try/except the execution of model materializations.

I prefer the second approach if possible; I'll open a dbt-core issue about this.

@jtcohen6 Thanks for your reply.

We already do this, in dbt-snowflake. Even though the query_id doesn't appear in run_results.json, it does still appear in the debug-level logs

Any idea on how to access those debug level logs via dbt template and print it ?

Given this is already in the debug logs, would y'all be open to optionally appending the query ID to the error message based on some config or env var?


Unresolvable Alias

# Unresolvable destination alias '' for rule 'NAT ' label "USER_RULE: NAT "

From rules.debug the rule must be

And the alias exists (at least in the configuration)

and it is even set up in the rules.debug
table <telegram> {   <IP_ADDRESS> }
telegram = "<telegram>"

@fabianfrz which version are you using? and can you dump the xml part of this rule? It doesn't look like it's easily reproducible.

Version is 17.1.2 and this is the rule:
    <rule>
      <protocol>tcp</protocol>
      <interface>lan</interface>
      <ipprotocol>inet</ipprotocol>
      <descr/>
      <associated-rule-id>nat_58bd8197899700.05313726</associated-rule-id>
      <nordr>1</nordr>
      <source>
        <any>1</any>
      </source>
      <destination>
        <address>telegram</address>
        <port>443</port>
      </destination>
      <updated>
        <username>root@<removed ip></username>
        <time>1488814487.5639</time>
        <description>/firewall_nat_edit.php made changes</description>
      </updated>
      <created>
        <username>root@<removed ip></username>
        <time>1488814487.5639</time>
        <description>/firewall_nat_edit.php made changes</description>
      </created>
      <disabled>1</disabled>
    </rule>

the associated rule cannot be found (does not exist) in the xml but there is another rule with the same associated rule:
    <rule>
      <source>
        <any>1</any>
      </source>
      <interface>lan</interface>
      <protocol>tcp</protocol>
      <ipprotocol>inet</ipprotocol>
      <destination>
        <address/>
        <port>80</port>
      </destination>
      <descr>NAT </descr>
      <associated-rule-id>nat_58bd8197899700.05313726</associated-rule-id>
      <created>
        <username>root@<removed ip></username>
        <time>1488814487.5637</time>
        <description>/firewall_nat_edit.php made changes</description>
      </created>
    </rule>

@fabianfrz I think this message is from another rule.... $rule['destination']['address'] doesn't appear to be empty. you could try naming your rule and inspect the message again, I expect it won't output the name, see https://github.com/opnsense/core/blob/master/src/etc/inc/filter.inc#L2112 .

I have added the descriptions and it fixed the problem. Beside the relevant changes (updated description and timestamps), the following happened:
The associated rule got emptied (as it should be)
-      <descr/>
-      <associated-rule-id>nat_58bd8197899700.05313726</associated-rule-id>
+      <descr>do not forward telegram to the transparent proxy</descr>
+      <associated-rule-id/>

And the strange rule deleted:
-      <source>
-        <any>1</any>
-      </source>
-      <interface>lan</interface>
-      <protocol>tcp</protocol>
-      <ipprotocol>inet</ipprotocol>
-      <destination>
-        <address/>
-        <port>80</port>
-      </destination>
-      <descr>NAT </descr>
-      <associated-rule-id>nat_58bd8197899700.05313726</associated-rule-id>
-      <created>
-        <username>root@<removed ip></username>
-        <time>1488814487.5637</time>
-        <description>/firewall_nat_edit.php made changes</description>
-      </created>
-    </rule>
-    <rule>

So for some reason, it looks like it created a broken associated rule.

Timeout + subsystem rewritten.

Didn't know that I had this ;)


Remove unnecessary rubygems filters

What was the end-user problem that led to this PR?
The problem was there's a lot of unnecessary code, specially in specs, that's never run.
What was your diagnosis of the problem?
My diagnosis was that since dropping support for old rubygems versions, this code is no longer necessary.
What is your fix for the problem, implemented in this PR?
My fix is to remove the code.
Why did you choose this fix out of the possible options?
I chose this fix because red is great on diffs.

Seems good.

@bundlerbot r=hsbt


MSFT_xExchWaitForADPrep: Fix PSSA Issues

PowerShell Script Analyzer is bringing up a few issues for the MSFT_xExchWaitForADPrep resource.
No errors, but there are a few warning/informational issues that should be either fixed or excluded.
This is the result of running Invoke-ScriptAnalyzer:

(Please disregard issues brought up by the PSDSCDscExamplesPresent and PSDSCDscTestsPresent rules for now.)

Consolidating into https://github.com/PowerShell/xExchange/issues/149


Unexpected token export

Actually subj. Uncaught SyntaxError: Unexpected token export error happens when I use the latest version of the library. If I use ~1.3.0 it does not happen.
What kind of additional input would you need to fix the issue?

Checking that.
You can use 1.4.1 for now. I will let you know when the bug will be fixed

@smnbbrv please try 1.4.5

@troyanskiy works fine, thank you Sir!


test

@laojiuh, you must sign our contributor license agreement before your changes are merged. Click here to sign the agreement. If you are a VMware employee, read this for further instruction.

@laojiuh, we have received your signed contributor license agreement. The review is usually completed within a week, but may take longer under certain circumstances. Another comment will be added to the pull request to notify you when the merge can proceed.

@laojiuh, we have received your signed contributor license agreement. The review is usually completed within a week, but may take longer under certain circumstances. Another comment will be added to the pull request to notify you when the merge can proceed.

@laojiuh, we have received your signed contributor license agreement. The review is usually completed within a week, but may take longer under certain circumstances. Another comment will be added to the pull request to notify you when the merge can proceed.


Bug: Mismatch in Studio and Engine handling of unresolved class mapping ID

Similar issues

[X] I have searched and found no existing similar issues

How are you using Studio?
Legend Studio
Current and expected behavior
A number of Pure users have mappings which contain class mapping IDs pointing at another mapping's class mappings. In Pure/Engine, we let this pass compilation phase. In Studio, we disallow this. This makes it hard for users to migrate to Studio. We should think of a strategy to make things loadable in Studio, but disallow users to make changes if they have this kind of error in their graph.
Steps to reproduce
Use the following grammar. Try compilation in text mode, Engine will pass while Studio graph builder will scream.
Model data
###Relational
Database model::Test
(
  Table FirmTable
  (
    id INTEGER PRIMARY KEY,
    Legal_name VARCHAR(200)
  )
  Table PersonTable
  (
    id INTEGER PRIMARY KEY,
    firm_id INTEGER,
    firstName VARCHAR(200),
    lastName VARCHAR(200)
  )

  Join FirmPerson(PersonTable.firm_id = FirmTable.id)
)

###Pure
Profile model::MyExtension
{
  stereotypes: [important];
  tags: [doc];
}

Enum model::IncType
{
  Corp,
  LLC
}

Class model::LegalEntity
{
  legalName: String[1];
}

Class model::Person
{
  firstName: String[1];
  lastName: String[1];
}

Class <<model::MyExtension.important>> {model::MyExtension.doc = 'This is a model of a firm'} model::Firm extends model::LegalEntity
[
  validName: $this.legalName->startsWith('')
]
{
  employees: model::Person[1..*];
  incType: model::IncType[1];
  employeeSize() {$this.employees->count()}: Number[1];
}

###Mapping
Mapping model::NewMapping
(
  *model::Firm: Relational
  {
    ~primaryKey
    (
      [model::Test]FirmTable.id
    )
    ~mainTable [model::Test]FirmTable
    legalName: concat([model::Test]FirmTable.Legal_name, '_LTD'),
    employees[idNotFound]: [model::Test]@FirmPerson
  }
  *model::Person: Relational
  {
    ~primaryKey
    (
      [model::Test]PersonTable.id
    )
    ~mainTable [model::Test]PersonTable
    firstName: [model::Test]PersonTable.firstName,
    lastName: [model::Test]PersonTable.lastName
  }
)

Environment
System:
    OS: macOS 11.6.4
  Binaries:
    Node: 16.8.0 - /private/var/folders/95/y32364512q77c1dvhkpnr_n80000gn/T/xfs-e9ca549e/node
    Yarn: 3.2.0 - /private/var/folders/95/y32364512q77c1dvhkpnr_n80000gn/T/xfs-e9ca549e/yarn
    npm: 7.21.0 - ~/.nvm/versions/node/v16.8.0/bin/npm
  Browsers:
    Chrome: 98.0.4758.102
    Firefox: 96.0.3

Possible solution and workaround
Discussed with @pierredebelen and @MauricioUyaguari. We will do the followings:

[x] Make this not hard-failing in Studio
[ ] Change engine to return this as a warning as part of compilation result - @pierredebelen
[ ] Change the behavior of the tool to disallow saving if compilation failed
[ ] Studio to handle these warnings and disallow saving when there are warnings like these

Contribution

[ ] I would like to work on the fix for this issue

Moved to milestone 5.0.0 because this work needs engine change from @pierredebelen


[WiP] Handle duplicate UE IP+TEIDs across multiple sessions

This patch makes sure that a PDIs don't conflict for the PDIs for the
same NWI from other sessions based on UE IPs and TEIDs. This is not
yet a complete implementation of the spec in this respect b/c other
IEs in PDI also need to be taken into account.
Previously, adding many duplicate UE IPs could cause UPG to
crash (about 255) due to FIB entry src count limit, and, in addition
to that, deleting sessions with duplicate UEs could lead to an
assertion failure due to a bad session reference.
Now, if a conflict is detected, the corresponding Session Establishment Request
or Session Modification Request fails with "rule creation/modification failure" as the cause.
Fixes #25.

[X] detect duplicate UE IP+TEIDs across multiple sessions
[X] E2E test for TDF case (no TEIDs, just UE IPs)
[ ] fix bihash display for the UE IP index (this problem causes some of the Scapy-based integration tests to fail)
[ ] E2E test for GTP-U case (should tolerate duplicate UE IPs if TEIDs differ)
[ ] verify session modification
[ ] take care of // TODO: display UE IP + teid (improve debug print)

I wonder if the UE IP check really needs a hash. Wouldn't it suffice to perform a FIB lookup on the SGi/N6 side fib table and reject the request if we find a /32 (/64 IPv6) entry for the IP that points to a different session?

@RoadRunnr I thought about that too initially, but note that this PR also checks for duplicate UE IP+TEID pairs on NWIs, in which case there are no session DPOs.

@RoadRunnr I thought about that too initially, but note that this PR also checks for duplicate UE IP+TEID pairs on NWIs, in which case there are no session DPOs.

Yes, but a duplicate TEID + UE IP within the same session are valid. I'm not sure what exactly you want to prevent with that check?
A duplicate Fq-TEID (GTP-IP + TEID) across multiple sessions is not valid regardless of the UE IP, but that should be handled already.

Hmm, probably I've missed TEID duplication check across multiple sessions. handle_f_teid() doesn't appear to have one https://github.com/travelping/upg-vpp/blob/master/upf/upf_pfcp_api.c#L677-L750 nor do handle_create_pdr() / handle_update_pdr(). Will check a bit more. The idea is to prevent the duplication early, before applying any changes to the session. Although, using UE IP for that is incorrect indeed, v[46]_tunnel_by_key should be used instead

Hmm, probably I've missed TEID duplication check across multiple sessions. handle_f_teid() doesn't appear to have one https://github.com/travelping/upg-vpp/blob/master/upf/upf_pfcp_api.c#L677-L750 nor do handle_create_pdr() / handle_update_pdr(). Will check a bit more. The idea is to prevent the duplication early, before applying any changes to the session. Although, using UE IP for that is incorrect indeed, v[46]_tunnel_by_key should be used instead

You are right, there is not check. I thought there was one. Duplicate TEID are prevented to create havoc by the the fact that the {v4,v6}_tunnel_by_key will not accept duplicate keys, newer tunnel will always replace existing tunnels.
We should probably check all use case of bihashes, change the is_add argument to 2 (meaning add but not overwrite) and check the return.
Checking the hashes for collisions is a good idea. Putting the check into handle_f_teid() https://github.com/travelping/upg-vpp/blob/master/upf/upf_pfcp_api.c#L745 looks like the best place. The TEID generation can not generate colliding TEIDs by design.


models/practice payload is missing direction ("t2d" or "d2t")

Unless I missed something

good point


Seeking relative milliseconds

public abstract void seekRelativeMillis (int milliSeconds)
Seeks forward or backward by the specified number of seconds.

milliSeconds
Number of milliseconds to seek forward, if positive, or backward, if negative.

This method is presented in YouTube API, is it possible to get something like, with using this Player?

@PierfrancescoSoffritti is this enhancement in version 4.1.7? I cannot find it.

Hi, no it is not. I don't want to clutter the API too much with noise, it s easy for library s users to build a method do to just that.
But,  I may release an helper for that in the next version.


Docker: init_by_lua:3: module 'vips' not found

Hi,
Currently I have been unable to self-hosting using Docker.  When a try use I get "500 Internal Server Error".
Checking the logs a found this.
docker exec imagesweserv cat /var/www/imagesweserv/logs/supervisor/nginx_error.log
nginx: [error] [lua] init_by_lua:10: Failed to disable libvips operations cache init_by_lua:3: module 'vips' not found:
        no field package.preload['vips']
        no file '/var/www/imagesweserv/app/vips.lua'
        no file '/var/www/imagesweserv/src/vips.lua'
        no file '/usr/local/openresty/site/lualib/vips.ljbc'
        no file '/usr/local/openresty/site/lualib/vips/init.ljbc'
        no file '/usr/local/openresty/lualib/vips.ljbc'
        no file '/usr/local/openresty/lualib/vips/init.ljbc'
        no file '/usr/local/openresty/site/lualib/vips.lua'
        no file '/usr/local/openresty/site/lualib/vips/init.lua'
        no file '/usr/local/openresty/lualib/vips.lua'
        no file '/usr/local/openresty/lualib/vips/init.lua'
        no file './vips.lua'
        no file '/usr/local/openresty/luajit/share/luajit-2.1.0-beta3/vips.lua'
        no file '/usr/local/share/lua/5.1/vips.lua'
        no file '/usr/local/share/lua/5.1/vips/init.lua'
        no file '/usr/local/openresty/luajit/share/lua/5.1/vips.lua'
        no file '/usr/local/openresty/luajit/share/lua/5.1/vips/init.lua'
        no file '/usr/local/openresty/site/lualib/vips.so'
        no file '/usr/local/openresty/lualib/vips.so'
        no file './vips.so'
        no file '/usr/local/lib/lua/5.1/vips.so'
        no file '/usr/local/openresty/luajit/lib/lua/5.1/vips.so'
        no file '/usr/local/lib/lua/5.1/loadall.so'
nginx: [error] [lua] init_by_lua:10: Failed to disable libvips operations cache init_by_lua:3: module 'vips' not found:
        no field package.preload['vips']
        no file '/var/www/imagesweserv/app/vips.lua'
        no file '/var/www/imagesweserv/src/vips.lua'
        no file '/usr/local/openresty/site/lualib/vips.ljbc'
        no file '/usr/local/openresty/site/lualib/vips/init.ljbc'
        no file '/usr/local/openresty/lualib/vips.ljbc'
        no file '/usr/local/openresty/lualib/vips/init.ljbc'
        no file '/usr/local/openresty/site/lualib/vips.lua'
        no file '/usr/local/openresty/site/lualib/vips/init.lua'
        no file '/usr/local/openresty/lualib/vips.lua'
        no file '/usr/local/openresty/lualib/vips/init.lua'
        no file './vips.lua'
        no file '/usr/local/openresty/luajit/share/luajit-2.1.0-beta3/vips.lua'
        no file '/usr/local/share/lua/5.1/vips.lua'
        no file '/usr/local/share/lua/5.1/vips/init.lua'
        no file '/usr/local/openresty/luajit/share/lua/5.1/vips.lua'
        no file '/usr/local/openresty/luajit/share/lua/5.1/vips/init.lua'
        no file '/usr/local/openresty/site/lualib/vips.so'
        no file '/usr/local/openresty/lualib/vips.so'
        no file './vips.so'
        no file '/usr/local/lib/lua/5.1/vips.so'
        no file '/usr/local/openresty/luajit/lib/lua/5.1/vips.so'
        no file '/usr/local/lib/lua/5.1/loadall.so'
nginx: [error] [lua] init_by_lua:10: Failed to disable libvips operations cache init_by_lua:3: module 'vips' not found:
        no field package.preload['vips']
        no file '/var/www/imagesweserv/app/vips.lua'
        no file '/var/www/imagesweserv/src/vips.lua'
        no file '/usr/local/openresty/site/lualib/vips.ljbc'
        no file '/usr/local/openresty/site/lualib/vips/init.ljbc'
        no file '/usr/local/openresty/lualib/vips.ljbc'
        no file '/usr/local/openresty/lualib/vips/init.ljbc'
        no file '/usr/local/openresty/site/lualib/vips.lua'
        no file '/usr/local/openresty/site/lualib/vips/init.lua'
        no file '/usr/local/openresty/lualib/vips.lua'
        no file '/usr/local/openresty/lualib/vips/init.lua'
        no file './vips.lua'
        no file '/usr/local/openresty/luajit/share/luajit-2.1.0-beta3/vips.lua'
        no file '/usr/local/share/lua/5.1/vips.lua'
        no file '/usr/local/share/lua/5.1/vips/init.lua'
        no file '/usr/local/openresty/luajit/share/lua/5.1/vips.lua'
        no file '/usr/local/openresty/luajit/share/lua/5.1/vips/init.lua'
        no file '/usr/local/openresty/site/lualib/vips.so'
        no file '/usr/local/openresty/lualib/vips.so'
        no file './vips.so'
        no file '/usr/local/lib/lua/5.1/vips.so'
        no file '/usr/local/openresty/luajit/lib/lua/5.1/vips.so'
        no file '/usr/local/lib/lua/5.1/loadall.so'

But as you can see below the library "vips" is already installed.
docker exec imagesweserv luarocks list
Warning: The directory '/root/.cache/luarocks' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing /usr/local/openresty/luajit/bin/luarocks with sudo, you may want sudo's -H flag.

Installed rocks:
----------------

argparse
   0.6.0-1 (installed) - /usr/local/openresty/luajit/lib/luarocks/rocks

busted
   2.0.rc13-0 (installed) - /usr/local/openresty/luajit/lib/luarocks/rocks

dkjson
   2.5-2 (installed) - /usr/local/openresty/luajit/lib/luarocks/rocks

lua-path
   0.3.1-1 (installed) - /usr/local/openresty/luajit/lib/luarocks/rocks

lua-resty-http
   0.12-0 (installed) - /usr/local/openresty/luajit/lib/luarocks/rocks

lua-resty-template
   1.9-1 (installed) - /usr/local/openresty/luajit/lib/luarocks/rocks

lua-term
   0.7-1 (installed) - /usr/local/openresty/luajit/lib/luarocks/rocks

lua-vips
   1.1-9 (installed) - /usr/local/openresty/luajit/lib/luarocks/rocks

lua_cliargs
   3.0-1 (installed) - /usr/local/openresty/luajit/lib/luarocks/rocks

luacheck
   0.23.0-1 (installed) - /usr/local/openresty/luajit/lib/luarocks/rocks

luacov
   0.13.0-1 (installed) - /usr/local/openresty/luajit/lib/luarocks/rocks

luacov-coveralls
   0.2.2-1 (installed) - /usr/local/openresty/luajit/lib/luarocks/rocks

luafilesystem
   1.7.0-2 (installed) - /usr/local/openresty/luajit/lib/luarocks/rocks

luassert
   1.7.11-0 (installed) - /usr/local/openresty/luajit/lib/luarocks/rocks

luasystem
   0.2.1-0 (installed) - /usr/local/openresty/luajit/lib/luarocks/rocks

mediator_lua
   1.1.2-0 (installed) - /usr/local/openresty/luajit/lib/luarocks/rocks

penlight
   1.5.4-1 (installed) - /usr/local/openresty/luajit/lib/luarocks/rocks

say
   1.3-1 (installed) - /usr/local/openresty/luajit/lib/luarocks/rocks

And the files are there!
docker exec imagesweserv find / -iname '*vips*'
/var/lib/yum/yumdb/v/71ddac65a02b916ee592948140621dceabc98225-vips-8.8.0-1.el7.remi-x86_64
/var/www/imagesweserv/.ci/install-vips.sh
/usr/share/licenses/vips-8.8.0
/usr/share/locale/en_GB/LC_MESSAGES/vips8.8.mo
/usr/local/openresty/luajit/share/lua/5.1/vips
/usr/local/openresty/luajit/share/lua/5.1/vips.lua
/usr/local/openresty/luajit/lib/luarocks/rocks/lua-vips
/usr/local/openresty/luajit/lib/luarocks/rocks/lua-vips/1.1-9/lua-vips-1.1-9.rockspec
/usr/lib64/girepository-1.0/Vips-8.0.typelib
/usr/lib64/libvips.so
/usr/lib64/libvips.so.42
/usr/lib64/libvips-cpp.so.42.10.0
/usr/lib64/libvips.so.42.10.0
/usr/lib64/ImageMagick-6.9.10/modules-Q16/coders/vips.la
/usr/lib64/ImageMagick-6.9.10/modules-Q16/coders/vips.so
/usr/lib64/libvips-cpp.so.42

I will be realy greatfull if someone give me a help. Unfontuly I don't know anything related to Lua, so a I'm completly lost.
If you need more information just ask. I will provide as fast I can.

Looks like vips is not fully installed. Did you ran the command  docker exec imagesweserv make dev?

Yes. And to double check I just create a new VPS then I installed Docker and followed the tutorial's instructions.

I'm sorry to hear that. We'll look into it tomorrow. It may be that NGINX/openresty needs to be restarted after the make dev, or it is a problem with the latest libvips release, or something else we didn't consider/test when making the Docker image.

I'm looking forward for the solution. This is a awesome project I really want to use.

I just pushed c42a769 which should fix this. Could you test this?
Note that running docker exec imagesweserv make dev isn't required anymore.

@kleisauke, you are awesome. Thank you very much. It worked as expected.


Support for scipy > 1.10.1

I saw from the list of changes in 0.5.9 that https://github.com/biocore/scikit-bio/pull/1867 caps the scipy version at 1.10.1. In nixpkgs, we carry only one version (usually the latest) of each Python package, and our scipy version is already at 1.11.2. I'm doing some information gathering to decide what we should do:

What is the incompatibility with later versions of scipy, and what sort of degradation in functionality exists?
Does this affect 0.5.8 as well?

Thank you for any insight into this!

If I remember correctly it was something with the kulsinski tests failing; you can check the build before the cap.
Anyway, looking online the solution might be as simple as replacing kulsinski for kulsinski1 but not sure:
https://docs.scipy.org/doc/scipy-1.10.0/reference/generated/scipy.spatial.distance.kulsinski.html
Thoughts?

@antgonza I think we should address this issue. kulsinski as a beta diversity metric is only mentioned in an internal list that is not exposed to the user. I performed literature search and couldn't find any paper that uses kulsinski or kulczynski1 as a beta diversity metric. As this comment pointed out, kulczynski1 returns inf for a vector against itself. In my test, kulsinski also returns a positive value for a vector against itself. For example, kulsinski(np.array([1,0,1,0,1]), np.array([1,0,1,0,1])) is 0.4. Therefore, I think either metric is not a suitable beta diversity metric, and we should simply remove it from skbio.

After a quick search I found these papers that cite this metric (used google vs. pubmed):

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8778696/
https://www.frontiersin.org/articles/10.3389/fbioe.2020.556413/full
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8418097/pdf/main.pdf
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2948603/#SD1

However, I agree that fixing or removing so we can remove the cap is a good idea.

@antgonza Thanks for the findings and opinion!
I checked the four literatures you found. I think that they may not be ideal cases supporting using Kulsinski as a beta diversity metric. Specifically, 1) used Kulsinski as an alpha diversity metric. 2) used it to assess genomic distance (along with metrics like L1/L2 and Hamming). 3) used it to evaluate clustering of disease groups (not microbial communities). 4) the first author's name is "Kulsinski" but the paper didn't use this metric.

Then sounds like it's fine to remove.
Note that the 4th the metric is used as part of the original comparison (and it might be one of the reason it was included in the original Qiime); if you check the supplemental you would see the results there.

@antgonza I did note that kulsinski is mentioned somewhere (a few places) in the current QIIME 2 documentation. Therefore to safely remove it we need to coordinate with @ebolyen .

According to https://github.com/scipy/scipy/issues/19274#issuecomment-1731862215 at least scipy 1.11.2 is needed for python 3.12. As long as scipy<1.10.1 is capped, scikit-bio is not able to run with python3.12.

@jowodo @antgonza Thanks for the comment! We have added support for SciPy 1.11+ in #1887 .
The next challenge is to add Python 3.12 support. Although scikit-bio runs on Python 3.12, some unit tests fail due to gzip file content discrepancy. This needs to be addressed as high priority.


Fix travis-ci

python3.4 compatibility
environment marker under setuptools 17.1
attributes of typing are changed

@admire93 Seems you need to upgrade setuptools to 17.1+.  >/< operators with environment markers were provided since setuptools 17.1.

@admire93 Travis CI 그 사이에 장애가 있었다고 합니다. 마지막 커밋만 amend해서 다시 올려주실 수 있을까요?


Add about Requirements documentation, building and installing process in Readme Section

What would you like to be added?
Requirements documentation, Building and installing process in Readme Section.
Why is this needed?
I like to add about the Requirements (dependecies), Building and installing process in Readme Section which will help to the new contributor to easily install our kubeedge dashboard project in their local system.

Hi @vincentgoat let me know is this make sense for you.

Hey,it will be done after the mentee selection not right now. Please wait.

@Amishakumari544 Is this impact on something if we do before mentee selection.

well,for now, there is nothing added in this repo then it might possible that you may not add all the requirements. For further @vincentgoat can tell you if it's good to add or not.🙂


[Authz v2] Add e2e test for Authz v2

Describe the feature request
E2E test is very important to prevent any regressions, we will need to update the current ones for the upcoming Authz v2 support.
/cc @pitlv2109
Additional context
Istio Authorization v2 design

Close this one. Tracker for Authz v2 is here https://github.com/istio/istio/issues/12394


Does anybody know if there are similar statistics for the other hardware de-/encoders? Maybe @nyanmisaka can help?
Thanks

According to MPP dev there is no direct way here. https://github.com/rockchip-linux/mpp/issues/129
But it can be roughly estimated by the hardware time spent per frame and chip specs (e.g. up to 8k60).
# 0x100 debug on; 0x0 debug off
sudo sh -c "echo 0x100 > /sys/module/rk_vcodec/parameters/mpp_dev_debug"

# dmesg
[37651.636118] rk_vcodec: fdc48100.rkvdec-core:1 session 34096:5 time: 1170 us hw 1158 us
[37651.636723] rk_vcodec: fdc38100.rkvdec-core:0 session 34096:5 time: 1166 us hw 1158 us

I would also recommend using the FPS given by ffmpeg for estimation.


Backport "fix(#16610): warn ignored Scaladoc on multiple enum cases" to LTS

Backports #19555 to the LTS branch.
PR submitted by the release tooling.
[skip ci]

No regressions detected in the community build up to lts-19921.
Reference


Option.None should be a value

The Option type is defined as data Option = None | Some x.
Option.None should be a value.
In the library, it is implemented as a type function.
None<int>() isn't semantic, since absence of a value does not carry type information.

Hi,
This is partially a limitation of the type system and partially a design choice.
Once a None value is created and is to be used as an option type, it will have to have a type Option where T is a concrete type. Hence, the type parameter in the None() function. Otherwise, None would have to be a separate type, which would introduce a lot of problems of its own.
A "workaround" for this, which is sometimes used, is to make a specific intermediate None type, that is implicitly convertible to an Option. Although it allow nice syntax such as return Option.None, it adds a layer of confusion that I would rather be without. For example, if you try and use such a "None" value without it first being converted to an actual Option, it will not work as expected - and the extra type alone will easily cause some misunderstandings.
In Optional, I have therefore decided not to provide such a value, to keep the implementation and API as simple and explicit as possible - even if it is a little more verbose than corresponding code in F# and other functional languages.

Nils

None would be Option.
Have an implicit case operator to convert from Option to None.
This would have the effect of making none a value, rather than a type
function - which it is currently.
On Wed, Feb 1, 2017 at 12:26 AM Nils Lück<EMAIL_ADDRESS>wrote:

Hi,
This is partially a limitation of the type system and partially a design
choice.
Once a None value is created and is to be used as an option type, it will
have to have a type Option where T is a concrete type. Hence, the type
parameter in the None() function. Otherwise, None would have to be a
separate type, which would introduce a lot of problems of its own.
A "workaround" for this, which is sometimes used, is to make a specific
intermediate None type, that is implicitly convertible to an Option.
Although it allow nice syntax such as return Option.None, it adds a layer
of confusion that I would rather be without. For example, if you try and
use such a "None" value without it first being converted to an actual
Option, it will not work as expected - and the extra type alone will easily
cause some misunderstandings.
In Optional, I have therefore decided not to provide such a value, to keep
the implementation and API as simple and explicit as possible - even if it
is a little more verbose than corresponding code in F# and other functional
languages.

Nils

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/nlkl/Optional/issues/24#issuecomment-276456588, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ACQ_PvXSxmnOrspWwB0KTjZ7UmYanZGWks5rX4PcgaJpZM4LxtAM
.

Also, I noticed that there are instance methods on the Option struct.
These would require boxing to call - you should move everything to the
extension methods.
On Sun, Feb 12, 2017 at 2:59 PM Asti R<EMAIL_ADDRESS>wrote:

None would be Option.
Have an implicit case operator to convert from Option to None.
This would have the effect of making none a value, rather than a type
function - which it is currently.
On Wed, Feb 1, 2017 at 12:26 AM Nils Lück<EMAIL_ADDRESS>wrote:
Hi,
This is partially a limitation of the type system and partially a design
choice.
Once a None value is created and is to be used as an option type, it will
have to have a type Option where T is a concrete type. Hence, the type
parameter in the None() function. Otherwise, None would have to be a
separate type, which would introduce a lot of problems of its own.
A "workaround" for this, which is sometimes used, is to make a specific
intermediate None type, that is implicitly convertible to an Option.
Although it allow nice syntax such as return Option.None, it adds a layer
of confusion that I would rather be without. For example, if you try and
use such a "None" value without it first being converted to an actual
Option, it will not work as expected - and the extra type alone will easily
cause some misunderstandings.
In Optional, I have therefore decided not to provide such a value, to keep
the implementation and API as simple and explicit as possible - even if it
is a little more verbose than corresponding code in F# and other functional
languages.

Nils

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/nlkl/Optional/issues/24#issuecomment-276456588, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ACQ_PvXSxmnOrspWwB0KTjZ7UmYanZGWks5rX4PcgaJpZM4LxtAM
.

An actual None value will always have a type Option<T>. In ML like languages, type inference makes it much less clumsy to create a value, as T can be inferred instead of being specified explicitly, but a None value is still of type Option<T>.
C# does not have such sophisticated type inference, and one has to decide how to support creating None values. The most direct translation from the F# equivalent would probably be Option<T>.None as a static singleton value on Option<T>. However, as a matter of preference, I have opted for a generic create-method instead of calling a property on a generic type (Option.None<T>() vs Option<T>.None).
It is purely a matter of my own personal aesthetic preference. Both cases require the type parameter to be specified explicitly, as type inference in C# isn't powerful enough to easily circumvent this. Performance should be no problem, as creating a simple struct (such as an option) is very cheap.
The only workaround for the explicit type annotation, is to create an implicit type conversion from another type. This reduces some of the clumsiness, but at the cost of unnecessary complexity in my point of view. If you have a preference for this method, you are of course very welcome to implement and use your own such type in your code, but it will, for now at least, not be part of Optional.
Regarding instance methods, your assumption is wrong. Instance methods on structs do not lead to boxing (the exception being methods derived from object, such as ToString(), but only in case they are not overriden in the struct itself - which they are in this case). As such, I see no value in moving these to extension methods.
/Nils

Here's a basic implementation of what I was talking about:
public struct Unit
{
    public static readonly Unit Default = default(Unit);
}

public struct Option<T>
{
    public static readonly Option<T> None;

    public bool HasValue { get; }
    public T Value { get; }
    public Option(T value)
    {
        Value = value;
        HasValue = true;
    }

    public static implicit operator Option<T>(Option<Unit> value)
    {
        return Option<T>.None;
    }
}

public static class Option
{
    public static Option<Unit> None => Option<Unit>.None;
    public static Option<T> Some<T>(T value) => new Option<T>(value);

   //simpler operator implementation
    public static Option<TResult> SelectMany<T, TResult>(this Option<T> self, Func<T, Option<TResult>> selector)
    {
        return self.HasValue ? selector(self.Value) : None;
    }
}

It behaves like a true value:

var someint = Option.Some(2);
var somefloat = Option.Some(1f);
var none = Option.None;
someint = none; //compiles
somefloat = none; //compiles

I am not saying that it isn't possible - it definitely is. I am just saying that it isn't going to be part of Optional for now, as a conversion is a hack that saves a tiny bit of verbosity, but adds some less-than-obvious complexity underneath.

Okay, Nils. I can respect that. :)
Good luck with your future projects!


Remove version: DoltHub.Dolt version 1.40.0

[Automated] It returns code over 400 in all urls
Microsoft Reviewers: Open in CodeFlow

For moderators or/and anyone else

[!Caution]
As a result of several personal problems I've had over the last few months + my final exams coming up very soon, I'm currently unable to look at my GitHub notifications every day and maintain my executions correctly.
If you detect a certain problem on a certain pul request:

Either you can close the pull request directly without mentioning me.
Or you can send me a message on X (Twitter): @Exorcism0666

Thank you for your understanding, and I'm sorry if some pull requests can't be approved due to the situation I'm in.

This account is automated by Github Actions and the source code was created by CoolPlayLin. If you have any questions about any pull request, don't hesitate to ping Exorcism0666.

[!warning]
With the new version of Komac, numerous Pull Requests may contain bugs or unwanted elements. If such issues arise, please tag russellbanks so that these bugs can be addressed in future updates.

[!important]
Please carefully review these Pull Request before merging. If it is a Pull Request for removing incorrect content and the URLs are issue, free upon manual checking, please close this Pull Request directly. (It is best to inform Exorcism0666 of the closure.)

良い一日をお過ごしください！

URL: https://github.com/dolthub/dolt/releases/download/v1.40.0/dolt-windows-amd64.msi
Response status code does not indicate success: 404 (Not Found).
(Automated message - build 886)

/azp run

https://github.com/microsoft/winget-pkgs/issues/142854


Fix inaccurate MySQL/MariaDB, Redis metrics

[x] Update the CHANGES log.
snapshots:

I think those service dashboards should be updated to use MQE to aggregate instance metrics.

Redis e2e fails, please fix it.

please check and help fix the hierarchy part: https://github.com/apache/skywalking/blob/7e978261fcb2bf2f619b8f2a9586e1703107f0a3/oap-server/server-starter/src/main/resources/ui-initialized-templates/mysql/mysql-service.json#L301

you didn't add instance_id in these expressions:
https://github.com/apache/skywalking/blob/7e978261fcb2bf2f619b8f2a9586e1703107f0a3/oap-server/server-starter/src/main/resources/otel-rules/mysql/mysql-service.yaml#L37

Got it, sorry I don't have time tonight, I'll fix it tomorrow.

This PR is not yet complete (the UI part has not been modified). I'm very sorry as I had many things at school this weekend and didn't have time to work on it. I will definitely finish this PR tomorrow.

Please fix the relative e2e.

PR is ready if e2e pass.


SPI DMA PSRAM QuadSPI  half_duplex_write() doesn't resolve

Bug description
Exact cause in not clear to me, but attempting to send a larger buffer with SPI DMA using PSRAM causes no interrupts to trigger (manually verified), and thus never resolves is_done().
I can only reliably reproduce it with half_duplex_write, although I think I've also seen it with regular write().
To Reproduce
Run example: https://github.com/i404788/esp-hal/blob/spi-dma-wip/examples/src/bin/spi_loopback_dma_psram.rs
Logs:
 Chip type:         esp32s3 (revision v0.1)
Crystal frequency: 40 MHz
Flash size:        16MB
Features:          WiFi, BLE
MAC address:       68:b6:b3:31:64:e8
App/part. size:    113,824/1,048,576 bytes, 10.86%
[2024-12-23T13:22:30Z INFO ] Segment at address '0x0' has not changed, skipping write
[2024-12-23T13:22:30Z INFO ] Segment at address '0x8000' has not changed, skipping write
[00:00:04] [========================================]      44/44      0x10000                                                                                [2024-12-23T13:22:36Z INFO ] Flashing has completed!
Commands:
    CTRL+R    Reset chip
    CTRL+C    Exit

ESP-ROM:esp32s3-20210327
Build:Mar 27 2021
rst:0x1 (POWERON),boot:0x2b (SPI_FAST_FLASH_BOOT)
SPIWP:0xee
mode:DIO, clock div:2
load:0x3fce3818,len:0x16f8
load:0x403c9700,len:0x4
load:0x403c9704,len:0xc00
load:0x403cc700,len:0x2eb0
entry 0x403c9908
I (30) boot: ESP-IDF v5.1-beta1-378-gea5e0ff298-dirt 2nd stage bootloader
I (30) boot: compile time Jun  7 2023 08:07:32
I (31) boot: Multicore bootloader
I (35) boot: chip revision: v0.1
I (39) boot.esp32s3: Boot SPI Speed : 40MHz
I (44) boot.esp32s3: SPI Mode       : DIO
I (49) boot.esp32s3: SPI Flash Size : 4MB
I (53) boot: Enabling RNG early entropy source...
I (59) boot: Partition Table:
I (62) boot: ## Label            Usage          Type ST Offset   Length
I (70) boot:  0 nvs              WiFi data        01 02 00009000 00006000
I (77) boot:  1 phy_init         RF data          01 01 0000f000 00001000
I (85) boot:  2 factory          factory app      00 00 00010000 00100000
I (92) boot: End of partition table
I (96) esp_image: segment 0: paddr=00010020 vaddr=3c010020 size=03fe4h ( 16356) map
I (109) esp_image: segment 1: paddr=0001400c vaddr=3fc8a010 size=00aa8h (  2728) load
I (114) esp_image: segment 2: paddr=00014abc vaddr=40378000 size=02010h (  8208) load
I (124) esp_image: segment 3: paddr=00016ad4 vaddr=00000000 size=09544h ( 38212)
I (139) esp_image: segment 4: paddr=00020020 vaddr=42000020 size=0bc50h ( 48208) map
I (153) boot: Loaded app from partition at offset 0x10000
I (153) boot: Disabling RNG early entropy source...

Expected behavior
Expected it to either error, or complete successfully. See the Done log repeatedly.
Environment

Target device: ESP32-S3]
Crate name and version: main

Update: found that the bus speed is relevant: <20Mhz still works 25Mhz fails

Thanks for the tidbit about bus speed. It's possible that PSRAM can't keep up with the SPI.
Would you mind printing out the SPI interrupt registers (with the PAC) a few seconds after the transfer starts?

@Dominaezzz attempted to add your change but it seems adding a delay of >600us (bisected to +-50us, in this range it occasionally works) causes it to succeed again. So it seems like there might be a race condition with an interrupt or something.
The result I'm getting is always 0 but I may be doing it wrong:
      log::info!("SPI2: {}", unsafe {
          *(SPI2::ptr().add(0x0040) as *const u32).as_ref().unwrap()
      });

I'm looking for INT_RAW rather than INT_ST, which is 0x003C.
Did you add the delay to the example or to the SPI driver? I was thinking of here.
Since this example is blocking and not async, a race condition with an interrupt is very unlikely.

Did you add the delay to the example or to the SPI driver? I was thinking of here.

Yes exactly there, I've pushed my latest version now.

Can you add a larger delay? I still want to see what the interrupts look like if it succeeds

Hmm switched to read_volatile but still no change either way:
        log::info!("pre-SPI2: {}", unsafe {
            core::ptr::read_volatile(SPI2::ptr().add(0x003C) as *const u32)
        });

        delay.delay_millis(2000);

        log::info!("post-SPI2: {}", unsafe {
            core::ptr::read_volatile(SPI2::ptr().add(0x003C) as *const u32)
        });
        (spi, dma_buf) = transfer.wait();

INFO - pre-SPI2: 0
INFO - post-SPI2: 0

However it does not resolve again anymore. Given that; maybe it has to do with memory layout since that will only be randomized with some significant change? This seems to be confirmed by placing the delay between the prints also fails with 593us, even though placing it before the prints will succeed once then fail once. Likewise placing 750us between prints fails, placing it before the prints get repeated successes.
I'm not sure which parts require a specific alignment and how I can check if the are aligned.

The alignment requirements should be checked by DmaTxBuf, which Daniel improved the checks for recently.
Can you print out the addresses of tx_descriptors and tx_buffer? We need to make sure the latter is 64 byte aligned.

Oh looks like it's already done here.
    info!(
        "TX: {:p} len {} ({} descripters)",
        tx_buffer.as_ptr(),
        tx_buffer.len(),
        tx_descriptors.len()
    );

Hmm yeah the descriptors/buffer addresses don't seem to different:
Working:
INFO - TX: 0x3c020000 len 8192 (3 descripters)
INFO - Alignment: 64, Chunk size: 4032
INFO - First dmabuffer: Some(DmaTxBuf { descriptors: DescriptorSet { descriptors: [DmaDescriptor { flags: DmaDescriptorFlags { size: 4095, length: 4095, suc_eof: false, owner: "CPU" }, buffer: 0x3c020000, next: 0x3fc8aacc }, DmaDescriptor { flags: DmaDescriptorFlags { size: 4095, length: 4095, suc_eof: false, owner: "CPU" }, buffer: 0x3c020fff, next: 0x3fc8aad8 }, DmaDescriptor { flags: DmaDescriptorFlags { size: 2, length: 2, suc_eof: false, owner: "CPU" }, buffer: 0x3c021ffe, next: 0x0 }] }

Failing:
INFO - TX: 0x3c020000 len 8192 (3 descripters)
INFO - Alignment: 64, Chunk size: 4032
INFO - First dmabuffer: Some(DmaTxBuf { descriptors: DescriptorSet { descriptors: [DmaDescriptor { flags: DmaDescriptorFlags { size: 4095, length: 4095, suc_eof: false, owner: "CPU" }, buffer: 0x3c020000, next: 0x3fc8aacc }, DmaDescriptor { flags: DmaDescriptorFlags { size: 4095, length: 4095, suc_eof: false, owner: "CPU" }, buffer: 0x3c020fff, next: 0x3fc8aad8 }, DmaDescriptor { flags: DmaDescriptorFlags { size: 2, length: 2, suc_eof: false, owner: "CPU" }, buffer: 0x3c021ffe, next: 0x0 }] },

After much testing it's seems to be an very perplexing issue. Ghidra shows effectively no difference between a working example and a broken example (apart from the delay moving):

Also of note it seems debug builds are always broken, esp-hal@main requires 1.83 so I haven't been able to test the previous compiler versions.
Further looking through Ghidra, some of the data sections do have a layout shift of 3:

So the only conclusion I can make is that,  the layout of some unknown data (non-descriptor non-buffer?) is important.

I've run out of ideas for what could be going on but as a final hail mary, try adding a delay to the update function here. You can copy that into the gdma section and make it bigger, like 10 instead of just 1.

I tried the linked code and can reproduce the issue.
When I change DataMode to Dual it works fine - even at higher frequencies. On a device with octal-PSRAM it also works fine (with Quad)
Placing most of the involved code into RAM lets me go up to 32MHz
(maybe not all is needed but I used ESP_HAL_PLACE_SWITCH_TABLES_IN_RAM, ESP_HAL_PLACE_ANON_IN_RAM, ESP_HAL_PLACE_SPI_DRIVER_IN_RAM, put delay_micros, wait and main into RAM, removing the logs before wait )
Seems to be a bandwidth issue (i.e. placing code in RAM "eliminates" concurrently accessing flash)

I'll try both suggestions and report back. @bjoernQ assuming it's the same issue I encountered; any chance it can be detected; I'd rather it panic or Err than hang.

Hmm I put everything relevant in ram removed logs before wait, and added:
ESP_HAL_PLACE_SPI_DRIVER_IN_RAM = "true"
ESP_HAL_PLACE_ANON_IN_RAM = "true"
ESP_HAL_PLACE_SWITCH_TABLES_IN_RAM = "true"

However the env variables actually causes it to fail (commenting succeeds again). If it is flash contention, it seems like it would be at the half_duplex_write()-side rather than the wait() side, since placing a log after the delay works, but before doesn't.
I'll see if I can get a minimal break condition (e.g. call to an empty function or similar) to maybe get a better look in ghidra.
@Dominaezzz I tried your hail-mary, unfortunately doesn't seem to change anything.

I've reduced the break condition to a few vec::pushes: https://github.com/i404788/esp-hal/blob/spi-dma-wip/examples/src/bin/spi_loopback_dma_psram2.rs (note new file).  Even with black_box, ghidra shows large code shifts when changing the hard-coded bool which makes it hard to compare, I tried using bindiff but it seems like the ghidra plugin is broken on xtensa.
I've also tried manually allocating and generating some PSRAM traffic (see commented fail code) but that didn't seem to matter much.
In the fail=false condition, it just works.
In the fail=true condition without tuning I'm getting the original failure mode (infinite wait).
In the fail=true condition with the ram envs i'm getting:
====================== PANIC ======================
panicked at /home/null/.rustup/toolchains/esp/lib/rustlib/src/rust/library/core/src/panicking.rs:221:5:
unsafe precondition(s) violated: slice::from_raw_parts requires the pointer to be aligned and non-null, and the total size of the slice not to exceed `isize::MAX`

Backtrace:

0x4200b878
0x4200b878 - core::slice::raw::from_raw_parts::precondition_check
    at /home/null/.rustup/toolchains/esp/lib/rustlib/src/rust/library/core/src/ub_checks.rs:70
0x4200b586
0x4200b586 - <esp_println::Printer as core::fmt::Write>::write_str
    at /home/null/Projects/esp-hal-spi/esp-println/src/lib.rs:82
0x42005237
0x42005237 - core::fmt::Formatter::pad
    at /home/null/.rustup/toolchains/esp/lib/rustlib/src/rust/library/core/src/fmt/mod.rs:1438
0x4200bb88
0x4200bb88 - <log::Level as core::fmt::Display>::fmt
    at /home/null/.cargo/registry/src/index.crates.io-6f17d22bba15001f/log-0.4.22/src/lib.rs:534
0x42005950
0x42005950 - core::fmt::rt::Argument::fmt
    at /home/null/.rustup/toolchains/esp/lib/rustlib/src/rust/library/core/src/fmt/rt.rs:177
0x4200b54c
0x4200b54c - <&mut W as core::fmt::Write::write_fmt::SpecWriteFmt>::spec_write_fmt
    at /home/null/.rustup/toolchains/esp/lib/rustlib/src/rust/library/core/src/fmt/mod.rs:226
0x42003372
0x42003372 - log::__private_api::log_impl
    at /home/null/.cargo/registry/src/index.crates.io-6f17d22bba15001f/log-0.4.22/src/__private_api.rs:61
0x4200c037
0x4200c037 - Reset
    at /home/null/Projects/esp-hal-spi/xtensa-lx-rt/src/lib.rs:82
0x40379aa6
0x40379aa6 - ESP32Reset
    at /home/null/Projects/esp-hal-spi/esp-hal/src/soc/esp32s3/mod.rs:164

The panic looks sus - we have seen some miscompiles / misoptimizations lately.
However I tried your changed code on 36095e447e6b99115c34e591f3c29e292638b9e0 plus my changes and some more changes and it seems to work

Here are my changes (on top of the latest commit on main)
diff --git a/esp-hal/src/delay.rs b/esp-hal/src/delay.rs
index a361aabc..5fb26144 100644
--- a/esp-hal/src/delay.rs
+++ b/esp-hal/src/delay.rs
@@ -68,6 +68,7 @@ impl Delay {
     }

     /// Delay for the specified number of microseconds
+    #[procmacros::ram]
     pub fn delay_micros(&self, us: u32) {
         let delay = MicrosDurationU64::micros(us as u64);
         self.delay(delay);
diff --git a/esp-hal/src/spi/master.rs b/esp-hal/src/spi/master.rs
index 48116c03..6e3d6d01 100644
--- a/esp-hal/src/spi/master.rs
+++ b/esp-hal/src/spi/master.rs
@@ -1316,6 +1316,7 @@ mod dma {
         /// This method blocks until the transfer is finished and returns the
         /// `SpiDma` instance and the associated buffer.
         #[instability::unstable]
+        #[procmacros::ram]
         pub fn wait(mut self) -> (SpiDma<'d, Dm, T>, Buf) {
             self.spi_dma.wait_for_idle();
             let retval = unsafe {
@@ -1581,6 +1582,7 @@ mod dma {
         #[allow(clippy::type_complexity)]
         #[cfg_attr(place_spi_driver_in_ram, ram)]
         #[instability::unstable]
+        #[ram]
         pub fn half_duplex_write<TX: DmaTxBuffer>(
             mut self,
             data_mode: DataMode,
diff --git a/examples/.cargo/config.toml b/examples/.cargo/config.toml
index 600f74b2..d25c3d15 100644
--- a/examples/.cargo/config.toml
+++ b/examples/.cargo/config.toml
@@ -35,5 +35,9 @@ GATEWAY_IP = "<IP_ADDRESS>"
 HOST_IP = "<IP_ADDRESS>"
 ESP_WIFI_CSI_ENABLE = "true"

+ESP_HAL_PLACE_SPI_DRIVER_IN_RAM = "true"
+ESP_HAL_PLACE_ANON_IN_RAM = "true"
+ESP_HAL_PLACE_SWITCH_TABLES_IN_RAM = "true"
+
 [unstable]
 build-std = ["alloc", "core"]
diff --git a/examples/src/bin/spi_loopback_dma_psram.rs b/examples/src/bin/spi_loopback_dma_psram.rs
index 41b785b4..61937507 100644
--- a/examples/src/bin/spi_loopback_dma_psram.rs
+++ b/examples/src/bin/spi_loopback_dma_psram.rs
@@ -16,13 +16,16 @@
 //! If your module is quad PSRAM then you need to change the `psram` feature in the
 //! in the features line below to `quad-psram`.

-//% FEATURES: esp-hal/log esp-hal/octal-psram esp-hal/unstable
+//% FEATURES: esp-hal/log esp-hal/quad-psram esp-hal/unstable
 //% CHIPS: esp32s3

 #![no_std]
 #![no_main]
+#![feature(vec_push_within_capacity)]

 use esp_backtrace as _;
+use esp_hal::macros::ram;
+use esp_hal::peripherals::SPI2;
 use esp_hal::{
     delay::Delay,
     dma::{DmaRxBuf, DmaTxBuf, ExternalBurstConfig},
@@ -51,13 +54,48 @@ macro_rules! dma_alloc_buffer {
     }};
 }

-const DMA_BUFFER_SIZE: usize = 8192;
 const DMA_ALIGNMENT: ExternalBurstConfig = ExternalBurstConfig::Size64;
 const DMA_CHUNK_SIZE: usize = 4096 - DMA_ALIGNMENT as usize;
+const DMA_BUFFER_SIZE: usize = 8192;
+
+// #[ram]
+fn effectively_empty(fail: bool) // -> bool
+{
+    const size: usize = 2;
+    let mut x: alloc::vec::Vec<usize> = alloc::vec::Vec::with_capacity(size);
+    // unsafe { x.set_len(size) };
+
+    // let layout = core::alloc::Layout::new::<[usize; size]>();
+    // let ptr = unsafe { alloc::alloc::alloc(layout) as *mut usize };
+    // let mut lsize = 0;
+
+    if fail {
+        // for i in 0..size {
+        //     // unsafe { *(ptr.add(i)) = i }
+        //     unsafe { core::ptr::write(ptr.add(i), i) };
+        //     let j = unsafe { *(ptr.add(i)) };
+        //     lsize += j;
+        // }
+        for i in 0..size {
+            // ram op
+            // x.push(i);
+            x.push_within_capacity(i);
+            // x.get(i).unwrap_or(&0);
+        }
+    } else {
+        for i in 0..size {
+            // nop
+            unsafe { core::ptr::read_volatile(SPI2::ptr().add(0x003C) as *const u32) };
+        }
+    }
+
+    // x[size - 2] % 2 == 0
+}

+#[link_section = ".rwtext"]
 #[entry]
 fn main() -> ! {
-    esp_println::logger::init_logger(log::LevelFilter::Info);
+    esp_println::logger::init_logger(log::LevelFilter::Trace);
     info!("Starting SPI loopback test");
     let peripherals = esp_hal::init(esp_hal::Config::default());
     esp_alloc::psram_allocator!(peripherals.PSRAM, esp_hal::psram);
@@ -67,6 +105,8 @@ fn main() -> ! {
     let mosi = peripherals.GPIO48;
     let miso = unsafe { mosi.clone_unchecked() };
     let cs = peripherals.GPIO38;
+    let sio2 = peripherals.GPIO7;
+    let sio3 = peripherals.GPIO6;

     let (_, tx_descriptors) =
         esp_hal::dma_descriptors_chunk_size!(0, DMA_BUFFER_SIZE, DMA_CHUNK_SIZE);
@@ -78,66 +118,85 @@ fn main() -> ! {
         tx_descriptors.len()
     );
     let mut dma_tx_buf =
-        DmaTxBuf::new_with_config(tx_descriptors, tx_buffer, DMA_ALIGNMENT).unwrap();
-    let (rx_buffer, rx_descriptors, _, _) = esp_hal::dma_buffers!(DMA_BUFFER_SIZE, 0);
-    info!(
-        "RX: {:p} len {} ({} descripters)",
-        rx_buffer.as_ptr(),
-        rx_buffer.len(),
-        rx_descriptors.len()
+        Some(DmaTxBuf::new_with_config(tx_descriptors, tx_buffer, DMA_ALIGNMENT).unwrap());
+
+    log::info!(
+        "Alignment: {}, Chunk size: {}",
+        ExternalBurstConfig::Size64 as usize,
+        DMA_CHUNK_SIZE
     );
-    let mut dma_rx_buf = DmaRxBuf::new(rx_descriptors, rx_buffer).unwrap();
-    // Need to set miso first so that mosi can overwrite the
-    // output connection (because we are using the same pin to loop back)
+
+    let (rx_buffer, rx_descriptors, tx_buffer, tx_descriptors) =
+        esp_hal::dma_buffers!(DMA_BUFFER_SIZE);
+
+    let dma_rx_buf2 = DmaRxBuf::new(rx_descriptors, rx_buffer).unwrap();
+    let mut dma_tx_buf2 = DmaTxBuf::new(tx_descriptors, tx_buffer).unwrap();
+
     let mut spi = Spi::new(
         peripherals.SPI2,
         Config::default()
-            .with_frequency(100.kHz())
+            .with_frequency(25.MHz())
             .with_mode(Mode::Mode0),
     )
     .unwrap()
     .with_sck(sclk)
     .with_miso(miso)
     .with_mosi(mosi)
+    .with_sio2(sio2)
+    .with_sio3(sio3)
     .with_cs(cs)
     .with_dma(peripherals.DMA_CH0);

     delay.delay_millis(100); // delay to let the above messages display

-    for (i, v) in dma_tx_buf.as_mut_slice().iter_mut().enumerate() {
+    for (i, v) in dma_tx_buf
+        .as_mut()
+        .unwrap()
+        .as_mut_slice()
+        .iter_mut()
+        .enumerate()
+    {
         *v = (i % 256) as u8;
     }

+    log::info!("First dmabuffer: {:?}", dma_tx_buf.as_mut());
+
     let mut i = 0;

     loop {
-        dma_tx_buf.as_mut_slice()[0] = i;
-        *dma_tx_buf.as_mut_slice().last_mut().unwrap() = i;
-        i = i.wrapping_add(1);
-
+        use esp_hal::spi::master::{Address, Command};
+        let mut dma_buf = dma_tx_buf.take().unwrap();
         let transfer = spi
-            .transfer(dma_rx_buf.len(), dma_rx_buf, dma_tx_buf.len(), dma_tx_buf)
-            .map_err(|e| e.0)
+            .half_duplex_write(
+                esp_hal::spi::DataMode::Quad,
+                Command::None,
+                Address::Address24(0, esp_hal::spi::DataMode::Quad),
+                0u8,
+                dma_buf.len(),
+                dma_buf,
+            )
             .unwrap();
+        // delay.delay_micros(1000);
+        // delay.delay_micros(1000);
+        // delay.delay_micros(1000);
+        // let transfer = spi.write(dma_buf.len(), dma_buf).unwrap();
+        // log::info!("pre-SPI2: {}", unsafe {
+        //     core::ptr::read_volatile(SPI2::ptr().add(0x003C) as *const u32)
+        // });
+
+      //  core::hint::black_box(effectively_empty(false));
+
+     //   delay.delay_micros(1000);
+        // delay.delay_millis(1000);
+
+        // log::info!("post-SPI2: {}", unsafe {
+        //     core::ptr::read_volatile(SPI2::ptr().add(0x003C) as *const u32)
+        // });
+        (spi, dma_buf) = transfer.wait();
+        dma_tx_buf.replace(dma_buf);
+
+        log::info!("Done");

-        (spi, (dma_rx_buf, dma_tx_buf)) = transfer.wait();
-        for (i, v) in dma_tx_buf.as_mut_slice().iter_mut().enumerate() {
-            if dma_rx_buf.as_slice()[i] != *v {
-                error!(
-                    "Mismatch at index {}: expected {}, got {}",
-                    i,
-                    *v,
-                    dma_rx_buf.as_slice()[i]
-                );
-                break;
-            }
-        }
-        info!(
-            "{:0x?} .. {:0x?}",
-            &dma_rx_buf.as_slice()[..10],
-            &dma_rx_buf.as_slice().last_chunk::<10>().unwrap()
-        );
-        dma_tx_buf.as_mut_slice().reverse();
         delay.delay_millis(1000);
     }
-}
+}
\ No newline at end of file

(I copied your new example as the original one)

@bjoernQ Can confirm your example works (similar to fail=false in my case), however I'm a bit concerned on the fragility. Adding the log back will halt it again, and so the thing I'm trying to find out is if half_duplex_write().wait() always works or if it just happens to work in this exact configuration. And those experiments do point toward fairly consistent miscompiles/layout mismatches, but I'll see if I can unexpectedly break it without inserting anything in between.

Yeah it doesn't work in the project I originally found the issue (although I can't set ESP_HAL_PLACE_ANON_IN_RAM because it breaks embassy) even without anything in-between.


Add a stable odlm api version v1

What this PR does / why we need it:
Add a new apiversion v1 in CRD, current we can support 2 apiversion v1 and v1alpha1
Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #
Special notes for your reviewer:

Please confirm that if this PR changes any image versions, then that's the sole change this PR makes.

Release note:

@horis233 please check if this is fine to upgrade to new apiversion v1.

@gyliu513 Because the dev sprint is over, add v1 api will generate too many code change, maybe have some potential risks, so just add a v1 version into CRD, do you think we need add a real v1 API?

If want to migrate the existing CR instance to the new API version, it need some manual steps:
https://kubernetes.io/docs/tasks/access-kubernetes-api/custom-resources/custom-resource-definition-versioning/#upgrade-existing-objects-to-a-new-stored-version


Assemble Parallel Given Element Matries

Hi everyone! I have a question when I try to run my project with parallel mesh. When I assemble element matries (serial) with the given matries I can code:
for (int i=0; i<TotalElement; i++){
 mfem::BilinearForm::AssembleElementMatrix(i, elmat)
}

But ParBilinearForm does not have the function AssembleElementMatrix like it. I would like to ask whether there is a good way to do it with Parallel mesh.
Thank you!!!

I don't quite understand your syntax because AssembleElementMatrix is a member function of BilinearForm so you would need to do something like
ParBilinearForm a(fespace);
for (int i=0; i < TotalElement; ++i)
{
  a.AssembleElementMatrix(i, elmat);
}

Also, ParBilinearForm is inherited from BilinearForm so you can call it just them same.
All this being said, if you just want the full parallel matrix, you should us ParBilinearForm::ParallelAssemble().

I mean: for examples, I have some dense matries K_1, K_2, ... K_n; I want to assemble them. With FiniteElementSpace, I can code:
BilinearForm a(fespace);
for (int i=0; i < n; ++i)
{
  a.AssembleElementMatrix(i,K_i);
}

but with ParFiniteElementSpace, when I code like above, It's wrong because it seems to have divided into many parallel parts and each parts include some shared faces. So the result when finishing assemble is wrong. I have no idea about it. May you recommend me a good way to parrallelly assemble n matries?

Yes. Each parallel process only solves part of the mesh so the mesh has been partitioned using metis.
You shouldn't be assembling the global matrix yourself, even for the serial problem. For some problems, you'll use BilinearForm::Assemble() and others you'll use ParBilinearForm::ParallelAssemble(). You'll want to take a look at ex1 and ex1p.  For parallel, look at these lines. For serial, look at these lines.
You'll then use a parallel solver like the CGSolver to solve the problem.

Yes. Each parallel process only solves part of the mesh so the mesh has been partitioned using metis.
You shouldn't be assembling the global matrix yourself, even for the serial problem. For some problems, you'll use BilinearForm::Assemble() and others you'll use ParBilinearForm::ParallelAssemble(). You'll want to take a look at examples ex1 and ex1p. For parallel, look at these lines. For serial, look at these lines.
You'll then use a parallel solver like the CGSolver to solve the problem.

Thank you for your supporting. I have read the examples you mentioned. I have a question: How can I set another test function. From the examples, the used test function is a basic function. But I want to change it.

Thanks for your support


ItemizedOverlayWithFocus is not scaled for screen resolution

As noted in PR #209 the description of the itemized overlay is very small on a large screen.
This is presumably due to all the hard-coded calculations in ItemizedOverlayWithFocus.  These calculations should account for screen resolution, or use a layout file.

@neilboyd i pushed a change set that fixes this. Although it's fixed, i'm not exactly sure what "right" looks like here. This also required an API change for ItemizedOverlayWithFocus and everything else that uses it. I've made the font sizes and whatnot adjustable by classes that extend this but this could potential be simplified or done in a better way. I'll defer to you on that one

It would be better to add the two properties that are needed as properties on ResourceProxy instead of passing Context through a load of methods.  ResourceProxy already has one property getDisplayMetricsDensity().

i added a new method to the default resource proxy and the interface. it'll be a potentially breaking change, but the impact should be minimal

You added getDisplayMetrics() so you could remove getDisplayMetricsDensity().

I thought about it, but maybe we could leave it and flag as deprecated. No
idea who else may be relying on it
On Tue, Jan 5, 2016 at 4:34 PM, Neil Boyd<EMAIL_ADDRESS>wrote:

You added getDisplayMetrics() so you could remove
getDisplayMetricsDensity().
—
Reply to this email directly or view it on GitHub
https://github.com/osmdroid/osmdroid/issues/214#issuecomment-169140497.

Deprecated is good.

@neilboyd is this good to merge then?

I guess so


card-image + material box

I would like to use the card-image and material box combined.

After clicking the image this is the result:

Is it "fixable", or is it not possible to use both of them together?

See #825


Fixes issue #105 - typo in README.md line 59

added single quotes around cos(x)

@devinbmiller thanks for working on this :)


Spec compliance for null linkage for to-one relationships.

I think I'm reading the spec correctly here, it says:
Resource linkage MUST be represented as one of the following:
- null for empty to-one relationships.

http://jsonapi.org/format/#document-structure-resource-relationships
But right now these relationships return empty hashes.

Yes, I read the spec the same way. Thanks for catching and fixing this.


Implement substring search on components palette

This PR replaces the Trie-based approach to searching component names to do arbitrary substring matching over the set of components. It also fixes an issue where extensions were not included in the search.
Resolves #1750
Change-Id: I64feb9023bf8fdc9dfa9213ab5112ce96b46fa5c

Note that because of a weird interplay between SimpleComponentDescriptor and SimplePaletteItem I was unable to reuse descriptors across palette items. This means that we keep around more things in memory by having multiple invisible mock components (on SimpleComponentDescriptor) that are needed for drawing the search's icons. Trying to reuse the SCD objects results in the icons of visible components disappearing from the palette.

From testing: If I click the "x" in the search field to clear it, the search results don't clear as expected.

Also from testing: Extensions don't seem to be included in the search results in most cases.


Can not detect loop closure sometimes!

My experiment on SuMa++ and KIITI always cannot detect loop closure at some place, i have tried many loop closure parameters, but i think my parameters have problems, could you tell me how to change the parameters to make it work??

Hey @lymdove, thanks for using our code. Yes, we use a geometric-based loop closing method in SuMa++. You may try to use some other methods to close loops. For example, our overlapnet, a deep learning-based approach.

Thanks, I'll have a try. @Chen-Xieyuanli


Optimize requests on the vendor product page

Instead of pulling each row of the different tables one by one, pull them all at once and only pull one by one when individual tests are uploaded.
Results before when creating a new product and staying on the page for the duration of the processing:

Results after:

Tests for this may be failing at the moment, I just rebased against master however it seems master may be failing as well.

Does this still allow the table to be updated as individual measure tests are being built?

@thecristen Yes, the UI still updates nearly the same, a lot of tasks are just done in bulk and a little less often.


added line in supported devices

Added a line saying 'Don't see your device here? Contact<EMAIL_ADDRESS>to discuss our custom hardware support packages.'

Accept and roll into another PR


feat: Avoid leaking the registry

Hi!
I'm working for Wasmer (https://github.com/wasmerio/wasmer), and we are using rayon. Thanks for the awesome project!
We have a memory leak with rayon. Valgrind gives us the following trace:
==1319== 8 bytes in 1 blocks are definitely lost in loss record 1 of 58
==1319==    at 0x102025635: malloc (in /usr/local/Cellar/valgrind/HEAD-6049595/lib/valgrind/vgpreload_memcheck-amd64-darwin.so)
==1319==    by 0x1001D35D6: alloc::alloc::alloc (alloc.rs:74)
==1319==    by 0x1001D3689: alloc::alloc::Global::alloc_impl (alloc.rs:153)
==1319==    by 0x1001D43FA: <alloc::alloc::Global as core::alloc::AllocRef>::alloc (alloc.rs:203)
==1319==    by 0x1001D3535: alloc::alloc::exchange_malloc (alloc.rs:281)
==1319==    by 0x1001D4B93: rayon_core::util::leak (boxed.rs:175)
==1319==    by 0x1001C2F90: rayon_core::registry::set_global_registry::{{closure}}::{{closure}} (registry.rs:196)
==1319==    by 0x1001EFB78: core::result::Result<T,E>::map (result.rs:508)
==1319==    by 0x1001C2FE8: rayon_core::registry::set_global_registry::{{closure}} (registry.rs:195)
==1319==    by 0x1001DAC6B: std::sync::once::Once::call_once::{{closure}} (once.rs:265)
==1319==    by 0x100BEFDB0: std::sync::once::Once::call_inner (once.rs:421)
==1319==    by 0x1001DABF7: std::sync::once::Once::call_once (once.rs:265)

While trying to fix the leak, I noticed a code that could be optimized, hence #814.
But shortly after, I then came with this patch that actually fixes the memory leak. An immediate consequence is that util.rs is no longer required and #814 is useless. Feel free to close #814 if you consider this patch is a good candidate to land in the main branch of this repository.
How to reproduce the memory leak
Follow the README.md at https://github.com/Hywan/rayon-fix-registry-leak. It contains a usecase you can execute freely.

(meta: I think your description meant to reference #814).

(meta: I think your description meant to reference #814).

Correct. Sorry for the mistake. Fixed.

Tests are failing with:
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 11, kind: WouldBlock, message: "Resource temporarily unavailable" }', library/test/src/lib.rs:473:32

I don't see the link with OnceCell. I'm investigating.

I've rebased my attempts into a single commit. I've adopted the same approach than my first commit, however we don't clone the Arc. We return a &'static Arc<…> by using Option::as_ref().

I think the resource problem in CI was from unconditionally calling registry, creating a whole bunch of threads even if THE_REGISTRY is already initialized.

Yes, correct, but I assumed this behavior could happen in the real life. I prefered to find an alternative.

I would expect using get_or_try_init instead.

get_or_try_init does not generate an error when the cell is already initialized. It's required in this case.

get_or_try_init does not generate an error when the cell is already initialized. It's required in this case.

No, but you can set a local boolean whether the closure was actually called.
I'm fine with the new approach that does it without once_cell though.

I discussed this with @nikomatsakis because I wanted to be sure there wasn't some historical reason why this needed to be leaked, but we couldn't come up with anything. It's been a leaked reference ever since it was introduced in here, but there was no explanation why it must be that way. So let's go ahead and change that!
bors r+

Thanks :-). It's better to double-check, that's my first patch to rayon!


g_ip_id_counter overflow

English Only (except for bug reporting).
文件:   network.cpp
函数:   int send_raw_ip(raw_info_t &raw_info,const char * payload,int payloadlen)
unsigned short g_ip_id_counter=0;
iph->id = htons (g_ip_id_counter++); //Id of this packet

g_ip_id_counter
是 uint16 类型，65535次执行后，会溢出吧

no problem, it's as expected. when it overflows it just wrap back to zero.

no problem, it's as expected. when it overflows it just wrap back to zero.


FLOPs?

Using the default imagenet 128 x 128 model and model.summary(), the number of parameters is 32M. There is no model with 67M parameters as mentioned in the paper.


Hashes at the end of URLs?

Why do links from the index now have hashes at the end? ie. https://lectures.quantecon.org/py/python_essentials.html#

this is an issue with sphinxcontrib-jupyter. I will work on a fix when generating links to remove # when there are none.

this PR will fix this issue: https://github.com/QuantEcon/sphinxcontrib-jupyter/pull/162


Multiple virtual text after are displayed below each other and not in one line as they're supposed to

Steps to reproduce
Run vim --clean test.vim -S test.vim with
vim9script

colorscheme habamax
prop_type_add('test', {highlight: 'Error'})

const opts: dict<any> = {
    type: 'test',
    text: range(50)->join(' '),
    text_align: 'after',
    text_padding_left: 2
}

prop_add(1, 0, opts)
prop_add(1, 0, opts)
prop_add(1, 0, opts)

Result:

Expected behaviour
Since all three text-properties have text_align set to after, they should be displayed in one line.
Version of Vim
9.0.718
Environment
Linux
Logs and stack traces
No response

The props are displayed in one line, but the line wraps.  They are each truncated to fit in the current screenline.
An alternative would perhaps be to behave like "nowrap", but then some props may not show at all, and there is no way to see them.
Another alternative would be to truncate each shorter, so that they fit together in the one screen line.  However, they might get so short that the text is not useful.  Especially if the text line only leaves few columns for the props.

Thanks for clarifying this. So the text_wrap attribute applies to the individual virtual text whereas wrap applies to consecutive virtual texts in the same line. Maybe this could be mentioned somewhere in the documentation.
I don't have an issue with the current behavior. But there seems to be a redrawing bug when list is enabled and wrap disabled.
Run the following code:
vim9script

colorscheme habamax
set list nowrap
prop_type_delete('test')
prop_type_add('test', {highlight: 'Error'})

const Opts: func = (str: string, length: number): dict<any> => ({
    type: 'test',
    text: $'{repeat(str, length)}',
    text_align: 'after',
    text_padding_left: 2
})

prop_add(1, 0, Opts('AAA', 20))
prop_add(1, 0, Opts('BBB', 10))
prop_add(1, 0, Opts('CCC', 20))

Append some text to the first line of the buffer.
Result: right before the 2nd virtual text disappears, the text-properties are wrapped even though nowrap is set (see first screenshot below).

Result: the screen is not redrawn after pressing u. This becomes more apparent when you keep pressing u and <C-R> a couple of times. Pressing <C-L> will redraw the screen again.

Screenshot:

Here's a video that demonstrates the above behavior:
https://user-images.githubusercontent.com/6266600/195165973-e3ec0c5b-2324-4a5b-8a03-b97d010948d6.mp4

Not directly related to the issue but wouldn't it be more useful to have a global virtwrap option (or similar) that changes the wrapping of all virtual text simultaneously, similar to wrap? Currently we need to delete all virtual text and then reapply them just for the sake of toggling text_wrap. That seems quite cumbersome. And every plugin author will have to go through the same steps to support toggling of virtual-text wrapping.

Fixed in https://github.com/vim/vim/commit/7e017461e21ac534f8a0d59c1165d11358c69d87.