Alloy Smelter doesn't accept oredict variants of Graphite Dust

The alloy smelter machine can only accept the NuclearCraft item graphite dust to make steel.
The problem is in clients with many mods, using the NuclearCraft method of obtaining graphite dust can end up yielding a non-NuclearCraft oredict variant of dustGraphite, such as Graphite dust from Extreme Reactors.
I am simply requesting that you add the functionality of accepting any oredict variant of graphite dust. If this is a problem with other items, please make that obey the ore dictionary too.

Another example I forgot to mention is Yellorium from Extreme Reactors. Functionally it is identical to uranium according to the oredict.

The alloy furnace should not be accepting graphite dust of any kind - it should only take in graphite ingots - if it is accepting the dust, then that is a bug.
I can't think of any mods that add the dust and not the ingot off of the top of my head, but in case there are any, I'll added a graphite dust -> graphite ingot manufactory recipe :)
As for the yellorium issue - that's been there for a while, even back in 1.7.10. The problem arises because Big/Extreme Reactors adds yellorium to both the 'ingotUranium' and 'ingotYellorium'.

Recipe added for 2.3


when dsn use https will throw SentryError: HTTP Error (400): No JSON data was found

{ SentryError: HTTP Error (400): No JSON data was found
    at new SentryError (/Volumes/west/rishiqing/rishiqing-electron/node_modules/@sentry/core/src/error.ts:7:5)
    at ClientRequest.req.on (/Volumes/west/rishiqing/rishiqing-electron/node_modules/@sentry/electron/src/main/transports/net.ts:39:12)
    at emitOne (events.js:96:13)
    at ClientRequest.emit (events.js:191:7)
    at URLRequest.ClientRequest.urlRequest.on (/usr/local/lib/node_modules/electron/dist/Electron.app/Contents/Resources/electron.asar/browser/api/net.js:207:12)
    at emitOne (events.js:96:13)
    at URLRequest.emit (events.js:191:7) name: 'SentryError' }

I think that's a response from the sentry server. If you're hosting your own server it might need updating.

@HazAT This is a bug in the base transport, we should use the DSN's protocol and pass it to the protocol option for transports.
@qinyang912 As a workaround, you can enable HTTP for the /api/x/store/ endpoint.


FromPyObject for HashSet and BtreeSet

We already have FromPyObject for HashMap and  BtreeMap, so it should be straightforward to implement these for HashSet and BtreeSet.

This has been done in https://github.com/PyO3/pyo3/pull/842


PopoverService: Attemp to fix SemaphoreFullException

Description
Attempt to fix https://github.com/MudBlazor/MudBlazor/pull/8210#issuecomment-1985671425

Made InitializeServiceIfNeededAsync lock free
Added _disposed

We do not accept new subscribers after dispose.
We do not let to create a new popovers after dispose.
We do not let to update an existing popover after dispose.
We ignore if Dispose is called multiple times.

Added CancellationToken

We pass it to the AsyncKeyedLock.
We pass it to the JSRuntime - I had to extend the IIJSRuntimeExtentions.
We pass it to the BatchPeriodicQueue.
We pass it now to the IPopoverObserver.PopoverCollectionUpdatedNotificationAsync - this can be handy to abort for loops.

We do not initialize service anymore on GetProviderCountAsync call. I had idea to initialize it at any opportunity, but don't think it's worth.
Removed tickOnDispose in BatchPeriodicQueue, it doesn't have any use in the code anymore and it implements now IDisposable instead.

Open topics:

Should we add _disposed for the DestroyPopoverAsync? Technically this method doesn't use any locking, only the batch does but the list is empty anyway so do we care about early exit?
Should I use Lazy for InitializeServiceIfNeededAsync instead?

How Has This Been Tested?
Unit tests
Types of changes

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist:

[x] The PR is submitted to the correct branch (dev).
[x] My code follows the code style of this project.
[x] I've added relevant tests.

@haas-daniel @Yomodo asking in case if anyone wants to play with this PR and put it in their app.
I sadly can't effectively reproduce it, I do not use BSS (only WASM) for any of my prod applications to use it as "lab rat". This is all kinda "theory" code.
If not then I merge it after I will think about it for a day or two in case I came up with any other ideas.

I'll play with this later tonight.

I'll play with this later tonight.

Unfortunately I can't proceed at the moment due to security restrictions in our dev environment,
that only allows to run executables from a specific folder.

C:\Users\ikke\.nuget\packages\aspnetcore.sasscompiler\<IP_ADDRESS>\build\AspNetCore.SassCompiler.targets(11,5): error MSB4018: The "CompileSass" task failed unexpectedly.

C:\Users\ikke\.nuget\packages\aspnetcore.sasscompiler\<IP_ADDRESS>\build\AspNetCore.SassCompiler.targets(11,5): error MSB4018: System.ComponentModel.Win32Exception (0x80004005): This program is blocked by group policy. For more information, contact your system administrator

Unfortunately I'm unable to help out as I can't compile the MudBlazor solution anymore.
Culprit is the sascompile nuget package that contains a script which starts an executable: dart.exe
In our DEV environment policies are in effect that will block any executable unless it is
started from a certain exclusion folder.
@echo off
REM This script drives the standalone dart-sass package, which bundles together a
REM Dart executable and a snapshot of dart-sass.

set SCRIPTPATH=%~dp0
set arguments=%*
"%SCRIPTPATH%\src\dart.exe" "%SCRIPTPATH%\src\sass.snapshot" %arguments%

C:\Users\admin.nuget\packages\aspnetcore.sasscompiler\<IP_ADDRESS>\runtimes\win-x64\sass.bat

https://www.myget.org/F/mudblazor/api/v3/index.json
PM> Install-Package MudBlazor -Version 6.18.0 -Source https://www.myget.org/F/mudblazor/api/v3/index.json

I just installed your package and deployed our code.
Will keep you updated, thanks!

I've used 3 different environments to test around.
Almost immediately after pushing, two exception are thrown but only in 1 environment.

Are there more detailed stack-traces?

I added 6.18.1 to the feed.

Are there more detailed stack-traces?

Unfortunately this is as detailed at is gets:

Up until now these are the only 2 exceptions. No exceptions to be seen in the other environments.
I'll try the 6.18.1 version.

Unfortunately this is as detailed at is gets, no stack traces etc.

Ye, sadly it's not really helpful as it doesn't show the caller of the PopoverService.

I have the same problem, like @Yomodo sample.

Is this with with 6.17.0 or you installed the dev version 6.18.1 from my myget?

I have the same problem, like @Yomodo sample.

Is this with with 6.17.0 or you installed the dev version 6.18.1 from my myget?

I use 6.17.0

I use 6.17.0

Thank you, we are already aware of this issue in the 6.17.0 and this PR is an attempt to fix it.

Yes: Up until now, NO more exceptions. All is well.

Thanks, i will merge it since it's better than nothing, because our members want to release new version soon.
But if you get exceptions, let me know.

@haas-daniel, @Yomodo @onatsko is the exception totally gone?
I'm wondering if I should drop now the legacy popover for v7.


Creating a loop & move mkdir

Less code, less bugs

The commit logs don't mention dropping the chown to ceph. Why drop it?

@ktdreyer because the last line of create_mandatory_directories() does it. That's implicit. Should I make it explicit in the commit message ?

Whoops, I should've noticed that. Thanks for pointing it out to me!


Howto -> linuxpro.com.br

Install Wide on Linux 32/64 Bits
http://linuxpro.com.br/desenv/web-editor-golang

Hi, you can download Wide from here, or build it yourself follows this steps.


Don't have the Rust runtime as the default for bend run

Is your feature request related to a problem? Please describe.
The Rust runtime is the reference implementation, but it's slow and doesn't have IO.
It's confusing for new users.
Describe the solution you'd like
Change bend run command into bend run-rs.
Describe alternatives you've considered

Also making run default to run-c.

@developedby I'd like to work on this issue. Is there anything I should know before working on this?

You need to update the cli tests to the new command. in tests/golden_tests/cli/*
To run the tests you need to install cargo-insta cargo install cargo-insta, run them with cargo insta test and then update the snapshots with cargo insta review if anything changed.

fixed #630


Update zio-prelude to 1.0.0-RC29

About this PR
📦 Updates dev.zio:zio-prelude from 1.0.0-RC26 to 1.0.0-RC29
📜 GitHub Release Notes - Version Diff
Usage
✅ Please merge!
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

⚙ Adjust future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "dev.zio", artifactId = "zio-prelude" } ]

Or, add this to slow down future updates of this dependency:
dependencyOverrides = [{
  pullRequests = { frequency = "30 days" },
  dependency = { groupId = "dev.zio", artifactId = "zio-prelude" }
}]

labels: library-update, early-semver-pre-release, semver-spec-pre-release, commit-count:1

Superseded by #1329.


upgrade netty and bcprov

Signed-off-by: Jan N. Klug<EMAIL_ADDRESS>

If you rebase this, does it pass travis now (using openjdk8)?

Unfortunately there is another problem which I'm currently investigating. If I use bcprov>1.51 (which should be done due to security issues with versions <1.60) I get a 'bad record mac' error in the final pairing phase. With 1.51 it works.

Will re-open when problems are solved.


Show Loader when adding Workspace Edit

Details

Add loading flags for Avatar upload and form submission
IMP: Added isUploading flag to AvatarWithImagePicker with uploadIndicator
Replaced js loop to Animated.loop

Fixed Issues

$ https://github.com/Expensify/App/issues/4487
Tests

Tested by keeping the form contents the same
Updating the workspace name and image
Removing the workspace image
Submitting the form successfully
Uploading a large image

QA Steps

Go to Workspace and click on the Avatar
Check button state when the values are not updated
Change the content/upload the image and see if values are updated
Test Removing the workspace image as well
Submitting the form successfully
Uploading a large image

Tested On

[x] Web
[x] Mobile Web
[x] Desktop
[x] iOS
[x] Android

Screenshots

Web
v1
https://user-images.githubusercontent.com/3069065/131385161-e70fc702-9ef7-4515-bacd-86ac728d5c03.mp4
Updated
https://user-images.githubusercontent.com/3069065/132609914-524caa8b-ead7-4ec1-97c4-6288d6575dd7.mp4
Mobile Web
v1
https://user-images.githubusercontent.com/3069065/131384341-a5d4da09-e5e7-4807-8cf9-3724c797d2c6.mp4
Updated
https://user-images.githubusercontent.com/3069065/132610024-500c89e5-83d0-4c05-91c6-61688fd223c1.mp4
Desktop
v1
https://user-images.githubusercontent.com/3069065/131384466-c488f178-844d-471b-b9b6-9ca68fa5de65.mp4
Updated
https://user-images.githubusercontent.com/3069065/132610083-e81b6468-c976-4ec2-83cc-f7932de947f3.mp4
iOS
v1
https://user-images.githubusercontent.com/3069065/131384510-c38701d8-74ae-49c2-a116-e50e2fdc19a5.mp4
Updated
https://user-images.githubusercontent.com/3069065/132610283-1a851e62-640f-496d-86d2-83376a6e7125.mp4
Android
v1
https://user-images.githubusercontent.com/3069065/131384679-c6b796b0-f49e-4e76-8d48-3991d2cac241.mp4
Updated
https://user-images.githubusercontent.com/3069065/132610438-29e4b86c-81aa-45fc-ad41-7d77e0ba6c92.mp4

@roryabraham I've raised a draft PR. Need to clarify a couple of things:

Should we throw error messages on any form error or avatar upload? or even a success message?
I've noticed when we upload an Avatar image, there isn't any feedback shown to the user. For now, what I've done is added an isUploading flag to AvatarWithImagePicker which shows the Sync icon instead of Pencil when set true. While it did solve the purpose, it doesn't look great. I would recommend an animation similar to AvatartWithIndicator. Shall I go ahead and add it?
If we agree to do the loader thing in #2, a similar UX is missing for the Profile image update as well. Shall I cover that too in this PR if is a 🟢 for the above approach.

Should we throw error messages on any form error or avatar upload? or even a success message

If there's a form error or API error, we should display a user-friendly error message. A success message would be good too, maybe using Growl or some other visual indication.

I've noticed when we upload an Avatar image, there isn't any feedback shown to the user. For now, what I've done is added an isUploading flag to AvatarWithImagePicker which shows the Sync icon instead of Pencil when set true. While it did solve the purpose, it doesn't look great. I would recommend an animation similar to AvatartWithIndicator. Shall I go ahead and add it?

I'm not sure. I'll defer to @Expensify/design here

If we agree to do the loader thing in Fix spaces #2, a similar UX is missing for the Profile image update as well. Shall I cover that too in this PR if is a 🟢 for the above approach?

I assume you meant item 2 of your list, not the PR linked in the comment. I would think yes 🟢, but again will defer to @Expensify/design here.

Lastly, I'll warn you that what I see in this PR so far violates the Expensify/App Philosophy, specifically UI Binds to Disk. My interpretation of that philosophy is that actions should never really be returning a promise – instead the UI binds to data in Onyx, and calls actions that manage the data. It should never be explicitly waiting for a response.

Thanks for the clarification @roryabraham.

If there's a form error or API error, we should display a user-friendly error message. A success message would be good too, maybe using Growl or some other visual indication.

I'll put in a message.

I'm not sure. I'll defer to @Expensify/design here

I'll wait for the comments from the design team.

I assume you meant item 2 of your list, not the PR linked in the comment.

Yes. I meant point# 2 from my list. It seems GH linked to the PR due to #.

Lastly, I'll warn you that what I see in this PR so far violates the Expensify/App Philosophy, specifically UI Binds to Disk.

Thanks for highlighting.  I only went ahead with the approach because the same file had the following line:
this.uploadAvatarPromise = uploadAvatar(image).then(url => new Promise((resolve) => {

So should I store these loading props to Onyx?

So should I store these loading props to Onyx?

For now, I believe this is the approach we'll want to take.

@roryabraham I've updated the PR with the following changes:

Growl messages in catch block
Moving all flags to Onyx
Uploading Indicator in the AvatarImagePicker

I haven't added Growl on success because immediately after the policy is saved, we redirect to workspace card with updated info. Let me know if you still feel we should add a Growl message for success.

@mananjadhav No growl on success sounds fine to me 👍

@stitesExpensify @roryabraham I've updated the PR. Additionally, I've also modified the size of the indicator to get it closed to the Avatar and adjusted the icon size.
https://user-images.githubusercontent.com/3069065/131907746-2cb1af94-8dd4-45c3-a338-ddc49f27863e.mov

PR updated

I’ll update the AvatarWithIndicator too.
Yeah I’ve worked on Reanimated v1. But I’ve heard Reanimated V2 last year reduces a lot of boilerplate code too.

PR updated

Okay, I'll check the Animation updates and implement them.

Lastly, I'm looking at the screenshots provided in the PR description, and it doesn't seem like the indicator is getting bigger > when the animation starts. Instead, it actually looks like it's getting smaller and is outside the avatar circle.

I had updated this comment but not the PR description. Let me update it in the PR description.
https://github.com/Expensify/App/pull/4801#issuecomment-912008496

@roryabraham There's a comment in AvatarWithIndicator.startRotation,
We need to manually loop the animations as `useNativeDriver` does not work well with `Animated.loop`.

@roryabraham I am updating the SpinnerAnimation and having the following issue with AvatarWithIndicator, which I can't reproduce in staging.new.expensify.com
If I add a couple of messages in Offline mode, and then disable throttling, the Avatar scale doesn't change back to 1. (stays 1.66 and hence it looks bigger), neither does the rotation stop. I've pushed my change and attaching the video of the issue.
https://user-images.githubusercontent.com/3069065/132413898-d7d3abb1-168b-4c01-af82-75e828a6f056.mov
I am guessing it is because after this.animation.stop() the component isn't rendered and getIndicatorStyles don't have the updated values of this.scale and this.rotate.  I also feel Animated.parallel use must be causing it. I am a little confused that I am unable to reproduce the same in the staging URL where we have the old Animation in place.

@roryabraham Did you get a chance to look at my comments above?
Meanwhile I'll resolve the conflicts.

@mananjadhav Regarding this comment:

We need to manually loop the animations as useNativeDriver does not work well with Animated.loop.

is there some observable problem that occurs when you use Animated.loop as I suggested above? I looked back at the PR which added that comment and there wasn't any additional context explaining the problem. AFAICT there's nothing that seems to go wrong when I switch to using Animated.loop locally, and it makes the code clearer and more performant. So I'd prefer to do that unless we can somehow expand on how "useNativeDriver does not work well with Animated.loop". Hopefully that makes sense!
We can skip the use of Animated.parallel though since you've found an issue with my suggested implementation.

Well, I didn't find any issue on the emulator but I haven't been able to check on a real Android device. Let me see if I can test and accordingly update the PR.

@roryabraham PR updated. I checked on a real Android device and it worked fine. So I am guessing it shouldn't be a problem.
I've also updated the videos with the latest testing of indicator closer to the image with increased size.

:hand: This PR was not deployed to staging yet because QA is ongoing. It will be automatically deployed to staging after the next production release.

🚀 Deployed to staging by @roryabraham in version: 1.0.95-2 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

This has been deployed to production and is now subject to a 7-day regression period.
If no regressions arise, payment will be issued on 2021-09-17.  :confetti_ball:

🚀 Deployed to production by @roryabraham in version: 1.0.96-0 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅


New method to do lane line detection

What changes do you make in this PR?
Our previous method decreases the difficulty of getting the distance between the vehicle and lane lines.
Now the distance to the most left continuous line  and the most right continuous line will be calculated by cay casting

to do so, the vehicle will get more freedom in Intersection and Split/Merge

Besides, this method can also be applied to get the distance to the left and right lane lines of the current lane.

Checklist

[ ] I have merged the latest main branch into current branch.
[ ] I have run bash scripts/format.sh before merging.
Please use "squash and merge" mode.

SO, how we determine the maximum lateral distance?

Add  config "use_lane_line_detector" to vehicle config. Enabling it can change the way to detect lane line


Support querying against data portal annotation metadata to select picks/segmentation

Overrides root.get_picks and root.get_segmentations in order to allow querying against any scalar field of cryoet_data_portal.Annotation and the list of author namees.
The signature of root.get_picks changes to:
    def get_picks(
        self,
        object_name: str = None,
        user_id: str = None,
        session_id: str = None,
        portal_meta_query: Dict[str, Any] = None,
        portal_author_query: List[str] = None,
    ) -> List["CopickPicksCDP"]:
        """Get picks by name, user_id or session_id (or combinations). Portal metadata are compared for equality. Portal
        authors are checked for inclusion in the full author list.

        Args:
            object_name: Name of the object to search for.
            user_id: User ID to search for.
            session_id: Session ID to search for.
            portal_meta_query: Dictionary of values to compare against portal metadata of this annotation. Allowed keys
                are the scalar fields of [cryoet_data_portal.Annotation](https://chanzuckerberg.github.io/cryoet-data-portal/python-api.html#annotation)
            portal_author_query: List of author names. Segmentations are included if this author is in the portal
                annotation's author list.

        Returns:
            List[CopickPicks]: List of picks that match the search criteria.
        """

The signature of root.get_segmentations changes to:
    def get_segmentations(
        self,
        user_id: str = None,
        session_id: str = None,
        is_multilabel: bool = None,
        name: str = None,
        voxel_size: float = None,
        portal_meta_query: Dict[str, Any] = None,
        portal_author_query: List[str] = None,
    ) -> List["CopickSegmentationCDP"]:
        """Get segmentations by user_id, session_id, name, type or voxel_size (or combinations) and portal metadata and
        authors. Portal metadata are compared for equality. Portal authors are checked for inclusion in the full author
        list.

        Args:
            user_id: User ID to search for.
            session_id: Session ID to search for.
            is_multilabel: Whether the segmentation is multilabel or not.
            name: Name of the segmentation to search for.
            voxel_size: Voxel size to search for.
            portal_meta_query: Dictionary of values to compare against portal metadata of this annotation. Allowed keys
                are the scalar fields of [cryoet_data_portal.Annotation](https://chanzuckerberg.github.io/cryoet-data-portal/python-api.html#annotation)
            portal_author_query: List of author names. Segmentations are included if this author is in the portal
                annotation's author list.

        Returns:
            List[CopickSegmentation]: List of segmentations that match the search criteria.
        """

@jtschwar This might be interesting.


[Game]: Inventory cannot cycle through items.

Inventory ring cycling does not work despite the input code being implemented.

Solved.


[HUDI-1156] Remove unused dependencies from HoodieDeltaStreamerWrapper Class

Tips

Remove unused dependencies from HoodieDeltaStreamerWrapper Class

What is the purpose of the pull request
(For example: This pull request adds quick-start document.)
Brief change log
(for example:)

Modify AnnotationLocation checkstyle rule in checkstyle.xml

Verify this pull request
(Please pick either of the following options)
This pull request is a trivial rework / code cleanup without any test coverage.
(or)
This pull request is already covered by existing tests, such as (please describe tests).
(or)
This change added tests and can be verified as follows:
(example:)

Added integration tests for end-to-end.
Added HoodieClientWriteTest to verify the change.
Manually verified the change by running a job locally.

Committer checklist

[ ] Has a corresponding JIRA in PR title & commit

[ ] Commit message is descriptive of the change

[ ] CI is green

[ ] Necessary doc changes done or have another open PR

[ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.

Please fix the build.


Connection using complex custom JDBC URL throws I/O error

System information:

Operating system (distribution) and version: MacOS and Windows 10, both Up-to-date and earlier
DBeaver version: 7.0.2
Additional extensions: None

Connection specification:

Database name and version: Oracle 11g - <IP_ADDRESS>.0
Driver name: ojdbc8 <IP_ADDRESS>
Do you use tunnels or proxies (SSH, SOCKS, etc)? No

Describe the problem you're observing:
Using a complex connection URL results in IO Error.
Used URL in Connection Settings > Custom:
jdbc:oracle:thin:@(DESCRIPTION=(FAILOVER=on)(LOAD_BALANCE=OFF)(CONNECT_TIMEOUT=5)(RETRY_COUNT=3)(RETRY_DELAY=1)(TRANSPORT_CONNECT_TIMEOUT=3)(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=XXXX)(PORT=YYYY))(ADDRESS=(PROTOCOL=TCP)(HOST=XXXX2)(PORT=YYYY2)))(CONNECT_DATA=(SERVICE_NAME=DUMMY)))
Include any warning/errors/backtraces from the logs
I/O-Error: The Network Adapter could not establish the connection
The Network Adapter could not establish the connection
The Network Adapter could not establish the connection
Socket read timed out, socket connect lapse 5 ms. XXXX YYYY 3 2 true
Socket read timed out, socket connect lapse 5 ms. XXXX YYYY 3 2 true
Socket read timed out
Socket read timed out

thanks for the bug report

I think the problem is in URL itself.
At least I can connect to our test server with custom URL.
Parameters like RETRY_COUNT, RETRY_DELAY and TRANSPORT_CONNECT_TIMEOUT may cause IO errors, try to remove them.
Also you could try to leave just one target host name.

I think the problem is in URL itself.
At least I can connect to our test server with custom URL.
Parameters like RETRY_COUNT, RETRY_DELAY and TRANSPORT_CONNECT_TIMEOUT may cause IO errors, try to remove them.
Also you could try to leave just one target host name.

Removing RETRY_COUNT, RETRY_DELAY and TRANSPORT_CONNECT_TIMEOUT worked. Thx!

But it was woking fine for 7.0.0 with RETRY_COUNT, RETRY_DELAY and TRANSPORT_CONNECT_TIMEOUT

JDBC8 changed the TRANSPORT_CONNECT_TIMEOUT to milliseconds. JDBC 7 this value is in seconds
Change your connection string to TRANSPORT_CONNECT_TIMEOUT=5000 and you are grand.


Downscale images in comment markup to 50%

Feature Description
Gitea renders markup images it at 100% scale, often resulting in overly large images for screenshots from HiDPI screens. I suggest we apply a reduction to 50%, which also matches the current GitHub rendering.
Screenshots
GitHub - 324px image width:

Gitea - 648px image width:

Looks like this is more nuanced: GitHub only applies the downscaling to comment markup, not markup documents which still render at 100%.

Interestingly enough, they do not downscale all screenshots, here's one taken on Windows:
https://github.com/silverwind/symlink-test/issues/13
https://try.gitea.io/silverwind/symlink-test/issues/12
They definitely downscale Mac screenshots, so I think there must be some metadata attached to the screenshot upload that contains the dppx information.

I figured out what's happening. It's conditional downscaling of uploaded images based on PNG  pHYs metadata present at least in MacOS screenshots. This issue is not accurate anymore so I will close and follow up with an actual fix later.


Infrastructure for publishing status events from transports

The plan is to build upon the intrastructure added in #979.

Depends on #979, otherwise ready for review.

Ready for review now.

:+1:


Using validate method on RearrangementReader

I am using the RearrangementReader class as follows:
   airr_data = airr.read_rearrangement(file_handle)
   if airr_data.validate():
            print("File", path, "is a valid AIRR TSV file")

When I do this with a TSV file generated with igblast's AIRR TSV generation, I get a set of messages that says:
Warning: productive is not boolean for row # 1.
Warning: productive is not boolean for row # 1.
Warning: productive is not boolean for row # 1.
Warning: productive is not boolean for row # 1.
Warning: productive is not boolean for row # 1.
Warning: productive is not boolean for row # 1.
Indeed, the TSV file has no data for the productive field, so the warning is valid... The data is not boolean, it is empty. This implies I think that igblast is generating invalid AIRR TSV for this field, but that is not my main concern here...
I have two questions:

It seems like the row number in this message is incorrect, as I suspect there is the same error for each row in the file...
I would like to turn off the warnings in this case, as really all I care about is that the data is valid and I can live without the warnings. It seems like the default for the class is to be quiet, but I can't seem to stop this message from being printed and I don't see where in the code the error message is being generated...

Thoughts?
Brian

Yes, I think row_num = 1 inside def validate(self), in io.py, doesn't get increased.
I am  going to check what the R package does in the same situation.

I used the pip install method, and it appears that I am an early adopter. My AIRR python version is 1.1.0 8-)
Let me upgrade and check if that fixes the problem...

I was just about the mention that. :)

airr.read_rearrangement now takes a filename (string) instead of a file handle.
io.RearrangementReader doesn't have a validate method anymore in favor of the validate=True argument to __init__

We made a bunch of other changes to validation to address #142.  I'd check through the end of that issue for details on how the behavior differs.

Changed code to 1.2.1 - now working after updating my code to use new version...


new theme for search button

This PR can be reviewed when we're ready to make the next release.
This PR updates the theme for the Search button:

Uses Assembly btn classes for consistency
Makes button a little more prominent, without drawing too much attention
Uses accessible colors (current contrast ratio is too low)

Default

Status
Image

Before

After

Dark

Status
Image

Before

After

@colleenmcginnis if we have time to review, this would be good to add in the next release!

I'm noticing the formatting gets a little messy in your example right before the break when the search bar jumps onto its own line:

@colleenmcginnis The test case was using code that we normally don't use in the topbarsticker, I updated it and it scales much cleaner!


Ignore function's main description

Hi,
First thing : mkdocstrings is awesome.
That being said, I'd like to hide the function's description when using the ::: app.function syntax.
For example, if I have this function in an app.py file:
def mkdocstrings_is_awesome(fish: str, percolator: set) -> bool:
    """But I'd be better if I could hide this description!

    Args:
        fish (str): a fish.
        percolator (set): this thing in a coffee machine.

    Returns:
        bool: whether there is a fish in the percolator.
    """
    return fish in percolator

and the mkdocstrings plugin called in my mkdocs.yml, the ::: app.mkdocstrings_is_awesome line renders this first screenshot.

I'd like an option like ignore_description to be able to render the second screenshot.
::: app.mkdocstrings_is_awesome
    options:
        ignore_description: true

And that's about it. Thanks again for this plugin! :smile:

As discussed privately, there's probably a way to do this with CSS only, by hiding the first paragraph(s?) of doc-contents divs.

Workaround
@artperrin As said by @pawamoy your request could be achieved with CSS.
This is a dirty but working workaround:
.doc > .doc-contents > p,
.doc > .doc-contents > div.admonition,
.doc > .doc-contents > details {
    display: none;
}

Before

After

Proposal
@pawamoy It would be cleaner and more precise if every section (returns, args, etc.) would have a custom class. For example, .doc-section-returns or .doc-section-args (or .doc-content-args).
This could make the filtering/styling way cleaner and easier, allowing us to rewrite the code used before like this:
.doc > .doc-contents > *:not(.doc-section-returns):not(.doc-section-args) {
    display: none;
}

@pawamoy What do you think? I don't think it should be difficult.
I could even try making a PR.

Hey @LukeSavefrogs! I'm open to adding more CSS classes to the HTML templates :slightly_smiling_face:
A pull request would be most welcome. If you add classes to the Material theme, I'd love that you also add them to other supported theme, when relevant of course.
Now, please note that there's an unreleased yet feature in the Python handler that allows to choose which sections to render, see: https://github.com/mkdocstrings/python/pull/56. I'll release once the other PRs there are merged :slightly_smiling_face:

Hey @LukeSavefrogs! I'm open to adding more CSS classes to the HTML templates 🙂 A pull request would be most welcome. If you add classes to the Material theme, I'd love that you also add them to other supported theme, when relevant of course.

I can't seem to find where the classes are declared though 🤔

Now, please note that there's an unreleased yet feature in the Python handler that allows to choose which sections to render, see: mkdocstrings/python#56. I'll release once the other PRs there are merged 🙂

I saw that, it would be really cool!

I can't seem to find where the classes are declared though :thinking:

That would be in the Python handler. The HTML templates are always contained in handlers. See https://github.com/mkdocstrings/python/tree/master/src/mkdocstrings_handlers/python/templates :slightly_smiling_face:


Add support for admin tab

"materializeTab": false, //false = tab.html, true = tab_m.html
"adminTab": {
"singleton": true,
"name": "Info", //or as Object "name" : {"de": "Info", "en": "Info"...}
"link": "", // optional - if not use tab.html
"fa-icon": "info" //(Materialize Icons) https://materializecss.com/icons.html
}

That's on the roadmap, but having an issue for it doesn't hurt :)

And I won't support non-materialize tabs for new adapters.


Do not treat errors in reading config as fatal

There are cases where rustfmt's call to fs::metadata() will fail
(due to permissions) and panic. Such cases should not be treated as
fatal.
For example, an ancestor directory or the home directory may exist but
the user may not have permissions to list the contents of the directory.
Some build systems may set HOME to something like
/var/empty/, which may not be readable/executable for the build user.
In such cases, rustfmt will fail.
Example on a Linux system where the user cannot read the home directory:
$ ls -ld /opt/root-only
drwx------ - root 2022-01-19 15:26 /opt/root-only/

$ HOME=/opt/root-only rustfmt --check /dev/null
Permission denied (os error 13)

Thanks for sharing your perspective and being willing to submit a pull request! However, I'm going to close this because this is change is not desirable.
I'm open to making the error behavior more friendly, but the inability to access the documented locations for config files is an error from the rustfmt pov and needs to be treated as such. It's not uncommon for users to unintentionally butcher permissions, and running rustfmt without the defined config values can result in very different formatting than with, something especially important in ide/editor  formatting scenarios.
I'm not sure exactly when the scenarios you've posited would typically occur, neither as part of the inner dev loop nor in a build context. If there are some realistic/non-hypothetical situations, they'd be fringe at most, and should be manageable via including the --config-path . option

To be more concrete, I actually ran into this error at work. The build system sets HOME set to /var/empty.  The /var/empty directory had was owned by the "root" user and had permissions set to 0700 (only owner can read/write/enter directory). This prevents programs (not just cargo/rustfmt) from saving state to the home directory. The idea is that the build is "hermetic" and not affected by any particular user's environment/config.
A workaround for this issue was to set HOME to a non-existent directory, like /var/this-does-notexit.
However, it seems arbitrary that rustfmt would distinguish between:

the home directory exists but cannot stat() the directory (treated as an error)
the home directory does not exist (silently ignores and falls back to default config)

https://github.com/rust-lang/rustfmt/blob/5c558e2c11f33e1e25932607458396be883d36b5/src/config/mod.rs#L355
As you mentioned, the error message could be more friendly (print more context like what file could not be opened).
The only error was Permission denied (os error 13).
To avoid confusion, let's track the "error message friendliness" in a separate issue.

I made issue #5188 to track the error message context.

Also, thanks for the pointer the --config-path option. I was confused by the help text, so I raised #5206.


Threads not closing properly

Currently no proper way of closing the Threads used for Queues and other Long- running tasks.
Since threads are "Run Forever" the .join() does not work. Workaround is to force-kill the application leaving these threads as Zombies.
The impact depends on your system. But a few megabytes may be wasted for zombies whenever restart the application.
Solution for this will come. May move to Redis + Celery

Not relevant anymore with new design


Enhanced Wayland support

Hello,
this pull request enhances current Wayland support by adding the following features:

support for multi-window applications
basic client-side window decorations
support for xdg-shell and ivi-application protocols

NOTE: xdg-shell and ivi-application are not core protocols and thus require some additional source code to be supported; typically, Wayland protocols are distributed as XML files and .c/.h files are generated by the wayland-scanner utility at build time. This pull request also contains such auto-generation, but an additional (manual) step is required at build time. Details are reported inside the README.

Since the changes requires a break in API compatibility (that is, periodic call to a cycle function is required), this pull request also changes the names for current functions, in order to stress the existence of the breakage.
API breakage has two reasons:

efficient multi-window support requires wl_display_flush to be called once per LVGL rendering and not once per window - thus the need to call periodically (ideally after lv_timer_handler(), as reported in the README) lv_wayland_cycle()
pthread-based event dispatching does not match really well with the rest of LVGL infrastructure (I know, my fault during first implementation)

About client-side window decorations: they are very basic, just a simple gray bar on top with close and (only for XDG shell) minimize buttons. Even when enabled at compile time, decorations can be disabled at runtime setting as environment variable LV_WAYLAND_DISABLE_WINDOWDECORATION=1. Resizing and maximization are not supported, since there is no way to notify LVGL that window size is changed (correct me on this point if I'm wrong).
Support for IVI shell (that is, the ivi-application prototcol) has only been smoke-tested at the moment.
Since changes are quite invasive, I marked this as RFC - in particular, I'd like to hear opinions about the code auto-generation for extra protocols.

I don't know wayland, so can not comment on the implementation details.  :slightly_frowning_face:
We have a discussion about how to handle small breaking changes but couldn't draw a conclusion yet. IMO it's acceptable to add these minor breaking changes. @embeddedt ?

Resizing and maximization are not supported, since there is no way to notify LVGL that window size is changed (correct me on this point if I'm wrong).

If you see a Wayland event you can use lv_disp_drv_update to set the a new resolution. It's already works with the windows port.

Why not ask the user to use monitor_cb to run lv_wayland_cycle? That should run it after the entire frame has been drawn (regardless of how many flush_cb calls it takes), which sounds like it's what you want.
Aside from that I think minor API tweaks to Wayland are fine. It's not one of the most popular drivers at the moment anyways.

Hello,
following received suggestion about monitor_cb, I refactored a bit the implementation. Now a call to lv_wayland_cycle is no more required: first call to lv_wayland_flush will install a custom monitor_cb callback, which will perform the flush & dispatch to the Wayland compositor. Eventually, if the user application installed an own monitor_cb callback, it will be saved and called as part of the custom one.
I'll keep the PR in draft for now, as I'm still working on removing the memcpy at flush.

thanks I'll try to test, something that would be welcome too, is maximization,
I remember I implemented it for qt-wayland years ago:
https://mastodon.social/@rzr/106170248748212334#qt-tizen-qtwayland-xdg-shell-20140425-rzr

Hello,
I added another round of modifications, with support for:

window maximization and un-maximization
close callback (that is, a function that is called when the user clicks on the X button on titlebar - can be useful for example to track window status and/or to implement for example save popups before exiting an application).

I reworked yet another time the display initialization, in order to provide the user a cleaner interface; now a single call to lv_wayland_create_window() is required to setup a LVGL display. An example follows:
lv_init(); lv_wayland_init(); disp1 = lv_wayland_create_window(DISP1_HRES, DISP1_VRES, "Window 1", close_cb); disp2 = lv_wayland_create_window(DISP2_HRES, DISP2_VRES, "Window 2", close_cb);
Left to be done:

support for window resizing through border dragging (should be simple, now that maximization is in place)
support for zero-copy double buffering

Hi,
Sounds very good! Thank you! I don't have a device with Wayland to test it now, but I'm sure that you have tested it and it works well. :slightly_smiling_face:
Let me know if I should merge it now, or after adding the mentioned last two features.

Hello,
I just added window resizing through dragging of borders, while I was considering whether to add double-buffering in this round (inital testing shows a significant loss in performance, in fact).
In order to simplify integration by end-user of this library, I decided to embed the initialization of the input devices inside the initialization of each window, giving the possibility to get an handle for them (similarly to what is done inside the win32drv driver). I updated the README to reflect this change.
Let's wait another couple of days, while I try to find some touchscreen-equipped device to test the touch integration, then I'll remove the draft flag and clear it for integration.
Thank you

Hello,
I tested (and fixed) the implementation also on a PC with a touch screen, so I think it can be ready for integration. I decided not to integrate (at least for now) the double-buffered feature, since using it performances are really poor. I squashed the commits into three, as they did not really add information about the implementation.
Please let me know if something more is required for the integration.
As a side note: if needed, the implementation can be easily tested on any machine running Linux using the Weston compositor, as it supports an X11 backend and can thus run also on systems still relying on the X window server.
Thank you

Hello,
Thank you very much! I haven't tried it out yet, just reviewed but it looks good in general. :+1:
I merge it now.


Integrate-Answers-in-QA-Page - Added Small Code

Closes: #885 (Draft PR)
Proposed changes
Currently the Q&A page only have the question list and form to submit the questions. There should be a way to see all the answers of the particular question sorted by the number of upvotes
Brief description of what is fixed or changed
Added The Answer Page For Specific Question When  User Clicks On The Get Answer Button
Types of changes

[x] Integrating Front End Answer For Specific Question. (Non -Breaking Changes)
[x] New feature (non-breaking change which adds functionality)
[x] Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[x] My changes generate no new warnings
[x] My changes does not break the current system and it passes all the current test cases.

Screenshots
Added Button In Question Card :

Answer Page For Specific Question By User IDs :

Other information
There Answer In Past Days But They Deleted The Answers And Question Also So I Will Work On The Issue  #886 And Then Back To This 😁

I will add all this soon I have exam that why I am inactive it's draft PR


Terminal: permission denied

<EMAIL_ADDRESS>Permission denied (publickey).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.

Resolved:
git clone https://github.com/zenangst/C.mode.git


Update groups-settings-cmdlets.md

Grammar fix

@alialo : Thanks for your contribution! The author(s) have been notified to review your proposed change.

@alialo : Thanks for your contribution! The author(s) have been notified to review your proposed change.

Looks good; thanks.

Looks good; thanks.


Update sql.html

解决null或者undefined情况下，str.replace报错。

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


nixpkgs-unstable fails on Hydra because of mariadb

Describe the bug
All last builds on Hydra have failed because of mariadb (top level package).
E.g. https://hydra.nixos.org/build/121987551

Nr
What
Duration
Machine
Status

1
Build of /nix/store/cpd44xn7kqjm88nmd7bh1b6jj4zbrb17-mariadb-server-10.4.13-man,/nix/store/n2nvzbpclai3jwfnzvz7wf8d27anmsjs-mariadb-server-10.4.13
n/a
n/a
Cached failure (log, raw, tail) (propagated from build 121469753)

First broken build 2020-06-05 19:14:56: https://hydra.nixos.org/build/121385458

Nr
What
Duration
Machine
Status

1
Build of /nix/store/gid5m5kxdh4q3mv8ar2a7l4wgr35g512-mariadb-server-10.4.13-man,/nix/store/8ksp3vsqycq52vrjx527qijbsbxlkzlk-mariadb-server-10.4.13
n/a
mac2-guest
Failed (log, raw, tail) (propagated from build 121385505)

Log: https://hydra.nixos.org/build/121385458/nixlog/1
...
-- Installing: /nix/store/8ksp3vsqycq52vrjx527qijbsbxlkzlk-mariadb-server-10.4.13/lib/pkgconfig/mariadb.pc
-- Installing: /nix/store/8ksp3vsqycq52vrjx527qijbsbxlkzlk-mariadb-server-10.4.13/share/aclocal/mysql.m4
-- Installing: /nix/store/8ksp3vsqycq52vrjx527qijbsbxlkzlk-mariadb-server-10.4.13/share/doc/mysql/mysql.server
mv: cannot stat '/nix/store/8ksp3vsqycq52vrjx527qijbsbxlkzlk-mariadb-server-10.4.13/OFF/suite/plugins/pam/pam_mariadb_mtr.so': No such file or directory
builder for '/nix/store/qm5xwgfb5cr1jw2zsiryp997czs8ja21-mariadb-server-10.4.13.drv' failed with exit code 1

To Reproduce
I can't reproduce the issue locally though, nix-build . -A mariadb does compile.
Expected behavior
Hydra job should not fail on mariadb.
Notify maintainers

@thoughtpolice
Metadata
Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.
Maintainer information:
# a list of nixpkgs attributes affected by the problem
attribute:
# a list of nixos modules affected by the problem
module:

Reproduction: are you sure you tried it on darwin?  (That's what fails.)

Probably caused by a1ec0288122ab8e1ebcf800a0c4fadcf51c87414, Darwin doesn't have .so 's right? cc @Izorkin


Suggestion - Update G&W platform

Suggestion to update the Game & Watch platform to support the Mame Retroarch core for those roms.

I'll give this a check when I can

I could not get them to work on the Android build of RetroArch's (1.18.0) MAME Current core (02.64 170929e0). Can you provide a tutorial?
Also you can add the MAME players yourself by manually adding it to the json file or downloading the mame platform > add mame players to the gw platform.

Found the issue. They are actually using the latest MESS core (which labels itself in retroarch as the mame core).
Here is the line from the platforms file for ES-DE
<command label="Multi (MESS)">%EMULATOR_RETROARCH% %EXTRA_CONFIGFILE%=/storage/emulated/0/Android/data/%ANDROIDPACKAGE%/files/retroarch.cfg %EXTRA_LIBRETRO%=mamemess_libretro_android.so %EXTRA_ROM%=%ROM%</command>
I tested on retroarch and I was able to load the MESS emulator, and then the mame based g&w roms loaded no problem.

So I confirmed it works by editing and then sideloading the current Game&watch.json platform file.
No idea if this is correct, but just tried to match the current naming conventions of things.
Also not sure if MESS is available on on the 32bit version of Retroarch or not. I know initially it was 64bit only, but that was a year or so ago.
    {       "name": "gw - RetroArch 64 - mess",       "uniqueId": "gw.ra64.mess",       "description": "Supported extensions: 7z, chd, cmd, zip.",       "acceptedFilenameRegex": "^(.*)\\.(?:7z|chd|cmd|zip)$",       "amStartArguments": "-n com.retroarch.aarch64/com.retroarch.browser.retroactivity.RetroActivityFuture\n  -e ROM {file.path}\n  -e LIBRETRO /data/data/com.retroarch.aarch64/cores/mamemess_libretro_android.so\n  -e CONFIGFILE /storage/emulated/0/Android/data/com.retroarch.aarch64/files/retroarch.cfg\n  -e IME com.android.inputmethod.latin/.LatinIME\n  -e DATADIR /data/data/com.retroarch.aarch64\n  -e APK /data/app/com.retroarch.aarch64-1/base.apk\n  -e SDCARD /storage/emulated/0\n  -e EXTERNAL /storage/emulated/0/Android/data/com.retroarch.aarch64/files\n  --activity-clear-task\n  --activity-clear-top",       "killPackageProcesses": true,       "killPackageProcessesWarning": true,       "extra": ""     },

Sorry for the late reply, I updated the platform c08b60c. It should work in both RA64 and RA (mess is not available on 32bit)


Add gtfs-viz from Gabriel Hidalgo

Project: https://github.com/gabrielAHN/gtfs-viz
Referenced in this LinkedIn post: https://www.linkedin.com/feed/update/urn:li:activity:7246478313114804224/
@gabrielAHN do you want to add this awesome tool to this list? 🙂
FYI this list is rendered here: https://gtfs.org/resources/overview/

Hey @isabelle-dr 👋 that would sound great, let me know what you need from me to make it happen 😊.
Maybe something like this for the blur:

GTFS Viz 🚉 - A web app that visualizes GTFS Data on the browser at scale without a backend on the client side using duckdb-wasm 🦆.

But yea let me know if you need anything else from me

Just opened a PR for it https://github.com/MobilityData/awesome-transit/pull/285


Syntax error: Piped optionals check

This code is not allowed in RiveScript-JS version 1.17.2
+ hello [|]
- hi

Syntax error: Piped optionals can't begin or end with a | at stream() line 1 near + hello [|]
But this error check is not detected in Python version. In Python, this pair matches the word hello exactly, without any additional word.

I look at the code to identify where the empty option is parsed in brain.py L447. It shows that the list of parts (after splitting by |) is not checked for empty string.
# Optionals.
optionals = re.findall(RE.optionals, regexp)
for match in optionals:
    parts = match.split("|")
    new = []
    for p in parts:
        p = r'(?:\\s|\\b)+{}(?:\\s|\\b)+'.format(p)
        new.append(p)

This bug does no harm if we miss empty options. However if we want to place a check for empty string here, check_syntax function alone doesn't check this bug. This is only uncovered during the replacement of our symbols (i.e. wildcards by its corresponding Regex).
With this finding, can you suggest what to do here? Do you want to parse the optional during the check_syntax (like JS version)?

Yeah it would be nicer to get the versions to all behave the same. The JavaScript version picked up the "empty pipe check" somewhere along the way via pull request #101, and the other versions didn't get updated to follow. Feel free to open a PR to apply the update to the Python version. :smile:


encrypt only secret fields

instead of encrypting the whole config we should encrypt keys that ends? or starts? with _ maybe we can have SecretField type?

Added fields.Secret. (see #68).

verified
created Digital Ocean client which has token property defined as secret
ssh = j.clients.sshkey.get(name= "testing_enc")
ssh.private_key_path = "/home/rafy/.ssh/id_rsa"
ssh.load_from_file_system()
dg = j.clients.digitalocean.get("testDG_enc")
dg.token = "f4f2bea............."
dg.save()

checked Configuration path
/home/rafy/.config/jumpscale/secureconfig/jumpscale/clients/digitalocean/digitalocean/DigitalOcean/testDG_enc

The token were encrypted


Fix installation for bash 3.2

Installation currently fails on bash 3.2 (the default version of bash that ships with macOS).
This is due to a bug in bash 3.2 related to sourcing from a process substitution.
This is a known workaround that will work for bash 3.x and newer versions.

deploying 0.15.2


Upgrade firebase messaging dependency

Upgrade plugin to support background notifications on Android O.

@wtrocki I have tested it in a device with Android 8.1.0 and it works like a charm

Can confirm that it works with cordova android 6 and cordova android 7.
No dex file issues!


Quarter placements

Can you add more placements:
top-quarter
top-three-quarters
Or is there some solution how can I add custom placements without modify your code?
Thanks

Sorry, I overlooked offset setting :)


Add GUADEC 2016 version with GNOME foot bullet points.

Changes:

Change aspect ratio to 16:10 (which should be what we have)
Change year and dates
use GNOME foot for bullet points
change skyline
Change city name

Hey, this repo has moved to https://git.gnome.org/browse/presentation-templates/

/me sets up an autoresponse

Alright, https://github.com/sils1297/guadec-presentation-templates/issues/12#issuecomment-224430472 is set up for issues and PRs.

Aha! And I tried to search git.gnome.org for "guadec" …


Some more documentation.

Adding some documentation to assist with the set up with diesel, #6

Will do 😁

I also added a link to the instructions from the README, if not desired let me know and can easily change it back 👍


internal, trailing spaces, folding, scriptanalyzer

Type of Change

[x] Bug fix (non-breaking change, fixes #)
[ ] New feature (non-breaking change, adds functionality)
[ ] Breaking change (effects multiple commands or functionality)
[ ] Ran manual Pester test and has passed (`.\tests\manual.pester.ps1)
[ ] Adding code coverage to existing functionality
[ ] Pester test is included
[ ] Nunit test is included
[ ] Documentation
[ ] Build system

Purpose
keeping internals up with 1.0 standards

hey niph, can you look into these failures?

yup

did it, fingers crossed

yessssssssssssss


Browser version: loading database causes quota exceeded error due to rename

Hi
I'm using nedb with wonderful results in a mixed nwjs/browser setting.
Lately I have come across a problem whereby loading a database that is more than half the localStorage quota will cause a QuotaExceededException, at least in Chrome 44.
The stacktrace shows the failure is happening because nedb is renaming from myDatabase~ to myDatabase during the loadDatabase call, though I can't tell why it should be doing that.
It seems counter-intuitive that a read operation should cause a quota exception – afterall the data has already been stored!
Any thoughts?

Hello Josh,
This mechanism is indeed strange at first sight but is used by the server version of nedb. The goal is to use temporary files when the cached database is persisted (which happens at startup in loadDatabase and depending on configuration can happen when the user requests it or at regular intervals) to avoid data loss issues if the machine crashes. Without this system if the machine crashes when nedb is rewriting the data file, the data file becomes empty and all data is lost.
It doesn't seem to be useful in a browser setting I agree, but the way the code is factorized makes it use this system. There are ways to fix it but since I'm quite short on time I'd first like to understand the use for your app, if you are already reasching 1/2 of the limit isn't there a good chance you reach the limit too ?

Thanks for the reply.
My app is for social research and stores a network-like data structure. The majority of the volume of the data results from every interaction with the app (touch or mouse click) being logged, which allows the network to be reconstructed and the interactions analysed. You're right that there's a good chance I could use the limit, but I would like to be able to use as much data as possible, and then catch the quota exception myself in order to manage it within my application logic.
There are a few options:

Could there be an optional parameter for loadDatabase which either loads the data with the risk of data loss during a crash? It could even be a different method which is browser specific.
Could there be a read-only loadDatabase that would at least allow the app to load?
Could you handle the quota exception within nedb, and use one of the above options within the catch statement?

I can look into implementing this, if needs be.

Any thoughts?

As of v1.4, NeDB automatically selects the best method to persist data on the browser (IndexDB then WebSQL then localStorage), so the limit is removed. WARNING: it's not back compatible with v1.3, but that should not be a major issue as user apps can resync entirely.
Also in the browser the crash safe mechanism is not used anymore, so you get the full available disk space, way more than 5MB (typically in 100s of MB)


Cannot install on windows

Cannot install the plugin on windows app, get an error each time i tried, 'ln -s' not recognised as an internal or external command, any help please

I have the same issue, ln is a linux command not available on windows. I have access to a macbook that I can use for now but I'm evaluating this library for a new project and some of the developers may only have windows machines. It would be nice if it could be installed on all platforms.

I think i am able to install it on my window machine now, i used Git Bash console, but at first try it doesnt work but on the second try it worked.

Adding through git bash worked for me as well, thanks.

Why is ln necessary?
Usually in react-native packages, the ios/android folder is in the root folder. Alternatively you can request users of the library to link manually to src/ios or src/android.

Symlinks play havoc with a lot of systems (Yarn and npm for one) and should be done manually via a script in very rare cases.

Fixed in ver 0.2.0

It is still not working in windows

It is still not working in windows


Specify Default Values in Local environment variables (secretstores.local.env) yaml?

Does the Local environment variables (secretstores.local.env) supports specifying default values?
I am trying to include Secrets as ENV VARs for Local Development purposes and I was wondering if there was a way to specify default values if ENV VARs does not exist.
I don't know if this is currently supported but I was thinking that default values could be specified like this:
apiVersion: dapr.io/v1alpha1
kind: Component
metadata:
  name: bill360-secret-store
  namespace: default
spec:
  type: secretstores.local.env
  version: v1
  metadata:
  - name: defaultValues
    value:
    - ENV_VAR_NAME_1=ENV_VAR_DEFAULT_VALUE_1
    - ENV_VAR_NAME_2=ENV_VAR_DEFAULT_VALUE_2

Can you elaborate on your scenario? The use case seems like it overlaps with the local file secretstore and it seems like it would be a gotcha for local development if a secret value can end up being different silently when an env variable was not set.

I believe you are right. I'll try using a File SecretStore.
Thanks @CodeMonkeyLeet!


Can I select causal attention for retrieval embeddings when using GritLM

In the paper, the ablation study about attention emb and gen is interesting.
Are these models all different models using each attention?
Can I select causal attention for both cases when using GritLM-7B?
If not, could you share the model using causal attention for both?
I selected 'cccc' and 'mean' for retrieval embeddings, but the performance was significant degraded :(

The models are all uploaded here: https://huggingface.co/collections/GritLM/gritlm-65cc1403df78d51bb89f1651
and linked to from the Appendix table of the paper: https://arxiv.org/pdf/2402.09906
It includes cccc models, for best performance, you probably want the one with cccc & weightedmean. Lmk if sth is unclear.

Thanks for your help.
I found the models for CCCC WM and CCCC LT.
CCCC WM https://hf.co/GritLM/gritlm_m7_sq2048_medi
CCCC LT https://hf.co/GritLM/gritlm_m7_sq2048_medi_lasttoken
I wonder if there is a CCCC & Mean model available and if there is severe performance degradation using the WM. I'm curious if using the CCCC & WM model is permissible for CCCC & Mean.

There's no trained CCCC + M model because WM is better. Why do you not want to use WM?

It does not matter to use CCCC + WM model for me.
I am just wondering why CCCC & M models have lower performance than CCCC & WM models.
Also, I have one question.
Excluding the prompt section, can cache reuse still be possible when using this model? If so, could it potentially improve the performance of Table 7 on the main paper?

I am just wondering why CCCC & M models have lower performance than CCCC & WM models.

The intuition for why WM is better than M for decoders and more comparisons are in this paper: https://arxiv.org/pdf/2202.08904 ; It's not super well-written though, sorry!

Also, I have one question. Excluding the prompt section, can cache reuse still be possible when using this model? If so, could it potentially improve the performance of Table 7 on the main paper?

Yes if you use CCCC then the performance in Table 7 with caching will be much better (at the cost of slightly worse retrieval as BB is better).

I selected 'cccc' and 'mean' for retrieval embeddings, but the performance was significant degraded :(

I'm surprised GritLM-7B gets that much worse if you do cccc & mean. Did you benchmark on MTEB how much of a difference it makes? It is probably worth checking that & comparing with "CCCC WM https://hf.co/GritLM/gritlm_m7_sq2048_medi".

Thank you for sharing your good paper :)
We are experimenting with GritLM in our application, and the top-2 hit rate is around 98% using bidirectional, but when using causal, the performance is below 50%.

We are experimenting with GritLM in our application, and the top-2 hit rate is around 98% using bidirectional, but when using causal, the performance is below 50%.

Oh that's poor ; https://hf.co/GritLM/gritlm_m7_sq2048_medi2 should be even better than https://hf.co/GritLM/gritlm_m7_sq2048_medi (Table 15). Unfortunately, I didn't train a CCCC E5 model :/ I think the caching issue can be easily solved by further finetuning GritLM to make it get used to that format but I haven't had the time to try.

Thanks for your suggestion.
I think that the suggested model is the embedding-only model.
Our goal is to share both the retriever and the generator, and to reduce inference cost by also sharing the cache.
I will try to utilize cccc & wm. Thank you.

Oh https://huggingface.co/GritLM/gritlm_m7_sq2048_medi2 should be both embedding & generation.
Did GritLM-7B not work well for your use case? Do you want to cache docs or queries or both?

In our case, we want to cache queries.
In the GritLM-7B case, query caches are calculated using bidirectional attention, so the generation performance is degraded a lot when using our instruction (prompt).
Therefore, we want to use [https://huggingface.co/GritLM/gritlm_m7_sq2048_medi2] model.

I see, let me know how it goes!

We are experiencing the following issue and are hoping you can provide some comments or solutions:
We are using the aforementioned causal/causal model and are proceeding with the following process:
Step1: We perform query forwarding for retrieval embedding, store that cache, and extract the Top-2 documents.
Step2: We put the query cache into the past key values, forward the documents, and generate the response.
When performing the process as described above, the results differ between when the cache is stored and when it is not. When we input the query and documents without storing the cache, the response is accurate. However, when we input the cache and documents, the response becomes completely strange (with repetitive phrases or nonsensical output).
We found that it seems that the causal/causal model uses a system prompt at the beginning unlike the released GritLM, and even after removing it for evaluation, we still get similarly strange and incorrect results.

causal/causal model uses a system prompt

What do you mean by this? There should be no system prompt besides the formatting i.e. <|embed|> etc. If you include the formatting in both with and without the cache, the results should be exactly the same.

Thank you for answering.
When we fixed the code, the results were same :)

It seems these two outcomes should be exactly the same, but since different results are coming out, I am asking this question.
When using both the query and text, the following format is utilized: FULL_FORMAT = "<|embed|>\n{query}\n<|user|>\n {text}\n\nOptionally using the prior context answer the query prior to it\n<|assistant|>\n"
When forwarding the query, "<|embed|>\n" + query is included, and when forwarding the text, the format "\n<|user|>\n {text}\n\nOptionally using the prior context answer the query prior to it\n<|assistant|>\n" is utilized.
It seems that both should be exactly the same, but the result of 1) comes out much better.

Yes, that looks good to me and it should be the same. A few checks:

Where is the boss token <s> being added? Make sure this is also the same. The tokenizer adds it automatically unless you pass add_special_tokens=False
Try forwarding "<|embed|>\n{query}\n<|user|>\n {text}\n\nOptionally using the prior context answer the query prior to it\n<|assistant|>\n" and check that the key value states are the same for the first X tokens as with "<|embed|>\n" + query
Ensure you use causal attention, the correct pooling method, and are passing the key value states

Thank you for answering.

I used add_special_tokens=True for query caching and add_special_tokens=False for generation. I checked the decoded result of the input, and confirmed that two inputs are same.
Let me check :). Thanks for valuable comments
In the generation stage, I think that causal attention is automatically used and pooling method is not used. Is it correct?

yes


Gradle plugin not resolving

I followed the instructions here:
https://plugins.gradle.org/plugin/org.owasp.dependencycheck
and here
https://jeremylong.github.io/DependencyCheck/dependency-check-gradle/
and get the following output with gradle v2.2.1 and with v2.14.1

* What went wrong:
A problem occurred evaluating script.
> Failed to apply plugin [id 'org.owasp.dependencycheck']
   > Plugin with id 'org.owasp.dependencycheck' not found.

* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output.

Is there a minimum gradle version require for the plugin? I'm not sure what I'm doing wrong. I did notice that gradle pulled down the following dependencies
onfiguration on demand is an incubating feature.
Download https://jcenter.bintray.com/org/owasp/dependency-check-gradle/1.4.3/dependency-check-gradle-1.4.3.pom
Download https://jcenter.bintray.com/org/owasp/dependency-check-core/1.4.3/dependency-check-core-1.4.3.pom
Download https://jcenter.bintray.com/org/owasp/dependency-check-parent/1.4.3/dependency-check-parent-1.4.3.pom
Download https://jcenter.bintray.com/org/owasp/dependency-check-utils/1.4.3/dependency-check-utils-1.4.3.pom
Download https://jcenter.bintray.com/org/slf4j/slf4j-api/1.7.21/slf4j-api-1.7.21.pom
Download https://jcenter.bintray.com/org/slf4j/slf4j-parent/1.7.21/slf4j-parent-1.7.21.pom
Download https://jcenter.bintray.com/org/apache/commons/commons-compress/1.12/commons-compress-1.12.pom
Download https://jcenter.bintray.com/org/apache/lucene/lucene-core/4.7.2/lucene-core-4.7.2.pom
Download https://jcenter.bintray.com/org/apache/lucene/lucene-parent/4.7.2/lucene-parent-4.7.2.pom
Download https://jcenter.bintray.com/org/apache/lucene/lucene-solr-grandparent/4.7.2/lucene-solr-grandparent-4.7.2.pom
Download https://jcenter.bintray.com/org/apache/lucene/lucene-analyzers-common/4.7.2/lucene-analyzers-common-4.7.2.pom
Download https://jcenter.bintray.com/org/apache/lucene/lucene-queryparser/4.7.2/lucene-queryparser-4.7.2.pom
Download https://jcenter.bintray.com/org/glassfish/javax.json/1.0.4/javax.json-1.0.4.pom
Download https://jcenter.bintray.com/org/glassfish/json/1.0.4/json-1.0.4.pom
Download https://jcenter.bintray.com/com/sun/mail/mailapi/1.5.5/mailapi-1.5.5.pom
Download https://jcenter.bintray.com/com/sun/mail/all/1.5.5/all-1.5.5.pom
Download https://jcenter.bintray.com/com/h2database/h2/1.3.176/h2-1.3.176.pom
Download https://jcenter.bintray.com/org/apache/lucene/lucene-queries/4.7.2/lucene-queries-4.7.2.pom
Download https://jcenter.bintray.com/org/apache/lucene/lucene-sandbox/4.7.2/lucene-sandbox-4.7.2.pom
Download https://jcenter.bintray.com/org/owasp/dependency-check-gradle/1.4.3/dependency-check-gradle-1.4.3.jar
Download https://jcenter.bintray.com/org/owasp/dependency-check-core/1.4.3/dependency-check-core-1.4.3.jar
Download https://jcenter.bintray.com/org/owasp/dependency-check-utils/1.4.3/dependency-check-utils-1.4.3.jar
Download https://jcenter.bintray.com/org/slf4j/slf4j-api/1.7.21/slf4j-api-1.7.21.jar
Download https://jcenter.bintray.com/org/apache/commons/commons-compress/1.12/commons-compress-1.12.jar
Download https://jcenter.bintray.com/org/apache/lucene/lucene-core/4.7.2/lucene-core-4.7.2.jar
Download https://jcenter.bintray.com/org/apache/lucene/lucene-analyzers-common/4.7.2/lucene-analyzers-common-4.7.2.jar
Download https://jcenter.bintray.com/org/apache/lucene/lucene-queryparser/4.7.2/lucene-queryparser-4.7.2.jar
Download https://jcenter.bintray.com/org/glassfish/javax.json/1.0.4/javax.json-1.0.4.jar
Download https://jcenter.bintray.com/com/sun/mail/mailapi/1.5.5/mailapi-1.5.5.jar
Download https://jcenter.bintray.com/com/h2database/h2/1.3.176/h2-1.3.176.jar
Download https://jcenter.bintray.com/org/apache/lucene/lucene-queries/4.7.2/lucene-queries-4.7.2.jar
Download https://jcenter.bintray.com/org/apache/lucene/lucene-sandbox/4.7.2/lucene-sandbox-4.7.2.jar

Can you share more of how you actually configured dependency check in your build?  What error do you get with --stacktrace and/or --debug?
I am running the following and it works for me:

------------------------------------------------------------
Gradle 2.13
------------------------------------------------------------

Build time:   2016-04-25 04:10:10 UTC
Build number: none
Revision:     3b427b1481e46232107303c90be7b05079b05b1c

Groovy:       2.4.4
Ant:          Apache Ant(TM) version 1.9.6 compiled on June 29 2015
JVM:          1.8.0_92 (Oracle Corporation 25.92-b14)
OS:           Windows 10 10.0 amd64

Sure thing.
https://github.com/spyhunter99/gradle-fury/blob/feature/%23178/gradle/quality.gradle

Okay, I think I have the necassary changes to your quality.gradle (based on bugs or poor documentation within gradle ;) ).
First, in an external script it appears you must reference the plugin using the fully qualified class name instead of the plugin id. So if you change your apply plugin line to:
apply plugin: org.owasp.dependencycheck.gradle.DependencyCheck

Next, you need to reference a task using the quoted task name. So add the single quotes around 'dependencyCheck' in the following section:
//dependencyCheck --info
//OWASP dependency-check is a utility that detects publicly disclosed vulnerabilities in application dependencies.

afterEvaluate {
    try {
        check.dependsOn 'dependencyCheck'
    } catch (Exception ex) {
        System.out.println(project.name + ": " + ex.getMessage())
    }
}

Those changes seemed to make the dependency-check plugin work on my local system. Let me know if this works for you.

Those changes exactly work with gradle 2.14.1. I'll update again once travis-ci has finished building. Kick ass! Very exciting. thanks!

Works on all tested versions of gradle over here. Awesome. The irony is that it found that findbugs uses a dependency on the list, whoops!


add missing dep: requests

Example code breaks due to missing dep: requests

I just added this to requirements.txt, maybe more stuff needs to be changed to fix this idk

Thanks for the PR! Looks good to me


Support optional serializing of properties

Consider:
@json
class Person {
  constructor(public firstName: string, public lastName: string) {}

  get fullName(): string {
    return `${this.firstName} ${this.lastName}`;
  }
};

...
const p = new Person("John", "Doe")
const s = JSON.stringify(p);

output:
{"firstName":"John","lastName":"Doe"}

How might I indicate that I want fullName included in the output?
With JavaScript, I can do that by explicitly specifying the requested fields.  In other words:
const s = JSON.stringify(p, ['firstName', 'lastName', 'fullName']);

I could see that working here as well.  Alternatively, perhaps we can do something with decorators on the property itself, such as:
  @json(include: true)
  get fullName(): string {
    return `${this.firstName} ${this.lastName}`;
  }

Thoughts?

Perhaps we could make it default to serialize/deserialize if a getter/setter pair exists which would be implemented in the transform.
I like the idea of having an option argument JSON.stringify<T, C>(data: T, config: C): string, but I'm not sure how deeply string matching would impact performance.
Another option could be to use some sort of comment...
// include: fullName
@json
class Vec3 {
class Person {
  constructor(public firstName: string, public lastName: string) {}

  get fullName(): string {
    return `${this.firstName} ${this.lastName}`;
  }
}

Any of these could totally be implemented. I ordered them top-to-bottom for complexity

@casimiro, looping you in here.
Supporting default values (https://github.com/JairusSW/as-json/issues/51) kind of loops in here.
Currently, as-json just allocates empty space for the class, so default values and all values are uninitialized.
One other solution here is to parse out to a object, string, number, variable, ect..
@json
class Vec3 {
    x: f64 = 3.4;
    y: f64 = 1.2;
    z: f64 = 8.3;
}

const parsedVec = new Vec3();
// Default values initialized. Do whatever logic you want here.

JSON.parseTo<Vec3>(parsedVec, '{"y":8.4}');

console.log(JSON.stringify(parsedVec));
// {
//    "x": 3.4,
//    "y": 8.4,
//    "z": 8.3
// }

@JairusSW I really like the idea of parsing out to an object -- it favours efficiency and the interface remains simple and clear.

I'm on Thanksgiving break, so I'm planning to spend some time on this. :)

Implemented in latest version. I will implement parsing out to an object later as that is really cool


feat(calcite): support in-predicate (#204)

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.Roman Kondakov seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.


[BUG] image-load rsults in ctr: executable file not found in $PATH"

What did you do?

How was the cluster created?

k3d create

then
$ k3d i alpine
2019/10/12 00:09:52 INFO: Saving images [alpine] from local docker daemon...
2019/10/12 00:09:53 INFO: saved images to shared docker volume
2019/10/12 00:09:53 INFO: Importing images [alpine] in container [k3d-k3s-default-server]
2019/10/12 00:09:53 ERROR: seems like something went wrong using `ctr image import` in container [k3d-k3s-default-server]. Full output below:
OCI runtime exec failed: exec failed: container_linux.go:345: starting container process caused "exec: \"ctr\": executable file not found in $PATH": unknown

The above flow works when running locally, but not in CI. Our CI setup is a GKE cluster, where the test runs as a pod. Inside the pod, we run daemon -U -- dockerd to get docker in docker support.
What did you expect to happen?
Loading image works.
I verified its not just a weird path issue, ctr exist in bin/ when run locally but not in CI, which it is still the same rancher/k3s:v0.7.0 running in both cases.
Concise description of what you expected to happen after doing what you described above.
Screenshots or terminal output
If applicable, add screenshots or terminal output (code block) to help explain your problem.
Which OS & Architecture?

Linux, Windows, MacOS / amd64, x86, ...?

Which version of k3d?
v1.3.1
Which version of docker?
19.03 in CI, 18.09 locally

Hi @howardjohn, thanks for opening this issue :+1:
Looks like an interesting problem, where I don't think that we have to search for the problem in k3d or k3s...
Can you post the pod manifest used in CI here please? And then the steps from starting the docker daemon over creating the cluster to importing the image?
This way I can try to replicate and debug your issue.

Pod spec:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: k3d
spec:
  selector:
    matchLabels:
      app: k3d
  template:
    metadata:
      labels:
        app: k3d
    spec:
      containers:
      - name: k3d
        image: gcr.io/istio-testing/build-tools:2019-10-11T13-37-52
        securityContext:
          privileged: true
        command:
        - bash
        - -c
        - |-
          echo starting
          (cd /tmp; go get<EMAIL_ADDRESS>          echo starting dockerd
          dockerd > /dev/null 2>&1 &

          echo "Waiting for dockerd to start..."
          while :
          do
            echo "Checking for running docker daemon."
            if [[ $(docker ps) ]]; then
              echo "The docker daemon is running."
              break
            fi
            sleep 1
          done
          docker pull alpine
          k3d c --wait 60
          k3d i alpine || k3d d
          k3d d

Logs
starting dockerd
Waiting for dockerd to start...
Checking for running docker daemon.
The docker daemon is running.
Using default tag: latest
latest: Pulling from library/alpine
9d48c3bd43c5: Pulling fs layer
9d48c3bd43c5: Verifying Checksum
9d48c3bd43c5: Download complete
9d48c3bd43c5: Pull complete
Digest: sha256:72c42ed48c3a2db31b7dafe17d275b634664a708d901ec9fd57b1529280f01fb
Status: Downloaded newer image for alpine:latest
docker.io/library/alpine:latest
2019/10/12 15:05:27 Created cluster network with ID 6b61297ec7d49e759488d9f3221e527ee05cb46e8ded86b1c362d6a5dc766d9e
2019/10/12 15:05:27 Created docker volume  k3d-k3s-default-images
2019/10/12 15:05:27 Creating cluster [k3s-default]
2019/10/12 15:05:27 Creating server using docker.io/rancher/k3s:latest...
2019/10/12 15:05:27 Pulling image docker.io/rancher/k3s:latest...
2019/10/12 15:05:42 SUCCESS: created cluster [k3s-default]
2019/10/12 15:05:42 You can now use the cluster with:

export KUBECONFIG="$(k3d get-kubeconfig --name='k3s-default')"
kubectl cluster-info
2019/10/12 15:05:42 INFO: Saving images [alpine] from local docker daemon...
2019/10/12 15:05:42 Pulling image docker.io/iwilltry42/k3d-tools:v0.0.1...
2019/10/12 15:05:46 INFO: saved images to shared docker volume
2019/10/12 15:05:46 INFO: Importing images [alpine] in container [k3d-k3s-default-server]
2019/10/12 15:05:46 ERROR: seems like something went wrong using `ctr image import` in container [k3d-k3s-default-server]. Full output below:
OCI runtime exec failed: exec failed: container_linux.go:345: starting container process caused "exec: \"ctr\": executable file not found in $PATH": unknown
2019/10/12 15:05:46 Removing cluster [k3s-default]
2019/10/12 15:05:46 ...Removing server
2019/10/12 15:05:46 ...Removing docker image volume
2019/10/12 15:05:46 SUCCESS: removed cluster [k3s-default

Another one with a differnt base image that sets up docker differently (service docker start instead of dockerd):
apiVersion: apps/v1
kind: Deployment
metadata:
  name: k3d
spec:
  selector:
    matchLabels:
      app: k3d
  template:
    metadata:
      labels:
        app: k3d
    spec:
      containers:
      - name: k3d
        image: gcr.io/istio-testing/istio-builder:v20190829-4edc0e70
        securityContext:
          privileged: true
        env:
        - name: GO111MODULE
          value: "on"
        - name: GOPROXY
          value: "https://proxy.golang.org"
        command:
        - entrypoint
        - bash
        - -c
        - |-
          export PATH=$HOME/go/bin:$PATH
          echo starting
          go get<EMAIL_ADDRESS>          docker pull alpine
          k3d c --wait 60
          k3d i alpine || k3d d
          k3d d

Hi @howardjohn , I just found time to check this out again.
Can you please try to specify a specific version of k3s?
E.g. k3d c --wait 60 --image rancher/k3s:v0.9.1?
I just tried your first DeploymentSpec and it failed because of other reasons, however, I saw, that it uses rancher/k3s:latest (it doesn't use a specific tag, because you're pulling it via go get, which doesn't "bake" a k3s tag into the code like make build does).
rancher/k3s:latest however is not kept up-to-date (it's 5 months old right now) and thus may not include the ctr executable.
My solutions would be:
a) run make build in the pulled source code directory (wouldn't recommend this, as it might fail if GitHub's API request limit was exceeded)
b) use curl or wget and our install.sh to get the latest (or a specific) release
c) use --image to choose a k3s version >= v0.7.0 which includes ctr

Thank you for the --image rancher/k3s:v0.9.1 flag, fixed this issue for me ❤❤❤ Also nicer to run the same version of k3s as in prod.

Perfect :+1::grin:


[IMAGING-319] Bugfix for lost characters & fixed test

Combined PR of bugfix #359 & fixed test #366 plus added test image.

@kinow FYI

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 71.64%. Comparing base (fd2b5b8) to head (2977e09).
Report is 19 commits behind head on master.

Additional details and impacted files
@@             Coverage Diff              @@
##             master     #367      +/-   ##
============================================
+ Coverage     71.48%   71.64%   +0.15%
- Complexity     3580     3588       +8
============================================
  Files           352      352
  Lines         16919    16908      -11
  Branches       2648     2648
============================================
+ Hits          12095    12114      +19
+ Misses         3785     3756      -29
+ Partials       1039     1038       -1

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

@garydgregory It's fixed. You can approve the workflow again.

?

?

Er... we are volunteers with busy lives ;-) Tomorrow... I'll have to find out if we can use a file licensed under the "Unsplash license".

?

Er... we are volunteers with busy lives ;-) Tomorrow... I'll have to find out if we can use a file licensed under the "Unsplash license".

https://github.com/apache/commons-imaging/pull/359#issuecomment-1925741932

I do not see the "Unsplash license" listed as a Category A or Category B license on https://www.apache.org/legal/resolved.html so I this will take a bit more digging.

Thomas said „no“.
I disagree with his reasoning because Apache Commons Imaging is not a service like Unsplash and also is not selling anything.
The image is used according to the terms.
Maybe other legal team members will share my view.
However, I can remove it from the PR if you want.
Maybe if you and/or @kinow  have seen and verified that both the bugfix and test are correct, this is enough proof.
There is really a serious bug present and my PR fixes it. I swear!
The only thing that will be lost without the image is a regression protection.

And from a quick read at the code I think you might be correct 🙂

By the time I was porting Commons Imaging to Kotlin I created my own exif writer roundtrip test to make sure no data get lost. And to have a broad set of test files I used the Unsplash lite set, which contains 25k images from all kinds of cameras - with all kinds of oddities, too.
For a good chunk of these files I found that there was a data corruption and not knowing what it caused I wanted to open up a issue first, but I saw on the tracker that someone else also encountered this problem and @gwlucastrig luckily already found the cause and provided a fix. :)
That's why I can be so certain that the bugfix is correct. ;)
So all credit for #359 goes to him.

We can wait a bit longer for other to review the legal issue.

Yes, we can. There is no hurry. I would just recommend to not release a M1 milestone release with that bug in the interest of the project.

Which is why we try to enforce the policy to always include tests with the changes.

I agree that this is in general a good policy. Especially for new features that you want to be covered. But I don't feel that it's right to block a critical bugfix that can be manually verified because the lack of an test.

from what I understood, you wrote the fix and a test.

Yes, the fix was @gwlucastrig work, but the correction of the exif roundtrip test was my own.

You used an image with a license that could be blocked by ASF's legal.

At first I did not include it, because I can't decide such a thing for Apache.
Gary wanted me to include it without checking with legal first. That's one of the things that went wrong in our communication.

But IMHO, you went the extra mile already by writing the test and trying to sort out the issue.

Yes, even if I'm using a fork without this bug and personally have no issue anymore, I felt I wanted to contribute something important back in my free time to thank Apache. Writing my lib from scratch would be much harder, because on porting the code to Kotlin I first grasped how image metadata actually works thanks to this project.
I was paid for my time figuring out how the exif roundtrip test actually must be crafted. Porting my findings from Kotlin back to Java was just doing and I somehow owed it.

You don't have to do everything yourself

Right :)

You can leave the PR as-is, and one of us can take a look to see if we can find other images, or even craft one using some editor like GIMP, or searching in other archives, etc.

That's right. You can drop in images from allowed sources and see where the test fails on.
Having access to Unsplash images, which should be totally possible according to the lincese, would still be a benefit, because it's really a large collection. And a lot of metadata there is partly broken, because apparently many cameras write fault metadata and a lot of tools corrupt it.
So Apache Commons Imaging is by far not the only tool corrupting metadata. Sadly a lot of tools do it. And I try very hard that my app won't become one of them. :D

@garydgregory @kinow
In the mean time legal decided that the test image can't be included.
So I close this PR and re-open #359 for the bugfix and #366 for the completed unit test. I close this PR.
I think by now I did everything that I can do to contribute back something and you can either take my donation or leave it. That will be up to you.
Have a nice day!


Enhancement :  Update username for Assets Moved with Renovator JCR_LAST_MODIFIED_BY

Required Information

[ ] AEM Version, including Service Packs, Cumulative Fix Packs, etc: AEM 6.5.2
[ ] ACS AEM Commons Version: 4.8.6
[ ] Reproducible on Latest? yes

Expected Behavior
Update the JCR_LAST_MODIFIED_BY field of the assets moved by Renovator with username "Renovator".
Having a username updated for assets moved by "Renovator" helps organizations that have custom Asset workflows to exclude the assets moved from standard workflows (Activation Workflow) to control the publish of these moved assets to scene7 etc and also get a report of assets moved.
Actual Behavior
Currently, the JCR_LAST_MODIFIED_BY is untouched.
Steps to Reproduce
Upload a spreadsheet with a list of assets that need to be moved.
Links

It sounds like a compelling feature but we have to be careful not to impose additional metadata rules on the user.  For example, jcr last modified attributes are generally updated by the product when you save changes to a node and I'm not 100% sure we can change that to something other than the user name of the current session.  On the other hand, if we apply a new property called something slightly different we can store whatever we want and achieve the desired effect.   Unfortunately over time having two last modified by fields could get very confusing because we don't know which one is more correct by looking at it.
Instead, we might consider an alternative: An option exposed to the user that, when checked, will record the timestamp in a new property like "renovator_process_time" or something like that.   This tells us two useful things: 1) That renovator did something to this content and 2) When it happened.  So we could then use a date range to narrow down the time as well, making our query even more useful.
Thoughts?

(it should be an optional feature that the user must opt-in, that much I'm pretty sticky about because some people might not want metadata updates of any kind because they could cause workflow triggers or other undesired side-effects)

@badvision
We drew inspiration for using the "LastModifiedBy" field after looking at some code in ACS Commons.
https://github.com/Adobe-Consulting-Services/acs-aem-commons/blob/4b3da3f5fdf0b59049b966bea9306d24c87d0293/bundle/src/main/java/com/adobe/acs/commons/dam/impl/ReviewTaskAssetMoverHandler.java#L145
We could go the route of adding a new property to have some constant value as we need that to exclude in workflow config. Date ranges may not be a great way to set up exclude for us.
As for having the user opt-in if he wants the field updated, this was one of our initial ideas then we saw the User name being thrown in as constant.  Here is our proposal
We add user opt-in for updating the user-field. If they choose it we will update it with a default value "Renovator"
This provides the users not to choose "user-field" updates and if they choose for reasons similar to us they get something constant they could use as a hook for their workflows or processes.
We are open to more ideas. Thank  You!

@badvision  : Waiting for your thoughts on the above so we can determine how we can proceed. Please let me know if you would like more details.


Cryptsy 502 error

I was wondering if anybody is experiencing random or 502 errors from making calls to Cryptsy.
For instance one of the domain (ie. pubapi1, http://pubapi1.cryptsy.com/api.php?method=singlemarketdata&marketid=132) would give me a 502 error, and the other (ie. pubapi2, http://pubapi2.cryptsy.com/api.php?method=singlemarketdata&marketid=132) works fine.
The default pubapi, gives me error from time to time. Are you experiencing this?
Was wondering if anybody is voting to migrate the API endpoint to Cryptsy's V2 API over at https://www.cryptsy.com/pages/apiv2/markets ?

I'd like it to use a combination of V1 and V2. e.g. the withdrawal stuff in V2 requires a callback url in the settings. so id say use V2 for everything besides deposits and withdrawals.

For several hours i'm getting only errors 502 and 503 from cryptsy. And indeed, pubapi2 seems to work with the given link where pubapi1 gives the 502.
So I agree with tmlee, why not change to v2 if that is giving less problems ?

I too was also getting this.  For me anyway, I reached the conclusion that I was making too many (too frequent) getExchangeSymbols calls in an effort to populate CryptsyCurrencyUtils.  The reason I'm confident that it was this is that I changed my API key in between making sequential calls and it allowed one to go through immediately after the new key.


improve performance (cargo bench) by 20-30%

The RootIterator is the heart of all detection logic. It performs up to tens or even hundreds of thousands of iterations per beat detection.
There is good potential to reduce the amount of needed iterations with only sacrificing minimal loss in accuracy - which is even irrelevant given the high sampling rates.
Benchmark cargo bench on my 11th Gen Intel(R) Core(TM) i7-1165G7 @ 2.80GHz:
before
simulate beat detection (with lowpass) with 4096 samples per invocation
                        time:   [1.9795 ms 1.9882 ms 1.9988 ms]
                        change: [+21.442% +22.009% +22.798%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 11 outliers among 100 measurements (11.00%)
  4 (4.00%) high mild
  7 (7.00%) high severe

simulate beat detection (no lowpass) with 4096 samples per invocation
                        time:   [2.0585 ms 2.0809 ms 2.1089 ms]
                        change: [+20.353% +21.636% +23.305%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 10 outliers among 100 measurements (10.00%)
  5 (5.00%) high mild
  5 (5.00%) high severe

after
simulate beat detection (with lowpass) with 4096 samples per invocation
                        time:   [1.6477 ms 1.6498 ms 1.6523 ms]
                        change: [-17.421% -16.968% -16.584%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 10 outliers among 100 measurements (10.00%)
  1 (1.00%) low mild
  6 (6.00%) high mild
  3 (3.00%) high severe

Benchmarking simulate beat detection (no lowpass) with 4096 samples per invocation: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 8.7s, enable flat sampling, or reduce sample count to 50.
simulate beat detection (no lowpass) with 4096 samples per invocation
                        time:   [1.7221 ms 1.7230 ms 1.7239 ms]
                        change: [-18.165% -17.062% -16.138%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 9 outliers among 100 measurements (9.00%)
  1 (1.00%) low mild
  6 (6.00%) high mild
  2 (2.00%) high severe

On my Intel i5 10th Generation, the performance boost is +30%.
I think, this is the best I can get for now. Further performance improvement ideas that I have:

Internally, scale down the signal to 1/100 of the sampling rate. I only need a sampling rate of 200Hz to detect beats of 20-80 Hz.
drop envelope detection, use state machine-like approach to detect beat (should work pretty well, I think)


[v4.4] DHCP Server registering multiple callbacks on multiple start/stop (IDFGH-9680)

Answers checklist.

[X] I have read the documentation ESP-IDF Programming Guide and the issue is not addressed there.
[X] I have updated my IDF branch (master or release) to the latest version and checked that the issue is present there.
[X] I have searched the issue tracker for a similar issue and not found a similar issue.

IDF version.
v4.4.4-116-g00bb43ff24
Operating System used.
Windows
How did you build your project?
VS Code IDE
If you are using Windows, please specify command line type.
None
Development Kit.
ESP32S3-WROOM N8R2
Power Supply used.
USB
What is the expected behavior?
Coming after https://github.com/espressif/esp-idf/issues/10762 .
The DHCP Server now is able to restart each time with our patch (no further changes provided from espressif to date).
When one client connects to the mesh root, we expect the root to provide one IP address
What is the actual behavior?
The root provides multiple times the IP address, having a callback triggered for each time the dhcps was started
Steps to reproduce.

Start DHCPS
Connect child and give IP
Disconnect
Stop DHCPS
Repeat from 1
...

Debug Logs.
W (10:52:40.010) mesh_main: <MESH_EVENT_ROUTING_TABLE_ADD>add 1, new:2
I (153191982) mesh: <nvs>write assoc:1
I (10:52:40.027) mesh_main: <MESH_EVENT_CHILD_CONNECTED>aid:1, 7c:df:a1:e0:8b:7c
I (10:52:40.261) esp_netif_lwip: DHCP server assigned IP to a station, IP is: <IP_ADDRESS>
W (10:52:40.262) mesh_main: IP EVENT 2
I (10:52:40.297) esp_netif_lwip: DHCP server assigned IP to a station, IP is: <IP_ADDRESS>
W (10:52:40.299) mesh_main: IP EVENT 2
I (10:52:40.303) esp_netif_lwip: DHCP server assigned IP to a station, IP is: <IP_ADDRESS>
W (10:52:40.309) mesh_main: IP EVENT 2
I (10:52:40.351) esp_netif_lwip: DHCP server assigned IP to a station, IP is: <IP_ADDRESS>
W (10:52:40.352) mesh_main: IP EVENT 2
I (10:52:40.357) esp_netif_lwip: DHCP server assigned IP to a station, IP is: <IP_ADDRESS>
W (10:52:40.362) mesh_main: IP EVENT 2
I (10:52:40.367) esp_netif_lwip: DHCP server assigned IP to a station, IP is: <IP_ADDRESS>
W (10:52:40.375) mesh_main: IP EVENT 2
I (10:52:40.404) esp_netif_lwip: DHCP server assigned IP to a station, IP is: <IP_ADDRESS>
W (10:52:40.406) mesh_main: IP EVENT 2
I (10:52:40.417) esp_netif_lwip: DHCP server assigned IP to a station, IP is: <IP_ADDRESS>
W (10:52:40.418) mesh_main: IP EVENT 2
I (10:52:40.452) esp_netif_lwip: DHCP server assigned IP to a station, IP is: <IP_ADDRESS>
W (10:52:40.453) mesh_main: IP EVENT 2
I (10:52:41.359) mesh_main: Tried to publish layer:1 IP:<IP_ADDRESS>

More Information.
No response

hi @KonssnoK
I (10:52:40.452) esp_netif_lwip: DHCP server assigned IP to a station, IP is: <IP_ADDRESS>
Does this log have any impact?

hi @KonssnoK
dhcp_server.txt
Please add this log test. Thank you

i can't currently proceed due to crashes on other issues.
I'll continue when those are solved.
https://github.com/espressif/esp-idf/issues/9955
https://github.com/espressif/esp-idf/issues/11006

@xueyunfei998 as expected the issue is the udp pcbs not being correctly cleaned, as can be seend from the following log
I (16:18:09.176) mesh_main: <MESH_EVENT_CHILD_CONNECTED>aid:1, 7c:df:a1:e0:8b:7c
tcpip_thread: PACKET 0x3fcf3094
pcb (100000A, 67) <-- (0, 0)
pcb (0, 68) <-- (0, 67)
pcb (100000A, 67) <-- (0, 0)
pcb (100000A, 67) <-- (0, 0)
pcb (0, 56474) <-- (0, 0)
tcpip_thread: PACKET 0x3fcf7ef0
pcb (100000A, 67) <-- (0, 0)
pcb (0, 68) <-- (0, 67)
pcb (100000A, 67) <-- (0, 0)
pcb (100000A, 67) <-- (0, 0)
pcb (0, 56474) <-- (0, 0)
handle_dhcp,1064
send_ack,706

so, from further analysis it seems the issue is triggered by the example code.
if esp_netif_dhcps_stop(netif_ap); is not called when changing netif interface, the udp pcb will continue to contain references , triggering multiple times on a single packet.
i will change the code of the example grouping the destroy code as follows>

(the example ip_internal_network should be updated)

(the example ip_internal_network should be updated)

It has been updated to 4.4