[RFC] New Hire Buddies

Insight: assigning someone as a mentor to a new hire that has been at Artsy for a while is good because they know lots of things, but it's bad because it's probably been a long time since they set certain things up. Another new hire is in a much better position to assist with these types of things because they just solved these problems!
For example, @pepopowitz just started and @starsirius asked if I'd like to be a mentor for him and I was happy to. I was able to help with a couple questions he had. Later he was asking about creating AWS and Jira accounts - something I haven't really thought much about.
@javamonn (for example) would be better suited to help getting someone setup on Jira since he's just done it and I actually don't even remember how I did it. Similarly, @ashleyjelks has just setup her AWS creds and ran into gotchas - she's in a much better position to help him out.
As a bonus, it feels good when you're new to a company to be asked to help with things. Makes one feel like they know stuff and are being helpful!
So, I wonder if there's a way to connect new hires so that they can learn from each other. It could be as informal as a slack group or formalized by assigning buddies in some way. Open to ideas here!

Nice, yeah, we've discussed this idea before and I've been a fan of the idea. 👍

I agree with the points raised here as well. WRT ideas about how to execute it effectively:
-perhaps new hires could be assigned a new hire contact person (not to replace the onboarding buddy/mentor role currently in place) to help with the sort of issues that come up when you first join an org and are trying to get everything set up and properly configured
-another idea could be an onboarding Slack group where more recent hires post config/onboarding specific questions and less recent hires are responsible for helping/pointing them in the right direction.
-another possibility here could be a point person list that is kept up to date so new folks know the best people to reach out to for specific questions?
All suggestions here to take or leave, but great proposal!

Love this idea!

-perhaps new hires could be assigned a new hire contact person (not to replace the onboarding buddy/mentor role currently in place) to help with the sort of issues that come up when you first join an org and are trying to get everything set up and properly configured

👍 on this idea. I think it's really useful to have an explicit contact that's been at the org for a while to bombard with questions, but might also be helpful to have a peer new hire buddy to help with onboarding, ask other newbie questions to, etc. FWIW, when people start at roughly the same time, I think that happens informally and somewhat naturally - with that said, never hurts to be explicit.

The #onboarding-eng 🔒Slack channel has been a great resource for me this week.

 I'm not opposed to it, but wonder if it could lead to some overlap with mentor duties that make it unclear who is responsible for what.

Ok, after chatting with @orta on this one, I'm proposing to resolve it with this PR:
https://github.com/artsy/README/pull/115
It seems like to a certain degree new hire buddies is happening organically, so I've just added a mention of the slack channel and then a nudge to have the new hire wave at the person most recently onboarded.
Thanks for all the feedback!! ❤


Remove svelte as peerDependency

fixes #4409

That was added not without a reason I guess https://github.com/nolimits4web/swiper/pull/4351 Maybe @Filyus knows what happens?

Humm strange... why not just a peerDep in swiper svelte? And how could be 2 instances of svelte if swiper does not add another?

It was the simplest solution to get Snowpack to work, I guess peerDependency can be moved inside swiper/svelte. In theory, you can also completely remove it, but then you have to fix Snowpack, which is probably more correct but more difficult.

Let's merge it for now. I will look into Snowpack/Svelte issue today


Upgrade packages

Blind execution of npm upgrade. This still needs to be reviewed.
I am trying to get rid out from console warnings about caniuse-lite and unfetch.js

package.json looks okay to me. @Joozty what do you think?

Also, the project builds and runs. I wonder if I need to check anything else 😄

Wow! This looks nice 😀


Guide for SDK maintainers

Create a markdown document in the sdk/ directory with instructions on creating an SDK, which is also useful for existing maintainers to keep consistency with the other SDKs.

Added in https://github.com/dagger/dagger/pull/5098


MGDOBR-721: ManagedResource: Validate and trim name

https://issues.redhat.com/browse/MGDOBR-721
PR owner checklist
Please make sure that your PR meets the following requirements:

[x] Your code is properly formatted according to this configuration.
Run mvn clean verify -DskipTests so that imports are correctly set.
[x] Your commit messages are clear and reference the JIRA issue e.g: "[MGDOBR-1] - $clear_explanation_of_what_you_did"
[x] The layers in the kustomize folder have been updated accordingly.
[x] All new functionality is tested
[x] Pull Request title is properly formatted: MGDOBR-XYZ Subject
[x] Pull Request contains link to the JIRA issue
[x] Pull Request contains link to any dependent or related Pull Request
[x] Pull Request contains description of the issue
[x] Pull Request does not include fixes for issues other than the main ticket

PR reviewer(s) checklist

[ ] Check if code and Github action workflows have been modified and if the modification is safe
[ ] If the modification is safe, add the safe to test label

How to trigger pipelines and use the bots:

Run the end to end pipeline
Annotate the pull request with the label: safe to test. If you want to run the pipeline again, remove and add it again.

Rebase the pull request
Comment with: /rebase.

Deploy BOT
Comment with `/deploy <dev,stable> when the PR has been merged to deploy to a target environment.

@sutaakar Please feel to reassign.

Hi @jomarko

...should user find some message about it calling errors api? i.e. "http://localhost:8080/api/v1/errors" ?

Sorry, I do not understand. Are you suggesting an error should be returned?

Two approvals obtained; merging.


Remove return statement support.

For #6333 we worked on improving syntax auto-indent/dedent (e.g. else).  We were able to get dedent mostly right for return (see https://github.com/microsoft/vscode-python/pull/6497#issuecomment-509772399), but weren't able to find a good solution for return followed by a [, (, or { right before the cursor (see examples in #6564).  We've determined that it would be better to remove dedent support for return entirely than to have some minor cases where it does the wrong thing.
Later, we will work on sorting out indent/dedent for returnin #6564.

Validated ✅


[3.0] - Ideas

Please, add some AI facility such as: neural networks, behaviour trees. These should not be full implementations, but allow these to be implemented in scripting with ease.
Also, I don't know if terrains are still used today, but it would be cool to see them in.
Finally, physics should be enhanced with fluids and soft bodies.
Please, share your toughts.

3.0 is too close, maybe 3.1...
AI, there are infinite ways to implement in Godot, more now with DLScript because we can implement existing external libraries that may be more efficient.
Terrains, yes, there are some terrain tools I would love to see in the core, I will like to see GridMap at the level of TileMap too.
I believe soft bodies are planned for later and fluid physics will be nice to see, at least something like LiquidFun.

For 3.0 we need a official way to centralize tutorials and tutorial requests, maybe linked to the asset lib or internal help...

@berserk82 BTW, with that issue title, you are basically asking to get it closed...
Would you mind to:

Search existing issues (I'm absolutely sure there are some for softbodies and terrains) for the requests you have, and comment/:+1: them,
Remove those point that were covered by them for this issue,
Leave only one point in here, and
Open other issues for the leftover points?

That way, we can keep the discussion focused, and also, give developers to just hit "Close issue" after a  day's worth of hard work, sit back, and relax.
P.S. Please check "I have a great idea that will make Godot better, What do you think?" in the well-hidden FAQ too, since it is probable that some parts of this issue would fall under it.

I think @bojidar-bg comment was detailed enough, I will archive this issue. However you might continue loose discussion here if you want :)

As a sidenote: Godot devs are totally aware of the fact that these things exists in some other engines and they totally will implement them for sure at some point of time. Furthermore some of those are already considered for 3.1 :)

Thanks for your 2 cents.
I just wanted to know exactly what I knew.

May I know which of these are considered for 3.1?


[BUG] Allow string constructor to take 0 arguments

Here taking no arguments should be allowed and the variable should be initialized with an empty string.
Python
>>> s = str()
>>> s
''

Pocketpy main
>>> i = int()   # Supported and default assigned to variable
>>> i
0
>>> f = float() # Supported and default assigned to variable
>>> f
0.0
>>> s = str()
Traceback (most recent call last):
  File "<stdin>", line 1
    s = str()
TypeError: expected 2 arguments, got 1

C:\Users\kunni\pocketpy>git diff
diff --git a/src/pocketpy.cpp b/src/pocketpy.cpp
index 790189a1..999d01c2 100644
--- a/src/pocketpy.cpp
+++ b/src/pocketpy.cpp
@@ -496,7 +496,12 @@ void init_builtins(VM* _vm) {
     });

     // tp_str
-    _vm->bind_constructor<2>(_vm->_t(VM::tp_str), PK_LAMBDA(vm->py_str(args[1])));
+
+    _vm->bind_constructor<-1>(_vm->_t(VM::tp_str), [](VM* vm, ArgsView args) {
+        if(args.size() == 1) return VAR(Str());
+        return vm->py_str(args[1]);
+    });

This diff fixes it and passes all other tests in place. Let me know if I can send in a quick fix.

Go ahead, please.


Tab accessibility on web (expo-web)

Hi,
I am using expo, specifically expo for web, to build a login form. I have some pretty straightforward code, 2 inputs, and a submit button. I have some icons being used in the accessoryLeft on both inputs.
When I attempt to tab between the two inputs in Chrome, it takes 11 keystrokes on the tab key to move from one input to the next and you can see the outline move through the icon in the accessoryLeft in what appears to be layers of the svg elements.
I don't see any way to control this but it seems like icons and other elements should probably get the tabIndex="-1" in the resulting html so as to avoid this kind of issue.
Is there a way to specific what the next focused element is when tabbing out of an input?
The regular react-native solution of handling the onSubmitEditing and focusing into the next field via ref does not work unless you hit the enter key in the browser.
I am using the latest version of UI Kitten and expo SDK 39.

Ok, for anyone else hitting this issue, this is how I worked around it:
<Input
onKeyPress={e => {
   if (
      Platform.OS === 'web' &&
      e.nativeEvent.key === 'Tab'
   ) {
      e.preventDefault();
      passwordRef.current.focus();
   }
}}
/>

passwordRef is the ref to the password input, and it's important to include preventDefault() or it will add an extra tab to your outcome.

Also experiencing this when using accessoryRight. The workaround above is good in some circumstances, but managing refs for each input can be messy with repeated use of the same component.
I would say that https://github.com/akveo/react-native-ui-kitten/issues/1422 is somewhat related as it deals with focus state on other components around the input.


Extensions

What would you think about some useful extensions right inside the lib? For exampe, User.IsSameAs(User u) or User.FullName{ get => $"{this.FirstName} {this.LastName}".Trim() }

Sounds good. we could put the extension methods in Telegram.Bot.Types namespace for convenience.
Here are the suggestions I had:
bool isSameUser = someUser.IdEquals(otherUser);
bool hasSameSender = someUpdate.IsFromSameUserId(otherUser); // for messages, channel posts, callback query,...
int? msgId = someUpdate.TryGetMessageId(); // for messages, channel posts, callback query,...
long? chatId = someUpdate.TryGetChatId(); // for messages, channel posts, callback query,...
string queryIdOrNull = someUpdate.TryGetCallbackQueryId();

I hope users of the project participate here and share what they need to be added here.

I'd also suggest EditMessageTextAsync(Message msg, string text) and similar ;)

Hmm. That could be handy. As an extension method, that could be
await someCallbackQueryUpdate.CallbackQuery.Message
   .EditMessageTextAsync("Edited message");

Closed due to inactivity. Due to recent decision this can be implemented in extension package rather than in the library. Library stays as close to pure API as possible.


Encode images

Let's make this a python package. That'll be 3 separate components then:

UI
Django Service
Face scripts


Fix deprecation warnings due to invalid escape sequences.

Fixes #84

I was intending to address this issue, but it got lost in the shuffle.
Thanks for fixing it!


(B)Create API end points[1pt]

Create the file structure for the Api routings

https://github.com/DuaneDave/Bike-rental-api/pull/4


wheres the extension UI?

If a user install a chrome exension they should see the UI ... but this and other JSON viewers just points to github....how come its so user unfriendly? Where is the UI for the extension?

there is no UI... It used to just render json when you opened an URL with json content. but it seems to be broken in the latest chrome version

82


Bump bundled JRE to 15

Bumps everything to OpenJDK 15.
Docker image switched to Azul OpenJDK, because they already have 15 and also because they don't require glibc on Alpine, possibly leading to even smaller images.
Javadoc no longer fails on warnings, as 15 has some unwelcome changes there.
Finally, CI now caches Maven artifacts for build speedup.

Single mutation coverage failure appears to be a flake.


http request keeps the response order and add unit tests for that

Summary (What and Why)

asyncio.wait can be out of order if one request timeout. change to asyncio.gather to keep the order of request
Test Plan

Implemented unittest
Release Plan

GitHub Issue:
https://github.com/tecton-ai/tecton-http-client-python/issues/XXX

nit, prefer using semantic-commit style commit headings. This will make automating releases easier in the future.
https://www.conventionalcommits.org/en/v1.0.0/


Crash when calling performUpdates with workingRangeDelegate

New issue checklist

[x] I have reviewed the README and documentation
[x] I have searched existing issues and this is not a duplicate

General information

IGListKit version: 1.0 Master Thread
iOS version(s): 10
CocoaPods/Carthage version: 1.1
Xcode version: 8
Devices/Simulators affected: All
Reproducible in the demo project? (Yes/No): Yes
Related issues:

I have my workingRangeDelegate set, with a range of 2.
If you adjust the DemosViewController so that you:

Initially return zero items to your Adapter Data Source (return a property demos2 where it's defined as var demos2: [DemoItem] = [])
Try to access a cell in your sectionControllerWillEnterWorkingRange function by using the following code:

if let cell = self.collectionContext?.cellForItem(at: 0, sectionController: self) as? LabelCell {
    print("hey!")
}

Change the adapter data source to have some items upon viewWillAppear (imitating loading some data from the server). (set demos2 = demos).
Then run performUpdates in viewWillAppear straight after that.

It will crash with the following exception:
'NSInternalInconsistencyException', reason: 'request for number of items before section 2 when there are only 0 sections in the collection view'
I initially encountered it within my own project but found it to be reproducible using the steps above!

@yusuftor why are you accessing the cell in -sectionControllerWillEnterWorkingRange:? If you just entered the range you shouldn't have a cell yet. If you remove that line does it still repro?

Ok I'm a bit confused then.
In the WorkingRangeSectionController in the example project, it accesses the cell in -sectionControllerWillEnterWorkingRange: using the same code as I just tried to use. It does that to set the image in the cell, and I'm also trying to set something within the cell. But if you say that you shouldn't have access, why does the example project access it?
And no it doesn't crash if I remove that line.
What happens if you're scrolling, download a large image that takes a while in -sectionControllerWillEnterWorkingRange: and set a property downloadedImage with the new image. As that image is being downloaded -cellForItem: might be called, but at that moment downloadedImage hasn't been set. Eventually downloadedImage would be set, but if the cell isn't accessible in -sectionControllerWillEnterWorkingRange: the cell image would be nil until -cellForItem: is called again.
I think this is the problem that has happened to me when I have small section controllers (150px in height) with a small workingRange. I've had to increase the working range just so that -sectionControllerWillEnterWorkingRange: can set the downloadedImage property before -cellForItem: is called.

@yusuftor the idea in that project is that if in the time since you started downloading, a cell exists, then assign the image data to that cell's image view. (your fourth paragraph is spot on)
It does seem like there is a real crash/issue in here though. I'll get the sample into the config that you outlined and see what is going on. Either a UICollectionView annoyance that we can avoid by returning nil, or a bug in the working range stuff.
Thanks for all the detail!

@rnystrom That makes sense. That's why I've been trying to access the cell to update it, but the occasional crash issue has prevented me from doing that.
No worries, if you need any extra info from me let me know!

I just encountered this same error again. I decided to investigate and the place where it crashes is in IGListAdapter.m in this function:
- (__kindof UICollectionViewCell *)cellForItemAtIndex:(NSInteger)index
                                    sectionController:(IGListSectionController <IGListSectionType> *)sectionController {
    IGAssertMainThread();
    IGParameterAssert(sectionController != nil);

    // if this is accessed while a cell is being dequeued, just return nil
    if (_isDequeuingCell) {
        return nil;
    }
    NSIndexPath *indexPath = [self indexPathForSectionController:sectionController index:index];
    // prevent querying the collection view if it isn't fully reloaded yet for the current data set
    if (indexPath != nil
        && indexPath.section < [self.collectionView numberOfSections]) {
        // only return a cell if it belongs to the section controller
        // this association is created in -collectionView:cellForItemAtIndexPath:
        UICollectionViewCell *cell = [self.collectionView cellForItemAtIndexPath:indexPath];
        if ([self sectionControllerForCell:cell] == sectionController) {
            return cell;
        }
    }
    return nil;
}

Specifically on the line: UICollectionViewCell *cell = [self.collectionView cellForItemAtIndexPath:indexPath];. I tried printing out the number of sections in the collectionView and it correctly prints out the number that should be in the collectionView... Yet at the same time the error is saying that there are 0 items!
I tried to trace -performUpdates: and that goes all the way through until it reaches the line [collectionView performBatchUpdates:updateBlock completion:completionBlock]; in  -performBatchUpdatesWithCollectionView: inside IGListAdapterUpdate.m. It does everything in the updateBlock, but then the exception occurs so the completion block never happens. Perhaps what's happening is that the workingRangeDelegate is being called before the collectionView has updated properly?
Hopefully this extra info can help you! If I find any more info I'll update this thread...

@yusuftor -- Can you provide repro steps or a stack trace?

@jessesquires Repro steps are the same as in my first post. The stack trace is :
2016-11-14 10:44:03.223 IGListKitExamples[78257:9572078] *** Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'request for number of items before section 2 when there are only 0 sections in the collection view'
*** First throw call stack:
(
	0   CoreFoundation                      0x000000010cf3434b __exceptionPreprocess + 171
	1   libobjc.A.dylib                     0x000000010c99521e objc_exception_throw + 48
	2   CoreFoundation                      0x000000010cf38442 +[NSException raise:format:arguments:] + 98
	3   Foundation                          0x000000010c52be4d -[NSAssertionHandler handleFailureInMethod:object:file:lineNumber:description:] + 195
	4   UIKit                               0x000000010e66f661 -[UICollectionViewData numberOfItemsBeforeSection:] + 231
	5   UIKit                               0x000000010e66f70d -[UICollectionViewData globalIndexForItemAtIndexPath:] + 52
	6   UIKit                               0x000000010e61b542 -[UICollectionView _cellForItemAtIndexPath:includePrefetchedCells:] + 118
	7   IGListKit                           0x000000010c3d34d4 -[IGListAdapter cellForItemAtIndex:sectionController:] + 1236
	8   IGListKitExamples                   0x000000010c24b75b _TFC17IGListKitExamples21DemoSectionController11listAdapterfTCSo13IGListAdapter38sectionControllerWillEnterWorkingRangeCSo23IGListSectionController_T_ + 155
	9   IGListKitExamples                   0x000000010c24b98f _TToFC17IGListKitExamples21DemoSectionController11listAdapterfTCSo13IGListAdapter38sectionControllerWillEnterWorkingRangeCSo23IGListSectionController_T_ + 79
	10  IGListKit                           0x000000010c408aeb -[IGListWorkingRangeHandler updateWorkingRangesWithListAdapter:] + 10699
	11  IGListKit                           0x000000010c405a8d -[IGListWorkingRangeHandler willDisplayItemAtIndexPath:forListAdapter:] + 6189
	12  IGListKit                           0x000000010c3d1a54 -[IGListAdapter collectionView:willDisplayCell:forItemAtIndexPath:] + 660
	13  UIKit                               0x000000010e60a116 -[UICollectionView _createPreparedCellForItemAtIndexPath:withLayoutAttributes:applyAttributes:isFocused:notify:] + 2409
	14  UIKit                               0x000000010e6097a7 -[UICollectionView _createPreparedCellForItemAtIndexPath:withLayoutAttributes:applyAttributes:] + 35
	15  UIKit                               0x000000010e628c28 __51-[UICollectionView _viewAnimationsForCurrentUpdate]_block_invoke.1912 + 597
	16  UIKit                               0x000000010e625753 -[UICollectionView _viewAnimationsForCurrentUpdate] + 3529
	17  UIKit                               0x000000010e62b7e1 __71-[UICollectionView _updateWithItems:tentativelyForReordering:animator:]_block_invoke.1983 + 197
	18  UIKit                               0x000000010dd806e6 +[UIView(Animation) performWithoutAnimation:] + 90
	19  UIKit                               0x000000010e62a337 -[UICollectionView _updateWithItems:tentativelyForReordering:animator:] + 3942
	20  UIKit                               0x000000010e624369 -[UICollectionView _endItemAnimationsWithInvalidationContext:tentativelyForReordering:animator:] + 17765
	21  UIKit                               0x000000010e62cdba -[UICollectionView _endUpdatesWithInvalidationContext:tentativelyForReordering:animator:] + 71
	22  UIKit                               0x000000010e62d0fc -[UICollectionView _performBatchUpdates:completion:invalidationContext:tentativelyForReordering:animator:] + 432
	23  UIKit                               0x000000010e62cf29 -[UICollectionView _performBatchUpdates:completion:invalidationContext:tentativelyForReordering:] + 91
	24  UIKit                               0x000000010e62ceab -[UICollectionView _performBatchUpdates:completion:invalidationContext:] + 74
	25  UIKit                               0x000000010e62ce00 -[UICollectionView performBatchUpdates:completion:] + 53
	26  IGListKit                           0x000000010c3de5fc -[IGListAdapterUpdater performBatchUpdatesWithCollectionView:] + 2956
	27  IGListKit                           0x000000010c3e1535 __54-[IGListAdapterUpdater queueUpdateWithCollectionView:]_block_invoke + 357
	28  libdispatch.dylib                   0x000000010ff6d980 _dispatch_call_block_and_release + 12
	29  libdispatch.dylib                   0x000000010ff970cd _dispatch_client_callout + 8
	30  libdispatch.dylib                   0x000000010ff778d6 _dispatch_main_queue_callback_4CF + 406
	31  CoreFoundation                      0x000000010cef84f9 __CFRUNLOOP_IS_SERVICING_THE_MAIN_DISPATCH_QUEUE__ + 9
	32  CoreFoundation                      0x000000010cebdf8d __CFRunLoopRun + 2205
	33  CoreFoundation                      0x000000010cebd494 CFRunLoopRunSpecific + 420
	34  GraphicsServices                    0x00000001111b5a6f GSEventRunModal + 161
	35  UIKit                               0x000000010dcc7964 UIApplicationMain + 159
	36  IGListKitExamples                   0x000000010c25df7f main + 111
	37  libdyld.dylib                       0x000000010ffe368d start + 1
)
libc++abi.dylib: terminating with uncaught exception of type NSException

Would it be possible to look into this bug before v2.0 is released? It seems like a fairly major bug to me as until this is fixed I can't reliably use the working range in production. No worries if not though as I know you guys are working really hard on other issues. As a side note, IGListKit has made the scrolling within my app waaay smoother than before, so thanks for that!

@yusuftor I'm on this today, ideally I should be able to make a repro unit test and fix the crash. Very bizarre. Agree tho this should be fixed before 2.0.

Haven't fixed it, but do have my sample project in a setup where this crash occurs.


grain/detail is retained on every other frame

simple settings, but multiple quant and speed settings
rav1e.exe --metrics --threads 8 --speed 6 --quantizer 80 fightclubsnippet.y4m -o fightclubsnippet-rav1e-q80-speed6.ivf
also when i tried quant 72, 64, 56, and speeds down to 1
every other frame seems to have detail: https://cdn.discordapp.com/attachments/587033245061873759/736876069537316864/mpv-shot0002.png
and every other frame seems to wipe it out:
https://cdn.discordapp.com/attachments/587033245061873759/736876046753988634/mpv-shot0001.png
source: http://www.mediafire.com/file/55j542f7ws24ygo/fightclubsnippet.mkv/file
encodes: http://www.mediafire.com/file/q10922pyrepzhop/fightclubsnippet-rav1e-q72-speed6.ivf/file
http://www.mediafire.com/file/dn55dfvvlfc1gig/fightclubsnippet-rav1e-q64-speed6.ivf/file
http://www.mediafire.com/file/f5jl6ugeamjdu3h/fightclubsnippet-rav1e-q64-speed1.ivf/file
http://www.mediafire.com/file/kxzgzza2x323kby/fightclubsnippet-rav1e-q56-speed6.ivf/file

I can't find a related open issue, but I remember this having been reported before. Will look into it when I have some time.

hugely appreciated! i've been bashing my head into a wall trying to get aomenc to retain grain at reasonable bitrates and this is my first hope so far!

fan comment
sorry really appreciate everyone's av1 work!
hugely hoping we get a jump on vvc/evc and gain popularity to make it as popular
or even more than the terrible mpeg (for its licensing costs)

Does it reproduce with --low-latency true?

At first I thought this was something about key frames vs inter frames, but there really is significant smoothing every other frame.
@tdaede --low-latency makes the inter frame avg QP correct (same as --quantizer) while the default behavior doesn't.
I've uploaded my samples and logs here: https://mega.nz/folder/xMsHXTqD#4BYnbCiUVHM5-z4Fy3_Rng
I used the CLI from OP, these are the stats:
Default: Inter frame:          91 | avg QP: 119.00 | avg size:  182092 B
Low latency: Inter frame:          93 | avg QP:  80.00 | avg size:  302079 B
Interestingly, the metrics don't change much but the inter frame avg size is much smaller.

--low-latency seemed to improve it (tho resulting in more than a 100% size increase), but it still has less detail/less prominent grain every other frame https://www.mediafire.com/file/wevkq9dufjgfwdp/fightclubsnippet-rav1e033-q72-speed6-lowlatency.ivf/file

--quantizer does not guarantee the frame average QP, it only guarantees the "base pyramid" frame QP. By default depth 2 pyramid coding is used (3 levels) so average QP will be higher. Pyramid coding is also better for compression.
Thanks, I also see it on the file you provided.

FWIW nothing obvious on the analyzer. Either incorrect reference frames are being used, or somehow lambda or distortion scale is getting set wrong every other frame.

Another Sample:

Frames

Encoding settings are
` --tiles 16 --quantizer 90 -s 5 --matrix BT2020NCL --trasnfer BT20202_10Bit `
Video:
vid.tar.gz

Is this a regression for 0.5? I'm not sure whether to mark it as a blocker.

I was looking into this a bit, and there's only one particular pattern that I found when going through the analyzer, which is present even in the low latency encode.
The even numbered frames are more likely to choose larger, DCT based blocks.
However, I haven't yet determined if this is the cause of the issue or a result of it.

 Fix rav1e destroying grain in every other frame

Furthermore, this is an instance where limited SGR in CPU-6 improves subjective quality a bit in some blocks.
I also encounter the issue with even numbered frames choosing bigger partitions.
Currently investigating that last one.

It looks like there was a bit of this effect present in the source, which let to some red herrings in investigating.
The end result we came to is that this is a natural effect of using frame pyramids, and all encoders are prone to it. aomenc definitely exhibits the same effect.
I do have a couple of ideas I want to try to see if I can improve this, however, so I'll leave this ticket open but remove the "bug" tag since it's functioning "as expected". (But we want it to function better than expected.)

For posterity and the ability to reproduce this using publicly available source files, I encoded the 1080p files from the objective-2-slow test set to a VMAF target of 93 using ab-av1. This alternating amount of grain seems to be easiest to notice in the bottom third of the frame in ducks_take_off_1080p50_60f when playing back frame-by-frame.

old_town_cross_1080p50_60f is another file to reproduce this with. The alternating grain amount is visible against the gray clouds, sometimes more, sometimes less.

x264 also exhibits this behavior:


[CI] mark failed cases as xfail caused by llvm error

to unblock CI first, will raise a ticket for the failure to complier team

file ticket to trac the build errors with compiler team
https://ontrack-internal.amd.com/browse/SWDEV-454948


#1375 skip ill-quoted CSV lines

Description
When ran into an ill-quoted CSV line, like 10,"abc"d,xy, skip the line.
Currently, the CSV parser stops and the result is up to the previous line.
Please refer to the related issue below. There's a sample file attached and detailed bug scenario.
Related Issue : https://github.com/metatron-app/metatron-discovery/issues/1375
How Has This Been Tested?
Ran locally.
Types of changes

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[ ] My code follows the code style of this project. it will be added soon
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[ ] I have read the CONTRIBUTING document. it will be added soon
[ ] I have added tests to cover my changes.
[ ] All new and existing tests passed.

Additional Context
I'll be appreciated if you test many cases that use CSV, JSON files: import, wrangle and snapshot each, because I've change rather many lines that handles those file formats.

run build
deploy to 1


updated catalog base image with 4.16

Description
Updated the base image with 4.than using latest. We want to  build from SCRATCH , but currently I see a problem with running in 4.15 OCP clusters . We will fix using https://issues.redhat.com/browse/OLS-863
Type of change

[ ] Refactor
[ ] New feature
[x] Bug fix
[ ] CVE fix
[ ] Optimization
[ ] Documentation Update
[ ] Configuration Update
[ ] Bump-up dependent library

Related Tickets & Documents

Related Issue #
Closes #

Checklist before requesting a review

[ ] I have performed a self-review of my code.
[ ] PR has passed all pre-merge test jobs.
[ ] If it is a core feature, I have added thorough tests.

Testing

Please provide detailed steps to perform tests related to this code change.
How were the fix/results from this change verified? Please provide relevant screenshots or results.

/assign @raptorsun

/test bundle-e2e-4-16

/lgtm
/approve


Q: Is WebAuthn planned to be implemented by aspnet.security?

Hello,
The API should be available in Chrome and Firefox May release.
Starting to look at it and be ready for it could be a good idea?
webauthn specs

FIDO2/WebAuthn is definitely the future and I'd personally love to see it adopted everywhere.
That said, it essentially dictates how things happen client-side, not server-side. The specification doesn't say how you're supposed to retrieve the challenge from the server or how you send the public key when creating an account or adding a new key to an existing account: all this stuff is necessarily implementation-specific (which gives you total flexibility).
As such, I'm not sure aspnet/Security will ever have anything related to FIDO, just like there's nothing for "password authentication" in this repo (on the other hand, membership frameworks like ASP.NET Core Identity will have to offer ways to store the public keys in the DB).

About validating the client/user signature, I thought it would be considered as an external authentication provider.
But you are probably right about adding this as a whole feature into Identity.

Should we close the issue here then? A new Identity issue was just logged at https://github.com/aspnet/Identity/issues/1747.

I think it belongs in both, because it's not just an identity issue.


Not all APIs have rate limits

When working with the integrations API, I noticed that not all APIs have rate limits, which causes gidgethub to fail hard:
Traceback (most recent call last):
  File "/Users/dstufft/.virtualenvs/browntruck/lib/python3.6/site-packages/twisted/internet/defer.py", line 459, in callback
    self._startRunCallbacks(result)
  File "/Users/dstufft/.virtualenvs/browntruck/lib/python3.6/site-packages/twisted/internet/defer.py", line 567, in _startRunCallbacks
    self._runCallbacks()
  File "/Users/dstufft/.virtualenvs/browntruck/lib/python3.6/site-packages/twisted/internet/defer.py", line 653, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "/Users/dstufft/.virtualenvs/browntruck/lib/python3.6/site-packages/twisted/internet/defer.py", line 1357, in gotResult
    _inlineCallbacks(r, g, deferred)
--- <exception caught here> ---
  File "/Users/dstufft/.virtualenvs/browntruck/lib/python3.6/site-packages/twisted/internet/defer.py", line 1301, in _inlineCallbacks
    result = g.send(result)
  File "t.py", line 46, in inner
    async for installation in gh.getiter("/integration/installations"):
  File "/Users/dstufft/.virtualenvs/browntruck/src/gidgethub/gidgethub/abc.py", line 64, in getiter
    authorization)
  File "/Users/dstufft/.virtualenvs/browntruck/src/gidgethub/gidgethub/abc.py", line 48, in _make_request
    data, self.rate_limit, more = sansio.decipher_response(*response)
  File "/Users/dstufft/.virtualenvs/browntruck/src/gidgethub/gidgethub/sansio.py", line 267, in decipher_response
    return data, RateLimit.from_http(headers), _next_link(headers.get("link"))
  File "/Users/dstufft/.virtualenvs/browntruck/src/gidgethub/gidgethub/sansio.py", line 211, in from_http
    limit = int(headers["x-ratelimit-limit"])
builtins.KeyError: 'x-ratelimit-limit'

Great, so their docs are lying that "any API request" will have the rate limit headers.

@brettcannon it's possible it is a bug that this API doesn't have it, the entire integrations API is early preview right now.

I am not seeing ratelimits on github api requests on my github enterprise account. There needs to a be a safe way to support this change.

Probably if you update
https://github.com/brettcannon/gidgethub/blob/4d5819cf50f3ad896713364d5e749bea466b9f05/gidgethub/sansio.py#L233-L234
to catch the KeyError and return None instead that should do it (plus tests and docs, of course). If anyone wants to provide a PR I will happily review it.

[ @eliot1019 edited your message ⬆ to come off as more 😃 and less 😠 ]


Docs: Document new v1beta1 webhook-config standard

Please document the newly created standard for v1beta1 version of webhook-config

Part of https://github.com/keptn/keptn.github.io/pull/1329


Crash when clicking on Golem Press in Clockwork Minds (TC-B17)

When trying to open the tab for visualization of the multiblock structure for Golem Press, the game crashes. Tried 3 times in a row. There is no problem with the visualization of the Infernal furnace tho.
Image of what I mean: https://imgur.com/a/AUbc3Lo
Do not know if it helps, attaching crash report.
crash-2018-07-07_20.44.18-client.txt

Looks like a Optifine crash to me, have you tried without it?

Tried it now and yeah... Optifine was the problem. Thanks


Fix build error on win32/mingw

Also requires https://github.com/an-tao/trantor/pull/270 to fully build.


Idea: Modify Email Link to "Find your games"

idea: instead of it emailing me the link to my game, it emails me the link to find “your games” (ie my games). that way if im confused ah when i go to the school to work on sprig, i can find the email that says “link to your sprig games”

Is this still the case. Can you include a screenshot of the current email?

What I get when I save a game not logged in:

What I get when I log in:


Update ja.json

Fixed some translations in FullBodyCalibrator.

Thank you for the fixes!

Thank you for the fixes!


Fixed bugs related to INCLUDE_CONTAINERS

Hey I realize I accidentally implemented "exclude containers" and had a syntax error in https://github.com/looplab/logspout-logstash/pull/64.  My bad!  Updated code here.

For anyone looking for a container with this INCLUDE_CONTAINERS supported, I built one at vertoforce /logspout-logstash

Thanks!


The unchecked subscription under another account shows after running 'Search From Here' for the subscription under one account

Storage Explorer Version: 1.19.1
Build Number: 20210423.2
Branch: hotfix/1.19.1
Platform/OS: Windows 10/ Linux Ubuntu 16.04/ MacOS Catalina
Architecture: ia32/x64
Regression From: Previous release (1.18.1)
Steps to Reproduce

Sign in two Azure accounts (A and B) that have the same subscription (like: 'subscription1').
Check  'subscription1' for the two accounts -> Click 'Open Explorer'.
Go to 'ACCOUNT MANAGEMENT' panel -> Uncheck 'subscriptioin1' for account A -> Click 'Open Explorer'.
Right click 'subscription1' -> Click 'Search From Here'.
Check whether there is only one 'subscription1'

Expected Experience
There is only one 'subscription1'.

Actual Experience
There are two 'subscription1'.

Hi Matthew,
This issue also reproduces following the below steps:

Sign in two Azure accounts that have the same subscription (like: 'subscription1').
Check 'subscription1' for the two accounts -> Click 'Open Explorer'.
Select one of the subscription1 -> Right click it -> Click ‘Search From Here’ -> Observe the focused item -> Reset the search box.
Go to 'ACCOUNT MANAGEMENT' panel -> Uncheck 'subscriptioin1' for the account which was focused in Step3 -> Click 'Open Explorer'.
Right click existing 'subscription1' -> Click 'Search From Here'.
There are two ‘subscription1’ in the tree view.

@v-xianya the behavior you are now seeing (as specified in this comment), that is not a regression? Correct? I tested on 1.18.1 and I see the subscription two times in the tree view, once for each of the Azure accounts which have access to it. Let me know if you see the same thing.

@MRayermannMSFT It is a regression. I verified on 1.18.1 following the steps in comment. Not reproduce.

@v-xianya ah I misread the instructions! I'll have a fix for this shortly.

@v-xianya I've checked in a fix for this. :)


Auto-update Quest

The tool, once installed, should offer to update itself directly and thus avoid going through the install process again. Electron Forge proposes something quite simple, if the tool is public (which is the case): https://www.electronforge.io/advanced/auto-update

For Debian and Ubuntu, the best option is launchpad : https://help.launchpad.net/Packaging/PPA/Uploading


rpc: incompatibilities

I've tested the (ignored) integration tests for rpc against a tendermint 0.32.7 version and the following tests fail:
abci_info and genesis

genesis fails with: called Result::unwrap() on an Err value: Error { code: ParseError, message: "Parse error. Invalid JSON", data: Some("invalid digit found in string at line 11 column 26") }
abci_info with: called Result::unwrap() on an Err value: Error { code: ParseError, message: "Parse error. Invalid JSON", data: Some("missing field last_block_height at line 9 column 5") }

https://github.com/interchainio/tendermint-rs/pull/85#issuecomment-564584641
I've also noted that example responses on https://docs.tendermint.com/master/rpc/#/ have a numeric id (every JSON rpc request comes with such an id):
{
  "jsonrpc": "2.0",
  "id": 0,
  "result": {

while on my local 0.32.7 returns a string based id  "id": "",.
We should make sure to support both as the spec says:

An identifier established by the Client that MUST contain a String, Number, or NULL value if included.

https://www.jsonrpc.org/specification#request_object
Compatibility with some tendermint version is also relevant in the context of a light client making requests via rpc.

@Liamsi can we figure out a way to run the currently #[ignore]d integration tests in CI? I think that'd be the ideal way to spot incompatibilities

Yes, that would be ideal! A naive approach that should work: grab a docker image (with the latest / a pinned tendermint version), pull in tendermint-rs source, spin up a single tendermint node and run the ignored tests against this node (via cargo test -- --ignored). Wasn't there a script that done exactly this in the past?
It makes sense to sync with @greg-szabo and @thanethomson on this too. Not likely that I'll work on this this year but I I'll open a PR with a fix for the regressions.

https://github.com/interchainio/tendermint-rs/pull/169 surfaced new incompatibilities with the latest tendermint version.

As mentioned above the spec defines:

An identifier established by the Client that MUST contain a String, Number, or NULL value if included.

https://www.jsonrpc.org/specification#request_object
I'll go on and change the Id to be a enum to match above behaviour instead of currently a String only id: https://github.com/interchainio/tendermint-rs/blob/471ac8faa79c3072a220e09228bffae157fab2f0/tendermint/src/rpc/id.rs#L8

I think this has been fixed with #183. The resolution (for 0.32) was to introduce the Rust version of omitempty for the structures.
Oh, I see the id thing is mentioned here too. That's not fixed yet. But CI works, so I'm guessing we're not testing for the id thingy?

so I'm guessing we're not testing for the id thingy

Yes, the JSON-only tests still use a string based ID (see all files here). When I run a tendermint instance locally it returns an integer though (ad do all examples in https://docs.tendermint.com/master/rpc/). Interestingly, gaia seems to return a string based ID: https://rpc.cosmos.network/commit?height=1
That said, we might not explicitly test for the ID thingy but it's effectively parsed on each test / rpc call.

@Liamsi Can we close this issue? Integration tests run and although they are not mandatory, they are a good indicator of issues.

@Liamsi Can we close this issue? Integration tests run and although they are not mandatory, they are a good indicator of issues.

As far as I can see all known issues are fixed. I have another comment on how to improve the integration test situation but I'll open a separate issue for this or use the #120 to track this.

Closing


Added support for Mono's implementation of SQLite.

Mono ships its own implementation of SQLite: Mono.Data.Sqlite
Note the casing: Sqlite instead of SQLite.  This is the same across all the classes and functions, hence a pile of #if __MonoCS__ tests.

I don't recall needing to use a special version of SQLite in the past. I thought the DLL included both native and portable code. But I can revisit and check it.  I'd hate to see the need for all the case maping #if changes if we don't absolutely need it.

Agreed.  What prompted this was that while the code compiled fine using System.Data.SQLite, it didn't have a valid SQLite DLL and so crashed at runtime.  Since Mono includes its own, I figured it was easier to just use that than to try and find and require people to go get the relevant DLL.

Well we can check the DLL into the project if needed.  I know I fixed this once before I just dont remember how I did it.   What you will need to do is have the system .so installed on the linux side so the DLL can find and call into that.
We can always back this out when we have a solution for the portable DLL.  I just wanted to say I'd worked this out once before I just cant remember what the solution was.  LOL. Need to jog some brain cells and reconstruct what I did.

LOL, that's Ok.  I do remember seeing some scattered #if __MonoCS__ mostlye in Util.cs locking out some Kernel32 calls.
And, apparently, I missed making the changes to Scene.cs a separate pull request: I'd found some more kernel32 calls that needed blocking out.  No implementation, so I can only hope the watchdog at least sorta works, but with this and some upcoming patches I did get a region booted and connected to our local grid.  Can see it on the map - though the maptile isn't rendered because of missing the OpenJPEG DLL or .so. Unable to TP into it, most likely because I'm hacking the connection: I'm attempting to run my region with all the ports tunneled through SSH to the actual grid...  Yeah, it's not happy about that!  But it's running! :D

Cool!  How did you get past the physics initialization?
Interesting... you should be able to get at least the grid services running in Mono in thinking this through.  I'll make some time tonight and give some of that a go.  That should help to flesh out the platform dependencies a bit. I have been building on both Mono/Linux and Xamarin/Mac so it would be nice to address both platforms.

For how I got past the physics implementation, see https://github.com/InWorldz/halcyon/pull/20

During testing of #20, I replicated the core situation  hat stimulated me into creating this patch and got the crash output:
APPLICATION EXCEPTION DETECTED: System.UnhandledExceptionEventArgs

Exception: System.DllNotFoundException: System.Data.SQLite.dll
  at (wrapper managed-to-native) System.Data.SQLite.UnsafeNativeMethods:sqlite3_config (int,System.Data.SQLite.SQLiteLogCallback,intptr)
  at System.Data.SQLite.SQLite3.StaticIsInitialized () [0x00025] in <filename unknown>:0
  at System.Data.SQLite.SQLiteLog.Initialize () [0x00000] in <filename unknown>:0
  at System.Data.SQLite.SQLiteConnection..ctor (System.String connectionString) [0x0000e] in <filename unknown>:0
...

If not the patch I made or a DLL-based fix, then a simple wrapper class could be written that makes everything consistent and internally handles the Mono-vs-System styles.

@mdickson is working on restoring the native components that he believes are missing on Linux to fix the functionality there.

@mdickson How's progress?

I have the openjpeg DLL's generated but the default versions that are released for SQLite are not cross platform.  I may need to rebuild from source to enable platform detection.    I was sidetracked for a while without a windows machine to test on but I have that fixed now.
An alternative is to switch to the Mono SQLite provider for everything since it will also work on Windows.   That would remove the need for the special case code w/ifdefs if we are comfortable using that provider across the board.  Looking for feedback on that.  BTW thats the approach used in OpenSim.

I have a hesitation about switching to the mono implementation since it is completely untested on the InWorldz grid and some pretty critical code currently relies on the existing implementation. Script state saves are one example and we have thousands of state files saved for all different regions across the grid. It is difficult to know what kind of impact (performance, etc) switching to the mono implementation would have.
I'm open for more input, but I'd rather keep what we have rather than rock the boat. Maybe @kf6kjg could assist with the cross platform DLL builds?

http://www.codeproject.com/Articles/821149/Universal-System-Data-Sqlite-binary-for-NET-and-Mo
URL that describes the build process for the DLL's.

Sounds like and adventure.  From the comments it looks like OSX might be an
issue.
It also looks like while he's packaging the dll with the release, he's
relying on the Linux ssytem to have SQLite preinstalled.
I'd have to dig further, but this might mean that Halcyon may need a build
system that can check for and download precompiled binaries from a central
storage location a la the client build process...  What do you think?
On Wed, Oct 7, 2015 at 10:03 AM, mdickson<EMAIL_ADDRESS>wrote:

http://www.codeproject.com/Articles/821149/Universal-System-Data-Sqlite-binary-for-NET-and-Mo
URL that describes the build process for the DLL's.
—
Reply to this email directly or view it on GitHub
https://github.com/InWorldz/halcyon/pull/19#issuecomment-146263283.

Ok, so I got it working on Linux, see https://github.com/kf6kjg/halcyon/tree/fix_mono_sqlite_trial2
However I've been unable to test on Windows 7 building with VS2015: no matter whether it is with my edition of the DLL or with the master branch's edition I get an error about assembly version conflicts on System.Data.SQLite.dll.  Any of you have issues getting a build to run with VS2015 under Win7?

Closing in favor of #48.


DOC: Working example of upload_to_pypi tag customization

It is explained in README that it can be customized but would be nice to have an example on how to drop the leading "v". Some packages tag X.Y.Z instead of vX.Y.Z. Thanks!
https://github.com/OpenAstronomy/github-actions-workflows/blob/ce411ebef92d6048328bee7486f0e2c8de1bece4/.github/workflows/publish_pure_python.yml#L21-L24
cc @tepickering

Some packages tag X.Y.Z instead of vX.Y.Z

more fool them :stuck_out_tongue:

@Cadair , what do you mean? Please clarify.

I was just joking that they should tag with a v ;)

Har har... awkward silence ;)

I haven't tested but probably just setting the option to refs/tags/ instead of refs/tags/v might work?

Like this?
jobs:
  publish:
    uses: OpenAstronomy/github-actions-workflows/.github/workflows/publish_pure_python.yml@v1
    with:
      upload_to_pypi: refs/tags/

I think so but @ConorMacBride should confirm


Fix #118

Refer to this link for build results (access rights to CI server needed):
http://jenkins.guh.guru/job/guh-webinterface-pull-request/78/


Fix header links for #2153

header links weren't taking into account external config
need to do a test on staging behind nginx

Coverage increased (+10.4%) to 77.267% when pulling 35431f2b69d749c4f966f64a20203edb809bbc3e on feature/ga4gh_header_links into 045adcf322282c420f5ea83cd37cfcb344920b90 on develop.


[RFC] core::marker::Freeze in bounds

Rendered

IIRC @joshlf was also asking about exposing Freeze; maybe they can help provide some more motivating examples. Currently there's only really one. (I don't understand the "key of a map" note, and the RFC doesn't explain it in more than 5 words either.)

The main reason for not stabilising Freeze was because it would now add an additional burden for API contracts: adding interior mutability could be considered a breaking change for some APIs, and thus people might want to add "phantom cells" of UnsafeCell<()> inside their types to ensure that they have this option for the future.
I don't think that just stabilising it for bounds affects this concern. The issue isn't the trait itself being exposed in the library, but APIs having to worry about whether they are Freeze or not as an API contract.

I don't think that just stabilising it for bounds affects this concern. The issue isn't the trait itself being exposed in the library, but APIs having to worry about whether they are Freeze or not as an API contract.

A bound is exactly what one is worried about here? If one writes a function fn myfunc(x: impl Freeze), then if I pass a value I got some some other crate to myfunc this will work off that type is Freeze -- meaning it's a breaking change for that crate to add non-Freeze fields.
This is exactly the same as Send and Sync.

IIRC @joshlf was also asking about exposing Freeze; maybe they can help provide some more motivating examples. Currently there's only really one. (I don't understand the "key of a map" note, and the RFC doesn't explain it in more than 5 words either.)

Sure!
If you want to dive deeper, look at uses of Immutable in zerocopy's 0.8 alpha docs.
On zerocopy stable, we provide traits like FromBytes. T: FromBytes guarantees that transmute::<[u8; size_of::<T>()], T>(bytes) is sound regardless of the value of bytes. We also want to support reference transmutations (e.g., &[u8; N] to &T). Reference transmutations are necessarily more restrictive than value transmutations since they must ban interior mutation. For example, &[u8; N] to &Cell<U> is unsound because it would allow the referent bytes to be mutated while a &[u8; N] reference exists referencing the same memory. For this reason, T: FromBytes also requires that T contains no UnsafeCells.
However, this is overly-restrictive. The "no UnsafeCells" requirement is only relevant to reference transmutations, but not to value transmutations. E.g., transmute::<[u8; size_of::<T>()], UnsafeCell<T>>(bytes) can be sound (depending on T), but we can never implement FromBytes for UnsafeCell<T>. Our API supports value transmutations in places like FromBytes::read_from and transmute!. APIs like these are made less powerful because of this restriction.
Another restriction is that some authors want to use FromBytes as a bound to justify their own unsafe code blocks. For example, https://github.com/google/zerocopy/issues/251 was motivated by a user who wanted to derive FromBytes on a type containing an UnsafeCell for this purpose (see the "Motivation" section of that issue; cc @korran).
Our solution in the upcoming zerocopy 0.8 is to add a separate Immutable trait that is semantically very similar to Freeze*. We require T: Immutable where reference transmutations are happening, but don't require it where mutable reference transmutations (e.g. &mut [u8; N] to &mut T) or value transmutations are happening.

* Currently, Immutable bans UnsafeCells recursively, even via indirection. We will likely lift this restriction, at which point Immutable will be semantically identical to Freeze.

Letting people write unsafe impl Freeze for Type is even worse, given its role in the operational semantics, so I think we disagree on what the main reason was. ;)

I should clarify, what I meant here was the adding of stuff like UnsafeCell<()> to make your type !Freeze, so that there's the option to add real interior mutability later without breaking existing APIs that rely on the type being Freeze.
Not the ability to make something Freeze despite interior mutability.

For another "motivating example", bytemuck is in basically the same position as zerocopy, in that some of the bounds are currently more strict than necessary (discussion). I have a branch with a sketch of bytemuck's API in the presence of core::mem::Freeze. (cc @Lokathor)

One possible alternative that would "work around" the semver issue would be to have Freeze be a "normal" (non-auto) trait that, like Copy, can only be implemented for types where all of their fields are also Freeze. This would require types to "opt-in" to guaranteeing they do not have interior mutability, just like types now have to "opt-in" to being Copyable. I don't necessarily think this option is better than the status quo[^1], but it is something to consider.
[^1]: since Freeze can be somewhat observed on stable via static-promotion, and this would either break that until libraries add impl Freeze for MyPODType, or require a second BikeshedRawFreeze actually-auto-trait as a supertrait of Freeze for the compiler to use for determining static-promotion etc.

Yeah that could work -- as you say, the existing Freeze trait has to stay around; we'd also have to make static promotion accept a type that's either AutoFreeze or OptInFreeze for the motivating example to work (which is all about static promotion of data at a generic type).

Thanks for writing this up. I appreciate the effort that went into it and the desire to push things forward.
But... I think this RFC needs a fair bit of work. After reading it, I'm still left with some very fundamental questions (and I expect these to be answered in the RFC):

What is the Freeze trait used for? How will Rust programmers use it?
What is an example of a typical use of the Freeze trait in Rust programs?
What would happen if we didn't stabilize Freeze?
The guide level explanation doesn't say anything about what Freeze is. The guide level section should not be a stub.
The reference level explanation should be reasonably self-contained and explain things in a fair bit of detail. I also have to say that I followed the links and they did not help me to understand what Freeze is.
There is zero mention of the fact that this is a new marker trait in the drawbacks section. Marker traits present significant annoyances. This needs more discussion and an exploration for why a new marker trait is really truly warranted.
I don't understand why we also aren't trying to stabilize unsafe impl Freeze. The RFC just says it's orthogonal, and while that may be true, that isn't a good enough reason to split them apart. The default position should be to stabilize both, and only after some compelling justification to do otherwise should we try to split it apart. If we're splitting them apart, for example, there should be an exploration of the drawbacks of doing that. For example, I sometimes need to write unsafe impl Send for MyType. What happens when I need to do that for Freeze but can't? (And if that can't happen or is unlikely to happen, then that is a very interesting difference from other marker traits that should have discussion about it in the RFC.)

I don't understand why we also aren't trying to stabilize unsafe impl Freeze.

IIUC, we couldn't do that today since Freeze promises that a type contains no UnsafeCells. That property is entirely visible to the compiler - either you have no UnsafeCells, in which case the compiler knows your type is Freeze, or you have some, in which case it would be unsound to implement Freeze for your type. We'd need to relax it to simply say "doesn't permit interior mutation" or something to that effect, in which case it'd be more about whether any of your public API permits interior mutation (so it's more of a runtime property and less of a type property).
If we decide to keep it a type property - ie, the absence of UnsafeCells - then we could do impl Freeze rather than unsafe impl Freeze per @zachs18:

One possible alternative that would "work around" the semver issue would be to have Freeze be a "normal" (non-auto) trait that, like Copy, can only be implemented for types where all of their fields are also Freeze. This would require types to "opt-in" to guaranteeing they do not have interior mutability, just like types now have to "opt-in" to being Copyable. I don't necessarily think this option is better than the status quo1, but it is something to consider.
Footnotes

since Freeze can be somewhat observed on stable via static-promotion, and this would either break that until libraries add impl Freeze for MyPODType, or require a second BikeshedRawFreeze actually-auto-trait as a supertrait of Freeze for the compiler to use for determining static-promotion etc. ↩

Freeze is used by the compiler (codegen, const-check) and Miri to determine whether a type allows shared mutation. We definitely don't want to allow people to disallow shared mutation on their UnsafeCell-carrying types -- I don't see when that would ever be useful, it opens so many questions, and it's not what anyone asked for.
So if we want to expose the Freeze trait that exists, that decides for codegen and Miri and const-checking whether there's any interior mutability, then we can't allow impls.

@joshlf For the purpose of zerocopy (and bytemuck), what advantages does exposing Freeze brings over a custom trait? You will have to derive it anyway because you have other required traits, right?

For the purpose of zerocopy (and bytemuck), what advantages does exposing Freeze brings over a custom trait?

IMO the main advantage is that Freeze can be definitively checked by the compiler. Under the current Freeze as an auto trait, no derives or impls are needed (or possible) for it since it is automatically implemented by the compiler when it can be. [^1]
An additional advantage is that there would be only one trait to encode the invariant. bytemuck and zerocopy (and any other cratr) could each implement their own Immutable trait, but with Freeze in the stdlib this would not be necessary.

You will have to derive it anyway because you have other required traits, right?

For my sketch of bytemuck's API with Freeze, Freeze is simply added as an additional bound on the functions where it is needed, it is not a supertrait or subtrait of any bytemuck trait itself.
[^1]: Also, even a hypothetical Copy-like "opt-in-but-still-validated-impls" version of Freeze would still be easier to correctly implement since an incorrect implementation simply wouldn't compile (just like impl Copy for struct Wrapper(String) doesn't compile today).

@joshlf For the purpose of zerocopy (and bytemuck), what advantages does exposing Freeze brings over a custom trait? You will have to derive it anyway because you have other required traits, right?

The most important reason is that the compiler can see into the internals of types in the standard library. For example, I filed this extremely silly issue because there's technically no guarantee provided to code outside of the standard library that Box<T> doesn't contain any UnsafeCells. That's obviously true, but in order for zerocopy to uphold its soundness policy, we need an explicit guarantee. Such a guarantee would be useful to basically nobody but us. By contrast, Box already implements Freeze, so if we could use Freeze directly, we wouldn't have to quibble about things like that. Here's another PR that would be at least partially obviated by Freeze.
Besides that, here are some other reasons:

All of the normal advantages of not having to own something like this. For example, zerocopy and bytemuck - if we were both to implement the same idea - would duplicate code between the two of us.
We have to manually implement Immutable for a huge number of types, all of which is extra code we have to write and maintain (this file contains the keyword Immutable 159 times!). Worse, since it's an unsafe trait, it's also safety proofs we have to write and maintain. Maintaining safety proofs is a giant headache since there's no programmatic way to discover that your safety proof isn't valid anymore since it's just prose, so we do a lot of manual work to make sure our safety proofs are forwards-compatible (this relates to the soundness policy I mentioned before). Often a single safety comment will be blocked for weeks or months while we try to get guarantees landed in the language documentation.
The custom derives, while not complicated by custom derive standards, are much uglier than the equivalent in-compiler implementation since the latter has direct access to a more natural representation of a type's fields and their types. We have to hack it by adding clumsy where bounds and such.
Proc macros are slow to compile and slow to execute, while the compiler-supported implementation will presumably be much faster
The compiler might support fancier reasoning at some point, e.g. permitting UnsafeCell<()>. Supporting this natively in zerocopy would be very difficult. This particular example may be contrived, but my point is that, in general, the compiler can support much richer analysis that would require feats of code gymnastics to pull off outside the compiler.

Thanks to everyone taking an interest in this RFC.
First, a quick apology for it starting up so half-assed, I clearly underestimated the task. I started this RFC as a knee-jerk reaction to 1.78 merging the breaking change the RFC mentions without providing a way for const references to generics to exist.
I'm still committed to getting this to move forward, but I have limited availability for the next 2 weeks, so please bear with me as I rewrite this entire thing to the standard I should have started it with.
I would really appreciate getting some early feedback on the direction people around here would like to see this RFC take regarding:

Should this RFC target the existing core::marker::Freeze or propose an alternative (let's strawman it as Frieza) with similar purpose?

A personal priority I have for this RFC is allowing const references to generics in stable Rust again. Frieza: core::marker::Freeze is my highest priority.

Recursivity:

core::marker::Freeze only accounts for "local" immutability. This is good enough for certain purposes (such as static promotion), but not for stuff like maps that would like to ensure their keys are totally immutable.
Would your application benefit from Frieza being recursive? Would it suffer from it being so?

Auto-trait-iness: do you feel happy/neutral/unhappy about Frieza being an auto-trait?

If the consensus is that more auto-traits=bad, creating Frieza seems like the right approach to me.
Otherwise, we could just expose core::marker::Freeze, and a recursive equivalent could be built either inside or outside core later.

Sorry for my high latency for the beginnings of this RFC. I really appreciate all the feedback you can give, and would like to mention that PRs to this PR's branch are very welcome :)

@p-avital happy to hear that you're not discouraged from all the extra work we're throwing your way. :)

core::marker::Freeze only accounts for "local" immutability. This is good enough for certain purposes (such as static promotion), but not for stuff like maps that would like to ensure their keys are totally immutable.

Even if Freeze was recursive, types can use raw pointers or global static to implement shared mutable state. So I don't think this RFC should try to do anything in that direction -- between promotion of generic consts, zerocopy, and bytemuck we have three users that all only care about "shallow" immutability. A map can use an unsafe trait PartialEqPure or so if a proof is required that partial_eq (and/or other functions) are pure.

@p-avital It's hard for me to give you good feedback here because I think I lack a fair bit of context. Most of the discussion (and the RFC itself) seems to be very high context here, and it's context that I don't have.
With that said, I can say that adding a new auto/marker trait is a Very Big Deal. It's not just that "auto traits = bad," but that adding a new one comes with very significant trade offs. That doesn't mean that adding one is impossible, but it does mean, IMO, that adding one needs to come with significant benefits. Or that it's the only path forward for some feature that most everyone deems to be essential.
From the discussion, it sounds like there are alternatives. So those alternatives, at minimum, should be explored.

@BurntSushi note that the Freeze marker trait, albeit being unstable, is already a semver hazard. Whenever I have a constant of type T, there are things the compiler will let me do only if T: Freeze. So, while new auto traits are a Very Big Deal, this is not an entirely new auto trait, and it merely extends the existing semver hazard to also affect crates that do not allow creating const values of their type. But any crate that has a const fn new() -> Self already needs to worry about Freeze, in today's Rust.
Here's a demonstration.

@RalfJung Yikes. OK. Thank you for pointing that out. That's exactly the kind of context I was missing.

Good news! I did find time to rewrite this as a proper RFC. Now ready for the next wave of comments :)

(Btw, github supports committing multiple of these proposals into a single commit.  You just have to go to the diff view and click "add to batch" on all the proposals you want to include.)

(Btw, github supports committing multiple of these proposals into a single commit.  You just have to go to the diff view and click "add to batch" on all the proposals you want to include.)

Oooh, didn't spot that one, sorry for the noise :(

I've not read the updated RFC, but I wanted to share a few general thoughts re: semver, opt-in, etc...
First, in terms of semver, there are two mitigations

As @RalfJung pointed out, there is already a semver hazard around freeze-ness, because the compiler requires that types in some contexts are Freeze (even if you don't have a way to name it).
It's already the case today that adding "interior mutability" in the form of a Cell is a breaking change (because Sync); with Freeze, adding a Mutex or AtomicI32 would now be a breaking change in addition -- but note that adding an Arc<Mutex> or Arc<AtomicI32> would not.

In general I feel that the semver rules for Rust are something like: using tree-like ownership with static dispatch is "idiomatic" and the most flexible. Straying from that is a kind of breaking change because some low-level software will stop working. I know that's not what we actually say and it's more complex, but it seems to be true.
I do think it's important that we have (at minimum) a way to opt out from Freeze. For example, I think that Mutex may on some platforms require an allocation, which means that it would be "shallowly Freeze", but it would want to opt-out from that. (There's an interesting interaction with MSRV here, in that older versions compiled with newer compilers cannot opt-out...i.e., if I have version X that uses Freeze, then Freeze is introduced in Rust 1.Y, and in version X.1 I opt-out, code could still have upgraded rust to version 1.Y and used version X of my library, making X.1 a breaking change; annoying).
Another option of course would be to make Freeze be like Copy: opt-in but checked.
I also feel there is a general tension between semver and safety, particularly in the context of systems programming. When you want to be able to do low-level hacks, you often need to rely on certain properties of types (e.g., their size, whether they can have interior mutation, whether they make use of thread-safety) that go beyond the traditional abstraction boundaries we see in other languages -- these boundaries are intentionally intended to obscure exactly what data is accessed and how, but that is precisely the kind of information you need to create these safety abstractions. This is why people used to say that parallelism is "anti-encapsulation". I think this existing semver hazard, which has to do with our ability to put statics into a read-only data section etc, is a similar example.
Overall I think it's important we expose the tools people need to build low-level optimizations. I also think it's important that people are able to check that these low-level optimziations are safe. This is true even if the combination works against semver.

I do think it's important that we have (at minimum) a way to opt out from Freeze

So, like a PhantomNotFreeze type?

So, like a PhantomNotFreeze type?

That would be the obvious way to go that matches precedent. I hope we can shift to negative impls but that's a separate effort.

I've had some use cases for this, I think in salsa, I have to think about whether shallow- or deep- would've been useful. I suspect shallow is the thing I really need for the same reason as everyone else.

Doesn't UnsafeCell<()> have the same effect as PhantomNotFreeze, admittedly with less ergonomics? Or have I been suggesting that incorrectly so far?

Reasonable optin for "disable all auto traits", I suppose.
On Wed, May 15, 2024, at 5:51 PM, Clar Fon wrote:

Doesn't UnsafeCell<()> have the same effect as PhantomNotFreeze, admittedly with less ergonomics? Or have I been suggesting that incorrectly so far?
—
Reply to this email directly, view it on GitHub https://github.com/rust-lang/rfcs/pull/3633#issuecomment-2113512677, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABF4ZR2IT557Y5X3CJGXD3ZCPKE5AVCNFSM6AAAAABHQ6KFQKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMJTGUYTENRXG4.
You are receiving this because you commented.Message ID: @.***>

Oh, fair point. I guess that Send and Sync also break with that as well.

Doesn't UnsafeCell<()> have the same effect as PhantomNotFreeze, admittedly with less ergonomics? Or have I been suggesting that incorrectly so far?

It currently does, but with some core questions around UnsafeCell still being open, maybe we want to provide something more explicit.

Reasonable optin for "disable all auto traits", I suppose.

And also disable inference of variance?
Can we then also allow unused type and lifetime parameters, please? ;)
But all of that seems like outside the scope of a Freeze RFC.

I was thinking more of that the value in memory is stuck at whatever bytes are put into .rodata

That's far from it, though. Freeze isnt just about rodata, it's about all shared references without interior mutability.
I find "stuck" extremely unintuitive. It conveys negative feelings to me, "we're stuck somewhere or with something (and want to get unstuck again)". I don't think it is a good term for "this memory is immutable".

since the LLVM IR op is already called freeze, they've effectively already laid claim to that name, so anyone who's used LLVM IR and tries to figure out rust will be confused if we use freeze to mean something completely different where it could be confused with an operation (since freeze is present-tense, unlike stuck where being past-tense and being applied to types makes it much less likely to be an operation).

LLVM also uses poison/undef, neither of which we use in Rust (we call it "uninitialized memory" or just "uninit"). LLVM calls it getelementptr, we call it offset. LLVM calls it load/store, we call it read/write. The list can probably go on for a while. I don't think we should let LLVM dictate our terminology.

I like the name Immutable. Maybe it will help shift people away from using the term "immutable reference" vs "shared reference". Agreed that the shallow vs deep distinction is potentially confusing, particularly in contrast to Send, and agreed that it is orthogonal.
Message ID: @.***>

Hello again everyone,
I'm back and ready to keep pushing on this. I've already committed the result of every suggestion that seemed immediately actionable to me.
It appears to me that 2 remaining roadblocks are:

[ ] Should we rename Freeze?
[ ] Should we provide PhantomNotFreeze in the same stride?

As an aside to maintainers around here, is there a chance we can slip this in 1.79 if we're quick enough about resolving those questions, or is the release cycle already too far gone?

One remaining question in my mind. I'm not sure whether this would affect the design, but it might.
Do we want to leave open the possibility of implementing Freeze conditional on how many bytes are covered by UnsafeCell, even via generis? Ie:
struct Foo<T>(UnsafeCell<T>);

static_assertions::assert_not_impl_any!(Foo<u8>: Freeze);
static_assertions::assert_impl_all!(Foo<()>: Freeze);

@joshlf, I believe that possibility is permitted. See the reference level explanation's proposal of Freeze's documentation: https://github.com/p-avital/rfcs/blob/stabilize-marker-freeze/text/0000-stabilize-marker-freeze.md#reference-level-explanation

As an aside to maintainers around here, is there a chance we can slip this in 1.79 if we're quick enough about resolving those questions, or is the release cycle already too far gone?

1.79 is in beta and not going to receive any new features. If this got implemented before June 7th, it would be part of 1.80. But given t-lang capacity and the 10-day FCP period for the RFC, I don't want to raise any false expectations -- that's very unlikely. If it makes it into 1.81 (which branches on July 19th) that would be extremely fast by RFC standards.
Rust is moving too fast for some people's liking, but not that fast.

One remaining question in my mind. I'm not sure whether this would affect the design, but it might.
Do we want to leave open the possibility of implementing Freeze conditional on how many bytes are covered by UnsafeCell, even via generis? Ie:
struct Foo<T>(UnsafeCell<T>);

static_assertions::assert_not_impl_any!(Foo<u8>: Freeze);
static_assertions::assert_impl_all!(Foo<()>: Freeze);

I think it does. I think a nice way to allow for this to fit the RFC nicely would be to:

Highlight the RFC's definition that Freeze is about byte-fiddling, and therefore does not promise that T: Freeze doesn't guarantee that T doesn't have some ZST UnsafeCell.
Propose the opt-out mechanism at the same time, such that maintainers can opt out of Freeze "upon release". To me, this could come in a second time since they didn't have access to such a mechanism for previous releases either way, but I do think it's a valuable addition, even if it just starts as a wrapper around UnsafeCell<()>.

Rust is moving too fast for some people's liking, but not that fast.

Bunch of grumps! :smiling_face_with_tear:

If it makes it into 1.81 (which branches on July 19th) that would be extremely fast by RFC standards

So at least 2 more months of sad stabby. Imma push as hard as I can for this to happen without putting a sour taste in t-lang's mouth :)

I'm afraid it's going to be more than 2 months -- 1.81 branches on July 19th, i.e. that's when the 6-week beta period starts. It is going to be released on Sep 5th.
I hope this is not too much of a downer, your help here is much appreciated! It's just generally not  good idea to rush irreversible decisions that the entire Rust community will have to live with forever.

I hope this is not too much of a downer, your help here is much appreciated! It's just generally not good idea to rush irreversible decisions that the entire Rust community will have to live with forever.

Honestly, the downer was 1.78; now I'm just fired up to get that situation resolved!
I fully understand wanting to do due diligence on these things, I've been hurt by both ends of the stabilization stick before (looking at you, forever ABI-unstable core::task::Waker because the constructor takes extern "Rust" fns).
But just to keep a sense of urgency: not making it to 1.81 would be a downer :stuck_out_tongue:
As an aside: I've updated the proposal to include PhantomNotFreeze for semver nerds :)

Hiya @RalfJung, do you see any work this RFC still needs before it can enter FCP? :)

Hiya @RalfJung (sorry if the ping is a bit cavalier), do you see any work this RFC still needs before it can enter FCP? :)

That's a question for t-lang, not for me. :) I have nominated the RFC to be discussed by the lang team.

Hi @RalfJung, sorry for pestering you, any updates on the RFC's status?
Let me know if there's a more appropriate place/person to follow up with on this RFC :)

It's in the lang team's nomination queue. They probably have several dozen items in that queue so we'll have to see when they get around to discussing this RFC.

Sorry if this is the wrong place to comment this, but I wanted to note that using Freeze bounds, it is possible to use arrays based on type-level numbers (like the generic_array crate) as a replacement for generic-const-expr and create associated constant strings of dynamic sizes over generic type-numbers.
Here is a working example for turning peano-style typenums into their unary representation at compile-time: playground
This requires Freeze because the custom array type's contents depend on a generic number type, so the compiler cannot rule out the existence of some implementation that contains an UnsafeCell.
I have also written a working conversion to base 10 using binary type numbers (similar to those of typenum), but it's too much to post here.


Operational query on running stunner in headless mode

Currently I am a bit confused with the scaling operation of stunner in one-to-one call scenario. This the setup, initially I would have one stunner with LoadBalancer service (Public facing IP) and cluster service IP for WebRTC client within k8s. This works fine as long as there is only one stunner pod. But once I scale the stunner pods to 3 instances I would assume the WebRTC would not establish because there is no control over the LB and cluster service to actually land the BIND requests to which stunner correct?
So in this case what should be done to scale ? A naive way I could think of is that I need to assign a new LB public address for each stunner and use headless service within the k8s. But this adds extra complexity on how should I ensure the both clients can use the same stunner ?
Thanks in advance

Oh sorry, they can use arbitrary stunner and talk to each other

Well ... I guess the confusion is still here ... Let's say I have stunner-A and stunner-B. A browser as a video player and an application in k8s which streams video frames. So now when the browser connects to stunner-A and the application connects to stunner-B, will this work out ? So browser would have a relay addr which is the pod IP + random port of stunner-A and the application most likely can uses its pod IP + random port and it is host candidate. So how does stunner-A forwards to application on its host candidate?

IIUC, if the application talks to stunner-B than by default, application would not be able to whitelist the peer addr in stunner-A, so the browser should not be able to reach the application via stunner-A, is it correct ?

I'll try to answer your  questions below, let me know if I am being too fast (or too slow).

Let's say I have stunner-A and stunner-B. A browser as a video player and an application in k8s which streams video frames. So now when the browser connects to stunner-A and the application connects to stunner-B, will this work out ?

The idea here is that the user, located outside the cluster behind potentially several layers of NAT, will connect via STUNner-A (or STUNner-B), by creating a transport relay connection on STUNner. The user-facing side of the transport relay connection runs TURN, so that it is immune to NATs, while the cluster-facing side uses plain UDP. Now the client will ask STUNner to return the IP address and port corresponding to the cluster-facing side of the transport relay connection, this is the so called "relay candidate". Note that the IP address here is STUNner-A's pod IP, while the port is some semi-random (STUNner allows you to customize the upstream port range).
The application on the other hand, being located inside the cluster, will not need STUNner at all: it will create a humble host candidate over its pod IP. (Note that this is contingent on that you configure your application with no external STUN or TURN servers at all; otherwise the client and the application may establish a media connection circumventing STUNner, which is most probably not what you want. Curiously, TURN magic also allows us to successfully connect the client and the application in the case when you mistakenly set STUNner as a TURN server for the application, but this is much rather a funny side-effect than a useful feature here.)

So browser would have a relay addr which is the pod IP + random port of stunner-A and the application most likely can uses its pod IP + random port and it is host candidate. So how does stunner-A forwards to application on its host candidate?

Exactly! Now the client passes over the relay candidate to the application and the application returns its own host candidate to the client over the signaling channel, and then they try to connect via this candidate pair. The client sends a packet to the application's host candidate via STUNner A, which it will conveniently receive over the external TURN connection and forward over its pod IP:port (the relay candidate) to the application's pod IP:port (the host candidate). Now, since pod-pod communication in Kubernetes must never ever involve a NAT, this connection setup attempt will be successful: the application will find that it receives the packet from the IP:port it was told in the relay candidate, and the same applies in the other way around. ICE goes to connected state and voila, your WebRTC media is ingested into the cluster despite that it commonly traverses 2-3 NATs while traveling from the client to the application via the Kubernetes container network.
Does this answer your question?

@rg0now Excellent explanation! Huge Thanks!

(Note that this is contingent on that you configure your application with no external STUN or TURN servers at all; otherwise the client and the application may establish a media connection circumventing STUNner, which is most probably not what you want. Curiously, TURN magic also allows us to successfully connect the client and the application in the case when you mistakenly set STUNner as a TURN server for the application, but this is much rather a funny side-effect than a useful feature here.)

I noticed this too, but I was wondering in the one-to-one call example, when we loop back the connection to stunner itself, it is then the case that both sides end up using stunner provided relay addr for the media transport. However, I do find that the most intuitive way is to configure the UDPRoute to the media service as backend so Pod-to-Pod communication is right there.

I noticed this too, but I was wondering in the one-to-one call example, when we loop back the connection to stunner itself, it is then the case that both sides end up using stunner provided relay addr for the media transport.

I'm not sure we're using the word "looping back" in the same sense. We use this term to mean that in the headless model, when there is no media server at all, both clients open a relay candidate on STUNner and then these magically get looped back to one another during the ICE conversation, connecting the two clients. This works because both relay candidate's contain pod IPs, so this is a fairly standard TURN server use case it is just that the relay candidates are opened "inside" the cluster.
The one2one call demo uses the "media-plane setup" on the other hand, in which case the flow of packets is client_a -> stuner_a -> kurento -> stunner_b -> client_b and vice versa. In some sense this is also "looping back" media to STUNner; I guess this is what you meant.

Is this correct ? So it really does not matter which one connects to which stunner as the ICE server

Almost correct: I would make this more precise in the following way: "a client external to the cluster MUST use STUNner to open a transport relay connection, and the application located inside the cluster MAY use STUNner to open a transport relay candidate" (where MUST and MAY mean what they usually do in IETF parlance).
Observe the difference: the case when the client generates only a host candidate (or a server reflexive candidate using a public STUN server) but it does not use STUNner, while the application opens a relay candidate via STUNner (and potentially further host candidates) will not work. This is because in this case the clients' ICE candidates all contain public IPs while the application's candidate (obtained from STUNner) contains a private IP (pod IP), and the former has no way to route packets to the latter (the other way around should work though but I don't know ICE enough to know whether or not it handles half-duplex connections, plus there is a NAT in the middle so I guess this should not work either).
The other case (both the client and the application use STUNner) should work though (not that I ever tested this). Feel free to try, let me know what you obtained.

To that end, I also believe once we have a large number of pion streamers in our cluster, we need to have a relatively small sized media server group in front of the pion streamers so the STUNner would not need to loop over like thousands or tens of thousands Route targets.
In my imagination, for instance, I can have a media server group of 5 servers. Each of them acting as SFU? So the current streamers could just send the frames to the media servers and our clients should fetch corresponding media tracks from those media servers. This is how I imagined to scale our service.
Any suggestions or corrections would be greatly appreciated!

To that end, I also believe once we have a large number of pion streamers in our cluster, we need to have a relatively small sized media server group in front of the pion streamers so the STUNner would not need to loop over like thousands or tens of thousands Route targets.

Can you give more detail? We are extremely interested in production deployments of STUNner: we want to learn what works and what not in your setup. Let us know if we could maybe organize a call or something.
That being said, STUNner does not quite pre-condition the size of the backend pool, as long as all backends belong to the same Kubernetes service.  (Well, this is not entirely true: the operator actually injects all the pod IPs as endpoint IP addresses into the STUNner config so when you have like thousands of pods in a single backend service then this may take a while, but if you have thousands of media servers in your cluster then I guess this is your least worry). So if it is an option to you to put all your pion streamers into a single Kubernetes service then all should be fine.
Quick note: STUNner in fact does not load-balance across the backend pool (yet): by the time STUNner receives a TURN allocation request from the client the client actually knows which media-server/application/backend it wants to talk to (the one corresponding to the ICE candidate received from the application), so STUNner just merely checks whether that pod-ip belongs to the Kubernetes service you specified as the UDPRoute backend, and it if does then it lets the request through, otherwise it blocks it.
Does this make any sense?

Totally!
Sure I will try to put more details of our setup. Besides, I believe we could probably find a time by mid of Sep. to have a conversation via concall. Stay in touch!
The deployment is as follows.
Our application in the cloud (k8s) is more or less like a online gaming, where users will be assigned to a dedicated server. The server renders user input into h264 frames and stream those frames to our pion based proxy sidecar. The user is using WebSocket for signaling with this pion proxy sidecar directly and they use STUNner as the broker. For the latest configuration, user would get a relay addr from STUNner and the pion proxy would just use POD IP for the RTC communication. We do not possess any media servers I would say.  So that being said, let's say if we want to serve 1K users simultaneously, we will then scale 1K server apps + proxies so that STUNner will have 1K routes in this case. What I was saying previously is that perhaps when we have 10K users, we should have a pool of media servers and STUNner only takes care of the media servers. Then our application server's proxy would somehow stream the videos to those media servers.

Thanks @imcom, this was super insightful. Two minor notes:

As per STUNner's scaling per backend pod: The dataplane (stunnerd) should be able to handle tens of thousands of simultaneous connections into the cluster, and if not then that's a bug. Another blocker may be raw packet speed: in standalone benchmarks STUNner can handle 50-100k packet per sec with 1-2 cores and our GKE benchmarks show somewhere around 30-50 kpps per 1.c cores, but this is with only 10-50 simultaneous connections.
As per the control plane: at present the Kubernetes operator watches all the Kubernetes Endpoint resources in the cluster and it will re-generate the config for each change, so it is definitely not designed to handle 10k endpoints per service and it is absolutely not designed for massive-scale Endpoint churn. It should work, but it might be slow.

That being said, I don't see a reason why the current STUNner version would not work with 10k endpoints, provided that the dataplane is scaled-out to 10-20 pods and you give sufficient CPU to the operator. And if not, then your idea to inject media servers between can be a good solution.
Backend scalability has not been in the focus yet, but I guess your input should be a good motivation for us to work on this: we know how to handle this, just haven't had the time/resources to work on it yet.

Lovely, we have not got chance to do the benchmark on our side, mind to elaborate more on the tooling or methodology you used for it ? Anyhow your benchmark result looks just sufficient and reasonable for our use case. Glad to hear that !

Just wanted to drop a line here that we have documented the ideas that came up during our discussion:

in the "Asymmetric ICE mode" the client uses STUNner to generate a TURN relay candidate via STUNner and the server generates only a host candidate, while
in the "Symmetric ICE mode" both the client and the server generate TURN a relay candidate and they connect via STUNner.

I'm closing this issue for now, feel free to reopen if there's any related issue you'd like to discuss.


Stubs mapping directory isn' t created in sub module

If you have gradle multi project the verifier doesn' t create the mapping with Json folder. The workaround consists to change the stubs directory name.

Hi @Alvise88 . Do you think it would be possible for you to create a sample Github project to recreate that? Thanks a lot!

+1
Similar issue in my case: simple SpringBoot service. gradle task generateWireMockClientStubs is not executing since it iss marked as UP-TO-DATE. as a result, the mappings dir is not created.
Workaround: use --rerun-tasks on gradle build to force UP-TO-DATE tasks to run anyway

Do you have a sample project? It would help me speed things up with fixing this

nope, sorry.
It occured in my production project. Worked for some time, then stopped just like that. I can't figure out what could happen. There were changes in codebase, but none of them touched CDC. Stubs are copied to the jar. Only json mappings are missing (the whole dir)

ok so if I have a multimodule Gradle project then this doesn't work? How do you apply the plugin?

Ok I've replicated that - https://github.com/marcingrzejszczak/sc-contract-159 . ./gradlew checkMappings blows up :/


Allow Jobs to be marked as read-only

Environment

Python version: Python 3.8.5
Nautobot version: 1.0.0b2

Proposed Functionality
It would be helpful to be able to mark a Job as being read-only, i.e. not modifying the database. This would then remove "Commit message" checkbox from the Job form, and would stop "Database changes..." log messages from showing up in the Job results.

Use Case
As an end user running Jobs that are read-only (formerly known as Reports) I might not understand why I'm seeing the "Commit changes" checkbox and why "Database changes..." logs show up in Job results.
As a programmer writing a Job I want to be able to specify that given Job will not modify database objects.

Database Changes
None.

External Dependencies
None.

Thanks @progala for submitting this. :)


https://link.trustwallet.com/send?coin=195&address=TYqUrXbGHg8hsdoiUXnvHzXFYYBrc6Fnmd

My Public Address to Receive TRX TYqUrXbGHg8hsdoiUXnvHzXFYYBrc6Fnmd
Pay me via Trust Wallet: https://link.trustwallet.com/send?coin=195&address=TYqUrXbGHg8hsdoiUXnvHzXFYYBrc6Fnmd

Please submit the correct information, invalid information will cause the issue to close


Install dotnet 2.2 on Ubuntu 19.04 missing dependencies

Problem encountered on https://dotnet.microsoft.com/download/linux-package-manager/ubuntu18-10/sdk-current
Operating System: Linux Ubuntu 18.10 - x64
Provide details about the problem you are experiencing. Include your operating system version, exact error message, code sample, and anything else that is relevant.
Install of SDK fails with
dotnet-sdk-2.2 : Depends: aspnetcore-runtime-2.2 (>= 2.2.4) but it is not going to be installed
Depends: dotnet-runtime-2.2 (>= 2.2.4) but it is not going to be installed
E: Unable to correct problems, you have held broken packages.
sudo apt-get install dotnet-sdk-2.2
Reading package lists... Done
Building dependency tree
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:
The following packages have unmet dependencies:
dotnet-sdk-2.2 : Depends: aspnetcore-runtime-2.2 (>= 2.2.4) but it is not going to be installed
Depends: dotnet-runtime-2.2 (>= 2.2.4) but it is not going to be installed
E: Unable to correct problems, you have held broken packages.

Hi, this is my solution
wget -q https://packages.microsoft.com/config/ubuntu/18.10/packages-microsoft-prod.deb
sudo mv prod.list /etc/apt/sources.list.d/microsoft-prod.list
sudo gedit /etc/apt/sources.list.d/microsoft-prod.list
and change in string
deb [arch=amd64] https://packages.microsoft.com/ubuntu/19.04/prod **xenial** main
to
deb [arch=amd64] https://packages.microsoft.com/ubuntu/19.04/prod **disco** main

Duplicate of https://github.com/dotnet/core/issues/2526, we don't have packages for 19.04 quite yet.

Any news on when we will get 2.1 and 2.2 working for Ubuntu 19.04?