Pinned tabs: show them in a secondary tab row above others

Refs: https://github.com/microsoft/vscode/issues/12622
Pinned tabs by default will show to the left of other tabs. An alternative presentation is to do what Visual Studio does and have a secondary tab row above the others:

vs what we do today:

Here's why I would like to get the pinned tab VS style:
I use pinned tab has my main work focus. I usually have between 2 to 5 tabs pinned and all the tab noise of opening / looking / searching / inspecting is kept out of it. So when I find what I'm looking for, I can instantly go back to the focused tabs and they are clearly delimited on their own row.
It's a huge productivity booster for me to be able to organise tab like that.

@yannlairdc pinned tabs in VSCode are always the first tabs of the row and never scroll out of view, does that still require a secondary row then?

@bpasero It's all about visibility and speed of access to what you're looking for. I will say that having this improvement without the other feature to display the full name of the pinned files reduces the value of that one alone.
When I'm looking for something or iterating through references in the code, I can go back to where I was at the glance of an eye. I know it's going to be on the first row, I know it's not going to move whatever I do and I know which tab I need to go back to with the name displayed.
It allows me in my head to have the first row as my area of focus, while everything in the second row is just "noise" that I disregard.
Having everything pinned in the first rows without the name display is a nice addition, but not quite there for me.
The reason it works well with chrome tabs is because the content of those pinned tab never changes and it's easy to recognise the icon of the website in case of doubt. Witt VS Code, the files pinned changes regularly and most of the time they are of the same type.
Having 7 pinned tabs with a C# icon on them would still force me to iterate on each tab until I find back the tab I'm interested in, while pushing to the right all the others opened tab.

@yannlairdc yeah understood, assume you have a setting to show the name of a tab even when pinned. Would that be sufficient to not add another row of tabs?

@bpasero That would be another good step in the right direction for sure, but it wouldn't solve all the problem. If you have 6 or 7 pinned tab with their full name displayed, their's a high chance that it will occupy most of the screen and left out the non pinned tabs.
By the way, I just realised something by looking at both VS and Code and I realised that the tab in Code are much bigger than in VS, which makes the above problems more critical (see below).

If I have 6 pinned tabs, it would occupy most of the space opened on my 27" screen. Is there any option to be able to reduce the size? Or put them on several rows maybe?

@yannlairdc you might want to check out "workbench.editor.tabSizing": "shrink" setting which will ensure tabs start to shrink to a minimum of 80px when space is limited.
To be clear: I would only suggest a solution that makes pinned tabs shrink too so that you can have many pinned tabs side by side without starting to scroll. E.g. one idea is to make every pinned tab shrink as if "workbench.editor.tabSizing": "shrink" as configured.

@bpasero If we can get the pinned tab on a separated row, this problem almost solves itself.

I am trying to find a design alternative that does not require a second row of tabs which I personally find visually very heavy.

I understand that not everyone would like a second row, but that's what I'm looking for (and that's the title of this enhancement request ^^).
I'm not asking for it to be the default behavior or to replace current implementation, I would just like the ability to configure it this way for me and other likeminded people (lot of them requested that specific behavior to be implemented on the original pinned tabs topic).

I 100% agree with @yannlairdc. I have exactly the same use case. In most cases, I have 3-5 pinned tabs and a bunch of tabs on the 2nd row in VS. The ability to have full names for the pinned tabs is crucial as well because e.g. if I'm working on some backend feature then all or most of my files will be *.cs.
One more thing, with the ability to pin tabs, it would be nice to have the option to "Close all but pinned" tabs.

The ability to have full names for the pinned tabs is crucial

Which https://github.com/microsoft/vscode/issues/98161 would try to cover too.

One more thing, with the ability to pin tabs, it would be nice to have the option to "Close all but pinned" tabs.

That should already work today, pinned tabs should stay open for mass closing commands.

Not intending to "beat the drums", but I definitely believe that having pinned tabs on a separate row will be more beneficial than "shrunk" tabs on the same row as the rest of the tabs.
From a functional perspective, we want to be able to:

Read the whole file name (showing an icon doesn't help when all the file types we're working with are the same)
Quickly determine which tabs are pinned and which ones are not (which is challenging when all the tabs on are the same line)

I understand the desire to come up with a less "heavy" solution, but I'm not quite sure what that would look like without sacrificing some functionality.

Current pinned tab style may also cause misunderstanding frequently, like this(I pinned a JS file):

I also agree with @yannlairdc that the size of tabs in VS Code is too large, either in width or height. Hope there are options to modify the tab size more flexibly.

Another vote for pinned tabs having their own row, possibly with a different background color too. When I'm working in a large project with many files of the same type, I end up having anywhere from 4-8 files that I need quick access to. I need to be able to see the file names, not just the icon.
Having 2 rows would be extremely helpful to my workflow.
The visual studio example looks great.

I would love for Pinned Tabs to have their own row, but I would also like to see them get some UI treatment in the File Explorer pane. Giving them either their own group or section, or a pinned indicator could alleviate the frustration, at least in a workaround fashion.

Having pinned tabs that exist on a separate row is a fantastic productivity booster, but the brownie points version of this is to support a dynamic number of rows. IMO pinning a tab should not automatically create a separate row. But when a user drags a pinned tab up or down (beyond some threshold) the IDE would spawn a new row onto which the user can drop said pin. This would then allow for multiple rows of pins. Perhaps I want a row for configs, another for base classes, another for utility classes, etc. I've looked for this feature in every IDE ever, but have never found it.

Not sure about the secondary row, it steals some space and doesn't attack the source of the issue: as soon as you start working with files from one or other project, you end with tons of files opened.
Now that you can pin a Tab, probably is time to have a solution which grows from Dumb to Smarter:
Dumb Tab Row
The Dumb Tab row just shows the Tabs related to the active repo/folder in Workspace. If you change to a file from a different repo/folder, files unrelated to the new folder are hidden but the pinned ones.
Ie:  if I am working in the Frontend Repo, with F1, F2, F3 files/tabs opened and I am going to open the file B1 from my Backend Repo I can pin the, ie, F1 tab, before clicking in the B1 file.
As soon as I open/click the B1 file the F2,F3 tabs are hidden but not the F1 one since it was pinned.
I can keep working in my Backend project opening files, B2,B3,B4, B5. But as soon as I click in the F1 tab/file, the F2, F3 files which were hidden are revealed and B1,B2,B3,B4,B5 are now hidden.
Tabs are not closed but hidden, so it is fast to swap from project F to B while keeping the Tab bar tidy.
Also it will help us with two scenarios I usually find:
1 - Workspace with unrelated projects.

Sometimes I have 4 landing pages/projects totally unrelated. But I have them in the same Workspace. I start working in one of them, opening dozen of files(html, css, js..etc). I finish my work there so I click in a file of a different project, automatically the Dumb Tab Row would hide the Tabs of the previous project, cleaning my Tabs row. Since they are unrelated, I dont need them at all. But Ey! I suddenly realize I missed a detail in the previous project. I have just to click in any file in the Explorer window of the previous project, and then the hidden Tabs are revealed and the current one are hidden.

2 - Workspace with related project. Typical Front-Back.
I am working in a GET controller in the B1 file, implementing a middleware B2, a router B3, etc...
Now I decide to swap to the frontend file F1 which is calling the controller in B1. I would be pinning B1 tab, to not lose it from view when opening F1. As soon as I start editing F1, the rest of B files, but B1 which is pinned, are hidden, making room and reducing noise from my frontend work (as I will start open F2, F3,F4 files...). However if I suddenly detect a problem in my B code not sitting at B1, as soon as I click in the B1 file, B2, B3 and the B files I was using are shown..and F are hidden, helping me to swap fast from Front to Back.
This Dumb behavior would help a lot because it kills tons of unneeded files/leftovers but at the same time helps to swap easily from project to project.
I usually work with 3 projects at the same time, Front Web, Front Mobile, Backend...so the mess is 3x and Dumb Tab Row will help a lot.
Smarter Tab Row

It would work pretty similar to the Dumb one, however it doesnt hide the last files touched of the previous project. So, if I have B1,B2,B3,B4,B5 tabs opened of my Background project they would be hidden as soon as I click in a file from my Frontend project unless 1)the tab was recently touched, 2)the tab was recently saved, or 3)the tab is pinned.
This way when swapping from project to project, the Smarter Tab does not hide latest dirty or saved files, since the chances they are related to my new work is pretty high (or higher at least that those untouched for days but they are still hanging around).

TLTR: Just show related Files to the Project, unless they are pinned or recently touched/saved. It helps when swapping from project to project in the same Workspace by hidding unneeded files.

I usually work with 3 projects at the same time, Front Web, Front Mobile, Backend...so the mess is 3x and Dumb Tab Row will help a lot.

A humble suggestion, @vicmarcal, which, it seems, could side-step your raised issues while also offering other benefits to your workflow: What if you opened three instances of VSCode (one for each project). Each project would naturally have its own tab set and previously opened files, etc. This, presumably, would also help with things like file searches, debugging, commits, console commands, etc as an IDE instance would be constrained to the particular project's workspace.
I've recently adopted this strategy and have found it works better for me than one instance with multiple open projects.

This is a great feature. Thanks for it. I would prefer to have an option to show the file name or not. If I pin multiple javascript or same type of files. It shows only icons but it's difficult to remember the file names.
Thanks

Personally, I like the idea of dropdown boxes of @dean1010. Given the big support for having a secondary row makes me wonder if it would be feasible to have both, and leave it up to the user to configure which (if any) (s)he wants to use.

Working predominantly in settings that requires integration between files from various sources where each source addresses different topics, I would love to see multiple (named) dropdown boxes, that allow me to tack files from one source under the corresponding dropdown box. It's a bit like what workona does in browsers.

I usually work with 3 projects at the same time, Front Web, Front Mobile, Backend...so the mess is 3x and Dumb Tab Row will help a lot.

A humble suggestion, @vicmarcal, which, it seems, could side-step your raised issues while also offering other benefits to your workflow: what if you opened three instances of VSCode (one for each project). This, presumably, would also help you with things like file searches, debugging, commits, console commands, etc as each IDE instance is constrained to the particular project's workspace. After a reboot or closing your VSCode instances, you can easily get back to work by popping up a console, cd'ing to a project's folder, and running 'code .'. Using this setup, you'll find that each project maintains its own tab set, previously opened files, etc.
I've recently adopted this strategy and have found it works a lot better for me than one instance with multiple open projects.

Thanks for the suggestion, however, for me, that kind of "opening two Visual Codes" seems like an UIX hack after all. Since Visual Studio is not providing a good experience when having two projects opened, let's open two different windows.
I know a lot of mates working that way, so basically it means there is an UIX issue to be fixed, and Pinned tabs seems to be an approach to try to fix that. My suggestion tries to enhance a little bit more the Tab row behavior to make it a little bit better. I highly doubt that Pinned Tabs just on its own will fix the mess of tabs I already have.

Would be great to keep the pinned editors in the "explorer" section, and not have anything that occupies the row (not even a second row)
I work with projects with hundreds of files and usually I only deal with a handful. Would be great to use the "explorer" like a bookmark to quickly navigate between these files.

I work with projects with hundreds of files and usually I only deal with a handful. Would be great to use the "explorer" like a bookmark to quickly navigate between these files.

Hey there, @dashesy -- seems perhaps "pinning" isn't really what you're in the market for. I wonder if perhaps Zuo's "favorites" extension is a better match for your workflow?

I'm on the side of having a separate row for pinned tabs.
The points made by @yannlairdc are spot on. If you have been using Visual Studio IDE, you would know about this productivity boost. I hope this can be shipped!

I am trying to find a design alternative that does not require a second row of tabs which I personally find very heavy visually.

That's fair, but if you have a large monitor and high resolution then a second row barely takes up any screen real estate and can do wonders for productivity since you don't have to discern between pinned/unpinned tabs.
Perhaps a configurable "Pin Tabs to Separate Row" option would make everyone happy.
If it's not set then pinned tabs are just pinned to the far left of the single row as expected.
Does a second row appear to be too heavy in this situation?

If the goal is to maximize space-saving, then I think the very top frame of the window could be used as the second row. There's a bunch of dead space, and the main thing in the center is "file name - project name - Visual Studio Code". I don't know where this info could be moved to, but if you could move it, then that very top frame becomes a second row in itself.

i like that, please do it right now.

My preference is 1 row with the pinned tabs all the way on the left, but 2 rows is doable on my monitor.  Either way, really looking forward to pinned tabs that have the file name/path on them!

Given our wide variety of preferences wrt VSCode's default handling of rows/pins, there really is no silver bullet--we could debate for or against one, two, or n rows, but this won't get us anywhere. Really, the UI ought to be moldable to suit whatever need the user wants for a particular project. People who want only one row can have it. People that want more can have that, too. I'd like to point back to my above suggestion (on Jun 11) which takes this into account.

Having pinned tabs that exist on a separate row is a fantastic productivity booster, but the brownie points version of this is to support a dynamic number of rows. IMO pinning a tab should not automatically create a separate row. But when a user drags a pinned tab up or down (beyond some threshold) the IDE would spawn a new row onto which the user can drop said pin. This would then allow for multiple rows of pins. Perhaps I want a row for configs, another for base classes, another for utility classes, etc. I've looked for this feature in every IDE ever, but have never found it.

This issue feels like the Lord of the Rings trees who want to talk about something really basic for 1000 years before making a decision. Just add an OPTION for there to be pinned tabs like how it is in regular Visual Studio and be done with it. The fact that it's been stuck in limbo for this long makes me become suspicious if this is truly about difficulty/perfection of implementation or if it's some other secret reason. I agree with others. If nothing else, just make this an OPTION!!! Not mandatory, but an option that people can choose to enable or not so that no one can complain. You already have proof it works in regular Visual Studio, so just add it.

Not having those simple options for tab control is pushing me out of using VSCode now. I'm looking into Rider as a replacement for Visual Studio, because at least I can get some level of similarity with VS just for that part (and the fact that it embed resharper as well.)
It appears evident that VSCode developers, with their answers to both topics, are not willing to change their vision and make modifications to allow more flexibility and more use cases. They want it browser-tab like and that's it.

Aside from all its other advantages, allowing us to have a second tab row has one major advantage: it would make it easier for developers to come up with extensions to handle a lot more scenarios for tab sorting (such as per project, per status (pinned, dirty)).

Not sure how drag & drop will be supported with two tab rows. But it could make the pinned icon redundant. If one wants to unpin a tab one can drag & drop it to the desired location (this would also allow one to choose the position of the tab among others).
It's indeed a fidgety problem to solve, although I don't get some of the negativity surrounding this. It's wonderful we got the feature, now it's simply a matter of patiently tweaking it. The very fact we have somebody from the team discussing this with us (not on OUR expense) should make everybody do their utmost to participate rather than call it quits.

@bpasero any update on this? General consensus has been very clear about enabling a second row for pinned tabs as an option.

@bpasero any update on this? General consensus has been very clear about enabling a second row for pinned tabs as an option.

Anything?

I just requested this same feature, to "Show pinned tabs in a second row" at this thread.  https://github.com/microsoft/vscode/issues/110477
My request was rapidly closed as a duplicate. Please give us the ability to add a second row, or to create multiple rows in the code editor.
The latest release, October 2020 (1.51.1) offered a "Workbench" enhancement to make pinned tabs more prominent.  A person or team at Microsoft just worked on extending the tabs.
Please ask them to give us multiple rows in the code editor. Thank you.

I'm with everyone else. Having come from C# to JavaScript, the one feature I've missed about Visual Studio compared to VS Code is the second row of pinned tabs. I really don't get why this hasn't just been added as a feature.

I agree, @hersman. My understanding is that this is on their list of items. Apparently we need to upvote the original post from May 19 to increase its priority.

I'd also like to add the 1018 likes from #12622 making it the fourth most liked feature amongst open issues

Does this in the backlog? Anyone knows?

any update on this?

Can't believe this feature has been requested since 2016 and outright ignored, seems like such a basic requirement. With my current setup I can only see around 6 files in the tab bar - having more than 10-15 files open means that I have to scroll annoyingly through the top bar to find a file. The pinning doesn't start a new row or anything so pinning files right now doesn't make it any more organized than it already is. I need VS Code for typescript since VS 2019 goes crazy with the syntax highlighting but this feature would be a real productivity/organization booster.

Try to use open multiple file and find code you will know why we request a second row not this design. Second row can help to find a new file and when it used often will be category in pinned. Pinned file can help us open them quickly. If vscode has this function, I don't need to open vscode and vs2019 simultaneously when I read code. Personnally I would say your design is ridiculous because it doesn't help create a clear "pin" category but like a folder. Openning folder and find files which we would like open frequently is so ridiculous.

This feature should be implemented, how do I open the file on the left ?
Really it should just work like Visual Studio, or at least have an option to choose what do you prefer.

This feature should be implemented, how do I open the file on the left ?
Really it should just work like Visual Studio, or at least have an option to choose what do you prefer.

@virok Easiest way i've found is to put your mouse in the row and then use the scroll wheel up and down. That'll left or right scroll the file tabs.

@virok Easiest way i've found is to put your mouse in the row and then use the scroll wheel up and down. That'll left or right scroll the file tabs.

Is this still being discussed?

Crossing fingers it still is, fortunately not closed yet.
More info at bottom of #12622

As someone having used Visual Studio on a daily basis for more than 10 years, this feature is really lacking in VS Code. In VS2017 I will usually have 3 to 5 pinned tabs which I'm mainly working on at the moment, and the non-pinned tabs are there for minor changes or exploration/reference, so they tend to grow in number.
The way pinned tabs work in VS Code is not satisfactory. I do not want to scroll all the way to the left to see them (by the way there are no arrows to scroll the tabs as @virok said, I need to guess that I'm supposed to use the mousewheel). In compact mode they stay, but the names are too small and it takes space away from the other tabs.

@yannlairdc pinned tabs in VSCode are always the first tabs of the row and never scroll out of view, does that still require a secondary row then?

Is there a setting to ensure that, in current implementation, pinned tabs never scroll out of view?
I'm seeing the pinned tabs scroll out of view which makes the pinned tabs a bit useless in its current form (would really love that separate row for pinned tabs that actually never scrolls out of view).

@ChuckkNorris see #111896

I use this feature constantly in Visual Studio and I am baffled why adding it as an optional item is so controversial. I find it very useful and time saving to have the core tabs I am working on pinned in a separate row so the unpinned tabs are used as I bounce from file to file. Out of all the changes to Visual Studio Code this is the one that I would love to see the most.

The feature will be incredibly useful. I can't wait when it will be finally implemented.

Since tab wrap has recently been added as a feature, I'm thinking it might not be too difficult to implement a pinned tab row? We already have multiple tab rows anyways, all that's needed is to seperate the pinned tabs into the topmost row.

Hi, any updates on this? Its been a while now. Ive been missing this feature for 3 years now since I switched from visual studio

GIVE ME A SECOND TAB ROW, OR GIVE ME DEATH

It is all about comfortable navigation, you don't have to scroll loads of tabs to focus back on what you were on. To have an easy glance of what you were working on made you follow a lot of threads and open many other tabs.
Actually having this option(OPTION) is not comparable to any other solution.
BUT actually I think it is impossible for vscode or having this option is highly expensive. When we are debugging , for example in a reactjs or typescript project, we cannot move the debugging button to any location that we desire and it has a limitted range of relocation. I think that limitations are applied to the tabs too and it is main reason of the vscode team's resistance against this feature.

Here's a workaround for anyone that wants pinned tabs on their own row.
It requires you to use the Custom CSS and JS Loader extension.
I added the following custom CSS:
.tab {
    flex-grow: unset !important;
}

And the following custom JS:
setInterval(() => {
    const tabsContainer = document.querySelector('.tabs-container');
    if (tabsContainer == null) {
        return;
    }

    const tabs = Array.from(tabsContainer.querySelectorAll('.tab'));
    const stickyTabs = Array.from(tabsContainer.querySelectorAll('.tab.sticky'));

    tabs.forEach(tab => {
        tab.style.marginRight = '0';
    });

    stickyTabs[stickyTabs.length - 1].style.marginRight = (
        tabsContainer.clientWidth -
        (stickyTabs[stickyTabs.length - 1].getBoundingClientRect().right - tabsContainer.getBoundingClientRect().left) -
        5
    ) + 'px';
}, 100);

I didn't bother with throttling or anything like that because I didn't notice performance issues, but feel free to add it if you want.
I also have "workbench.editor.wrapTabs": true set in my settings.json.

@JohnKurlak Thanks!!! Awesome!!! How can I make tabs a bit larger?

@JohnKurlak Thanks!!! Awesome!!!
In my case tabs where too small. Solved with this bit:
tab.style.minWidth = "150px";
Result:
tabs.forEach((tab) => {
    tab.style.marginRight = "0";
    tab.style.minWidth = "150px";
  });

Probably better to just put it in the custom.css, e.g.:
.tab {
    flex-grow: unset !important;
    min-width: 150px !important;
}

"workbench.colorCustomizations": {
    "tab.lastPinnedBorder": "#ff0"
},
"workbench.editor.wrapTabs": true

Is a decent compromise if you don't want to fool around too much with customization until (or if) this feature is actually implemented, though I would love it if I could set that border a bit wider so it's easier to spot at a glance.

What's the status on this one? The ease in which you can pin tabs in Visual Studio and have them organized in a separate row is very productive for me, and apparently for a lot of others here as well. I miss it daily in VS Code and easily get lost in all my open tabs. Pinning them on a single row isn't going to work when you pin more than a few files.
I don't really understand why we're having a big discussion about this that drags on for years now. For me it is simple: make it optional :) Or is it technically very difficult? I do understand that we shouldn't make every feature request optional, as that would be bad design. But in this case it's been proven design that is part of the core Visual Studio interface for many years now. People are used to it.
Please bring it to VS Code as well.

@MarByteBeep at this point I think everything comes down to the same situation that happened with the hot reload stuff in the CLI, it is a feature that they wanted to keep on Visual Studio only, maybe that is the same reason why this issue has been stuck for years and no one from the VSCode wants to address it because management does not want VSCode to be at the same quality of some features as VS.

Solution that does not require JS

vscode v1.62.3
extension: Custom CSS and JS Loader v6.0.2

settings.json
{
  "workbench.editor.wrapTabs": true,

  "git.decorations.enabled": false,
  "workbench.editor.decorations.badges": false,
  "workbench.editor.decorations.colors": false,

  "vscode_custom_css.imports": ["file:///C:/temp/VScode/custom.css"],
  "vscode_custom_css.policy": true
}

p.s. In config above I also disabled some git stuff from tabs to make them more readable.. not sure if it affects other places, use with caution.
custom.css
.tabs-container {
  display: flex !important;
  flex-wrap: wrap !important;
  height: auto !important;
}

.tabs-container::before,
.tabs-container::after {
  content: "" !important;
  width: 100% !important;
  order: 1 !important;
}

.tab {
  display: flex !important;
  border-bottom: 2px solid #006cbe !important;
}

/* that is how you locate last pinned tab.
use: tab.lastPinnedBorder to set identifiable color */
.tab[style*="1px solid rgb(0, 0, 0)"]:after {
}

.tab.sticky {
  font-weight: 600 !important;
}

.tab:not(.sticky) {
  order: 1 !important;
}

It's unfortunate to realize that I have spent 3 hours of my life on something that should have been supported out-of-the-box. Hopefully it will save somebody else's time. I also miss the tab height size adjustment functionality very badly..
Result

Solution that does not require JS

The hero we needed that the VS code team couldn't accomplish in several years!

@pavlexander  Fantastic work. This works really well.
One thing I did have to do was execute "Reload Custom CSS and JS" via the Command Palette + Restart VSCode to load the css file, but after I did that it worked perfectly.

PAVLEXANDER
YOU MAGNIFICIANT MAN
https://imgur.com/gallery/ES45L2o
On Thu, Dec 9, 2021 at 5:20 PM welcometochristown @.***>
wrote:

@pavlexander https://github.com/pavlexander Fantastic work. This works
really well.
One thing I did have to do was execute "Reload Custom CSS and JS" via the
Command Palette + Restart VSCode to load the css file, but after I did that
it worked perfectly.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/microsoft/vscode/issues/98160#issuecomment-990393437,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AIUALFZFN2I4WMEAUKFKIRLUQE2UJANCNFSM4NE5HBNQ
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

To be completely honest, I have noticed yesterday during extensive work, that sometimes the top-most row would partially cover itself under the top menu. (like 20% cover). Seems like it happened when the number of rows started to exceed 3 rows.
I have tried to debug the problem but it goes away soon as you resize the window (hence opening the dev tools also fixes the problem...). This is not really a problem if you don't mind pulling the windows down and up from time to time (i.e. change vscode window dimensions). After that manipulation - you can forget about the problem for the hours to come :)
I am sure it has something to do with some flexbox styling and can be fixed by aligning the items in some weird way but it's nearly impossible to debug it. I will reply if/when I found some solution to this problem :) I figured I just could have the dev tools opened at all times :)
p.s. This could also be a problem specific to my personal config since I have other styles applied (tabs height size decreased, borders shown, ..).
Regardless, I am glad this solution is of some use to you, colleagues :) My only wish now that some VSCode dev guy pulls this feature into the main code repo and makes this a feature :)

To be completely honest, I have noticed yesterday during extensive work, that sometimes the top-most row would partially cover itself under the top menu. (like 20% cover).

I ran into the same issue yesterday with the minimal code setup. I don’t have custom borders or custom tab heights configured. I’ll post back if I can figure out a consistent repro.

Thank you for all the effort here! I've been so covered up this week I haven't had time to try this solution. Based on the animation, it looks fantastic!

@JohnKurlak
that would be nice! I have just had a situation when I had devtools opened and the "issue" reoccurring again.. Soon after I hovered over devtools window - the topmost row normalized itself :) So yet again I have no clue how to reproduce the problem.
In theory it has something to do with "height" not readjusting itself correctly in accordance with DOM changes.. as far as my CSS understanding goes - this can be mitigated by setting the height to be auto scalable..
In my latest variation I have added some additional styling to address the issue:
.tabs {
  height: auto !important;
}

.tabs .tabs-and-actions-container {
  height: auto !important;
}

.tabs .tabs-and-actions-container .monaco-scrollable-element {
  height: auto !important;
}

.tabs-container {
  /*...;*/

  flex-direction: row !important;
  justify-content: flex-start !important;
  align-content: flex-start !important;
}

I will update the original post with the new variation. In the very least it does not break the already working functionality :)

@Chifilly
Hey, nice update! Hopefully it fixes it.
FYI. I have also noticed another issue when the scrollbar would go under the bottom console panel:
https://user-images.githubusercontent.com/20644772/161376266-7998edff-d712-44ad-9793-2a5390c15087.mp4
This is another byproduct of pinned tabs styling. You can repro it by opening a relatively large document and simply scrolling down. It looks like the space that is occupied by tabs is somehow appended(/reserving) to the bottom of the page.
Hopefully someone could find a fix for that as well.

Hi @pavlexander I couldn't be happier finding your CSS solution for having second-row-pinned tabs in vscode. but recently I am also using Azure Data Studio, which is quite similar to vscode, but not sure if there was Custom CSS and JS Loader extension that can be added to enable it there, then can apply same solution for pinned tabs.

@pavlexander this is brilliant mate.  I followed the steps in your post and the tabs are now wrapping, but pinned ones are not moving above.  They just dock to the left as per normal.  I am running VSCode 1.68.0 on Windows 10, and I have the extension extension: Custom CSS and JS Loader v6.0.2 installed.  I have restarted VS Code, but still pinned tabs do not move above.  Also VSCode is telling me that "vscode_custom_css.policy" is an Unknown Configuration Setting.  Any ideas on what I could do to get this to work, I am so close...... :-)

@pavlexander I have found a problem where, when I have multiple rows of tabs, pinned or unpinned, the horizontal scroll bar for the code window no longer shows when I mouse over the area where it should be, as it normally does.  If I close enough tabs to get back to a single row, then the horizontal scroll for the code window shows again when I mouse over the area where it should be.  Any ideas on how to solve this one, we are sooooo close.

Since we still have to do this ourselves in 2022 and I prefer having the style directly in settings.json instead of a separate file. Here's the fix formatted in json:
"customizeUI.stylesheet": {
	// Hack to have pinned tab on another row <https://github.com/microsoft/vscode/issues/98160>
    ".title.tabs": "height: auto !important;",
    ".title.tabs .tabs-and-actions-container": "height: auto !important;",
    ".title.tabs .tabs-and-actions-container .monaco-scrollable-element": "height: auto !important;",
    ".title.tabs .tabs-container": "display: flex !important; flex-wrap: wrap !important; height: auto !important; flex-direction: row !important; justify-content: flex-start !important; align-content: flex-start !important;",
    ".title.tabs .tabs-container::before": "content: \"\" !important; width: 100% !important; order: 1 !important;",
    ".title.tabs .tabs-container::after": "content: \"\" !important; width: 100% !important; order: 1 !important;",
    ".title.tabs .tabs-container .tab": "display: flex !important;",
    ".title.tabs .tabs-container .tab:not(.sticky)": "order: 1 !important;",
    ".title.tabs .tabs-container .tab.last-in-row:not(:last-child)": "flex-grow: 0 !important;"
}

Just thought I'd drop an update to the above CSS by @pavlexander to fix an issue that would break the integrated terminal

An alternate way of using this is installing https://windhawk.net/, with the VSCode tweaker extension

Hmm, pinned tabs is implemented but pinning on a seperate row isn't yet?  Grr... this would be very useful.

Since we still have to do this ourselves in 2022 and I prefer having the style directly in settings.json instead of a separate file. Here's the fix formatted in json:
"customizeUI.stylesheet": {
	// Hack to have pinned tab on another row <https://github.com/microsoft/vscode/issues/98160>
    ".title.tabs": "height: auto !important;",
    ".title.tabs .tabs-and-actions-container": "height: auto !important;",
    ".title.tabs .tabs-and-actions-container .monaco-scrollable-element": "height: auto !important;",
    ".title.tabs .tabs-container": "display: flex !important; flex-wrap: wrap !important; height: auto !important; flex-direction: row !important; justify-content: flex-start !important; align-content: flex-start !important;",
    ".title.tabs .tabs-container::before": "content: \"\" !important; width: 100% !important; order: 1 !important;",
    ".title.tabs .tabs-container::after": "content: \"\" !important; width: 100% !important; order: 1 !important;",
    ".title.tabs .tabs-container .tab": "display: flex !important;",
    ".title.tabs .tabs-container .tab:not(.sticky)": "order: 1 !important;",
    ".title.tabs .tabs-container .tab.last-in-row:not(:last-child)": "flex-grow: 0 !important;"
}

For some reason this, and the other CSS hacks to get the pinned tabs in a new row, cause my horizontal scrollbar along the bottom of editors to disappear. :-(

Coming from Visual Studio, would REALLY love to see this in Code!

Coming from IntelliJ, would REALLY love to see this in Code!

This is probably the most significant usability improvement that can be made in the realm of code navigation.
I feel the need for this feature scales linearly with the number of files in the project.

Looking at the above Iteration Plan for September 2023:
This puppy is actually planned to get implemented this month??? I haven't been this excited since Avengers Endgame came out. Haven't waited this long for something since the last Avatar movie came out.

In todays insider we released a new setting for pinned editor tabs: workbench.editor.pinnedTabsOnSeparateRow
This will display all your pinned tabs on a tab bar above your regular tabs.
Please give it a try and let me know what you think.
You can give our preview releases a try from: https://code.visualstudio.com/insiders/

Works great! Pinned tabs will finally be useful, thank you!

this is fantastic🎉
but is there a way I can pin multiple tabs simultaneously?

Multi-select of tabs is covered in: https://github.com/microsoft/vscode/issues/119327

In insider I couldn't access the tab
Tabs are half hidden
though I haven't turn on this setting
but I thought could be this that might have affect the tabs in certain cases

Can you create an issue? Thanks!

8 years later and it's finally here! Hurrah!
Better late than never 😄  Thank you!

Amazing, so glad to see this !! Any timeline on when this gets GA'ed into the stable version of Code ?

@prasad-kamat expect it before the end of next week. See https://github.com/microsoft/vscode/issues/193938

Just got this today in an update.  Thanks so all for this journey.

Yaayyyy. Its everything I dreamed it would be!
I can't believe how long the journey has been but we finally made it! :)


Update prevents VLAN0?

I'm trying to be bug an issue that just happened. I updated my controller software and now my internet that has been so reliable doesn't work.
I suspect the controller update included a firmware update to the USG because internet died shortly after it updated. Or it could have been a reprovision. Or it could have just been a restart.
Anyway, now I'm in my USG, and ifconfig doesn't show a second VLAN for eth0, my WAN device.
Running eap_proxy.py, I get errors:

root@router:~# /usr/bin/python /config/scripts/eap_proxy.py eth0 eth2 --restart-dhcp --ignore-when-wan-up --ignore-logoff --ping-gateway --set-mac --vlan-id 0 --pidfile /var/run/eap_proxy.pid --debug-packets
[2022-11-30 18:25:43,982]: starting proxy_loop
[2022-11-30 18:25:47,176]: eth2: recv 60 bytes:
0x0000:  0180 c200 0003 8c5a 25db 5131 888e 0201
0x0010:  0000 0000 0000 0000 0000 0000 0000 0000
0x0020:  0000 0000 0000 0000 0000 0000 0000 0000
0x0030:  0000 0000 0000 0000 0000 0000
[2022-11-30 18:25:47,179]: eth2: 8c:5a:25:db:51:31 > 01:80:c2:00:00:03, EAPOL start (1) v2, len 0
Traceback (most recent call last):
  File "/config/scripts/eap_proxy.py", line 869, in <module>
    sys.exit(main())
  File "/config/scripts/eap_proxy.py", line 865, in main
    return proxy(args, log)
  File "/config/scripts/eap_proxy.py", line 817, in proxy_loop
    proxy.proxy_loop()
  File "/config/scripts/eap_proxy.py", line 602, in proxy_loop
    self.on_poll_event(socks[fd], event)
  File "/config/scripts/eap_proxy.py", line 623, in on_poll_event
    self.on_router_eap(eap)
  File "/config/scripts/eap_proxy.py", line 651, in on_router_eap
    if self.os.getmac(if_vlan) == eap.src:
  File "/config/scripts/eap_proxy.py", line 478, in getmac
    return getifhwaddr(ifname)
  File "/config/scripts/eap_proxy.py", line 180, in getifhwaddr
    with open("/sys/class/net/%s/address" % ifname) as f:
IOError: [Errno 2] No such file or directory: '/sys/class/net/eth0.0/address'

Looks like the vlan isn't getting set for some reason. Any ideas?

Looking into this more, VLAN 0 is being created... So that may not be it.
I managed to get it to connect once, but I killed the proxy binary and then something else reset and I haven't managed to get it back.
Playing with code trying to see how far it's getting, it doesn't seem to get past this poll.poll():
https://github.com/jaysoffian/eap_proxy/blob/78a058ffe67c253da5153d82a3a729588966f02e/eap_proxy.py
I'm not super familiar with python put I'll try to dig into this some more

Restarting the ONT seems to have gotten the polling to run. Maybe it was just waiting for events the whole time. It's still not 100% clear to what what has broken here, or why those events would have stopped coming in.

Well, in the end, it looks like it was a confluence of things. Posting here in case this helps someone in the future, but this doesn't seem to have been an eap_proxy issue.

Unifi seems to be pushing to not allow VLAN 0 directly. There seems to be some conflict with QoS tags or something. I'm not super familiar with this low level ethernet functionality. Maybe I'll read some more and update this comment.
AT&T seems to have added some restrictions - if you try talking to them without securing the connection first or on the wrong VLAN, they stop trying to reconnect until you cycle the physical link. I think this is why I thought eap_proxy was hanging - AT&T was just not starting the authentication process.
The not working uplink meant many client devices would self disconnect from the WiFi to use GSM, then get back onto WiFi with a new generated MAC address, and the local DHCP pool was exhausted, making me think something more significant had changed/broken on the USG.


[BUG] Dark mode issue in email form on premium website

Description
Email form does not support dark mode
Screenshots

Additional information
No response

I will try to remove the background in the Kartra form builder but I don't think it looks good

Thanks for raising Tom! The best I could do it make it 50% transparent, so it doesn't show in light mode, but in dark mode the text is still readable


handle "global" side-nav and catalog's side-nav

On smaller screens, the top navigation items (About/Installation/Project Status/Help) are hidden and displayed in a side-nav instead.
The component catalog also brings its own side-nav with navigation to component samples.
These two side-navs don't work well together.

component side-nav introduces a padding on the left edge which doesn't disappear oin small screens
the global side-nav's hamburger icon button (how is that thing called really?) only shows that global side-nav
the component side-nav is only accessible from the "samples" folder

Either merge these two side-navs into one global fixed side-nav or display one of them on the right edge instead of the left edge.
Right now, I'd prefer merging.

All of our catalog apps are direct descendants of Aurelia Navigation Skeleton, which uses some of the Bootstrap features (navbar and layout on some pages).
<template bindable="router">

      <button type="button" class="navbar-toggle" data-toggle="collapse" data-target="#skeleton-navigation-navbar-collapse">
        Toggle Navigation

      </button>

        ${router.title}

          ${row.title}

</template>

The Project status page uses the bootstrap row/col layout which also behaves as you described above.

So, what would you do if you had two navbar-collapse? That is, if I got Bootstrap right (which need not be the case). Combine ("merge") them or put them in distinct places?
The point is really: how to best combine two different collapsing/collapsed navigation panels?
Btw: there is no Bootstrap in this bridge anymore. :smile:


Creating data_extraction code

This issue focusses on building the code to extract the needed data from: https://datasets.imdbws.com/.
As mentioned in our team meeting in week 2 we chose to work with the datasets from imdbws.com. In addition, we concluded that we needed data about the ratings, episodes and genres. Meaning that we need to download the following files: "title.ratings.tsv.gz", "title.basics.tsv.gz" and "title.episode.tsv.gz".
To automate this proces code was written for the assignment in week 2. Meaning that the code needs to be placed at the right location in the new structure. In the assignment the code needed is placed on line 36 to 64, so these lines need to be copied to "Data_Extraction". Do not forget to check if it still runs as intended.
Kind regards,
Julian Peters

Added the code :)

The code isn't showing. Check the location you pushed it to.
Kind regards,
Julian Peters


Add Bemidji Fun Walk/Run Route

Contact: Geoff

Because Geoff is awesome, and already ahead of the game
2017_FunRunMap.pdf


I wanna add Mandarin translation in the zoneRegex under ui/raidboss/data//.js

I wanna add Mandarin translation in the zoneRegex under ui/raidboss/data/*/*.js
This will not change any existing functions and all I wanna do is to add cn: /^...$/ to raidboss js files.
Is that OK? I'll make a pull request when the works done.

Thanks for the pull request! Feel free to add translations wherever they are missed!


ServiceNotFoundException: lexik_jwt_authentication.handler.authentication_success

Hello,
I followed a documentation over here: https://github.com/lexik/LexikJWTAuthenticationBundle/blob/master/Resources/doc/7-manual-token-creation.md and tried to implement the last part with handleAuthenticationSuccess method. However, it seems it's not working in Symfony 4 anymore.
The "lexik_jwt_authentication.handler.authentication_success" service or alias has been removed or inlined when the container was compiled. You should either make it public, or stop using the container directly and use dependency injection instead.
So I tried to inject it in controller's constructor, but...
Cannot autowire service "App\Controller\UserController": argument "$authenticationSuccessHandler" of method "__construct()" references class "Lexik\Bundle\JWTAuthenticationBundle\Security\Http\Authentication\AuthenticationSuccessHandler" but no such service exists. It cannot be auto-registered because it is from a different root namespace.
It was necessary to add these four lines to config/services.yaml:
Lexik\Bundle\JWTAuthenticationBundle\Services\JWTManager:
        autowire: true

Lexik\Bundle\JWTAuthenticationBundle\Security\Http\Authentication\AuthenticationSuccessHandler:
        autowire: true

Is this actually the right way of doing this? It feels a bit hacky to me, but I am not really a Symfony expert :) In any way, the documentation seems to be obsolete in this case.

Thanks for reporting this bug, I will try to find the best approach for fixing it. Keep you informed

Fixed by #465

Please open a new issue with a full reproducer.


[BUG] All my apps suddenly stopped working

Describe the bug
All my apps stopped working. They all show an error but some apps show different error codes.
Instructions to reproduce
Just try to open an app.
What version of LiveContainer are you using?
Version 2.0-release (main/ebea623)
Other info
Error codes :dlopen(/var/mobile/Containers/Data/Application/448194FB-B4F4-4081-9A07-7C859BB3C629/Documents/Applications/SimplePlanes.app/SimplePlanes, 0x0109): tried: '/var/mobile/Containers/Data/Application/448194FB-B4F4-4081-9A07-7C859BB3C629/Documents/Applications/SimplePlanes.app/SimplePlanes' (code signature invalid in <6DCA2CBE-B5FB-38A9-B7A8-D33C36D3A4E9> '/private/var/mobile/Containers/Data/Application/448194FB-B4F4-4081-9A07-7C859BB3C629/Documents/Applications/SimplePlanes.app/SimplePlanes' (errno=85) sliceOffset=0x00000000, codeBlobOffset=0x0000C330, codeBlobSize=0x000008B0), '/private/preboot/Cryptexes/OS/var/mobile/Containers/Data/Application/448194FB-B4F4-4081-9A07-7C859BB3C629/Documents/Applications/SimplePlanes.app/SimplePlanes' (no such file), '/var/mobile/Containers/Data/Application/448194FB-B4F4-4081-9A07-7C859BB3C629/Documents/Applications/SimplePlanes.app/SimplePlanes' (code signature invalid in <6DCA2CBE-B5FB-38A9-B7A8-D33C36D3A4E9> '/private/var/mobile/Containers/Data/Application/448194FB-B4F4-4081-9A07-7C859BB3C629/Documents/Applications/SimplePlanes.app/SimplePlanes' (errno=85) sliceOffset=0x00000000, codeBlobOffset=0x0000C330, codeBlobSize=0x000008B0), '/private/var/mobile/Containers/Data/Application/448194FB-B4F4-4081-9A07-7C859BB3C629/Documents/Applications/SimplePlanes.app/SimplePlanes' (code signature invalid in <6DCA2CBE-B5FB-38A9-B7A8-D33C36D3A4E9> '/private/var/mobile/Containers/Data/Application/448194FB-B4F4-4081-9A07-7C859BB3C629/Documents/Applications/SimplePlanes.app/SimplePlanes' (errno=85) sliceOffset=0x00000000, codeBlobOffset=0x0000C330, codeBlobSize=0x000008B0), '/private/preboot/Cryptexes/OS/private/var/mobile/Containers/Data/Application/448194FB-B4F4-4081-9A07-7C859BB3C629/Documents/Applications/SimplePlanes.app/SimplePlanes' (no such file), '/private/var/mobile/Containers/Data/Application/448194FB-B4F4-4081-9A07-7C859BB3C629/Documents/Applications/SimplePlanes.app/SimplePlanes' (code signature invalid in <6DCA2CBE-B5FB-38A9-B7A8-D33C36D3A4E9> '/private/var/mobile/Containers/Data/Application/448194FB-B4F4-4081-9A07-7C859BB3C629/Documents/Applications/SimplePlanes.app/SimplePlanes' (errno=85) sliceOffset=0x00000000, codeBlobOffset=0x0000C330, codeBlobSize=0x000008B0).            AND ALSO:The bundle “OpenSSL” couldn’t be loaded.

Refresh both LiveContainer, SideStore then refresh the certificate and try again.

Refresh both LiveContainer, SideStore then refresh the certificate and try again.

I'm experiencing the same problem. And I've tried renewing all the certificates(including Sidestore, Livecontainer and tapping the renew jit-less certificates section). All the operations have done smoothly. But it doesn't work.
Then I try to uninstall Livecontainer and SideStore and reinstall them and reconfigure jit-less in Livecontainer. But it still popup the same error message when I launching the app in Livecontainer.
BTW. It just working fine two days ago. And I saw this error message first time when I reinstalled Sidestore because it expired,

Update. I tap the remove all apps button in VPN & Device Management -> my Apple ID  page to erasing all the app and data which is signed from my Apple ID. And then reinstall every thing (Sidestore, Livecontainer…).
After that, configure jit-less and install app in Livecontainer, it works.


[Enhancement] Support Analyze to collect statistics of replace type aggregated columns

What type of PR is this：

[ ] BugFix
[ ] Feature
[x] Enhancement
[ ] Refactor
[ ] UT
[ ] Doc
[ ] Tool

Which issues of this PR fixes ：

Fixes #12766
Problem Summary(Required) ：

Checklist:

[ ] I have added test cases for my bug fix or my new feature
[ ] I have added user document for my new feature or new function

BE Build Success

[FE PR Coverage Check]
:heart_eyes: pass : 11 / 11 (100.00%)
file detail

path
covered_line
new_line
coverage
not_covered_line_detail

:large_blue_circle:
com/starrocks/sql/ast/ShowAnalyzeStatusStmt.java
1
1
100.00%
[]

:large_blue_circle:
com/starrocks/sql/ast/ShowBasicStatsMetaStmt.java
1
1
100.00%
[]

:large_blue_circle:
com/starrocks/statistic/StatisticUtils.java
8
8
100.00%
[]

:large_blue_circle:
com/starrocks/statistic/StatisticsCollectJobFactory.java
1
1
100.00%
[]

BE Build Success

run starrocks_admit_test

run starrocks_admit_test


Support java 11 http2 client gh-689

Support java 11 http2 client  gh-689

Fixes gh-689.

hello @OlgaMaciaszek , Thank you for your advice

Thanks for the changes @galaxy-sea. This last one still needs to be addressed.


Add a link graph node position setting

Description
I'd love to see a option in the right click menu to bind graph node positions together when multiple graph nodes are selected. This is an ergonomics/nice to have thing so that once you are happy with the relative positioning of a group of event nodes, you don't have to group select them all to move the group around.
The intended behavior would be that if I move one node when bound to one or more other nodes, all the nodes would move; just like what happens when multiple graph nodes are selected. I'd also like this behavior to apply to comments too. Ideally this would be hierarchical in the sense that a bound node group could be bound to another node group and then that mega group could be bound to a larger group, etc.
Implementation ideas
No response

So you can already do this with comment nodes, although it may not be well documented. If you double-click the comment node title bar, all nodes within the comment node's frame are selected, and moving the comment node will move all the nodes within the comment together.
There is a new GraphFrame implementation based on the "frame" concept in the Visual Shader that we'll be implementing soon, which should make this more prominent and easier to work with.
As for right-click and binding nodes, would you be opposed that doing that would construct a comment/frame node around the selected nodes?

Oh, I just tried that with a comment node and I like the way that works! I'll start using that immediately for my sanity.
That frame sounds like a really good solution to both define the exact confines of what is bound together visually and a easy thing to learn with with perhaps a "Graph Frame" node in the utilities list. I think I'd use frames in a lot of situations where I'd just have awkwardly blank comment nodes.
I can't speak for everyone of course, but I think the value is there for me personally in it's creation. Ideally it would work almost exactly how the comment node works now, but no title bar for the node and no properties in the inspector except background color. Just a rectangle that, when selected (LMB single click), selects all the nodes inside it.
Let me know what I can do to help. I'm not a C++ wiz, so a feature PR is probably a bit beyond me. I could definitely compile some test builds on MacOS and Linux and feature/bug test though.


Migrate from using flag to pflag

The reason for this change is to be consistent with the flags that are used by genericapiserver.

This change is

cc @piosz

I think this looks good to me.  I'm glad we're explicitly passing around the options object instead of using globals now.  I'd prefer if the commit had a description (e.g. "This commit brings Heapster inline with the conventions used by Kubernetes' GenericAPIServer")

@DirectXMan12 thanks for having a look! I updated the description and title here and in the commit message.

Let's wait with merging this until we cut release-1.2 branch (hopefully tomorrow).

The branch is cut now. Please update godeps.

LGTM. Will merge once you will update Kubernetes version in godeps


Ruff fails to detect invalid Python syntax in f-string with single quotes

I found an issue where Ruff doesn't detect a syntax error in f-string. Consider this simple source:
import os

location = f's3://{os.environ.get('bucket', 'dev')}'

Ruff reports no issues:
$ uv tool run ruff check ruff_bug.py
All checks passed!
$ uv tool run ruff format ruff_bug.py
1 file left unchanged

However, the module is invalid:
$ python ruff_bug.py
  File "ruff_bug.py", line 3
    location = f's3://{os.environ.get('bucket', 'dev')}'
                                       ^
SyntaxError: invalid syntax

Tested in ruff 0.5.1 and 0.6.4.  It seems like Ruff isn't catching this invalid syntax with mismatched quotes in the f-string.
Is this an expected behaviour?

Yes, and no.
It's expected behavior because this is a known limitation of Ruff and there have been a few related issues like https://github.com/astral-sh/ruff/issues/10163
But no, because we agree that this is not a good user experience. The problem is that our parser currently doesn't respect the project's target version. It always uses the latest syntax which allows using the same quotes for the f-string and inner expressions. There's an issue to either make the parser target-version aware OR create a lint rule that flags all syntax that isn't compatible with the project's target version.
https://github.com/astral-sh/ruff/issues/6591


added redux and redux thunk dependencies

Description
added dependencies that were not in JSON file since we know for sure that we will be using them later on.
https://www.loom.com/share/4cce8accb6ad4274b7bd58ba010a79b5
Fixes # (issue)
sets things up in the future
Type of change
Please delete options that are not relevant.

[x] New feature (non-breaking change which adds functionality)

Checklist:

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[x] I have removed unnecessary comments/console logs from my code
[x] I have made corresponding changes to the documentation if necessary (optional)
[x] My changes generate no new warnings
[x] I have checked my code and corrected any misspellings
[x] No duplicate code left within changed files
[x] Size of pull request kept to a minimum
[x] Pull request description clearly describes changes made & motivations for said changes

@collerhatguy @paulstgermain I'm just now seeing this PR, and I'm not sure why it was merged nor made as there have been no decisions on state management with our FE crew, nor has any decision been passed by the TPMs. We have ContextAPI as an option as well and I have not heard a sell for either. Just adding deps is not a thorough or lean aligning PR, especially without discussion with the team.


Can't create account with len > 13

Is 13 symbols - max possible length?
2155421ms thread-0   main.cpp:1101                 main                 ] Failed with error: 10 assert_exception: Assert Exception
len <= 13:
    {}
    thread-0  native.hpp:111 set

Yes, 13 chars of: .12345abcdefghijklmnopqrstuvwxyz


'connectionString' is a required parameter and must be of type: 'string'.

I had set the connection string and env variables in the launch.json in visual studio code.
When I debug I get the error

<EMAIL_ADDRESS>start C:\Users\Nithish.Kalakonda\Desktop\web-apps-node-iot-hub-data-visualization
node server.js

Listening on 3000.
'connectionString' is a required parameter and must be of type: 'string'.

Hi nithishk -
Your launch.json should look like this. Does it?
{
    "version": "0.2.0",
    "configurations": [
        {
            "type": "node",
            "request": "launch",
            "name": "Launch Program",
            "program": "${workspaceFolder}\\server.js",
            "protocol": "inspector",
            "smartStep": true,
            "showAsyncStacks": true,
            "sourceMaps": true,
            "env": {
                "NODE_ENV": "local",
                "IotHubConnectionString": "HostName=redacted;SharedAccessKeyName=service;SharedAccessKey=redacted",
                "EventHubConsumerGroup": "monitoring"
            }
        }
    ]
}

Regards,
David

Yes it does. I removed my ConnectionString here 👍
{
// Use IntelliSense to learn about possible attributes.
// Hover to view descriptions of existing attributes.
// For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
"version": "0.2.0",
"configurations": [
    {
        "type": "node",
        "request": "launch",
        "name": "Launch Program",
        "program": "${workspaceFolder}\\server.js",
        "protocol": "inspector",
        "smartStep": true,
        "showAsyncStacks": true,
        "sourceMaps": true,
        "env": {
           // "NODE_ENV": "local",
           // "IotHubConnectionString": "",
            //"EventHubConsumerGroup": "gobconsum"
        }
    }
]

}

In your json snippet above, the 3 env properties are remarked out. Can you confirm if you get the error with the slashes removed?

yes I do

Seems I have used wrong connection string from device instead of the HUB

Seems I have used wrong connection string from device instead of the HUB

Thanks Nithishk. Let us know if you run into any other problems with the sample, or if you think it could be improved in any way.


Allow latest node to fail

In Travis, the latest LTS version of node (8.11.1) passes the build, but the latest version of node (10.0.0) doesn't, because node-sass doesn't support it yet. I think we can ignore this failure for now.
With this PR, Travis will still keep building on the latest node version, but won't call it a fail if that build fails.

Just realized that it's a bad idea to merge from my master copy. Recreated at #236.


thunderpush pidfile

I use thunderpush a week ago and would monitor the process with monit but found the pidfile, it is created?
Thanks

https://github.com/thunderpush/thunderpush/pull/47

Available in 1.0.1


ambient: introduce Service for waypoint proxy

This is only step 1, some TODOs left in comments to do this properly.
Currently, we rely on assumptions that waypoints are implemented in the cluster as pods with specific labels. Really, Waypoints can be anything

the Gateway already has a way to reference how to reach the Gateway, and we should use that.

As step one of this, we give each Waypoint a headless service and associate it with the Gateway in the status. This at least allows some other implementation to properly reach a waypoint just by reading the Gateway status.
Please provide a description of this PR:

@howardjohn: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

unit-tests-arm64_istio
bdc51cb8e13c94c545d73cd406d2914d6c301d06
link
true
/test unit-tests-arm64

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@howardjohn: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

unit-tests-arm64_istio
bdc51cb8e13c94c545d73cd406d2914d6c301d06
link
true
/test unit-tests-arm64

gencheck_istio
bdc51cb8e13c94c545d73cd406d2914d6c301d06
link
true
/test gencheck

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@howardjohn: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

unit-tests-arm64_istio
bdc51cb8e13c94c545d73cd406d2914d6c301d06
link
true
/test unit-tests-arm64

gencheck_istio
bdc51cb8e13c94c545d73cd406d2914d6c301d06
link
true
/test gencheck

unit-tests_istio
bdc51cb8e13c94c545d73cd406d2914d6c301d06
link
true
/test unit-tests

integ-pilot-istiodremote_istio
bdc51cb8e13c94c545d73cd406d2914d6c301d06
link
true
/test integ-pilot-istiodremote

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@howardjohn: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

unit-tests-arm64_istio
bdc51cb8e13c94c545d73cd406d2914d6c301d06
link
true
/test unit-tests-arm64

gencheck_istio
bdc51cb8e13c94c545d73cd406d2914d6c301d06
link
true
/test gencheck

unit-tests_istio
bdc51cb8e13c94c545d73cd406d2914d6c301d06
link
true
/test unit-tests

integ-pilot-istiodremote_istio
bdc51cb8e13c94c545d73cd406d2914d6c301d06
link
true
/test integ-pilot-istiodremote

integ-cni_istio
bdc51cb8e13c94c545d73cd406d2914d6c301d06
link
true
/test integ-cni

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@howardjohn: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

unit-tests-arm64_istio
bdc51cb8e13c94c545d73cd406d2914d6c301d06
link
true
/test unit-tests-arm64

gencheck_istio
bdc51cb8e13c94c545d73cd406d2914d6c301d06
link
true
/test gencheck

unit-tests_istio
bdc51cb8e13c94c545d73cd406d2914d6c301d06
link
true
/test unit-tests

integ-pilot-istiodremote_istio
bdc51cb8e13c94c545d73cd406d2914d6c301d06
link
true
/test integ-pilot-istiodremote

integ-cni_istio
bdc51cb8e13c94c545d73cd406d2914d6c301d06
link
true
/test integ-cni

integ-pilot_istio
bdc51cb8e13c94c545d73cd406d2914d6c301d06
link
true
/test integ-pilot

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

I am not sure i understand what it resolves. Current limitation is cannot lb across waypoints, if there are multi vips of the new waypoint service. The problem is still.

I'm not sure I understand the root cause for this limitation - why can't LB
across waypoints ? It is the same LB model do have for multi-network.
With ambient we want to simplify things - not sure we need to support all
the options at once, so maybe if there are local waypoints in
same cluster - we just use the endpoints and ignore everything else. If
there are no endpoints - maybe we can restrict it to a single
local VIP (for cluster.local) and multiple remote ones.
On Sun, Mar 12, 2023 at 6:28 PM Zhonghu Xu @.***> wrote:

I am not sure i understand what it resolves. Current limitation is cannot
lb across waypoints, if there are multi vips of the new waypoint service.
The problem is still.
—
Reply to this email directly, view it on GitHub
https://github.com/istio/istio/pull/43889#issuecomment-1465371888, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAAUR2T4BTK5ARCOYZZUUSLW3ZZ3NANCNFSM6AAAAAAVW6FOOE
.
You are receiving this because your review was requested.Message ID:
@.***>

VIP allows load balancing between Waypoints in a simple way. The issue is
that HBONE is a session-based protocol, so it'll stick to an endpoint for
far longer than regular HTTP requests. There is an easy way around it, by
putting a short limit on the length of the stream, for example.
What I think is too complicated, is trying to load balance between
Waypoints while load balancing L7 destinations. E.g. use one waypoint for
one destination pod and another waypoint for another destination pod
(assuming both pods share waypoints). That's unnecessarily complicated IMHO.
On Sun, Mar 12, 2023 at 9:36 PM Costin Manolache @.***>
wrote:

I'm not sure I understand the root cause for this limitation - why can't LB
across waypoints ? It is the same LB model do have for multi-network.
With ambient we want to simplify things - not sure we need to support all
the options at once, so maybe if there are local waypoints in
same cluster - we just use the endpoints and ignore everything else. If
there are no endpoints - maybe we can restrict it to a single
local VIP (for cluster.local) and multiple remote ones.
On Sun, Mar 12, 2023 at 6:28 PM Zhonghu Xu @.***> wrote:

I am not sure i understand what it resolves. Current limitation is cannot
lb across waypoints, if there are multi vips of the new waypoint service.
The problem is still.
—
Reply to this email directly, view it on GitHub
https://github.com/istio/istio/pull/43889#issuecomment-1465371888, or
unsubscribe
<
https://github.com/notifications/unsubscribe-auth/AAAUR2T4BTK5ARCOYZZUUSLW3ZZ3NANCNFSM6AAAAAAVW6FOOE
.
You are receiving this because your review was requested.Message ID:
@.***>

—
Reply to this email directly, view it on GitHub
https://github.com/istio/istio/pull/43889#issuecomment-1465498719, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/ACIYRRQZBTLRRKVMZSVNRZ3W32P6NANCNFSM6AAAAAAVW6FOOE
.
You are receiving this because your review was requested.Message ID:
@.***>

On Sun, Mar 12, 2023 at 9:44 PM Kuat @.***> wrote:

VIP allows load balancing between Waypoints in a simple way. The issue is
that HBONE is a session-based protocol, so it'll stick to an endpoint for
far longer than regular HTTP requests. There is an easy way around it, by
putting a short limit on the length of the stream, for example.

That's a feature - not a bug. It's the job of the waypoint to load balance.
Clients who need can open multiple h2 connections, max concurrent requests
can also be used
to tune that.
It won't be different with an external/managed LB - when clients connect to
google they
also get a mostly sticky session with a frontend.
What I think is too complicated, is trying to load balance between

Waypoints while load balancing L7 destinations. E.g. use one waypoint for
one destination pod and another waypoint for another destination pod
(assuming both pods share waypoints). That's unnecessarily complicated
IMHO.

Waypoints need to do the real LB, I don't think load balancing across
waypoints is a real
problem. We may want to add some 'GOAWAY' if a waypoint is at the limit,
and adjust max connections -
but if you consider that all 'out of cluster' LBs will not typically
provide any of this ( or maybe they
will, it may be out of our control ) - and most of the internet exposed
services work in the same model,
I think we can keep it pretty simple.

On Sun, Mar 12, 2023 at 9:36 PM Costin Manolache @.***>
wrote:

I'm not sure I understand the root cause for this limitation - why can't
LB
across waypoints ? It is the same LB model do have for multi-network.
With ambient we want to simplify things - not sure we need to support all
the options at once, so maybe if there are local waypoints in
same cluster - we just use the endpoints and ignore everything else. If
there are no endpoints - maybe we can restrict it to a single
local VIP (for cluster.local) and multiple remote ones.
On Sun, Mar 12, 2023 at 6:28 PM Zhonghu Xu @.***> wrote:

I am not sure i understand what it resolves. Current limitation is
cannot
lb across waypoints, if there are multi vips of the new waypoint
service.
The problem is still.
—
Reply to this email directly, view it on GitHub
https://github.com/istio/istio/pull/43889#issuecomment-1465371888,
or
unsubscribe
<

https://github.com/notifications/unsubscribe-auth/AAAUR2T4BTK5ARCOYZZUUSLW3ZZ3NANCNFSM6AAAAAAVW6FOOE

.
You are receiving this because your review was requested.Message ID:
@.***>

—
Reply to this email directly, view it on GitHub
https://github.com/istio/istio/pull/43889#issuecomment-1465498719, or
unsubscribe
<
https://github.com/notifications/unsubscribe-auth/ACIYRRQZBTLRRKVMZSVNRZ3W32P6NANCNFSM6AAAAAAVW6FOOE
.
You are receiving this because your review was requested.Message ID:
@.***>

—
Reply to this email directly, view it on GitHub
https://github.com/istio/istio/pull/43889#issuecomment-1465502738, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAAUR2QGY4TI7UUBLLX25YTW32Q3LANCNFSM6AAAAAAVW6FOOE
.
You are receiving this because your review was requested.Message ID:
@.***>

@howardjohn: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

unit-tests-arm64_istio
bdc51cb8e13c94c545d73cd406d2914d6c301d06
link
true
/test unit-tests-arm64

gencheck_istio
bdc51cb8e13c94c545d73cd406d2914d6c301d06
link
true
/test gencheck

unit-tests_istio
bdc51cb8e13c94c545d73cd406d2914d6c301d06
link
true
/test unit-tests

integ-pilot-istiodremote_istio
bdc51cb8e13c94c545d73cd406d2914d6c301d06
link
true
/test integ-pilot-istiodremote

integ-cni_istio
bdc51cb8e13c94c545d73cd406d2914d6c301d06
link
true
/test integ-cni

integ-pilot_istio
bdc51cb8e13c94c545d73cd406d2914d6c301d06
link
true
/test integ-pilot

integ-security-multicluster_istio
fbde9a7dde4ba08c8c4faee2f61e24244e1fab35
link
true
/test integ-security-multicluster

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/retest

@howardjohn: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

integ-security-multicluster_istio
fbde9a7dde4ba08c8c4faee2f61e24244e1fab35
link
true
/test integ-security-multicluster

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/retest
On Mon, Mar 13, 2023 at 6:32 PM Istio Automation @.***>
wrote:

@howardjohn https://github.com/howardjohn: The following test failed,
say /retest to rerun all failed tests or /retest-required to rerun all
mandatory failed tests:
Test name Commit Details Required Rerun command
integ-security-multicluster_istio fbde9a7
https://github.com/istio/istio/commit/fbde9a7dde4ba08c8c4faee2f61e24244e1fab35
link
https://prow.istio.io/view/gs/istio-prow/pr-logs/pull/istio_istio/43889/integ-security-multicluster_istio/1635424871926206464
true /test integ-security-multicluster
Instructions for interacting with me using PR comments are available here
https://git.k8s.io/community/contributors/guide/pull-requests.md. If
you have questions or suggestions related to my behavior, please file an
issue against the kubernetes/test-infra
https://github.com/kubernetes/test-infra/issues/new?title=Prow issue:
repository. I understand the commands that are listed here
https://go.k8s.io/bot-commands.
—
Reply to this email directly, view it on GitHub
https://github.com/istio/istio/pull/43889#issuecomment-1467208812, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAEYGXK2FYOUD4PO4RT7M3TW37DB5ANCNFSM6AAAAAAVW6FOOE
.
You are receiving this because you were mentioned.Message ID:
@.***>

@howardjohn I think that's our request- add a VIP and we can use that over directly waypoint IP in the clients.

VIP doesn't work in multicluster as someone mentioned in the other PR. I am not opposed to adding it but I don't think its appropriate for LB

@howardjohn I don't understand - all Google frontends have static IPs at the edge, and that works fine globally. Why would VIP break multi-cluster?

To continue analogy, you don't expect a local gRPC client to resolve Google's load balancer pool between L1 GFEs - why would we require this for Ambient? Any client should be able to use Waypoint VIP to use Waypoint features by going directly to it.

curl VIP will use kube-proxy routing which is single cluster. So a client in cluster-a cannot reach a waypoint in cluster-b.
Imagine we have Team-A-Cluster and Team-B-Cluster. Team A calls Team B. Team B only runs workloads in Team-B-Cluster, so they will only have a waypoint in Team-B-Cluster. Team-A cannot access that by a VIP.

That effectively makes multi-cluster the same as multi-network, right?
I am not sure why that makes sense for traffic to a waypoint but not for traffic to a workload directly?

Because waypoints are services and not workloads? It seems that k8s doesn't have a "flat" service IPAM in multi-cluser? One cluster cannot assume another cluster implements waypoints in-cluster or via some other cluster?

I mean when we connect to httpbin - we use EDS to do clientside LB. But now for waypoints we want to use the VIP? It feels like they are the same problem.
Just to be clear, I am only suggesting we do this when a waypoint points to a Service - if its an opaque address like waypoint.example.com we can just go directly to that (DNS resolution may be tricky though, but can be solved).

It seems that k8s doesn't have a "flat" service IPAM in multi-cluser?

Correct, only pods

The only way to do that, as far as I can tell, is sandwich. kube-proxy is not going to become multi-cluster. do you think we should block this in favor of sandwich

The point me and Costin are making - let's not special case Waypoint clients in any way with custom EDS. The problem is the same when using any service VIP with multi-cluster, so the solution should be the same. It's also a problem only with multi-cluster, so you could separate data paths easily, and use VIP directly for in-cluster waypoints, and punt on kube-proxy/MCS/ztunnel for cross-cluster waypoints.

The point me and Costin are making - let's not special case Waypoint clients in any way with custom EDS. The problem is the same when using any service VIP with multi-cluster, so the solution should be the same. It's also a problem only with multi-cluster, so you could separate data paths easily, and use VIP directly for in-cluster waypoints, and punt on kube-proxy/MCS/ztunnel for cross-cluster waypoints.

Well, I generally agree we should use the same solution.. but the solution we use for those is EDS...
kube-proxy and MCS are not viable options; ztunnel is but requires sandwich.

I doubt this is  going to happen in upstream k8s. Even MCS isn't upstream,
it's a custom extension that is not deployed by default on any K8s platform
that I know of, and it still doesn't meet our needs.
On Tue, Mar 14, 2023 at 10:34 AM Kuat @.***> wrote:

Let's not use "sandwich", that means too many things to confuse everyone.
We need to implement multi-cluster VIPs somehow, and if that requires a
'bump" with ztunnel, that's OK, but ideally, we need a KEP and bring it
upstream.
—
Reply to this email directly, view it on GitHub
https://github.com/istio/istio/pull/43889#issuecomment-1468536871, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAEYGXIX47YTBGU65YD6CK3W4CTZZANCNFSM6AAAAAAVW6FOOE
.
You are receiving this because you were mentioned.Message ID:
@.***>

Sounds like this is a complex topic we should split out into a design doc. Do you want to own this?

For multi-cluster - I think we are getting close to consensus that ServiceImport will be supported, and that assigns a VIP as well. It would also be clear and give user control on having a Waypoint exported or not.
Sandwitch is another important use case - I understand for our own implementation we can and should optimize it away, but other implementations of Gateway talking with mesh workloads may have to rely on ztunnel to handle mTLS, HBone and istio authz. They would need some way to detect the waypoints and use the Status - but that is not so hard and I'm sure it can be agreed as a cross-vendor mechanism.
I completely agree that the spec - and good practice - needs some IP, and the natural thing is a Service with ClusterIP or ServiceImport cluster IP.

To be clear: not proposing to do ServiceImport in this PR, just add a Cluster IP to the service - and some comments explaining that the plan is to use SI or 'managed VIP' when they are available ( TODO works ).

Just to clarify a concern I have. "Using MCS as an API" is very different than "Platform provides MCS functionality out of the box". We cannot rely on the latter; no platforms do it. It only works when we are the MCS implementation as well (resolving DNS for clusterset.local, doing the endpoint selection, etc).
otherwise we are suddenly going to make Istio available to only 0.1% of k8s users

To confirm: users can still create the Service object on their own and may choose an internal load balancer type ( which may work for multi cluster ) - so the code that is adding the IP to the cluster status should pick the 'external' IP if it is available. I would go as far as documenting that if user creates a Service object with type externalName or has externalIPs, loadBalancerIP - we should use that as waypoint address and in the gateway status.

so using a ClusterIP is not a problem.

Why not ? Ztunnel talks with Istiod, Istiod knows the cluster IP is from clusterB - and returns the endpoints behind it.

Kuat is suggesting the clients sending directly to the VIP I believe, not using EDS to translate the VIP to endpoints.

@costinm We don't do load balancing for waypoints right now at all. It's reasonable, but it's another xDS pass just to perform load balancing, which is IMHO not justified. We are talking about adding O(#waypoints) listeners to every single sidecar.

I'm ok with any implementation we agree on - I think we must use EDS or
PTR-DS at least for the multi-cluster use cases, but if we
use the VIP in cluster for now ( because it's easier in envoy or any other
reason ) - I think it's ok.
My concern is with the API and model - we should use the VIPs ( local or
clusterset or external ), and not be tied to an implementation detail,
we'll likely change or have multiple implementations anyways ( in
particular if non-istio gateway start using this ).
On Tue, Mar 14, 2023 at 11:08 AM John Howard @.***>
wrote:

Kuat is suggesting the clients sending directly to the VIP I believe, not
using EDS to translate the VIP to endpoints.
—
Reply to this email directly, view it on GitHub
https://github.com/istio/istio/pull/43889#issuecomment-1468591402, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAAUR2RXBDMGPQVEYGJUJZLW4CX3VANCNFSM6AAAAAAVW6FOOE
.
You are receiving this because your review was requested.Message ID:
@.***>

Adding more - sidecar need a way to access VIPs in another cluster, so there is a datapath for that already. What we want to chain this datapath for Waypoint VIPs with the tunneling datapath. We need to maximize the reuse of xDS, and that starts with treating Waypoints as regular services, and ends with specific changes to Envoy to make the tunneling simpler.

Kuat - I am not sure I understand how we don't do ANY load balancing - my
understanding is that different clients or connections may still hit
different waypoint pods.
Is anyone working on scale / load testing ? It is something we should
verify.
I don't think PTR-DS is so high price - Istiod must return a list of IPs to
connect to, it doesn't have to be EDS ( but it can be ).
My point is that we should not require a specific implementation - the IP
in Gateway should be a VIP, and implementations may
chose to use it directly and rely on lower-level data plane ( or ztunnel -
for a non-istio K8S Gateway) to resolve it, or directly
integrate with EDS or PTR-DS to get the endpoints. Whatever works - and
implementations may be optimized if we find it makes a difference.
On Tue, Mar 14, 2023 at 11:30 AM Kuat @.***> wrote:

Sure, that's why I'm trying to advocate - we use standard cluster IP for
in-cluster waypoints, and punt on a special data path for cross-cluster
waypoints.
EDS has a cost, and I don't want simple use cases to pay the price.
Especially, since we don't have agreement EDS is the way to implement
cross-cluster VIPs in the more general context (not just waypoints).
—
Reply to this email directly, view it on GitHub
https://github.com/istio/istio/pull/43889#issuecomment-1468625512, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAAUR2T7ZZYMHKIN3Y3FQ3TW4C2NBANCNFSM6AAAAAAVW6FOOE
.
You are receiving this because you were mentioned.Message ID:
@.***>

The data path to access a VIP in another cluster is EDS.
On Tue, Mar 14, 2023 at 11:33 AM Kuat @.***> wrote:

Adding more - sidecar need a way to access VIPs in another cluster, so
there is a datapath for that already. What we want to chain this datapath
for Waypoint VIPs with the tunneling datapath. We need to maximize the
reuse of xDS, and that starts with treating Waypoints as regular services,
and ends with specific changes to Envoy to make the tunneling simpler.
—
Reply to this email directly, view it on GitHub
https://github.com/istio/istio/pull/43889#issuecomment-1468629241, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAAUR2W2XD4PJGHY6EMZDWLW4C2Y3ANCNFSM6AAAAAAVW6FOOE
.
You are receiving this because you were mentioned.Message ID:
@.***>

Kuat - I am not sure I understand how we don't do ANY load balancing - my understanding is that different clients or connections may still hit different waypoint pods. Is anyone working on scale / load testing ? It is something we should verify. I don't think PTR-DS is so high price - Istiod must return a list of IPs to connect to, it doesn't have to be EDS ( but it can be ). My point is that we should not require a specific implementation - the IP in Gateway should be a VIP, and implementations may chose to use it directly and rely on lower-level data plane ( or ztunnel - for a non-istio K8S Gateway) to resolve it, or directly integrate with EDS or PTR-DS to get the endpoints. Whatever works - and implementations may be optimized if we find it makes a difference.

Ambient does no load balancing for waypoints. It's a single pod IP right now.
I agree, it should be a VIP at least. Since waypoints are not services yet, there's no xDS/EDS for them.

On Tue, Mar 14, 2023 at 11:37 AM Kuat @.***> wrote:

Kuat - I am not sure I understand how we don't do ANY load balancing - my
understanding is that different clients or connections may still hit
different waypoint pods. Is anyone working on scale / load testing ? It is
something we should verify. I don't think PTR-DS is so high price - Istiod
must return a list of IPs to connect to, it doesn't have to be EDS ( but it
can be ). My point is that we should not require a specific implementation

the IP in Gateway should be a VIP, and implementations may chose to use
it directly and rely on lower-level data plane ( or ztunnel - for a
non-istio K8S Gateway) to resolve it, or directly integrate with EDS or
PTR-DS to get the endpoints. Whatever works - and implementations may be
optimized if we find it makes a difference.

Ambient does no load balancing for waypoints. It's a single pod IP right
now.
I agree, it should be a VIP at least. Since waypoints are not services
yet, there's no xDS/EDS for them.

You mean ztunnel makes a PTR-DS request with the VIP of an actual Service,
Istiod detects a waypoint is present - and knows the IPs associated with
the waypoint - but only
returns one ( first ? ). And for Envoy - I assume this also goes to Istiod
chains and a cluster doing the hbone to ztunnel - the EDS response from
Istiod will have a single IP ?
It doesn't seem hard to fix.
I'm getting re-familiarised with the code, sorry for obvious questions.
Message ID: @.***>

You mean ztunnel makes a PTR-DS request with the VIP of an actual Service,
Istiod detects a waypoint is present - and knows the IPs associated with
the waypoint - but only
returns one ( first ? ). And for Envoy - I assume this also goes to Istiod
chains and a cluster doing the hbone to ztunnel - the EDS response from
Istiod will have a single IP ?

Tunneling is an overlay in Envoy, it simply replaces an actual EDS pod
endpoint with a different IP. It's not doing another load balancing, and
the filter chain for all HBONE is shared.
There's no secondary EDS, one EDS says something like {endpoint: pod_IP, metadata: {tunnel_through: waypoint_IP}}.
We can create a separate filter chain for each waypoint with an upstream
cluster. The problem is that this is additive to regular sidecar xDS which
is already quite large, and we're trying to make enablement cost-effective.

It doesn't seem hard to fix.
I'm getting re-familiarised with the code, sorry for obvious questions.
Message ID: @.***>

—
Reply to this email directly, view it on GitHub
https://github.com/istio/istio/pull/43889#issuecomment-1468761123, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/ACIYRRWLMCWOMBZP7PU5M6DW4DGABANCNFSM6AAAAAAVW6FOOE
.
You are receiving this because your review was requested.Message ID:
@.***>

On Tue, Mar 14, 2023 at 1:21 PM Kuat @.***> wrote:

You mean ztunnel makes a PTR-DS request with the VIP of an actual
Service,
Istiod detects a waypoint is present - and knows the IPs associated with
the waypoint - but only
returns one ( first ? ). And for Envoy - I assume this also goes to
Istiod
chains and a cluster doing the hbone to ztunnel - the EDS response from
Istiod will have a single IP ?

Tunneling is an overlay in Envoy, it simply replaces an actual EDS pod
endpoint with a different IP. It's not doing another load balancing, and
the filter chain for all HBONE is shared.

Still confused - the request goes from client app to ztunnel to waypoint.
How is Envoy involved in this chain
except as a receiver ?
The request from Envoy to the server app does go through the per-node
ztunnel, so it's nomal to
have a single IP.
Also pod -> Service usually gets an EDS response with multiple pod IPs -
that's what we do for multi-network for example.
So even in the 'envoy sidecar to waypoint' - we should be able to do the
same thing we do for multi-network, and populate
the EDS response with multiple IPs.

There's no secondary EDS, one EDS says something like {endpoint: pod_IP, metadata: {tunnel_through: waypoint_IP}}.

Sure, but even normal EDS can include more than one IP.

We can create a separate filter chain for each waypoint with an upstream
cluster. The problem is that this is additive to regular sidecar xDS which
is already quite large, and we're trying to make enablement cost-effective.

It doesn't seem hard to fix.
I'm getting re-familiarised with the code, sorry for obvious questions.
Message ID: @.***>

—
Reply to this email directly, view it on GitHub
https://github.com/istio/istio/pull/43889#issuecomment-1468761123, or
unsubscribe
<
https://github.com/notifications/unsubscribe-auth/ACIYRRWLMCWOMBZP7PU5M6DW4DGABANCNFSM6AAAAAAVW6FOOE
.
You are receiving this because your review was requested.Message ID:
@.***>

—
Reply to this email directly, view it on GitHub
https://github.com/istio/istio/pull/43889#issuecomment-1468782604, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAAUR2ULEOPK5MWSAINCLX3W4DHMLANCNFSM6AAAAAAVW6FOOE
.
You are receiving this because you were mentioned.Message ID:
@.***>

On Tue, Mar 14, 2023 at 3:00 PM Kuat @.***> wrote:

Still confused - the request goes from client app to ztunnel to waypoint.
How is Envoy involved in this chain
except as a receiver ?
Sidecars go to waypoints directly without ztunnel AFAIK.

Ok, so it's not 'ambient doesn't load balance', but only 'sidecars talking
with waypoints', and only because
we don't want to mess up weights ?
I still don't see the problem with 'messing up weights' - we are in the
same situation with multi-network,
we can compute the weights of workloads in each cluster.

The request from Envoy to the server app does go through the per-node
ztunnel, so it's nomal to
have a single IP.
Not always. Sidecars don't and I think waypoints, too
Also pod -> Service usually gets an EDS response with multiple pod IPs -
that's what we do for multi-network for example.
So even in the 'envoy sidecar to waypoint' - we should be able to do the
same thing we do for multi-network, and populate
the EDS response with multiple IPs.
That is a bad practice - we're messing up the weights and the load reports
if we modify EDS heavily. This would also prevent integration with normal,
not-Ambient aware load balancers (e.g. GSLB).

Agreed, non-ambient aware LBs may need special code - like registering the
waypoint addresses as endpoints. There are other problems with smart load
balancers - their
LB will not see the loads correctly. But that doesn't change the data path

any client must hit one of the waypoints, and both VIP and EDS/PTR-DS are
valid and
should work.

Message ID: @.***>

It's certainly possible - the question is whether it is the right thing to do? I think we should punt on it in Envoy xDS when it doesn't work one way or another.
So there are two general data paths:

generic, depends on ztunnel to deal with multi-cluster VIPs, including Waypoints.
fast path, sidecars use it to connect to waypoints directly.

We're trying to add a lot of complexity to the fast path, while what we should be doing is optimizing the generic path. If the fast path doesn't work, we fall back to ztunnel. In other words, in-cluster, sidecars refer to waypoints by cluster IP. Otherwise, traffic is intercepted by ztunnel and resolved there.
I'm not sure why we need to argue about this in depth. We certainly should stop adding hacks with re-weighting endpoints in the model by their gateways, even if we chose to do the complicated fast path optiimization.

@howardjohn: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

unit-tests-arm64_istio
e321aeeb21c0fa0bcb86a382a9476f54be0860db
link
true
/test unit-tests-arm64

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@howardjohn: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

unit-tests-arm64_istio
e321aeeb21c0fa0bcb86a382a9476f54be0860db
link
true
/test unit-tests-arm64

unit-tests_istio
e321aeeb21c0fa0bcb86a382a9476f54be0860db
link
true
/test unit-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

PR needs rebase.

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Kuat - I'm not sure a path is much faster than the other, but speed is not the reason we need ztunnel to send to endpoints or middle boxes - I think that's the source of the confusion.
On VMs or from other clusters  - the Service VIP is not usable.  If we need to cross gateways (east/west mutli network) - same thing. If L4 workloads run on VMs - VIPs are not currently usable ( we may go back to adding EndpointSlices for
VMs, but that's not what we do now)
I don't disagree with you that if the waypoint or L4 pods are in same cluster - VIP is the easiest. It is also good for interop with non Istio components, etc. But we do need code to handle both, and in most 'mesh is more than one cluster' cases the VIP is not sufficient.
I also agree it's a good opportunity to revisit and simplify - but there are important use cases that we still want to support.

@howardjohn: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

unit-tests_istio
e321aeeb21c0fa0bcb86a382a9476f54be0860db
link
true
/test unit-tests

unit-tests-arm64_istio
7557fd8e20c9a61e6a5faf2f86cd8c1e7a406b41
link
true
/test unit-tests-arm64

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@howardjohn: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

unit-tests-arm64_istio
7557fd8e20c9a61e6a5faf2f86cd8c1e7a406b41
link
true
/test unit-tests-arm64

unit-tests_istio
7557fd8e20c9a61e6a5faf2f86cd8c1e7a406b41
link
true
/test unit-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.