Support Markdown component

Duplicates

[X] I have searched the existing issues

Latest version

[X] I have tested the latest version

Summary 💡
The TextField component is quite limited. By default, I would expect to be able to use Toolpad's text component like I use Notion 😁
Motivation
In https://master--toolpad.mui.com/_toolpad/app/cl4hla83p01949xoizo5uxf2a/editor/pages/cl6i430vg00003b6ghpxth1dz, I had to mix two different components, but they are not part of the same layout flow, they should:

Examples 🌈
https://retool.com/blog/text-v2-app-documentation/

When we add it, should we remove typography component?
Are images, videos and text links different?

When we add it, should we remove typography component?

I wonder, Notion is made of small blocks and it feels great, so maybe keep both.

Should we call it Rich text or MD?

I would be afraid that if we call it Rich text, then people will confuse this with a Rich text editor. Markdown sounds clear.


feat: errors tracking

Description
Introduces the Errors tab within the app.
Motivation and Context
This feature allows users to view errors associated with Docker operations.
For instance, it highlights issues such as occupied ports.
How Has This Been Tested?
Manually, CI

Depends on #238


mptt admin indent issue

Following http://django-parler.readthedocs.org/en/latest/advanced/mptt.html#integration-with-django-mptt, all seems okay but with below in admin.py

without list_display as below, title indent is okay.

from django.contrib import admin
from parler.admin import TranslatableAdmin, TranslatableModelForm
from mptt.admin import MPTTModelAdmin
from mptt.forms import MPTTAdminForm
from .models import Category

class CategoryAdminForm(MPTTAdminForm, TranslatableModelForm):
    pass

class CategoryAdmin(TranslatableAdmin, MPTTModelAdmin):
    form = CategoryAdminForm

    def get_prepopulated_fields(self, request, obj=None):
        return {'slug': ('title',)}  # needed for translated fields

admin.site.register(Category, CategoryAdmin)

BUT if  list_display is used, then somehow second column value is indented. In this case, is_published is indented.

from django.contrib import admin
from parler.admin import TranslatableAdmin, TranslatableModelForm
from mptt.admin import MPTTModelAdmin
from mptt.forms import MPTTAdminForm
from .models import Category

class CategoryAdminForm(MPTTAdminForm, TranslatableModelForm):
    pass

class CategoryAdmin(TranslatableAdmin, MPTTModelAdmin):
    form = CategoryAdminForm
   ** list_display = ('title','is_published','order','language_column',)**

    def get_prepopulated_fields(self, request, obj=None):
        return {'slug': ('title',)}  # needed for translated fields

admin.site.register(Category, CategoryAdmin)

What am I missing?

@sunjoomoon what do you mean with "indented"? could you post a screenshot?

If I put below in admins.py, then mptt indent is applied to the is_published field instead of title field
    list_display = ('title','is_published','order','language_column',)
    list_editable = ('is_published','order')

However, if I just pass all in admins.py, then it looks okay.
[

](url)

Any tip? Is this a complex one? or just css or js thing? Any clue as to what to look into?

Further testing tells me this happens only with some fields BooleanField, IntegerField, CHOICES
And css somehow is applied to the unwanted field.

Here we go, reminded me of importance of reading the doc again.
https://django-mptt.github.io/django-mptt/admin.html#mptt-admin-mpttmodeladmin
class CategoryAdmin(TranslatableAdmin, MPTTModelAdmin):
    form = CategoryAdminForm
    mptt_indent_field = "title"         # replace this with your choice of indented field
    list_display = ('title','is_published','order','language_column',)

    def get_prepopulated_fields(self, request, obj=None):
        return {'slug': ('title',)}  # needed for translated fields

admin.site.register(Category, CategoryAdmin)

Thanks @sunjoomoon for clearing that up! Is this needed because our fields are translated, thus not standard model fields?

Not sure but in this case being explicit makes more sense. As I am unaware of what goes on under the surface, something to do with css class inserted in the unwanted field somehow.  Sorry for my ignorance..


Support for RTL languages (such as Arabic / Hebrew / Persian etc.)

My name is Tomer Mahlin. I lead a development team in IBM named Bidi Development Lab. We are specializing (for more than 20 years) in development of support for languages with bidirectional scripts (or "bidi lang." for short) .
We recently ran a sniff assessment on Monaco capabilities with respect to bidi lang. display. We believe there are several functional areas which require improvements (please see more details below).
My team can work on necessary modifications and suggest them via separate pull request, assuming community is interested in addressing the requirements detailed below.
Plain text editing

There should be a parameter through which it will be possible to communicated the default text direction for content being authored in specific instance of editor. This is a similar parameter to what is used in CKEditor: contentsLangDirection ( http://docs.ckeditor.com/#!/api/CKEDITOR.config-cfg-contentsLangDirection ).
Possible values should be:

ltr (left-to-right),
rtl (right-to-left),
contextual (or auto as used in HTML)

In addition to that, there should be explicit way for the end user to interactively change text direction for selected text (or for current paragraph in which cursor is positioned in case current selection is of zero length). This can be achieved via:

GUI buttons - similar to all rich text editors (i.e. http://ckeditor.com/addon/bidi)
AND / OR (in case there is no toolbars for any new buttons)
Keyboard shortcuts (i.e. in Notepad it is Ctrl - <Left|Right-Shift>)

Programming lang. editing

As opposed to plain text, programming lang has well defined syntax. Some part of this syntax is visualized via color schema used for coloring different elements (i..e comments vs variables etc.) of the language.  It is critical to enforce visual appearance associated with the syntax regardless of language used for different elements (i.e. comments, variables etc.). If this is not done, it becomes virtually impossible to work with the code when bidi text is used.  Simple English example:
a = b + c; // hello world
If bidi characters are used instead you would expect to see:
A = B + C; // DLROW OLLEH
Instead at the moment you see:
DLROW OLLEH // ;C+B=A
The more complex example can be, less intuitive the display will become.

Special case is the case of comments or/and constants. Those by all means usually include bidi characters (or at least much more frequently than variables names for example). It is thus preferable to display text in those contexts using natural text direction for bidi languages (which is RTL). We can't store text direction information with text (namely source code file is still a plain text file which can't include any meta information about text such as font size, color, direction etc.). Consequently we should be able to make a smart choice while displaying the text (relying just on the text itself). Most straightforward approach is to enforce auto (aka contextual or first strong) direction of text for each paragraph included in comments.
For example, currently the display of sample text is as follows:
res = var1 + var2; // SI EMAN YM tomer !!!
If we enforce auto text direction on the comment we will see:
res = var1 + var2; // !!! tomer SI EMAN YM
Namely text of comment will appear with actual RTL direction which is a natural one for bidi lang.
Display of text with natural text direction makes it considerably more readable and thus should greatly enhance user experience for bidi users.

Relevant requests
At some point support for bidi lang. was requested in vscode via https://github.com/Microsoft/vscode/issues/4994

@microsoftsam , can you please review this issue ?

@tomerm PR welcome. If anything special needs to be done, it could be done in viewLineRenderer.ts, where the actual HTML for a line is produced.
Our goal with the monaco editor and vscode is to be a code editor, i.e. the plain text case editing is not a priority.
Regarding programming language editing, have you actually tried these scenarios in the monaco-editor, I am not an expert, but we do support RTL and bi-di to quite some extent:

var b = 3;
var c = 2;
var x = "מיותר קודמות צ'ט של, אם לשון העברית שינויים ויש, אם";
var a = b + c; //מיותר קודמות צ'ט של, אם לשון העברית שינויים ויש, אם

@alexandrudima thanks a lot for your quick reply. The examples you provided are rendered properly. However, this is just one use case - you used exclusively Hebrew characters (no numbers, no English text in the middle of Hebrew text etc. ).
In other words I totally agree that Monaco editor provides correct basic reordering (taking text stored in the buffer and transforming it for the sake of presentation on the screen).
If you start using mixed (English + Hebrew) text in comments / constants (not mentioning using Hebrew in variable, class names etc.) the display will become considerably less readable. It will be very similar to display we get in Notepad. However, while Notepad is not aware of any programming language syntax (and thus is not expected to enforce it), Monaco is aware of such syntax and should assure it is preserved on the display layer.

Thank you for explaining the issue. Indeed, it would be better to interpret syntax characters as Left-To-Right instead of Neutral when rendering source code.
Here is a quick test I did for:
var ת = "מיותר קודמות צ'ט של, אם לשון העברית שינויים ויש, אם";

Monaco Editor

Ace Editor

CodeMirror

Word

Notepad

A browser textarea

And a couple that get it right:
SublimeText

Visual Studio

@alexandrudima ,  several examples of editors in which we do get expected behavior:
Orion
This is a web based editor https://orionhub.org/

Eclipse
This is a desktop editor (since you mentioned SiblimeText which is also desktop based )

ACE
Regarding ACE. Currently ACE has much more serious problems with support for RTL languages. My team is contributing the solution. Eventually we will support correct behavior inside ACE. By the way Github is also using ACE (I think).
Non code editors
Notepad, browser textarea, Word - all those editors are not meant to edit code written in some programming language. Thus I don't think we should expect them to enforce any syntax associated with any of programming languages.

Ok, here is what it looks like on master:
using JS syntax:

using plaintext:

@alexandrudima , I have tested this fix using the latest code.
Indeed it is resolved in Chrome and IE browsers. However, it is not for FF.
See the following screenshot.

@amirbrans Thank you for verifying.
I have FF 50.1.0 on Windows 10 and it appears to work there.
Does your test URL contain editor=dev in the query param (I often mix this up myself)?
e.g. file:///C:/Alex/src/monaco-editor/test/index.html?editor=dev#Y___DefaultJS

@alexandrudima , actually yes. it does contain the editor=dev parameter.
(BTW, if that was not the case then it wouldn't work on other browsers as well...)
I'm using FF ESR 45.6
Very weird it doesn't work on it.

Following up, document with Bidi issues discovered on Monaco editor:
Monaco Bidi issues.docx

@alexandrudima STT mentioned in @amirbrans report above refers to "STructured Text". It is a general term we use to describe a text with internal structure which in general is not preserved by UBA (Unicode Bidi Algorithm) in case it includes bidi text.
Please let us know if you need any help with fixing issues in the report above. I just want to make sure we don't duplicate our efforts if you have plans to address those issues yourself. Many thanks in advance for your attention and help.

We're looking for a great code editor for our content authors (using Jekyll, YAML, html, css, ..) and I am interested in VS Code or may be even just in Monaco. Since the content is in hebrew, arabic and english, this RTL issue is critical for us.
Please keep us posted here about the plan to fix the issues reported in @amirbrans report, I will be glad to test once solved. Thanks!

Thank you for the extra testing. The root cause is a bit silly: the same colors are used by the theme for identifiers and syntactical text. If the theme would give even a slight different color, the text would be split into multiple tokens and the "dir" trick might work...

@alexandrudima
Any update for BIDI support? As you know aside from web development in VSCode , the great extension LaTex Workshop (https://github.com/James-Yu/LaTeX-Workshop) with more than 132000 downloads prove that many people around the globe use VSCode for web and docs development. There are many users with daily involvement with RTL language and needs BIDI support from VSCode.
I highly appreciate if you kindly update us how it is going in VSCode team side.

@tomerm
Tomer, as you are with great background on BIDI and RTL language implementation is it possible for short time to develop an extension to remedy the issue as while as VSCode development team working to add this feature to VSCode.

Unfortunately vs code has no this feature .I hope to see it soon. I found the best editor to support bidirectional editing of scripts written in rtl is Emacs. So the trick for time being, is editing rtl scripts in emacs and then upload it to vs code


feat(mongo-connector): support copy.existing in source and topics/topic.override in sink

Motivation
Compares to mongodb-kafka-connector, pulsar-kafka-connector is quite simple. So it will be better to  add some useful features on pulsar-kafka-connector to enhance:

copy.existing config in source: https://docs.mongodb.com/kafka-connector/current/source-connector/configuration-properties/copy-existing/
topics/topics.regex in sink: https://docs.mongodb.com/kafka-connector/current/sink-connector/configuration-properties/kafka-topic/
topic.override in sink: https://docs.mongodb.com/kafka-connector/current/sink-connector/configuration-properties/topic-override/

Pls refer the open source code of mongo-kafka.

Okay, I will read the code of mongo-kafka firstly!

I will read the pulsar io source code next week

In pulsar io of mongodb, the offset is not saved , whether  this feature should be supported or not  in this  issue?

In pulsar io of mongodb, the offset is not saved , whether this feature should be supported or not in this issue? @ericsyh

@lzqdename Sorry I am not the maintainer of mongo pulsar io, can you give more details about the offset you mentioned that not saving? In source or sink connector? And in which process that not saving?

okay，I know, the offset is the position at which  the consumer will consume when restarted,
you can see :
com.mongodb.kafka.connect.source.MongoSourceTask.createCursor
in this function, there is a sub function call :  getResumeToken(sourceConfig)
in function getResumeToken(sourceConfig), it will read the last consumer offset position
in product environment,we shoudl save the offset of the consumed record to avoid repeated cosume action
okay, I will write a basic implementation of "copy.existing" due to lackness of time

@lzqdename Cause Pulsar consumer has no need to save offset since the offset info will be saved by Pulsar broker. For example, Pulsar consumer use a subscription name sub-1 to consume for a while and exit for some reason. This consumer just need to connect to pulsar with the same subscription name sub-1, Pulsar broker will send the next message from the last consumed position to this consumer.

consume mongodb,not pulsar,
u can see https://github.com/mongodb/mongo-kafka/blob/master/src/main/java/com/mongodb/kafka/connect/source/MongoSourceTask.java
line 397  ,  getResumeToken(sourceConfig)
the resume token is token of mongdb

@lzqdename OK i see, i think there is no need for you to support saving mongodb offset in this task.


Add serde support

What about to add serde support?
extern crate serde;
extern crate serde_json;
extern crate serde_derive;
extern crate css_colors;

use css_colors::{RGB, rgb};
use serde_derive::{Serialize, Deserialize};

#[derive(Serialize, Deserialize)]
struct Example {
	color: RGB,
}

#[test]
fn example() {
	let example = Example {
		color: rgb(100, 200, 1),
	};

	let json = serde_json::to_string(&example).unwrap();

	assert!(json, "{\"color\":\"rgb(100,200,1)\"}");
}

This would be a nice addition indeed.
Best would be support of hex codes (in both short #abc and long #abcdef form) for RGB. The 8-hex version for RGBA is not standardised, but could be added as well.

im sorry i dont know how to do this...8m sick


FileNotFoundException: ServiceFabricServiceModel.xsd ?!?!

Been working over a week trying to deploy a standalone Service Fabric secured cluster on premise for our development/staging environment. I've been following ALL the official guides and experimental blogs. Been tackling one roadblock after the other. This last one left me completely lost and would greatly appreciate assistance.
TestConfiguration.ps1 passes with flying colors.
All 17 requirements for the environment set up have been implemented and tested (ports, services, remote access, accounts, etc).
during the creation of the cluster, right after configuring all the nodes, I get a System.IO.FileNotFoundException: ServiceFabricServiceModel.xsd error and from there, a cascade rollback with no further detail.
All logs seem to be as mysterious with no exact explanation as to where was this file attempted to be found (it's there where it should be with the rest of the cluster setup!), at what stage is this thrown, why is this needed, etc.
Any suggestions where to look or what to test?
For reference, here's the exact powershell output:

PS C:\Users\Administrator\Downloads\AzureServiceFabric> .\CreateServiceFabricCluster.ps1 -ClusterConfigFilePath .\Dev.ClusterConfig.Windows.Multimachine.json -FabricRuntimePackagePath .\MicrosoftAzureServiceFabric.<IP_ADDRESS>94.cab -AcceptEULA
Creating Service Fabric Cluster...
If it's taking too long, please check in Task Manager details and see if Fabric.exe for each node is running. If not, please look at: 1. traces in DeploymentTraces directory and 2. traces in FabricLogRoot configured in ClusterConfig.json.
Trace folder already exists. Traces will be written to existing trace folder: C:\Users\Administrator\Downloads\AzureServiceFabric\DeploymentTraces
Running Best Practices Analyzer...
Best Practices Analyzer completed successfully.
Creating Service Fabric Cluster...
Processing and validating cluster config.
Configuring nodes.
Default installation directory chosen based on system drive of machine 'Node01'.
Copying installer to all machines.
Configuring machine 'Node01'.
Configuring machine 'Node03'.
Configuring machine 'Node02'.
Configuring machine 'Node04'.
System.IO.FileNotFoundException: ServiceFabricServiceModel.xsd
at System.Fabric.Common.SchemaLocation.GetWindowsFabricSchemaLocation()
at System.Fabric.FabricDeployer.DeploymentParameters.Initialize()
at System.Fabric.FabricDeployer.DeploymentOperation.ExecuteOperation(DeploymentParameters parameters, Boolean disableFileTrace)
at System.Fabric.FabricDeployer.ConfigurationDeployer.NewNodeConfigurationInner(String clusterManifestPath, String infrastructureManifestPath, String jsonClusterConfigPath, String fabricDataRoot, String fabricLogRoot, String fabricHostCredentialUser, SecureString fabricHostCredentialPassword, Boolean runFabricHostServiceAsManual, Boolean removeExistingConfiguration, FabricPackageType fabricPackageType, String fabricPackageRoot, String machineName, String bootstrapPackagePath)

We have added extra fallback steps in cleanup to purge old versions from GAC as part of uninstall.


Code | Alphabetic Pattern 94

@aryashah2k kindly assign this one to me.

@aryashah2k kindly assign this one to me.


[Mouse Guard 2e] new character sheet

Changes / Comments
Mouse Guard 2e character sheet basen on an existing Mouse Guard 1e sheet.
Changes against Mouse Guard 1e character sheet:
+aligned condition top to bottom as per original character sheet
+added Level 2 trait checkbox
+added a separate Wises section
+added Manipulator skill
+marked patrol skills with [p] (for Winter Session)
-removed Healthy condition
-removed surplus pass/fail checkboxes for skills and abilities
-removed Deceiver skill
Roll20 Requests
Comments are very helpful for reviewing the code changes. Please answer the relevant questions below in your comment.

[x] Does the pull request title have the sheet name(s)? Include each sheet name.
Mouse Guard 2e
[ ] Is this a bug fix?
No.
[ ] Does this add functional enhancements (new features or extending existing features) ?
No - it's a rework of an existing Mouse Guard 1e sheet as a completely new, separate sheet.
[ ] Does this add or change functional aesthetics (such as layout or color scheme) ?
No.
[ ] If changing or removing attributes, what steps have you taken, if any, to preserve player data ?
Does not interfere with existing Mouse Guard 1e sheets - it's a new sheet
[x] If this is a new sheet, did you follow Building Character Sheets standards ?
Yes I did - at least I tried. :) I'm new at this.

If you do not know English. Please leave a comment in your native language.

This doesn't comply with minimum requirements for new sheet submissions. It contains <table> elements. (Old sheets are obviously grandfathered into this and doesn't have to comply to this rule)

This doesn't comply with minimum requirements for new sheet submissions. It contains <table> elements. (Old sheets are obviously grandfathered into this and doesn't have to comply to this rule)

Thanks. I can try removing tables but it will take significant time. I was hoping on adding just a few quick amendments. The current 1e sheet is a bit cumbersome while playing 2e. :/

Hi @darkseji when you get the the new HTML update in without the tables we can merge it in that upcoming week.  Thanks!

Hi @Reonin. I'll try making required changes next week. Not enough free time, unfortunately.
A question, as I'm pretty new to Github - should I cancel this request and submit a new one with amendments or is there a way to incorporate changes to an existing pull request? Thanks!

You can keep this same pull request open! And as you make changes and commit then to this same pull request it will refresh to the latest version. If you feel like it's at a good state you can leave a comment or I'll check through it regardless next week.

Hey @darkseji,
I have a 2E Mouseguard sheet I was planning on submitting. I don't want to pull the rug from underneath you so let me know if you'd be cool with me posting.

Hi @Medieve. Sure, go on and submit! I'll bet it is far better than mine quick and dirty rework of the 1e sheet. :) Thanks!

Medieve have now submitted his Mouseguard 2E PR #6276, so this could be closed @darkseji.
You should see a "comment and close"-button at the bottom, next to the "comment" button.

Closing request as agreed.


cmd/go: "help test" caching paragraph is out-of-date and potentially misleading

First the out-of-date part: In https://github.com/golang/go/commit/c55a50edb9454dbdaca165be4b030a1e0cfbaa19 the special-casing of -timeout was removed, but the documentation for this behaviour at https://github.com/golang/go/blob/master/src/cmd/go/internal/test/test.go#L131 is still there.
Now the misleading: To correctly interpret "the run involves the same test binary and the flags on the command line", one has to understand some amount of the inner workings of the test process - specifically that we are talking about only those flags that are actually forwarded to the binary under test, and not all flags passed to go test itself. So for example -vet is fine to use for a cacheable run. Further, AFAIK the flags that are/aren't forwarded aren't enumerated in any documentation, or even in a single place in the code - there's a little discussion at go help testflag but no list.
I think the clearest way to resolve this would be to remove the flag-level detail in that paragraph, and instead explictly state the caching behaviour of each flag in go help testflag, i.e. "Cacheable" / "Ignored for caching" / "Prevents caching".

@josharian

It would be very helpful to clear up this documentation. Another one not discussed is -race. I've been experimenting with caching test results with -race in https://github.com/stellar/go/pull/3727 and in one build I see test results get cached even when that flag is enabled, but I don't see it mentioned in these at all.
I have another build that doesn't cache at all though under the same situation.


feat(docker): add Dockerfile and release script

I would rather have the whole building process in the Dockerfile, then hub.docker.com would be able to build it automatically on every push.


Dead link

The link to https://nafi-lang.github.io/rust-nafi/nafi_tokens at the top of the github page is dead.

Link is removed as I decide to continue experimenting 😜


adding simple example

This is the simplest attack file that Gauntlt needs to be able to run.

Oh one thing that is interesting here is that in the new parser I deleted "Feature" in lieu of "Attack"...  We either need backwards capability or some way to say this is gauntlt 2.0....

Makes sense. Checks passed, so merging.


Grafana container cannot connect to InfluxDB via service IP

Heapster container can connect to InfluxDB sink via service IP, but not Grafana container.
root@heapster-3c83g:/go# curl --connect-timeout 10 http://monitoring-influxdb:8086
404 page not found

root@influxdb-grafana-7cmy4:/# curl --connect-timeout 10 http://monitoring-grafana:8086
curl: (28) Connection timed out after 10001 milliseconds

For now we are using localhost since Grafana and InfluxDB containers are in the same pod.

This is related to the kubelet's hairpin mode or promiscuous mode: https://github.com/kubernetes/kubernetes/issues/20475

@zihaoyu did you set the --hairpin-mode=hairpin-veth kubelet flag? After that just kill your pod and it should work.
You can check on your hosts whether the hairpin mode is enabled or not on each virtual interface of your docker bridge:
for intf in /sys/devices/virtual/net/docker0/brif/*; do
  cat $intf/hairpin_mode
done

@antoineco No I did not set that flag. kubelet runs with default values for --configure-cbr0 and --hairpin-mode options.

Though there is a bug with the --hairpin-mode flag not taking effect in
certain cases that is fixed for inclusion into 1.2.1. (don't have time to
look up the issue number atm)
On Wed, Mar 30, 2016 at 10:57 AM Zihao Yu<EMAIL_ADDRESS>wrote:

@antoineco https://github.com/antoineco No I did not set that flag.
kubelet runs with default values for --configure-cbr0 and --hairpin-mode
options.
—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub
https://github.com/kubernetes/heapster/issues/1079#issuecomment-203552705

It doesn't fallback properly to hairpin-veth in case --configure-cbr0 is false, however @zihaoyu said it was set to true in his cluster, so this might be yet a different case.

Not an issue any more.


K8S client support for clients using the library

https://github.com/IBM/secret-common-lib/pull/18/ - raised a new PR here


Ssl support implementation

Please review whether the implementation is aligned with Your ideas.
Successfully tested on my local workstation.
Example codes for implementing the authentication related delegates in ISslConfiguration:
https://msdn.microsoft.com/en-us/library/system.net.security.localcertificateselectioncallback.aspx
https://msdn.microsoft.com/en-us/library/system.net.security.remotecertificatevalidationcallback.aspx

Not sure how I missed this before -- I apologize.
I'll take a look shortly.

Pls review my updates!

Looking great -- thanks for updating! If you don't mind adding in some unit tests, I'd gladly merge it in.

Dont merge yet, my tests shows reconnection issues! Will push the fixes later.

I also wanted to merge the SSL implementation against Your latest changes from master branch, but could not.
You've removed TcpClient, and used Socket directly for network communication.
Unfortunately SSL support could not be built on top of Socket, because it needs NetworkStream.
If You don't wish to use TcpClient, we can keep using the Socket approach, but network reading/writing should happen through a Stream, not through Socket directly. That can be NetworkStream for unencrypted, and replaced by SslStream, for encrypted connections. The NetworkStream can be easily created from the Socket, and we should keep the Stream reference in the Connection class, with/instead of Socket.

I absolutely missed the fact that the ssl implementation required TcpClient when I was doing the performance work -- I'm trying to tie something off wrt the consumer groups right now, then I'll take a look at reintroducing Streams into the Connection (so we can use the SslStream like you suggested).

I'm trying to do the reintroducing of Streams. SSL support is pretty urgent for us, and I could not found any other lib supporting it. I'll come up with a PR soon.

Not sure how far you got with this, but I've finally made progress on the groups side (which was my highest priority). I've got a bit of cleanup to do as a result, then I can come back and look at this -- likely get to it Monday if you don't beat me to it.
You mentioned earlier that you had tests that showed issues (pre merge with master). If you've got those tests (and fixes) handy, can you push them to the PR? If you do, I can help with the merging (and reintroduction of streams) to leverage your work.

Here is a solution for re-introducing streams, and adding SSL support.
https://github.com/ligu/KafkaClient/tree/streams
(Latest changes from You master repo added, altough the tests are not fixed yet.)
I've separated the low-level transport-related functionality from Connection, and created an ITransport interface, what Connection can use to do the low-level network communication. There is the StreamTransport implementation for the interface. We can also have a SocketTransport implementation (without the SSL support) if You think it can give advantages.

About the issues.
What I've noticed is that in case of the TCP connection goes down, (which is quite common after a couple of seconds, if You don't have keep-alive configured), then upon trying to publish the next message, it fails, and that message get lost. I'm still investigating this issue, no solution yet.
One idea is to add the TCP keep-alive functionality on a configurable way. But that's just a partial solution, still doesn't solve the message loosing problem in case of lost connection.

The code in https://github.com/ligu/KafkaClient/tree/streams generally looks good to me -- I like the split with ITransport. There may be something we need to do for reconnection attempts inside the semaphore, but a bit hard to read fully on github. As for the message lost issue, if we can identify that there's a partial read, the comment in Connection: // when reconnecting, will the data continue to stream or start again?? would be the place to distinguish between lost or partially read. If the message is actually lost, the request will timeout and cleanup appropriately otherwise.

Since I've made changes that (again) break the work you've done, I'm going to go through the merge. I don't want to lose the work that you've done. I'll pull from your updated branch and resolve conflicts on my side.

Closing this in favour of #22, to get the code merged in.


bcdb rebuild is not rebuilding the index correctly

Steps:

start the explorer. j.servers.threebot.local_start_explorer(background=True)
create any object. I use the farm actors as an example:

    cl = j.clients.threebot.explorer.actors_get("tfgrid.directory")
    farm = {
        "threebot_id": 1,
        "name": f"stresstest{j.data.idgenerator.generateXCharID(5)}",
        "email"<EMAIL_ADDRESS>        "wallet_addresses": ["address"],
    }
    farm_id = cl.farms.register(farm).farm_id

stop threebot server, but keep zdb running.
get the farm model and list the farms, it lists fine.

> bcdb = j.data.bcdb.instances["threebot_ZDB_tfgrid"]
JSX> bcdb.storclient
# JSXOBJ:
## jumpscale.zdb.client.1
ID: 92
 - name                : threebot_ZDB_tfgrid
 - addr                : <IP_ADDRESS>
 - port                : 9,900
 - secret_             : c2ace231021d72fb0329f0f7fb564a4a
 - nsname              : tfgrid
 - admin               : False
 - mode                : SEQ

JSX> farm_model = bcdb.model_get(url="tfgrid.directory.farm.1")
model:tfgrid.directory.farm.1
JSX> farm_model.find()
[## tfgrid.directory.farm.1
ID: 1
 - threebot_id         : 1
 - iyo_organization    :
 - name                : stresstestknzo7
 - wallet_addresses    :
    - address
    ## tfgrid.directory.location.1
     - city                :
     - country             :
     - continent           :
     - latitude            : 0
     - longitude           : 0

 - email<EMAIL_ADDRESS> - resource_prices     : []
 - prefix_zero         : ::/128
]

delete the farm using the storclient directly, not through object deletion. The aim is to have inconsistent index.

bcdb.storclient.delete(farm_id)

list the farms again, it will crash because the id is sitll in the index but removed from zdb. This should be fixed by rebuilding the index.

exit kosmos and run bcdb rebuild in the shell.

open the shell, and try to list the farms again, you'll still find the error.

It worked for me only when I deleted the sql index file and rebuild again. Also, @andrewayoub couldnt reproduce it with the rebuild_index function on the bcdb instance. It happens only with the bcdb  rebuild command.

when you rum bcdb rebuild it starts a new process and since bcdb models are lazy loaded, it will only be loaded when used, that's why I had to get all objects in order to enforce loading all models before rebuilding.
https://github.com/threefoldtech/jumpscaleX_core/commit/3a67d530e16766db7695d0e3288308b4e2be156d

verified


Add Trace ID Validation to Integration Tests for Organization Management Error Responses

Description
This pull request adds trace ID validation to the integration tests for error responses within the Organization Management API. The updates ensure that the test cases now verify the presence and format of trace IDs in error responses, improving traceability and supporting effective debugging.
Summary of Changes

Called the validateErrorResponse method, which includes trace ID validation, in the OrganizationManagementFailureTest.java file.

These changes contribute to better observability and align with best practices for error handling and logging within the codebase.

PR builder started
Link: https://github.com/wso2/product-is/actions/runs/10366690441

PR builder completed
Link: https://github.com/wso2/product-is/actions/runs/10366690441
Status: success


Fixes react-native link for Windows

Improve resilience of link command on Windows
There were various bugs related to react-native link for react-native-windows:    - NPM projects with multiple .csproj files could link the wrong projects

Recent addition of ReactNativeHost (in lieu of ReactPage) was not supported
Project GUIDs were not correctly referenced when linking dependencies
NPM dependency projects needed to unnecessarily publish a Visual Studio solution file (i.e., .sln)
Linking a project in a solution did not set up a default build configuration

This change set fixes all of the above. There are some remaining things to do for fit and finish:

Support linking projects as well as their source code project (i.e., .csproj) dependencies to the RNW app solution

Fixes Microsoft/react-native-windows#738
Fixes Microsoft/react-native-windows#1576
Fixes Microsoft/react-native-windows#1578

Motivation
The react-native link command is broken for version 0.50+ of react-native-windows.
Test Plan

Start localhost NPM registry (i.e., sinopia)
Apply patch to facebook/react-native repo
Publish patch to localhost registry
Create a new project with react-native init from patched version of react-native
Install a package with windows support (e.g., react-native-fs)
Run react-native link
Confirm link and unlink commands are successful for Windows.

Related PRs
(If this PR adds or changes functionality, please take some time to update the docs at https://github.com/facebook/react-native-website, and link to your PR here.)
Release Notes

[WINDOWS][BUGFIX][CLI] - Fix the link and unlink commands for react-native-windows

Sorry, i just copy and paste your commit change in my react native but i get this err :
\node_modules\react-native\local-cli\core\index.js:65
reduce((acc, pathToPlatforms) => {
^
TypeError: Reduce of empty array with no initial value
at Array.reduce ()
at Object. (C:/Users//android/MusicProject/node_modules/react-native/local-cli/core/index.js:65:4)
at Module._compile (module.js:649:30)
at loader (C:\Users*****\android\MusicProject\node_modules\babel-register\lib\node.js:144:5)
at Object.require.extensions.(anonymous function) [as .js] (C:\Users*\android\MusicProject\node_modules\babel-register\lib\node.js:154:7)
at Module.load (module.js:561:32)
at tryModuleLoad (module.js:501:12)
at Function.Module._load (module.js:493:3)
at Module.require (module.js:593:17)
at require (internal/module.js:11:18)

Why did you copied those changes to your code in first place?


feat: increase asset code length to 6 chars

Changes proposed in this pull request
Allow asset code length to be a maximum of 6 characters.
Context
Fixes #2139
Checklist

[x] Related issues linked using fixes #number
[ ] Tests added/updated
[ ] Documentation added
[ ] Make sure that all checks pass
[ ] Postman collection updated

Thank you for Contributing to Interledger during Hacktoberfest. If you have any accepted PRs, You can claim your Swag here. Last date to fill up the form is December 10th, 2023. @Dhoni77


Custom ModelBinder to handle nested derived classes

Hi, following @rynowak answer in #4188 I extended and replaced the original MutableObjectModelBinder to handle a case of nested inherited classes (abstract section and derived classes). However when debuging I realized that the model binder was only called for the top level class and not the nested collection (I thought binding would be a recursive process).
The classes I use are the following :
public class Template {
   //other things...
   public List<Section> Sections { get; set; }
}

public abstract class Section{}

public class ContentSection : Section
{
   //some things
}
//other derived section classes

My actual ModelBinder :
public class TemplateSectionModelBinder : MutableObjectModelBinder
    {
        /// <inheritdoc />
        protected override object CreateModel(ModelBindingContext bindingContext)
        {
            if (bindingContext == null)
            {
                throw new ArgumentNullException(nameof(bindingContext));
            }

            if (bindingContext.ModelType == typeof(Section))
            {
                var typeValue = bindingContext.ValueProvider.GetValue("ModelType");
                var type = Type.GetType(
                    (string)typeValue.ConvertTo(typeof(string)),
                    true
                );
                if (!typeof(Section).IsAssignableFrom(type))
                {
                    throw new InvalidOperationException("Bad Type");
                }
                return Activator.CreateInstance(type);
            }
                // If the Activator throws an exception, we want to propagate it back up the call stack, since the
                // application developer should know that this was an invalid type to try to bind to.
                return Activator.CreateInstance(bindingContext.ModelType);

            }
        }

So wich ModelBinder should I change to be able to detect the nested sections? Or should I just modify what I did to also handle the specific case of a Template containing List<Section>?
Thanks!

Model binding is indeed recursive but the MutableObjectModelBinder doesn't handle collections; it's specific to complex objects. Suggest adding a binder before the GenericModelBinder for List<Section>. Or ignore the list and change your MutableObjectModelBinder to handle Section as well. It'll get called as long as there is some data for a list element.

@dougbu I did not find much documentation on this, which generic Binder should I use as a base for my custom one if I want to add one before the GenericModelBinder? the CollectionModelBinder?

the CollectionModelBinder?

It depends on the details of your requirements. But it's not clear why your TemplateSectionModelBinder isn't already used to create Sections. I suspect the full names of the fields in the submission aren't matching the conventions model binding uses.

I used Model.GetType().FullName for my fields. When in debug my breckpoint in the TemplateSectionModelBinder was only hit once when receiving Template. I'll do more tests tomorrow as you're saying it is supposed to work 😄

Field names are important. The system expects ?Sections[0].ModelType={name1}&Sections[1].ModelType={name2} or a few more-complicated variants.

What you have introduced with Activator.CreateInstance(type) is a painful serialization vulnerability, which we worked very hard to eliminate, and one which the Java community is currently having to deal with,
You are basically taking untrusted user input, and allow an attacker to create any type of object they wish on your server. We, that is ASP.NET and the .NET Framework, have had multiple MSRC cases and Patch Tuesday events to fix this kind of thing, which can result in side effects ranging from simple file deletions to full blown remote code execution.
One of the issues was so bad that we issued the first ASP.NET patch which deliberately broke back compatibility and ran the risk of taking web sites offline (although we worked very hard to make sure that it didn't, and had no known incidents of this happening).
Please don't do this.

?Sections[0].ModelType={name1}&Sections[1].ModelType={name2}

@dougbu I confirm that my fields are posted like this
@blowdart What is your recomandation for this use case? I thought that if (!typeof(Section).IsAssignableFrom(type)) would prevent types other than sections to be created.

After some more testing, looking at the sources for CollectionModelBinder and this other issue #4196,  I managed to write a ModelBuilder that "work" but not so sure if it is done in a "good" way. There's probably some security issues too but TBH I don't know how to it without the custom model binder. 😿
This is my new implementation :
public class TemplateSectionModelBinder : IModelBinder
    {
        public async Task<ModelBindingResult> BindModelAsync(ModelBindingContext bindingContext)
        {
            if (bindingContext.ModelType == typeof(Section) && bindingContext.Model == null)
            {
                var typeName = bindingContext.ValueProvider.GetValue(bindingContext.ModelName + ".ModelType").FirstValue as string;
                if (typeName != null)
                {
                    var type = Type.GetType(typeName, true);
                    if (!typeof(Section).IsAssignableFrom(type))
                    {
                        throw new InvalidOperationException("Bad Type");
                    }
                    var model = Activator.CreateInstance(type);
                    bindingContext.Model = model;
                    var result = await bindingContext.OperationBindingContext.ModelBinder.BindModelAsync(bindingContext);

                    var metadataProvider = bindingContext.OperationBindingContext.MetadataProvider;
                    var elementMetadata = metadataProvider.GetMetadataForType(type);
                    var childBindingContext = ModelBindingContext.CreateChildBindingContext(
                        bindingContext,
                        elementMetadata,
                        fieldName: bindingContext.FieldName,
                        modelName: bindingContext.ModelName,
                        model: model);

                    var childResult = await bindingContext.OperationBindingContext.ModelBinder.BindModelAsync(childBindingContext);

                    return result;
                }
            }
            return await ModelBindingResult.NoResultAsync;
        }
    }

Why call BindModelAsync() the second time? You're throwing childResult away anyway and result should already be correct.
I'll let @blowdart make any security recommendations.

@dougbu I tried a first time without the following part but only the base class attributes were binded and I was missing all the specific values for the children class.
var metadataProvider = bindingContext.OperationBindingContext.MetadataProvider;
var elementMetadata = metadataProvider.GetMetadataForType(type);
var childBindingContext = ModelBindingContext.CreateChildBindingContext(
                        bindingContext,
                        elementMetadata,
                        fieldName: bindingContext.FieldName,
                        modelName: bindingContext.ModelName,
                        model: model);

var childResult = await bindingContext.OperationBindingContext.ModelBinder.BindModelAsync(childBindingContext);

Is AssignableFrom might protect you, but it's I'm never going to say its 100% safe. It depends on your object model.

I tried a first time without the following part but only the base class attributes

Model binding based on runtime types has performance and security issues. But if you want to pay those penalties, use childResult and do not call BindModelAsync() the first time.

IsAssignableFrom might protect you

May add a typeName.StartsWith("known namespace") for some additional protection.

@dougbu @blowdart Thanks to both of you, I'll modify the code to take into account your recommandations 😄

Hi, I'm struggling a little bit with the migration of this modelbinder to RC2.
I did some changes to add the GetBinder method like you wrote in #4703 but I could not find documentation on how to declare the custom binders (did not find some on the changes that occured for modelbinders with RC2 neither).
Thanks for the answers!

@dougbu , when you have a chance, can you answer?

@pierre-weceipt our documentation hasn't (yet) caught up w/ the new IModelBinderProvider split from IModelBinders. My #4703 response left out a bit mentioned earlier in that issue.
Add the following to your Startup.ConfigureServices() method:
services.AddMvc().Services.Configure<MvcOptions>(options => {
    options.ModelBinderProviders.Insert(0, new MessageModelBinderProvider());
});

Please respond here if I haven't answered your question or close the issue if your original question is now resolved.

I managed to declare my provider in startup.cs but the binder is not doing anything anymore. Not so sure I understood the whole changes or how the ChildBindingContext was replaced :/
Here my code :
public class TemplateSectionModelBinderProvider : IModelBinderProvider
    {
        public IModelBinder GetBinder(ModelBinderProviderContext context)
        {
            if (context == null)
            {
                throw new ArgumentNullException(nameof(context));
            }

            if (context.Metadata.ModelType == typeof(ContentBlock) || context.Metadata.ModelType == typeof(Condition))
            {

                var metadata = context.MetadataProvider.GetMetadataForType(context.Metadata.ModelType);
                var binder = context.CreateBinder(metadata);

                return new TemplateSectionModelBinder(context.MetadataProvider, binder);
            }
            return null;
        }
    }

 public class TemplateSectionModelBinder : IModelBinder
    {
        private readonly IModelMetadataProvider _metadataProvider;
        private readonly IModelBinder _modelBinder;

        public TemplateSectionModelBinder(
            IModelMetadataProvider metadataProvider,
            IModelBinder modelBinder)
        {
            _metadataProvider = metadataProvider;
            _modelBinder = modelBinder;
        }

        public Task BindModelAsync(ModelBindingContext bindingContext)
        {
            var typeNameResult = bindingContext.ValueProvider.GetValue(bindingContext.ModelName + ".ModelType");
            if (typeNameResult == ValueProviderResult.None)
            {
                bindingContext.Result = ModelBindingResult.Failed(bindingContext.ModelName);
                return Task.FromResult(0);
            }
            var typeName = typeNameResult.FirstValue;
            var type = Type.GetType(typeName, true);

            if (!typeof(ContentBlock).IsAssignableFrom(type) && !typeof(Condition).IsAssignableFrom(type) || !typeName.StartsWith("Thelos.NAO.Models.Templates"))
            {
                throw new InvalidOperationException("Bad Type");
            }
            var metadata = _metadataProvider.GetMetadataForType(type);
            var model = Activator.CreateInstance(type);
            bindingContext.Model = model;
            using (bindingContext.EnterNestedScope(
                metadata,
                bindingContext.FieldName,
                bindingContext.ModelName,
                model
                ))
            {
                return _modelBinder.BindModelAsync(bindingContext);
            }
        }
    }

What do you mean by "not doing anything"? I'm surprised the code you show doesn't overflow the stack; your IModelBinderProvider needs a guard to ensure context.CreateBinder(metadata) doesn't also return a TemplateSectionModelBinder.
As @blowdart mentioned, "IsAssignableFrom might protect you but ...".
In any case, please see #4703 for a custom model binder & provider pair that at least create the correct inner binders and do it once. But note you probably hit the same issue as I did in #4703: The original bindingContext.Result is not set.

Ok, when I say "not doing anything" I mean that the model does not get binded and I end up with an empty model in my controller. I'll try with the changes you suggested in your last comment.
I'm not so clear about why context.CreateBinder(metadata) could return a TemplateSectionModelBinder.
I think that when the doc will be out I'll better understand the whole concepts between the binder and the provider.
Thanks a lot for the time spent answering these questions 👍

I managed to make it work again, thanks :)

@pierre-weceipt - what was your fix?

@janhartmann I used the following classes
public class TemplateSectionModelBinderProvider : IModelBinderProvider
    {
        public IModelBinder GetBinder(ModelBinderProviderContext context)
        {
            if (context == null)
            {
                throw new ArgumentNullException(nameof(context));
            }

            if (context.Metadata.ModelType == typeof(ContentBlock) || context.Metadata.ModelType == typeof(Condition))
            {
                var binders = new Dictionary<string, IModelBinder>();
                foreach (var type in typeof(TemplateSectionModelBinderProvider).GetTypeInfo().Assembly.GetTypes())
                {
                    var typeInfo = type.GetTypeInfo();
                    if (typeInfo.IsAbstract || typeInfo.IsNested)
                    {
                        continue;
                    }

                    if (!(typeInfo.IsClass || typeInfo.IsPublic))
                    {
                        continue;
                    }

                    var metadata = context.MetadataProvider.GetMetadataForType(type);
                    var binder = context.CreateBinder(metadata);
                    binders.Add(type.FullName, binder);
                }

                return new TemplateSectionModelBinder(context.MetadataProvider, binders);

            }
            return null;
        }
    }

public class TemplateSectionModelBinder : IModelBinder
    {
        private readonly IModelMetadataProvider _metadataProvider;
        private readonly Dictionary<string, IModelBinder> _binders;

        public TemplateSectionModelBinder(
            IModelMetadataProvider metadataProvider,
            Dictionary<string, IModelBinder> binders
            )
        {
            _metadataProvider = metadataProvider;
            _binders = binders;
        }

        public async Task BindModelAsync(ModelBindingContext bindingContext)
        {
            var typeResult = bindingContext.ValueProvider.GetValue(bindingContext.ModelName + ".Type");
            if (typeResult == ValueProviderResult.None)
            {
                bindingContext.Result = ModelBindingResult.Failed();
                return;
            }

            IModelBinder binder;
            if (!_binders.TryGetValue(typeResult.FirstValue, out binder))
            {
                bindingContext.Result = ModelBindingResult.Failed();
                return;
            }

            // Now know the type exists in the assembly.
            var type = Type.GetType(typeResult.FirstValue);

            if (!typeof(ContentBlock).IsAssignableFrom(type) && !typeof(Condition).IsAssignableFrom(type) || !typeResult.FirstValue.StartsWith("Thelos.NAO.Models.Templates"))
            {
                throw new InvalidOperationException("Bad Type");
            }

            var metadata = _metadataProvider.GetMetadataForType(type);
            var model = Activator.CreateInstance(type);
            var result = bindingContext.Result;
            bindingContext.Model = model;

            using (bindingContext.EnterNestedScope(
                metadata,
                bindingContext.FieldName,
                bindingContext.ModelName,
                model: model))
            {
                await binder.BindModelAsync(bindingContext);
            }

            bindingContext.Result = ModelBindingResult.Success(model);
            return;
        }
    }


components 里的模块更改不会触发 hot reload

App.vue 会触发 hot reload
Hello.vue 不会

@haozime Can you link to a simple example project with the problem?

@chrisvfritz
+1
App.vue hot reload
Hello.vue do not hot reload
Default Template can show this issue.

Could you use browser-sync replace express?

This is working as intended. What you did is update the initial value of that data property, but after the component is loaded, hot reloading maintains the current state, so that change in initial state makes no difference.


android-messages: Update to 5.3.4

and switch to maintained fork.
The current version 3.0.1 points to 1.0.1.
Source: https://github.com/rasa/scoops/blob/master/bucket/android-messages.json

[x] I have read the Contributing Guide.

/verify
Is this the official fork?

Is this the official fork?

@rashil2000 If you define "the official fork" as it's being provided by Homebrew, then yes, though Homebrew provides the stale version too. It has 493 quality commits since forking, so it's worth the switch.


Bit operations

Logical bit operations
The and, or, not, xor bit operations on int types are fairly straightforward and uncontroversial. They can be implemented by std library functions such as bit_and, etc.
Making them into operators would necessitate using some obscure symbols for some of them (e.g. not and xor don't have obvious good candidates). Since these aren't necessarily super-common operations, we can use functions instead.
Bit shifting
There is some general concensus in programming when it comes to how to apply bit shifting to int types, when the number of bits used in bit shifting is between 0 and the bit length of the int representation.
When shifting an unsigned int, the bit shifting happens as one might expect the logical operation to happen and the newly added bits are 0.
When shifting a signed int, so-called "arithmetic" shifting is used, which preserves the sign bit. Well, the bit is preserved for right shifts while it can be overwritten in left shifts.
There's no concensus in programming languages on how bit shifting operators are generalized when the operand indicating the number of bits is either negative or greater than the bit length of the int in question.
Postgres
First let's see what happens as the number of bits right-shifted increases in Postgres in more detail:
 val | numbits | result |            bitresult
-----+---------+--------+----------------------------------
 123 |       0 |    123 |<PHONE_NUMBER>0000000000000001111011
 123 |       1 |     61 |<PHONE_NUMBER>0000000000000000111101
 123 |       2 |     30 |<PHONE_NUMBER>0000000000000000011110
 123 |       3 |     15 |<PHONE_NUMBER>0000000000000000001111
 123 |       4 |      7 |<PHONE_NUMBER>0000000000000000000111
 123 |       5 |      3 |<PHONE_NUMBER>0000000000000000000011
 123 |       6 |      1 |<PHONE_NUMBER>0000000000000000000001
 123 |       7 |      0 |<PHONE_NUMBER>0000000000000000000000
 123 |       8 |      0 |<PHONE_NUMBER>0000000000000000000000
 123 |       9 |      0 |<PHONE_NUMBER>0000000000000000000000
 123 |      10 |      0 |<PHONE_NUMBER>0000000000000000000000
 123 |      11 |      0 |<PHONE_NUMBER>0000000000000000000000
 123 |      12 |      0 |<PHONE_NUMBER>0000000000000000000000
 123 |      13 |      0 |<PHONE_NUMBER>0000000000000000000000
 123 |      14 |      0 |<PHONE_NUMBER>0000000000000000000000
 123 |      15 |      0 |<PHONE_NUMBER>0000000000000000000000
 123 |      16 |      0 |<PHONE_NUMBER>0000000000000000000000
 123 |      17 |      0 |<PHONE_NUMBER>0000000000000000000000
 123 |      18 |      0 |<PHONE_NUMBER>0000000000000000000000
 123 |      19 |      0 |<PHONE_NUMBER>0000000000000000000000
 123 |      20 |      0 |<PHONE_NUMBER>0000000000000000000000
 123 |      21 |      0 |<PHONE_NUMBER>0000000000000000000000
 123 |      22 |      0 |<PHONE_NUMBER>0000000000000000000000
 123 |      23 |      0 |<PHONE_NUMBER>0000000000000000000000
 123 |      24 |      0 |<PHONE_NUMBER>0000000000000000000000
 123 |      25 |      0 |<PHONE_NUMBER>0000000000000000000000
 123 |      26 |      0 |<PHONE_NUMBER>0000000000000000000000
 123 |      27 |      0 |<PHONE_NUMBER>0000000000000000000000
 123 |      28 |      0 |<PHONE_NUMBER>0000000000000000000000
 123 |      29 |      0 |<PHONE_NUMBER>0000000000000000000000
 123 |      30 |      0 |<PHONE_NUMBER>0000000000000000000000
 123 |      31 |      0 |<PHONE_NUMBER>0000000000000000000000
 123 |      32 |    123 |<PHONE_NUMBER>0000000000000001111011
 123 |      33 |     61 |<PHONE_NUMBER>0000000000000000111101
 123 |      34 |     30 |<PHONE_NUMBER>0000000000000000011110
 123 |      35 |     15 |<PHONE_NUMBER>0000000000000000001111
 123 |      36 |      7 |<PHONE_NUMBER>0000000000000000000111
 123 |      37 |      3 |<PHONE_NUMBER>0000000000000000000011
 123 |      38 |      1 |<PHONE_NUMBER>0000000000000000000001
 123 |      39 |      0 |<PHONE_NUMBER>0000000000000000000000
 123 |      40 |      0 |<PHONE_NUMBER>0000000000000000000000

It appears that the way shifting works is that the number of bits shifted gets truncated by taking a positive mod 32 of it. Same thing appears to happen for negative vaues of bits shifted:
 val | numbits | result |            bitresult
-----+---------+--------+----------------------------------
 123 |       0 |    123 |<PHONE_NUMBER>0000000000000001111011
 123 |      -1 |      0 |<PHONE_NUMBER>0000000000000000000000
 123 |      -2 |      0 |<PHONE_NUMBER>0000000000000000000000
 123 |      -3 |      0 |<PHONE_NUMBER>0000000000000000000000
 123 |      -4 |      0 |<PHONE_NUMBER>0000000000000000000000
 123 |      -5 |      0 |<PHONE_NUMBER>0000000000000000000000
 123 |      -6 |      0 |<PHONE_NUMBER>0000000000000000000000
 123 |      -7 |      0 |<PHONE_NUMBER>0000000000000000000000
 123 |      -8 |      0 |<PHONE_NUMBER>0000000000000000000000
 123 |      -9 |      0 |<PHONE_NUMBER>0000000000000000000000
 123 |     -10 |      0 |<PHONE_NUMBER>0000000000000000000000
 123 |     -11 |      0 |<PHONE_NUMBER>0000000000000000000000
 123 |     -12 |      0 |<PHONE_NUMBER>0000000000000000000000
 123 |     -13 |      0 |<PHONE_NUMBER>0000000000000000000000
 123 |     -14 |      0 |<PHONE_NUMBER>0000000000000000000000
 123 |     -15 |      0 |<PHONE_NUMBER>0000000000000000000000
 123 |     -16 |      0 |<PHONE_NUMBER>0000000000000000000000
 123 |     -17 |      0 |<PHONE_NUMBER>0000000000000000000000
 123 |     -18 |      0 |<PHONE_NUMBER>0000000000000000000000
 123 |     -19 |      0 |<PHONE_NUMBER>0000000000000000000000
 123 |     -20 |      0 |<PHONE_NUMBER>0000000000000000000000
 123 |     -21 |      0 |<PHONE_NUMBER>0000000000000000000000
 123 |     -22 |      0 |<PHONE_NUMBER>0000000000000000000000
 123 |     -23 |      0 |<PHONE_NUMBER>0000000000000000000000
 123 |     -24 |      0 |<PHONE_NUMBER>0000000000000000000000
 123 |     -25 |      0 |<PHONE_NUMBER>0000000000000000000000
 123 |     -26 |      1 |<PHONE_NUMBER>0000000000000000000001
 123 |     -27 |      3 |<PHONE_NUMBER>0000000000000000000011
 123 |     -28 |      7 |<PHONE_NUMBER>0000000000000000000111
 123 |     -29 |     15 |<PHONE_NUMBER>0000000000000000001111
 123 |     -30 |     30 |<PHONE_NUMBER>0000000000000000011110
 123 |     -31 |     61 |<PHONE_NUMBER>0000000000000000111101
 123 |     -32 |    123 |<PHONE_NUMBER>0000000000000001111011
 123 |     -33 |      0 |<PHONE_NUMBER>0000000000000000000000
 123 |     -34 |      0 |<PHONE_NUMBER>0000000000000000000000
 123 |     -35 |      0 |<PHONE_NUMBER>0000000000000000000000
 123 |     -36 |      0 |<PHONE_NUMBER>0000000000000000000000
 123 |     -37 |      0 |<PHONE_NUMBER>0000000000000000000000
 123 |     -38 |      0 |<PHONE_NUMBER>0000000000000000000000
 123 |     -39 |      0 |<PHONE_NUMBER>0000000000000000000000
 123 |     -40 |      0 |<PHONE_NUMBER>0000000000000000000000

Left-shifting has a similar repeating pattern. Since we're shifting a signed integer, eventually the sign bit starts getting affected producing negative values, but nothing surprising here:
 val | numbits |   result    |            bitresult
-----+---------+-------------+----------------------------------
 123 |       0 |         123 |<PHONE_NUMBER>0000000000000001111011
 123 |       1 |         246 |<PHONE_NUMBER>0000000000000011110110
 123 |       2 |         492 |<PHONE_NUMBER>0000000000000111101100
 123 |       3 |         984 |<PHONE_NUMBER>0000000000001111011000
 123 |       4 |        1968 |<PHONE_NUMBER>0000000000011110110000
 123 |       5 |        3936 |<PHONE_NUMBER>0000000000111101100000
 123 |       6 |        7872 |<PHONE_NUMBER>0000000001111011000000
 123 |       7 |       15744 |<PHONE_NUMBER>0000000011110110000000
 123 |       8 |       31488 |<PHONE_NUMBER>0000000111101100000000
 123 |       9 |       62976 |<PHONE_NUMBER>0000001111011000000000
 123 |      10 |      125952 |<PHONE_NUMBER>0000011110110000000000
 123 |      11 |      251904 |<PHONE_NUMBER>0000111101100000000000
 123 |      12 |      503808 |<PHONE_NUMBER>0001111011000000000000
 123 |      13 |     1007616 |<PHONE_NUMBER>0011110110000000000000
 123 |      14 |     2015232 |<PHONE_NUMBER>0111101100000000000000
 123 |      15 |     4030464 |<PHONE_NUMBER>1111011000000000000000
 123 |      16 |     8060928 |<PHONE_NUMBER>1110110000000000000000
 123 |      17 |    16121856 |<PHONE_NUMBER>1101100000000000000000
 123 |      18 |    32243712 |<PHONE_NUMBER>1011000000000000000000
 123 |      19 |    64487424 |<PHONE_NUMBER>0110000000000000000000
 123 |      20 |   128974848 |<PHONE_NUMBER>1100000000000000000000
 123 |      21 |   257949696 |<PHONE_NUMBER>1000000000000000000000
 123 |      22 |   515899392 |<PHONE_NUMBER>0000000000000000000000
 123 |      23 |<PHONE_NUMBER> |<PHONE_NUMBER>0000000000000000000000
 123 |      24 |<PHONE_NUMBER> |<PHONE_NUMBER>0000000000000000000000
 123 |      25 |  -167772160 |<PHONE_NUMBER>0000000000000000000000
 123 |      26 |  -335544320 |<PHONE_NUMBER>0000000000000000000000
 123 |      27 |  -671088640 |<PHONE_NUMBER>0000000000000000000000
 123 |      28 | -1342177280 |<PHONE_NUMBER>0000000000000000000000
 123 |      29 |<PHONE_NUMBER> |<PHONE_NUMBER>0000000000000000000000
 123 |      30 | -1073741824 |<PHONE_NUMBER>0000000000000000000000
 123 |      31 | -2147483648 |<PHONE_NUMBER>0000000000000000000000
 123 |      32 |         123 |<PHONE_NUMBER>0000000000000001111011
 123 |      33 |         246 |<PHONE_NUMBER>0000000000000011110110
 123 |      34 |         492 |<PHONE_NUMBER>0000000000000111101100
 123 |      35 |         984 |<PHONE_NUMBER>0000000000001111011000
 123 |      36 |        1968 |<PHONE_NUMBER>0000000000011110110000
 123 |      37 |        3936 |<PHONE_NUMBER>0000000000111101100000
 123 |      38 |        7872 |<PHONE_NUMBER>0000000001111011000000
 123 |      39 |       15744 |<PHONE_NUMBER>0000000011110110000000
 123 |      40 |       31488 |<PHONE_NUMBER>0000000111101100000000

A thing worth noting, though, is that left-shifting is not the same as simply multiplying by 2. It is fundamentally an agnostic manipulation of the underlying bit pattern, otherwise sign-flipping wouldn't happen.
Left-shifting by negative values has effects consistent with just taking a positive mod 32 value of the bit shift argument.
Right-shifting negative integers preserves the sign bit, resulting in the following:
 val  | numbits | result |            bitresult
------+---------+--------+----------------------------------
 -123 |       0 |   -123 |<PHONE_NUMBER>1111111111111110000101
 -123 |       1 |    -62 |<PHONE_NUMBER>1111111111111111000010
 -123 |       2 |    -31 |<PHONE_NUMBER>1111111111111111100001
 -123 |       3 |    -16 |<PHONE_NUMBER>1111111111111111110000
 -123 |       4 |     -8 |<PHONE_NUMBER>1111111111111111111000
 -123 |       5 |     -4 |<PHONE_NUMBER>1111111111111111111100
 -123 |       6 |     -2 |<PHONE_NUMBER>1111111111111111111110
 -123 |       7 |     -1 |<PHONE_NUMBER>1111111111111111111111
 -123 |       8 |     -1 |<PHONE_NUMBER>1111111111111111111111
 -123 |       9 |     -1 |<PHONE_NUMBER>1111111111111111111111
 -123 |      10 |     -1 |<PHONE_NUMBER>1111111111111111111111
 -123 |      11 |     -1 |<PHONE_NUMBER>1111111111111111111111
 -123 |      12 |     -1 |<PHONE_NUMBER>1111111111111111111111
 -123 |      13 |     -1 |<PHONE_NUMBER>1111111111111111111111
 -123 |      14 |     -1 |<PHONE_NUMBER>1111111111111111111111
 -123 |      15 |     -1 |<PHONE_NUMBER>1111111111111111111111
 -123 |      16 |     -1 |<PHONE_NUMBER>1111111111111111111111
 -123 |      17 |     -1 |<PHONE_NUMBER>1111111111111111111111
 -123 |      18 |     -1 |<PHONE_NUMBER>1111111111111111111111
 -123 |      19 |     -1 |<PHONE_NUMBER>1111111111111111111111
 -123 |      20 |     -1 |<PHONE_NUMBER>1111111111111111111111
 -123 |      21 |     -1 |<PHONE_NUMBER>1111111111111111111111
 -123 |      22 |     -1 |<PHONE_NUMBER>1111111111111111111111
 -123 |      23 |     -1 |<PHONE_NUMBER>1111111111111111111111
 -123 |      24 |     -1 |<PHONE_NUMBER>1111111111111111111111
 -123 |      25 |     -1 |<PHONE_NUMBER>1111111111111111111111
 -123 |      26 |     -1 |<PHONE_NUMBER>1111111111111111111111
 -123 |      27 |     -1 |<PHONE_NUMBER>1111111111111111111111
 -123 |      28 |     -1 |<PHONE_NUMBER>1111111111111111111111
 -123 |      29 |     -1 |<PHONE_NUMBER>1111111111111111111111
 -123 |      30 |     -1 |<PHONE_NUMBER>1111111111111111111111
 -123 |      31 |     -1 |<PHONE_NUMBER>1111111111111111111111
 -123 |      32 |   -123 |<PHONE_NUMBER>1111111111111110000101
 -123 |      33 |    -62 |<PHONE_NUMBER>1111111111111111000010
 -123 |      34 |    -31 |<PHONE_NUMBER>1111111111111111100001
 -123 |      35 |    -16 |<PHONE_NUMBER>1111111111111111110000
 -123 |      36 |     -8 |<PHONE_NUMBER>1111111111111111111000
 -123 |      37 |     -4 |<PHONE_NUMBER>1111111111111111111100
 -123 |      38 |     -2 |<PHONE_NUMBER>1111111111111111111110
 -123 |      39 |     -1 |<PHONE_NUMBER>1111111111111111111111
 -123 |      40 |     -1 |<PHONE_NUMBER>1111111111111111111111

This is equivalent of first performing a bit_not on the integer, perform a bit shift and another bit_not at the end. It happens to be the case that the arithmetic unary - performs a 2's complement on an integer and so the combination of the bit_not and bit shifting works.
Also worth noting that in Postgres right-shifting a negative value is not the same as division by 2. E.g. select -123/2 gets -61.
Left-shifting a negative integer will overwrite the sign bit with whatever bits that get shifted, just like for positive integers. This results in the sign potentially flipping due to left-shifting:
 val  | numbits |   result    |            bitresult
------+---------+-------------+----------------------------------
 -123 |       0 |        -123 |<PHONE_NUMBER>1111111111111110000101
 -123 |       1 |        -246 |<PHONE_NUMBER>1111111111111100001010
 -123 |       2 |        -492 |<PHONE_NUMBER>1111111111111000010100
 -123 |       3 |        -984 |<PHONE_NUMBER>1111111111110000101000
 -123 |       4 |       -1968 |<PHONE_NUMBER>1111111111100001010000
 -123 |       5 |       -3936 |<PHONE_NUMBER>1111111111000010100000
 -123 |       6 |       -7872 |<PHONE_NUMBER>1111111110000101000000
 -123 |       7 |      -15744 |<PHONE_NUMBER>1111111100001010000000
 -123 |       8 |      -31488 |<PHONE_NUMBER>1111111000010100000000
 -123 |       9 |      -62976 |<PHONE_NUMBER>1111110000101000000000
 -123 |      10 |     -125952 |<PHONE_NUMBER>1111100001010000000000
 -123 |      11 |     -251904 |<PHONE_NUMBER>1111000010100000000000
 -123 |      12 |     -503808 |<PHONE_NUMBER>1110000101000000000000
 -123 |      13 |    -1007616 |<PHONE_NUMBER>1100001010000000000000
 -123 |      14 |    -2015232 |<PHONE_NUMBER>1000010100000000000000
 -123 |      15 |    -4030464 |<PHONE_NUMBER>0000101000000000000000
 -123 |      16 |    -8060928 |<PHONE_NUMBER>0001010000000000000000
 -123 |      17 |   -16121856 |<PHONE_NUMBER>0010100000000000000000
 -123 |      18 |   -32243712 |<PHONE_NUMBER>0101000000000000000000
 -123 |      19 |   -64487424 |<PHONE_NUMBER>1010000000000000000000
 -123 |      20 |  -128974848 |<PHONE_NUMBER>0100000000000000000000
 -123 |      21 |  -257949696 |<PHONE_NUMBER>1000000000000000000000
 -123 |      22 |  -515899392 |<PHONE_NUMBER>0000000000000000000000
 -123 |      23 | -1031798784 |<PHONE_NUMBER>0000000000000000000000
 -123 |      24 | -2063597568 |<PHONE_NUMBER>0000000000000000000000
 -123 |      25 |   167772160 |<PHONE_NUMBER>0000000000000000000000
 -123 |      26 |   335544320 |<PHONE_NUMBER>0000000000000000000000
 -123 |      27 |   671088640 |<PHONE_NUMBER>0000000000000000000000
 -123 |      28 |<PHONE_NUMBER> |<PHONE_NUMBER>0000000000000000000000
 -123 |      29 | -1610612736 |<PHONE_NUMBER>0000000000000000000000
 -123 |      30 |<PHONE_NUMBER> |<PHONE_NUMBER>0000000000000000000000
 -123 |      31 | -2147483648 |<PHONE_NUMBER>0000000000000000000000
 -123 |      32 |        -123 |<PHONE_NUMBER>1111111111111110000101
 -123 |      33 |        -246 |<PHONE_NUMBER>1111111111111100001010
 -123 |      34 |        -492 |<PHONE_NUMBER>1111111111111000010100
 -123 |      35 |        -984 |<PHONE_NUMBER>1111111111110000101000
 -123 |      36 |       -1968 |<PHONE_NUMBER>1111111111100001010000
 -123 |      37 |       -3936 |<PHONE_NUMBER>1111111111000010100000
 -123 |      38 |       -7872 |<PHONE_NUMBER>1111111110000101000000
 -123 |      39 |      -15744 |<PHONE_NUMBER>1111111100001010000000
 -123 |      40 |      -31488 |<PHONE_NUMBER>1111111000010100000000

Contrasting this with bit shifting operations on bitstrings:
V2f144f3c41_edgedb=# select (123::int4)::bit(32) >> 2;
             ?column?
----------------------------------
<PHONE_NUMBER>0000000000000000011110
(1 row)

V2f144f3c41_edgedb=# select (123::int4)::bit(32) >> 34;
             ?column?
----------------------------------
<PHONE_NUMBER>0000000000000000000000
(1 row)

V2f144f3c41_edgedb=# select (123::int4)::bit(32) >> -2;
             ?column?
----------------------------------
<PHONE_NUMBER>0000000000000111101100
(1 row)

V2f144f3c41_edgedb=# select (123::int4)::bit(32) >> -30;
             ?column?
----------------------------------
<PHONE_NUMBER>0000000000000000000000
(1 row)

V2f144f3c41_edgedb=# select (-123::int4)::bit(32) >> 2;
             ?column?
----------------------------------
<PHONE_NUMBER>1111111111111111100001
(1 row)

V2f144f3c41_edgedb=# select (-123::int4)::bit(32) >> 34;
             ?column?
----------------------------------
<PHONE_NUMBER>0000000000000000000000
(1 row)

V2f144f3c41_edgedb=# select (-123::int4)::bit(32) >> -2;
             ?column?
----------------------------------
<PHONE_NUMBER>1111111111111000010100
(1 row)

V2f144f3c41_edgedb=# select (-123::int4)::bit(32) >> -30;
             ?column?
----------------------------------
<PHONE_NUMBER>0000000000000000000000
(1 row)

The positive and negative shift values work about as directly as you'd expect, actually being the bit offset. Any time the offset is larger than the length of the bitstring, the bits get shifted outside of the string without any looping effects. So shifting a bit(32) string by 32, 40, or 4000 all result in simply a bitstring full of 0s, regardless of direction of the shift.
To summarize, the bit shifting operators for integers appear to take the positive value of mod 32 of the second argument before performing the bit shift. Right-shifting negative integers preserves the sign bit.
I used this query (and some variations of it) to get the Postgres data:
SELECT data.val, data.numbits, data.result, data.result::bit(32) AS bitresult
FROM (
    SELECT raw.val, raw.numbits, raw.val >> raw.numbits AS result
    FROM (
        SELECT
            123::int4 AS val, n AS numbits
        FROM generate_series(0, 40) AS n
    ) AS raw
) AS data;

Python
Without going into as many details, here's the summary of Python's approach to bit shifting operators:

It is a ValueError to shift by a negative value.
Any positive shift value is valid for both left and right bit shifting.
A positive integer being bit shifted behaves as if there was an unlimited number of bits available to represent the integer, the sign never flips.
A negative integer being bit shifted always preserves the sign bit no matter what. Otherwise the operation behaves as if there's an unlimited number of bits available for the integer.
This is about as close as it gets to the pure "it's a logical bit shift for positive integers and an arithemtic bit shift for negative integers".

GO
Without going into as many details, here's the summary of GO's approach to bit shifting operators:

It is an error to shift by a negative value.
Any positive shift value is valid for both left and right bit shifting.
A positive integer being bit shifted behaves as if we are performing a logical bit shift of the underlying bit pattern without ever changing the total number of significant bits we consider. We then interpret the result as an integer, potentially flipping the sign. Shifting by a large enough number of bits in either direction just results in 0.
A negative integer being right-shifted preserves the sign bit. Eventually right-shifting resutls in -1 regardless of how much more you shift. Left-shifting a negative integer works the same as for positive integers, eventually shifting it all the way to 0.
This is similar to Python, but because there's no unlimited number of bits available to represent the underlying integers, the results of logical and arithmetic bit shifts gets clipped to the integer size.

JavaScript
Without going into details JavaScript treats bit shift operators pretty much the same as Postgres including taking the positive mod 32 of the second argument. The main difference is in how many significant bits integers have and when exactly sign bit gets affected by bit shifting.
Recommendation for EdgeDB

It may be reasonable to require the second argument in bit_rshift and bit_lshift to be positive and an error otherwise. Generally, shifting by negative amounts is either forbidden in other languages (Python and GO) or behaves unintuitively (Postgres and JavaScript).
None of the other examples treat shifting a number beyond either edge of the bit pattern as overflow or underflow. It is simply allowed. This suggests that EdgeDB should allow shifting in either direction by any positive amount.
It seems that there's concensus that negative values should use arithemtic (sign-bit preserving) shifting when shifted to the right, specifically.
It seems that shifting to the left can eventually flip sign bits in any language where shifting applies to a finite integer (Postgres, GO, JavaScript).
It seems intuitive that shifting by n bits in some direction should get the same result regardless of whether the shift happens in a single step or multiple steps. This makes any approach taking a mod of the second argument unacceptable. If it's legal to shift a number by 1 bit 100 times, it should be just as legal to shift it by 100 bits 1 time and it should produce the same result.
Postgres does not seem to allow casting numeric into bit, so we probably can't have a practical implementation of bit shifting for bigint (like Python does) in EdgeDB.

From all of the above it seems that EdgeDB should have pretty much the same approach as GO. This can be achieved by using the Postres's operators for shifting by a positive amount smaller than the bit length of a given integer type, while using CASE WHEN branches to provide an error for negative shifts and 0 or -1 for shifts beyond the bit length.
P.S. The reason why Postgres examples use int4 and not a shorter int2 is because you cannot cast an int2 into bit(16) and so showing the explicit bit-pattern would not be easy. The >> and << are still valid for int2, though.
P.P.S. The mod 32 behavior is applicable to both int2 and int4 in Postgres, whereas bit shifting int8 uses mod 64 for the second argument instead.
P.P.P.S. I haven't tried checking whether big- and little-endian encoding has any effect on the interaction of bit-shifting operators and ints, but it might. In practical terms it seems that modern systems use LE or support LE, so this concern is not likely to come up.

So these operators in postgres, C, Rust behave like that because most common processors do that without additional checks. So technically if someone ever needs that performance we can provide quick_shl and quick_shr functions that compile directly to the operator. I still have concerns whether the performance difference will be noticeable in Postgres (i.e. at least only for JIT'ed queries, and so forth). And I guess that if noticeable amount of bit manipulation is needed it should be better solved by WebAssembly function eventually.
So yes, throwing on negative values and capping shift amount (instead of using mod) is the most promising solution.

To do a logical shift in terms of arithmetic shifts, we can do an arithmetic shift, and then mask off the top bits:
int16_t logical_rsh(int16_t n, int amt) {
    int16_t mask = ((int16_t)(1 << (16-1)) >> (amt)) << 1;
    return (n >> amt) & ~mask;
}

We've implemented logical operations as well as a few more standard library functions in #3784


weave dns-add/remove IP addresses should be optional

One should be able to invoke
$ weave dns-add container1 -h foo.weave.local

which should add DNS records for all the weave IP addresses of that container.
Ditto for removal.
This is actually made painfully obvious by the current dns-add example in the docs:

If you want to give the container a name in DNS other than its hostname, you can register it using the dns-add command.

You rang?

Just in case you fancy working on this :)

Just in case you fancy working on this :)

:) Yep will take a look after I've worked out this niggle with the release script.


[frontend] Create a working search bar

deprecated earlier, see #65 and #115
Would be cool to have

What is this search bar? Don't we already have a search bar on the Rankings page?


OPAM pin of submodules during CI checkout, not in Docker

When we build our Docker toolchain image, we do an OPAM pin for the version of the submodules that are currently installed on the docker build machine.
In CI, we do a submodule update, which puts particular versions of submodules in src/external, but has no effect on the pinned submodules in the Docker image. So if you update a submodule in a branch, without updating the Docker image, the updates won't be use.
It might be better to skip the pins in the Docker build, and do them after the submodule update in CI. That way, branches can freely use different versions of submodules, without having to build new images (a long and painful process).
That will mean that users of the image have to pin the submodules after pulling the Coda repo, and the README-dev.md will have to be updated accordingly.

See #3471 for an example of this.

We'll have to remove the -p flag to dune in the .opam files for submodules, else the pin will fail in CI.

In #3471, observed that pinning all submodules takes about 2min 30sec.

Note that DUNE_PROFILE should be set only for the OCaml build and the .deb build. The pinning breaks if it's set there.

Done in #3619.


Serious SQL generation issue

Steps to reproduce
I have some related models, limited for brevity:
public class Ticket : IDataModel<int>
{
    [Key]
    public int Id { get; set; }

    [Required]
    public User Creator { get; set; }

    [Required]
    public string Subject { get; set; }
}

public class User
{
    [Key]
    public int Id { get; set; }

    public int Importance { get; set; }
}

Here is an example of the linq I'm trying to do:
db.Set().OrderBy(t => t.Concerning.Importance).Select(t => new { Subject = t.Subject }).ToList();
The issue
The query results is "SqlException: Invalid column name 'Subject'.
The generated sql looks like:
SELECT [t1].[Subject]
FROM (
SELECT [t.Concerning0].*
FROM [Ticket] AS [t0]
INNER JOIN [User] AS [t.Concerning0] ON [t0].[ConcerningId] = [t.Concerning0].[Id]
ORDER BY ([t0].[Importance] + [t.Concerning0].[Importance])
) AS [t1]
The issue is that the generated sql is NOT returning the correct columns, it's not returning the columns from [Ticket] (ie: [t0]) as it should and is only returning the columns from the [User] table (ie: [t.Concerning0]) which aren't even used anyway.  If I get rid of the linq Select statement, then the query simplifies and is no longer nested and I get the Ticket instance as expected.  This only happens when trying to project with Select.
This is causing a complete work stoppage with my project and I expected better from the RTM release.  This problem was also present in RC2 when I checked.
Further technical details
EF Core version: 1.0.0
Operating system: Windows 7 Pro
Visual Studio version: VS2015

Im trying to reproduce this in a console app and haven't been able yet.  The SQL looks completely different.  I'm not sure why yet.


How can I add a folder_id to an upload?

I am combining django_filer with django_chunked_upload, and I am trying to upload a file to the folder location my url is currently pointed to. Instead, all chunked uploads go to an unsorted folder.
Is there a way I can add a folder_id to the chunked_upload views.py file?

@joeco23 Have you tried to change the CHUNKED_UPLOAD_TO settings variable?


envoy_reloadable_features_postpone_h3_client_connect_to_next_loop deprecation

Your change #19860 (quic: postpone conn pool connect ) introduced a runtime guarded feature. It has been 6 months since the new code has been exercised by default, so it's time to remove the old code path. This issue tracks source code cleanup so we don't forget.

@alyssawilk  Can I work on this task?

There are TODOs added in https://github.com/envoyproxy/envoy/pull/19860 which requires some code refactoring. Would you like also to work on those?

@danzh2010 Sure I can work on TODOs as well.

If you grep postpone_h3_client_connect_to_next_loop, you will fine TODOs mentioning it, basically:
https://github.com/envoyproxy/envoy/blob/84ca17add2171c04876aaf345762f03b4d6c86b8/source/common/http/codec_client.h#L313 and
https://github.com/envoyproxy/envoy/blob/763891dbf61e2027463b4e0f701c19ec843ef6a7/source/common/http/http3/conn_pool.cc#L181
Thanks for help cleaning it up!

/assign @sgovinda-ub

If you grep postpone_h3_client_connect_to_next_loop, you will fine TODOs mentioning it, basically:
https://github.com/envoyproxy/envoy/blob/84ca17add2171c04876aaf345762f03b4d6c86b8/source/common/http/codec_client.h#L313
and
https://github.com/envoyproxy/envoy/blob/763891dbf61e2027463b4e0f701c19ec843ef6a7/source/common/http/http3/conn_pool.cc#L181
Thanks for help cleaning it up!

@danzh2010  I have questions.

Should I create one PR with both TODO and runtime flag depreciation changes?
Regarding the TODO, can you please elaborate on deprecate this class and make CodecClientProd to have the option to defer connect

Should I create one PR with both TODO and runtime flag depreciation changes?

Preferably make a 2nd PR to address the TODOs.

Regarding the TODO, can you please elaborate on deprecate this class and make CodecClientProd to have the option to defer connect

CodecClientProd constructor is just NoConnectCodecClientProd constructor + connect():
https://github.com/envoyproxy/envoy/blob/84ca17add2171c04876aaf345762f03b4d6c86b8/source/common/http/codec_client.cc#L221. We should be able to add a bool parameter in CodecClientProd() for caller decide if they want to connect() or not, and get rid of NoConnectCodecClientProd().


Undefined object in "Add primary configuration items" task

I'm having trouble running the playbook on three Ubuntu 20.04 machines from an Ubuntu 20.04 host running Ansible 2.10.
The message I see, after running ansible-playbook -i inventory/my-cluster/hosts.ini site.yml -vvv, is:
TASK [rke2_common : Add primary configuration items] **************************************************************************************************************************************************************
task path: /home/jon/rke2-ansible/roles/rke2_common/tasks/config.yml:17
The full traceback is:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/ansible/executor/task_executor.py", line 585, in _execute
    self._task.post_validate(templar=templar)
  File "/usr/lib/python3/dist-packages/ansible/playbook/task.py", line 307, in post_validate
    super(Task, self).post_validate(templar)
  File "/usr/lib/python3/dist-packages/ansible/playbook/base.py", line 431, in post_validate
    value = templar.template(getattr(self, name))
  File "/usr/lib/python3/dist-packages/ansible/template/__init__.py", line 844, in template
    d[k] = self.template(
  File "/usr/lib/python3/dist-packages/ansible/template/__init__.py", line 798, in template
    result = self.do_template(
  File "/usr/lib/python3/dist-packages/ansible/template/__init__.py", line 1066, in do_template
    res = j2_concat(rf)
  File "<template>", line 12, in root
  File "/usr/lib/python3/dist-packages/ansible/template/__init__.py", line 264, in wrapper
    ret = func(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/ansible/plugins/filter/core.py", line 69, in to_nice_yaml
    transformed = yaml.dump(a, Dumper=AnsibleDumper, indent=indent, allow_unicode=True, default_flow_style=False, **kw)
  File "/usr/lib/python3/dist-packages/yaml/__init__.py", line 290, in dump
    return dump_all([data], stream, Dumper=Dumper, **kwds)
  File "/usr/lib/python3/dist-packages/yaml/__init__.py", line 278, in dump_all
    dumper.represent(data)
  File "/usr/lib/python3/dist-packages/yaml/representer.py", line 27, in represent
    node = self.represent_data(data)
  File "/usr/lib/python3/dist-packages/yaml/representer.py", line 58, in represent_data
    node = self.yaml_representers[None](self, data)
  File "/usr/lib/python3/dist-packages/yaml/representer.py", line 231, in represent_undefined
    raise RepresenterError("cannot represent an object", data)
yaml.representer.RepresenterError: ('cannot represent an object', AnsibleUndefined)
fatal: [head]: FAILED! => {
    "changed": false
}

It seems to refer to line 17 of roles/rke2_common/tasks/config.yml, specifically it's looking for an rke2_config variable. A search on the repository doesn't show this defined anywhere. Am I missing something? Thanks!

I goofed by having inconsistent group names. I followed the README's group names, which has slightly outdated info, instead of just copying the example folder.
I made PR #45 to update the README's group names.


Is pretrained model available?

Hi, thank you for your great works. When I tried to use the pretrained model embedded in the top page, I found it was just scaler value. In order to load the pickle file, I used python pickle library, and loaded it as binary data. Is there any way to successfully load the pretrained model?
Thank you.

Please use torch.load to load the model file like:
trfm = TrfmSeq2seq(len(vocab), 256, len(vocab), 3)
trfm.load_state_dict(torch.load('.save/trfm_12_23000.pkl'))

Thank you. I got the pretrained-model weights.


Separate HydeServiceProvider into multiple providers

Abstract
The main HydeServiceProvider class manages a lot of things. Splitting it out to multiple providers, each with their own responsibility domain, will lead to more manageable and modifiable code.
Tasks

[x] https://github.com/hydephp/develop/pull/940
[x] https://github.com/hydephp/develop/pull/942

Related

[x] https://github.com/hydephp/develop/pull/943
[x] https://github.com/hydephp/develop/pull/944

Closing as I think it's implemented


Fixes Swift Package Manager installation

This fixes the Swift Package Manger installation issue described in https://github.com/tbaranes/IncrementableLabel/issues/17

Thanks for the contribution, looks all good!


Is it possible to use Burrow with Kafka cluster running in the KRaft mode?

Zookeeper ensemble address is a required parameter in Burrow, but KRaft mode doesn't have any zookeeper. Is there any plan to remove dependency on zookeeper?

@bai could you help please?

Afaik, zookeeper is just needed to store burrow specific data, it's doesn't need to read Kafka broker or cluster specific data.


Failed to load/initialize native-bzip2 library system-native, will use pure-Java version

Hadoop Native Libraries are installed but apparently there is a bug upstream that fails for BZIP2.
Links:

Hadoop Native Libraries: http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/NativeLibraries.html

BZip2 didn't work for them either: http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/NativeLibraries.html#Check

https://issues.apache.org/jira/browse/HADOOP-10409
http://askubuntu.com/questions/469528/install-native-bzip2-on-ubuntu-14-04

docker run --rm -i hadoop-test hadoop checknative
15/10/01 23:27:46 WARN bzip2.Bzip2Factory: Failed to load/initialize native-bzip2 library system-native, will use pure-Java version
15/10/01 23:27:46 INFO zlib.ZlibFactory: Successfully loaded & initialized native-zlib library
Native library checking:
hadoop:  true /usr/local/hadoop/lib/native/libhadoop.so.1.0.0
zlib:    true /lib/x86_64-linux-gnu/libz.so.1
snappy:  true /usr/lib/libsnappy.so.1
lz4:     true revision:99
bzip2:   false
openssl: true /usr/lib/x86_64-linux-gnu/libcrypto.so

This issue is related to issue #8 which installed the hadoop native libraries, but was not able to fix it for bzip2.


Avoid use of global variables whenever possible

In Python, we very rarely use global variables. I spotted a global variable on this line:
https://github.com/TwilioDevEd/airtng-flask/blob/master/airtng_flask/views.py#L104
Do we really need this variable to be global? Is there any reason why it can't be local?

That was suggested by PyCharm, I read a little about that and it seemed kind of straight to me. I can remove it

Done


Fix write/close deadlock

When the monitor goroutine sees that the connection has closed, it is
sometimes possible for resetChan to be full and a Write() invocation
blocked sending to resetChan. If this happens, and monitor attempts to
grab the write lock, it will result in a deadlock because Write() has
the lock, monitor is trying to acquire the lock, and nothing is draining
resetChan.
Start a goroutine in monitor to drain resetChan, just before trying to
acquire the lock.
Fixes #49
Signed-off-by: Andy Goldstein<EMAIL_ADDRESS>

LGTM


Why not replace Frame's backend with Aqua's backend?

It looks like there's a hanging PR for /api/logout
I'm just wondering, Why not just gut the frontend from Aqua, and drop it into Frame?
All the tests are passing for Aqua I presume.

Aqua started with Frame as the base. Besides the open PR they should be almost identical. There may be some really minor differences (like the baseUrl) that help support the front-end experience.

I still don't see why Aqua's backend isn't dropped into Frame...? It seems like a simple matter to me.

You mean to fix resolve the open PR?

No. I assume the tests are passing for Aqua. This means you can close that PR if you drop in the Aqua backend.
Does this makes sense?

Resolving the open PR will put them back in line. Otherwise I'm not sure I follow.

It seems to me if Frame is being touted as the backend which Aqua is built upon, it would make sense that the backend would be identical.
For me, I don't understand how a rationale can be made to not drop in the Aqua framework. Either that, or drop the Frame backend into Aqua.

The current PR isn't getting much attention at the moment. If you'd like to create a PR to align them I'd be happy to review it.

I understand that putting a PR through will patch up Frame nicely. That's not in question for me at all.
Why I don't get is why you're not answering my question:
"If Frame is being touted as the backend which Aqua is built upon, it would make sense that the backend would be identical."
Why is it not identical?

Please give me an answer to this question @jedireza.

Aqua started with Frame as the base. Besides the open PR they should be almost identical. There may be some really minor differences (like the baseUrl) that help support the front-end experience.

Since they are different code bases they need to maintained separately. When a change in Aqua should be made in Frame, it's a manual change that needs to be done. And it looks like the login patch was missed.


Question about the correct way to enforce semi orthogonality

Hey man, and thank you for this great repository, it saved me from implementing and debugging TDNN/FTDNN and instead I can focus on experiments. I have one question regarding the correct way to step the optimizer, because it seems that I have been doing it wrong until now. After reading the "usage" part of the README it seems to me that the correct way to step during the training is to do something like that: assuming that the most basic way to train network in pytorch is to:
for input, target in dataset:
    optimizer.zero_grad()
    output = model(input)
    loss = loss_fn(output, target)
    loss.backward()
    optimizer.step()

and you wrote
tdnn_f.step_semi_orth() # The key method to constrain the first two convolutions, perform after every SGD step

Than I should add this call tdnn_f.step_semi_orth() after the optimizer.step() call, making it like this, right?
for input, target in dataset:
    optimizer.zero_grad()
    output = model(input)
    loss = loss_fn(output, target)
    loss.backward()
    optimizer.step()
    tdnn_f.step_semi_orth()

Thank you for your time if you happen to answer this question!

Yes that seems right (to take an orth step at every SGD step)


Add difficulty and type of project to header?

Can we add the type of project (scratch, python etc) to the header so that it will be easy to see the type without knowing the project itself?
Also, add the difficulty-icon to the header.

I think collection.name already should be available, so it's a matter of adding images in the lesson+scratch template and styling them to fit at top right.

Fixed in #99: http://arve0.github.io/oppgaver/python/bokstaver/bokstaver.html


Improve documentation for equalTo property in firebase-query

It seems like equalTo property causes a lot of confusion among users, see
https://github.com/firebase/polymerfire/issues/169
https://github.com/firebase/polymerfire/issues/243
https://github.com/firebase/polymerfire/issues/59
And a few more. We should enhance the docs for this prop.

Closing due to inactivity.


Audit use of Font Awesome on website

Not sure how this is being used, but it's a chunk of CSS we download on every page.

One way it's being used is the much-loved icon that denotes external links. FTR I'm not suggesting we remove that, just saying it's one known use case.

We removed that "external link" icon :)

@chalin, have we removed font awesome from the new site? I can't remember.

We are using it now for GitHub icons. We'll likely be reintroducing the external link icon. We'll likely be adding a few more FA icons ("build" comes to mind -- like what we have in the www footer).

OK, thx. I'm going to close this.


Could not connect to a default or fallback server

When joining to Bungee server that redirects to the actual server without any lobby between them, causes the next error on connection: Could not connect to a default or fallback server. Incorrectly configured address/port/firewall?
I assume that it's limitation of the bungeecord, but if it is not, is there a possibility to make it work when no lobby server with portals to join the server is used, and if yes, then how because seems like I messed up somewhere.

Thanks for your request!
It is true that if you set the lobby itself to start up with BungeePteroPower, the message on kick will show the Bungeecord error message.
I would like to have the kick message tell you that the server is starting up when you log in, but it seems to be a bit difficult due to Bungeecord limitations.
I am unable to respond immediately, but will investigate when I have time!

Cool, I will make a lobby server until it will (if it ever will) be possible to startup server without triggering that in the lobby

Happy to announce that I have implemented a feature in v1.7 that you have been asking for!
You can now see a custom message when logging in if the lobby server is down.

However, this feature is disabled by default.
To enable it, please turn on useSynchronousPing in config.yml.
Due to the limitations of Bungeecord, during a ServerConnectEvent this feature blocks the thread until the ping is completed.
After the ping completes, let the thread continue and disconnect the player with a message if the lobby server is not open.

is there something other than useSynchronousPing that needs to be on as the name on the plugin and on bungeecord is the same and i still get the default message even though if i run the start command it works is there a bungeecord setting that interferes with it as i have tried turning off ping passthrough

Currently BungeeCord does not support to remove default message via plugin, so this is not possible.
We have submitted a pull request to BungeeCord to add an event for login failures, so that the default message can be removed once it is merged.
▼ Pull request to BungeeCord
https://github.com/SpigotMC/BungeeCord/pull/3638
▼ If the pull request is merged, the following modifications can be made to improve the situation.
https://github.com/Kamesuta/BungeePteroPower/pull/11


docs: add license task doc

json

Thanks for the reminder, I'll add it right away

Don't forget update docs/directory.json

Thanks for the reminder, I'll add it right away