KAFKA-7730: Limit number of active connections per listener in brokers (KIP-402)

Adds a new listener config max.connections to limit the number of active connections on each listener. The config may be prefixed with listener prefix. This limit may be dynamically reconfigured without restarting the broker.
This is one of the PRs for KIP-402 (https://cwiki.apache.org/confluence/display/KAFKA/KIP-402%3A+Improve+fairness+in+SocketServer+processors). Note that this is currently built on top of PR #6022
Committer Checklist (excluded from commit message)

[ ] Verify design and implementation
[ ] Verify test coverage and CI build status
[ ] Verify documentation (including upgrade notes)

[~gwenshap] The behaviours are intentional - not necessarily correct though :-)

If we exceed per-IP limit, we disconnect any new connections. We need to accept the connection first to detect the source IP, so this is a connect followed by a disconnect. This is the existing behaviour.

If we exceed the overall broker limit, we handle inter-broker connections differently from other connections. We never block inter-broker connections due to overall limit - i.e a runaway client on another listener cannot bring the broker down by preventing inter-broker connections from being made. We allow inter-broker connection to succeed regardless of the limit and then close the least recently used connection even though it may not have reached the idle timeout.

If we exceed overall broker limit on a non-inter-broker listener or exceed the listener limit on any listener, then we don't accept new connections until some connection is closed normally or due to hitting the idle timeout. If this is a short connection storm, then new connections are blocked to protect the broker, but as connections are closed, new connections will be accepted.  Clients will just see a short delay in connection establishment. If we have a long period of time with a large number of active connections hitting broker or listener limit, then we may block new connections for a long time. In this case clients will timeout and attempt to reconnect. We can hit this scenario if brokers are configured with lower limits than required by client applications. We have metrics that show active connections as well as the amount of time blocked, so brokers can be reconfigured if required.

Got it.

This is the "protectedListener" part, yes? It is smart and awesome.

I think I got it. We accept a connection, we see that we now exceed the broker limit and we block. Which means that we are not accepting any other connection until we have a free slot. One thing that is maybe worth documenting: The listening socket is created with a backlog of 50 (nio default), so clients will start getting errors if more of those requests accumulate. I'd probably want to monitor the backlog of the TCP socket to make sure it remains reasonable.  Or, maybe we want to expose an option for users to override this? I guess a good value depends on the connection storms you anticipate?

I'm concerned that:

  def maxConnectionsExceeded(listenerName: ListenerName): Boolean = {
    totalCount > brokerMaxConnections && !protectedListener(listenerName)
  }

Doesn't take per-listener limits into account. Doesn't it mean that if we have a limit on SSL listener but not on the broker, we'll never close excess connections on that listener? I feel like I'm missing something.

We didn't add a configuration option for server backlog queue size since it is very hard to come up with a good value. As you said it would depend on the nature of connection storms that are hard to predict.

Let's say we have two listeners, a SASL_SSL listener for external connections and an SSL listener for internal connections. We configure listener limits for both and an overall broker limit to protect the broker.
4a) If we hit the listener limit on the SSL listener, we will block the next connection until a connection is closed. We would expect the limit to be reasonably high for the SSL listener, so we would never expect to block with well-behaved brokers.
4b) If we hit the listener limit on the SASL_SSL listener, we will block the next connection until a connection is closed. We would expect the limit to be set according to the connections expected from client applications. You can monitor blocking times and increase limits if required.
4c) You can set a broker limit that is less than the sum of listener limits. The higher listener limits improve utilization of connection slots and the lower broker limit protects the broker. We allow both the SASL_SSL and SSL listeners to use up to their individual listener limit, but beyond that they are blocked until a connection on that listener is closed. If the total exceeds the broker limit without hitting the listener limits, then we allow SSL connection to succeed and close the least recently used SASL_SSL connection. We dont allow any more SASL_SSL connections until a connection is closed normally (disconnected or idle timeout).

The asymmetric behavior where we accept connections up to the listener limits and disconnect connections based on the broker limits (but done separately on each processor for each listener) makes the interactions really complex to reason about.
If you have a broker limit, an internal (protected) listener and 2 external listeners (not totally unreasonable scenario), thats a lot of combinations to consider.
You can have one listener with blocked connections (because the listener limit is exceeded), while another listener connection getting closed (because broker limit is exceeded) but still accepting new connections (because its own limit wasn't exceeded yet). It just seems difficult to reason about the possible scenarios and figure out the right limits to set and the expected behavior.
I think I'm missing something about the reasoning for the different behaviors.
Why not close connections on a listener, if that listener limit is exceeded? Why wait for the broker limit to exceed?
And the inverse, why close connections if the broker limit is exceeded? Why not have connections wait for other connections to "naturally" close, the way you currently do for listeners?

@gwenshap Agree that connection limit handling is complex and the fact that we are handling different limits in different ways makes implementation and debugging more complex. Not sure if it makes usage more complex though since the limits serve different purposes. Listener limits require the user to understand the number of applications connecting to each listener and connections per-broker for those. While broker limits should be configured based on the total number of connections and total memory reserved for those connections (e.g. SSL buffers).

Why not close connections on a listener, if that listener limit is exceeded? Why wait for the broker limit to exceed?
If we close old connections to accommodate new connections on the broker, old ones would retry and we would go into a loop of connection plus expensive authentications, overloading the broker. By preventing new connections, we would still have connection retry, but without expensive authentications.

And the inverse, why close connections if the broker limit is exceeded? Why not have connections wait for other connections to "naturally" close, the way you currently do for listeners?
We ensure that application connections cannot bring a broker down by blocking inter-broker connections by closing external connections to make space for inter-broker connections. When we complete each iteration, the total number of connections is guaranteed to be within the broker limit. If we allow new connections without closing existing ones, total number of connections can be above broker limit for unbounded periods of time. And users will need to configure broker limit to be (broker-limit-based-on-resource-allocation minus expected-inter-broker-connections).

Thanks for the patient explanation. I think it is reasonable to want both high limit to protect the broker that is aggressively enforced, and also "QoS" limits to allocate connections to listeners and allow for something like "fast lane" listeners for important apps.

LGTM. Thank you.


refactor: Updated droplet icon

Updated the droplet icon to match the icon style in general

This is fantastic, I've already updated my local copy.

@bdbch The droplet icon you have created it more looks like an egg.

why @bdbch ?


Add tests for Windows OS

Description
At the moment tests for Windows OS is commented out.
There are few issues that needs to be sorted out, mainly the installation of fonts withing GHA runner.
Code of Conduct

[X] I agree to follow the IETF's Code of Conduct

PR #1177 adds reduced tests for the Windows OS. Tests include pytests without PDF checks.
#1181 is created to address adding rest of the tests for Windows OS.


Incorrect status of a single declined recurring event

I have a weekly recurring event. I declined one of the weeks, but accepted all of the others.
When I fetch_events I'm getting that event back in the list. I updated to use the master version of icalevents, which looked like it gave me access to an Event.status attribute. But the attribute always says "CONFIRMED".
Happy to work on this myself and submit a PR if you think it's a good first issue?

Hi @WilliamMayor
Interesting case! Thank you for bringing this up.
Yes - if you can re-test with the newest published version. If the error still exists can you provide the .ics sample and I am happy to guide/help/asisst you through your contribution.
cheers

close this due to inactivity


Missing information on where to find [Disable] for isolated libraries?

I am migrating to isolated mode, and can not find [Disable] attribute anymore. The documentation does not provide enough details to know where to find this

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 3b7f85a6-7b15-afe0-6443-27512c831553
Version Independent ID: 5caea59e-779c-7b0e-e30b-497d4d1df4c7
Content: How to disable functions in Azure Functions
Content Source: articles/azure-functions/disable-function.md
Service: azure-functions
GitHub Login: @ggailey777
Microsoft Alias: glenga

@santo2
Thanks for your feedback! We will investigate and update as appropriate.


[Backport 2.x] Bumps snakeyaml to 2.0

Description
Backport of https://github.com/opensearch-project/OpenSearch/pull/6511
Issues Resolved
[List any issues this PR will resolve]
Check List

[ ] New functionality includes testing.

[ ] All tests pass

[ ] New functionality has been documented.

[ ] New functionality has javadoc added

[x] Commits are signed per the DCO using --signoff
[x] Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Codecov Report

Merging #6516 (4efa769) into 2.x (ed73582) will increase coverage by 70.41%.
The diff coverage is 100.00%.

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
@@            Coverage Diff             @@
##             2.x    #6516       +/-   ##
==========================================
+ Coverage       0   70.41%   +70.41%
- Complexity     0    59309    +59309
==========================================
  Files          0     4798     +4798
  Lines          0   284439   +284439
  Branches       0    41330    +41330
==========================================
+ Hits           0   200276   +200276
- Misses         0    67371    +67371
- Partials       0    16792    +16792

Impacted Files
Coverage Δ

...a/org/opensearch/extensions/ExtensionsManager.java
45.62% <100.00%> (ø)

...search/analysis/common/BasqueAnalyzerProvider.java
0.00% <0.00%> (ø)

...h/index/store/OpenSearchBaseDirectoryTestCase.java
75.00% <0.00%> (ø)

.../org/opensearch/gradle/test/RestIntegTestTask.java
0.00% <0.00%> (ø)

...java/org/opensearch/nio/utils/ByteBufferUtils.java
90.90% <0.00%> (ø)

...ava/org/opensearch/painless/node/EBooleanComp.java
100.00% <0.00%> (ø)

...search/common/concurrent/RefCountedReleasable.java
100.00% <0.00%> (ø)

...histogram/AutoDateHistogramAggregationBuilder.java
83.84% <0.00%> (ø)

...org/opensearch/ResourceAlreadyExistsException.java
100.00% <0.00%> (ø)

.../org/opensearch/repositories/ShardGenerations.java
81.42% <0.00%> (ø)

... and 4789 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


Potential bug with IXON handling in WSL

Environment
WSL Ubuntu 20.04
Steps to reproduce

Enter Raw Mode
Turn off IEXTEN behavior by using the IXON flag
Press Ctrl + V on

Here is a small program in Go that demonstrates the issue

package main

import (
	"bufio"
	"io"
	"os"
	"log"

	"golang.org/x/sys/unix"
)

func main() {

	termios, err := unix.IoctlGetTermios(unix.Stdin, unix.TCGETS)
	if err != nil {
		log.Fatalf("could not fetch console settings: %s", err)
	}

	termios.Lflag = termios.Lflag &^ unix.ICANON
	termios.Iflag = termios.Iflag &^ unix.IXON

	if err := unix.IoctlSetTermios(unix.Stdin, unix.TCSETSF, termios); err != nil {
		log.Fatalf("could not set console settings: %s", err)
	}

	r := bufio.NewReader(os.Stdin)

	for {

		b, err := r.ReadByte()

		if err == io.EOF {
			break
		} else if err != nil {
			log.Fatalf("Error reading from Stdin: %s\n", err)
			os.Exit(1)
		}
		log.Printf("%d\n", b)
	}
}

Expected behavior
Reading Ctrl + V key from keyboard should return 22. This is what happens in the default Ubuntu WSL terminal
Actual behavior
Returns a whole bunch of characters as keypresses

This is probably because Ctrl+V is bound to paste by default for new settings files. You can remove that binding from your settings by just deleting that line from your settings.json. Then, the ^V will get passed through to the commandline application as normal.


How to set properly "config_data_single_pair.json" samples + delay?

How do I know what delay I need to set? Is there an easy way to find out the sample numbers?
{     "train": {         "start": null,         "stop": 9124249,         "ny": 1024     },     "validation": {         "start": 9124249,         "stop": null,         "ny": null     },     "common": {         "x_path": "/Users/.../bin/train/outputs/v1_1_1.wav",         "y_path": "/Users/.../bin/train/outputs/output.wav",         "delay": 0     } }

Also what does "ny" stand for?

How do I know what delay I need to set?
Set it according to how many samples the output lags the input by. For example, look at this comparison of the input & output:

Based on my eyeballing it, the output first seems to move around sample 12029 (where I've put the vertical ruler). The input that cause this starts at sample 12000. To align them, I would set "delay": 29 in the config.
However, if you make a mistake and over-correct, then the output would happen before the input that caused it, which would be...hard to predict 😅 So, putting in a little safety factor (say, subtract a few samples) can help make sure you aren't trying to "predict the future" because you keyed in on a blip in the noise instead of a real response to the input.  In easy_colab.ipynb, See the safety_factor in nam/train/colab.py.
This is probably the "easiest" way to do it if you aren't using "easy mode".

Also what does "ny" stand for?

This is the number of samples in a single output "datum" that the model is asked to predict at once. See the definition of Dataset._ny and how it's used to get data.


FEATURE :: search bar

Codecov Report

Merging #248 into master will increase coverage by 0.01%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master     #248      +/-   ##
==========================================
+ Coverage   98.03%   98.05%   +0.01%
==========================================
  Files          53       54       +1
  Lines         814      822       +8
  Branches      159      160       +1
==========================================
+ Hits          798      806       +8
  Misses         16       16

Impacted Files
Coverage Δ

src/components/ActionToolbar.vue
100% <ø> (ø)
:arrow_up:

src/components/constants.ts
100% <ø> (ø)
:arrow_up:

src/components/SearchBar.vue
100% <100%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d2ca0a5...b7591eb. Read the comment docs.

@adimasci Thanks for your feedback, this PR is in WIP I forgot to put the label.
About the keyboard navigation yes we should have that but I'm not sure if I will do it in this PR.
I use the popover to display the search results so we have two possibilites :

Implement this feature in the popover component like that we could navigate with keyboard in all popover
don't use the popover and develop a separate dropdown (not a big work)

What do you think about that ?

This PR is replaced by this one now => https://github.com/ToucanToco/vue-query-builder/pull/250


Using type as the Field ID will result in no data being outputted for that Content Type when building the YAML files

I ran into this odd error while trying to figure out why a specific Content Type wasn't being returned anymore when running middleman contentful. The Content Type has 10 fields, and only the ID was being returned now. Very strange. I disabled each field one by one until I discovered that if it was a Short Text field that was causing this. More specifically, it was because I named the field ID type.  Once I changed the field ID to something like itemType it worked.
Why does this happen?

This happens due to an issue when there are conflicting names between the sys properties and the fields. This will be fixed with the update to the 2.0.0 contentful.rb SDK, I'll have this done within the next few days. For the moment, you can change your field ID.
Cheers


The LFS checkout option doesn't run "git lfs fetch", so if the repository has been previously checked out, then the git lfs files will never get checked out.

git lfs fetch should be required to run, since lfs use git hook, a git checkout should trigger git lfs fetch. waiting for customer provide more detail information.


How can i add samplegrabber to gamecapture to save stream to video file ?

Hi,
Great work. The problem is I don't know c++ i'm working with c#
And I don't have a clue how to make a wrapper of this gamecapture from c++ to c#
Instead maybe you can add to the gamecapture project to the c++ a samplegrabber or something like a button with file dialog so you select a directory and file name when you click ok it will start saving the streaming video to a video file ? I'm not sure how to use a samplegrabber for it and for sure not in c++
Maybe you can explain how to do it or maybe to add a button near the OK button and call it RECORD ?
Thank you very much.

Hi,
Unfortunately we cannot provide much support for this in C#. You could try using "Custom RTMP" streaming to a local RTMP server using Game Capture and grabbing data from there. Also, perhaps this project on SourceForge might help with accessing DirectShow from C#: http://directshownet.sourceforge.net http://directshownet.sourceforge.net/
Best,
Florian

On Apr 12, 2016, at 10:57 PM, Csharper1972<EMAIL_ADDRESS>wrote:
Hi,
Great work. The problem is I don't know c++ i'm working with c#
And I don't have a clue how to make a wrapper of this gamecapture from c++ to c#
Instead maybe you can add to the gamecapture project to the c++ a samplegrabber or something like a button with file dialog so you select a directory and file name when you click ok it will start saving the streaming video to a video file ? I'm not sure how to use a samplegrabber for it and for sure not in c++
Maybe you can explain how to do it or maybe to add a button near the OK button and call it RECORD ?
Thank you very much.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub https://github.com/elgatosf/gamecapture/issues/4


How to set speech language in streaming_transcribe

Hello , Could you please tell me where can I set the recognition language Japanese or korean in streaming_transcribe ,which API can set the speech language,thank you

This kind of question is better answered by the reference documentation and guides than by the samples.  A good starting point is:
https://cloud.google.com/speech-to-text/docs
The list of supported languages is found here:
https://cloud.google.com/speech-to-text/docs/languages
In the C++ examples this is configured here:
https://github.com/GoogleCloudPlatform/cpp-samples/blob/main/speech/api/parse_arguments.cc#L63
The full reference documentation for the protos is found here:
https://cloud.google.com/speech-to-text/docs/reference/rpc/google.cloud.speech.v1
And the full reference documentation for the C++ client API is found here:
https://googleapis.dev/cpp/google-cloud-speech/latest/

thank you very much , I can use the command streaming_transcribe.exe --language-code ko-KR D:\resource\KoRean \2.wav test KoRean audio file .but  have no recognizer Result return

I am not a Cloud Text-to-Speech expert.  The best I can do is offer some general troubleshooting guidance.  I tested with a public file in es-ES encoding:
gsutil cp gs://cloud-samples-data/speech/multi_es.flac .
streaming_transcribe --language-code=es-ES --bitrate=44100 multi_es.flac

That worked for me.  That leads me to believe that setting the language encoding in the examples "works", in that it is sent and interpreted correctly by the service.  In your place I would try to use an existing tool to see if it works, for example, try using the gcloud command-line tool:
https://cloud.google.com/speech-to-text/docs/transcribe-gcloud#make_an_audio_transcription_request
I think in your case you would try:
gcloud ml speech recognize --language-code ko-KR "D:\resource\KoRean \2.wav"

Note the use of quotes around a filename with spaces in it.
If using gcloud does not work, then it is a problem with the service.  I think you would need to contact support in that case.
If gcloud does work, then it is probably the examples.  With the current version of the examples you can use some environment variables to troubleshoot the messages.  On Windows that would be:
set GOOGLE_CLOUD_CPP_ENABLE_CLOG=yes
set GOOGLE_CLOUD_CPP_ENABLE_TRACING=rpc,rpc-streams

and then run the commands as before.

thank you very much ,I will do the relevant tests as you suggest

Closing for now, please reopen if you have more questions on this topic.


Path error in reproduction test when run via docker container

PASSED: docker container built correctly and launched!
PASSED: build docker container + pytest
FAILED: scripts/reproduction.ipynb

DESC:  runtime error. If I have understood correctly this is because it is trying to access ../../original_study/[CSV file]. But docker container launches notebook in reproduction/ as the root (i.e. it can't go ../../original study...).

Options:

Launch Jupyter lab at one level higher than reproduction/ or duplicate CSV files in reproduction directory.

Resolved in commit a0612be


suppress some server logs

i am interested in most logs like who logged in and what they clicked on but want to suppress these:
time="2023-02-10T04:04:06.868Z" level=info duration="108.602µs" method=GET path=index.html size=473 status=0
time="2023-02-10T04:04:26.868Z" level=info duration="99.842µs" method=GET path=index.html size=473 status=0
time="2023-02-10T04:04:26.868Z" level=info duration="76.141µs" method=GET path=index.html size=473 status=0
time="2023-02-10T04:04:45.138Z" level=info duration="100.382µs" method=GET path=index.html size=473 status=0
time="2023-02-10T04:04:45.463Z" level=info duration="106.572µs" method=GET path=index.html size=473 status=0
time="2023-02-10T04:04:46.478Z" level=info duration="100.362µs" method=GET path=index.html size=473 status=0
time="2023-02-10T04:04:46.531Z" level=info duration="100.941µs" method=GET path=index.html size=473 status=0
time="2023-02-10T04:04:46.868Z" level=info duration="107.773µs" method=GET path=index.html size=473 status=0
time="2023-02-10T04:04:47.102Z" level=info duration="97.922µs" method=GET path=index.html size=473 status=0
time="2023-02-10T04:04:47.283Z" level=info duration="103.831µs" method=GET path=index.html size=473 status=0
time="2023-02-10T04:04:53.071Z" level=info duration="212.254µs" method=GET path=index.html size=473 status=0
time="2023-02-10T04:04:53.819Z" level=info duration="94.782µs" method=GET path=index.html size=473 status=0
time="2023-02-10T04:04:55.440Z" level=info duration="339.266µs" method=GET path=index.html size=473 status=0

do u know how to stop these logs @tczhao ?


validation hostnames

I am trying to add a hostname from a non azure created domain using this code:
 webapp.Update()
                .DefineHostnameBinding()
                    .WithThirdPartyDomain(domain)
                    .WithSubDomain("www")
                    .WithDnsRecordType(CustomHostNameDnsRecordType.CName)
                    .Attach()
                .Apply();

The command executes without error and the hostname is not created. In the azure activity log I see a succesfull update. When trying to add the hostname via the blade in azure it gives me an error vith the validation.
I could not find functionallity to validate if a hostname can be added,
Next I changed the domain to a valid domain. Executed the same code and that did not add a hostname either. What is it I am doing wrong?

Is there documentation I have missed discussing how to attach a Third Party domain?

@bankzitter01 please reopen this issue in https://github.com/Azure/azure-libraries-for-net/ repo since Fluent SDK has been moved to a different place.


Use 2-space indenting

To avoid that kind of unreadable super-indentation https://github.com/Egoscio/gelato/blob/master/lib/proxy.js#L71

https://github.com/Egoscio/gelato/commit/d8156223e09eac51145a1641fe99bc5684a1b721 fixed the problem as well as anything else related to syntax inconsistency, etc...

@Egoscio as you're following standard style, you might want to do http://standardjs.com/#what-you-might-do-if-youre-clever so things are checked automatically ;) (might even add a circle.yml so checks get done when you push)


Username/password length

With the introduction of https://github.com/StackStorm/st2web/pull/887 there is a username and password limit set, but there is no equivalent check in the CLI.
Some feedback from @m4dcoder
"There's going to be some challenges there because the st2auth backends are not managed by st2 (i.e. htpasswd, LDAP, etc.). If those auth providers allow longer usernames and passwords, there's not much we can do other than throw an error that says username/passwords is beyond supported length when user login via the CLI."

htpasswd appears to support up to 255 characters for the username
These two locations hint at username < MAX_STRING_LEN - 1
https://github.com/apache/httpd/blob/21f16155c38e406e0a0daaa60a539d66128cf044/support/htpasswd.c#L91
https://github.com/apache/httpd/blob/21f16155c38e406e0a0daaa60a539d66128cf044/support/htpasswd.c#L239
MAX_STRING_LENGTH is set to 256
https://github.com/apache/httpd/blob/21f16155c38e406e0a0daaa60a539d66128cf044/support/passwd_common.h#L35
Linux systems appear to agree with this character limit (although not directly related to St2 Authentication)
256 is the limit for ubuntu, debian and centos
getconf LOGIN_NAME_MAX
256

I saw in some forums that AD has shorted constraints but St2 should allow the largest constraint IMO:

Active Directory has constraints
Cn (Common-Name): 64
There is no attribute like 'fullname' the closest one is 'display-Name' and has a limit on 256


[DOCS] Fix documentation typo

What is this PR for?
Change mistyped name from Heliun to Helium
What type of PR is it?
Documentation
Questions:

Does the licenses files need update? No
Is there breaking changes for older versions? No
Does this needs documentation? No

LGTM


Create window in the sRGB colorspace

It is often useful to create windows that use the sRGB colorspace natively. This is vital for maintaining a linear colorspace through to the screen. The WGL/GLX_framebuffer_sRGB (http://www.opengl.org/registry/specs/ARB/framebuffer_sRGB.txt) explains how to do this.
ContextSetting should have a boolean field for this.

Is this really an important feature? Is the difference significant?
I have the feeling that you're the only one to care/know about this kind of feature here.

If you need to ask about whether gamma-correction, the sRGB colorspace, and maintaining a linear color pipeline is important, then I humbly suggest that you need to learn more about graphics. Gamma correction and gamma correct texturing is not something that should be ignored, and tools should not make it easy to ignore them. Indeed, good tools should encourage you to use these important features.
Here are some links for you to educate yourself on how important this is for actually drawing what you intend to draw:
http://http.developer.nvidia.com/GPUGems3/gpugems3_ch24.html
http://maddieman.wordpress.com/2009/06/23/gamma-correction-and-linear-colour-space-simplified/
http://www.arcsynthesis.org/gltut/Illumination/Tut12 Monitors and Gamma.html
http://www.arcsynthesis.org/gltut/Texturing/Tutorial 16.html
The last two are part of my graphics tutorial book.

This is only supported on newer hardware. Also you can do this manually by writing in the end of your fragment shader:
void main()
{
        /* Code */
        gl_FragColor = pow( finalColor, 2.2 );
}

Let's ignore the fact that your suggestion would mean that any blending happens in sRGB space instead of linear space (and therefore the blending would be wrong), as well as the fact that your code would also raise the alpha to a power. Let's just look at the simple practicalities of the suggestion.
How much does pow(finalColor, 2.2); cost in performance? I don't know off the top of my head, but I do know that sRGB conversion is free. So I'm guessing the power function is more expensive. Why waste performance?
We're not talking about something that's difficult to implement here; this isn't a massive implementation burden. We're talking about adding a parameter to ContextSettings and adding another pixel format setting internally when creating a context. That's all. It might take 15 minutes, 45 on the outside.

Dude if you're taking a little example I gave you explain how to achieve it and dissect it like if it were supposed to be a 100% accurate then it's going to be impossible to discuss with you.
No this is not difficult to implement. And that is not the issue at hand. The issue is not all hardware supports it. And the OpenGL version SFML will mainly run on does not support it. There will be more problems in using this and having it implemented than you expect.

It is an important issue, you get a point (and thanks for the links).
But you'll still be the only one to care about it. Most people using SFML are beginners, or write 2D games. There are very few (if not none) who develop a 3D engine with complex lighting/HDRL/... using SFML to create their OpenGL contexts.
And I don't want to add a feature for a single user, even if it takes me 5 minutes to implement. There are so many important things to do before.

Actually, two people care about it. It would be nice if you could provide more access to things like the context settings so people who know what they're doing can implement these things themselves rather than asking you to change SFML.

Wouldn't it be better to just add this to #166, either way?

I'm not very knowledgeable on this, but wouldn't it be better to just allow users to manage graphics contexts on their own? That way, they could create an sRGB context and pass it to sf::Window, and then, you cover all cases.

Context creation is complex, highly platform specific, and cannot easily be done outside SFML.

So is window creation, but you can pass custom window handlers to SFML, too.

So is window creation, but you can pass custom window handlers to SFML, too

The purpose is to allow integrating SFML into other GUI libraries. Not to create your own window from scratch with OS-specific code.

To be fair RetroX, any user who wanted to write and maintain their own context creation code (which also means writing and maintaining Window creation), probably doesn't want to use SFML.
Indeed, the idea with this was for me to basically use SFML as a means to do sound, input, and manage window/GL context creation, while completely ignoring all of SFML's actual rendering routines. I do that with Allegro 5 now, which is an API that I like. But I was hoping to be able to trade it in for a nicer, more C++-ish API. But sRGB support isn't optional for me, so I guess I'll skip SFML and see if I can get the Allegro developers to add sRGB support.
Oh well.

Eh, good point. :/

Just out of interest, since I might pick this up at some point: Does OS X support sRGB framebuffers? I know OS X supports the OpenGL extension, but the pixel format documentation doesn't provide any hints as to how an sRGB framebuffer would be created.
@mantognini do you have any ideas?

I haven't found a clear answer but I think it is supported.
However, using OpenGL Extensions Viewer, with a Compatibility profile on a NVIDIA or Intel card, I could find those extensions (some might be aliased):
https://www.opengl.org/registry/specs/ARB/framebuffer_sRGB.txt
https://www.opengl.org/registry/specs/EXT/framebuffer_sRGB.txt
https://www.opengl.org/registry/specs/EXT/texture_sRGB.txt
https://www.opengl.org/registry/specs/EXT/texture_sRGB_decode.txt
Does it help you in some way?

Like I said... I know those extensions are supported. What I want to know is whether I can select a pixel format that provides me with an sRGB framebuffer through the OS X API. If you look in the specification for ARB_framebuffer_sRGB, you will find the new tokens GLX_FRAMEBUFFER_SRGB_CAPABLE_ARB and WGL_FRAMEBUFFER_SRGB_CAPABLE_ARB. These can be used when specifying a pixel format using GLX (Unix) and Wgl (Windows). There are no tokens for OS X, because... well... OS X context management isn't described in any ARB specifications.
OS X describes its pixel formats through an NSOpenGLPixelFormat object, and its API is maintained by Apple. I've (briefly) read through the documentation and they don't mention sRGB support anywhere.
It could very well be that even in 2015, Apple systems still don't support sRGB OpenGL framebuffers, but if you ask me it is hard to believe. Maybe they do it automatically? I've read that in other areas (not related to OpenGL), Apple has given some thought to supporting sRGB in some way so it seems strange they didn't do it here as well. I was hoping you had come across some OS X OpenGL documentation I have not yet seen regarding this.

Unfortunately no, I've never seen sRGB in the document I've read.
But maybe other libs such as sdl or glfw do implement that...

It's always nice when you learn something new. :grin:
I still find it strange though, that I couldn't find any official documentation/information from Apple.

Why am I not even surprised that it was not documented? :unamused:

Any progress on this issue? sRGB is also holding me back from using SFML.

I'll probably look into this and the other OpenGL related stuff after gl_dev gets merged. Shouldn't be too long.

Any news? Would be a nice feature to have.

Work is scheduled to begin after pending pull requests affecting the same files have been merged.

Merged in e00d160224fbdf519aa9e420e57cc8e29e244f31


add: a new method String::reverse()

I have added a reverse() method to String.
I think the reverse() method has some good points. First, it makes the String class more useful and easier to work with. It makes reversing a string simple, which is needed often in programming.
I have tested it in my computer. I can run correctly.
And you can using it like this:
fn main {
  let x: String = "what is a dog";
  println(x.reverse())
}

Pull Request Test Coverage Report for Build 2770
Details

3 of 3 (100.0%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.009%) to 85.469%

Totals

Change from base Build 2768:
0.009%

Covered Lines:
3741

Relevant Lines:
4377

💛  - Coveralls

LGTM. Could you rename String::reverse() to String::rev(), to make it look consistent with other collections? (Array::rev(), List::rev())

I did it just now.


Is it possible to support subscription for flask

Hi there,
I am building an app with Flaks framework and migrating to fastAPI seems not feasible at the moment. Would it be possible to add subscription feature to flask here?
I digged a little into this myself but failed to enable asgi/websockets at the same time.
Would love to learn more about if it is possible to do this for flask.

Hey @zhouyuf6741, are you still interested in this?
From the top of my head, two solutions come into my mind.

One obviously would be to implement the graphql subscription protocols for Flask. That could be tricky though, since Flask apparently has no official websocket support (correct me if I'm wrong).

Maybe you could run Flask as an ASGI app and the strawberry-graphql ASGI app in parallel and delegate requests based on the used protocol (http vs websocket) to either of the ASGI apps.

Hi @DoctorJohn, thank you for the suggestions! We ended up deployed another service to delegate subscription requests.


chore(context): add properties for d.ts generation

this will add the properties to the typescript defintions

@bryanrsmith Can you get this PR in as part of the beta docs/api finalization?

You got it. Thanks @JeroenVinke!


Error while calling lua chunk: module 'conf' not found:could not convert modname to path

I did the following:
git clone --depth 1 https://github.com/shaunsingh/nyoom.nvim.git && cd nyoom.nvim
nix develop
nvim

Which gave me this error when opening nvim:
Error detected while processing /home/hosk/nyoom.nvim/init.lua:
E5113: Error while calling lua chunk: /home/hosk/nyoom.nvim/init.lua:43: module 'conf' not found:could not convert modname to path
        no field package.preload['conf']
        no file './share/lua/5.1/conf.lua'
        no file './conf.lua'
        no file './conf/init.lua'
        no file './lib/lua/5.1/conf.so'
        no file './conf.so'
        no file './lib/lua/5.1/loadall.so'
stack traceback:
        [C]: in function 'require'
        /home/hosk/nyoom.nvim/init.lua:43: in main chunk

It seems like when this repo is cloned anywhere but ~/.config/nvim (or platform equivalent) this error occurs (since neovim still looks there for modules to load)

Could you try switching fennel_compiler to aniseed in init.lua? In that case, it should compile all the files to a lua folder which I imagine would work with nix. Sorry, I haven't tested nix support in a while


cdc  add multiple duplicated index when add an index without specifying a name in upstream

Bug Report
Please answer these questions before submitting your issue. Thanks!

What did you do? If possible, provide a recipe for reproducing the error.
the workflow is TiDB -> TiCDC -> TiDB
add an index without specifying a name in upstream

What did you expect to see?
one index added in downstream

What did you see instead?
more than one indexes added in downstream

Versions of the cluster

Upstream TiDB cluster version (execute SELECT tidb_version(); in a MySQL client):
v5.1.1

TiCDC version (execute cdc version):
v5.1.1

the root cause is: cdc try to do ddl job if timeout will keep retry. but downstream tidb will not cancel pervious ddl job even after receive next one.
have provide workaround:

add index with a name
increase the timeout in cdc retry.

Change to severity/moderate, as it has workarounds.

/assign @asddongmen

alter table exchange partition has the same problem.

More context:
When adding index on table with a name can prevent add dup index to the same table.
my:root@<IP_ADDRESS>:4000=> create index idx_name on sample_table (name);
CREATE INDEX
my:root@<IP_ADDRESS>:4000=>show index from sample_table;
    Table             | Non_unique | Key_name | Seq_in_index | Column_name |
---------------------+--------------------+----------------+--------------------+-----------------------+
 sample_table |                     0 | PRIMARY   |            1           | id                        |
 sample_table |                     1 | idx_name  |            1           | name                 |
my:root@<IP_ADDRESS>:4000=> create index idx_name on sample_table (name);
error: mysql: 1061: index already exist idx_name

adding index without name will lead to dup index to be added to the same table
my:root@<IP_ADDRESS>:4000=> alter table sample_table add index(name);

ALTER TABLE
my:root@<IP_ADDRESS>:4000=> alter table sample_table add index(name);

ALTER TABLE

my:root@<IP_ADDRESS>:4000=> show index from sample_table;
    Table             | Non_unique | Key_name | Seq_in_index | Column_name |
---------------------+--------------------+----------------+--------------------+-----------------------+
 sample_table |                     0 | PRIMARY   |            1           | id                        |
 sample_table |                     1 | idx_name  |            1           | name                 |
 sample_table |                     1 | name         |            1           | name                 |
 sample_table |                     1 | name_2     |            1           | name                 |

/assign

PR #9034 mitigate this issue by disabling timeout when syncing physical ddls.


I'm having No module named 'cortex_DIM' error

hello!
thanks for your great repo.
I followed all instructions and made a new conda environment specific to this code.
I'm not sure why but it seems to fail on loading modules
I'm working on ./DIM directory and wrote command python scripts/main.py --help
thanks for your support in advance

sorry, additional note!
tried pip install . on ./Dim directory
such error shows up
ERROR: Could not find a version that satisfies the requirement cortex==0.13a0 (from cortex-DIM==0.11) (from versions: 0.1a0, 0.11)
ERROR: No matching distribution found for cortex==0.13a0 (from cortex-DIM==0.11)

after installing I get this error
KeyError: 'torchvisionnot found in .cortex.yml data_paths'
.. and now I'm really stuck

sorry, I solved above problem with pip install . on ./DIM directory
I also had cortex version mismatch so I chagned required cortex version in setup.py to 0.11, will it be fine?
and now when I try to run the code
cortex GAN --d.source CIFAR10 --d.copy_to_local
I get
KeyError: 'torchvision not found in .cortex.yml data_paths'
any suggestions?

hi，how did you solve this problem？
(pytorch_2.2.1) root@wxy1:~/wangxy/DIM# python scripts/main.py --help
/root/anaconda3/envs/pytorch_2.2.1/lib/python3.8/site-packages/torch/nn/_reduction.py:42: UserWarning: size_average and reduce args will be deprecated, please use reduction='none' instead.
warnings.warn(warning.format(ret))
Traceback (most recent call last):
File "scripts/main.py", line 12, in
from cortex_DIM.evaluation_models.classification_eval import ClassificationEval
ModuleNotFoundError: No module named 'cortex_DIM'


Getting status Unauthorized on login attempt.

I was  logged in and use this lib.
Then i call client.destroy instead of client.dispose by accident.
Now each time i try to authorize, i receive:
 {"@type":"error","code":401,"message":"Unauthorized","@extra":"some extra"}
For example:
client.set_authentication_phone_number(phone_number: '+tel number', settings: nil).wait
-> getting code
client.check_phone_number_verification_code(code:  'received code').wait
-> {"@type":"error","code":401,"message":"Unauthorized","@extra":"some extra"}

Or:
client.request_qr_code_authentication(other_user_ids: nil).wait
-> getting link
client.confirm_qr_code_authentication(link: "tg://login?token=received link")
-> {"@type":"error","code":401,"message":"Unauthorized","@extra":"some extra"}

Does anyone have any clue?

Hello.
Try to use client.check_authentication_code(code: 'your code') instead of check_phone_number_verification_code.
Write me if it helps, please

Well, now after attempt to get a code i receive this:
client.set_authentication_phone_number(phone_number: '+ phone', settings: nil).wait

->  #<TD::Types::Error code=406 message="UPDATE_APP_TO_LOGIN">
 {"@type":"updateServiceNotification","type":"API_64BIT_LOGIN_APP_OUTDATED_262","content":{"@type":"messageText","text":{"@type":"formattedText","text":"You are using an outdated app that is no longer supported. To access your messages, please update your app to the latest version. If you don\u2019t know how to update your app, go to https://telegram.org/dl","entities":[{"@type":"textEntity","offset":91,"length":37,"type":{"@type":"textEntityTypeBold"}},{"@type":"textEntity","offset":178,"length":23,"type":{"@type":"textEntityTypeUrl"}}]}}}


hello.fm fails to compile

I started the tutorial, ran npm i -g formality-lang (got formality-lang@0.3.98), wrote
import Base@0 open

main : Output
  print("Hello, world!")

into hello.fm, and ran fm hello/main.  I received the following output:
Downloading files to `fm_modules`. This may take a while...
[PARSE-ERROR]
[PARSE-ERROR]
[PARSE-ERROR]
ReferenceError: TextEncoder is not defined

I noticed the problem on line 12, col 25:
   5|
   6| import Data.List@0 open
   7| import Data.Maybe@0 open
   8| import Data.Maybe.Control@0 open
   9| import Data.Nat@0 open
  10| import Data.Or@0 open
  11| import Data.String@0 open
  12| import Data.Unit@0 open                      --red line
  13| import Data.Vector2D@0 open
  14|
But it could have happened a little earlier.
If this is hard, consider relaxing. You deserve it!

I noticed the problem on line 4, col 0:
   1| import Algebra@0 as Algebra                      --red line
   2| import Control@0 as Control
   3| import Data@0 as Data
   4| import Induction@0 as Induction                      --red line
   5| import Logic@0 as Logic
   6|
But it could have happened a little earlier.
It takes me some time to process things. Have patience with me!

I noticed the problem on line 1, col 18:
   1| import Base@0 open                      --red line
   2|
   3| main : Output
   4|   print("Hello, world!")
But it could have happened a little earlier.
It takes me some time to process things. Have patience with me!

(--red line added by me to indicate red lines)
Is there a library that wasn't packaged right, or something?
I'm on Windows 10.
(By the way, the randomized flavor text at the bottom of each error chunk was funny for a second or two, but almost immediately became annoying.  It distracts from the actual data.  If you want to keep it, I'd recommend at least restricting it to once per run, and putting it somewhere out of the thick of things, like right after the error is named.)

Tried it on Ubuntu, on a different computer.  Same result.

TextEncoder should come from Node.
If i remember correctly, i had a similar error with node 10 (on nixos though).
But with Node 12 everything worked.

@Erhannis let me know if the problem still happens after updating Node as @LoPoHa said

That fixed it.  Perhaps you could make a note on the installation page, about minimum Node version?

@Erhannis i've made an explicit note of the Node 12 dependency in the README in c1a54ba, will add to the docs repo as well.
@LoPoHa I made the Nix build a little simpler in 67023ff, using just node2nix, but since Node 12 isn't in stable nixpkgs yet you still need to build from unstable. I have a quick and dirty hack for this right now, but there ought to be a way to just tell node2nix to generate the expressions directly


Unexpected behavior in cross-schema_prefix foreign key references

Environment

Elixir version (elixir -v):
Erlang/OTP 22 [erts-10.4.4] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] [hipe]
Elixir 1.9.1 (compiled with Erlang/OTP 21)

Database and version (PostgreSQL 9.4, MongoDB 3.2, etc.):
psql (PostgreSQL) 11.5 (Ubuntu 11.5-0ubuntu<IP_ADDRESS>)

Ecto version (mix deps): {:ecto_sql, "~> 3.2"}

Database adapter and version (mix deps): {:postgrex, ">= 0.0.0"} (v. 0.15.0 per mix.lock)

Operating system: Linux (Ubuntu 19.04)

Current behavior
For your convenience (?) I've created a small project reproducing the error here: https://github.com/davidsulc/ecto_demo
The issue is documented in that project's README, which features reproduction steps & code. The gist is that in certain circumstances, Ecto queries the table in the wrong "schema prefix" both when fetching related records (e.g. belongs_to) and when handling foreign keys.
Reproducing the issue:

create an Ecto schema with no schema_prefix (let's call it Employee for consistency with the linked project)
create an Ecto schema with the same name (not sure if this is relevant to the problem) in some schema_prefix (GdprEmployee)
have GdprEmployee refer to Employee

Now the foreign key constraint present in GdprEmployee will get checked against the GdprEmployee table instead of the Employee table. In additon, querying a GdprEmployee record and attempting to load its related Employee record will fail.
Expected behavior
I would expect the cross-schema_prefix queries to be handled transparently, or Ecto warning me I'm doing something wrong/unsupported.

@josevalim Thank you so much for looking into this, especially since it turned out to be a user error...

We could force you to specify a prefix for each reference whenever the parent table has a prefix, that would at least solve the ambiguity.

Suggested documentation change: https://github.com/elixir-ecto/ecto_sql/pull/146


added cycle for recurring treasury proposal

When users make a proposal for funding by the treasury:
⦁	they must indicate what cycle they are proposing the budget for (the current one or any cycle in the future).
⦁	they must indicate the duration of their proposal - segments (one cycle or more, these cycles must be contiguous).
⦁	Users will only be able to vote on proposals proposing in the present cycle.

@zycon91 @VinD22 who is going to review this? cc @DesmondSanctity

@VinD22 can help review the pull request.

Okay will effect changes now


S2 Playoffs Error Found

In the Red Reserve vs eUnited London Docks HP that took place during Losers Round 3 of S2 Playoffs, the total kills of the lobby add up to 255, and the total deaths add up to 220. Something is off with that map.

Here's the VOD link with the end of game scoreboard https://youtu.be/n7VoPvxRLb4?t=58m40s
It looks like 5 or 6 players had the wrong kill total so go ahead and fix the raw file. (I don't know if I can do that myself but I'm unfamiliar with GitHub). I just went thru and double checked this is the only map with this issue.

Good find!
For CWL Birmingham I just uploaded the CSV to my GitHub after I made the revisions, but I'm not sure if that made it to the official CSV. I still don't completely know how GitHub works in full either 😃.
Justin will likely see this in the coming days and give a better response.


Add missing requirements

These are the things I had to install in order to successfully run the app in a virtualenv (aka empty Python environment).
Partial output of my pip freeze:
pyOpenSSL==0.14
simplejson==3.6.4

For what it's worth.

The issue with pyOpenSSL 0.14 has been resolved, and the requirements.txt was updated in another commit.

Thanks for commenting!


Added README in bengali

Dear Sir,
I have added readme translated in bengali. Please review the changes and merge my pull request.
Thanking you,
Shruti Sen

@Harold-lkk Please review my changes.


Fix Lsp Symbol Rename Bug

closes #1042

Ok cool, make sense. I'll omit the lock file in future PR's unless it is necessary for the PR to work.
@leviathanbeak not sure why this function was once needed?

@JoshuaBatty  I don't remember exactly for renaming, but I know for example when hovering over symbols there was a "off by 1" bug, where you would hover over the last character and you would not get anything so I added +1.
So I probably added +1 to the rename range when I shouldn't, if this fix passes all the possible scenarios:

requesting "rename" while hovering over the last character
requesting "rename" while hovering over the next whitespace after the last character (and not getting anything back)

then I guess it's fine :)

Is this good? Seems like everybody is happy with it and it has been 7 days since last activity. cc @JoshuaBatty @adlerjohn

Is this good to go? Seems like everybody is happy with it and it has been 7 days since last activity. cc @JoshuaBatty @adlerjohn @mitchmindtree

Sorry, have been sick so haven't had a chance to double-check what Elvis recommended, going to do that now and finish this PR off.

This should be good to go now, have double-checked it passes the scenarios that Elvis brought up.


How to import a static library as an IDF component (IDFGH-13362)

Answers checklist.

[X] I have read the documentation ESP-IDF Programming Guide and the issue is not addressed there.
[X] I have updated my IDF branch (master or release) to the latest version and checked that the issue is present there.
[X] I have searched the issue tracker for a similar issue and not found a similar issue.

IDF version.
v5.2.2
Operating System used.
Windows
How did you build your project?
Command line with CMake
If you are using Windows, please specify command line type.
PowerShell
What is the expected behavior?
see more information
What is the actual behavior?
see more information
Steps to reproduce.
see more information
Build or installation Logs.
No response
More Information.
I have read the relevant documentation, including the examples in "esp-idf\examples\build_system\cmake", but there are no examples of importing a static library as an IDF component. I referred to the "import_lib" example to try to add an IDF component, and below is my build project.
The directory of my project is as follows
my_project/
├── components/
│   ├── mylibA/
│   │   ├── CMakeLists.txt
│   │   ├── include/
│   │   │   └── myliba.h
│   │   └── lib/
│   │       └── libmyliba.a
│   └── mylibB/
│       ├── CMakeLists.txt
│       ├── include/
│       │   └── mylibb.h
│       └── lib/
│           └── libmylibb.a
├── CMakeLists.txt
└── ...

In components/mylibA/CMakeLists.txt, the code is as follows:
idf_component_register()
set(lib_path            "${CMAKE_CURRENT_LIST_DIR}/lib/libmyliba.a")
set(include_path        "${CMAKE_CURRENT_LIST_DIR}/include")
set(private_requires    mylibB)
add_prebuilt_library(lib_${COMPONENT_NAME} ${lib_path}
                     PRIV_REQUIRES ${private_requires}
                    )
set_target_properties(lib_${COMPONENT_NAME} PROPERTIES INTERFACE_INCLUDE_DIRECTORIES ${include_path})

target_link_libraries(${COMPONENT_LIB} INTERFACE lib_${COMPONENT_NAME})
set_target_properties(${COMPONENT_LIB} PROPERTIES IMPORTED_LINK_OPTIONS "-Wl,--whole-archive ${lib_path} -Wl,--no-whole-archive")

In components/mylibB/CMakeLists.txt, the code is as follows:
idf_component_register()
set(lib_path            "${CMAKE_CURRENT_LIST_DIR}/lib/libmylibb.a")
set(include_path        "${CMAKE_CURRENT_LIST_DIR}/include")

add_prebuilt_library(lib_${COMPONENT_NAME} ${lib_path})
set_target_properties(lib_${COMPONENT_NAME} PROPERTIES INTERFACE_INCLUDE_DIRECTORIES ${include_path})

target_link_libraries(${COMPONENT_LIB} INTERFACE lib_${COMPONENT_NAME})
set_target_properties(${COMPONENT_LIB} PROPERTIES IMPORTED_LINK_OPTIONS "-Wl,--whole-archive ${lib_path} -Wl,--no-whole-archive")

Is this way of writing CMake correct? Currently, testing this method shows that the unused symbols in the two static libraries do not appear in the project's map file, as if the WHOLE_ARCHIVE parameter has not taken effect.

Hi @TxRxTxRx,
We will take a look at it shortly.

Hi @TxRxTxRx, We will take a look at it shortly.

Thank you very much for your reply. I will supplement the relevant information.
In my project,  "myliba" depends on "mylibb".
If there is only the mylibb component, the code in components/mylibB/CMakeLists.txt is modified as follows:
set(lib_path            "${CMAKE_CURRENT_LIST_DIR}/lib/libmylibb.a")
set(include_path        "${CMAKE_CURRENT_LIST_DIR}/include")
idf_component_register()

add_prebuilt_library(lib_${COMPONENT_NAME} ${lib_path})

target_include_directories(lib_${COMPONENT_NAME} INTERFACE ${include_path})
target_include_directories(${COMPONENT_LIB} INTERFACE ${include_path})

target_link_libraries(${COMPONENT_LIB}
    INTERFACE
    -Wl,--whole-archive lib_${COMPONENT_NAME} -Wl,--no-whole-archive)

The WHOLE_ARCHIVE attribute is effective, and the weak symbols in the mylibb component can be overridden by strong symbols.
However, when I write the CMake for myliba in the same way, WHOLE_ARCHIVE does not work, and the weak symbols in both myliba and mylibb are not correctly overridden. The code in components/mylibA/CMakeLists.txt is as follows:
set(lib_path            "${CMAKE_CURRENT_LIST_DIR}/lib/libmyliba.a")
set(include_path        "${CMAKE_CURRENT_LIST_DIR}/include")
set(private_requires    mylibB)

idf_component_register()

add_prebuilt_library(lib_${COMPONENT_NAME} ${lib_path}
                     PRIV_REQUIRES ${private_requires}
                    )

target_include_directories(lib_${COMPONENT_NAME} INTERFACE ${include_path})
target_include_directories(${COMPONENT_LIB} INTERFACE ${include_path})

target_link_libraries(${COMPONENT_LIB}
    INTERFACE
    -Wl,--whole-archive lib_${COMPONENT_NAME} -Wl,--no-whole-archive)

I would like to know the correct way to do this. After many tests, I still can't correctly add WHOLE_ARCHIVE for multiple static libraries with dependencies.

Hi @TxRxTxRx,
Had your project not included this dependency between libA and libB, I would have suggested to refer to https://github.com/espressif/esp-idf/tree/master/examples/build_system/cmake/import_prebuilt as the basic example on how to import prebuilt static libraries in the ESP-IDF build system.
Now, since you have a dependency between your static libs, it is probably not as straightforward as if those library where independent.
Could you maybe provide the CMakeLists of the main and base directory of your project if you are able to do so?

@SoucheSouche
Of course, here is the directory structure of my project:
my_project/
├── components/
│ ├── demo/
│ │ ├── CMakeLists.txt
│ │ ├── include/
│ │ ├── dummy.c                       // This is a dummy C file
│ │ └── libdemo.a
│ └── platform/
│   ├── CMakeLists.txt
│   ├── include/
│   ├── dummy.c                       // This is a dummy C file
│   └── libplatform.a
├── main
│ ├── main.c
│ └── CMakeLists.txt
└── CMakeLists.txt

The code in my_project/components/demo/CMakeLists.txt is:
# Set the static library file path
set(lib_path            "${CMAKE_CURRENT_LIST_DIR}/libdemo.a" )

# Set the header file path
set(include_path        "${CMAKE_CURRENT_LIST_DIR}/include" )

# Set public dependencies
set(public_requires     platform)

# Set private dependencies
set(private_requires    log )

idf_component_register(SRCS dummy.c)

add_prebuilt_library(lib_${COMPONENT_NAME} ${lib_path}
                     REQUIRES ${public_requires}
                     PRIV_REQUIRES ${private_requires}
                    )

target_include_directories(lib_${COMPONENT_NAME} INTERFACE ${include_path})

# target_link_options(lib_${COMPONENT_NAME} INTERFACE -Wl,--whole-archive ${lib_path} -Wl,--no-whole-archive)

# Link the imported library to the current component.
target_link_libraries(${COMPONENT_LIB} INTERFACE -Wl,--whole-archive lib_${COMPONENT_NAME} -Wl,--no-whole-archive )

The code in my_project/components/platform/CMakeLists.txt is:
# Set the static library file path
set(lib_path            "${CMAKE_CURRENT_LIST_DIR}/libplatform.a" )

# Set the header file path
set(include_path        "${CMAKE_CURRENT_LIST_DIR}/include"
                        "${IDF_PATH}/components/driver/gptimer/"
                        "${IDF_PATH}/components/esp_netif/include"
                        "${IDF_PATH}/components/esp_event/include"
                        "${IDF_PATH}/examples/system/console/advanced/components/cmd_nvs"
                        "${IDF_PATH}/examples/system/console/advanced/components/cmd_system"
                        "${IDF_PATH}/examples/system/console/advanced/components/cmd_wifi" )

# Set public dependencies
set(public_requires    console driver esp_adc nvs_flash unity vfs fatfs cmd_nvs cmd_system )

# Set private dependencies
set(private_requires    app_update espressif__esp_encrypted_img espressif__esp-dsp )

idf_component_register(SRCS dummy.c)

add_prebuilt_library(lib_${COMPONENT_NAME} ${lib_path}
                     REQUIRES ${public_requires}
                     PRIV_REQUIRES ${private_requires}
                    )

target_include_directories(lib_${COMPONENT_NAME} INTERFACE ${include_path})

# target_link_options(lib_${COMPONENT_NAME} INTERFACE -Wl,--whole-archive ${lib_path} -Wl,--no-whole-archive)

# Link the imported library to the current component.
target_link_libraries(${COMPONENT_LIB} INTERFACE -Wl,--whole-archive lib_${COMPONENT_NAME} -Wl,--no-whole-archive )

The code in my_project/main/CMakeLists.txt is:
idf_component_register(SRCS "main.c"
                    INCLUDE_DIRS "."
                    )

The code in my_project/CMakeLists.txt is:
cmake_minimum_required(VERSION 3.16)

include($ENV{IDF_PATH}/tools/cmake/project.cmake)

project(main)

I look forward to your reply.

@TxRxTxRx, thanks a lot for sharing. I will take a look at it now.

@fhrbata Thank you for your suggestion
Using this method, it seems that the main component cannot find the header files of the mylibA and mylibB components. If we add
set_target_properties(${COMPONENT_NAME} PROPERTIES INTERFACE_INCLUDE_DIRECTORIES ${include_path})

the REQUIRES and PRIV_REQUIRES in add_prebuilt_library will become invalid, and the compiler will show undefined reference errors.

Hello @TxRxTxRx ,

Using this method, it seems that the main component cannot find the header files of the mylibA and mylibB components.

Yeah, the set_target_properties seems to just overwrite everything in INTERFACE_INCLUDE_DIRECTORIES, including what was added e.g. by target_link_libraries, so the info about  directory includes is lost. This can be seen by printing the INTERFACE_LINK_LIBRARIES property before and after calling set_target_properties.
BEFORE set_target_properties: __idf_cxx;__idf_newlib;__idf_freertos;__idf_esp_hw_support;__idf_heap;__idf_log;__idf_soc;__idf_hal;__idf_esp_rom;__idf_esp_common;__idf_esp_system;__idf_xtensa;__idf_cxx;__idf_newlib;__idf_freertos;__idf_esp_hw_support;__idf_heap;__idf_log;__idf_soc;__idf_hal;__idf_esp_rom;__idf_esp_common;__idf_esp_system;__idf_xtensa;lib_mylibA
AFTER set_target_properties: -Wl,--whole-archive /home/fhrbata/work/esp-idf/examples/get-started/hello_world/components/mylibA/lib/libmyliba.a -Wl,--no-whole-archive

I thought that appending to INTERFACE_LINK_LIBRARIES should solve this.
set_property(TARGET ${COMPONENT_LIB} APPEND PROPERTY INTERFACE_LINK_LIBRARIES "-Wl,--whole-archive ${lib_path} -Wl,--no-whole-archive")

but there is another problem with the
target_link_libraries(${COMPONENT_LIB} INTERFACE lib_${COMPONENT_NAME})

because cmake will add the library into the link twice, first without --whole-archive as a result of the  target_link_libraries  and second time with it as result of setting  INTERFACE_LINK_LIBRARIES property , leading to  multiple definition of function within the archive. I haven't been able to find out how to tell cmake to link it just once with --whole-archive while using target_link_libraries.
Appending the prebuild lib directly into INTERFACE_LINK_LIBRARIES and adding its interface to the component seems to be working.  Here is what I ended up with
components/mylibA/CMakeLists.txt
set(include_path        "${CMAKE_CURRENT_LIST_DIR}/include")
set(lib_path            "${CMAKE_CURRENT_LIST_DIR}/lib/libmyliba.a")
idf_component_register(
    INCLUDE_DIRS ${include_path}
    SRCS dummy.c
    REQUIRES mylibB
)
set_property(TARGET ${COMPONENT_LIB} APPEND PROPERTY INTERFACE_LINK_LIBRARIES "-Wl,--whole-archive ${lib_path} -Wl,--no-whole-archive")

components/mylibB/CMakeLists.txt
set(include_path        "${CMAKE_CURRENT_LIST_DIR}/include")
set(lib_path            "${CMAKE_CURRENT_LIST_DIR}/lib/libmylibb.a")
idf_component_register(
    INCLUDE_DIRS ${include_path}
    SRCS dummy.c
)
set_property(TARGET ${COMPONENT_LIB} APPEND PROPERTY INTERFACE_LINK_LIBRARIES "-Wl,--whole-archive ${lib_path} -Wl,--no-whole-archive")

main/CMakeLists.txt
idf_component_register(SRCS "hello_world_main.c"
    PRIV_REQUIRES spi_flash mylibA
                    WHOLE_ARCHIVE
                    INCLUDE_DIRS "")

I'm not sure if this is what you are looking for. Maybe others will have other/better idea how this should be handled, because I cmake knowledge is pretty limited.
Thank you

@fhrbata Thank you for your suggestion. I tried your method and indeed found that the pre-built libraries on build.ninja all had the -Wl,--whole-archive attribute added. However, I still couldn't pass the compilation. The compiler gave an "undefined reference" error, indicating that it couldn't find the symbols of the IDF component. I have checked and confirmed that the components with missing symbols were added to REQUIRES.

@fhrbata Thank you for your suggestion. I tried your method and indeed found that the pre-built libraries on build.ninja all had the -Wl,--whole-archive attribute added. However, I still couldn't pass the compilation. The compiler gave an "undefined reference" error, indicating that it couldn't find the symbols of the IDF component. I have checked and confirmed that the components with missing symbols were added to REQUIRES.

hi @TxRxTxRx, even after adding the PRIV_REQUIRES to the component registration in your static libs, you still get the undefined references to esp-idf components ?

@SoucheSouche
Yes, I can find the components and their static library files that indicate undefined reference errors in build/esp-idf, and confirm adding them to REQUIRES or PRIV_REQUIRES in the component registration interface of the static library.

Hi @TxRxTxRx,
I tried to create a dummy project locally with 2 static libraries ( one of them depending on the other ).
Running idf.py build in "lib_comp" will compile the dummy project to generate the .a files and place them in "main" directory together with the include files from both static libs.
Running idf.py build in "my_project" will build the main application.
CmakeLists.txt files are named according to their place in the structure so they are easy to identify later in the message.
The project is inspired from https://github.com/espressif/esp-idf/tree/master/examples/build_system/cmake/import_prebuilt as I was mentioning in a previous message.
my_project/
├── lib_comp/
│ ├── main/             // this is a dummy main to compile the libraries
│ │ ├── CMakeListsDummy.txt
│ │ └── main.c
│ ├── components/
| | ├── lib_a/
| | │ ├── CMakeListsLibA.txt
| | │ ├── lib_a.c
| | | └── include/
| | └── lib_b/
| |   ├── CMakeListsLibB.txt
| |   ├── lib_b.c
| |   └── include/
| └── CMakeListsLibComp.txt
├── main
│ ├── main.c
│ └── CMakeListsMain.txt
└── CMakeListsProject.txt

CMakeListsDummy:
    idf_component_register(SRCS "main.c"
                        INCLUDE_DIRS "")

CMakeListsLibA:
set(include ${CMAKE_CURRENT_LIST_DIR}/include)

    idf_component_register(SRCS lib_a.c
                        INCLUDE_DIRS ${include}
                        PRIV_REQUIRES app_update spi_flash log)

    # After build, copy the archive file and header file to parent example directory's main component
    add_custom_command(TARGET ${COMPONENT_LIB}
                    POST_BUILD
                    COMMAND ${CMAKE_COMMAND} -E copy $<TARGET_FILE:${COMPONENT_LIB}> ${CMAKE_SOURCE_DIR}/../main
                    COMMAND ${CMAKE_COMMAND} -E copy ${include}/lib_a.h ${CMAKE_SOURCE_DIR}/../main
                    COMMENT "Copying built archive file and header to parent example directory...")

CMakeListsLibB:
    set(include ${CMAKE_CURRENT_LIST_DIR}/include)

    idf_component_register(SRCS lib_b.c
                        INCLUDE_DIRS ${include}
                        PRIV_REQUIRES app_update spi_flash log lib_a)

    # After build, copy the archive file and header file to parent example directory's main component
    add_custom_command(TARGET ${COMPONENT_LIB}
                    POST_BUILD
                    COMMAND ${CMAKE_COMMAND} -E copy $<TARGET_FILE:${COMPONENT_LIB}> ${CMAKE_SOURCE_DIR}/../main
                    COMMAND ${CMAKE_COMMAND} -E copy ${include}/lib_b.h ${CMAKE_SOURCE_DIR}/../main
                    COMMENT "Copying built archive file and header to parent example directory...")

CMakeListsLibComp:
    cmake_minimum_required(VERSION 3.16)

    # Since we're only interested in the prebuilt libraries, trim
    # the build
    set(COMPONENTS lib_a lib_b main esptool_py)
    include($ENV{IDF_PATH}/tools/cmake/project.cmake)

    project(lib_comp C)

CMakeListsMain:
    idf_component_register(SRCS "main.c"
                        INCLUDE_DIRS ".")

    # Import the libraries, specifying target names and the libraries paths.
    add_prebuilt_library(my_lib_a "liblib_a.a")
    add_prebuilt_library(my_lib_b "liblib_b.a")

    # `main` calls a function from the libraries, so link them to `main`
    target_link_libraries(${COMPONENT_LIB} PRIVATE my_lib_a)
    target_link_libraries(${COMPONENT_LIB} PRIVATE my_lib_b)

CMakeListsProject:
    cmake_minimum_required(VERSION 3.16)

    include($ENV{IDF_PATH}/tools/cmake/project.cmake)
    project(import_prebuilt)

Hi @TxRxTxRx, I will close this issue now since I didn't get any update after my last suggestion. Feel free to reopen if you still can't solve your problem with the insights provided so far.


Add serializer to constructor for COMMS API

Might want to make it optional, might even want to make it legal for implementation to reject the argument because it has to use a specific serializer.

Update components.rst section in docs, remove reference to this ticket when complete, also add some text about supplying serializer.

in 2c433e5


Problems reported by test/run-tests.py on OS X, Ubuntu

I grabbed the current master and built the debug version on OS X 10.11.2 with Xcode 7.2. Then I ran test/run-tests.py.
./build.py --confirm --debug --git-clean-qtbase --git-clean-qtwebkit > buildout.txt 2>&1
test/run-tests.py >test/testout-debug.txt 2>&1
The result is a bunch of failures due to leak reports written to stderr as well as a crash due to a failed assertion. There's also an HTTP server error. Attached are the outputs of build.py and run-tests.py as well as the crash report.
I don't know if this is new behavior as I haven't tried run-tests.py previously.
buildout.txt
testout-debug.txt
phantomjs_2016-01-11-094955_Marcs-MacBook-Pro.crash.zip

I get similar results with Ubuntu-64 14.04.3.
testout-ubuntu.txt

Landed in master! Thank you!


NO-JIRA fix log message format in ActiveMQRALogger

MessageFormat uses single quotes to escape so string ${0} was logged.

I have no way to test this, but I will trust you :)


Updated working model of perceiver tranform

The project has been modified, and the model works as intended on the ADNI brain dataset. It does not show 100% accuracy on everything anymore, as the dataset path was changed correctly to work on split datasets, image conversion was appropriately done, and RGB was not used as all images are in grayscale. Image' labelling was fixed to identify the difference between AD and NC. A validation set was also added to work correctly after each training epoch.  The input dimensions of the image the model was being parsed have been changed accordingly to predict correctly. Finally, the plots and visualization have also been updated according to the new working code and results.

This is an initial inspection, no action is required at this point
Difficulty: Hard
Readme:

Need more explanation on how Perciever architecture works and how you developed your algorithm.
No plots of training and val accuracy
Reference section is missing

Commit messages:

clear
submitted commits on two days.

Code:

Uses Perceiver
Commented
Good model design

Functionality/Performance:

Test accuracy is 50.44%
Loss and accuracy plots are unclear (unclear legends).
Only have training ACC plot, no val ACC plot
No evidence of patient-level split
Preprocssing data correctly
No train, val and test split. Selected model only based on test set results.
No hyperparameter tuning.
The reference section is missing

Marking
Good Practice (Design/Commenting, TF/Torch Usage)
Adequate design and implementation
Good spacing and comments
Header blocks missing -1
Recognition Problem
Solves problem , but poor performance < 0.6 -1
Driver Script present
File structure present
Shows Usage & Demo & Visualisation & Data usage, no validation plot -1
Module present
Commenting
No Data leakage, no patient level split -1, no validation set -1
Difficulty: Hard
Commit Log
Meaningful commit messages
Progressive commits used but over two days -1
Documentation
ReadMe acceptable/good
Model/technical explanation
Good Description and Comments
Markdown used and PDF submitted
Pull Request
Successful Pull Request (Working Algorithm Delivered on Time in Correct Branch)
Feedback required remove pyc files and move README into folder restore repo README -2
Request Description minimal -1

Feedback marks possible +2 if the requested changes are made (see above).

No feedback attempt and no feedback marks granted.


feat: printing out deno version in command line

What kind of change does this PR introduce?
Feature
What is the current behavior?
vscode ➜ /workspaces/edge-runtime $ ./target/debug/edge-runtime -V
edge-runtime 0.1.0

What is the new behavior?
vscode ➜ /workspaces/edge-runtime (cli-deno-version) $ ./target/debug/edge-runtime -V
edge-runtime 0.1.0
deno 1.37.2 (debug, aarch64-unknown-linux-gnu)

Additional context
Resolves #236

Nice one! Thanks for the contribution.


[DO NOT MERGE] Add switch comparison experiment

For @samuelsycamore 's lighting talk at React Summit.

[ ] I have followed (at least) the PR section of the contributing guide.

Thank you so much for this!! 💙 It was the perfect demo for the talk. We should publish it in some other format! Maybe we could turn the lightning talk material into a blog post?


Not saved

Made subdir under demo => uploads"
upload.php => $output_dir = "uploads/";
Webpage works: files are uploaded
But not saved!
No errormessge.
What could be wrong?

Issue might be with folder permissions.
Can you share me your URL ?

Hello, and thanks for the reaction:
http://venes.biz/vergadering/_testmap/uploaderhayageek/demo/demo.html
The uplaodmap=
http://venes.biz/vergadering/_testmap/uploaderhayageek/demo/uploads
folder permission=777
greetings Jos
Verzonden vanuit Mailhttps://go.microsoft.com/fwlink/?LinkId=550986 voor Windows 10
Van: Ravishanker<EMAIL_ADDRESS>Verzonden: maandag 24 april 2017 16:06
Aan<EMAIL_ADDRESS>CC<EMAIL_ADDRESS><EMAIL_ADDRESS>Onderwerp: Re: [hayageek/jquery-upload-file] Not saved (#145)
Issue might be with folder permissions.
Can you share me your URL ?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/hayageek/jquery-upload-file/issues/145#issuecomment-296679100, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AGdJy21Y4rXmekSArxUbCMAracAbEiOoks5rzKxegaJpZM4NF81P.

Did you see upload.php response. This is the error.

Fatal error:  Call to undefined function file_put_content() in /home/p22562/domains/venes.biz/public_html/vergadering/_testmap/uploaderhayageek/php/upload.php on line 14
actually it is file_put_contents() not file_put_content().

Ravishankar
I removed the line. I added it for test.
Checking in firefox/network therer seems to be no troubles but the files arent moved.
Strange
//Jos
Verzonden vanuit Outlookhttp://aka.ms/weboutlook

Van: Ravishanker Kusuma<EMAIL_ADDRESS>Verzonden: dinsdag 25 april 2017 05:18
Aan: hayageek/jquery-upload-file
CC: josvenes; Author
Onderwerp: Re: [hayageek/jquery-upload-file] Not saved (#145)
Did you see upload.php response. This is the error.
Fatal error: Call to undefined function file_put_content() in /home/p22562/domains/venes.biz/public_html/vergadering/_testmap/uploaderhayageek/php/upload.php on line 14
actually it is file_put_contents() not file_put_content().
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/hayageek/jquery-upload-file/issues/145#issuecomment-296883531, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AGdJy2ceU9Phox4VdvFK--S6c7S6aIGsks5rzWYBgaJpZM4NF81P.

can you send your upload.php code ?

Warning:  move_uploaded_file(uploads/Screen Shot 2017-04-18 at 1.51.56 PM.png) [function.move-uploaded-file]: failed to open stream: No such file or directory in /home/p22562/domains/venes.biz/public_html/vergadering/_testmap/uploaderhayageek/php/upload.php on line 20

Warning:  move_uploaded_file() [function.move-uploaded-file]: Unable to move '/tmp/php0Y8lr3' to 'uploads/Screen Shot 2017-04-18 at 1.51.56 PM.png' in /home/p22562/domains/venes.biz/public_html/vergadering/_testmap/uploaderhayageek/php/upload.php on line 20
["Screen Shot 2017-04-18 at 1.51.56 PM.png"]

This might help you

upload.php code

Verzonden vanuit Outlookhttp://aka.ms/weboutlook

Van: Ravishanker Kusuma<EMAIL_ADDRESS>Verzonden: dinsdag 25 april 2017 08:57
Aan: hayageek/jquery-upload-file
CC: josvenes; Author
Onderwerp: Re: [hayageek/jquery-upload-file] Not saved (#145)
can you send your upload.php code ?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/hayageek/jquery-upload-file/issues/145#issuecomment-296933096, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AGdJy52mi2e9zLM1q8SbVDCsdinhLKwtks5rzZk_gaJpZM4NF81P.

Ravishanker
I've solved it!
Got warning in firefox: HTTP wrapper does not support writeable connections
So i changed http:// in //home (server path)
And it worked!
Thanks for your kind help!
//Jos Venes
Verzonden vanuit Outlookhttp://aka.ms/weboutlook

Van: Ravishanker Kusuma<EMAIL_ADDRESS>Verzonden: dinsdag 25 april 2017 08:57
Aan: hayageek/jquery-upload-file
CC: josvenes; Author
Onderwerp: Re: [hayageek/jquery-upload-file] Not saved (#145)
can you send your upload.php code ?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/hayageek/jquery-upload-file/issues/145#issuecomment-296933096, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AGdJy52mi2e9zLM1q8SbVDCsdinhLKwtks5rzZk_gaJpZM4NF81P.


Sockets are not working with CloudFlare

I have nodejs web socket server running on port 3000 on my Debian based VPS. I am currently using free plan of cloudflare.
socket.mydomain.com is gray record in CloudFlare and sockets are working with this URL but I don't want to work this way because I don't want to expose my server's IP address. Main problem is that sockets are not working on main orange record in cloudflare. Cloudflare have sockets support in free plan too but why these are not working? What I am doing wrong?
I have Laravel application and it was not working with Cloudflare initially and I had to install https://github.com/fideloper/TrustedProxy to work with cloudflare.
I tried to follow this guide but no luck:
http://expressjs.com/en/guide/behind-proxies.html
I have been trying to solve this issue for weeks.
Here is the nodejs socket server codes:
var express = require('express');
var https = require('https');
var http = require('http');
var app = express();
var port = '3000';

//app.enable('trust proxy fn');
//app.set('trust proxy fn', 'loopback, linklocal, uniquelocal');
app.set('trust proxy fn', [
	// Ipv4
	'<IP_ADDRESS>/22',
	'<IP_ADDRESS>/22',
	'<IP_ADDRESS>/22',
	'<IP_ADDRESS>/12',
	'<IP_ADDRESS>/18',
	'<IP_ADDRESS>/22',
	'<IP_ADDRESS>/18',
	'<IP_ADDRESS>/15',
	'<IP_ADDRESS>/13',
	'<IP_ADDRESS>/20',
	'<IP_ADDRESS>/20',
	'<IP_ADDRESS>/20',
	'<IP_ADDRESS>/22',
	'<IP_ADDRESS>/17',
	'<IP_ADDRESS>/21',
	// Ipv6
	'2400:cb00::/32',
	'2405:8100::/32',
	'2405:b500::/32',
	'2606:4700::/32',
	'2803:f800::/32',
]);

var server = http.createServer(app);

var io = require('socket.io')(server);
var Redis = require('ioredis');
var redis = new Redis({password: "myredispassword"});

server.listen(port, function () {
	var addr = server.address();
//	console.log(addr);
	console.log('   server listening on ' + addr.address + ':' + addr.port);
});

//function handler(req, res) {
//	console.log(req);
//	res.writeHead(200);
//	res.end('');
//}

io.on('connection', function (socket, req) {
//	console.log(socket);
//	console.log(req);
});

redis.psubscribe('*', function (err, count) {
	//
});

redis.on('pmessage', function (subscribed, channel, message) {
	message = JSON.parse(message);
	io.emit(channel + ':' + message.event, message.data);
});

Client-side code:
<script src="{{ asset('assets/js/socket.io.js') }}"></script>
//	var socket = io('<?php echo url('/'); ?>:3000', {secure: false, port: 3000}); // does not work
var socket = io('socket.mydomain.com:3000', {secure: false, port: 3000}); // works but prone to DDoS

socket.on('test-event:App\\Events\\TestEvent', notifyUser_mine);
</script>

@salmanaligeek none of your code is using Express. I think you posted this on the wrong repository.

@dougwilson I have updated my code. Please check.

@saintedlama, your question is still about WebSockets, which Express has nothing to do with. If you are looking for support, please post on our support channel: https://gitter.im/expressjs/express

@salmanaligeek I think what Doug was trying to say is that Express doesn't have anything to do with the sockets.  Those are all handled between the core node server and socket.io.  Even in the updated code above, express and socket.io dont talk.  You pass the core node http server to socket.io.

@wesleytodd, the original post literally did not even have Express in the code in any way. This question was "carpet bombed" across a bunch of repos:

The correct repo for @salmanaligeek's question is probably the Socket.io one.


Init redit work

Potential solution for #31

Can this be merged?

@roscoegray needs some work still - e.g. storing the persistence layer elsewhere for Redis


Addition of FieldData

Any issues with me adding the ability to write a bit of global data, via the FieldData element?  Attached screenshot indicates what I am thinking . . .  Thx.

Now that I look at the screenshot more carefully, I will say the following:

I won't use self-closing tags, and will use separate, explicit closing tags, per the current formatting convention.
I wouldn't bother with the string array element, as that is likely an extension for our particular case.  I really just want to get the float guy in there.

Sure, that would be great! Feel free to make a pull request with your additions and, if you can, add a test so that I can verify the expected behavior before merging.

Heh, lets see if I did the fork+PR thing correctly . . .


Client Error? Where should I add the IAM role to allow the client to access stats?

[ec2-user@ip-10-6-22-221 prometheus-ecs-sd]$ python discoverecs.py --directory /opt/prometheus-ecs
Starting. Directory: /opt/prometheus-ecs. Interval: 60s.
Traceback (most recent call last):
File "discoverecs.py", line 410, in
main()
File "discoverecs.py", line 407, in main
Main(args.directory, float(args.interval)).loop()
File "discoverecs.py", line 398, in loop
self.discover_tasks()
File "discoverecs.py", line 359, in discover_tasks
targets = self.get_targets()
File "discoverecs.py", line 353, in get_targets
infos = self.discoverer.get_infos()
File "discoverecs.py", line 227, in get_infos
for clusters in clusters_pages:
File "/home/ec2-user/.local/lib/python2.7/site-packages/botocore/paginate.py", line 255, in iter
response = self._make_request(current_kwargs)
File "/home/ec2-user/.local/lib/python2.7/site-packages/botocore/paginate.py", line 332, in _make_request
return self._method(**current_kwargs)
File "/home/ec2-user/.local/lib/python2.7/site-packages/botocore/client.py", line 357, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/home/ec2-user/.local/lib/python2.7/site-packages/botocore/client.py", line 661, in _make_api_call
raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (UnrecognizedClientException) when calling the ListClusters operation: The security token included in the request is invalid.

Hi HIMADRI518, are you running discoverecs.py from an ec2 instance? If that is the case, you'll need to attach an IAM instance profile (with an attached role) to your ec2 instance so discoverecs.py can pick it up through the boto3 library. Check our README.md file (https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-ec2_instance-profiles.html) for the kind of policy you will need to attach to your IAM role. This also might help:
https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-ec2_instance-profiles.html

Thanks for responding. I am aware about attaching an instance profile with relevant role but I am still seeing the same error. That's why I would really appreciate if you can guide with what specfic policy to attach that grants access to applications running on ec2 instances ?

Here is a list of policies I have in my role,
AmazonEC2RoleforSSM
AmazonEC2ContainerRegistryFullAccess
AmazonSSMFullAccess
AmazonEC2ContainerServiceFullAccess
AmazonECSTaskExecutionRolePolicy
AmazonEC2ContainerServiceforEC2Role
AmazonEC2ContainerServiceRole
AmazonSSMReadOnlyAccess

Looks like I had to make some changes on my instance credentials. Running into this error now,
self.write_jobs(jobs)

File "discoverecs.py", line 347, in write_jobs
with open(tmp_file_name, 'w') as f:
IOError: [Errno 2] No such file or directory: '/opt/prometheus-ecs/5m-tasks.json.tmp'

Sorry for the delay. From the error, it seems the /opt/prometheus-ecs doesn't exist. You will have to create it first:
mkdir /opt/prometheus-ecs
and run the tool again. You can of course, choose a different directory. Make sure both the tool and Prometheus have the permissions to access.

Closing this for now.


Missing flag to avoid decryption in tests

Is it possible to tur off decryption of properties in integration tests?
I've got a bunch of @SpringBootTests which raise Spring context including property source beans, so in case I don't provide jasypt.encryptor.password, context creation fails.
I don't necessarily want to take care of storing the password a safe way for tests only. In my tests I don't also need the properties to be decrypted. Is there a way of turning off the decryption process?
If not - a property like jasypt.encryptor.enabled would come in very handy.

It should not bother you if you don’t have any encrypted properties. But I’ll doble check

I am using 2.1.0 and I am seeing behavior I didn't expect.
ENC property like this works fine
spring.datasource.password=ENC(1W+NaD8JG7UvJE/7tc8iQg==)
if I use the clear text version it is still trying to decrypt it for some reason.  Is it looking for properties that contain the word "password" instead of just properties that use ENC()?
spring.datasource.password=clear

Ok, do you have another ENC() property somewhere else? I just tested one of the sample apps and as far as I don't have any ENC() decorated property I don't need to set the password.
Check this sample where app and tests don't provide password and everything runs properly: https://github.com/ulisesbocchio/jasypt-spring-boot-samples/tree/master/jasypt-spring-boot-demo-no-password


PDF CAN'T DOWNLOAD

All files in Ad-papers can't download!~

Both preview on web and download work for me. Perhaps it's your system/browser. What error do you see?


Add ValidateTemplate function

Discussed this with @azazeal. We tried Option("missingkey=zero", but that results in nil pointer errors. There may be ways to circumvent these, but not trivial to do. Also discussed trying to come up with a set of properties we know should work and providing these as template data for full execution and parsing. This is also not trivial to do, but might be worth it at a later time.
In the tests I have two test cases that won't return an error when validating the template, but would result in a failure when signing certs. It would be nice to negate those tests in the future with improved validation.


set mongo port in config

Actually, this doesn't need to be a config option - we should be able to grab it from the MONGO_URI


Flutter plugin onAttachedToEngine called twice

Steps to Reproduce

I have attached the test plugin to check. testplugin.zip
Run the example
When you check on logcat you will see that the "I GOT ATTACHED....." called twice

This bug only happen when I have the android_alarm_manager in the example project and called AndroidAlarmManager.initialize()

Doctor summary (to see all details, run flutter doctor -v):
[✓] Flutter (Channel stable, 1.20.2, on Linux, locale en_US.UTF-8)

[✓] Android toolchain - develop for Android devices (Android SDK version 29.0.2)
[✓] Android Studio (version 3.5)
[✓] VS Code (version 1.50.1)
[✓] Connected device (1 available)

• No issues found!

@apinprastya
Can you upgrade to latest stable and try again to see if issue perists ?

Still happen with 1.22.3
Doctor summary (to see all details, run flutter doctor -v):
[√] Flutter (Channel stable, 1.22.3, on Microsoft Windows [Version 10.0.19041.572], locale en-US)

[√] Android toolchain - develop for Android devices (Android SDK version 29.0.3)
[√] Android Studio (version 4.0)
[√] VS Code, 64-bit edition (version 1.50.1)
[√] Connected device (1 available)

flutter doctor -v

[✓] Flutter (Channel stable, 1.22.3, on Mac OS X 10.15.4 19E2269, locale en-IN)
    • Flutter version 1.22.3 at /Users/dhs/documents/Fluttersdk/flutter
    • Framework revision 8874f21e79 (6 days ago), 2020-10-29 14:14:35 -0700
    • Engine revision a1440ca392
    • Dart version 2.10.3

[✓] Android toolchain - develop for Android devices (Android SDK version 30.0.0)
    • Android SDK at /Users/dhs/Library/Android/sdk
    • Platform android-30, build-tools 30.0.0
    • Java binary at: /Applications/Android
      Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build
      1.8.0_242-release-1644-b3-6222593)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 12.0.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 12.0.1, Build version 12A7300
    • CocoaPods version 1.9.3

[✓] Android Studio (version 4.0)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin version 46.0.2
    • Dart plugin version 193.7361
    • Java version OpenJDK Runtime Environment (build
      1.8.0_242-release-1644-b3-6222593)

[✓] VS Code (version 1.50.1)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.15.1

[✓] Connected device (1 available)
    • SM A260G (mobile) • 5200763ebcfa861f • android-arm • Android 8.1.0 (API
      27)

• No issues found!

We hit the similar issue when integrate our flutter plugin with firebase_messaging, here is a repro PR https://github.com/square/in-app-payments-flutter-plugin/pull/153.

We faced the same issue and this caused other library failed to work (Segment)

Same issue. I didn't yet manage to figure out what's the cause of it but I'm using firebase_messaging too

Closing as android_alarm_manager is deprecated in favor of android_alarm_manager.

For anyone finding this in the future, this is expected behavior, not a bug. See https://github.com/flutter/flutter/issues/150855.


SQL files under docker-entrypoint-initdb.d are not executed even if data directory is not initialized

According to the documentation running docker compose on this yaml file:
services:
  masterdb:
    image: postgres:15.6-alpine
    container_name: primary-server
    ports:
      - 5432:5432
    environment:
      POSTGRES_USER: admin
      POSTGRES_PASSWORD: psw
    volumes:
      - ./init.sql:/docker-entrypoint-initdb.d/init.sql
  readonlydb:
    image: postgres:15.6-alpine
    container_name: hot-standby
    ports:
      - 5430:5432
    environment:
      POSTGRES_USER: admin
      POSTGRES_PASSWORD: psw

should initialise:
primary-server with init.sql.
However it does not happen even if the file is correctly mounted:

Ii tried, to delete/re build the images, delete all the volumes but nothing. The init.sql script is not executed.

Compose works really hard to keep and re-use volumes between runs (including and especially "anonymous" volumes that aren't specified in the file, but are specified in the images in use), which is probably what you're running into.
Have you tried docker-compose down --volumes and/or docker-compose up --renew-anon-volumes?


[GLUTEN-4548][VL] Ubuntu 18 Support

What changes were proposed in this pull request?
This change adds support for Ubuntu 18 builds by adding the necessary setup steps in dev/vcpkg/setup-build-depends.sh
(Fixes: #4548)
How was this patch tested?
This was tested by running the setup script and a full build with vcpkg enabled on Ubuntu 18.

did you try vcpkg on? in long term we are going to only support vcpkg on

@FelixYBW Yes, I built with ENABLE_VCPKG=ON

===== Performance report for TPCH SF2000 with Velox backend, for reference only ====

query
log/native_4549_time.csv
log/native_master_01_28_2024_d041ce584_time.csv
difference
percentage

q1
35.77
32.99
-2.778
92.23%

q2
24.47
23.85
-0.621
97.46%

q3
37.67
37.74
0.061
100.16%

q4
38.32
38.98
0.653
101.70%

q5
71.12
67.95
-3.177
95.53%

q6
7.24
7.02
-0.212
97.08%

q7
83.38
80.87
-2.511
96.99%

q8
85.45
84.35
-1.099
98.71%

q9
122.61
121.55
-1.061
99.13%

q10
44.43
43.06
-1.374
96.91%

q11
19.72
20.27
0.556
102.82%

q12
30.44
27.38
-3.052
89.97%

q13
46.50
45.45
-1.046
97.75%

q14
15.70
22.94
7.245
146.15%

q15
29.27
28.71
-0.559
98.09%

q16
14.32
14.08
-0.239
98.33%

q17
101.91
102.67
0.762
100.75%

q18
150.97
148.89
-2.080
98.62%

q19
14.02
12.47
-1.550
88.94%

q20
26.14
26.11
-0.029
99.89%

q21
224.74
226.03
1.294
100.58%

q22
13.75
13.55
-0.202
98.53%

total
1237.95
1226.93
-11.021
99.11%


[release/1.5] test: introduce failpoint control to runc-shimv2 and cni

backport #7069

cherry-pick: https://github.com/containerd/containerd/pull/7069/commits/ffd59ba6001af605e57c08a5fc07c8337954e190
cherry-pick and write:

https://github.com/containerd/containerd/pull/7069/commits/822cc51d840775f0c9aa48e6e93cecc657a90f06
https://github.com/containerd/containerd/pull/7069/commits/5f9b318e506b9ebf16fa5caf7a63b47b9ad0b0e5
These patches depends on the https://github.com/containerd/containerd/pull/6021 so we can't use rpc middleware to invoke the failpoint. We use wrapper to invoke failpoint in release/1.5.

cherry-pick: https://github.com/containerd/containerd/pull/7069/commits/1ae6e8b076bd2a02cf622b5bf79c17c253dae845
cherry-pick: https://github.com/containerd/containerd/pull/7069/commits/be91a219c2ac5e65c00bbe85c5dff0827d41958b
cherry-pick: https://github.com/containerd/containerd/pull/7069/commits/cbebeb9440b236e92e0ac8555aab1c72721b2864
cherry-pick: https://github.com/containerd/containerd/pull/7069/commits/e6a2c079025aa2db5eeb635d4269925d8d3ac0de
Rewrite:

https://github.com/containerd/containerd/pull/7069/commits/3c5e80b63e683453fee48093347c7c1e969450a7
The patch depends on https://github.com/containerd/containerd/pull/4695. Since we don't have concurrent test cases in cri-integration, we can replace the CNI config for the case.

cc @samuelkarp @qiutongs

Would we backport changes that utilizes the failpoints to 1.5.x?

Would we backport changes that utilize the failpoints from main to 1.5.x?

@kzys YES. This pull-request is backporting the failpoint to 1.5. We have plan to backport sandbox rollback bugfix, which relies on this pull-request.


fix: ensure only rank 0 process creates output directories

When running with multiple GPUs, prevent directory creation conflicts
by having only the main process (rank 0) create output directories.
Other processes will wait at a barrier until directories are created.

Add rank check before directory creation
Add distributed barrier for process synchronization
Keep existing directory existence check for safety

I have tested this fix using 2 GPUs and AWQ and it works.
Let me know any thoughts!

Please use pre-commit to check the format before pr.

Please use pre-commit to check the format before pr.

Please excuse me. Yes, I'll do that in the future 🫡


jshint comments breaking into new line

First, really awesome tool!
Second, is there any way to prevent //jshint comments from breaking ?
ex:
function test(arg1,arg2,arg3) { //jshint ignore:line
  code here..
}

become:
function test(arg1,arg2,arg3) {
  //jshint ignore:line
  code here...
}

Comments are currently somewhat buggy and I'm working on a rehaul of them now.

See #275