template: Process Directories and Files

In order to properly allow for templates in directory names, we must
adjust the WalkDir to include both files and directories AND to yield
contents first.
Given that the project name is bar, and the directory tree:
| {{project-name}}/foo.rs
| {{projcet-name}}

{{project-name}}/foo.rs is read and the contents rendered
fs:create_dir_all will create the bar directory
fs::write will write bar/foo.rs
the {{project-name}} directory is then deleted with fs::remove_dir_all

Which will provide:
| bar/foo.rs
Fixes: #396
Command output from this commit:
🔧   Creating project called `foo`...
[ 1/12]   Done: .appveyor.yml
[ 2/12]   Done: .gitignore
[ 3/12]   Done: .travis.yml
[ 4/12]   Done: Cargo.toml
[ 5/12]   Done: LICENSE_APACHE
[ 6/12]   Done: LICENSE_MIT
[ 7/12]   Done: README.md
[ 8/12]   Done: src/lib.rs
[ 9/12]   Done: src/utils.rs
[10/12]   Done: src
[11/12]   Done: tests/web.rs
[12/12]   Done: tests
✨   Done! New project created /home/dave/dev/cargo-generate/foo

thanks for your fix!

One odd side effect is that I can't figure out how to prune the root directory from the WalkDir:

Above you'll find a solution for the issue

Thanks @sassman :pray: I've incorporated your suggested fix and it works as expected!

Thanks a lot @dave-tucker


xserver: check case of multiple keyboard layouts

Motivation for this change
#26961
Things done

[x] Tested using sandboxing
(nix.useSandbox on NixOS,
or option build-use-sandbox in nix.conf
on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] Linux

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[x] Fits CONTRIBUTING.md.

cc @aszlig

I'm going to commit a different fix of #26961 which is similar to your PR, but with the comments addressed. Basically I was the one who broke it, so I'll fix it :-D

Alright.


Pull dialog: Usability

Ref https://github.com/abapGit/abapGit/issues/6107

Move "Continue" button to left
It was there before and matches layout for other AG forms.
Add "Ctrl-S" shortcut for "Continue"
This allows using "Enter" to select/deselect rows and submitting popup without having to tab anywhere else

After:

Move "Continue" button to left, It was there before and matches layout for other AG forms.

I object. This does not matter how it was before or how it's done elsewhere. It matter what's better. As I mentioned in my other messages:

there is POV of a mouse user: for them it is better that most used buttons have fixed positions, independent of dynamic toolbar change (jumping) or further development improvements. Besides, OK and Cancel just group together naturally.
then there is a Keyboard user POV (as Christian described, and I confess I missed that perspective initially). what he actually described is that after selecting the lines with keyboard, the next TAB-stop was the Continue button (before) and that was fast and convenient. And I totally agree. But my idea with Ctrl+S or F2 was that it is actually event faster and more universal and accessible from any point at the screen, including, when there is no need to select (and thus tab-through) all lines. Thus a dedicated shortcut is just superior to the button position for a keyboard user.

I'd suggest moving the button back.
Using the opportunity, I'd also suggest adjusting the keys a bit (as a proposal).
Move SEL_CAT to F8, so that all selection related buttons are in line and in the same sequence. Also I had an idea to change the logic of (De)Select All as they are a bit confusing now when there is a filter applied. But I can do it n a separate PR as well

I made this change based on my experience with this (changed) dialog. I use mouse and keyboard.
It's significantly more difficult to find/click the button in the middle of a toolbar esp. since you use "ok" almost every time (in general and here too). It's also a shorter distance from the checkboxes. Consistency matters as well. Why should this dialog be different from all others? It's better with the button on the left.
Ctrl-S was added for keyboard and it works very well. Where's the problem?
PS: I moved SEL_CAT to F8.

I guess the problem is in different approach to product design. Whatever. It's a tool, it works

As a most of the time keyboard only user I now have several options to confirm the popup.

CTRL+TAB, Enter/Space
Enter when cursor is on any non hotspot column
CTRL-S
As I like to have options I like this change especially as I can still use my old way to confirm this popup (first bullet point)

When I'm forced to use the mouse, e.g. while working on Citrix with different keyboard layout, this point is crucial to me:

It's significantly more difficult to find/click the button in the middle of a toolbar esp. since you use "ok" almost every time (in general and here too)

So while understanding Alex arguments I vote for merging this PR. Maybe we can get a few more opinions.


Trailing slash handling causes error "invalid field key: must not be empty"

It seems that attempting to list flags (v0.2.2 and v0.2.3), the trailing slash in the request url causes the api to error out with:
{
"code": 3,
"message": "invalid field key: must not be empty",
"details": []
}

https://github.com/flipt-io/flipt-node/blob/1522bcc1bf02ad2bfffddfd866e0af3998a578f4/src/api/resources/flags/client/Client.ts#L40

@markphelps thanks for the report -- will get back to you when fixed

PR here to upgrade fern + the TS generator to the latest versions: https://github.com/flipt-io/flipt-api/pull/69. This includes the fix for not generating trailing slashes!

@R0hitYadav this should be fixed now in the latest release https://www.npmjs.com/package/@flipt-io/flipt
sorry for the inconvenience, please let us know if that fixes the issue

@markphelps @dsinghvi @zachkirsch Thanks for creating Flipt, I recently migrated from Unleash and I've been enjoying Flipt a lot. Can confirm that the issue's fixed now, thanks again for the exceptional response time.


Add documentation to public APIs and to tasks in tasks.py

Need to add documentation to our APIs, but also to tasks in our tasks.py. The latter needed also to allow projects using us to copy-paste them to their similar tasks.
SeleniumLibrary's tasks should be updated as part of this issue. See PR robotframework/SeleniumLibrary#895 and issue robotframework/SeleniumLibrary#634.

Tasks have docs. That ought to be enough at this point.


Intermittent PiCameraValueError raised when using picamera.array.PiRGBArray and KeyboardInterrupt is raised

I have intermittently encountered a PiCameraValueError when I Ctrl-C out of my application that is capturing images in a while True loop. I'm using PiRGBArray. It looks like this happens when the Ctrl-C occurs before all the bytes have been received by the encoder and encoder.stop() is called, meaning when flush is called on the stream bytes_to_rgb has less data than it expects.
Not entirely sure whether this needs to be fixed, and if so, where the best place to fix it is. Perhaps we could add an optional parameter in PiEncoder._close_output(suppress_value_errors=False) that catches and suppresses  PiCameraValueError errors raised if suppress_value_errors == True?
Thanks,
Ken
Stacktrace:
File "./motion.py", line 136, in <module>
  motion_detector.run()
File "./motion.py", line 91, in run
  next_img = self.camera.capture_opencv_image()
File "./motion.py", line 33, in capture_opencv_image
  self.camera.capture(stream, use_video_port=True, format='bgr')
File "/usr/lib/python2.7/dist-packages/picamera/camera.py", line 1466, in capture
  encoder.close()
File "/usr/lib/python2.7/dist-packages/picamera/encoders.py", line 758, in close
  self.stop()
File "/usr/lib/python2.7/dist-packages/picamera/encoders.py", line 746, in stop
  self._close_output()
File "/usr/lib/python2.7/dist-packages/picamera/encoders.py", line 660, in _close_output
  output.flush()
File "/usr/lib/python2.7/dist-packages/picamera/array.py", line 275, in flush
  self.array = bytes_to_rgb(self.getvalue(), self.size or self.camera.resolution)
File "/usr/lib/python2.7/dist-packages/picamera/array.py", line 173, in bytes_to_rgb
  'Incorrect buffer length for resolution %dx%d' % (width, height))
picamera.exc.PiCameraValueError: Incorrect buffer length for resolution 320x240

Hi Ken,
Sorry for the long delay in getting to this. The problem is that when you hit Ctrl+C, sooner or later the main thread stops the recording. When stopping the recording, the encoder will attempt to flush the output object. The stream-array classes use the flush call to determine when to try and construct an array from the output. In the event of a partially written buffer, the stream-array classes raise the PiCameraValueError.
In other words, the PiCameraValueError is a secondary exception raised in response to the KeyboardInterrupt exception. This is quite normal and not a bug (it's more a consequence of how generic the output mechanism is in picamera), but it is difficult to detect and respond to in Python 2 (the usual method there would probably be to catch the PiCameraValueError as well in whatever loop your main script runs). Under Python 3, exception chaining is implemented so you could potentially look back through the stack of exceptions to find the KeyboardInterrupt one.
Basically this isn't something I can easily fix in picamera, and certainly not in a way that is compatible with all python versions that are currently in the wild. Sorry!

you need to clear down the stream between captures. The PiRGBArray docs cover doing this with truncate function see http://picamera.readthedocs.org/en/release-1.7/array.html#picamera.array.PiRGBArray

Ah, thanks for answering. That makes perfect sense. I'll catch the exception then.


bug

Basic information

My operating system and version:win11
My mzmine version: 4.0.3

What happened

When processing .raw raw data in bulk, sometimes the program crashes directly after lagging, leading to the loss of all progress.
When using NIST MS search, the results of annotations for Greek letters such as α, β, etc., cannot be recognized, and there is a desire to improve the encoding format.

Please describe what happened and how it differs from what you expected to happen.
Please include all parameter values for the mzmine modules you used.
Please upload screenshots of any errors and your log/mzmine.log file.
Please upload a standalone mzmine project file (or a batch + raw files) that demonstrates the problem you are reporting. You can use Google Drive, Dropbox, or a similar repository to upload the project.
Issues opened without an example project are unlikely to be processed

Hi,

Did you check if there is enough space available on the disc with the temporary file path? (See the preferences where it is currently located). Make sure there is enough space and that the folder is not on a network drive or a HDD, but an SSD (nvme at best).
Otherwise please provide the log file.

do you mean the result is not properly imported into mzmine? please specify

Hi,

Did you check if there is enough space available on the disc with the temporary file path? (See the preferences where it is currently located). Make sure there is enough space and that the folder is not on a network drive or a HDD, but an SSD (nvme at best).
Otherwise please provide the log file.
do you mean the result is not properly imported into mzmine? please specify

1.The temporary file storage disk is a solid-state drive, and the capacity should be sufficient.
2. The results annotated with NIST MS search may contain unrecognizable symbols as shown in the figure.
Additional Information:
3. When using version 3.9, although there may be situations where the system freezes, it does not crash. After waiting for a sufficient amount of time, it will resume normal operation, and the process will not be lost. However, in version 4.0.3, the software automatically closes after freezing for a period of time.
4. The ADAP resolver is not available in the new version. In this case, what should be used as a replacement for processing GC-MS data?

mzmine_console（1. The situation when the system freezes.）.txt

The log at the time of data entry.txt

Hi,
ok thanks for the info. We recommend to not put the temp folder in the MZmine folder, but rather next to mzmine or somewhere different. E.g. E:\software\mzmine_temp or E:\mzmine_temp in your case.
I agree that there should be enough disk space in your case.
Does mzmine always crash when importing a large number of files? this might be related to another issue we observed ourselves. Do you have close to 26 GB of ram? maybe 32 or so?

Hi,
ok thanks for the info. We recommend to not put the temp folder in the MZmine folder, but rather next to mzmine or somewhere different. E.g. E:\software\mzmine_temp or E:\mzmine_temp in your case.
I agree that there should be enough disk space in your case.
Does mzmine always crash when importing a large number of files? this might be related to another issue we observed ourselves. Do you have close to 26 GB of ram? maybe 32 or so?

My computer has 32GB of memory. I attempted to change the location of the temp folder, but still couldn't avoid the freezing when inputting a large amount of data.When using Mzmine 3.9.0, it also lags, but it will recover after a while. However, when using Mzmine 4, it freezes when there are many input files, and it does not return to normal.

Ok, thanks for the info. I think as a workaround you can import the files in 2 (or more) steps. We will fix this for the next version.


What is the Button called "Text" in codeblock WYSIWYG.  Can it be removed

Summary
I cannot find any documentation about how the WYSIWYG Codeblock button named "Text" is supposed to be used.
For my application I find it disturbing, and would really like to removed or hide it.
Screenshots
In attached screenshot, the button says "js" and has a pen.  I do realize that I can change the text, and I assume it is for syntax highlighting.  However, I wish to find a way to remove it.
Version
3.0
Additional context

I guess I may answer it myself.  Added the following CSS:
    .toastui-editor-ww-code-block:after{         visibility: hidden !important;     }
The !important was exactly that; important!


Fixed the build

The most recent CLI tools update started to cause some problems with our project structure.
This fixes the issues and makes it possible to build cofxlab from the command line.

Aside from adding 2 new projects to the solution, this change appears to be benign. I'm curious how the projects before were causing problems building. Seems like a bug in the CLI that should be reported.

@piotrpMSFT, it seems like the CLI tools we picked up sometimes don't properly handle package reference in separate item groups.

It seems like this is not actually fixing anything. It just happened to built on my machine after these changes. some race is happening in the toolchain.


IDENTITY-5995 [LDAPConnector] Decouple username attribute with user entry RDN

Currently RDN coupled with username attribute;
means

user entry created with the username attribute as its RDN and also
we try to authenticate with the same attribute.

This lead us to impossible to rename usernames.
Labels
Type: Improvement
Priority: High
Component/s: user-core, user-store, user-store-ldap
Severity: Major
Estimated Complexity: Moderate
Issue migrated from: https://wso2.org/jira/browse/IDENTITY-5995

This issue is being closed due to extended inactivity. Please feel free to reopen it if further attention is needed. Thank you for helping us keep the issue list relevant and focused!


Connection to trace server fails in gitpod demo

Server starts successfully (in the gitpod demo the server is started manually in the gitpod.yml script), but when opening a trace, the front end appears to fail to connect to the trace server. (The front end tries to automatically restart the server and fails with "Error opening serial port 8080 (Port busy)" since there's already a server running on that port.)
More context courtesy of @arfio:

Dev console shows: "has been blocked by CORS policy: Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource."
Others experiencing the same issue recently: CORS errors due to new Gitpod port policy?

https://user-images.githubusercontent.com/28311615/125679569-b3e002cd-365d-4891-8897-f2540cc6bf7e.mp4
Hypothesis 1 - Externally started server is confusing the front end (incorrect)
Stopped the manually started server and allowed the front end to start the server. Initially, this appears to work (trace file explorer comes up), but when a trace is opened it seems to lose connection to the server and tries to restart it (but fails because there's already a server running).
https://user-images.githubusercontent.com/28311615/125683099-5c545102-c020-4c5b-94c9-c49558bc40ac.mp4
Hypothesis 2 - Gitpod is using port 8080 in some way and this is causing issues (incorrect)
Changed the port in the Trace Viewer settings and tried to open traces. Got the same behaviour as the previous hypothesis (file explorer comes up, but when try to open trace it breaks) and restart server, but always receives the same port busy error.
https://user-images.githubusercontent.com/28311615/125683734-1cea9a74-a470-4350-b9d2-9d5ede3424b6.mp4
Unverified hypotheses

Server hasn't started completely when the trace files are opened and so it thinks no server is running and tries to restart it

Gitpod configuration was changed with this pull request: https://github.com/gitpod-io/gitpod/pull/4548
This breaks the connection between the trace-compass server and the theia-trace-extension frontend. Changing the port visibility to public makes the gitpod demo work again. https://www.gitpod.io/docs/config-ports#configure-port-visibility
Also it is possible to test it quickly if you click on the lock in the list of ports in Gitpod. This makes the port public without having to modify the configuration.

I confirm that manually changing the service on 8080 to public makes things work again! Thanks @arfio!
With private 8080 and then public 8080:
https://user-images.githubusercontent.com/28311615/125689970-fb35967e-fb06-4b2f-b039-4cacfe897c26.mp4


Healtcheck with custom listen-http

I'm running Ntfy in a docker-compose setup with Traefik in front.
Upon upgrading from 1.31.0 to 2.0.1 I'm no longer able to access Ntfy through Traefik.
I've traced the issue down to the container health checks added in #555.
The health check doesn't take the  listen-http setting into account, causing issues for those of us who's not running Ntfy on localhost:80.
This issue is easily fixed by either disabling the health check or overriding the default health check URL (See https://docs.docker.com/compose/compose-file/compose-file-v2/#healthcheck).

Do you suggest I remove the HEALTHCHECK in the Dockerfile entirely?  Maybe this can be added to the docs somewhere?
I am leaning towards removal entirely, honestly. The healthcheck doesn't add much.

I would remove it and add a section to https://docs.ntfy.sh/install/#docker describing how to configure it.
I suspect leaving it in the Dockerfile is gonna be a source of confusion and it will most likely affect more people than removing it again.

I am leaning towards removal entirely, honestly. The healthcheck doesn't add much.
Deleting the healthcheck entirely gets my +1

I have a supposedly "unhealthy" ntfy because I don't run it on port 80 outside the container.
As I use watchtower to automatically update, the healthcheck disable or override using composer gets reset on the updates. So I would prefer to remove the healthcheck.
We will notice a not running ntfy container soon enough anyway when scripts fail to send notifications. ;)

I quickly whipped together a fix here .
Let me know if you want me to make a PR :+1:

@Andersbiha That looks great. I'll happily accept that PR.

@binwiederhier Thank you for a great project and the quick merge. :+1:


here's the code for a tiny "clear contents" plugin. Feedback?

I wrote a very simple plugin for adding a clear button (an "x" to the left of the arrow plugin) It's basically the same as the clear button in google docs; clicking it removes all tags and text from the textarea
js:     http://pastebin.com/C7kJig1R
css:  http://pastebin.com/dBQXBv8T
Anyone find this useful? I'd love feedback on the code; i basically just copied the arrow plugin and changed the p.onArrowClick function. I'm not sure if I'm following textext conventions in that code.

Can you elaborate a little more? Are you suggesting having the plugin fire an event before it's actually cleared to provide a hook for confirm dialogs, etc?

Yes :) typing one value is not bad. But typing all (after accidental remove) of is not.

dang, that is the fastest reply ever! :P
Ok, I think that's a good idea. I'm not entirely sure how to create a cancelable event in textext but I'll take a look at the other plugins and see if I can mangle something together. :)

btw is there a preferred way to share the progress? pastebin rocks but I feel dirty using it here. Maybe a gist instead?

I think making an personal repository is best, also because your having an CSS file.
Making a confirmation check can be tricky, just calling an function and checking whether it returned false does not work in a JavaScript world since for example a jQuery UI dialog uses callbacks that get fired later.
Maybe you can get some inspiration from the newly added tagClick event handler.


[Todo-MVP] Statistics Activity back button exits app

When the user clicks on the navigation drawer from the Tasks activity and navigates to the Statistics activity the backstack is erased.  When the user clicks on the back button on the Statistics page it exits the app instead of taking back to the Tasks page.  Deleting these two intent flags fixes this behavior.

I signed it.

Issue: #287

good!


[REQUEST-FOR-EXTERNAL] Table 5965 "Service Contract Header" Function CalculateEndPeriodDate

Can you make CalculateEndPeriodDate to be global/external, curently it is local.
I have subscribed to Event OnBeforeValidateNextInvoicePeriod and I would like to call this function

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. Please do not reply to this, as we do not monitor closed issues. If you have follow-up questions or requests, please create a new issue where you reference this one.


Error when assigning to read-only interface properties from merged interfaces

From @nbransby on https://github.com/Microsoft/TypeScript/pull/3333#issuecomment-207822397:
interface Foo {
    readonly name: string
}

class Foo {
    constructor(name: string) {
        this.name = name; //error
    }
}

The above errors stating that name is a read-only property; however, if you move the declaration of name into the class, then everything works again.

This looks right to me. if you want to assign to it from the constructor define it in the class.

That's fair, though it is a strange inconsistency.
@nbransby, what is the scenario you had in mind?

Well I had the interface defined in one shared project then different implementations in sub projects via declaration merging. Obviously I had to remove the readonly to get it to work but I wanted the objects to be immutable

@sandersn i assume we are missing one more getMergedSymbol call

@nbransby so the class is aware of all of its extensions in subprojects? Or is there a common core between subprojects that all share some common fields which are assumed to be assigned to?

Yes there is a common core that contains the interfaces and the subprojects define classes with the same name instead of implementing the interfaces. The advantage is not having to repeat the variables defined in the interface.
Although since I switched to modules this pattern does not work due to the restrictions on cross module merging. So I've switched to classes in the shared core being extended in the subprojects (from which I've encountered https://github.com/Microsoft/TypeScript/issues/7590)

It turns out that isReferenceToReadonlyEntity intentionally does not allow assignment to readonly properties from a different, merged declaration. I think this is correct. The constructor assignment exception shouldn't extend from an interface to a merged class.


[Bug] Memory leakage detected when IndexStoreService is enabled and TieredMessageStore's FileSegment is PosixFileSegment

Before Creating the Bug Report

[X] I found a bug, not just asking a question, which should be created in GitHub Discussions.

[X] I have searched the GitHub Issues and GitHub Discussions  of this repository and believe that this is not a duplicate.

[X] I have confirmed that this bug belongs to the current repository, not other repositories of RocketMQ.

Runtime platform environment
CentOS 7
RocketMQ version
branch: develop
JDK Version
OpenJDK 8
Describe the Bug

We deployed a rocketmq cluster with TieredStore and offload cold data to hdd with PosixFileSegment. After about 16 hours of producing messages, the shutdownHook was called and there is no other error log.

2024-04-11 09:57:22 INFO ShutdownHook - Shutdown hook was invoked, 1

By checking monitor, we found out that the direct memory usage kept on increasing in a stable rate,

which reminds us of checking if there is memory leak. Actually the increase rate is a direct clue -- about 570 MB each time,  exactly the size of a compacted IndexFile, which leads us to here at IndexStoreService.java:

public boolean doCompactThenUploadFile(IndexFile indexFile) {
        ...
        boolean result = flatAppendFile.commitAsync().join();
        ...
}

Add a watch of direct memory when debugging the UT doCompactionTest at IndexStoreServiceTest.java can prove that doCompactThenUploadFile may cause direct memory leak.

By diving in the code,  the root cause actually is inappropriate use of FileChannel at PosixFileSegment.java:

@Override
    @SuppressWarnings("ResultOfMethodCallIgnored")
    public CompletableFuture<Boolean> commit0(
        FileSegmentInputStream inputStream, long position, int length, boolean append) {

        return CompletableFuture.supplyAsync(() -> {
            try {
                byte[] byteArray = ByteStreams.toByteArray(inputStream);
                writeFileChannel.position(position);
                ByteBuffer buffer = ByteBuffer.wrap(byteArray);
                while (buffer.hasRemaining()) {
                    writeFileChannel.write(buffer);
                }
                writeFileChannel.force(true);
            } catch (Exception e) {
                return false;
            }
            return true;
        }, MessageStoreExecutor.getInstance().bufferCommitExecutor);
    }

The line writeFileChannel.write(buffer); would allocate a DirectByteBuffer whose size is same with buffer, which means in uploading of IndexFile, the DirectByteBuffer's size will be about 570MB. This DirectByteBuffer is an element of an array of ByteBuffer of a ThreadLocal variable in sun.nio.ch.Util.class:

private static class BufferCache {
    private ByteBuffer[] buffers;
    private int count;
    private int start;
    ...
}

private static ThreadLocal<BufferCache> bufferCache;

The actual process of writeFileChannel.write(buffer) is:

find a DirectByteBuffer which is big enough to store data of buffer, if there is not, allocate one;
write data from DirectByteBuffer to pagecache;
return the DirectByteBuffer to BufferCache and do not release it;

which means that the direct memory allocated would not be released until the thread is killed, which will never happen for teh thread runs commit0 is in a thread pool of MessageStoreExecutor.

Reproducing the bug with personal code is simple, writing a class like below, call FileWrite.run() and open jvisualvm, the increasing of direct buffer should be shown in Buffer Pools.

public class FileWrite {
    private FileChannel readFileChannel;
    private FileChannel writeFileChannel;
    private ExecutorService executor;
    private final String placeHolder = new String(new char[60000]).replace('\0', '1');

    public FileWrite(String path) throws Exception {
        this.readFileChannel = new RandomAccessFile(new File(path), "r").getChannel();
        this.writeFileChannel = new RandomAccessFile(new File(path), "rwd").getChannel();
        this.executor = new ThreadPoolExecutor(
                Math.max(16, Runtime.getRuntime().availableProcessors() * 4),
                Math.max(16, Runtime.getRuntime().availableProcessors() * 4),
                60 * 1000,
                TimeUnit.MILLISECONDS,
                new LinkedBlockingQueue<>(100));
    }

    public void run() throws InterruptedException {
        for (int i = 0;i < 100;++i) {
            writeAsync();
            Thread.sleep(5 * 1000)
        }
    }

    public CompletableFuture<Boolean> writeAsync() {
        return CompletableFuture.supplyAsync(this::write, executor);
    }
}

Atention that the size of thread pool will limit the increase of buffer usage, for BufferCache of each thread is reusable, the increase should stop as each thread in thread pool runs writeAsync() at least once.

For example, reduce the size of trhead pool to 4:

Steps to Reproduce
2 choices:

Deploy a rocketmq cluster with TieredStore and offload cold data to hdd with PosixFileSegment, keep on producing new messages and the direct memory usage should increase unlimitedly.
Or run IndexStoreServiceTest.doCompactionTest and keep an eye on the direct buffer usage.

What Did You Expect to See?
The direct memory usage should not increase unlimitedly.
What Did You See Instead?
The direct memory usage keeps on increasing until OOM or all threads in commitExecutor run commit0 at least once.
Additional Context
There might be some solution of this bug:

Reduce the size of bufferCommitExecutor;
Limit the size of BufferCache by config -Djdk.nio.maxCachedBufferSize;
Limit the size of inputStream in PosixFileSegment.commit0's parameter list;

Or we can introduce a DirectByteBuffer pool like TransientStorePool

ByteStreams.toByteArray() creates large array in memory and is unnecessary. It seems FileChannel.transferFrom() with NIO channel would be more efficient for writing streams to a file.

ByteStreams.toByteArray() creates large array in memory and is unnecessary. It seems FileChannel.transferFrom() with NIO channel would be more efficient for writing streams to a file.

Thx for reply, I tried FileChannel.transferFrom() but find out some unittest failed:
@Test
public void consumeQueueTest() throws ClassNotFoundException, NoSuchMethodException {
    ...
    ByteBuffer cqItem1 = fileSegment.read(initPosition, unitSize);
    Assert.assertEquals(baseOffset, cqItem1.getLong());  // failed
    ...
}

java.lang.AssertionError:
Expected :1000
Actual   :0

It turns out the cq data was not flush correctly by debugging, I wonder if the way I use FileChannel.transferFrom() is wrong:

ByteStreams.toByteArray() creates large array in memory and is unnecessary. It seems FileChannel.transferFrom() with NIO channel would be more efficient for writing streams to a file.

Thx for reply, I tried FileChannel.transferFrom() but find out some unittest failed:
@Test
public void consumeQueueTest() throws ClassNotFoundException, NoSuchMethodException {
    ...
    ByteBuffer cqItem1 = fileSegment.read(initPosition, unitSize);
    Assert.assertEquals(baseOffset, cqItem1.getLong());  // failed
    ...
}

java.lang.AssertionError:
Expected :1000
Actual   :0

It turns out the cq data was not flush correctly by debugging, I wonder if the way I use FileChannel.transferFrom() is wrong: Before:
                byte[] byteArray = ByteStreams.toByteArray(inputStream);
                writeFileChannel.position(position);
                ByteBuffer buffer = ByteBuffer.wrap(byteArray);
                while (buffer.hasRemaining()) {
                    writeFileChannel.write(buffer);
                }
                writeFileChannel.force(true);

After:
                ReadableByteChannel readableByteChannel = Channels.newChannel(inputStream);
                writeFileChannel.transferFrom(readableByteChannel, position, length);
                writeFileChannel.force(true);

It seems the modification is not equivalent. writeFileChannel.position(position) has side effect, making writeFileChannel written from position.  But writeFileChannel.transferFrom() exits when position is larger than the size of writeFileChannel.
You might extend the file of writeFileChannel to position and also check if the transferred bytes is equal to length.

ByteStreams.toByteArray() creates large array in memory and is unnecessary. It seems FileChannel.transferFrom() with NIO channel would be more efficient for writing streams to a file.

Thx for reply, I tried FileChannel.transferFrom() but find out some unittest failed:
@Test
public void consumeQueueTest() throws ClassNotFoundException, NoSuchMethodException {
    ...
    ByteBuffer cqItem1 = fileSegment.read(initPosition, unitSize);
    Assert.assertEquals(baseOffset, cqItem1.getLong());  // failed
    ...
}

java.lang.AssertionError:
Expected :1000
Actual   :0

It turns out the cq data was not flush correctly by debugging, I wonder if the way I use FileChannel.transferFrom() is wrong: Before:
                byte[] byteArray = ByteStreams.toByteArray(inputStream);
                writeFileChannel.position(position);
                ByteBuffer buffer = ByteBuffer.wrap(byteArray);
                while (buffer.hasRemaining()) {
                    writeFileChannel.write(buffer);
                }
                writeFileChannel.force(true);

After:
                ReadableByteChannel readableByteChannel = Channels.newChannel(inputStream);
                writeFileChannel.transferFrom(readableByteChannel, position, length);
                writeFileChannel.force(true);

It seems the modification is not equivalent. writeFileChannel.position(position) has side effect, making writeFileChannel written from position. But writeFileChannel.transferFrom() exits when position is larger than the size of writeFileChannel.
You might extend the file of writeFileChannel to position and also check if the transferred bytes is equal to length.

Agree with you. But then I was wondering if the failed unittest necessary, it seems acting like:

set a empty file's commitPosition to 100
append some mocked ConsumeQueue Buffer to it
commitPosition is larger than the size of writeFileChannel, commit failed

Is it acceptable to remove step 1 in consumeQueueTest? @lizhimins 'cause the scenario seems nonexistent in production.
Furthermore, FileChannel.read() has the same problem with FileChannel.write(), but to return a ByteBuffer, calling it looks like inevitable. I didn't find out how to use FileChannel.transferTo() to replace it completely. I'd like to use MappedByteBuffer as a alternative.

@bxfjb If you want to extend the size of writeFileChannel to make FileChannel.transferTo() successful, you may try writing a byte at position - 1:
writeFileChannel.write(ByteBuffer.wrap(new byte[]{0}), position - 1);

@bxfjb If you want to extend the size of writeFileChannel to make FileChannel.transferTo() successful, you may try writing a byte at position - 1:
writeFileChannel.write(ByteBuffer.wrap(new byte[]{0}), position - 1);

Eventually I decide to use MappedByteBuffer for FileChannel.transferFrom()'s worse IO preformance in tests, could you pls review the #8036?

The default implementation in the current code is DefaultMetadataStore and PosixFileSegment.
PosixFileSegment only as test cases and validation purposes. We used metadata management based on Rocksdb implementation and FileSegment based on object storage implementation in out production. In order to achieve higher performance and lower storage costs, this part of the code will contribute in future pr.

Adding a new file type for index looks good

@bxfjb If you want to extend the size of writeFileChannel to make FileChannel.transferTo() successful, you may try writing a byte at position - 1:
writeFileChannel.write(ByteBuffer.wrap(new byte[]{0}), position - 1);

Eventually I decide to use MappedByteBuffer for FileChannel.transferFrom()'s worse IO preformance in tests, could you pls review the #8036?

In general mmap has a greater impact on page cache and may cause resource contention with normal commit log. We need to evaluate the impact on overall performance carefully.

@bxfjb If you want to extend the size of writeFileChannel to make FileChannel.transferTo() successful, you may try writing a byte at position - 1:
writeFileChannel.write(ByteBuffer.wrap(new byte[]{0}), position - 1);

Eventually I decide to use MappedByteBuffer for FileChannel.transferFrom()'s worse IO preformance in tests, could you pls review the #8036?

In general mmap has a greater impact on page cache and may cause resource contention with normal commit log. We need to evaluate the impact on overall performance carefully.

If the target storage is based on the POSIX protocol, using directIO is a good choice.

@bxfjb If you want to extend the size of writeFileChannel to make FileChannel.transferTo() successful, you may try writing a byte at position - 1:
writeFileChannel.write(ByteBuffer.wrap(new byte[]{0}), position - 1);

Eventually I decide to use MappedByteBuffer for FileChannel.transferFrom()'s worse IO preformance in tests, could you pls review the #8036?

In general mmap has a greater impact on page cache and may cause resource contention with normal commit log. We need to evaluate the impact on overall performance carefully.

Got it, I am doing stress tests to reveal the impact as possible as I can. So far it looks good under about 60k producer TPS.

@bxfjb If you want to extend the size of writeFileChannel to make FileChannel.transferTo() successful, you may try writing a byte at position - 1:
writeFileChannel.write(ByteBuffer.wrap(new byte[]{0}), position - 1);

Eventually I decide to use MappedByteBuffer for FileChannel.transferFrom()'s worse IO preformance in tests, could you pls review the #8036?

In general mmap has a greater impact on page cache and may cause resource contention with normal commit log. We need to evaluate the impact on overall performance carefully.

If the target storage is based on the POSIX protocol, using directIO is a good choice.

I chose MappedByteBuffer cuz that:

FileChannel.write()/read() is banned for memory leak
FileChannel.transferFrom()'s performance is unacceptable low, more specifically, about 200 messages are written per second

I wonder what do you mean about  direct IO, does it mean DirectByteBuffer or something?

对于 SSD 转 HDD 场景，当前设计中 PosixFileSegment 主要是为了测试场景，可以改进。如果用 MappedByteBuffer，把数据从 CommitLog 复制到 FileSegment，也会有 pagecache 浪费内存的问题。至于如何实现，可以搜索 Java directIO lib
For copy data from SSD to HDD scenario, the PosixFileSegment in the current design is mainly for testing scenarios and can be improved. If you use MappedByteBuffer to copy data from CommitLog to FileSegment, there will also be a problem of pagecache wasting memory. We can use Java directIO lib to slove this problem


Added an ability to specify localdb name.

Fix for #80
This fix allows to run CCCheck on the machine with only VS2015 installed.
Please note, that new version (i.e. (LocalDb)\MSSqlLocalDb) is also available on machine with VS2013 installed. So minor performance degradation would be only for computers with ONLY VS2012 and VS2010 only.
We can enhance this strategy in the future, but this fix unblocks us to release VS2015 support (without such kind of fix CCCheck will fail unless the user manually change the options and disables caching).

For our purpose it would be totally sufficient to look at HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SQL Server Local DB\Installed Versions to decide which name we can use. So we would not need that library at all.

Superseded by #97.


entry_points install path is now 'bin' instead of 'lib' for setup.py installs

I believe a regression has been introduced with the merging of https://github.com/colcon/colcon-core/commit/5203c997af9a6d068908ac800506f3de23122376 (or I'm holding something wrong).
With colcon-core at 0.9.0, the test_py_package fails to execute it's entry_points scripts.
I created a workspace, added the content of https://github.com/colcon/colcon-bundle/tree/master/integration/ros2_workspace/src/test_py_package to the workspace and built.
When executing the run_py_package_tests script, I get:
# ros2 run test_py_package run_py_package_tests
No executable found

The install space for the workspace looks like this:
# tree -L 3
.
├── COLCON_IGNORE
├── _local_setup_util_ps1.py
├── _local_setup_util_sh.py
├── local_setup.bash
├── local_setup.ps1
├── local_setup.sh
├── local_setup.zsh
├── setup.bash
├── setup.ps1
├── setup.sh
├── setup.zsh
└── test_py_package
    ├── bin
    │   └── run_py_package_tests
    ├── lib
    │   └── python3.8
    └── share
        ├── ament_index
        ├── colcon-core
        └── test_py_package

8 directories, 12 files

Previously, with colcon-core at 0.8.3, the run_py_package_tests executable was located in lib as a cousin to the python3.8 directory.
Downstream, this is causing other issues with things like launch files, with errors like "libexec directory '/path/to/workspace/
install/pkg_name/lib/pkg_name' does not exist"

a workaround that seems to work for me is this
colcon build --symlink-install
colcon build

this creates the necessary directory in `install/<package_name>/lib/<package_name>

a workaround that seems to work for me is this
colcon build --symlink-install
colcon build

this creates the necessary directory in install/<package_name>/lib/<package_name>

Was recently bit by this on our CICD pipeline as colcon-core 0.8.3 upgraded. Can confirm this work around is great.

Faced the same issue.
In the setup.cfg I replaced
[develop]
script-dir=$base/lib/my_package
[install]
install-scripts=$base/lib/my_package

By
[develop]
script_dir=$base/lib/my_package
[install]
install_scripts=$base/lib/my_package

(so replacing dashes from variable's name by underscores)

Hi, I'm still running into this problem.
My setup is as follows: I install colcon and dependencies into a Python 3.12 venv managed by uv, then run colcon build there. It looks like this:
ARG OVERLAY_WS=/home/rosdev/ros_ws_install
RUN mkdir -p $OVERLAY_WS
WORKDIR $OVERLAY_WS

RUN ~/.local/bin/uv venv --python 3.12 venv
RUN . ./venv/bin/activate && \
    ~/.local/bin/uv pip install git+https://github.com/colcon/colcon-cargo.git && \
    ~/.local/bin/uv pip install git+https://github.com/colcon/colcon-ros-cargo.git && \
    ~/.local/bin/uv pip install colcon-common-extensions

RUN . ./venv/bin/activate && \
    ~/.local/bin/uv pip install empy==3.3.4 # Workaround https://github.com/ros2/rosidl/issues/779 https://robotics.stackexchange.com/questions/109912/problem-with-building-nav2-on-ros-humble
RUN . ./venv/bin/activate && \
    ~/.local/bin/uv pip install lark numpy Cython

RUN --mount=type=bind,target=$OVERLAY_WS/src,source=./src \
    . /opt/ros/humble/setup.sh && \
    . ./venv/bin/activate && \
    touch venv/COLCON_IGNORE && \
    colcon build  \
      --event-handlers console_direct+ \
      --packages-up-to my_package

The console_scripts from my_package end up in install/lib/my_package/bin instead of lib.
The fix from #530 looks promising, and it works if I replace the underscores in my setup.cfg with dashes, looking like this:
[develop]
script-dir=$base/lib/my_package
[install]
install-scripts=$base/lib/my_package

I get the warning about script-dir being unsupported:
#20 15.54 running install_scripts
#20 15.93 Installing my_node script to /home/rosdev/ros_ws_install/install/my_package/lib/my_package
#20 16.14 !!
#20 16.14
#20 16.14         ********************************************************************************
#20 16.14         Usage of dash-separated 'script-dir' will not be supported in future
#20 16.14         versions. Please use the underscore name 'script_dir' instead.
#20 16.14
#20 16.14         This deprecation is overdue, please update your project and remove deprecated
#20 16.14         calls to avoid build errors in the future.
#20 16.14
#20 16.14         See https://setuptools.pypa.io/en/latest/userguide/declarative_config.html for details.
#20 16.14         ********************************************************************************
#20 16.14
#20 16.14 !!
#20 16.14   opt = self.warn_dash_deprecation(opt, section)
#20 16.14 /home/rosdev/ros_ws_install/venv/lib/python3.12/site-packages/setuptools/_distutils/dist.py:261: UserWarning: Unknown distribution option: 'tests_require'
#20 16.14   warnings.warn(msg)

but it does install the script into the correct location.
@cottsay do you think it makes sense to also look for the install_scripts key in setup.cfg, or is there a better solution?
Note: ros2 pkg create --build-type ament_python creates a setup.cfg with script_dir and install_scripts keys.


Upgrade SDk to version 0.3.3

Features

SDK version is now 0.3.3
Removed session object and removed saving of KeyPair in session
Now purely using the publickey for all Horizon calls
This also fixes the bug that says no network connection on the first time.

@Daniel-Wang Small but a great PR! Good riddance of the keyPair 👍


DocsArchive-<IP_ADDRESS>Auto archive content

Auto archive content from https://github.com/OPS-E2E-PPE/docs-archive-test-source

Docs Build status updates of commit c727926:
:x: Validation status: errors
Please follow instructions here which may help to resolve issue.

File
Status
Preview URL
Details

:x:Error

Details

[Error]

Cannot sync git repo to specified commit because branch Release_Archive_master_2020-02-11-02-54-14 has been deleted or has been force pushed: fatal: couldn't find remote ref refs/heads/Release_Archive_master_2020-02-11-02-54-14

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


convert-hf-to-gguf.py should have underscores.

In the wiki (https://github.com/unslothai/unsloth/wiki) for manually saving to gguf it has the line:
python llama.cpp/convert-hf-to-gguf.py FOLDER --outfile OUTPUT --outtype f16
This should be:
python llama.cpp/convert_hf_to_gguf.py FOLDER --outfile OUTPUT --outtype f16
Thanks!

Oh yes!! Actually would you be interested in opening a PR and editing the Readme file, and then I'll copy paste your edits to the Wiki :)


Missing support for disassembling vendor specific extensions (TimeAMD).

Related issue: https://github.com/google/shaderc/issues/325
I looked around a bit and it seems related to spvOpcodeString not having it's tables set up for vendor extensions.
The offending shader compiled with shaderc
#version 450

#extension GL_ARB_separate_shader_objects : enable
#extension GL_ARB_shading_language_420pack : enable
#extension GL_ARB_gpu_shader_int64 : enable
#extension GL_AMD_gcn_shader : enable

void main()
{
	uint64_t v = timeAMD();
}

Hi David thanks for looking at this & the quick turnaround. After the shaderc issue I also run into this, when loading the SPIRV module with validation enabled:
SPIR-V module not valid: Invalid extended instruction import 'SPV_AMD_gcn_shader'
Does the above commit fix that as well?

SPIR-V module not valid: Invalid extended instruction import 'SPV_AMD_gcn_shader'

Does the above commit fix that as well?

Yes.  The validator recognizes the extension, but does not check the specific rules.  For the example you gave, the SPIR-V module passes validation silently.


Feature/custom translation en ar

Test merging from a branch to my forked branch

Coverage remained the same at 96.044% when pulling 536e02335269f10fe43012eb96c071aec617c12f on infinitecipher:feature/custom-translation-en-ar into 624ad01afbc86fa15b4e652406f3bdcd01f36e00 on Tivix:master.

Hi,
as a user of rest-auth, thanks for the contribution!
This repo is not maintained anymore, so the development moved to dj-rest-auth. (reference: https://github.com/Tivix/django-rest-auth/issues/568)
It may be best, if you move this PR there. (and upgrade to using dj_rest_auth)
new repo link: https://github.com/jazzband/dj-rest-auth (I'm not the upkeeper of that repo, it just makes sense for me to help you merge your PR)
Many Thanks,
Barney


Bicop factory

Warning: This PR also contains all changes suggested in #9 which should be merged first.
Here's my implementation of a factory for the Bicop class. It returns a std::shared_ptr<Bicop> (typedef'd as Bicop_ptr).

Well, for this one, I must admit that the factory implemented is not at all what I was thinking about. In my opinion, the create method should rather be called select and "mimic" the behaviour of BiCopSelect from the Vinecopula package. As such, I was thinking about a factory whose arguments would be data, familyset, selectioncrit, rotations, presel and method. I will implement such a factory and make a commit in this branch, and we can further discuss this before merging anything to the master branch.

OK
Then why about writing the typedef inside the class declaration?
OK
OK


SQS: Sent messages might not get picked up by ReceiveMessages on .NET Core running on Linux

Under .NET Core with Linux messages sent by the SQS client might not be picked up by ReceiveMessages  even though the messages are available in the queue, when the following factors are given:

Multiple clients are used in the same application domain
ReceiveMessages is done concurrently

Expected Behavior
Messages sent to the queue should be picked up by the ReceiveMessages call as soon as possible. Execute https://github.com/danielmarbach/SqsLinuxRepro under .NET Core in Windows and the following should happen:
PS C:\p\SqsLinuxRepro\SqsRepro> dotnet run
16/11/2017 10:43:35 (0) - Receiving
16/11/2017 10:43:35 (1) - Receiving
16/11/2017 10:43:37 (Main) - Sending attempt 1 1
16/11/2017 10:43:37 (1) - Received 1 / OK / 884
16/11/2017 10:43:37 (1) - deleting 16/11/2017 10:43:37 attempt 1 / 1
16/11/2017 10:43:37 (1) - deleted 16/11/2017 10:43:37 attempt 1 / 1
16/11/2017 10:43:37 (1) - Receiving
16/11/2017 10:43:38 (Main) - Sent attempt 1 1
16/11/2017 10:43:45 (0) - cancelled
16/11/2017 10:43:45 (1) - cancelled
16/11/2017 10:43:45 (0) - Receiving
16/11/2017 10:43:45 (1) - Receiving
16/11/2017 10:43:47 (Main) - Sending attempt 2 1
16/11/2017 10:43:47 (0) - Received 1 / OK / 884
16/11/2017 10:43:47 (0) - deleting 16/11/2017 10:43:47 attempt 2 / 1
16/11/2017 10:43:47 (0) - deleted 16/11/2017 10:43:47 attempt 2 / 1
16/11/2017 10:43:47 (0) - Receiving
16/11/2017 10:43:48 (Main) - Sent attempt 2 1
16/11/2017 10:43:55 (0) - cancelled
16/11/2017 10:43:55 (1) - cancelled
16/11/2017 10:43:55 (0) - Receiving
16/11/2017 10:43:55 (1) - Receiving
16/11/2017 10:43:57 (Main) - Sending attempt 3 1
16/11/2017 10:43:57 (1) - Received 1 / OK / 884
16/11/2017 10:43:57 (1) - deleting 16/11/2017 10:43:57 attempt 3 / 1
16/11/2017 10:43:58 (1) - deleted 16/11/2017 10:43:57 attempt 3 / 1
16/11/2017 10:43:58 (1) - Receiving
16/11/2017 10:43:58 (Main) - Sent attempt 3 1
16/11/2017 10:44:05 (1) - cancelled
16/11/2017 10:44:05 (0) - cancelled
16/11/2017 10:44:05 (0) - Receiving
16/11/2017 10:44:05 (1) - Receiving
16/11/2017 10:44:07 (Main) - Sending attempt 4 1
16/11/2017 10:44:08 (1) - Received 1 / OK / 884
16/11/2017 10:44:08 (1) - deleting 16/11/2017 10:44:07 attempt 4 / 1
16/11/2017 10:44:08 (1) - deleted 16/11/2017 10:44:07 attempt 4 / 1
16/11/2017 10:44:08 (1) - Receiving
16/11/2017 10:44:09 (Main) - Sent attempt 4 1
16/11/2017 10:44:15 (1) - cancelled
16/11/2017 10:44:15 (0) - cancelled

Result: all messages consumed

Current Behavior
One or multiple messages are in flight during the execution time but never picked up the the SDK and propagated to the user code, more information see Steps to reproduce.
Possible Solution
Steps to Reproduce (for bugs)
Execute https://github.com/danielmarbach/SqsLinuxRepro under .NET Core on Linux with a concurrency higher than 2 and observe a similar pattern to:
11/16/17 10:37:06 AM (0) - Receiving
11/16/17 10:37:06 AM (1) - Receiving
11/16/17 10:37:08 AM (Main) - Sending attempt 1 1
11/16/17 10:37:08 AM (0) - Received 1 / OK / 885
11/16/17 10:37:08 AM (0) - deleting 11/16/17 10:37:08 AM attempt 1 / 1
11/16/17 10:37:08 AM (0) - deleted 11/16/17 10:37:08 AM attempt 1 / 1
11/16/17 10:37:08 AM (0) - Receiving
11/16/17 10:37:09 AM (Main) - Sent attempt 1 1
11/16/17 10:37:16 AM (1) - cancelled
11/16/17 10:37:16 AM (0) - cancelled
11/16/17 10:37:16 AM (0) - Receiving
11/16/17 10:37:16 AM (1) - Receiving
11/16/17 10:37:18 AM (Main) - Sending attempt 2 1
11/16/17 10:37:19 AM (Main) - Sent attempt 2 1
11/16/17 10:37:26 AM (0) - cancelled
11/16/17 10:37:26 AM (1) - cancelled
11/16/17 10:37:26 AM (0) - Receiving
11/16/17 10:37:26 AM (1) - Receiving
11/16/17 10:37:28 AM (Main) - Sending attempt 3 1
11/16/17 10:37:28 AM (1) - Received 1 / OK / 885
11/16/17 10:37:28 AM (1) - deleting 11/16/17 10:37:28 AM attempt 3 / 1
11/16/17 10:37:28 AM (1) - deleted 11/16/17 10:37:28 AM attempt 3 / 1
11/16/17 10:37:28 AM (1) - Receiving
11/16/17 10:37:29 AM (Main) - Sent attempt 3 1
11/16/17 10:37:36 AM (0) - cancelled
11/16/17 10:37:36 AM (1) - cancelled
11/16/17 10:37:36 AM (0) - Receiving
11/16/17 10:37:36 AM (1) - Receiving
11/16/17 10:37:38 AM (Main) - Sending attempt 4 1
11/16/17 10:37:38 AM (1) - Received 1 / OK / 885
11/16/17 10:37:38 AM (1) - deleting 11/16/17 10:37:38 AM attempt 4 / 1
11/16/17 10:37:38 AM (1) - deleted 11/16/17 10:37:38 AM attempt 4 / 1
11/16/17 10:37:38 AM (1) - Receiving
11/16/17 10:37:39 AM (Main) - Sent attempt 4 1
11/16/17 10:37:46 AM (0) - cancelled
11/16/17 10:37:46 AM (1) - cancelled

As we can see sent attempt 2 was never received although the receive time is high enough in the repro to actually pick it up.

but the queue browser shows multiple receive attempts.
Context
At Particular Software we are preparing a Release candidate of a .NET Core SQS transport for NServiceBus and we detected this on our CI environment. Multiple of our tests failed under .NET Core under Linux only. This is currently a blocker for us.
Your Environment

AWSSDK.Core version used: 3.3.19
Service assembly and version used: <IP_ADDRESS>
Operating System and version: Windows 10 lates and Ubuntu 16.04
Visual Studio version: 2017.5.4
Targeted .NET platform: .NET Core 2.0.2 and 2.0.3

.NET Core Info

.NET Core version used for development:
.NET Core version installed in the environment where application runs:
Output of dotnet --info:

.NET Command Line Tools (2.0.0)

Product Information:
 Version:            2.0.0
 Commit SHA-1 hash:  cdcd1928c9

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.15063
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\2.0.0\

Microsoft .NET Core Shared Framework Host

  Version  : 2.0.3
  Build    : a9190d4a75f4a982ae4b4fa8d1a24526566c69df

.NET Command Line Tools (2.0.3)

Product Information:
 Version:            2.0.3
 Commit SHA-1 hash:  eb1d5ee318

Runtime Environment:
 OS Name:     ubuntu
 OS Version:  16.04
 OS Platform: Linux
 RID:         ubuntu.16.04-x64
 Base Path:   /usr/share/dotnet/sdk/2.0.3/

Microsoft .NET Core Shared Framework Host

  Version  : 2.0.3
  Build    : a9190d4a75f4a982ae4b4fa8d1a24526566c69df

Contents of project.json/project.csproj:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>netcoreapp2.0</TargetFramework>
    <CopyLocalLockFileAssemblies>true</CopyLocalLockFileAssemblies>
    <LangVersion>7.1</LangVersion>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="AWSSDK.SQS" Version="<IP_ADDRESS>" />
  </ItemGroup>

</Project>

Btw. the repro sample uses the http URI so that wireshark can be attached. I have a full wireshark capture of a test but since it contains my unencrypted credentials I won't attach it here and it can be easily recaptured with the repro if required.

@danielmarbach , thanks for the detailed report and repro. We'll investigate this and get back to you.

@gokarnm let me know if you need help or additional information

Some more information.
I created two more branches. One is https://github.com/danielmarbach/SqsLinuxRepro/tree/client-per-receive which creates a client per concurrent receive loop. The issue still occurs
Another one is https://github.com/danielmarbach/SqsLinuxRepro/tree/multiple-starts-bash where I start the same program with just one receive loop multiple times with a bash script. The issue still occurs.

I was able to narrow it down
if you enable the following in the repro
AWSConfigs.LoggingConfig.LogTo = LoggingOptions.Console;

you can observe
AmazonSQSClient 45|2017-12-19T16:54:41.797Z|ERROR|TaskCanceledException making request ReceiveMessageRequest to http://sqs.eu-central-1.amazonaws.com/. Attempt 1. --> System.Threading.Tasks.TaskCanceledException: A task was canceled.
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
   at System.Net.Http.HttpClient.<FinishSendAsyncUnbuffered>d__59.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Amazon.Runtime.HttpWebRequestMessage.<GetResponseAsync>d__20.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Amazon.Runtime.Internal.HttpHandler`1.<InvokeAsync>d__9`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Amazon.Runtime.Internal.Unmarshaller.<InvokeAsync>d__3`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Amazon.SQS.Internal.ValidationResponseHandler.<InvokeAsync>d__1`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Amazon.Runtime.Internal.ErrorHandler.<InvokeAsync>d__5`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Amazon.Runtime.Internal.CallbackHandler.<InvokeAsync>d__9`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Amazon.Runtime.Internal.CredentialsRetriever.<InvokeAsync>d__7`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Amazon.Runtime.Internal.RetryHandler.<InvokeAsync>d__10`1.MoveNext()

which seems to be related to
https://github.com/dotnet/corefx/issues/25800

Sorry for the delayed response.  We weren't able to reproduce this on Mac or Windows.  We are pretty confident that this is a .NET Core framework issue--probably with the linux httpclient it uses.
We've discussed a few workaround that we can provide in the SDK until a new HttpClient is released in .NET Core 2.1.  Once we have a workaround solution that is reasonable, we'll ping back on this thread.

@sstevenkang I'm not confident anymore that it is only related to .NET Core. I was able to repro the same problem on Windows consistently on multiple machines with FIFO queues. I updated the repro solution https://github.com/danielmarbach/SqsLinuxRepro/tree/fifo-queues // @seanfeldman

Running code from fifo-queues branch on Windows
2018-03-05 22:45:35 (0) - Receiving
2018-03-05 22:45:35 (1) - Receiving
2018-03-05 22:45:35 (0) - Received 2 / OK / 1290
2018-03-05 22:45:35 (0) - deleting 2018-03-05 22:23:56 attempt 4 / 1
2018-03-05 22:45:35 (0) - deleting 2018-03-05 22:23:45 attempt 3 / 1
2018-03-05 22:45:35 (0) - deleted 2018-03-05 22:23:56 attempt 4 / 1
2018-03-05 22:45:36 (0) - deleted 2018-03-05 22:23:45 attempt 3 / 1
2018-03-05 22:45:36 (0) - Receiving
2018-03-05 22:45:37 (Main) - Sending attempt 1 1
2018-03-05 22:45:37 (1) - Received 1 / OK / 776
2018-03-05 22:45:37 (1) - deleting 2018-03-05 22:45:37 attempt 1 / 1
2018-03-05 22:45:37 (Main) - Sent attempt 1 1
2018-03-05 22:45:38 (1) - deleted 2018-03-05 22:45:37 attempt 1 / 1
2018-03-05 22:45:38 (1) - Receiving
2018-03-05 22:45:46 (0) - cancelled
2018-03-05 22:45:46 (1) - cancelled
2018-03-05 22:45:47 (0) - Receiving
2018-03-05 22:45:47 (1) - Receiving
2018-03-05 22:45:49 (Main) - Sending attempt 2 1
2018-03-05 22:45:49 (Main) - Sent attempt 2 1
2018-03-05 22:45:58 (0) - cancelled
2018-03-05 22:45:58 (1) - cancelled
2018-03-05 22:45:59 (0) - Receiving
2018-03-05 22:45:59 (1) - Receiving
2018-03-05 22:46:01 (Main) - Sending attempt 3 1
2018-03-05 22:46:01 (Main) - Sent attempt 3 1
2018-03-05 22:46:10 (1) - cancelled
2018-03-05 22:46:10 (0) - cancelled
2018-03-05 22:46:10 (0) - Receiving
2018-03-05 22:46:10 (1) - Receiving
2018-03-05 22:46:12 (Main) - Sending attempt 4 1
2018-03-05 22:46:13 (Main) - Sent attempt 4 1
2018-03-05 22:46:19 (0) - Received 1 / OK / 776
2018-03-05 22:46:19 (0) - deleting 2018-03-05 22:45:49 attempt 2 / 1
2018-03-05 22:46:20 (0) - deleted 2018-03-05 22:45:49 attempt 2 / 1
2018-03-05 22:46:20 (0) - Receiving
2018-03-05 22:46:22 (1) - cancelled
2018-03-05 22:46:22 (0) - cancelled

The outcome is that the messages are not picked up, see

3/5/2018 11:51:34 PM (0) - Receiving
3/5/2018 11:51:34 PM (1) - Receiving
3/5/2018 11:51:34 PM (0) - Received 3 / OK / 1747
3/5/2018 11:51:34 PM (0) - deleting 3/5/2018 11:43:58 PM attempt 4 / 1
3/5/2018 11:51:34 PM (0) - deleting 3/5/2018 11:42:41 PM attempt 3 / 1
3/5/2018 11:51:34 PM (0) - deleting 3/5/2018 11:43:46 PM attempt 3 / 1
3/5/2018 11:51:34 PM (0) - deleted 3/5/2018 11:43:58 PM attempt 4 / 1
3/5/2018 11:51:35 PM (0) - deleted 3/5/2018 11:42:41 PM attempt 3 / 1
3/5/2018 11:51:35 PM (0) - deleted 3/5/2018 11:43:46 PM attempt 3 / 1
3/5/2018 11:51:35 PM (0) - Receiving
3/5/2018 11:51:36 PM (Main) - Sending attempt 1 1
3/5/2018 11:51:36 PM (0) - Received 1 / OK / 757
3/5/2018 11:51:36 PM (0) - deleting 3/5/2018 11:51:36 PM attempt 1 / 1
3/5/2018 11:51:36 PM (Main) - Sent attempt 1 1
3/5/2018 11:51:36 PM (0) - deleted 3/5/2018 11:51:36 PM attempt 1 / 1
3/5/2018 11:51:36 PM (0) - Receiving
3/5/2018 11:51:44 PM (1) - cancelled
3/5/2018 11:51:44 PM (0) - cancelled
3/5/2018 11:51:46 PM (0) - Receiving
3/5/2018 11:51:46 PM (1) - Receiving
3/5/2018 11:51:48 PM (Main) - Sending attempt 2 1
3/5/2018 11:51:49 PM (Main) - Sent attempt 2 1
3/5/2018 11:51:56 PM (1) - cancelled
3/5/2018 11:51:56 PM (0) - cancelled
3/5/2018 11:51:59 PM (0) - Receiving
3/5/2018 11:51:59 PM (1) - Receiving
3/5/2018 11:52:01 PM (Main) - Sending attempt 3 1
3/5/2018 11:52:02 PM (Main) - Sent attempt 3 1
3/5/2018 11:52:09 PM (1) - cancelled
3/5/2018 11:52:09 PM (0) - cancelled
3/5/2018 11:52:10 PM (0) - Receiving
3/5/2018 11:52:10 PM (1) - Receiving
3/5/2018 11:52:12 PM (Main) - Sending attempt 4 1
3/5/2018 11:52:12 PM (Main) - Sent attempt 4 1
3/5/2018 11:52:20 PM (1) - cancelled
3/5/2018 11:52:20 PM (0) - cancelled

It also doesn't appear to be limited to .NET Core. If the target framework of the repro project is changed to net452, the same problem is demonstrated.

The effect is that the message stays in flight for 30 seconds which means they are actually processed but never handed over to the upper layers of the code.

Reproed it again on freshly setup cloud Linux machine with Ubuntu 17.10 and dotnet core version 2.1.4. And like @bording said it is also possible to repro this on full framework.
Guys seriously, this is a major blocker for us. Can we get some turn around here?

For the record NET452 target

@sstevenkang any updates?

We don't have a definitive solution yet but here's an update on our running theory:
When a message is received but not deleted you have to wait for the message visibility timeout to elapse before receiving it again.  That's the well known pattern.
That's what's happening here - but with a twist.  It looks like there's a race condition when the request gets canceled.  When the CancellationSource gets triggered control returns to the calling code but the socket isn't closed yet.
So even though a message is sent after the previous ReceiveMessage request has been canceled the socket from the previous receive is still open.  SQS replies over the old socket with the new message, considers it received, and subjects the message to the visibility timeout.
We're still looking into how the CancellationToken works within the HTTP client, and possible solutions to that.
One possible workaround is to call CancellationSource.CancelAfter with a value greater than the visibility timeout (which is 30 seconds by default).  Another (less desirable) is to put a Thread.Sleep after any OperationCanceledExceptions to allow the socket time to be closed.
@danielmarbach Will either of those work for you?  Can you tell us more about your use case and how this is causing issues?

@vellozzi thanks for coming back to us and investigating more.
We'll investigate if we can incorporate your proposal with the CancellationTokenSource. What I'm not yet understanding is why that would change anything. The token source we currently pass to the SDK is only canceled from our perspective when we shut down the application and that is not what we are doing when we see that happening in our environments. The Thread.Sleep option is not something I would like to add to our production code TBH.

Can you tell us more about your use case and how this is causing issues?

We detected this in our acceptance testing environment. We are developing an SQS adapter for NServiceBus. We recently added support for .NET Core. Based on that we run a set of acceptance tests on .NET Core on Windows, Linux and also the full framework on Windows. The default test timeout was 20 seconds and we suddenly started seing those tests failing. When we investigated we found out that in certain circumstances the messages that we sent remain not visible for the visibility timeout and thus the tests fail. We got worried because we figured that could also happen to our customers. We got also worried because we figured it might also indicate a potential message loss (which we haven't confirmed yet). For now, we decided to lift the restriction to not run on Linux because we think in the worst case messages will get delayed by 30 seconds (which is something we'll document on our docs site). But of course we'd love to see the SDK just work ;)

Are you saying that the CancellationSource is only cancelled on shutdown and your problem isn't occurring at shutdown?  If that's the case then I may not be seeing the same problem you are.  The issue I'm seeing only happens when the CancellationSource is cancelled.
Can you change the sample code to take the CancellationSource out of the equation and commit that to your GitHub repo?  I would do it but I want to make sure the sample accurately represents your production code.

I agree that there is no message loss and that the worst case is that you have to wait for the message visibility period to expire.

I extended the program a bit to include the received timespan by doing simple math between ApproximateFirstReceiveTimestamp and SentTimestamp. A run with the existing settings showed again the delay
daniel@DESKTOP-SUIHDT7:/mnt/c/p/SqsLinuxRepro/SqsRepro$ dotnet run SqsRepro.csproj -c Release
3/27/18 7:09:27 AM (0) - Receiving
3/27/18 7:09:28 AM (1) - Receiving
3/27/18 7:09:30 AM (Main) - Sending attempt 1 1
3/27/18 7:09:30 AM (Main) - Sent attempt 1 1
3/27/18 7:09:30 AM (1) - Received 1 / OK / 1131
Elapsed '00:00:00.0030000' (1) - to receive message with body '3/27/18 7:09:30 AM attempt 1 / 1'
3/27/18 7:09:30 AM (1) - deleting 3/27/18 7:09:30 AM attempt 1 / 1
3/27/18 7:09:30 AM (1) - deleted 3/27/18 7:09:30 AM attempt 1 / 1
3/27/18 7:09:30 AM (1) - Receiving
3/27/18 7:09:37 AM (1) - cancelled
3/27/18 7:09:37 AM (0) - cancelled
3/27/18 7:09:38 AM (0) - Receiving
3/27/18 7:09:38 AM (1) - Receiving
3/27/18 7:09:40 AM (Main) - Sending attempt 2 1
3/27/18 7:09:40 AM (Main) - Sent attempt 2 1
3/27/18 7:09:48 AM (0) - cancelled
3/27/18 7:09:48 AM (1) - cancelled
3/27/18 7:09:48 AM (0) - Receiving
3/27/18 7:09:48 AM (1) - Receiving
3/27/18 7:09:50 AM (Main) - Sending attempt 3 1
3/27/18 7:09:50 AM (Main) - Sent attempt 3 1
3/27/18 7:09:50 AM (1) - Received 1 / OK / 1131
Elapsed '00:00:00.0010000' (1) - to receive message with body '3/27/18 7:09:50 AM attempt 3 / 1'
3/27/18 7:09:50 AM (1) - deleting 3/27/18 7:09:50 AM attempt 3 / 1
3/27/18 7:09:50 AM (1) - deleted 3/27/18 7:09:50 AM attempt 3 / 1
3/27/18 7:09:50 AM (1) - Receiving
3/27/18 7:09:58 AM (0) - cancelled
3/27/18 7:09:58 AM (1) - cancelled
3/27/18 7:09:58 AM (0) - Receiving
3/27/18 7:09:58 AM (1) - Receiving
3/27/18 7:10:00 AM (Main) - Sending attempt 4 1
3/27/18 7:10:00 AM (Main) - Sent attempt 4 1
3/27/18 7:10:08 AM (0) - cancelled
3/27/18 7:10:08 AM (1) - cancelled

daniel@DESKTOP-SUIHDT7:/mnt/c/p/SqsLinuxRepro/SqsRepro$ dotnet run SqsRepro.csproj -c Release
3/27/18 7:11:20 AM (0) - Receiving
3/27/18 7:11:21 AM (1) - Receiving
3/27/18 7:11:21 AM (1) - Received 1 / OK / 1131
3/27/18 7:11:21 AM (0) - Received 1 / OK / 1131
Elapsed '00:01:28.4026165' (1) - to receive message with body '3/27/18 7:10:00 AM attempt 4 / 1'
3/27/18 7:11:21 AM (1) - deleting 3/27/18 7:10:00 AM attempt 4 / 1
Elapsed '00:01:48.6386170' (0) - to receive message with body '3/27/18 7:09:40 AM attempt 2 / 1'
3/27/18 7:11:21 AM (0) - deleting 3/27/18 7:09:40 AM attempt 2 / 1
3/27/18 7:11:21 AM (0) - deleted 3/27/18 7:09:40 AM attempt 2 / 1
3/27/18 7:11:21 AM (0) - Receiving
3/27/18 7:11:21 AM (1) - deleted 3/27/18 7:10:00 AM attempt 4 / 1
3/27/18 7:11:21 AM (1) - Receiving
3/27/18 7:11:23 AM (Main) - Sending attempt 1 1
3/27/18 7:11:23 AM (Main) - Sent attempt 1 1
3/27/18 7:11:23 AM (1) - Received 1 / OK / 1131
Elapsed '00:00:00.0030000' (1) - to receive message with body '3/27/18 7:11:23 AM attempt 1 / 1'
3/27/18 7:11:23 AM (1) - deleting 3/27/18 7:11:23 AM attempt 1 / 1
3/27/18 7:11:23 AM (1) - deleted 3/27/18 7:11:23 AM attempt 1 / 1
3/27/18 7:11:23 AM (1) - Receiving
3/27/18 7:11:30 AM (0) - cancelled
3/27/18 7:11:30 AM (1) - cancelled
3/27/18 7:11:30 AM (0) - Receiving
3/27/18 7:11:30 AM (1) - Receiving
3/27/18 7:11:32 AM (Main) - Sending attempt 2 1
3/27/18 7:11:32 AM (Main) - Sent attempt 2 1
3/27/18 7:11:40 AM (1) - cancelled
3/27/18 7:11:40 AM (0) - cancelled
3/27/18 7:11:40 AM (0) - Receiving
3/27/18 7:11:40 AM (1) - Receiving
3/27/18 7:11:42 AM (Main) - Sending attempt 3 1
3/27/18 7:11:43 AM (Main) - Sent attempt 3 1
3/27/18 7:11:43 AM (0) - Received 1 / OK / 1131
Elapsed '00:00:00.0020000' (0) - to receive message with body '3/27/18 7:11:42 AM attempt 3 / 1'
3/27/18 7:11:43 AM (0) - deleting 3/27/18 7:11:42 AM attempt 3 / 1
3/27/18 7:11:43 AM (0) - deleted 3/27/18 7:11:42 AM attempt 3 / 1
3/27/18 7:11:43 AM (0) - Receiving
3/27/18 7:11:50 AM (0) - cancelled
3/27/18 7:11:50 AM (1) - cancelled
3/27/18 7:11:51 AM (0) - Receiving
3/27/18 7:11:51 AM (1) - Receiving
3/27/18 7:11:53 AM (Main) - Sending attempt 4 1
3/27/18 7:11:53 AM (Main) - Sent attempt 4 1
3/27/18 7:12:01 AM (0) - cancelled
3/27/18 7:12:01 AM (1) - cancelled

Then I increased the timeout of the cancellation token source to 40 seconds which is more than the visibility timeout and still I get
daniel@DESKTOP-SUIHDT7:/mnt/c/p/SqsLinuxRepro/SqsRepro$ dotnet run SqsRepro.csproj -c Release
3/27/18 7:15:50 AM (0) - Receiving
3/27/18 7:15:51 AM (1) - Receiving
3/27/18 7:15:53 AM (Main) - Sending attempt 1 1
3/27/18 7:15:53 AM (Main) - Sent attempt 1 1
3/27/18 7:15:53 AM (0) - Received 1 / OK / 1131
Elapsed '00:00:00.0030000' (0) - to receive message with body '3/27/18 7:15:53 AM attempt 1 / 1'
3/27/18 7:15:53 AM (0) - deleting 3/27/18 7:15:53 AM attempt 1 / 1
3/27/18 7:15:53 AM (0) - deleted 3/27/18 7:15:53 AM attempt 1 / 1
3/27/18 7:15:53 AM (0) - Receiving
3/27/18 7:16:11 AM (1) - Received 0 / OK / 240
3/27/18 7:16:11 AM (1) - Receiving
3/27/18 7:16:13 AM (0) - Received 0 / OK / 240
3/27/18 7:16:13 AM (0) - Receiving
3/27/18 7:16:30 AM (1) - cancelled
3/27/18 7:16:30 AM (0) - cancelled
3/27/18 7:16:30 AM (0) - Receiving
3/27/18 7:16:30 AM (1) - Receiving
3/27/18 7:16:32 AM (Main) - Sending attempt 2 1
3/27/18 7:16:32 AM (Main) - Sent attempt 2 1
3/27/18 7:16:32 AM (0) - Received 1 / OK / 1131
Elapsed '00:00:00.0040000' (0) - to receive message with body '3/27/18 7:16:32 AM attempt 2 / 1'
3/27/18 7:16:32 AM (0) - deleting 3/27/18 7:16:32 AM attempt 2 / 1
3/27/18 7:16:32 AM (0) - deleted 3/27/18 7:16:32 AM attempt 2 / 1
3/27/18 7:16:32 AM (0) - Receiving
3/27/18 7:16:50 AM (1) - Received 0 / OK / 240
3/27/18 7:16:50 AM (1) - Receiving
3/27/18 7:16:53 AM (0) - Received 0 / OK / 240
3/27/18 7:16:53 AM (0) - Receiving
3/27/18 7:17:10 AM (1) - cancelled
3/27/18 7:17:10 AM (0) - cancelled
3/27/18 7:17:10 AM (0) - Receiving
3/27/18 7:17:10 AM (1) - Receiving
3/27/18 7:17:12 AM (Main) - Sending attempt 3 1
3/27/18 7:17:12 AM (Main) - Sent attempt 3 1
3/27/18 7:17:30 AM (0) - Received 0 / OK / 240
3/27/18 7:17:30 AM (0) - Receiving
3/27/18 7:17:30 AM (1) - Received 0 / OK / 240
3/27/18 7:17:30 AM (1) - Receiving
3/27/18 7:17:43 AM (1) - Received 1 / OK / 1131
Elapsed '00:00:37.4222483' (1) - to receive message with body '3/27/18 7:17:12 AM attempt 3 / 1'
3/27/18 7:17:43 AM (1) - deleting 3/27/18 7:17:12 AM attempt 3 / 1
3/27/18 7:17:43 AM (1) - deleted 3/27/18 7:17:12 AM attempt 3 / 1
3/27/18 7:17:43 AM (1) - Receiving
3/27/18 7:17:50 AM (0) - cancelled
3/27/18 7:17:50 AM (1) - cancelled
3/27/18 7:17:51 AM (0) - Receiving
3/27/18 7:17:51 AM (1) - Receiving
3/27/18 7:17:53 AM (Main) - Sending attempt 4 1
3/27/18 7:17:53 AM (Main) - Sent attempt 4 1
3/27/18 7:17:53 AM (0) - Received 1 / OK / 1131
Elapsed '00:00:00.0010000' (0) - to receive message with body '3/27/18 7:17:53 AM attempt 4 / 1'
3/27/18 7:17:53 AM (0) - deleting 3/27/18 7:17:53 AM attempt 4 / 1
3/27/18 7:17:53 AM (0) - deleted 3/27/18 7:17:53 AM attempt 4 / 1
3/27/18 7:17:53 AM (0) - Receiving
3/27/18 7:18:11 AM (1) - Received 0 / OK / 240
3/27/18 7:18:11 AM (1) - Receiving
3/27/18 7:18:13 AM (0) - Received 0 / OK / 240
3/27/18 7:18:13 AM (0) - Receiving

Sending attempt 3 was never received. Was only received in the next run of the repro (output not shown here)
When I change
                    var receiveResult = await sqsClient.ReceiveMessageAsync(new ReceiveMessageRequest
                        {
                            MaxNumberOfMessages = 10,
                            QueueUrl = queueUrl,
                            WaitTimeSeconds = 20,
                            AttributeNames = new List<string>
                            {
                                "SentTimestamp",
                                "ApproximateFirstReceiveTimestamp",
                                "ApproximateReceiveCount"
                            },
                        },
                        token).ConfigureAwait(false);

to
                    var receiveResult = await sqsClient.ReceiveMessageAsync(new ReceiveMessageRequest
                        {
                            MaxNumberOfMessages = 10,
                            QueueUrl = queueUrl,
                            WaitTimeSeconds = 20,
                            AttributeNames = new List<string>
                            {
                                "SentTimestamp",
                                "ApproximateFirstReceiveTimestamp",
                                "ApproximateReceiveCount"
                            },
                        },
                        CancellationToken.None).ConfigureAwait(false);

the issue disappears. It looks like when passing in the cancellation token (no matter what the timeout is) under some circumstances things leak which causes messages to be delayed for the visibility timeout period.
One could say that this is an acceptable workaround for us for now. So in the worst case we will delay the shutdown of an endpoint by 20 seconds.

I pushed an update to the repro sample

I had a chat @normj and based on the repro steps we provided we are fairly certain what we are seeing here is what @vellozzi already explained in https://github.com/aws/aws-sdk-net/issues/796#issuecomment-375494537. Since this is not a message loss scenario for us and in the worst case messages will be delayed for the duration of the visibility timeout I'm closing this issue.
We will discuss internally whether we want to pass in the token or not pass in the token.
Thanks all for chiming in!

@vellozzi Any luck with We're still looking into how the CancellationToken works within the HTTP client, and possible solutions to that.? I have the same issue with Rebus, which I fixed using the workaround you recommended - passing CancellationToken.None into sqsClient.ReceiveMessageAsync(). Cheers.

I found this: https://stackoverflow.com/a/45562557/379279 HttpClient cancellation doesn't kill underlying TCP call: HttpClient re-uses connections as much as possible, hence the reason why the connection is not closed on cancel. When canceling a request like that, the HttpClient will just stop sending/receiving data to/from the other end. It will not send anything to inform the other end that it was cancelled.
So I guess that's it - the solution is to stop using cancellations with HttpClient, and the workaround looks like the proper solution.


Make Grimoire always accessible (less scrolling)

This is similar to issue #126.
If a golden cookie comes up I want to have easy access to "Hand of Fate" in case I get a duration effect. Often I'm looking at stats though and I've scrolled down far, so in order to see my spells I have to close stats then scroll up to the Grimoire.
One way to do this might be an option to collapse certain building types on the building view of the center column so there's less content and no need to scroll. I don't need to see my buildings that don't have minigames.

I feel a little silly for not noticing this until now, but just zooming out on the page helps work around this.

That is a workaround, but it won't be obvious to everyone.  I do see it being annoying, but when there are mini games for all buildings, the same problem may somewhat still be there.  I'll think about it.

You can mute all the non-minigame buildings. Maybe that wasn't available when this bug was opened? It can probably be closed.

That's true, does sound like this can be closed now.  I'll take a closer look before deciding.


Fix link to bird site

Currently the link to bird site is in the format of:
https://twitter.com/i/web/status/12345

This link is registered but ironically unrecognised by Twitter for Android.
Twitter does not care about the user part of the URL, so:
https://twitter.com/{anything}/status/12345

... works.
This PR replaces the i/web part with _.

Huh, good find!
I don't use Twitter for Android and I wouldn't have guessed this URL formatting would make it misbehave in any way.
I was a bit concerned this could break Nitter links but they seem to behave in the exact same way:
https://nitter.net/_/status/12345

Nice unconventional solution 👍
Thank you for your contribution!


Open Payments Mandates API not mentioned in docs

I'm drafting a proposal which entails calling the Open Payments Mandates API, much as a web browser would:
https://docs.openpayments.dev/mandates
I don't see mention of the Mandates API in the Rafiki docs. Is this in the plans? If so, is there a timeframe?
More context in https://github.com/WICG/webmonetization/issues/195
Thanks!

@sharafian is this something Rafiki currently supports, or is this going to be part of Stage 3?

It would really help to get clarification on this soon, as the deadline for GftW proposals is near.
cc @uchibeke @cairin

Mandates is part of Stage 3 of Rafiki, and the timeline revolves around this year. If you're looking at this as part of a GftW proposal, I'd say that's a good assumption to make in your proposal.

Awesome, thanks! I'll leave this issue open in case that API should be mentioned in Rafiki docs.

We've deprecated the Mandates concept in favour of Grants in Open Payments.


Initial release

@JuliaRegistrator register

Registration pull request created: JuliaRegistries/General/71283
After the above pull request is merged, it is recommended that a tag is created on this repository for the registered package version.
This will be done automatically if the Julia TagBot GitHub Action is installed, or can be done manually through the github interface, or via:
git tag -a v0.1.0 -m "<description of version>" 455293d7cbaec62949757d8a541a0a3df39cb32b
git push origin v0.1.0

Error while trying to register: "Tag with name v0.1.0 already exists and points to a different commit"

@JuliaRegistrator register

Error while trying to register: "Tag with name v0.1.0 already exists and points to a different commit"


x/build: use Google Cloud for aix/ppc64 and linux/ppc64x builders

Google Cloud now supports POWER:
https://cloud.google.com/blog/products/gcp/ibm-power-systems-now-available-on-google-cloud
Maybe we can start running the AIX and linux/ppc64x builders on GCP (and scale VMs dynamically as needed) instead of OSUOSL (where we have a fixed and limited capacity).
/cc @laboger @trex58 @toothrot

I came to this bug from the now-locked https://github.com/golang/go/issues/34850 and, after following the discussion, I just wanted to mention one thing regarding OSes and ppc64 big-endian support. While all Linux distributions seem to have dropped support for this platform, FreeBSD/powerpc64 still exists and works. It might be an option too.

I came to this bug from the now-locked https://github.com/golang/go/issues/34850 and, after following the discussion, I just wanted to mention one thing regarding OSes and ppc64 big-endian support. While all Linux distributions seem to have dropped support for this platform, FreeBSD/powerpc64 still exists and works. It might be an option too.

FreeBSD/powerpc64 support would be nice


segmentation fault using tensorflow debugger

Please make sure that this is a bug. As per our GitHub Policy, we only address code/doc bugs, performance issues, feature requests and build/installation issues on GitHub. tag:bug_template
When i type run in tfdbg i get a segmentation fault.
System information

Have I written custom code?
yes i have.
OS Platform and Distribution:
kali linux rolling 2019 (see tf_env)
Python version:
Bazel version (if compiling from source):
GCC/Compiler version (if compiling from source):
not compiled from source
CUDA/cuDNN version:
i don't know what CUDA is but i don't have Nvidia GPU i think but radeon amd?
i think tensorflow runs on my cpu?
GPU model and memory:
lspci -vnn | grep VGA -A 12
01:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Thames [Radeon HD 7500M/7600M Series] [1002:6840] (prog-if 00 [VGA controller])
Subsystem: Toshiba America Info Systems Radeon HD 7670M [1179:fb22]
Flags: bus master, fast devsel, latency 0, IRQ 29
Memory at b0000000 (64-bit, prefetchable) [size=256M]
Memory at c0000000 (64-bit, non-prefetchable) [size=128K]
I/O ports at 3000 [size=256]
Expansion ROM at 000c0000 [disabled] [size=128K]
Capabilities: [50] Power Management version 3
Capabilities: [58] Express Legacy Endpoint, MSI 00
Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
Capabilities: [150] Advanced Error Reporting
Kernel driver in use: radeon

You can collect some of this information using our environment capture script
output of tf_env_collect.sh:
tf_env.txt
You can also obtain the TensorFlow version
with
python -c "import tensorflow as tf; print(tf.GIT_VERSION, tf.VERSION)"
i installed tensorflow in pycharm using pip (my projects uses the system interpreter)
tf.VERSION:
1.13.1
tf.GIT_VERSION:
b'v1.13.0-rc2-5-g6612da8'
Describe the current behavior
I first go to a terminal and type the following:
tensorboard --logdir=/root/files/Tensorboard/logs/ --debugger_port=6064
then in another terminal i type:
python3 NeuralNetwork.py --debug
i then see the run-start CLI like this.

then i type run and i get an segmentation fault.

Describe the expected behavior
the expected behavior i think is that the run command would not crash the debugger.
Code to reproduce the issue
Example.txt
Provide a reproducible test case that is the bare minimum necessary to generate the problem.
Other info / logs
Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached.

@hugo-boser This is a stale issue. Just checking whether it is still an issue for you? Can you please check recent TF/TB versions and let us know whether the issue was resolved for you. Thanks!


Add a DummyProtocol or similar that functions as a placeholder for users, docs

We want a DummyProtocol or similar that allows users to prototype alchemical networks without having to make any actual Protocol choices upfront. This would be especially useful right now as actual protocol implementations are being actively developed.

Is this not in the test suite already? I've been using that for development purposes where I needed a protocol to use (e.g., for our quickrun CLI input setup: https://github.com/OpenFreeEnergy/openfe/blob/main/openfecli/tests/dev/write_transformation_json.py

Indeed, we could use that one. I was thinking of this as an "official" placeholder that might be used in docs, examples, etc., in which a user is more focused on building an alchemical network than their choice of protocol. It should probably have a different name than the DummyProtocol given in the test suite to avoid confusion.
I can work with the existing DummyProtocol in the test suite for now; happy to close this issue if the above isn't of interest.


exceptions that aren't defined by the library should print stack trace

A 'base' exception should be created and every exception defined by the library should inherit from that base exception.
Any exceptions that are not an instance of that base exception should have a full stack trace printed out for visibility into the problem.

Probably a reason but when I try to use traceback.format_exc() I'm getting the following 'NoneType: None\n'.
Found traceback.print_tb(search_exception.__traceback__) from https://stackoverflow.com/a/16946886 and this seems to work, although it's not a full stack trace. I think it should be enough for now though..?


Warnings for useRole

I've been getting a bunch of warnings for useRole

Hi @tenthyoung again it will be pretty helpful if you can give a example of how you are using the components. Thanks! :)

Hi @tenthyoung again it will be pretty helpful if you can give a example of how you are using the components. Thanks! :)

Closing this issue because of inactivity.
Please reopen if you still require further assistance.


Migration job fails to trigger on local deploy

Describe the bug
When running pnpm saas deploy, the migration job fails to trigger as it can't resolve localstack.
This happens when AWS_ENDPOINT_URL=http://localstack:4566 is present in packages/backend/.env, which is not needed anymore as that flag is being passes by docker-compose.local.yml now.
Therefore, it should be remove from packages/backend/.env.shared
Steps to reproduce

Create a .env for backend based on .env.shared
pnpm saas build
pnpm saas deploy

System Info
Any local deploy (not from CI) should face the same issue.

Logs
No response
Validations

[X] Follow our Code of Conduct.
[X] Read the Contributing Guidelines.
[X] Read the docs.
[X] Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
[X] Check that this is a concrete bug. For Q&A open a GitHub Discussion or join our Discord Chat Server.

Thank you for the contribution!:)


wrong input when edit in multy cursor mode after copied something in multy cursor mode

We have written the need
Type: Bug
1, Press alt key and use left mouse click to select multy words(in diffrent lines), use 'ctrl + c' to copy these words
2.  Past these words in a new line
3.  use 'ctrl + alt + ↓' to create multy cursor, the cursor number is the same as the length which copied in step 1;
4.  just edit and input anything:
eg.  Select 6 lines and then input a letter 'e',  the 1st line got 1 'e', but 2nd line got 5 'e', and 3rd line got 10 'e', 4th got 10,  5th got 5 , 6th got 1
VS Code version: Code 1.70.2 (e4503b30fc78200f846c62cf8091b76ff5547662, 2022-08-16T05:35:13.448Z)
OS version: Windows_NT x64 10.0.19042
Modes:

System Info

Item
Value

CPUs
Intel(R) Core(TM) i7-10700 CPU @ 2.90GHz (16 x 2904)

GPU Status
2d_canvas: enabledcanvas_oop_rasterization: disabled_offdirect_rendering_display_compositor: disabled_off_okgpu_compositing: enabledmultiple_raster_threads: enabled_onopengl: enabled_onrasterization: enabledraw_draw: disabled_off_okskia_renderer: enabled_onvideo_decode: enabledvideo_encode: enabledvulkan: disabled_offwebgl: enabledwebgl2: enabled

Load (avg)
undefined

Memory (System)
15.72GB (2.41GB free)

Process Argv
--crash-reporter-id 1a3b94b7-5db8-4aee-b502-5762e4fbaa84

Screen Reader
no

VM
0%

Extensions (23)

Extension
Author (truncated)
Version

project-manager
ale
12.7.0

conflict-squeezer
ang
1.0.1

vscode-django
bat
1.10.0

vscode-eslint
dba
2.2.6

python-environment-manager
don
1.0.4

python-extension-pack
don
1.7.0

gitlens
eam
12.2.1

prettier-vscode
esb
9.8.0

remotehub
Git
0.42.0

vsc-python-indent
Kev
1.17.0

template-string-converter
meg
0.6.0

vscode-docker
ms-
1.22.1

python
ms-
2022.14.0

vscode-pylance
ms-
2022.8.50

jupyter
ms-
2022.7.1102252217

jupyter-keymap
ms-
1.0.0

jupyter-renderers
ms-
1.0.9

remote-repositories
ms-
0.18.0

autodocstring
njp
0.6.1

vetur
oct
0.36.0

svg-preview
Sim
2.8.3

vscodeintellicode
Vis
1.2.24

jinja
who
0.0.8

A/B Experiments
vsliv368cf:30146710
vsreu685:30147344
python383cf:30185419
vspor879:30202332
vspor708:30202333
vspor363:30204092
vstes627:30244334
vslsvsres303:30308271
pythonvspyl392:30443607
vserr242:30382549
pythontb:30283811
vsjup518:30340749
pythonptprofiler:30281270
vshan820:30294714
vstes263cf:30335440
vscoreces:30445986
pythondataviewer:30285071
vscod805:30301674
binariesv615:30325510
bridge0708:30335490
bridge0723:30353136
cmake_vspar411:30542924
vsaa593:30376534
pythonvs932:30410667
wslgetstarted:30449410
cppdebug:30492333
pylanb8912:30545647
vsclangdf:30486550
c4g48928:30535728
hb751961:30543456
dsvsc012:30540252
azure-dev_surveyone:30548225

ed data into your clipboard because it was too large to send. Please paste.

Thanks for creating this issue! It looks like you may be using an old version of VS Code, the latest stable release is 1.71.0. Please try upgrading to the latest version and checking whether this issue remains.
Happy Coding!

I cannot reproduce this. Does this reproduce when disabling all extensions?

This issue has been closed automatically because it needs more information and has not had recent activity. See also our issue reporting guidelines.
Happy Coding!


Adjusted TopRow layout

fixed top menu layout issue on small screens
Adjusted styling for theme select and about button
added small margin to hamberger button
adjusted TopRow layout

Hamburger is wrapping to new column on small screens


keep current mapType when taking snapshot

Does any other open PR do the same thing?

No
What issue is this PR fixing?
When taking snapshot, it used default mapType. This PR will keep the current mapType then take snapshot
How did you test this PR?

Test in iphone simulator

Hi, thanks for the PR. Is this an issue on Android too? If yes can you fix it there too? 🤗

@rborn Android is working well, just happens on iOS


automatic close

The select options doesnt close automatically when a option is selected it needs to closed manually by clicking on other side of screen to close the options bar, how can we make it to close automaticallly

automatic close on selecting an multi option

@leosv123 by default component's behaviour is to allow users to select multiple options at a time so it would not close automatically but I think the options you might be looking for is shouldToggleOnHover prop


⚠ Bofrost.nl has degraded performance

In 5a44442, Bofrost.nl (https://www.bofrost.nl) experienced degraded performance:

HTTP code: 200
Response time: 952 ms

Resolved: Bofrost.nl performance has improved in 8964fac.