Reinstalling mmonit pack fails as the venv is being used

Im sure this is due the sensor being run, so you cant create the virtualenv again as its being used.
Should the packs action check if there is sensors and try to stop them before reinstalling a pack?
  "stderr": "st2.actions.python.SetupVirtualEnvironmentAction: DEBUG    Setting up virtualenv for pack \"mmonit\"\nst2.actions.python.SetupVirtualEnvironmentAction: DEBUG    Creating virtualenv for pack \"mmonit\" in \"/opt/stackstorm/virtualenvs/mmonit\"\nst2.actions.python.SetupVirtualEnvironmentAction: DEBUG    Creating virtualenv in \"/opt/stackstorm/virtualenvs/mmonit\" using Python binary \"/usr/bin/python2.7\"\nTraceback (most recent call last):\n  File \"/usr/lib/python2.7/dist-packages/st2actions/runners/python_action_wrapper.py\", line 116, in <module>\n    obj.run()\n  File \"/usr/lib/python2.7/dist-packages/st2actions/runners/python_action_wrapper.py\", line 61, in run\n    output = action.run(**self._parameters)\n  File \"/opt/stackstorm/packs/packs/actions/pack_mgmt/setup_virtualenv.py\", line 52, in run\n    self._setup_pack_virtualenv(pack_name=pack_name)\n  File \"/opt/stackstorm/packs/packs/actions/pack_mgmt/setup_virtualenv.py\", line 89, in _setup_pack_virtualenv\n    self._create_virtualenv(virtualenv_path=virtualenv_path)\n  File \"/opt/stackstorm/packs/packs/actions/pack_mgmt/setup_virtualenv.py\", line 125, in _create_virtualenv\n    (virtualenv_path, stderr))\nException: Failed to create virtualenv in \"/opt/stackstorm/virtualenvs/mmonit\": Traceback (most recent call last):\n  File \"/usr/local/lib/python2.7/dist-packages/virtualenv.py\", line 2363, in <module>\n    main()\n  File \"/usr/local/lib/python2.7/dist-packages/virtualenv.py\", line 832, in main\n    symlink=options.symlink)\n  File \"/usr/local/lib/python2.7/dist-packages/virtualenv.py\", line 994, in create_environment\n    site_packages=site_packages, clear=clear, symlink=symlink))\n  File \"/usr/local/lib/python2.7/dist-packages/virtualenv.py\", line 1288, in install_python\n    shutil.copyfile(executable, py_executable)\n  File \"/usr/lib/python2.7/shutil.py\", line 83, in copyfile\n    with open(dst, 'wb') as fdst:\nIOError: [Errno 26] Text file busy: '/opt/stackstorm/virtualenvs/mmonit/bin/python2.7'\n\n",

Perhaps. We have run into this issue a few times ourselves and have been debating how to get this fixed. Perhaps, shutting down the sensor from the reloaded pack might be a good answer or avoid creation of virtualenv might be another.
Thanks for the report!


Dataflow incremental refresh and datasets

[Enter feedback here]
Hello,
Could you please clarify and confirm that the following scenario is working in Power BI Service:

Incremental Refresh of a specific Dataflow
Incremental Refresh of Datasets depending on this specific Dataflow

is technically possible and supported by Microsoft Power BI (Service) .
Reason for question: If you have to ingest large amounts of data from a Data-Source (i.e. DWH) via Dataflows (incrementally refreshed), then the depending Datasets have to be refreshed incrementally as well to take full advantage of the incremental refresh, else a full refresh of the Datasets will be too slow (e.g. for an nightly end-to-end Data Refresh Data-Source => Dataflow => Dataset).
Thanks for your support !

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: b60093b2-2bdf-92e9-ee7c-de316b601beb
Version Independent ID: 25307bcb-648e-e620-3fe9-a1412e89c0b5
Content: Using incremental refresh with Power BI dataflows - Power BI
Content Source: powerbi-docs/transform-model/service-dataflows-incremental-refresh.md
Service: powerbi
Sub-service: powerbi-service
GitHub Login: @davidiseminger
Microsoft Alias: davidi

Thanks for your comment, @PowerBI-User999 . This issue has been assigned to the doc author for review.

Hi @PowerBI-User999 ,
I'm cleaning up our Git Issues queue, and I noticed that this Git issue is a bit old, so I'm going to close it.  I hope that you were able to get your questions answered. If not, please let us know.  Another option is to submit your questions to the Power BI community (community.powerbi.com).
All the best,
the DOCS team


Crash DDLog.m line 839 - [DDLog lt_log:]

New Issue Checklist

[x] I have read and understood the CONTRIBUTING guide
[x] I have read the Documentation or CocoaDocs
[x] I have searched for a similar issue in the project and found none

Issue Info

Info
Value

Platform Name
iOS

Platform Version
12.1.2

CocoaLumberjack Version
3.6.0

Integration Method
cocoapods

Xcode Version
Xcode 12.2

Repro rate
sometimes 67%

### Stack:
cocoa.lumberjack
0  libsystem_kernel.dylib         0x1a85539fc __ulock_wait + 8
1  libdispatch.dylib              0x1a8397238 _dispatch_ulock_wait + 56
2  libdispatch.dylib              0x1a8397c38 _dispatch_group_wait_slow + 40
3  CocoaLumberjack                0x104adae78 -[DDLog lt_log:] + 839 (DDLog.m:839)
4  CocoaLumberjack                0x104ad962c __40-[DDLog queueLogMessage:asynchronously:]_block_invoke + 358 (DDLog.m:358)
5  libdispatch.dylib              0x1a83f56c8 _dispatch_call_block_and_release + 24
6  libdispatch.dylib              0x1a83f6484 _dispatch_client_callout + 16
7  libdispatch.dylib              0x1a839dbd0 _dispatch_lane_serial_drain$VARIANT$mp + 592
8  libdispatch.dylib              0x1a839e718 _dispatch_lane_invoke$VARIANT$mp + 432
9  libdispatch.dylib              0x1a83a6eb8 _dispatch_workloop_worker_thread + 600
10 libsystem_pthread.dylib        0x1a85d90dc _pthread_wqthread + 312
11 libsystem_pthread.dylib        0x1a85dbcec start_wqthread + 4

The issue is still relevant, we still have a lot of similar crashes

Do you have some more stack traces? Possibly also including other threads? It looks like a deadlock, but for that we'd need to see where the call is coming from.

I am seeing this crash as well. Here is one kind of stack trace we see
Exception Type:  EXC_CRASH (SIGKILL)
Exception Codes: 0x0000000000000000, 0x0000000000000000
Exception Note:  EXC_CORPSE_NOTIFY
Triggered by Thread:  0

Thread 0 name:
Thread 0 Crashed:
0   libsystem_kernel.dylib        	0x00000001bc6a924c kevent_id + 8
1   libdispatch.dylib             	0x0000000182ddbfb4 _dispatch_kq_poll + 228 (event_kevent.c:750)
2   libdispatch.dylib             	0x0000000182ddc9f0 _dispatch_event_loop_wait_for_ownership$VARIANT$mp + 440 (event_kevent.c:2221)
3   libdispatch.dylib             	0x0000000182dcb820 __DISPATCH_WAIT_FOR_QUEUE__ + 288 (queue.c:1678)
4   libdispatch.dylib             	0x0000000182dcb464 _dispatch_sync_f_slow + 136 (queue.c:1762)
5   Logging                       	0x00000001020f3560 -[DDLog queueLogMessage:asynchronously:] + 95584 (DDLog.m:336)
6   Logging                       	0x00000001020f3b84 -[DDLog log:message:level:flag:context:file:function:line:tag:] + 97156 (DDLog.m:476)
7   Logging                       	0x00000001020f3784 -[DDLog log:level:flag:context:file:function:line:tag:format:] + 96132 (DDLog.m:0)

And another
Exception Type:  EXC_CRASH (SIGKILL)
Exception Codes: 0x0000000000000000, 0x0000000000000000
Exception Note:  EXC_CORPSE_NOTIFY
Triggered by Thread:  0

Kernel Triage:
VM - pmap_enter failed with resource shortage

Thread 0 name:
Thread 0 Crashed:
0   libsystem_kernel.dylib        	0x00000001b85bf1a4 __ulock_wait + 8
1   libdispatch.dylib             	0x00000001812ed064 _dlock_wait + 56 (lock.c:326)
2   libdispatch.dylib             	0x00000001812ed414 _dispatch_group_wait_slow + 60 (semaphore.c:214)
3   Logging                       	0x0000000106324dec -[DDLog lt_log:] + 101868 (DDLog.m:0)
4   Logging                       	0x00000001063235b4 __40-[DDLog queueLogMessage:asynchronously:]_block_invoke + 95668 (DDLog.m:325)
5   libdispatch.dylib             	0x00000001812ec670 _dispatch_client_callout + 20 (object.m:560)
6   libdispatch.dylib             	0x00000001812fb8c4 _dispatch_lane_barrier_sync_invoke_and_complete + 56 (queue.c:1016)
7   Logging                       	0x0000000106323560 -[DDLog queueLogMessage:asynchronously:] + 95584 (DDLog.m:336)
8   Logging                       	0x0000000106323b84 -[DDLog log:message:level:flag:context:file:function:line:tag:] + 97156 (DDLog.m:476)
9   Logging                       	0x0000000106323784 -[DDLog log:level:flag:context:file:function:line:tag:format:] + 96132 (DDLog.m:0)

Both crashes happen early on in our app's lifecycle, in methods called from application:didFinishLaunchingWithOptions:

@silverhammermba can you trace the issue back to a certain DDLogXYZ statement? If so, is that a synchronous log call by any chance (level error or greater by default)?

@ffried yes I can trace it back, and no both log statements are lower (former is info, latter is warn). We're on Cocoalumberjack 3.7.2

@silverhammermba Ok, thanks for the info! I'm still a bit confused why those calls would end up in a deadlock.
Maybe I can find some time to investigate this.
How reliably can you reproduce this? Would you be able to provide a sample project?

I periodically see deadlocks as well. But only when stepping in the Xcode debugger. (I try to avoid stepping over any log statements.)

@ffried I have not been able to reproduce this myself. I'm getting the stack traces from Apple's crash reporting. This is our  number 1 crash from customers installing our app in the app store :(

I may have reproduced it. But I can only get it to happen when our app is built in release mode with no debugger attached. Tried reverting to Cocoalumberjack 3.1.0 (which we used to use) but it still crashed on app launch, only after locking up for 20 seconds or so.

@silverhammermba The 20 seconds make sense. It's the watchdog from iOS that terminates apps that hang.
I've checked your stack traces against our code and it definitively seems to log synchronously!
Is you application in Swift or Objective-C?

@ffried our app is mixed swift and objective-c, this part it's crashing on is all objective-c.
I did find a workaround for the crash, still no root cause. I will try to put together a small app for reproducing it, though I'm a bit busy with rushing out a hotfix. Here is what our app is doing and what the workaround was:
In addition to Cocoalumberjack, we also use the WS1IntelligenceSDK pod for VMWare crash analytics (it sucks, don't give them money). The steps during launch in our app delegate are

Init WS1, give them the device UUID (this doesn't turn on crash reporting yet)
Init Cocoalumberjack, give it a log formatter which also passes every log message to our AppDelegate.

when the AppDelegate receives a log message, it forwards it to WS1 so that we see the latest logs in our crash analytics

Set up a notification observer on the main thread for WS1. If WS1 detects that the app crashed the last time it was used, WS1 sends us a notification. When we receive it, we log a warning and stop observing that notification.
Enable crash reporting in WS1

The workaround was to comment out the line in AppDelegate that forwards ours logs to WS1. In my mind the sequence of events is something like

App crashes for some reason
User restarts app
We get the WS1 notification that app crashed last time
Try to log a warning
Cocoalumberjack asks our log formatter to format it
Log formatter first tells AppDelegate about the log
AppDelegate passes the log to WS1
????
deadlock

I can see how that is risky. Basically our log formatter blocks Cocoalumberjack, which may be in the middle of some mulithreading shenanigans, and starts doing stuff in a totally separate library. I bet I could fix this by making the log formatter use some async mechanism like a notification to pass the log to the AppDelegate instead. Still would be nice to know the real root cause.

our app is mixed swift and objective-c, this part it's crashing on is all objective-c.

Do you somewhere change the define LOG_ASYNC_ENABLED? And are you using our default DDLogXYZ macros?

Basically our log formatter blocks CocoaLumberjack, which may be in the middle of some multihreading shenanigans, and starts doing stuff in a totally separate library

This sound very much like a possibility for a deadlock. Especially since everything happens on the main thread. Also, CocoaLumberjack has a (serial) dispatch queue for each logger as well as a (serial) queue for all loggers, that is synced using a dispatch group.
The log formatter is called on the serial queue of the logger. Making a blocking call here might block the whole logging system. While loggers are generally given log messages in parallel, each log message is awaited to not overflow a single logger that is slower than others.
I'd recommend to implement your own logger for passing on log messages to another system. If passing them on is slow, maybe even maintain your own queue inside this logger for this particular step. Using a dedicated logger allows the message to be logged by other loggers in parallel instead of blocking them.

Do you have the contents of the other threads in your backtrace by any chance? This would maybe show, where the deadlock happens. Your backtrace above showing thread 0 doesn't seem to be the cause for the deadlock - just the one running into it.

We don't mess with LOG_ASYNC_ENABLED. We don't directly call DDLogXYZ, we call our own macro which calls it, but I don't see any risky stuff there.
I checked the other stack traces we have and indeed all of them show some background thread in the middle of a lt_log call while thread 0 is stuck waiting on some sync mechanism. I also verified that our workaround was only a partial fix, we still have some users' apps crashing on log calls. I'm still a little confused how any of this can deadlock. The log calls that are blocking thread 0 are all async (DDLogInfo). Even if Cocoalumberjack is getting deadlocked internally, shouldn't those async logs immediately return?
We looked around for other mechanisms tied into logging and we similarly have another delegate call which is encrypting log messages. It sounds like that should also be a separate logger with its own internal queue. We'll try implementing that in the meantime.

@silverhammermba I'm probably as confused as you are 👀
I've triple checked our code by now. If the log call was made using an async macro, the first interaction with a queue is async on the _loggingQueue. There is some syncing across queues internally later, but that first call is async and thus shouldn't block...
What I've noticed from your backtraces is that the line numbers don't match with version 3.7.2. Do you have backtraces that happened with version 3.7.2 by any chance? Just so that I can confirm it's really using the async path.

🤦 if it's any consolation, you were completely right the whole time and we overlooked the obvious problem in our code.
Our code was only calling DDLogXYZ in Swift, in Objective-C we had a custom macro which was calling -[log:level:flag:context:file:function:line:tag:format:,...] and we had inverted the first argument. We thought the flag was for synchronous logging, not asynchronous. Consequently almost all of our log messages were synchronous (and we have a lot of log messages). Frankly I'm amazed our app wasn't crashing more.
My earlier workaround helped because it made each log slightly faster, which helped to mitigate all of the thread blocking.
So, at least for us, the RCA for this lt_log crash is way too much synchronous logging. Sorry for wasting your time, and thank you so much for your help.
Maybe in Cocoalumberjack 4.0 you can rename that method to -[logAsync:level:flag:context:file:function:line:tag:format:,...], that would have made it much more obvious that we were doing a dumb thing.

@silverhammermba As awkward as this might be for you, I'm actually relieved. I was really starting to doubt everything here. 😅
I agree that the naming isn't great here. Maybe I'll just deprecated the old methods and add new versions of them. That way we don't have to wait until version 4.0 😉

For anyone else with issue: Please first make sure you are not logging synchronously and then provide back traces for your crashes. They help a lot in diagnosing the underlying cause! Until then I'll close this issue.


Documentation for JRPC

Is there a documentation available for the ProxyFS JRPC protocol, that is used in this library?

Great question. First things first... the actual JRPC used is JSON RPC 2.0 implementing the APIs listed in the jrpcfs subdirectory's api.go file in the ProxyFS repo... see https://github.com/swiftstack/ProxyFS/blob/development/jrpcfs/api.go
As mentioned in https://github.com/swiftstack/proxyfs-jrpc-client/issues/14, the client in this package (proxyfs-jrpc-client) also implements a separate RPC path for reads and writes. Alas, the documentation for that path is much more obscure at this point. You can get the gist of it by perusing the io.go file in that same subdirectory - specifically, the ioRequest and ioResponse struct's.
There are other caveats... ProxyFS, for one, has this notion of PrivateIPAddr vs PublicIPAddr. The PrivateIPAddr is assumed to be on a trusted network and, as such, has no authentication/encryption at all. Also, while reads and writes use the non-JSON RPC path fully supporting multiple async requests in flight, the JSON RPC path is not expected to do so. It turns out that the VFS API implemented by the companion submodule proxyfs-vfs only ever issues a single request at a time. The server should, however, have ample support for multiple requests being outstanding since it is also regularly used by the Object API (both Swift & S3) path (see pfs-middleware in the ProxyFS repo.
I'd like to add one more tidbit. While the JSON RPC interface implemented by this package is used by both SMB and Object API paths, an enhancement for it is used by PFSAgent (see pfsagentd subdirectory in the ProxyFS repo). For PFSAgent, the expectation is that it will not have access to the PrivateIPAddr network of a ProxyFS instance and, instead, need to use the PublicIPAddr to reach it. As such, a new interface (port) is exposed on each ProxyFS node that is on the PublicIPAddr. This public RPC port is TLS protected. The client gets the cert used to authenticate the server in the Mount* JSON RPC response and, there after uses this much more resilient and optimized RPC path for all subsequent non-Read/Write RPCs. While not strictly JSON RPC 2.0 anymore, the mechanism to use it is quite simple (see the retryrpc subdirectory in the ProxyFS repo.
Also pertinent to the move to the retryrpc method is that the Reads and Writes are more "physical" rather than "logical" in nature. What this means is that rather than saying "I want to read 5 bytes from the beginning of file foo", you use the RPC to find out where those 5 bytes are and read them directly via a Swift or S3 GET (with a special path that basically says "hey, I know what I'm doing"). Writing is quite a bit more complicated as the client has to take on the responsibilities of buffering write data and sending it to Swift in reasonable chunks... and updating the inode at appropriate times. The aim of this new division of labor - heavily utilized by PFSAgent - is to massively scale out the performance of ProxyFS while keeping the metadata operations simple and centralized.
I hope this helps...

So currently the best source of documentation for JRPC is reading the server implementation and PFSAgent as an advanced example of client implementation?
How stable is this API going to be? Does it make sense to depend on it for some external systems? We want to avoid extra layers like Samba/Ganesha/Fuse between our service and storage. Content is expected to be immutable, but write pattern may be random, i.e. our service can write random ranges of file (possibly retrying some on failures). Using Swift Object API for this purpose seems suboptimal, we'll have to divide the file in blocks, and do read-modify-write in case writes are not aligned to block boundary, but given eventual consistency model that Swift offers, that won't work well.

So currently the best source of documentation for JRPC is reading the server implementation
and PFSAgent as an advanced example of client implementation?

I would say a definite YES to that BUT...

How stable is this API going to be?...

The intention of the JRPC path to ProxyFS is (was) for:

The local (i.e. on the very same node and, hence, not scale out-able) instance of Samba, and
The set of SwiftProxy instances in the cluster to perform metadata operations to keep the file system's namespace in sync with what the object API presents out each of the SwiftProxy nodes (via either OpenStack Swift or S3 APIs)

A third intent of JRPC was to perform a similar function to a local Ganesha instance for NFS. As you are probably aware, Samba's VFS "plug-in" mechanism has a counterpart in Ganesha called FSAL. The original intent was to adopt Ganesha as the NFS solution, craft a proxyfs-fsal equivalent to proxyfs-vfs but plug it into Ganesha. This proxyfs-fsal would similarly map the calls it receives to execute the APIs of proxyfs-jrpc-client.
Along the way, we found that the FUSE exposure of ProxyFS itself sufficient when paired with the NFSd of Linux... thus never developed the proxyfs-fsal component to use Ganesha NFS instead.
As for stability, the JRPC API has, to date, received very little change over its four year existence. Indeed, the only changes I can recall were:

A work-around for JSON's inability to distinguish ints from uints. This was very unfortunate because things like InodeNumbers are uint64s in ProxyFS with the upper 'N' bits indicating which "view" of the File System one is interested in. When those upper 'N' bits are all zero, you are referencing the "live" view of the File System. When non-zero, the upper bits select the SnapShot identified by the value of those upper 'N' bits. And, finally, when all 'N' bits are set, you are looking at the "directory" listing/navigating-to all available SnapShots (specifically, you are looking at '/.snapshot/'). As that "directory" and half of the available SnapShot IDs will have the uppermost bit set, the various JSON RPC  "marshallers" (e.g. the C one!) made the assumption that the number was actually negative :-(. The change to the JRPC API was to account for this unfortunate behavior. Indeed, I've long regretted ever choosing JSON as the RPC format for this very reason.

As we introduced the nature of a resilient RPC mechanism (see ProxyFS's retryrpc sub-package), we needed a way for a reconnected RPC client to identify the mounted state they were re-establishing. At the time, the MountID was merely a uint64 that incremented each time someone connected. Over the trusted interface, this was of course vulnerable to malicious impersonation. The change was to use a cryptographically generated GUID as the MountID... thus requiring a very small enhancement to the Mount RPCs. At this time, we also wanted to use a trustable transport and shifted to TLS 1.3 (instead of just raw, unsecured TCP). The Mount RPC response carries with it the Ed25519 RootCA's public key (in PEM form) to simplify the process of client configuration of their TLS connection process.

Does it make sense to depend on it for some external systems?

I would say not, actually. For a number of reasons. Most important among these is the open trust the JRPC path has with its clients... hence, it is only listened to on the so-called PrivateIPAddr. Further, with the advent of the new retryrpc mechanism and development of PFSAgent, it is our hope that both NFS and SMB serving either become unnecessary or, if PFSAgent cannot be deployed on the client, that it instead be deployed on whatever (set of) server(s) that will stand in as proxy's over to Swift (for reads and writes) and ProxyFS (for all metadata). This PFSAgent approach allows for scaling out of performance dramatically surpassing the capabilities of the local Samba and NFSd servers co-resident with the ProxyFS process serving your volume.

We want to avoid extra layers like Samba/Ganesha/Fuse between our service and storage.
Content is expected to be immutable, but write pattern may be random, i.e. our service can
write random ranges of file (possibly retrying some on failures). Using Swift Object API for
this purpose seems suboptimal, we'll have to divide the file in blocks, and do read-modify-write
in case writes are not aligned to block boundary, but given eventual consistency model that
Swift offers, that won't work well.

Interesting challenge. As you notice, the RESTful APIs of Swift and S3 are not on the face of things conducive to random write patterns... hence the interest in a file system semantic. Your desire to avoid the FUSE et. al. overhead is a challenge for that.
Let me propose two alternatives... The first sounds much more like what you in general might be looking for. That is, it sounds like you'd prefer a callable library of sorts way to interact with Swift and ProxyFS presenting a POSIX-like API but staying within the confines of your application's process. That is a very reasonable request... indeed, if you look at PFSAgent, it is very much architected to make that very possible. It leverages the fission package to issue callbacks to what you might call the back end of the PFSAgent. These callbacks are basically 1-for-1 with the FUSE upcalls received from the Linux Kernel by the PFSAgent user mode process. Seems to me you could just call those yourself. Let me think on that and see if we can expose that more directly.
Key question for this alternative is what language your application is in. If it's Golang, the adjustment would seemingly be very straight forward :-).
But now I'd like to propose a 2nd, more direct alternative. It's appropriateness would depend upon the number of ranges of writes your application might do. I'm really picking up on the fact that the written "files" are ultimately to be immutable. If it is the case that the "file" is written with a modest number of fragments (albeit randomly placed) with little or no overwrites, I think the "chopping the file up into blocks" unnecessary. Indeed, I think ProxyFS itself unnecessary.
Consider the S3 Multi Part Upload mechanism... or the Swift API's near equivalent Static Large Object. In both of these, your application can upload different fragments of the eventual "file" via distinct/separate PUTs. Then, once you are done, you issue the Multi Part Upload Complete or SLO Manifest PUT to tie things all together into a single "file" (Object). Indeed, if the Swift Account is "ProxyFS enabled", there is a fancy "COALESCE" verb you may use which stitches together all those "file fragments" you PUT without moving any data. Indeed, COALESCE is used under the covers by that S3 Multi Part Upload Complete method. COALESCE basically takes a manifest of Objects to logically concatenate (assuming no holes). Seems to me this would allow for the fastest possible upload mechanism possible (indeed, you may upload various parts in parallel).
Perhaps the MPU/SLO/Coalesce mechanism too limiting for you...love to hear more. Indeed, perhaps such a discussion better pursued on the proxyfs.slack.com Slack channel.


Not Saving System Resource Widgets

Dev Home version
<IP_ADDRESS>
Windows build number
10.0.22621.1778
Other software
OS Build Version: 10.0.22621.1778.amd64fre.ni_release.220506-1250
.NET Version: .NET 6.0.16
Steps to reproduce the bug
Set sys widgets, close app, start app
Expected result
widgets are saved after restarting app
Actual result
not saved
Included System Information
CPU: AMD Ryzen 5 2600 Six-Core Processor
Physical Memory: 31.93GB (16.24GB free)
Processor Architecture: x64
Included Extensions Information
Extensions:
Microsoft.Windows.DevHome_<IP_ADDRESS>_x64__8wekyb3d8bbwe
Microsoft.Windows.DevHomeGitHubExtension_<IP_ADDRESS>_x64__8wekyb3d8bbwe

Duplicate of #724


FOM: endpoint getFileContents() for downloading attachment not working

Description:
While working on ticket #158 testing out using new backend api NestJS Guard implementation for controllers, found there is a problem for AttachmentController.getFileContents() endpoint.
Initial investigation identifies that this bug isn't introduced from backend work at #158; it was already not working before FOM Angular upgrade.
This particular endpoint allows admin users to download attachments (for example, at "View FOM") page.
At the current production (I used a PR dev deployment without new code change, same as current main code for production) this endpoint will throw "500" internal error. With current backend api work using Guard for endpoint, this guard for this endpoint will return "403" error, which make more sense as the problem seems to be no header included in the request but endpoint is secured. Why the header is missing on this particular endpoint requests still needs investigation. Currently, it seems the problem is from frontend.

Acceptance Criteria

[ ] "Admin" client user can download the file content.
[ ] Ministry user can download the file content.
[ ] No public access for file content download.

Additional context

Probably need to re-write 'admin' frontend code to use api (not just click a link) to include 'auth' header in request for file download.

I've tested this as part of FOM upgrade testing and confirmed that both public notices and supporting document attachments suffer from this problem. This is a major issue. One quick fix would be to drop the need to secure access to public notices, and only secure the supporting documents. But still need a solution for FOM admin to view supporting documents in secured fashion though...

Confirmed that Stakeholder engagements (interactions) attachments are also experiencing the same problem.

It looks like we might need to do getFileContents for backend api according to NestJS: streaming-file.

While working on this issue and testing "public" side to download file, encountered strange 403 error even though backend permitted "public" to access the endpoint. Investigation found that one of the backend Attachment Entity property (attachmentTypeCode) is undefined, so the check for isViewAuthorized failed.
It is strange the entity is missing the value from TypeORM for attachmentTypeCode but has value for attachmentType relation from (findOne()); but the same entity with the same decorator @RelationId is used for projectId and has a value. Don't know why it behaviour differently.
However, I am able to work around the issue.


Starting point index of each section

Hello,
Currently I am trying to inject the morphology objects from MorphIO API into our package SpatialIndex. The SpatialIndex API has a method to add branches with an array of points (start/end points of segments), an array of radii, and an array of the first point index of each branch. For the first two arrays, I can use the property accessors of the MorphIO API (like Morphology.points, Morphology.diameters).
My question is that is it possible to get the index of the starting points of each section directly from the API? I am trying to avoid loops in Python for performance issue, so it would be better to have such array directly constructed by the API.
Suggestions and discussions are appreciated.
Cheers,
Weina

Hi,
Yes, I can add this as an attribute of the Morphology class. That will be a vector of int and each one will give you the offset in the point array. It will be available as section_indices. Sections will be in the depth first search order in a given neurite and the neurite order is the one of the file.
One thing to note is that Morphology.points only provides points belonging to neurites. Soma points can only be retrieved as Morphology.soma.points.
Does this sound good to you ?

Hello Benoit,
The section_indices sounds very good. Thanks !
Yes, I see that soma points are not included in the Morphology.points. In SpatialIndex we can add soma and segments separately per gid. Are the Morphology.points always connected to the first element of the array Morphology.soma.points ?

No, the neurite point are not necessarily connected to the soma points. It may be, but it does not have to be.


Unknown key not spawning on the dead scav body behind RUAF extract on Customs

SPT Version
3.10-BE
Project Type
No response
Expected Result
At least one unknown key spawn is guaranteed on that body
Actual Result
Key just doesn't spawn
Steps To Reproduce
Go into the raid with or without AI spawns and loot the body
Server Log
Click and attach file. ↓
BepinEx Log
Click and attach file. ↓
Client Log
Click and attach file. ↓
Player Profile
No response
Screenshots
The body on a raid without the AI

Attachments

server-2024-09-14.log.gz (69 KiB)
LogOutput.log (10 KiB)
2024.09.15_21-30-47_<IP_ADDRESS>.32291 application.log (33 KiB)
2024.09.15_21-30-47_<IP_ADDRESS>.32291 errors.log (4 KiB)
2024.09.15_21-30-47_<IP_ADDRESS>.32291 notifications.log (2 KiB)
2024.09.15_21-30-47_<IP_ADDRESS>.32291 spatial-audio.log (0 KiB)
2024.09.15_21-30-47_<IP_ADDRESS>.32291 traces.log (73 KiB)
2024-09-15[23-42]_28.2, 2.5, -109.5_0.2, -0.8, 0.4, 0.4_10.90 (0).png (1440 KiB)

Originally written by LOLZ

Fixed with: https://github.com/sp-tarkov/server/commit/022737ef356c7e807b89fc439b2444806f95a683

Originally written by chomp


Mssql fixes

This PR fixes the mssql driver's createTableIfNotExists() function, which previously (due to an inverted condition) would never actually create the table.
It also gets the integration tests mostly working; there's one last error I haven't resolved yet:
  1) Integration Tests mssql | mssql Schema createTable rejects setting foreign key where tableName is not typeof === string:
     TypeError: Cannot read property 'sql' of undefined
      at TableCompiler.alterTableForCreate (lib/schema/tablecompiler.js:9:10850)
      at TableCompiler.create (lib/schema/tablecompiler.js:9:2125)
      at TableCompiler.toSQL (lib/schema/tablecompiler.js:9:1456)
      at TableBuilder.toSQL (lib/schema/tablebuilder.js:9:1630)
      at SchemaCompiler.createTable (lib/schema/compiler.js:9:2304)
      at SchemaCompiler.toSQL (lib/schema/compiler.js:9:1782)
      at SchemaBuilder.toSQL (lib/schema/builder.js:9:2260)
      at lib/runner.js:9:1117
  From previous event:
      at Runner.run (lib/runner.js:9:733)
      at SchemaBuilder.Target.then (lib/interface.js:9:1345)

Thanks to @elhigu for all the help understanding how the knex internals work.

thanks!


Baudrate attribute

I could not find any attribute related to baudrate or bitrate. Shouldn't this be a network interface attribute or do you only target modules supporting a single baudrate?

There is not currently a baudrate/bitrate attribute for the network interface object (or any other object). Currently, the bitrate is indirectly selected based upon the modulation. For the modules I'm familiar with, each modulation technique selects a single network interface bitrate. For example, on the AC400 selecting 16-QAM modulation means the network interface is carrying 200Gbps of host interface traffic.

Yes, it's true that the AC400 and existing pluggable DCO modules only supports a single baudrate so the modulation decides the bitrate. But some system vendors' transponders (using in-house DSPs) already support two baudrates and the AC1200 will support variable baudrate. Maybe TAI doesn't have to support this today but it is something that will need to be considered in the future.

I think it is good to have a baud rate attribute in TAI. Thanks for bringing this up.
Even if the DSP doesn't support changing the baud rate, it might be useful to have a read-only attribute to know the current baud rate.

I agree to Wataru, baudrate must be read-only parameter from coherent DSP point of view. Generally baudrate is determined after the other setting parameters like modulation format, FEC type, FEC overhead, etc. have been configured.


Broken Link in Step 0 of Tutorial

README.md link is Broken in Step 0 of both VanillaJS and VueJS tutorial.

You're right. It should be a link to the Sugarizer Application paragraph of the README.md file at the root of the repo.

Fixed in https://github.com/llaske/sugarizer/pull/962


Examples of feature disparities - Legend vs Rosetta for CDM?

Hi, has anyone used both Legend + Rosetta and faced problems with inconsistencies/feature disparities between the two DSLs? I am polling for prevalent issues to investigate.
This is part of a potential research project on the compatibility of using two DSLs to work with the CDM.
My current trajectory
Given I am new to FINOS, I'm very open to and actively seeking feedback!
TODO
Assess feature disparity between Pure and Rosetta:

[ ] 1. Explore "diff" approaches to compare transpiled Rosetta (from Pure) against the original Rosetta, and vice versa.

The purpose of this post is to gather input samples/test cases for this.

[ ] 2. Manually evaluate modeling capabilities of each DSL (via documentation review and domain-agnostic modeling tests). Unclear if this is helpful.
[ ] 3. Test Morphir as a target IR.

What I know so far + additional questions

The Legend/Pure -> Rosetta pipeline seems fleshed out.

I can transpile the Legend CDM back to Rosetta (using "New File Generation.."). This appears to solve #300 , as well as the code generator discussion in #311.

How do members typically deploy the rosettaTypes.txt output? Is this out of scope of the "Pure to Rosetta translation" problem?
What else might I be missing? (e.g., problems people face using the generated Rosetta?)

The Rosetta -> Legend/Pure pipeline looks less developed.

Per #650 (under "Other"), Rosetta CDM versions are not pushed regularly to Legend. The problem cited is "opaque expressions".

What are some examples of these "opaque expressions"?
Besides opaque expressions, are there other known bottlenecks (technical or otherwise) for regularly pushing Rosetta versions to Legend?

(Sidenote: I recently learned that the Legend CDM was updated last month to match the latest Rosetta CDM.)

ISDA review is required for all CDM changes (e.g., those made in Legend) pushed to the official Rosetta CDM.

From: #335

Is this review stage beyond the scope of DSL compatibility?
Is this review done at a purely logical level? Or are technical inconsistencies ironed out between DSLs?

I am looking into Morphir as a target IR and am just starting to explore how FpML fits into all this.

Hi @smsant - would recommend that you reach out to the FINOS Technical SteerCo to find out more. I think they engaged in a feature comparison analysis.

@beekemarie - sounds good, thanks for the guidance!

This issue is stale because it has been open for 30 days with no activity. Please remove stale label or add any comment to keep this open. Otherwise this will be closed in 5 days.

This issue was closed because it has been inactive for 35 days. Please re-open if this issue is still relevant.


close keyboard when the user hits "return"

It would be nice if the keyboard closed after the user hits return.  I believe if you put it in  tags and give it an ng-submit= that calls the closeKeyboard() function, it would work.

+1

I already have a solution for that.
Can I submit this fix?

Hey @djett41 @felipevlourenco, we tested and its working fine, please could you generate a release with it? CC @RangerRick


Tests: BTree<T::combined()

Add tests to increase branch or path coverage.

There is no untested and reachable branch in this function.
The times that the function is called are these:
https://github.com/OLGKAD/sef18group15-coverage/blob/80bf99af73ab327465ed2ed10b9fdb5601ea5315/src/com/jwetherell/algorithms/data_structures/BTree.java#L190-L209
In these calls, the argument's parent is never null which is required for the only non-visited branch to be ventured.
I'll look at new paths, but I doubt there are any obvious.


Move configuration file + ability to connect to a protected MQTT server

Configuration file should not be versioned, let's give an example one to copy.
Signed-off-by: Gérondal Thibault<EMAIL_ADDRESS>

Nice Feature. Merged top master


[Accessibility] div.monaco-tokenized-source requires aria-live="assertive"

CC @isidorn and @meganrogge
Current Issue

Load NVDA or any screen reader.

Launch VSCode.

enable copilot extension.

Create any ts or js file.

Try to trigger copilot-powered inline suggestion (Alt+\).

See (hear) if suggestion pop-up is automatically read: it's not unless you press Ctrl+K, Ctrl+I to explicitly move your focus to the tooltip area.

Notes: This is because "div.monaco-tokenized-source" does not have aria-live="assertive" to announce the dymanic content to screen reader.

Notes: copilot is just one of the examples. Any extensions or widgets that depend "div.monaco-tokenized-source" have the same issue.

Expected Behavior
The suggested dynamic content needs to be automatically announced to screen reader users.

Note: I am willing to fix this issue and test with my screen reader.

System Info
Version: 1.74.0-insider (system setup)
Commit: d0e44be43287acadb13f7c2808332e00fe52568d
Date: 2022-11-25T08:00:08.922Z
Electron: 19.1.6
Chromium: 102.0.5005.167
Node.js: 16.14.2
V8: <IP_ADDRESS>-electron.0
OS: Windows_NT x64 10.0.22621
Sandboxed: Yes

@jooyoungseo thanks a lot for opening this issue and for the PR.
However I am not sure if this is the ideal experience. I worry that it might result in Screen Reader reading too much, due to inline completions popping up all the time.
fyi @hediet since we should think about inline completion accessibility improvements.

@isidorn -- I understand your concern. However, adding aria-live to div.monaco-tokenized-source only reads new suggested token instead of the whole tooltip content. This means screen reader users do not necessarily hear redundant and verbose tooltip instructions and button names all the time. Without this feature, using inline suggestion is quite tedious for screen reader users. Because you need to manually press Ctrl+K, Ctrl+I each every time to identify the new content in tooltip.
As far as I tested, this gave me a more seamless and accessible experience when using inline suggestions. However, I understand that I cannot represent other screen reader users.
As a fair compromise, I suggest adding an option to VSCode settings to adjust the aria-live level between "on" and "off" so that users can customize the inline suggestion announcement level per their preference.

I do not have copilot. @hediet could you pls try out the PR with a screen reader?

I will close my PR, but keep opening this issue because I receive too many GH action messages. However, this issue still needs attention.

The idea was to use audio cues to indicate that inline suggestions are available.
Then, Ctrl+K, Ctrl+I can be pressed to read the suggestion.
The problem is that inline suggestions might be there 50% of the time you type something.

I think it makes sense to let the user decide if the potential disruption is worth it and disable this by default.

I explored reading the hint automatically using aria-live assertive. It was very disruptive because would suggest something with every keystroke. I also tried debouncing and the experience still was not great. You can try my branch merogge/ariaAlert to see what you think.
Suggestion from @jooyoungseo :
We can consider replacing the “Suggest  List box” message with a specific audio cue like an inline hint.

@meganrogge-- I have just tested your branch and this is exactly what I intended and proposed. You may find aria-live very disruptive; however, this is what sighted people see visually and it is the matter of getting used to. Advanced screen reader users have very sophisticated techniques. For example, screen readers have a typing echo setting to choose whether you will hear the typing feedback based on character/word/none/both. I would turn off the typing echo so that I can only hear the suggestion live event while typing. You can simply keep typing if you don't want to what's suggested and you can hit tab/enter key to accept the suggestion. Some advanced users, including myself, would find this feature useful and I believe this can improve their productivity. That's why I am suggesting this feature as a user configuration. I would definitely turn this on because the auto-suggestion and copilot hint is immediately read aloud which saves me a lot of time!

Great, thanks for resting. I think it makes a lot of sense to have a setting which enables this.

Going to investigate adding a setting for this and not having a dialogue box open up.
alert does not work on macOS in any of the contexts for me, but I'm going to test on Windows

I have reached out to an apple contact about VoiceOver. This is fixed on Windows though

This now works on all platforms. I had an issue with my VoiceOver configuration


Reading from stdin hangs at EOF, unless submitted with IOSQE_ASYNC

With kernel Linux 5.8.3-arch1-1 #1 SMP PREEMPT Fri, 21 Aug 2020 16:54:16 +0000 x86_64 and liburing 0.7, running the following program as dd if=/dev/urandom count=1 | ./program hangs in io_uring_wait_cqe after ddexits and all the data is read from stdin
#define _GNU_SOURCE

#include <errno.h>
#include <stdio.h>
#include <string.h>
#include <unistd.h>
#include <liburing.h>

int
main(int argc, char **argv)
{
    const ssize_t bufsize = 512;
    char buf[bufsize];
    ssize_t size = 0;
    struct io_uring ring;
    struct io_uring_sqe *sqe;
    struct io_uring_cqe *cqe;

    io_uring_queue_init(8, &ring, 0);
    while (1) {
        sqe = io_uring_get_sqe(&ring);
        io_uring_prep_read(sqe, STDIN_FILENO, buf, bufsize, 0);
        /* sqe->flags |= IOSQE_ASYNC; */
        io_uring_submit(&ring);
        io_uring_wait_cqe(&ring, &cqe);

        if (cqe->res < 0) {
            fprintf(stderr, "Read error: %s\n", strerror(-cqe->res));
            return 1;
        }
        if (cqe->res == 0) {
            fprintf(stderr, "EOF\n");
            break;
        }
        size += cqe->res;
        io_uring_cqe_seen(&ring, cqe);
    }

    fprintf(stderr, "Read: %zd\n", size);
    return 0;
}

whereas it finishes at EOF when uncommenting the line that adds the IOSQE_ASYNC flag. I would expect that both cases behave the same, i.e. generate a CQE with zero bytes read after the writer has closed the pipe.
Is reading from/writing to pipes even a thing that is supposed to be supported by io_uring? If so, would it be possible to also support offset=-1 in the arguments to io_uring_prep_{read,write} instead of offset=0as used above? That seems to be more consistent with regular files, for which offset=0 has a different meaning. In particular, for programs that can read both from stdin or a regular file, that would obliterate the need to distinguish these cases (when reading the entire file sequentially).
Thanks for your work! I hope this is the right place to report the issue, since this is probably something in the kernel rather than liburing, but I saw a note of @axboe on the kernel bugzilla that he doesn't usually read anything there.

Oops, that's a bug! It should work just fine with pipes etc, just made a mistake in how the poll handling is done. This should work:
diff --git a/fs/io_uring.c b/fs/io_uring.c
index 384df86dfc69..d15139088e4c 100644
--- a/fs/io_uring.c
+++ b/fs/io_uring.c
@@ -4889,12 +4889,20 @@ static bool io_arm_poll_handler(struct io_kiocb *req)
 	struct async_poll *apoll;
 	struct io_poll_table ipt;
 	__poll_t mask, ret;
+	int rw;

 	if (!req->file || !file_can_poll(req->file))
 		return false;
 	if (req->flags & REQ_F_POLLED)
 		return false;
-	if (!def->pollin && !def->pollout)
+	if (def->pollin)
+		rw = READ;
+	else if (def->pollout)
+		rw = WRITE;
+	else
+		return false;
+	/* if we can't nonblock try, then no point in arming a poll handler */
+	if (!io_file_supports_async(req->file, rw))
 		return false;

 	apoll = kmalloc(sizeof(*apoll), GFP_ATOMIC);

Can I have your real name and email, so I can add proper reporting attribution to the patch I'll queue for upstream?

Here it is in patch form:
https://git.kernel.dk/cgit/linux-block/commit/?h=io_uring-5.9&id=68faa4727c0e2d7d255d9b97e754f2cda748f141

And I adapted your test case into something that fits with the liburing regression suite and pushed it out:
https://git.kernel.dk/cgit/liburing/commit/?id=43e05df7cd2b4463bec0cdf5457ac0e666334d01

Thanks for the quick fix! I'm Benedikt Ames, wisp3rwind at posteo dot eu.
Regarding

[...] would it be possible to also support offset=-1 in the arguments to io_uring_prep_{read,write} instead of offset=0 as used above? That seems to be more consistent with regular files, for which offset=0 has a different meaning. In particular, for programs that can read both from stdin or a regular file, that would obliterate the need to distinguish these cases (when reading the entire file sequentially).

What I meant is that in the program above, when changing
io_uring_prep_read(sqe, STDIN_FILENO, buf, bufsize, 0);

to
io_uring_prep_read(sqe, STDIN_FILENO, buf, bufsize, -1);

the cqe for this request returns with an EINVAL error code. I would have expected it to behave as described for IORING_FEAT_RW_CUR_POS in io_uring_setup(2). If it did so, the meaning of the offset -1 would also be equivalent for regular files and pipes, i.e. just read the whole data sequentially. The same happens with io_uring_prep_write.

The offset behavior should be the same as for eg pread and friends, is it not?

To be honest, I didn't check, but it makes sense that this should behave like pread(v2), however that treats the offset. I only searched for documentation, which is pretty sparse with regard to pipes. According to pread(3), the offset is ignored for pipes (but POSIX pread does not support -1 offsets for any kind of file). preadv2(2) doesn't really say anything about pipes in particular, and what an offset of -1 should do in their case.

I just checked pread and it behaves the same, you get -EINVAL if you specify an offset of -1. Let me check preadv2, but I'd be surprised if it's not the same.

In fact, the following (essentially your pipe eof testcase, but without io_uring) works with the offset -1
#define _GNU_SOURCE

#include <errno.h>
#include <pthread.h>
#include <stdio.h>
#include <string.h>
#include <sys/uio.h>
#include <unistd.h>
#include "liburing.h"

#define BUFSIZE	512

struct data {
	char *str;
	int fds[2];
};

static void *t(void *data)
{
	struct data *d = data;
	int ret;

	strcpy(d->str, "This is a test string");
	ret = write(d->fds[1], d->str, strlen(d->str));
	close(d->fds[1]);
	if (ret < 0)
		perror("write");

	return NULL;
}

int main(int argc, char *argv[])
{
	static char buf[BUFSIZE];
	static char str[BUFSIZE];
	pthread_t thread;
	struct data d;
	int ret;
	struct iovec iov;

	if (pipe(d.fds) < 0) {
		perror("pipe");
		return 1;
	}
	d.str = str;

	pthread_create(&thread, NULL, t, &d);

	while (1) {
		iov.iov_base = buf;
		iov.iov_len = BUFSIZE;
		/* signature: (fd, iov, iovcnt, offset, flags) */
		ret = preadv2(d.fds[0], &iov, 1, -1, 0);
		if (ret < 0) {
			fprintf(stderr, "Read error: %s\n", strerror(-ret));
			return 1;
		}
		if (ret == 0)
			break;
	}

	pthread_join(thread, NULL);
	return 0;
}

but fails with Read error: Operation not permitted when setting the offset to 0.

Yep same with preadv, agree with your test that preadv2 does work with -1. Pretty inconsistent...

But then, it's probably not io_urings job to do anything about it. Although io_uring_setup(2) claims

IORING_FEAT_RW_CUR_POS
If this flag is set, applications can specify offset == -1 with IORING_OP_{READV,WRITEV} , IORING_OP_{READ,WRITE}FIXED, and IORING_OP{READ,WRITE} to mean current file position, which behaves like preadv2(2) and pwritev2(2) with offset == -1. It'll  use  (and  update) the current file position. This obviously comes with the caveat that if the application has multiple reads or writes in flight, then the end result will not be as expected. This is similar to threads sharing a file descriptor and doing IO using the current file position.

Anyway, thanks again for all the support! I guess this issue can be closed :tada:

Well, we could still make it consistent. Should be pretty trivial:
diff --git a/fs/io_uring.c b/fs/io_uring.c
index d9b88644d5e8..bd2d8de3f2e8 100644
--- a/fs/io_uring.c
+++ b/fs/io_uring.c
@@ -2866,6 +2866,11 @@ static ssize_t io_import_iovec(int rw, struct io_kiocb *req,
 	return iov_iter_count(&req->io->rw.iter);
 }

+static inline loff_t *io_kiocb_ppos(struct kiocb *kiocb)
+{
+	return kiocb->ki_filp->f_mode & FMODE_STREAM ? NULL : &kiocb->ki_pos;
+}
+
 /*
  * For files that don't have ->read_iter() and ->write_iter(), handle them
  * by looping over ->read() or ->write() manually.
@@ -2901,10 +2906,10 @@ static ssize_t loop_rw_iter(int rw, struct file *file, struct kiocb *kiocb,

 		if (rw == READ) {
 			nr = file->f_op->read(file, iovec.iov_base,
-					      iovec.iov_len, &kiocb->ki_pos);
+					      iovec.iov_len, io_kiocb_ppos(kiocb));
 		} else {
 			nr = file->f_op->write(file, iovec.iov_base,
-					       iovec.iov_len, &kiocb->ki_pos);
+					       iovec.iov_len, io_kiocb_ppos(kiocb));
 		}

 		if (iov_iter_is_bvec(iter))
@@ -3139,7 +3144,7 @@ static int io_read(struct io_kiocb *req, bool force_nonblock,
 		goto copy_iov;

 	iov_count = iov_iter_count(iter);
-	ret = rw_verify_area(READ, req->file, &kiocb->ki_pos, iov_count);
+	ret = rw_verify_area(READ, req->file, io_kiocb_ppos(kiocb), iov_count);
 	if (unlikely(ret))
 		goto out_free;

@@ -3262,7 +3267,7 @@ static int io_write(struct io_kiocb *req, bool force_nonblock,
 		goto copy_iov;

 	iov_count = iov_iter_count(iter);
-	ret = rw_verify_area(WRITE, req->file, &kiocb->ki_pos, iov_count);
+	ret = rw_verify_area(WRITE, req->file, io_kiocb_ppos(kiocb), iov_count);
 	if (unlikely(ret))
 		goto out_free;

<EMAIL_ADDRESS>

Ignoring the offset entirely seems sensible :+1:,  even though preadv2 does fail with non-negative offset (probably it still attemps to seek in the file?).

Yep same with preadv, agree with your test that preadv2 does work with -1. Pretty inconsistent...

Actually, on my system, I can't make pread/preadv work at all, which would be consistent with their man pages:

The file offset is not changed by these system calls.  The file referred to by fd must be capable of seeking.

(from  preadv(2))

Actually, on my system, I can't make pread/preadv work at all, which would be consistent with their man pages:

Right, I didn't mean in terms of man pages, just in terms of behavior between the two.


Feature selection

I have uploaded here the latest feature_selection_analysis script for your review.

Looks like the PR is well submitted! =) I'll add some comments now.

Almost ready, @sahas111! I've added two more minor comments, then I will merge this.


Update conventions-and-technical-restrictions-f0f13bf.md

Support for substitutions depends on the task provider.

Closes #24

Hi Pieter,
Thanks for your feedback! We’ll look into it in more detail and come back to you.

Thank you for your valuable content contribution, @piejanssens! So that we can recognize your contribution in the SAP Community, please check your SAP Community user ID (this is a number) in your personal settings page and share it with us in a reply to this comment. Make sure you just include the number in the reply.
Your user ID is displayed as follows:
Change display name for User ID N
where N is your user ID. For example, 53 is the user ID of the user 'qmacro'.
Please note that we are currently refactoring our profile and badge system on the SAP Community, and will start assigning badges again when that's complete.

68447


Support .htpasswd format

We should probably not specify username and password on the command line but instead support something like the .htpasswd format that's used by web servers for basic authentication.
https://httpd.apache.org/docs/2.4/programs/htpasswd.html

Fixed by185b3dfabf3c3036d273b0dadc98fed13ed2c3aa


Deployment of demo

I think we should start putting this up so we can show people, and also so @tnum can review progress.
What's the state of deployment for Elm, Thomas? Where do you think we should host the demo? Is it a matter of bundling the Phoenix api with the Elm app and put it up on Heroku, for example?

As a start I would suggest to deploy the backend to Heroku and the frontend to GitHub Pages.
Reasoning:

Heroku is an easy and nice way to deploy Elixir and Postgres.
It would be possible to put the frontend code to Heroku too, including compilation of Elm code.
However, the code for one Heroku app has to be pushed from one git repo, I believe.
Currently backend and frontend are in separate repos. As the API may be used independently of the current frontend, it seems reasonable to keep the repos apart.
Deployed frontend code is static, so any static http hosting is fine (after local compilation).

Options for later:

Scripted synchronous deployment of both frontend and backend. May be based on GitHub tags.
Common deployment to Heroku (or another PaaS / cloud service) by means of an extra repo that pulls in the two original repos.

@oliverbarnes thoughts?
I will start with deploying the backend in https://github.com/oliverbarnes/participate-api/issues/35

Locally served frontend works with the Heroku-deployed backend.
Just set apiUrl ="https://participate-api.herokuapp.com"
@oliverbarnes For further tests and for deploying the frontend, could you please:

Public the branch gh-pages at GitHub Pages (in the settings of this repo), either at https://oliverbarnes.github.io/ if possible (not sure), otherwise at https://oliverbarnes.github.io/participate
Whitelist these domains (or URLs ?) in the Facebook app settings:

https://oliverbarnes.github.io/
https://oliverbarnes.github.io/participate
http://participate.thomasweiser.de/

Thanks!

Aside from Facebook authentication (see above): The test deployment at http://participate.thomasweiser.de/ seems to be working when I manually set the accessToken in session storage.

Done:

gh-pages published to https://oliverbarnes.github.io/participate/
Set up a new facebook app for this demo, with the above domain whitelisted and https://oliverbarnes.github.io/participate/facebook_redirect as the redirect url. Looks like I can only have one redirect url setup, btw. I'm sending you the app id and secret privately

Thanks!
Currently the app works only in a root directory. I will change that and deploy it to https://oliverbarnes.github.io/participate/

Hmm, this makes me wonder if it's not time to switch participate to its own organization, something I've been considering doing. Besides then having a https://participate.github.io where the app could run in root dir, it'd also be good for its future visibility. I might do this switch this weekend

Ok, created the organization but it looks like the github-pages is still published to a sub-directory, https://participateapp.github.io/web-client/, and that can't be changed as far as I can see. I'm updating the facebook app permissions

Cool! Having a dedicated organization is a great step for the project.
Sure, finally the app should be deployed to a URL without sub-dirs. But I think it's no harm if the app is prepared to work also from a sub-dir URL. It's already done actually (untested though).

Implemented in #62
Demo of current version: https://participateapp.github.io/

Cool, this is done!


Variables not expanded in repo names

I define variables both in templates and inline as you can see bellow:
variables:

name: infraRepoName
value: some-infra
template: vars_it.yml
template: vars_acc.yml

resources:
repositories:

repository: some-infra
type: git
name: project_name/$(infraRepoName)
ref: some-branch

The error I get:
The repository $(infraRepoName) in project project_name could not be retrieved. Verify the name and credentials being used.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 8a0499c0-bd1c-47a0-b472-54a5d6c159b4
Version Independent ID: ec3feae0-4f23-68ab-6307-75fde645468a
Content: Check out multiple repositories in your pipeline - Azure Pipelines
Content Source: docs/pipelines/repos/multi-repo-checkout.md
Product: devops
Technology: devops-cicd-repos
GitHub Login: @steved0x
Microsoft Alias: sdanie

This is not enabled;
https://developercommunity.visualstudio.com/idea/816606/allow-variables-at-resourcesrepositoriesrepository.html

As documented here, not supported.


correctly store user votes

@miltontony review please

@miltontony please help

@miltontony ready for review :)

@miltontony added functionality for ensuring user can only vote once

@miltontony changed the template. have a look?

cool, changed. @miltontony any ideas on how I should test the urls to make to coveralls pass?

@miltontony ready for review, also for advice on coveralls?

@miltontony ready fore review

:+1: :cake: :cake: :cake:


Use environment variables for required folders

JAYCONFIG is used for generate and migrate. env will only use the current folder.


[Enhancement] BERT pre-training data generation from sentencepiece vocab

Description

add sentencepiece vocabulary support
addresses #726
more documentation

Checklist
Essentials

[x] PR's title starts with a category (e.g. [BUGFIX], [MODEL], [TUTORIAL], [FEATURE], [DOC], etc)
[ ] Changes are complete (i.e. I finished coding on this PR)
[ ] All changes have test coverage
[ ] Code is well-documented

Changes

[ ] Feature1, tests, (and when applicable, API doc)
[ ] Feature2, tests, (and when applicable, API doc)

Comments

If this change is a backward incompatible change, why must this change be made.
Interesting edge cases to note here

Job PR-743/1 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-743/1/index.html

@davisliang FYI

Job PR-743/3 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-743/3/index.html

Job PR-743/4 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-743/4/index.html

Job PR-743/5 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-743/5/index.html

Job PR-743/6 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-743/6/index.html

Job PR-743/7 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-743/7/index.html

Job PR-743/8 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-743/8/index.html

Job PR-743/9 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-743/9/index.html

Job PR-743/10 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-743/10/index.html

Job PR-743/11 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-743/11/index.html

New create_pretraining_data.py seems slower than the previous version which do multiprocess during the whole reading input text and writing npz file procedure.

Job PR-743/12 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-743/12/index.html

Job PR-743/13 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-743/13/index.html

Job PR-743/15 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-743/15/index.html


Section "Kontakt z administratorem" accessible without login

 Issue by Margaret2015
Saturday Feb 27, 2016 at 00:17 GMT
Originally opened as https://github.com/stxnext-csr/volontulo/issues/551

When we click on "Kontakt z administratorem" then appears "Logowanie".
Contact with admin should be enabled without logging in.

There is an issue - in case using of using form as organization ("Kontaktuję się jako: organizacja"), user still gets email which states: "Kontakt wolontariusza z administratorem".

Work as expected.


pritunl-client: 1.3.4026.10 -> 1.3.4066.51

Bump printunl-client packages.
Also, add nix-update-script to derivations (to try to get this auto-updated by @r-ryantm

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[x] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

nixpkgs-review result
Generated using nixpkgs-review.
Command: nixpkgs-review pr 352578 --checkout commit

x86_64-linux

:white_check_mark: 1 package built:

pritunl-client

@NixOS/nixpkgs-merge-bot merge

nixpkgs-review result
Generated using nixpkgs-review.
Command: nixpkgs-review pr 352578 --checkout commit

x86_64-linux

:white_check_mark: 1 package built:

pritunl-client

@NickCao could you please help review this one?

I'm not entirely sure nix-update-script would work correctly in this kind of complex derivations, but giving it a try won't harm.


demos: Add MacOS surface output to vulkaninfo

Added output for a MacOSSurface in vulkaninfo
and modified the CMake files to include the
new files needed for this change:
metal_view.m Objective-C code to hook into Metal
metal_view.h metal_view.m's header file

I don't have a lot of Mac or Objective-C knowledge but from what I can tell this looks good to me.
With these changes, LVL and vulkaninfo still build and run on Linux and Windows.

@mikes-lunarg: Changes made.


PushNewNavigationServiceModal not showing the back button

Hi.
Here's the scenario.
I have a FreshTabbedFONavigationContainer with 3 tabs.
If I PushNewNavigationServiceModal from one of the 3 tabs, the new navigation page doesn't show the back button.
If I navigate deeper from one of the 3 first tabs with PushPageModel, then call PushNewNavigationServiceModal from the new page, it adds the back button and works as expected.
Any idea why and how to fix it?
Thanks

@dpedrinha If possible you should download the FreshMvvm source code and review the sample app, this will give you a better understanding of how to do Navigation in Xamarin.Forms and FreshMvvm. Otherwise you can provide a sample application and I can take a look.

@rid00z I can't test your samples because they are using .Net versions that I don't have installed and I can't install due to lack of space in my SSD. But I checked it and did my best to mimic what you did there.
So I made a sample app for you to see the problem, here's the link:
https://www.dropbox.com/s/1evujo6owbrv63r/navigationbackbuttontest.zip?dl=0
I haven't tested the iOS version because I don't have my Mac right now.
But i did test it on Android and Windows 10. On Android the problem I'm facing isn't a problem because it has a hardware back button. So the problem is on Windows and iOS versions.
You will notice that I made a custom NavigationContainer. And this is where the problem happens. If I open this NavigationContainer from the first page on my app, it doesn't add the back button to it, if I open it from a deeper page, it does appear.
On the first app page you have two buttons, one to open the custom NavContainer and another one to navigate deeper (MiddlePage).
If you navigate to the MiddlePage, there you can open the NavContainer too. If you do, the back button appears. If you do it from the first page, it doesn't.
And I know, you have a built-in tabbed navigation container but I need a custom one. I'm pretty sure the problem isn't there.

@rid00z Did you test the example?

I did as requested and created you a sample. Two months later and I didn't get any response. Not cool.

@dpedrinha I've reviewed the sample and it functions without issues. There will never be a NavigationBar when you push the CarouselFreshMVVMNavPage because it's not inside a Navigation page.

@rid00z so you are telling me that FreshTabbedFONavigationContainer is not a Navigation page?
If you look carefully, your statement that "there will never be a NavigationBar" is not true because the sample itself shows how the NavigationBar does appear if called from any page besides the first one. No new Navigation page created in between.


Integrate intellij-plugin-verifier

It would be nice if there was some kind of integration with the intellij-plugin-verifier to detect potential compatibility issues before the plugin is published. @serejke

Until this feature is added, it is possible to run the verifier as a regular Gradle task, as I did in one of my plugins:
https://github.com/bjansen/pebble-intellij/commit/72f3823f7b3dec7552e0dc15893057e2239edace
Replace "build/distributions/pebble-intellij.zip" with your actual distribution path, and configure extra IDEs to verify in gradle.properties:
pluginVerifierIdes=/Applications/WebStorm.app/Contents/,/Applications/IntelliJ IDEA CE.app/Contents/

@bjansen the task runs the verification using the very same IDEA that you use for building plugin, right? What's the point? What kind of incompatibilities do you want to find there?

You can add other IDEs in the pluginVerifierIdes property, as I explained in my previous comment.
But you're right about the IDE used for building the plugin, perhaps I shouldn't add it to the list of IDEs to be tested.

Got it. Well, if the plugin will download different IDE version to check, I'd prefer real testing against different IDEs, than just running verifier.
Related issue: https://github.com/JetBrains/gradle-intellij-plugin/issues/324.

Well the verifier can verify against non-Java IDEs. The verifier will hopefully soon be able to detect missing optional dependencies to avoid reporting false positives
If you run unit tests on the same non-Java IDEs, some will fail because features used in optional dependencies are missing, which will lead to false positives.

Makes sense. Thank you for details

I've written a workaround for this problem in my Randomness plugin.
The good part is that it will download the IDEs for you, so you don't have to pre-install them somewhere. Just copy the task from the link above, configure it (see below), and run it. That's it. This way, you don't have any OS-dependent paths in your configuration anywhere, and other people can replicate it without having to follow complicated instructions.
The bad part is that it stores the IDEs in a directory of your choice. A much better solution would be to store them in the Maven cache (which is already where the Gradle IntelliJ plugin stores them), but I couldn't figure out an easy way to do that.

You can configure it as follows:
runPluginVerifier {
    /// Required params
    // Name of the zip of your plugin in `build/distributions` (without `.zip`)
    pluginFileName = "intellij-randomness-$version"
    // The IDEs you want to verify against
    ides = ["idea-ideaIC-2018.1", "clion-clion-2019.3"]

    /// Optional params
    // Where to store the IDEs
    // Warning: If it's in build/, `gradle clean` will wipe the cache.
    ideCacheDir = file("$buildDir/ides/")
    // Whether to download the IDEs even if they're already in cache
    forceUpdateIdes = false
    // The version of the verifier plugin to use
    verifierVersion = "1.222"
}

A much better solution would be to store them in the Maven cache (which is already where the Gradle IntelliJ plugin stores them)

Perhaps you could use IdeaDependencyManager? Although it looks like it's adding a dependency to the project, which might not be what we want.

Perhaps you could use IdeaDependencyManager?

When I tried that I mistakenly believed it was a non-public class because IntelliJ couldn't import it. Manually writing the import works though, so I'll see if I can figure something out. It looks like I can set configureDefaultDependencies to false in the IntelliJPluginExtension to prevent it from adding dependencies.

I've updated the script by copying the essential parts of the IdeaDependencyManager into the script (which is only a few lines anyway). Like the manager, the dependency is added to a separate, detached configuration, so it won't bloat your distribution files.
I don't really plan on expanding or improving it much further because it is, after all, only a workaround for a problem that is much better solved by integrating it into the Gradle IntelliJ plugin.

Thanks @YannCebron for letting me know about this issue :raised_hands:
I just included in build.gradle:
apply from:'https://raw.githubusercontent.com/FWDekker/intellij-randomness/master/gradle/scripts/verifier.gradle'

runPluginVerifier {
    pluginFileName = "<name>-<version>"
    ides = ["IC-2019.3.5"]
    verifierVersion = "1.241"
}

Then:
./gradlew runPluginVerifier

it works like a charm. Thanks @FWDekker :clap:

Integration with IntelliJ Plugin Verifier is now available through the runPluginVerifier task.
Check the Plugin Verifier DSL README section for more details.

Awesome, @hsz!
Accordingly, I think it's time to deprecate my workaround script. Because some projects import the script directly from my repository, I will leave the script untouched for one year, but will remove it after that. I have added a notice of deprecation when you run my runPluginVerifier task. Best I could think of was adding a println; no idea how to add a formal deprecation notice in Gradle. (Does anyone else know?)
verifier.gradle will be removed from the intellij-randomness repository on 2021-10-29.
Meanwhile, I will no longer fix bugs or add features to my script.

Thanks, @FWDekker - that's a nice move. You may try with logger.warn().
Then - when you'll decide to shut down that script, simply throw an exception instead of removing it.
Thanks for your work, Felix!

@hsz Good idea! I've replaced it with a logger.warn. I'll probably throw an exception instead of removing the script once I delete it, though I do plan on removing it entirely eventually.


Custom view with custom layout

Feature
What's the current behavior?
Views layout is not flexible. For example, TimeGrid component using layout where Resources placed on top, and times on left, and DayColumn component as a columns.
What's the expected behavior?

There is no easy way to implement layout like this.
Maybe someone help me with this question?

@aleksays did you find any solution to that? Looks like you already closed the issue.

@aleksays did you find any solution to that? Looks like you already closed the issue.

@aleksays I need something exactly like that, any solutions? Is there some implementation in another library? I don't want to go full custom for this if it can be done.


add components-path references

This PR updates the docs to account for the fact the components dir is no longer automatically created.

Thanks for doing this!


make list page use psacingClasses syntax

What is this PR trying to achieve?
Seems we forgot to update this template.
Having typed templates would have picked up on this post haste.

Closed in favour of #686


Add support for changing app language

Have you thought about providing translation support?
I am looking for an rss app for our staff and would be happy to provide German and maybe polish translations

Yes! Thank you. Right now you can find the English version of the strings here. What I would recommend doing is, creating a Google Sheet with key and translated string for the language. I can then create the Kotlin class.
Or if you are comfortable cloning the project and creating new Kotlin classes, that would great as well. Here is the process.

Edit the Locales object in TwineStrings.kt to add a new locale ISO tag. In your case de for Germany and pl for Polish.

object Locales {
  const val EN = "en"
  const val DE = "de"
  // more locales
}

Then create a new file under the same package as EnTwineStrings, you just have to attach the ISO prefix to the TwineStrings name. So, we will have DeTwineStrings.

In that file create a new variable called DeTwineStrings and initialise the TwineStrings data class to add your translations.

@LyricistStrings(languageTag = Locales.DE, default = false)
val DeTwineStrings = TwineStrings(
  // translations
)

If this process is a lot, feel free to fallback to just providing me with the Google Sheet, and I can make these changes.

Thanks for the fast reply.
I am on vacation soon but will look into it when I get back

I was able to raise a PR for addition of a german translation

I would add support for other languages at the weekend


AssetLib: Long plugin name break the UI

Godot version
v4.1.1.stable.arch_linux
System information
OpenGL API 4.2 (Core Profile) Mesa 23.1.4 - Compatibility - Using Device: Intel - Mesa Intel(R) HD Graphics 2500 (IVB GT1)
Issue description
The AssetLib container is set to the size of their children and will automatic resize itself to display all the text, ignore user and windows manager setting.

Steps to reproduce
Find some plugin that has very long name like VRM Importer and MToon Shader - Humanoid 3d Avatar Model Format
Minimal reproduction project
n/a

The text should be made to wrap with an ellipsis so it doesn't exceed the column width.

I can't even get out of AssetLib without making maximizing the Godot window.

Sorry for the long addon name. I submitted an edit for my addon to make it a bit shorter: "VRM Importer for 3D Avatars and MToon Shader", though I think even then it needs line wrapping or ...

Hi, I'm hoping to make my first engine contribution, and I'd like to take a shot at fixing this. I'll start working on a PR if this isn't already being worked on. :)

@haruyou27 I submitted a PR #80555 that solves the problem of screen overflow on my local machine, but I was unable to reproduce the differently sized columns shown in your screenshots on my system on both the current master branch and 4.1.1 stable. I'm using Linux though, so there might be something window manager or DPI related going on here as well.
Can you try out the PR, and if the fix there doesn't solve the problem on your system, can you provide more information about your local machine: OS, Window Manager/DE (if applicable), monitor resolution and DPI?
Thanks!

@haruyou27 Sorry, please disregard my previous comment. Working on a different solution for the PR.

I have a dual monitor setup, and the issue only happens if I previously closed the godot editor while looking at AssetLib, so the AssetLib is focused when the editor starts up. In this case, it seems to be the width of my largest monitor, even on my small monitor. There are no long plugin names, so I don't think that's actually the issue.
See here when maximized on my 1080p monitor:

And now here I dragged it onto my 4k monitor:

Ok actually it's just horridly inconsistent. every time I test it's different. Also, if I try to resize the window to be big enough to contain the inspector, the assetLib just keeps popping more items onto the first two rows, and it sort of ratchets bigger and bigger and won't get smaller. Now I can't even fit the godot window on my 4k monitor without losing the inspector.

I have a dual monitor setup, and the issue only happens if I previously closed the godot editor while looking at AssetLib, so the AssetLib is focused when the editor starts up. In this case, it seems to be the width of my largest monitor, even on my small monitor. There are no long plugin names, so I don't think that's actually the issue.

This might be fixed by the PR I submitted for this. The AssetLib plugin calculates the number of columns based on the width of its control when it is created, but the width of the individual columns were being expanded to the width of their contents because of the long plugin names, so the overall width would extend past the screen width.
In your screenshots, the columns have varying widths, so I think the PR might fix it since it clamps the column contents to the uniform calculated column width, but if the window is being resized to the width of the larger screen, it might be a problem with the width reported by the get_size method of Control, which would probably be beyond the scope of this issue and affect other parts of the UI as well. I believe the get_size method returns a value from some kind of internal cache, so that might be what's causing it to report the wrong value on reload, but I'm not very familiar with the internals of the UI system, so I can't say for sure. A more experienced contributor would probably know. :)
Hopefully, the PR fixes this on the dual resolution setup, but if not, it might be a separate issue.
I have a two-monitor setup as well, but both of mine are 1080p, so I can't test this myself. :(


feat: get and delete checkin endpoints

Brief description. What is this change?

Get and Delete CheckIn endpoint

Implementation description. How did you make this change?

API endpoints for the get (GET) and delete (DELETE) were created, and corresponding services getAllCheckIns(), getCheckInsById(id: string), getCheckInsByVolunteerId(volunteerId: string), deleteCheckInById(id: string), deleteCheckInsByDateRange(startDate: string, endDate: string).

Steps to test
Get functions:

Insert data to PgAdmin under checkIn table (right click on checkIn table -> scripts -> INSERT Script)
For getting all: localhost:5000/checkIn and ensure it gets the right response
For getting by id: localhost:5000/checkIn/<id>
For getting by volunteerId: localhost:5000/checkIn?volunteerId=<volunteerId>
Make sure the get all, get by id, get by volunteer id returns the correct responses.

Delete functions:

insert data like above
For deleting by id, localhost:5000/checkin/<id> and ensures it gets deleted from table
For deleting by date range  localhost:5000//checkin?startDate=<startDate>?endDate=<endDate> -> make sure the parameters are correct in Postman before running the request
^ example:  localhost:5000//checkin?startDate=2022-01-31T05:00:00.000Z?endDate=2022-02-31T05:00:00.000Z

Checklist

[x] My PR name is descriptive and in imperative tense
[x] My commit messages follow conventional commits and are descriptive. My commits are atomic and trivial commits are squashed or fixup'd into non-trivial commits
[x] I have run the appropriate linter(s)
[ ] I have requested a review from the PL, as well as other devs who have background knowledge on this PR or who will be building on top of this PR
[x] The appropriate tests if necessary have been written

please do not review the "do not merge" label - this ticket is meant to merge after another PR and will need to address merge conflicts after :)


[7.x] Replace bootstrap-select with Tom Select

https://tom-select.js.org/

Already imlemented in next branch.


Handling Ports for DoltLab Deployment on AWS Private Network with Load Balancer

Hello,
I am attempting to deploy DoltLab on an AWS EC2 instance that is situated within a private network. For security reasons, this instance is not directly exposed to the internet. Instead, we are using an AWS Load Balancer to forward requests to the instance.
My concern is regarding the handling of specific ports that DoltLab requires for operation, specifically the gRPC port (50051) and another port (4321). Given the setup with the AWS Load Balancer, how can I properly configure the forwarding rules to ensure that DoltLab functions correctly and securely?
Any guidance or examples on setting up DoltLab in such an environment would be greatly appreciated.
Thank you!

@rinshadka this is covered in the the documentation here. Closing issue for now.


[core] Implement forwardRef

Hi,
First of all, thank for your work !
I would like to know if is it possible to pass ref to my DatePicker ?
In my case, I use your Date Picker in a form using react-final-form.
I have wrappers for each of my inputs to customize the error message. However, I am obliged to pass a ref to each of my components in order to recover the size of said component and display a popper of this size.
Currently, from what I have seen in the doc, it is impossible for me to pass a reference to my DatePicker.
Do you have a solution to offer me or can you add the possibility to pass some?
Thank you for your time and answer !

Did you find a solution on this?

You are right, all the public components should leverage React.forwardRef.

@LucaPizzagalli The implementation of such behavior should be relatively straightforward. Is this something you would want to work on? :) For instance, I would envision something similar to:
index d41a1e85..02a042f2 100644
--- a/lib/src/views/Calendar/Calendar.tsx
+++ b/lib/src/views/Calendar/Calendar.tsx
@@ -92,26 +92,29 @@ export const useStyles = makeStyles(theme => ({
   },
 }));

-export const Calendar: React.FC<CalendarProps> = ({
-  date,
-  isMonthSwitchingAnimating,
-  onMonthSwitchingAnimationEnd,
-  focusedDay,
-  changeFocusedDay,
-  onChange,
-  minDate,
-  maxDate,
-  slideDirection,
-  disableFuture,
-  disablePast,
-  currentMonth,
-  renderDay,
-  reduceAnimations,
-  allowKeyboardControl,
-  isDateDisabled,
-  disableHighlightToday,
-  showDaysOutsideCurrentMonth,
-}) => {
+export const Calendar = React.forwardRef(
+  (props: CalendarProps, ref?: React.Ref<HTMLDivElement>) => {
+  const {
+    date,
+    isMonthSwitchingAnimating,
+    onMonthSwitchingAnimationEnd,
+    focusedDay,
+    changeFocusedDay,
+    onChange,
+    minDate,
+    maxDate,
+    slideDirection,
+    disableFuture,
+    disablePast,
+    currentMonth,
+    renderDay,
+    reduceAnimations,
+    allowKeyboardControl,
+    isDateDisabled,
+    disableHighlightToday,
+    showDaysOutsideCurrentMonth,
+  } = props;
+
   const now = useNow();
   const utils = useUtils();
   const theme = useTheme();
@@ -159,7 +162,7 @@ export const Calendar: React.FC<CalendarProps> = ({

   return (
     <>
-

+

         {utils.getWeekdays().map((day, i) => (
           <Typography
             aria-hidden


Fix renewalTime skew issue

See #4345 and Slack for context.
It appears like in cases where a certificate has a slightly skewed duration, the certs don't get renewed.
This seems to happen because the renewal time value on the Certificate is truncated to the nearest second, whereas the function we use to calculate renewal time, returns a value to higher precision.
We use the truncated renewal time from the certificate's status when we re-queue Certificate for renewal here, but then when it has been popped from the queue at the set renewal time, we re-calculate the renewal time using our custom function so we get the higher precison value, which in case of a skewed duration will be some micro/milli/nano-seconds later from the current time, so we determine that the cert does not need re-issuing now. After that the Certificate will also not get re-queued again since that part looks at the renewal time on the Certificate's status and determines that it is being re-issued now.
I have reproduced this issue by modifying the self-signed issuer to issue certs with skewed duration (the different second values for notAfter, notBefore):
  notAfter: "2021-08-20T14:23:10Z"
  notBefore: "2021-08-20T13:23:11Z"
  renewalTime: "2021-08-20T14:03:10Z"

In this case the renewalTime returned by our function that calculates renewal time is 2021-08-20 14:03:10.333333334 +0000 UTC"=null which is 0.333333334s later than the renewalTime read from Certificate's status.
I verified that once this fix is applied, the renewal happens as expected.
What this PR does / why we need it:
This PR truncates the renewalTime returned by RenewalTime function to the nearest second, so it will always be the same value as that stored on Certificate's status.
Special notes for your reviewer:

This issue could have also been fixed by re-calculating the renewal time also when re-scheduling, but I think this would be extra work and isn't needed
We recalculate the renewal time when determining whether cert needs re-issuing now since v1.3.0 - see #3665 in short it is needed to prevent multiple issuances when readiness controller has not yet updated renewal time on certificate's status just after an issuance

Fixes renewal time issue for certs with skewed duration period.

fixes #4345
Signed-off-by: irbekrm<EMAIL_ADDRESS>/kind bug

FWIW, I tried issuing a cert from LetsEncrypt staging endpoint and got the 1 second skew:
  notAfter: "2021-11-18T15:46:00Z"
  notBefore: "2021-08-20T15:46:01Z"
  renewalTime: "2021-10-19T15:46:00Z"

Not sure if this is always the case with LetsEncrypt yet and whether that'd mean that many certs issued after v1.3.0 would have this issue- I would have thought this would have been noticed.

Firstly: this is a brilliant description of the issue; really thorough :clap:
Second: am I correct in saying this would also be a bugfix for 1.4 and should be cherrypicked into release-1.4?
I'll take another look at all this on Monday :+1:

Thank you @SgtCoDFish 🙏🏼 🙏🏼

Second: am I correct in saying this would also be a bugfix for 1.4 and should be cherrypicked into release-1.4?

I believe so (it would have been introduced in v1.3). I am somewhat surprised that this issue has not been reported more often, but I was able to reproduce it and to me it seems to be a real bug. The user in question saw this with their LetsEncrypt certs.

/cherry-pick release-1.4

Thanks for the review and the good suggestions @SgtCoDFish ! I've tried to address those.
I guess I need to do this for the 1.5 cherry-pick too
/cherry-pick release-1.5

presumably a flake
/test pull-cert-manager-e2e-v1-22

This appeared to not have created the second cherry-pick PR for release-1.5 and I seem to remember someone mentioning an issue about this.

/cherry-pick release-1.5


Add Flutter template

Can we please have the flutter template files here too?

Hi there! We're working on it! Unfortunately the Flutter template is the only one that's a bit more involved than just a simple template. We're working on bringing the Flutter environment 100% onto our templating system, which is taking some time.

/cc @davideast @kirupa for vis

@romannurik  Flutter, please check this link , I found most of part useless,

@mg3994 We have a number of alternative/secondary approaches to spinning up Flutter workspaces that are unofficial/experimental (that's one of them) ... but the default "new Flutter app" experience on IDX isn't currently replicable purely via custom templates unfortunately.

/cc @rodydavis

The flutter template I created can get pretty close to the environment with the default flutter template but still not 100%.

yeah, my understanding also is the default flutter template is hyper-optimized for startup performance, using mechanisms unavailable to other templates at the moment.

Yep but if speed is not the concern then the template I shared will still have flutter fully working with the previews on Android and web 👍🏼

I saw that nixpkg flutter package is not up to date, interesting how this is solved in idx, probably some custom nix flake?

I only use the flutter nix to bootstrap the files.

Also the clarify the flutter template I create is just as fast running and is still super quick opening.
It adds a lot of the flutter create commands to the template instead of caching the flutter counter app only.


Spurious blank lines before some headers in Markdown output

Whilst this issue is perhaps relatively minor, it was suggested we report on it here so it can be tracked.  Currently it seems there are two (as opposed to the expected one) blank lines before the following headers in the Markdown output (for each endpoint)...

Optional Parameters
Curl Example
Response Example

I just made a PR to add blank lines around tables and noticed these, but I'm not familiar enough with the template syntax to work out exactly where they're coming from.

I think this comes from the sub-template rendering of a collection of objects. Each sub-template ends in a new line (so that there will be white space between it and it's siblings). There is also a new line AFTER the rendering of the whole collection, which combined with the newline after the last child results in doubled newlines. Note I haven't dug deeply in to this, but I've run into this sort of issue enough times in the past to feel it is pretty likely to be at blame.
The naive solution would be to remove the newline after the collection, but this would then fail in the empty collection case (perhaps we would not get here in that case though, so it might be safe). In some contexts you (unfortunately) have to add this line conditionally based on whether the collection is empty or else conditionally add the trailing whitespace to the collection object based on whether it is the last element in the collection.
Hopefully that helps provide some context, if you'd like to work on it that would be great. Otherwise I may return to it, but it might not be any time soon. Thanks!


LayoutDirection - RTL support

I have read the README file from top to bottom, yet i couldnt find anything regarding layout direction, so I just want to clarify whether it is possible for me to apply rtl support for dialog boxes if so could you please add a small tutorial on how to apply rtl to dialog boxes to the README file.

There's no setting for it, RTL is applied automatically for Android versions that support it (4.2 Jellybean and above, I believe). I thoroughly tested it before and it works like it should.
Keep in mind Android automatically applies RTL layouts on devices with RTL languages, so your entire phone has to be switched to a language like Arabic.

https://github.com/afollestad/material-dialogs/releases/tag/<IP_ADDRESS>


-db-password option is inappropriately interpolated

It appears the value of -db-password (whether passed on command line or by environment variable) is being interpolated when injected into the connection string.
So far we've had trouble with %% and other sprintf-like strings, as well as #. I haven't looked into the code, so can't give a full scope, but here are some illustrative command line attempts:
$ PROMSCALE_DB_HOST=<IP_ADDRESS> PROMSCALE_DB_PORT=5432 PROMSCALE_DB_NAME=prometheus PROMSCALE_DB_USER=postgres PROMSCALE_DB_SSL_MODE=prefer PROMSCALE_WEB_LISTEN_ADDRESS=:9201 promscale -db-password '%s123'
... err="cannot parse `postgresql://postgres:xxxxxx<EMAIL_ADDRESS>failed to parse as URL (parse \"postgresql://postgres:%s123<EMAIL_ADDRESS>invalid URL escape \"%s1\")"

$ ... promscale -db-password '%%s123'
... err="cannot parse `postgresql://postgres:xxxxxx<EMAIL_ADDRESS>failed to parse as URL (parse \"postgresql://postgres:%%s123<EMAIL_ADDRESS>invalid URL escape \"%%s\")"

$ ... promscale -db-password '\%s123'
... err="cannot parse `postgresql://postgres:xxxxxx<EMAIL_ADDRESS>failed to parse as URL (parse \"postgresql://postgres:\\\\%s123<EMAIL_ADDRESS>net/url: invalid userinfo)"

$ ... promscale -db-password '123#456'
... err="failed to connect to `host=postgres user=promscale database=`: hostname resolving error (lookup postgres on <nameserver_ip>:53: no such host)"

$ ... promscale -db-password '123\#456'
... err="cannot parse `postgresql://postgres:xxxxxx<EMAIL_ADDRESS>failed to parse as URL (parse \"postgresql://postgres:123\\\\\": invalid port \":123\\\\\" after host)"

Expected behavior:
In assembling the db url, the -db-password value should be url-encoded directly from the value received with no other interpolation.

Thank you for reporting the issue. Masking passwords always creates some corner cases when using special symbols. We will take a look on this more.
cc @antekresic

https://www.postgresql.org/docs/current/libpq-connect.html#id-<IP_ADDRESS>.3.6

The connection URI needs to be encoded with percent-encoding if it includes symbols with special meaning in any of its parts.

https://datatracker.ietf.org/doc/html/rfc3986#section-2.1

This issue is not limited to just the password. Database names and user names can also contain characters that need to be url encoded.

@babbottscott this issue is fixed and should go out with the next release


chore: Improved Error Handling and User Messages

Description
This pull request enhances the error handling and user messaging in the codebase. The following improvements have been made:

Enhanced Error Handling: I've introduced more robust error handling to provide detailed error messages when issues occur, making it easier to identify and troubleshoot problems.
Informative User Messages: The code now provides informative messages to the user, improving the overall user experience. Users will receive clear and helpful messages to guide them through the application's processes.
Code Readability: While the primary focus of this pull request is on error handling and user messages, I've also maintained good code formatting and readability.

These improvements aim to make the application more user-friendly and maintainable. Feel free to review the changes, and your feedback is highly appreciated.

@Yisaer I request you to please have a look at this and let me know if the changes are relevant

@Yisaer ,I request you that if you are satisfied with the changes, could you please merge it?

Merged, thank you!

Hey @adarsh-jha-dev Could you pls reach out to<EMAIL_ADDRESS>or alternatively provide me with yours. Needed to connect with you regarding hacktoberfest goodies :)

Hey @adarsh-jha-dev Could you pls reach out to<EMAIL_ADDRESS>or alternatively provide me with yours. Needed to connect with you regarding hacktoberfest goodies :)

Could you please let me know what details I have to give for the Hacktoberfest goodies?

Hey @adarsh-jha-dev Could you pls reach out to<EMAIL_ADDRESS>or alternatively provide me with yours. Needed to connect with you regarding hacktoberfest goodies :)

Hey @manansethi23 , I have sent you an email with all my details , I request you to please let me know if I have missed something, my email<EMAIL_ADDRESS>


WIP : Fixes #577 : Add high contrast display

css variable for colors
detect if high contrast is active
take high contrast color and put it in css

This is a content-rich PR which unfortunately now suffers from conflicts due to recent codebase changes (notably: strongly-typed Redux actions). Let's keep this PR open to keep in mind this great start, but obviously the PR needs significant revisiting (and it was still a WIP anyway).