Sendfile support

Hi!
What's the best way to implement sendfile support? Does it belong to rotor-stream?
"Basic" functionality is wrapping libc::sendfile. But how to use it in rotor (-stream/-http)?
All the rotor-http stuff is based on the Buf abstraction - you can't use that with sendfile.
To send a meaningful HTTP response, we first need to send HTTP Status, then some headers, then call sendfile. In the case of chunked responses, we might also need send some other arbitrary data afterwards. So we need an abstraction over Buf and sendfile() data.
Proposal: "Write" buffers (e.g. Response in the server) use OutputBuf, which is a Vec<enum of Buf and Sendfile>. If you write data to an OutputBuf:

If last elem in Vec is Buf: just write to buf
Else: Append a Buf and write to that.
Similarly, if you call sendfile() on an OutputBuf, it always appends another Sendfile object.

Just a rough proposal from skimming through rotor-http and rotor-stream. In the blog posts, you mentioned you weren't sure yet how to enable sendfile. I personally don't like having the Vec<Buf> thing, but I see no alternative to allow interleaving sendfile & arbitrary sending of data.
(Maybe as an optimization, we could have an enum that's only Buf or only Sendfile or Vec<OutputBuf> so we don't have the double indirection of Vec<Buf> where Buf is Vec<u8> when we don't need it. But let's decide on the simple approach first)
What do you think? Does this make any sense?
Cheers,
Kosta

I'm a little bit in a hurry now, I'll make more thoughtful response later.
But as an alternative rotor-stream exposes socket directly in the transport interface. So probably we can use sendfile right on the socket, without any changes of the buffer. If there is something in the buffer you just wait for Flush event. This is fine in linux, as you don't have a way to combine sendfile with data from memory anyway.
But I'm not sure how well does it play with rotor-http. Probably it will make the latter much less safe to use.

Yeah, basically what I describe can be rephrased to "queue up Bufs and Sendfile and write the next one on Flush". I'll try to prototype something.

I'm not sure I was clear enough. I mean in rotor-stream we can do this without a Vec<Buf> using purely state machine approach. Here is an altered example from rotor-stream: https://gist.github.com/tailhook/7f9df0d53818549d40d7902779758771 (It's untested because I don't know is there any interface for sendfile), but I think the idea is pretty clear.
I think in rotor-http there is no flow control at the moment. I mean there is no way to wait for your data to be sent, the response can only be driven by the timeouts or external wakeups. This should be fixed, for example, by adding  continue_response method to rotor_http::server::Server which is called in bytes_flushed.
When the flow control problem is fixed, we just need a wrapper around Transport::sendfile to rotor_http::server::Response which ensures that Content-Length matches the length of file or some similar thing. (We should also support range requests)

Hm, another alternative, is just to add another option to an Expectation::ExpectFileSent { file: File, offset: u64, bytes: u64 }. Unfortunately, this either means a File is a single type that you can send from, or there is a Box<AsRawFd> which is excessive memory allocation. It's also not clear how to pass ownership of the file back to the application (so file descriptor can be cached) if needed. Similarly this means you can't use single file descriptor to send a 1Gb file to two clients fetching the same file.
On the other hand this makes less iterations back and forth between rotor-stream and the application. I.e. application can say: send this 4Gb file, everything following is a job of the rotor-stream.
The question is: is there any other use cases for sendfile except HTTP static file serving? This may guide our decision a little bit better.

Haha, yeah I noticed after your reply that my "solution" was to introduce a very weird state machine into the Buf object. Maybe not the smartest idea :)
I'm currently trying to prototype using sendfile at all. I'll try to look into Expectations, too.
I think serving static files is the main use case for sendfile(). That's also the feature I'm interested in :) But I think a good test to re-use this for other purposes is e.g. sending some bytes, then a file, then some more bytes, then another file. If that is possible, I think the API should be pretty general-purpose.
Currently, I'm struggling at knowing when I am done with writing to socket. That's outside the "server" part. I hope that I'll be able get to that tomorrow.

@kosta, do you already have a wrapper for a sendfile itself? (or is it already in mio/net2?)

Not really. I'm using libc directly at the moment. I looked in mio but wasn't aware of net2, so I didn't see anything. I might be able to take a look tonight, otherwise tomorrow.

Doesn't look it it. At least git grep sendfile doesn't find anything.

I mean you should write that first. This will make it available on in transport.socket() directly.
Then it's probably good idea to prototype it on http_server::server::Request directly. When that interface is made more or less clean, it would be easier to find out whether it's worth to add it to rotor-stream.

Do you mean rotor_http::server::Response?
Ok, I'll try to look into adding sendfile to net2. The linux API is pretty straightforward, but the MacOS sendfile() API wants a non-null pointer to a size_t that it will write the number of bytes written to. So using sendfile() requires an allocation that can only be free'd after sendfile() is done. Not sure how to abstract that in a safe way in net2. On the other hand, maybe focus on linux and worry about Mac later :)

MacOS sendfile() API wants a non-null pointer to a size_t that it will write the number of bytes written to

It's fine to put it on stack. sendfile is already done when it exited from the function call.

Do you mean rotor_http::server::Response?

Sure, I mean the response :)

Ah thank you, that makes much more sense and is much easier to use :)

Hi!
Sorry for being unresponsive for so long.
If I have a server that uses rotor-stream and writes to that, what state is the stream wrapper in when the writing is done? I.e. Can the server request handler be called back when writing is done?
I think that is crucial to have anyway to be able to
write foo to Response
do something else
write bar to Response
If that is possible, we can leverage the same thing to be called back after sendfile is done. Or if you want to write something and then call sendfile, then call sendfile again and then write something else can do that as well.
Does that make any sense? :)
Cheers,
Kosta

Note: I originally posted this with the wrong account. Sorry for the confusion :)
Hi!
Sorry for being unresponsive for so long.
If I have a server that uses rotor-stream and writes to that, what state is the stream wrapper in when the writing is done? I.e. Can the server request handler be called back when writing is done?
I think that is crucial to have anyway to be able to
write foo to Response
do something else
write bar to Response
If that is possible, we can leverage the same thing to be called back after sendfile is done. E.g. you write something and then call sendfile, then call sendfile again and then write something else.
Does that make any sense? :)
Cheers,
Kosta

Closing, I think what I'm asking for is actually a Future... :)


extensionService.activateByEvent should return even if some extension fails activation

Have some extension which fails to activate  on event myevent and crashes in its activation
Have the following code in vsocde

 this.extensionService.activateByEvent('myevent').then(() => console.log('hi'), err => console.log('err');

Notice that both code paths are not printed and the activateByEvent call does not return

I think this has been fixed in the meantime. Here is what I have tried:

package.json:

{
    "publisher": "alex",
    "name": "62660",
    "version": "0.0.0",
    "engines": {
        "vscode": "^1.0.0"
    },
    "activationEvents": [
        "special"
    ],
    "main": "index.js"
}

index.js:

const vscode = require('vscode');

exports.activate = function (ctx) {
    console.log(`62660 activated!`);
    // throw new Error('an error!');
    return Promise.reject('an error!');
}

Then, somewhere in the renderer code:
this._extensionService.activateByEvent('special').then(() => console.log('hi'), err => console.log('err'));

I have tried both variants, of return a rejected promise and throwing an error. In both cases I see hi printed.


#291 feat: vcs url for gopkg

It's an interesting idea. Can we not do this as part of FETCH_LICENSE for go?

Done
+tests for gopkg with FETCH_LICENSE=1

Could you kindly run pnpm run lint && pnpm run gen-types && pnpm test and commit?

Thank you so much for your patience and support!


Cannot install app - Failed to set data for 'AppsUseLightTheme'

Throws the attached error. Failed to set data for 'AppsUseLightTheme'

Same, Windows 10 LTSB 2016


Creating libraries of real data sets

@kbarbary has already begun creating the code for the library of real data sets in Python under the name sndatasets.
Kyle may need some assistance in this effort.

link: https://github.com/sncosmo/sndatasets


Cypress test that data is saved after batch closeDate

tested in 01_Normal_Paths_Omnibus.cy

testing that data is saved on manual batch close. removed auto batch close.


Allow selectrum-insert-current-candidate to take a prefix arg

OK, I've addressed your comments.


refactor(GRO-388): rebuilds view in room

Drafting this while I write some specs.
The architecture here is: There's a component called ViewInRoomRoom which displays the floor/bench and will scale itself and everything in it to fit in the browser, the same way object-fit: contain would. Nested in that is both the actual artwork image and the 8ft scale. We know that the scale should be the width of the bench and comes out to eight feet, so we use that to convert the artworks from centimeters into pixels. With the hanging level, we hardcode the distance from the ceiling to where the floor starts and then use that same scale conversion to measure out the hanging heights from the floor.

I basically can't get opacity animations to run on this container in browsers other than Chrome, so I've temporarily disabled the transition and I can tackle that as it's own ticket. I think the browsers are just dropping all of the transition frames. Interestingly even just doing a simple fade in with react-transition-group doesn't animate in Safari/Firefox/Mobile Safari — I have a few theories but I'd rather not get too bogged down when I think this is shippable sans-transition. It already fixes a number of bugs with some artworks being misshapen and corrects both the scaling and the hanging at eye level distance.
I also moved the close button to the right to match it up with the zoom view's close button.
Here's a comparison (new on left, existing on right):

Update: a quick theory I had was that since the room actually extends extremely far outside the browser window that this was the cause of the poor performance. Wrapping it in an overflow: hidden div and we've got the opacity transition back. 🧠


Fix available display modes

SDL display modes are not accurate right now.

Won't do for now

Won't do for now


chore: Reference Github code in Docusaurus

SUMMARY
This is a POC for referencing Github code in Docusaurus.
AFTER
https://user-images.githubusercontent.com/60598000/151806961-03c821e4-27d1-4e0c-af06-5967e3db1121.mp4
ADDITIONAL INFORMATION

[ ] Has associated issue:
[ ] Required feature flags:
[ ] Changes UI
[ ] Includes DB Migration (follow approval process in SIP-59)

[ ] Migration is atomic, supports rollback & is backwards-compatible
[ ] Confirm DB migration upgrade and downgrade tested
[ ] Runtime estimates and downtime expectations provided

[ ] Introduces new feature or API
[ ] Removes existing feature or API

I like the direction, but I am afraid about the maintenance of that solution in long term. We have a hard-coded commit ID (or git tag) and a hard-coded line number. This can be easily outdated:

the example will be updated and the commit IT will not be updated to point to the new code,
the reference will be updated with the new code, but the lines will not be changed, e.g. extended with an additional enter added.

As for the first case, we may use the version identifier (git tag) instead of commit in the future. However, then the second case is then even more likely.
To reduce this case scenario Apache Airflow uses comments in the referenced file ( https://github.com/apache/airflow/blob/main/docs/apache-airflow/tutorial_taskflow_api.rst ), predicable comment structure (tags eg. for examples is added before this is required in the documentation). It allows also check that references on CI.

As the POC was successful I am removing the POC file and moving this PR to mergeable


Add the ability to use args on swift lanes

New Issue Checklist

[x] Updated fastlane to the latest version
[x] I read the Contribution Guidelines
[x] I read docs.fastlane.tools
[x] I searched for existing GitHub issues

Issue Description
It would be a good idea to let lanes take args in swift modes like this:
class Fastfile: LaneFile {
    func hockey(client: String, env: String) { //lane takes here 2 args
    desc("Push a new beta build to TestFlight")
        buildApp(workspace: "myspace.xcworkspace", scheme: schemeFor(env, client))
        uploadToHockeyApp(Credentials.shared.username, clientFor(client))
    }
}

Environment
Stack

Key
Value

OS
10.13.2

Ruby
2.3.3

Bundler?
false

Git
git version 2.14.3 (Apple Git-98)

Installation Source
/usr/local/bin/fastlane

Host
Mac OS X 10.13.2 (17C88)

Ruby Lib Dir
/System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib

OpenSSL Version
LibreSSL 2.2.7

Is contained
false

Is homebrew
false

Is installed via Fabric.app
false

Xcode Path
/Applications/Xcode.app/Contents/Developer/

Xcode Version
9.2

System Locale

Variable
Value

LANG
fr_FR.UTF-8
✅

LC_ALL

LANGUAGE

fastlane gems

Gem
Version
Update-Status

fastlane
2.79.0
✅ Up-To-Date

Loaded fastlane plugins:

Plugin
Version
Update-Status

fastlane-plugin-mobile_center
0.1.5
✅ Up-To-Date

fastlane-plugin-versioning
0.3.4
✅ Up-To-Date

Loaded gems

Gem
Version

did_you_mean
1.0.0

slack-notifier
2.3.2

CFPropertyList
2.3.6

claide
1.0.2

colored2
3.1.2

nanaimo
0.2.3

xcodeproj
1.5.4

rouge
2.0.7

xcpretty
0.2.8

terminal-notifier
1.8.0

unicode-display_width
1.3.0

terminal-table
1.8.0

plist
3.4.0

public_suffix
2.0.5

addressable
2.5.2

multipart-post
2.0.0

word_wrap
1.0.0

tty-screen
0.6.4

tty-cursor
0.5.0

tty-spinner
0.8.0

babosa
1.0.2

colored
1.2

highline
1.7.10

commander-fastlane
4.4.5

excon
0.60.0

faraday
0.13.1

unf_ext
<IP_ADDRESS>

unf
0.1.4

domain_name
0.5.20170404

http-cookie
1.0.3

faraday-cookie_jar
0.0.6

fastimage
2.1.1

gh_inspector
1.0.3

json
1.8.3

mini_magick
4.5.1

multi_json
1.13.1

multi_xml
0.6.0

rubyzip
1.2.1

security
0.1.3

xcpretty-travis-formatter
1.0.0

dotenv
2.2.1

bundler
1.16.1

faraday_middleware
0.12.2

uber
0.1.0

declarative
0.0.10

declarative-option
0.1.0

representable
3.0.4

retriable
3.1.1

mime-types-data
3.2016.0521

mime-types
3.1

little-plugger
1.1.4

logging
2.2.2

jwt
2.1.0

memoist
0.16.0

os
0.9.6

signet
0.8.1

googleauth
0.6.2

httpclient
2.8.3

google-api-client
0.13.6

io-console
0.4.5

libxml-ruby
2.9.0

fastlane-plugin-mobile_center
0.1.5

fastlane-plugin-versioning
0.3.4

@taquitos i putted an example right above in section description
It's quit obvious: do the same behavior as ruby file to enable put params into func ... no ?

I guess I wasn't being imaginative enough. It's not a feature I use in fastlane, but you're totally right.

👍 Readability++

Is this not a duplicate of issue number #11370? https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&uact=8&ved=0ahUKEwjj5IW-_sbZAhXM1IMKHe3CAXUQFggpMAA&url=https%3A%2F%2Fgithub.com%2Ffastlane%2Ffastlane%2Fissues%2F11370&usg=AOvVaw1cX814OgJ1zr8fwUYvntQZ

@VivintMan I haven't tried it yet, but I think you should be able to do:
func makeBurritoLane(tacos: Integer, beans: Integer) { }
Then on the command line:
fastlane makeBurrito tacos:2 beans:1

@taquitos excellent! Thanks! TBH, that’s exactly  what I thought I’d try, but I’ve been prepping our codebase for a release and iterating on some policies around our release train, so I wasn’t in a position to try it out. I definitely will tomorrow, though!
Thanks again!

I have been trying to pass parameters to lane as it was presented by @juliengdt and @taquitos but it's not working as it is described in this issue tickets.
For that reason I have documented here the whole process:
https://medium.com/@doruvil/fatlane-continous-integration-swift-version-b8851f1f38b6
There are some changes needed still for  LaneFileProtocol


deconz - No direct access since ingress (4.0)

I often used direct access to the deconz addon as a quick and comfortable option to see and configure my zigbee devices. For example adding a new device....
Now that is pretty uncomfortable especially on smartphone.
I'd really like to see port forwarding of the internal http-ports again. Otherwise I'd have to run a deconz container standalone.

Even with Ingress that will still work (just tested it myself on my iPhone). Suggestion: Add it to the sidebar by toggling the "Show in sidebar" feature.
We can't offer Ingress and Direct access at the same time for deCONZ since it causes issues with the discovery feature inside Phoscon itself. So, unfortunately, enabling/restoring that functionality is not an option.

Is there any way to access the deconz api after this change?

You can use the Home Assistant API that handle the deconz API

I´m trying to get the Hue essentials app that were working from my phone with direct access to deconz before to work, is that still possible?

I cant access the "Old WebApp version" through ingress.. I get a 404..


Weird text resizing of SVG texts on Chrome

Hello,
I don't know if this is a Chrome issue or a Panzoom issue but, in any case, it would be important to fix it.
On Chrome (tested on Version 96.0.4664.45 (Official Build) (64-bit), Linux (Lubuntu)) I see weird resizing of SVG texts after panzooming, on mouse over the text.
I don't have the previous bug on Firefox (94.0 (64-bit)), nor on Chromium (Version 96.0.4664.45 (Official Build) snap (64-bit))
Please see this very short example:
https://jsfiddle.net/2stekjvm/9/
Steps to reproduce:

panzoom with the mouse wheel the svg image
move the mouse over the element, after panzooming. You will see the text resized as soon as the mouse pointer touches it.

Thanks for your help/feedback!

As a further information, I just verified that the bug doesn't happen if I add a text-rendering attribute to the  node. See:
https://jsfiddle.net/s68zcd3b/

I had the same issue. I've been 3 days crazy until I have found that the issue was with Chrome 96 (in 95 from 1st of Nov and below there is no problem) and thanks @paopre for your workaround! it works!

Thanks for opening an issue! I've added this workaround to the README.