Bugfix for takeOne (issue 16999)

Fixed a bug: takeOne with unslicabe ranges did not pop its source range when popped. This made it inconsistent with take and takeExactly when using reference ranges. Also corrected its documentation.

You need to rebase so that the first commit message starts with  issue 16999 - .... The bot doesn't detect bugzilla issues from the pull request title but from the commit messages.

git rebase -i origin/master (or replace origin with the name you gave to the official dlang source)
In the console editor patch the commit name then save (ESC then W then Q).
finally push again with the option --force.

Too hard for me. I can't use Vim controls at all... were I to mistype a single key, I would not know what to do.
Is there a non-interactive way to rename commits?

@dukc Then just change the editor, if you don't like vim ;)
https://git-scm.com/book/tr/v2/Customizing-Git-Git-Configuration
git config --global core.editor [editor exe name]
E.g. git config --global core.editor subl (if you use Sublime Text).

Is there a non-interactive way to rename commits?

Two other possibilities:

You can also use one of the GUI editors
I can also do the rebase for you - just let me know if you want to go this way

I had already changed the editor -- I just did not except that other editors could do interactive mode. Thanks for the info.

I just did not except that other editors could do interactive mode

@dukc now you seem to have added a lot more commits. Maybe a simple git rebase upstream/master will do?

i have a feeling i am screwing the whole thing up just trying to rename a single commit...

No, I can't get it to work now... I edit the message, save it and then get a merge problem or something. Too much new stuff to handle at once. I abourted the rebase.

I somehow managed to apparently include everything that has happened in the master branch recenty in the pr... I believe I have to close this and do the whole pr again?

I somehow managed to apparently include everything that has happened in the master branch recenty in the pr... I believe I have to close this and do the whole pr again?

Nope - fixed it for you ;-)

Thanks. I'm not going to be trying to rebase anything I've made pr's of anytime soon.

What's that CircleCI failure about... "reference is not a tree" sounds like a checker bug?

Please explain the above, pull or close. I messed up my private fork but I dare not to try to clean it up as long as this pr is based from it. That means my possible further work is blocked by this in the queue.

@wilzbach I will close this pr if no-one has anything to say about it anymore. Post above explains why.

What's that CircleCI failure about... "reference is not a tree" sounds like a checker bug?

Yep, I think it was due to this fixed problem. I rebased your PR and hopefully everything should be green now.
Also note that passing CircleCi is not enforced nor required, it's a mere tool to help reviewers.

Please explain the above, pull or close. I messed up my private fork but I dare not to try to clean it up as long as this pr is based from it.

 As long as you don't delete your fork or the takeOneFix branch nothing should be affected.

That means my possible further work is blocked by this in the queue.

Don't worry about - in the worst case - GitHub makes copies of all PRs and I have them on my machines, so I could easily recreate this PRs.

@wilzbach I will close this pr if no-one has anything to say about it anymore. Post above explains why.

Sorry, I was away for a bit. Please don't let yourself get blocked by this. As said we will manage to find a way ;-)

Finally, fixed that fork! Trough, the lesson appears to be that if something is rejected, force it... usually that's a bad sign.

A short note on why you need to force a push to a remote branch. I hope you will find this helpful.
After you've rebased your local branch, you will have to force the push to your remote branch. That is because git rebase rewrites your commits history and you want to force the rewrite to the remote branch as well. If you don't force, the push will be rejected because of a mismatch in the commits history.
When you pull from a remote branch, it is probably a good idea to use the --ff-only option. The --ff-only option is to ensure that your local branch is identical to the remote branch, since otherwise you will end up with a very messy history that will be hard to clean up.
I have found rebasing explained really well (imho) here.
Hope this helps.
Cheers

Thanks, perhaps it will help if/whn i encounter more problems. My problems probably were due to thinking that if I'm forcing, I'm doing something wrong.

@JackStouffer Nitpicks addressed. But are they ok now?

Please squash and push: https://wiki.dlang.org/Starting_as_a_Contributor#Squashing

Thanks. Done, hopefully without complications.

Auto-merge toggled on

Oh no, I screwed something with squashing, popBack is not aliased to popFront anymore. That's a secondary thing trough. Perhaps i'll do a separate pr about that.


Quests resseting on server restart

(Required) What is your Quests version / build number (do not say "latest")? Quests 3.2.7
(Required) Please describe the problem and how to replicate it. Any console errors?
Hello there are no console errors, the problem is that some players who have an active quest gets bugged, during the schedulled night server restart. When they login back, the situation is that they do not have active quest, and they can not take another one, until administrator forcibly didnt finish or quit the previous quest.
(Optional) Can you provide an example quest from quests.yml using Pastebin.com?
It usually happens within this quests:
custom36:
name: Problem s vlkmi
npc-giver-id: 14
redo-delay: 1209600
ask-message: Podarilo sa nam zistit, preco ma Rybar Galon pre nas nalovit dostatok
ryb! Nedaleko pristavu sa usidlili v Dungeone vlci Calu, ktori lovia nase ryby.
Zabi ich 50 a ziskas vzacnu odmenu! Vyhni sa ale BESNEMU Calu!
finish-message: Vyborne! Snad aspon na chvilu bude od vlkov pokoj. Tu je tvoja
odmena!
gui-display: name-GOLD_SWORD:amount-1
requirements:
quest-points: 40
fail-requirement-message: Musis mat alespon 40 questovych skusenosti, aby si
mohol prijat tento quest! Nemozem ho predsa dat ziadnemu novacikovi!
stages:
ordered:
'1':
custom-objectives:
custom1:
name: Kill MythicMobs Objective
count: 50
data:
Mob Level: '0'
Objective Name: Calu
Internal Mobnames: Calu

Thanks for submitting this issue and sorry I wasn't able to get to it. Chances are the issue has been resolved in recent builds of Quests. If you find that not to be the case, post the error generated by a recent version of Quests and I'll re-open the issue.

I am currently getting this issue in 3.6.7 and past 3.6.6. It only seems to be none cooldown quests having this issue.
Example,
I have a quest to dig 1000 dirt and another to break 100 of each wood log and if you log out while this is being done and the server restarts the quests lose their objectives in the quests journal and /quests stats. It still shows that they have the quest just nothing to do to finish the quest.


[WebSocket] RCTWebSocket crashes when receiving messages in binary format

WebSocket client doesn't currently contain any handling for binary messages and just passes the message directly to bridge. This crashes the application.
*** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: 'Invalid type in JSON write (_NSInlineData)'

Problematic code:
https://github.com/facebook/react-native/blob/0c9c6e89e36c3bd8c7926ad58d58c750975ae61a/Libraries/WebSocket/RCTWebSocketManager.m#L83-L89
Few ways to fix this.

Wait for https://github.com/facebook/react-native/issues/1424
Pass error describing the issue instead of crashing
Convert NSData to string
Base64 encode NSData and convert to blob in JS

Personally I'd prefer 4. as it brings the polyfill api more closer to actual WebSocket behaviour. I can take a stab at the problem once we've settled with the correct approach.
Thanks!

It seems that FileReader and atob (and probably Blob)  are not implemented in JSCore by default. Should be go with 3.?

We could also just push Apple to provide a native API for Typed Arrays in JSCore:
https://bugs.webkit.org/show_bug.cgi?id=120112

Hi there! This issue is being closed because it has been inactive for a while.
But don't worry, it will live on with ProductPains! Check out it's new home: https://productpains.com/post/react-native/websocket-rctwebsocket-crashes-when-receiving-messages-in-binary-format


chore(website): add descriptions for typescript options

Description

Added description to typescript options.

Related issue(s)
Fixes #1392

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 92.192%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
5259

Relevant Lines:
5562

💛  - Coveralls


Cache performance templates for JavaExecProjectGeneratorTask

Generating perfomance projects seems to be expensive. Let's just use the build cache for it!
For largeJavaMultiProject we have 2 minutes vs 20 seconds.

@lptr Could you have another look?

LGTM.


Error when using to_nf4 function, inside NF4Tensor Class

I have encountered a problem when using QLoRa from the LoRALinear class you have in Torchtune. Apparently, when the ‘quantize_base’ parameter is set to True, the NF4Tensor class is called, which transforms the tensor to 4 bits from the to_nf4 function.
This function set the tensor in bf16 without specifying the device, giving the problem of tensors on two different devices, CPU and GPU. When the tensor is set, by default it is assigned to CPU unless otherwise specified. I got this error trying to do QLoRA on my GPU.
I leave here the original function vs the one I modified to get out of it:
Original function according to torchao.dtypes.nf4tensor:
def to_nf4(tensor, block_size: int = 64, scaler_block_size: int = 256):     tensor1 = tensor.to(torch.bfloat16)     return NF4Tensor.from_tensor(tensor1, block_size, scaler_block_size)
Modified function for dealing with the problem:
def to_nf4(tensor, device, block_size: int = 64, scaler_block_size: int = 256):     tensor1 = tensor.to(device=device, dtype=torch.bfloat16)     return NF4Tensor.from_tensor(tensor1, block_size, scaler_block_size)

@FabioDataGeek would it work to do to_nf4([...]).to(device)?

@FabioDataGeek - I also sent a PR to add the functionality you mention: https://github.com/pytorch/ao/pull/324


Fix flaky test script bug

Commit Message:
#14731 introduced a bug where the script tried to dump the URL of a remote that might not be present.  This PR fixes that bug.
Signed-off-by: Randy Miller<EMAIL_ADDRESS>Risk Level: Low
Testing: Ran the script locally multiple times

cc @envoyproxy/windows-dev

cc @envoyproxy/windows-dev


Proposal of logo and icon for the application

Hello I am a graphic designer and I make contributions to open source projects through the Utopian-io platform, (with them I earn the rewards if you use my proposal) I attach a proposal for your project of an icon and a logo if you like I can send you all the files for you to attach to your project. :-)

Great work! I like it. What do you think @afaur?

@danielbayerlein Thank you so much that you liked it.

I am open to this proposal and have been trying to determine what changes would be required if the community decides to proceed with this.
I propose we create a branch or repo that tracks design files to make this an easier process going forward. It would also be a place where a designer could submit a PR with a proposal on how a logo would look in each placement scenario.
If this proposal is accepted we would need a member to make these changes and submit a PR with the correct modifications in the identified areas.
Here is an example of some identified areas where branding occurs in the application with a possible suggestion for how the logo might look in that scenario:

Web favicon

Before

After

Regular Tray

Before

After

Inverted Tray

Before

After

Launcher UI

Before

After

Print Media

I think zazu currently has some hex stickers associated with it.
Might be good to upload that template and see how the design could look in that format
Print media does not seem to be currently tracked through git and probably should be

If anyone can think of a situation I have failed to recognize please add it to the list.
xcf-gimp-icon-template.zip
^ Here are the template placeholders for the locations identified so far.
If you determine a new placement scenario that I did not think about and you are able to create an icon placeholder in xcf (gimp) format please attach it to a comment on this thread.
I will work to place these templates in a repo or branch based on community feedback here.

Seeing what you have placed I can notice that the icon is incorporated very well in different presentations and small sizes, within the proposal I do I can incorporate different variations of color and size so that they can use it in whatever they want. As you can see, the Z is better identified than the pen and in addition to this is a more personal image that users can easily remember, the pen reminds me of ENCARTA encyclopedias :-)
If you need something more from me, do not hesitate to write me and I will annex it. I will work on the different variations and leave them here.

Design mock-up


Add x back to functional evaluation while loading legacy files

Codecov Report
Patch and project coverage have no change.

Comparison is base (c78d077) 85.26% compared to head (93b0c3b) 85.26%.

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.
Additional details and impacted files
@@           Coverage Diff           @@
##             main     #341   +/-   ##
=======================================
  Coverage   85.26%   85.26%
=======================================
  Files          30       30
  Lines        1975     1975
  Branches      301      301
=======================================
  Hits         1684     1684
  Misses        209      209
  Partials       82       82

Impacted Files
Coverage Δ

btrack/io/hdf.py
76.42% <0.00%> (ø)

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


Object not supported

Displays when object is not supported rather than get error when doing a "add". Note this was branched from remove-option branch.

cool, thanks :smile: will take a look at the changes later this week


Unable to start docker

When I run the Docker command in the README, I receive the following error, does anyone know why this might be? @agladstein
docker: Error response from daemon: pull access denied for ncbihackathons/metadata-notebook, repository does not exist or may require 'docker login'

Sorry! Typo.
Should be ncbihackathons/metadata
Corrected the image name in README in https://github.com/NCBI-Hackathons/RNA-Seq-in-the-Cloud/commit/b6ab20317e8ebb22dceb71bdbe87751438bf4be8

Thanks!


OAuth2 and google script problem with meetup.com

I'm having an OAuth2 validation problem with Google Script

// ---- Oauth2 Google Script code

function getMeetupService() {
  return OAuth2.createService('meetup')

      // Set the endpoint URLs, which are the same for all Google services.
      .setAuthorizationBaseUrl('https://secure.meetup.com/oauth2/authorize')
      .setTokenUrl('https://secure.meetup.com/oauth2/access')

      // Set the client ID and secret, from the Google Developers Console.
      .setClientId(CLIENT_ID_MEETUP)
      .setClientSecret(CLIENT_KY_MEETUP)

      // Set the name of the callback function in the script referenced
      // above that should be invoked to complete the OAuth flow.
      .setCallbackFunction('authCallback')

      // Set the property store where authorized tokens should be persisted.
      .setPropertyStore(PropertiesService.getUserProperties())
      .setTokenFormat(OAuth2.TOKEN_FORMAT.JSON);

}

function authCallback(request) {
  var meetupService = getMeetupService();
  var isAuthorized = meetupService.handleCallback(request);
  if (isAuthorized) {
    return HtmlService.createHtmlOutput('Success! You can close this tab.');
  } else {
    return HtmlService.createHtmlOutput('Denied. You can close this tab');
  }
}

function clearService(){
  OAuth2.createService('meetup')
  .setPropertyStore(PropertiesService.getUserProperties())
  .reset();
}

_eventId = null;
function startService(eventId) {
  var meetupService = getMeetupService();
  if (!meetupService.hasAccess()) {
    var authorizationUrl = meetupService.getAuthorizationUrl();
    return authorizationUrl;
  } else {
    return null;
  }
}

Answer:
The state token is invalid or has expired. Please try again.
The returned property "state" to usercallback use signs "+" where it should be "_" both can represent spaces sometimes but I think here that is a problem.

If the state parameter is coming back altered in some way that sounds like a bug on our end. I'll have a look to see if we are replacing underscores for some reason

You should expect back + encoding for query string parameter values that contain spaces. This is not a bug. Url path elements that contain spaces are encoded with %20, query string parameter values and form urlencoded body values using + to encode spaces.

this should now be fixed


Missing command

Either I'm blind or there's a step missing at the end of step 5 or beginning of step 6 you should have a line to get the $groups variable.
get-unifiedgroup | where {$_.classification -eq "General"}

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 0d40ac52-6f02-b0f3-0c45-9f359473703b
Version Independent ID: 108355a9-05a3-4d17-c417-c4d5664a4506
Content: Use sensitivity labels with Microsoft Teams, Office 365 groups, and SharePoint sites (public preview)
Content Source: microsoft-365/compliance/sensitivity-labels-teams-groups-sites.md
Service: o365-seccomp
GitHub Login: @KCCross
Microsoft Alias: krowley

That's step 5 and 6 of the Relabel Office 365 groups with new sensitivity labels section

Hello @siegardj thanks for noticing and reporting.
I created a pull request to propose a correction to the article.
Thank you

@siegardj  Thanks for flagging this.  The extra line has been added & you should see it live within the next 24 hours.


authorized_userid() method in AbstractAuthorizationPolicy required?

I was wondering if authorized_userid() method in AbstractAuthorizationPolicy is necessary.
It seems like it duplicates functionality of identity() in AbstractIdentityPolicy.
@asyncio.coroutine
@abc.abstractmethod
def authorized_userid(self, identity):
    """Retrieve authorized user id.
    Return the user_id of the user identified by the identity
    or 'None' if no user exists related to the identity.
    """
    pass

In demo code, it is used to get identities from database that are not disabled.
@asyncio.coroutine
def authorized_user_id(self, identity):
    with (yield from self.dbengine) as conn:
        where = [db.users.c.login == identity, not db.users.c.disabled]
        query = db.users.count().where(sa.and_(*where))
        ret = yield from conn.scalar(query)
        if ret:
            return identity
        else:
            return None

I believe this check can be done in identify() method in AbstractIdentityPolicy instead.

Yes, it's required.
Just for note: userid is a persistent value but identity may be changed between user sessions.


proposal: net: TCPConn supports Writev

I have search go-nuts and google about the writev for unix, seems go not support it yet. And I found a project vectorio which support writev but not work.
I am rewriting the srs to go-oryx. SRS can support 7.5k clients per CPU, while golang version only support 2k, for the media streaming server need to delivery the video or audio packet to different parts, then use writev to send to client to avoid bytes copy.
I try to use reflect to implements the writev, but I found it's possible for the pollDesc is not exported, the commit is here.
Does go plan to support writev(netFD.Writev)?

I want to explain why writev is important for media streaming server:

The media streaming server always delivery some audio or video packets to a connection.
These audio or video packets can be sent in a writev to avoid too many syscall.
The writev can used to avoid the copy from videos to the buffer, for the video payload is very large.

I have use cpu and mem profile to fix the bottleneck, but the memory copy and syscall by use netFD.Write hurts the performance.

if has writev, it's easy for us to implement zero copy service....

+1

@Terry-Mao +1

+1

+1

@ianlancetaylor
As a network proxy, many network protocol has TWO parts HEADER & DATA part.
if netFD has writev and also support by bufio, it's no need do like:
// reader & writer goroutine call write concurrency, must lock.
lock.Lock()
w.Write(head)
w.Write(data)
lock.Unlock()

but the inner netFD has fd.writeLock(), so caller must do a wasting lock.Lock().
something like this is very nice:
// Write is atomic, and goroutine safe
w.Write(head, data)

func (w *Writer) Write(data ...[]byte) (n int, err error)

For streaming service, for example, to send a video packet to RTMP(over TCP) client, the video maybe:
video=make([]byte, 156 * 1024)

Then we need to add some small header every some video bytes, for instance, 10k:
h0=make([]byte, 12)
p0 = video[0:10240]

h1=make([]byte, 5)
p1=video(10240:20480)

......

// util the hN and pN
hN=make([]byte, 5)
pN=video(x:)

RIght now, we do it in a very very slow way:
// merge all to a big buffer
bigBuffer = bytes.Buffer{}
for b in (h0, p0, h1, p1, ..., hN, pN) {
    bigBuffer.Write(b)
}

// send in a syscall
netFD.Write(bigBuffer)

Because the syscall is expensive then copy buffer:
// very very slow for too many syscall
for b in (h0, p0, h1, p1, ..., hN, pN) {
    netFD.Write(b)
}

When golang support writev, we can send in a syscall and without copy to a big buffer.
Write(h0, p0, h1, p1, ......, hN, pN)

// merge all to a big buffer
bigBuffer = bytes.Buffer{}
for b in (h0, p0, h1, p1, ..., hN, pN) {
    bigBuffer.Write(b)
}

h0, p0, etc all derive from a single buffer, why do you slice them up, then copy them back together? Sending the original slice would avoid the copy, and avoid extending the net.Conn interface.

@davecheney Nop， only the p0,p1,...,pN is slice from video, the h0,h1,...,hN is another slice. That is, we slice the video payload:
// it's ok to use slice, without copy.
video = p0 + p1 + ... + pN

But the headers is another slice:
headers = h0 + h1 + ... + hN

We should interleave the header and  payload to a bit buffer:
big-buffer = h0 + p0 + h1 + p1 + ... + hN + pN

Because each header is belong to its payload, for instance, hN is only for pN. When use c/c++ to write to socket, we can use writev:
iovec iovs[N*2];
for h,p in (h0, p0, ..., hN, pN) {
    iovs[i].data = h
    iovs[i+1].data = p
}
socet.writev(iovs)

Does that make sense?

I'm fine with adding a Writev call to the net package.  I'm not aware of anybody actually working on it.

@ianlancetaylor Great~
I create SRS, and rewrite it by golang go-oryx, and the benchmark tool srs-bench for streaming server.
SRS can serve 7.5K clients, use 1CPU(about 80% usage), that is about 3.75Gbps. SRS is really very high efficient for use writev to avoid copy and use little syscall. But SRS is single process model, for steaming server is too complex.
Nginx-RTMP which is a plugin in nginx, which only support 2k clients per CPU, but nginx-rtmp support multiple-processes.
I create go-oryx, because I want to use the multile processes feature of golang. Right now, when I do lots of performance refine, the go-oryx can support 8k by 4CPU. If golang support writev, I think the performance can improve 200% to 400%, that is about 16k to 32k clients use 4CPU. It's really awesome.

@davecheney @ianlancetaylor What's the state of this issue now? Accept or postpone? Any plan?

@winlinvip, are you sure your problem is Writev and not, say, allocations? Some of your examples above look very allocation-happy.
Before accepting this proposal, I'd want to see before & after programs with numbers and profiles to show that it helps enough. There will definitely be pushback about expanding the net package's API, which many feel is already too large.
That is, convince us with data perhaps. Implement this in the net package and then show us numbers with how much it helped. We have to be able to reproduce the numbers.

@bradfitz I can only give c/c++ performance data for writev, but the golang not support writev right now.I can give a test example, written in c++ and golang, to show what streaming server should do.
The project is go-writev
go-writev
The test example for https://github.com/golang/go/issues/13451
Usage
The c++ version for writev:
# server side.
cd cpp && make && ./server 1985 true
# client side.
cd cpp && make && ./client 1985

The golang version for writev:
# server side.
cd golang && go build . && ./golang 1985 false true
# client side.
cd cpp && make && ./client 1985

Benchmark
| NO | OS   | Server      | Client | Syscall            | Bandwidth| Server Command     |
| 1 | ----- | ----------- | ------ | ------------------ | -------- |  ------------------  |
| 2 | Linux | c++,100%    | c++,90% | write(one-by-one) | 1172MBps | ./server 1985 false true |
| 3 | Linux | c++,86%%    | c++,100%| write(big-buffer) | 2016MBps | ./server 1985 false false |
| 4 | Linux | c++,73%     | c++,99% | writev            | 3576MBps | ./server 1985 true        |
| 5 | Linux | golang,100% | c++,78% | write(one-by-one) | 743MBps  | ./golang 1985 false true |
| 6 | Linux | golang,193% | c++,55% | write(big-buffer) | 1079MBps | ./golang 1985 flase false |
Conclude:

The No.4 is the highest performance, to use writev to avoid memory copy and low syscall.
The No.5 is the most weak performance, for lots of syscall.
The No.6 is the better solution for golang without writev, but waste lots of cpu(golang 193% compare to c++ 73%).

So, I think the writev can improve about 200%~400% performance for streaming server.
Winlin 2015

@bradfitz I try the no-copy version, to send the []byte one by one, but it will cause lots of syscall which hurts performance more. So I use a big buffer to copy the data to send, then use one syscall to send it. The big buffer solution is better than one by one, I have profile it and test it with mock clients. Please read https://github.com/ossrs/go-oryx/pull/20

@bradfitz And I think the writev is not a new API or language feature, it exists on all linux, unix and unix-like os. It's really very useful for high performance server, I found nginx also use writev:
find . -name "*.c"|xargs grep -in "= writev("
./os/unix/ngx_darwin_sendfile_chain.c:285:            rc = writev(c->fd, header.elts, header.nelts);
./os/unix/ngx_files.c:229:        n = writev(file->fd, vec.elts, vec.nelts);
./os/unix/ngx_freebsd_sendfile_chain.c:336:            rc = writev(c->fd, header.elts, header.nelts);
./os/unix/ngx_linux_sendfile_chain.c:294:            rc = writev(c->fd, header.elts, header.nelts);
./os/unix/ngx_writev_chain.c:113:        n = writev(c->fd, vec.elts, vec.nelts);

Seems writev was introduced at 2001, https://en.wikipedia.org/wiki/Vectored_I/O
Standards bodies document the applicable functions readv[1] and writev[2] in POSIX 1003.1-2001 and the Single UNIX Specification version 2.

@winlinvip, can you try to modify variant 6 to avoid allocations of group buffers by populating buf (https://github.com/winlinvip/go-writev/blob/master/golang/server.go#L123) directly. I think this will be roughly equivalent to calling writev on multiple buffers. What your benchmark shows for this program?

@kostya-sh Did u compare the c++ version? The allocation is not the bottle-neck, but the memory copy and syscall.

It's really a very basic problem for server. Eventhough the golang is modern language, but it's compile to binary code and execute on linux, the memory copy and syscall is always the bottle-neck for server, I have profile it.

Please post a profile here.

The execute time and command:
# we use netFD.Write and send []byte one by one to avoid copy them to a big buffer,
# which need more syscall.
[winlin@ftp golang]$ go build . && time ./golang 2015 false true

The cpu profile:
[winlin@ftp golang]$ go tool pprof golang cpu.prof
Entering interactive mode (type "help" for commands)
(pprof) top
2.95mins of 3.24mins total (91.23%)
Dropped 88 nodes (cum <= 0.02mins)
Showing top 10 nodes out of 29 (cum >= 0.03mins)
      flat  flat%   sum%        cum   cum%
  2.66mins 82.29% 82.29%   2.82mins 87.16%  syscall.Syscall
  0.04mins  1.21% 83.50%   0.09mins  2.78%  runtime.deferreturn
  0.04mins  1.16% 84.66%   0.04mins  1.16%  runtime.newdefer
  0.03mins  1.06% 85.73%   0.03mins  1.06%  runtime.cas
  0.03mins  1.03% 86.76%   3.14mins 97.13%  net.(*netFD).Write
  0.03mins  0.97% 87.72%   0.03mins  0.97%  runtime.freedefer
  0.03mins  0.96% 88.68%   0.12mins  3.81%  runtime.systemstack
  0.03mins  0.95% 89.63%   0.07mins  2.10%  runtime.reentersyscall
  0.03mins   0.8% 90.43%   3.19mins 98.66%  main.srs_send
  0.03mins  0.79% 91.23%   0.03mins  0.79%  sync/atomic.CompareAndSwapUint64

Too many syscalls, for each video(actually streaming server will slice video to smaller chunk). Which can be avoid by copy small piece to big buffer or use writev to send them in a syscall.
The memory profile:
(pprof) [winlin@ftp golang]$ go tool pprof golang mem.prof
Entering interactive mode (type "help" for commands)
(pprof) top
512.19kB of 512.19kB total (  100%)
Dropped 2 nodes (cum <= 2.56kB)
      flat  flat%   sum%        cum   cum%
  512.19kB   100%   100%   512.19kB   100%  runtime.malg
         0     0%   100%   512.19kB   100%  runtime.mcommoninit
         0     0%   100%   512.19kB   100%  runtime.mpreinit
         0     0%   100%   512.19kB   100%  runtime.rt0_go
         0     0%   100%   512.19kB   100%  runtime.schedinit

Seems memory is ok, right?
The cpu usage:
  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 9597 winlin    20   0  217m 9216 1616 R 100.4  0.1   0:13.94 golang
10113 winlin    20   0 11736  924  792 R 78.5  0.0   0:10.91 client

The client cpu is about 78.5%, that is, server is too busy to feed the client(where the c++ writev version the client is 100% and server is 73%, which indicates that server can feed more client, client is very busy).
The dstat data:
[winlin@ftp ~]$ dstat -N lo
----total-cpu-usage---- -dsk/total- ---net/lo-- ---paging-- ---system--
usr sys idl wai hiq siq| read  writ| recv  send|  in   out | int   csw
  1   5  90   0   0   4|   0     0 | 716M  716M|   0     0 |4898    34k
  1   7  87   1   0   4|   0    32k| 712M  712M|   0     0 |5000    34k
  1   6  88   0   0   5|   0    16k| 706M  706M|   0     0 |4806    34k
  1   6  87   0   0   5|   0     0 | 706M  706M|   0     0 |4927    34k
  1   6  89   0   0   4|   0     0 | 805M  805M|   0     0 |5237    43k
  2   7  86   1   0   5|   0    72k|1051M 1051M|   0     0 |6296    63k
  2   5  90   0   0   3|   0     0 |1050M 1050M|   0     0 |6473    63k
  2   7  86   0   0   5|   0    34M|1034M 1034M|   0     0 |6740    62k
  1   5  90   0   0   3|   0     0 |1039M 1039M|   0     0 |6428    63k
  2   6  88   0   0   4|   0     0 |1043M 1043M|   0     0 |6313    63k
  2   5  89   0   0   4|   0     0 |1042M 1042M|   0     0 |6268    63k
  2   7  86   0   0   5|   0    12k|1054M 1054M|   0     0 |6202    63k
  1   5  90   0   0   4|   0  4096B| 840M  840M|   0     0 |6115    50k
  1   6  88   0   0   5|   0     0 | 720M  720M|   0     0 |5670    42k
  1   6  89   0   0   4|   0     0 | 714M  714M|   0     0 |5611    41k
  1   7  86   0   0   5|   0     0 | 701M  701M|   0     0 |5057    36k

The c++ writev version can archive 3075MBps, actullay it maybe got more, for the server is not busy(about 73% cpu).

If I copy small pieces to a big buffer, the performance is better because the syscall is very expensive.
The command to use big buffer strategy:
(pprof) [winlin@ftp ggo build . && time ./golang 2015 false false
real	4m24.822s
user	4m31.110s
sys	2m57.815s

The cpu profile:
[winlin@ftp golang]$ go tool pprof golang cpu.prof
Entering interactive mode (type "help" for commands)
(pprof) top
371.03s of 448.79s total (82.67%)
Dropped 200 nodes (cum <= 2.24s)
Showing top 10 nodes out of 80 (cum >= 243.07s)
      flat  flat%   sum%        cum   cum%
   139.93s 31.18% 31.18%    140.77s 31.37%  syscall.Syscall
    51.39s 11.45% 42.63%     51.39s 11.45%  runtime.memclr
    46.57s 10.38% 53.01%     57.46s 12.80%  runtime.scanblock
    38.70s  8.62% 61.63%     38.70s  8.62%  runtime.memmove
    27.96s  6.23% 67.86%     27.96s  6.23%  runtime.futex
    24.66s  5.49% 73.36%     28.64s  6.38%  runtime.scanobject
    12.78s  2.85% 76.20%     46.11s 10.27%  runtime.scang
     9.83s  2.19% 78.39%     15.59s  3.47%  runtime.readgstatus
     9.62s  2.14% 80.54%      9.62s  2.14%  runtime.usleep
     9.59s  2.14% 82.67%    243.07s 54.16%  main.srs_send

The syscall is better than previous one.
The mem profile:
[winlin@ftp golang]$ go tool pprof golang mem.prof
Entering interactive mode (type "help" for commands)
(pprof) top
3120.57kB of 3120.57kB total (  100%)
Dropped 3 nodes (cum <= 15.60kB)
      flat  flat%   sum%        cum   cum%
 2093.75kB 67.10% 67.10%  2093.75kB 67.10%  main.srs_send
  514.63kB 16.49% 83.59%   514.63kB 16.49%  runtime.procresize
  512.19kB 16.41%   100%   512.19kB 16.41%  runtime.malg
         0     0%   100%  2093.75kB 67.10%  main.main.func2
         0     0%   100%  2093.75kB 67.10%  runtime.goexit
         0     0%   100%   512.19kB 16.41%  runtime.mcommoninit
         0     0%   100%   512.19kB 16.41%  runtime.mpreinit
         0     0%   100%  1026.82kB 32.90%  runtime.rt0_go
         0     0%   100%  1026.82kB 32.90%  runtime.schedinit

The CPU usage:
  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
17883 winlin    20   0  380m  15m 1652 S 167.6  0.1   0:15.45 golang
17891 winlin    20   0 11736  928  792 R 56.0  0.0   0:05.16 client

The client is hungery, only consume 56% cpu, but server is very busy about 167% cpu(max is 400%).
The dstat:
[winlin@ftp ~]$ dstat -N lo
----total-cpu-usage---- -dsk/total- ---net/lo-- ---paging-- ---system--
usr sys idl wai hiq siq| read  writ| recv  send|  in   out | int   csw
  1   0  99   0   0   0|5951B   68k|   0     0 |   1B   18B|2221  4291
  8   8  82   0   0   2|   0     0 |1135M 1135M|   0     0 |  20k   51k
  6   6  87   0   0   1|   0     0 |1210M 1210M|   0     0 |  20k   51k
  9   8  82   0   0   2|   0     0 |1168M 1168M|   0     0 |  20k   51k
  6   5  88   0   0   1|   0     0 |1159M 1159M|   0     0 |  21k   50k
  8   7  83   0   0   2|   0     0 |1146M 1146M|   0     0 |  21k   50k

This big-buffer is better than one-by-one solution, the bandwidth send archive 1146MBps. But it's bad than c++ writev version, which can archive 3576MBps(and server is only use 73% cpu).

@winlinvip, by removing allocations from your test program (variant 6) you will end up with a single syscall per write. This way you can estimate how fast would be your Go program if it used writev. If improvement of Go application is significant then it could be a pro argument for adding writev.
if you can implement writev in your local version of Go standard library, test the performance and post numbers here it would be even better.

@kostya-sh I can cache the big buffer to a global []byte, please see https://github.com/winlinvip/go-writev/blob/2afa1d3f4c73ada4d0f13a0f837947c7b92a4370/golang/server.go#L129
The result is better than previous, but still bad than c++-writev:
  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 7096 winlin    20   0 11736  928  792 R 97.1  0.0   0:51.16 client
 7088 winlin    20   0  301m  11m 1648 S 96.7  0.1   0:49.97 golang

usr sys idl wai hiq siq| read  writ| recv  send|  in   out | int   csw
  2  12  85   0   0   1|   0     0 |2092M 2092M|   0     0 |5645  6570
  1   8  90   0   0   1|   0    32k|2336M 2336M|   0     0 |4918  5133
  2  11  86   0   0   1|   0     0 |2159M 2159M|   0     0 |4710  4903
  2   9  88   0   0   1|   0     0 |2275M 2275M|   0     0 |4988  5896
  2  10  86   0   0   1|   0     0 |2151M 2151M|   0     0 |4823  4192
  2   9  88   0   0   1|   0     0 |2144M 2144M|   0     0 |4935  5142
  2  12  84   0   0   2|   0     0 |2254M 2254M|   0     0 |5492  5788
  1   8  90   0   0   1|   0     0 |1960M 1960M|   0     0 |5643  7195

[winlin@ftp golang]$ go build . && time ./golang 2015 false false
real	1m35.412s
user	0m21.500s
sys	1m7.571s

[winlin@ftp golang]$ go tool pprof golang cpu.prof
Entering interactive mode (type "help" for commands)
(pprof) top
86.57s of 88.93s total (97.35%)
Dropped 142 nodes (cum <= 0.44s)
Showing top 10 nodes out of 14 (cum >= 67s)
      flat  flat%   sum%        cum   cum%
    66.74s 75.05% 75.05%     66.98s 75.32%  syscall.Syscall
    15.59s 17.53% 92.58%     15.59s 17.53%  runtime.memmove
     3.63s  4.08% 96.66%     86.48s 97.25%  main.srs_send
     0.46s  0.52% 97.18%     87.31s 98.18%  main.main.func2
     0.03s 0.034% 97.21%     67.28s 75.66%  net.(*conn).Write
     0.03s 0.034% 97.25%      0.68s  0.76%  runtime.findrunnable
     0.03s 0.034% 97.28%      0.94s  1.06%  runtime.schedule
     0.02s 0.022% 97.30%      0.59s  0.66%  runtime.systemstack
     0.02s 0.022% 97.32%     67.02s 75.36%  syscall.Write
     0.02s 0.022% 97.35%        67s 75.34%  syscall.write

     0.02s 0.022% 97.35%        67s 75.34%  syscall.write
[winlin@ftp golang]$ go tool pprof golang mem.prof
Entering interactive mode (type "help" for commands)
(pprof) top
2605.94kB of 2605.94kB total (  100%)
Dropped 1 node (cum <= 13.03kB)
      flat  flat%   sum%        cum   cum%
 2093.75kB 80.35% 80.35%  2093.75kB 80.35%  main.init
  512.19kB 19.65%   100%   512.19kB 19.65%  runtime.malg
         0     0%   100%  2093.75kB 80.35%  runtime.goexit
         0     0%   100%  2093.75kB 80.35%  runtime.main
         0     0%   100%   512.19kB 19.65%  runtime.mcommoninit
         0     0%   100%   512.19kB 19.65%  runtime.mpreinit
         0     0%   100%   512.19kB 19.65%  runtime.rt0_go
         0     0%   100%   512.19kB 19.65%  runtime.schedinit

Seems the syscall is still the bottle-neck.

@kostya-sh I will try to add my writev version to golang stadard library, and give the result.

I implements the writev in my private fork https://github.com/winlinvip/go/pull/1/files
It's better than the big-buffer and one-by-one solution.
go build -a . && time ./golang 2015 true
real	1m9.798s
user	0m4.449s
sys	0m58.123s

The CPU profile:
[winlin@ftp golang]$ go tool pprof golang cpu.prof
Entering interactive mode (type "help" for commands)
(pprof) top
59.58s of 62.44s total (95.42%)
Dropped 142 nodes (cum <= 0.31s)
Showing top 10 nodes out of 37 (cum >= 58.83s)
      flat  flat%   sum%        cum   cum%
    56.29s 90.15% 90.15%     56.41s 90.34%  syscall.Syscall
     1.29s  2.07% 92.22%     58.80s 94.17%  net.(*netFD).Writev
     0.64s  1.02% 93.24%      0.64s  1.02%  runtime.futex

The mem profile:
[winlin@ftp golang]$ go tool pprof golang mem.prof
Entering interactive mode (type "help" for commands)
(pprof) top
2605.94kB of 2605.94kB total (  100%)
Dropped 6 nodes (cum <= 13.03kB)
      flat  flat%   sum%        cum   cum%
 2093.75kB 80.35% 80.35%  2093.75kB 80.35%  main.init
  512.19kB 19.65%   100%   512.19kB 19.65%  runtime.malg
         0     0%   100%  2093.75kB 80.35%  runtime.goexit
         0     0%   100%  2093.75kB 80.35%  runtime.main
         0     0%   100%   512.19kB 19.65%  runtime.mcommoninit
         0     0%   100%   512.19kB 19.65%  runtime.mpreinit
         0     0%   100%   512.19kB 19.65%  runtime.rt0_go
         0     0%   100%   512.19kB 19.65%  runtime.schedinit

[winlin@ftp ~]$ dstat -N lo
----total-cpu-usage---- -dsk/total- ---net/lo-- ---paging-- ---system--
usr sys idl wai hiq siq| read  writ| recv  send|  in   out | int   csw
  6  12  80   1   0   1|4096B   56k|2233M 2233M|   0     0 |  28k   46k
  6  12  81   0   0   1|   0    20k|2249M 2249M|   0     0 |  24k   40k
  6  12  79   2   0   1|   0    92k|2152M 2152M|   0     0 |  25k   40k
  9  13  76   0   0   1|   0    16k|2231M 2231M|   0     0 |  38k   70k
  6  11  81   1   0   1|   0   236k|2226M 2226M|   0     0 |  26k   50k

PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
27785 winlin    20   0 11736  928  792 R 99.8  0.0   1:13.91 ./client 2015
25941 winlin    20   0  376m 8652 1660 S 92.5  0.1   1:09.35 ./golang 2015 true

I will test the real streaming server go-oryx later.

go-oryx, use big-buffer solution to serve 6k clients, which play RTMP stream from go-oryx.
  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
19407 winlin    20   0 1706m 1.1g 3440 S 216.3  6.9   3:37.08 oryx
14664 winlin    20   0  436m 306m 1516 R 41.1  1.9   0:25.72 sb_rtmp_load
13920 winlin    20   0  435m 306m 1516 R 39.7  1.9   0:25.21 sb_rtmp_load
15658 winlin    20   0  436m 306m 1516 R 36.8  1.9   0:24.56 sb_rtmp_load

----total-cpu-usage---- -dsk/total- ---net/lo-- ---paging-- ---system--
usr sys idl wai hiq siq| read  writ| recv  send|  in   out | int   csw
 19   7  70   0   0   3|   0    16k| 242M  242M|   0     0 |  24k   41k
 19   7  71   1   0   3|   0    68k| 216M  216M|   0     0 |  24k   41k
 19   6  70   1   0   3|   0   696k| 182M  182M|   0     0 |  23k   40k
 21   8  68   0   0   3|   0   180k| 135M  135M|   0     0 |  36k   70k
 28   6  63   0   0   3|   0    84k| 141M  141M|   0     0 |  30k   59k
 23   8  65   0   0   3|4096B   44k| 266M  266M|   0     0 |  41k   99k
 18   7  72   1   0   3|   0   112k| 190M  190M|   0     0 |  24k   40k

Cpu Profile:
[winlin@ftp go-oryx]$ go tool pprof objs/oryx cpu.prof
Entering interactive mode (type "help" for commands)
(pprof) top
206.93s of 329.12s total (62.87%)
Dropped 479 nodes (cum <= 1.65s)
Showing top 10 nodes out of 130 (cum >= 26.92s)
      flat  flat%   sum%        cum   cum%
    95.43s 29.00% 29.00%     99.05s 30.10%  syscall.Syscall
    29.04s  8.82% 37.82%    176.10s 53.51%  github.com/ossrs/go-oryx/protocol.(*RtmpStack).SendMessage
    20.42s  6.20% 44.02%    143.60s 43.63%  github.com/ossrs/go-oryx/protocol.(*RtmpStack).slowSendMessages
    15.67s  4.76% 48.78%     36.42s 11.07%  runtime.selectgoImpl
    10.11s  3.07% 51.86%     10.11s  3.07%  runtime.memmove
     7.70s  2.34% 54.20%      7.72s  2.35%  github.com/ossrs/go-oryx/agent.(*Jitter).Correct
     7.62s  2.32% 56.51%     42.22s 12.83%  github.com/ossrs/go-oryx/agent.(*RtmpPlayAgent).Write
     7.62s  2.32% 58.83%     25.68s  7.80%  runtime.mallocgc
     6.67s  2.03% 60.85%      6.67s  2.03%  runtime.futex
     6.65s  2.02% 62.87%     26.92s  8.18%  github.com/ossrs/go-oryx/protocol.(*RtmpConnection).CacheMessage

go-oryx use writev to send:
  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 7134 winlin    20   0 1225m 786m 3412 S 207.6  4.9   2:52.78 oryx
 2560 root      20   0 1344m  36m 5024 S 68.3  0.2  40:25.03 bravo_data_anal
14664 winlin    20   0  435m 305m 1528 S 56.4  1.9   1:40.16 sb_rtmp_load
13920 winlin    20   0  434m 305m 1528 S 55.7  1.9   1:39.06 sb_rtmp_load
15658 winlin    20   0  434m 304m 1528 S 55.7  1.9   1:37.81 sb_rtmp_load

----total-cpu-usage---- -dsk/total- ---net/lo-- ---paging-- ---system--
usr sys idl wai hiq siq| read  writ| recv  send|  in   out | int   csw
 22   9  64   0   0   3|   0    76k| 204M  204M|   0     0 |  36k   64k
 19   8  69   1   0   3|   0   652k| 189M  189M|   0     0 |  25k   48k
 20   7  70   0   0   3|4096B   48k| 109M  109M|   0     0 |  26k   47k
 29   7  61   1   0   2|   0    32k| 170M  170M|   0     0 |  31k   59k
 19   8  70   0   0   3|   0    16k| 246M  246M|   0     0 |  24k   42k

Cpu profile:
[winlin@ftp go-oryx]$ go tool pprof objs/oryx cpu.prof
Entering interactive mode (type "help" for commands)
(pprof) top
207.34s of 332.93s total (62.28%)
Dropped 460 nodes (cum <= 1.66s)
Showing top 10 nodes out of 127 (cum >= 24.93s)
      flat  flat%   sum%        cum   cum%
   107.14s 32.18% 32.18%    109.54s 32.90%  syscall.Syscall
    23.06s  6.93% 39.11%    167.49s 50.31%  github.com/ossrs/go-oryx/protocol.(*RtmpStack).SendMessage
    18.22s  5.47% 44.58%        38s 11.41%  runtime.scanobject
    12.18s  3.66% 48.24%     28.46s  8.55%  runtime.selectgoImpl
     9.93s  2.98% 51.22%      9.93s  2.98%  runtime.heapBitsForObject
     8.89s  2.67% 53.89%     13.78s  4.14%  runtime.greyobject
     8.17s  2.45% 56.35%     37.90s 11.38%  runtime.mallocgc
     7.30s  2.19% 58.54%      7.30s  2.19%  github.com/ossrs/go-oryx/agent.(*Jitter).Correct
     6.42s  1.93% 60.47%     38.63s 11.60%  github.com/ossrs/go-oryx/agent.(*RtmpPlayAgent).Write
     6.03s  1.81% 62.28%     24.93s  7.49%  github.com/ossrs/go-oryx/protocol.(*RtmpConnection).CacheMessage

Seems there are still lots of syscalls. What's wrong with it? The writev code is https://github.com/winlinvip/go/pull/1/files

I adjust the group of videos to send to client, I test the 10k clients play RTMP from go-oryx:
  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 7602 winlin    20   0 8053m 7.0g 3432 S 253.6 45.2   9:43.28 oryx
14664 winlin    20   0  435m 422m 1528 S 38.7  2.7   9:13.22 sb_rtmp_load
15658 winlin    20   0  435m 421m 1528 S 38.4  2.6   9:17.62 sb_rtmp_load
10412 winlin    20   0  435m 342m 1528 S 38.1  2.1   6:03.18 sb_rtmp_load
13920 winlin    20   0  434m 421m 1528 S 37.7  2.6   9:12.67 sb_rtmp_load
24969 winlin    20   0  435m 422m 1528 S 37.4  2.6   6:18.85 sb_rtmp_load

It's great~

+1

@winlinvip It seems syscalls grow, I guess there are other side effect syscalls arisen.  From the diff maybe some of these calls ?
    18.22s  5.47% 44.58%        38s 11.41%  runtime.scanobject
    12.18s  3.66% 48.24%     28.46s  8.55%  runtime.selectgoImpl
     9.93s  2.98% 51.22%      9.93s  2.98%  runtime.heapBitsForObject
     8.89s  2.67% 53.89%     13.78s  4.14%  runtime.greyobject
     8.17s  2.45% 56.35%     37.90s 11.38%  runtime.mallocgc

@ggaaooppeenngg, selectGoImpl is about select. The rest are garbage collection: paying the cost of allocating memory.

A similar implemnetation for writev by coroutine:
ssize_t st_writev(_st_netfd_t *fd, const struct iovec *iov, int iov_size, st_utime_t timeout)
{
    ssize_t n, rv;
    size_t nleft, nbyte;
    int index, iov_cnt;
    struct iovec *tmp_iov;
    struct iovec local_iov[_LOCAL_MAXIOV];

    /* Calculate the total number of bytes to be sent */
    nbyte = 0;
    for (index = 0; index < iov_size; index++) {
        nbyte += iov[index].iov_len;
    }

    rv = (ssize_t)nbyte;
    nleft = nbyte;
    tmp_iov = (struct iovec *) iov;	/* we promise not to modify iov */
    iov_cnt = iov_size;

    while (nleft > 0) {
        if (iov_cnt == 1) {
            if (st_write(fd, tmp_iov[0].iov_base, nleft, timeout) != (ssize_t) nleft) {
                rv = -1;
            }
            break;
        }

        if ((n = writev(fd->osfd, tmp_iov, iov_cnt)) < 0) {
            if (errno == EINTR) {
                continue;
            }
            if (!_IO_NOT_READY_ERROR) {
                rv = -1;
                break;
            }
        } else {
            if ((size_t) n == nleft) {
                break;
            }
            nleft -= n;
            /* Find the next unwritten vector */
            n = (ssize_t)(nbyte - nleft);
            for (index = 0; (size_t) n >= iov[index].iov_len; index++) {
                n -= iov[index].iov_len;
            }

            if (tmp_iov == iov) {
                /* Must copy iov's around */
                if (iov_size - index <= _LOCAL_MAXIOV) {
                    tmp_iov = local_iov;
                } else {
                    tmp_iov = calloc(1, (iov_size - index) * sizeof(struct iovec));
                    if (tmp_iov == NULL) {
                        return -1;
                    }
                }
            }

            /* Fill in the first partial read */
            tmp_iov[0].iov_base = &(((char *)iov[index].iov_base)[n]);
            tmp_iov[0].iov_len = iov[index].iov_len - n;
            index++;
            /* Copy the remaining vectors */
            for (iov_cnt = 1; index < iov_size; iov_cnt++, index++) {
                tmp_iov[iov_cnt].iov_base = iov[index].iov_base;
                tmp_iov[iov_cnt].iov_len = iov[index].iov_len;
            }
        }

        /* Wait until the socket becomes writable */
        if (st_netfd_poll(fd, POLLOUT, timeout) < 0) {
            rv = -1;
            break;
        }
    }

    if (tmp_iov != iov && tmp_iov != local_iov) {
        free(tmp_iov);
    }

    return rv;
}

@winlinvip your C++ version  struct iovec local_iov[_LOCAL_MAXIOV]; is allocated in the stack, your Go version on the heap.

@Terry-Mao I will cache the iovs to avoid alloc in heap.
I refined the go-oryx code, descrease the chunks of video.
The clients is 10k, video is 300kbps.

OS
API
CPU
MEM
GC
Connections
Bitrate

Linux
Write
160%
5.4G
40ms
10k
300kbps

Linux
Writev
140%
1.5G
30ms
10k
300kbps

Conclude:

The writev(1440%) is almost the same than write(160%).
The writev(1.5G) use less memory than write(5.4G), for writev avoid of copy to a big buffer.
The writev gc(30ms) use less time than write(40ms).

The writev(1.5G) use less memory than write(5.4G), for writev avoid of copy to a big buffer.
The writev gc(30ms) use less time than write(40ms).

Will golang accept the proposal to support netFD.Writev?

@winlinvip, thanks for prototyping.
Can you make a small stand-alone program for profiling that can do either multiple writes or writev (controlled by a flag) and link to the code so others can reproduce your results and investigate more?
I also have an API idea. Instead of adding API like (*TCPConn).Writev([][]byte), we could have a special type that implements io.WriterTo that represents the multiple buffers (the [][]byte) and TCPConn's ReadFrom can special-case it into a writev internally, just like it does sendfile.
Then we can add the writev optimization gradually to different OS/conn types over time.

A time one especially wants writev is when writing out a UDP packet where the data comes from multiple places.  I guess you could do that using io.MultiReader with a list of bytes.Buffer values: io.Copy that to the UDPConn.  Sounds a little baroque, though.  Also the UDPConn ReadFrom method is not the one that io.ReaderFrom wants.  Hmmm.

@winlinvip,
Can you please change the description appropriately? For example, "proposal: net: do blah blah" (see https://github.com/golang/proposal for further information) because I'm still not sure what's the original objective; it's just to have scatter-gather IO APIs, or more stuff including fancy packet assembly framework and primitives. A few random thoughts.

Please use sendmsg instead of writev internally. Both have the same scatter-gather functionality but sendmsg is more useful for datagram-based unconnected-mode sockets and we can reuse it for #7882 later.

I think it's fine to have different scatter-gather IO APIs between stream-based connected-mode sockets and datagram-based unconnected-mode sockets because as @ianlancetaylor mentioned, interfaces defined in io package are not designed for the latter. Fortunately we can use UDPConn in both styles by using two connection setup functions Dial and ListenPacket appropriately.

@mikioh I changed the description to "proposal: net: TCPConn supports Writev", because I think at least the writev is very useful for tcp server. Maybe udp server need this api, but I am not sure.
@ianlancetaylor I only write a udp server(rtmfp server), and I am not sure whether writev is useful for udp servers, I will do some research.
@bradfitz I will write a standalone program for profiling the write or writev. I will rewrite the program go-writev to use golang code only(the c++ code will move to another branch).
About the api and implementation, for example, to use sendmsg or ReadFrom or MultipleReader, I need some time to study the code.

@bradfitz I have rewrite the go-writev to compare the write and writev, it's awesome. The code is go-writev
The bellow is benchmarks.
Darwin

OS
API
CPU
Memory
Bandwidth

Darwin
Write
99%
1.7M
2230.75Mbps

Darwin
Write
119%
1.8M
28618.23Mbps

Writev(28Gbps) is 10 times than Write(2.2Gbps).
The profile for Write:
winlin:tcpserver winlin$ go tool pprof ./tcpserver cpu.prof
Entering interactive mode (type "help" for commands)
(pprof) top
57.61s of 57.75s total (99.76%)
Dropped 16 nodes (cum <= 0.29s)
      flat  flat%   sum%        cum   cum%
    57.60s 99.74% 99.74%     57.62s 99.77%  syscall.Syscall
     0.01s 0.017% 99.76%     57.63s 99.79%  syscall.write
         0     0% 99.76%     57.67s 99.86%  main.main
         0     0% 99.76%     57.67s 99.86%  main.srs_serve
         0     0% 99.76%     57.67s 99.86%  net.(*conn).Write
         0     0% 99.76%     57.67s 99.86%  net.(*netFD).Write
         0     0% 99.76%     57.67s 99.86%  runtime.goexit
         0     0% 99.76%     57.67s 99.86%  runtime.main
         0     0% 99.76%     57.63s 99.79%  syscall.Write
(pprof)

The profile for Writev:
winlin:tcpserver winlin$ go tool pprof ./tcpserver cpu.prof
Entering interactive mode (type "help" for commands)
(pprof) top
103.27s of 103.39s total (99.88%)
Dropped 13 nodes (cum <= 0.52s)
Showing top 10 nodes out of 40 (cum >= 7.60s)
      flat  flat%   sum%        cum   cum%
    77.81s 75.26% 75.26%     77.81s 75.26%  runtime.mach_semaphore_signal
    11.23s 10.86% 86.12%     88.09s 85.20%  syscall.Syscall
     7.57s  7.32% 93.44%      7.57s  7.32%  runtime.kevent
     3.49s  3.38% 96.82%      3.49s  3.38%  runtime.usleep
     1.98s  1.92% 98.73%      1.98s  1.92%  runtime.mach_semaphore_wait
     1.07s  1.03% 99.77%      1.07s  1.03%  runtime.mach_semaphore_timedwait
     0.06s 0.058% 99.83%     88.15s 85.26%  net.(*netFD).Writev
     0.04s 0.039% 99.86%     79.96s 77.34%  runtime.systemstack
     0.01s 0.0097% 99.87%      9.63s  9.31%  runtime.findrunnable
     0.01s 0.0097% 99.88%      7.60s  7.35%  runtime.netpoll
(pprof)

Your example at https://github.com/winlinvip/go-writev/blob/master/tcpserver/server.go#L156 isn't exactly fair to the non-writev case. It sends a packet per write. Can you also add a mode where you use a bufio.Writer and write each small []byte to the bufio.Writer, then bufio.Writer.Flush at the end?

@bradfitz I use the bufio, it's better then multiple writes, but bad than writev. The code is go-writev
The bellow is benchmarks.
Darwin

OS
API
CPU
Memory
Bandwidth

Darwin
Write
99%
1.7M
2230.75Mbps

Darwin
Writev
119%
1.8M
28618.23Mbps

Darwin
Write+bufio
122%
2.3M
30732.18Mbps

Writev or Write+bufio (28Gbps) is 10 times than Write(2.2Gbps), it's awesome.
Write+bufio use the max memory for cache.
 But the bufio copy all iovs to buffer, maybe this test did not show this point. I will use bufio in go-oryx and give out the profile.

I refine the code, to always make the packets and send it, because the server will always recv packets from some clients and delivery the packets to another clients. The cr is here.

OS
API
CPU
Memory
Bandwidth

Darwin
Write+bufio
98%
10M
3283.85Mbps

Darwin
Writev
126%
6M
13674.30Mbps

The profile for Write+bufio:
winlin:tcpserver winlin$ go tool pprof ./tcpserver cpu.prof
45.21s of 45.58s total (99.19%)
Dropped 46 nodes (cum <= 0.23s)
Showing top 10 nodes out of 22 (cum >= 0.31s)
      flat  flat%   sum%        cum   cum%
    43.72s 95.92% 95.92%     43.74s 95.96%  syscall.Syscall
     1.12s  2.46% 98.38%      1.12s  2.46%  runtime.kevent
     0.27s  0.59% 98.97%      0.27s  0.59%  runtime.memmove

The profile for Writev:
winlin:tcpserver winlin$ go tool pprof ./tcpserver cpu.prof
62.14s of 63.93s total (97.20%)
Dropped 70 nodes (cum <= 0.32s)
Showing top 10 nodes out of 57 (cum >= 1.45s)
      flat  flat%   sum%        cum   cum%
    21.73s 33.99% 33.99%     39.19s 61.30%  syscall.Syscall
    17.67s 27.64% 61.63%     17.67s 27.64%  runtime.mach_semaphore_signal
    10.90s 17.05% 78.68%     10.90s 17.05%  runtime.kevent
     7.68s 12.01% 90.69%      7.68s 12.01%  runtime.mach_semaphore_wait
     2.41s  3.77% 94.46%      2.41s  3.77%  runtime.usleep

Seems the writev is 4 times faster than Write+bufio.

@winlinvip, but you're allocating per iteration. Let's leave all GC out of this benchmark.

@bradfitz Seems have nothing todo with gc, the writev also alloc for each send.

@bradfitz Sorry, it's a bug for write+bufio, I should NewWriterSize with the len(iovecs) to avoid multiple syscalls. I have fix it at here.
 I will test multiple clients about 10k and server with 8cpus.

I test the server with 8CPU. And client with 4CPU, which starts 2500 client(each use a goroutine) to recv packets from server.
The command to start server and client:
# start server, with 8CPU, write+bufio.
./tcpserver --port=2015 --cpus=8 --writev=false
# start server, with 8CPU, writev.
./tcpserver --port=2015 --cpus=8 --writev=true
# start client, with 4CPU, start 2500 goroutines.
./tcpclient --port=2015 --cpus=4 --clients=2500

The benchmark is:

OS
API
CPU
MEM
Bandwidth
GC Pause
ClientCPU

Linux
Write+bufio
188.4%
12M
6508.05Mbps
315.857µs
52.4%

Linux
Writev
185.8%
12M
6334.07Mbps
338.376µs
61.3%

The profile for write+bufio:
[winlin@dell-demo tcpserver]$ go tool pprof ./tcpserver cpu.prof
399.72s of 662.13s total (60.37%)
Dropped 179 nodes (cum <= 3.31s)
Showing top 10 nodes out of 109 (cum >= 45.50s)
      flat  flat%   sum%        cum   cum%
   141.49s 21.37% 21.37%    142.10s 21.46%  syscall.Syscall
    65.26s  9.86% 31.22%     65.26s  9.86%  runtime.futex
    29.29s  4.42% 35.65%     29.29s  4.42%  runtime.memclr
    26.60s  4.02% 39.67%     35.83s  5.41%  runtime.scanobject
    26.56s  4.01% 43.68%     26.56s  4.01%  runtime.memmove
    26.15s  3.95% 47.63%     26.15s  3.95%  runtime.xchg
    25.36s  3.83% 51.46%     29.56s  4.46%  runtime.scanblock
    19.99s  3.02% 54.48%     19.99s  3.02%  runtime.mSpan_Sweep.func1
    19.58s  2.96% 57.43%     97.85s 14.78%  runtime.mallocgc
    19.44s  2.94% 60.37%     45.50s  6.87%  bufio.(*Writer).Write

The profile for writev:
[winlin@dell-demo tcpserver]$ go tool pprof ./tcpserver cpu.prof
198.70s of 297.61s total (66.77%)
Dropped 163 nodes (cum <= 1.49s)
Showing top 10 nodes out of 103 (cum >= 56s)
      flat  flat%   sum%        cum   cum%
    92.53s 31.09% 31.09%     92.88s 31.21%  syscall.Syscall
    28.65s  9.63% 40.72%     28.65s  9.63%  runtime.futex
    12.78s  4.29% 45.01%     12.78s  4.29%  runtime.memclr
    11.72s  3.94% 48.95%     17.58s  5.91%  runtime.scanobject
    10.92s  3.67% 52.62%     10.92s  3.67%  runtime.xchg
     9.67s  3.25% 55.87%     10.96s  3.68%  runtime.scanblock
     9.23s  3.10% 58.97%      9.23s  3.10%  runtime.mSpan_Sweep.func1
     8.68s  2.92% 61.89%     43.84s 14.73%  runtime.mallocgc
     7.40s  2.49% 64.37%     48.41s 16.27%  runtime.findrunnable
     7.12s  2.39% 66.77%        56s 18.82%  runtime.makeslice

Seems the write+bufio is same to writev, it's great. I will test multiple clients.

@bradfitz To let GC out of this benchmark, I fix by commit0 and commit.

I test the server with one and multiple clients.
The benchmark is:

ID
OS
API
CPU
MEM
BW
GC
C
CCPU

1
Linux
Write
100%
8M
12810Mbps
0
1
78.5%

2
Linux
Writev
100.3%
7M
11519Mbps
0
1
105.3%

3
Linux
Write
800.1%
546M
77379Mbps
976µs
1000
799.9%

4
Linux
Writev
789.7%
39M
78261.76Mbps
2.19ms
1000
723.7%

5
Linux
Write
804.3%
1.1G
58926.29Mbps
3.63ms
2000
798.0%

6
Linux
Writev
790.1%
68M
77904.09Mbps
4.07ms
2000
674.5%

7
Linux
Write
1246.2%
1.6G
58800.03Mbps
3.15ms
3000
800.6%

8
Linux
Writev
802.6%
73M
78083.27Mbps
2.84ms
3000
704.9%

9
Linux
Write
597.8%
2.1G
63852.38Mbps
6.5ms
4000
599.8%

10
Linux
Writev
593.3%
89M
68355.00Mbps
6.4ms
4000
510.1%

11
Linux
Write
601.3%
2.6G
73028.46Mbps
1.06s
5000
542.6%

12
Linux
Writev
591.8%
114M
71373.89Mbps
6.5ms
5000
525.2%

13
Linux
Write
590.3%
3.1G
72853.05Mbps
8.04ms
6000
533.7%

14
Linux
Writev
598.6%
142M
82191.27Mbps
13ms
6000
542.5%

15
Linux
Write
599.8%
3.7G
71342.40Mbps
9.49ms
7000
547.4%

16
Linux
Writev
592.7%
154M
67619.66Mbps
26ms
7000
515.9%

17
Linux
Write
598.6%
5.2G
73846.97Mbps
7.02ms
10000
563.2%

18
Linux
Writev
601.8%
267M
70770.94Mbps
22.5ms
10000
543.9%

Remark:

C: The number of goroutine for client.
CCPU: The cpu usage for clients.
BW: The bandwidth consumed for server and clients.
API: Use writev or write(+bufio) API.
ID1-8: Server use 8CPUs, client use 8CPUs.
ID9-18: Server use 6CPUs, client use 6CPUs.

Thank you for the data and prototyping but we will not be looking at this until February, after Go 1.6 is out. I want to look at it more in depth but don't have the time at the moment.

@bradfitz Got it, thanks~

This looks like a good candidate for a specialized version of io.MultiWriter (just as http.MaxBytesReader is a specialization of io.LimitReader)

How could we optimize io.MultiWriter with Writev?
Writev is about writing multiple buffers to one fd, whereas
io.MultiWriter is about writing the same buffer to multiple
io.Writer's.

FYI, Java has a separate interface to support writev (https://docs.oracle.com/javase/7/docs/api/java/nio/channels/GatheringByteChannel.html).
How about
package io

type GatheringWriter interface {
    func Write(bufs byte[][])
}

?

@kostya-sh, that's not even Go syntax for a number of reasons. Go and Java have very different feels, too.
Let's discuss this more in February.

@bradfitz, the syntax can be corrected (and of course the method cannot be called Write because it would clash with io.Writer interface). This is just a suggestion that you might want to consider.

@Terry-Mao @davecheney @ianlancetaylor @bradfitz @kostya-sh @ggaaooppeenngg @mikioh @extemporalgenome @minux Seems writev is attractive for all of us, please update this post when discuss writev in February :smile_cat:

@minux ah. I misread it as writing one buffer to multiple fd's (and I wasn't suggesting modifying io.MultiWriter itself -- but rather adding a specialized function [which is now moot] to the net package)

And, should add new method to support sending multiple WSABuf with WSASend/WSASendto (similar to writev() and struct iovec) for windows.
### src/net/fd_windows.go:L502 Single WSABuf sending

func (fd *netFD) Write(buf []byte) (int, error) {
	if err := fd.writeLock(); err != nil {
		return 0, err
	}
	defer fd.writeUnlock()
	if raceenabled {
		raceReleaseMerge(unsafe.Pointer(&ioSync))
	}
	o := &fd.wop
	o.InitBuf(buf)
	n, err := wsrv.ExecIO(o, "WSASend", func(o *operation) error {
		 #### here arg 2,3 should be *WSAbuf, count ###
		return syscall.WSASend(o.fd.sysfd, &o.buf, 1, &o.qty, 0, &o.o, nil)
	})
	if _, ok := err.(syscall.Errno); ok {
		err = os.NewSyscallError("wsasend", err)
	}
	return n, err
}

@bradfitz ping.
Would it be possible for this to be revisited, since the release of Go >= 1.6?
Thank you!

@mbeacom, sorry, this also will not make Go 1.7. The 1.7 development freeze is May 1st and this isn't ready.

ping @bradfitz

@bradfitz What is the likelihood of this moving into the Go 1.8 Maybe milestone?

@ianlancetaylor and I are happy with this.
API-wise, though, I'm thinking:
package net

// Buffers contains zero or more runs of bytes to write.
//
// On certain machines, for certain types of connections, this is optimized
// into an OS-specific batch write operation.
type Buffers [][]byte

func (v Buffers) WriteTo(w io.Writer) (n int64, err error)

@bradfitz would we do func (v Buffers) ReadFrom(r io.Reader) (n int64, err error) for readv as well?
(Your proposed docstring says that Buffers is "bytes to write".)

Perhaps. But let's leave readv for another day.

@bradfitz: What is the expected behavior if WriteTo is called with zero runs of bytes to write? I would prefer that it is documented. (Both "does nothing" and "calls write with a zero-length buffer" are acceptable responses, I think.)

Why would a user care? I'd prefer the future implementation freedom and documentation clarity from leaving it unspecified.
Also, do you mean zero overall or an entry with zero length? Because currently (for ease of implementation), I do nothing if the overall vector is zero, but if one of the vector entries is empty, I still include it in the set, but zero length. I'd prefer to not get into that level of detail in docs.

I sent out https://golang.org/cl/29951
It works, but the thing I don't like about it is net.Buffers, despite being an io.WriterTo is not an io.Reader, so it can't be used as the src of an io.Copy call, which is kinda weird. The whole point of io.ReaderFrom and io.WriterTo was to optimize io.Copy, so it feels weird to have any WriterTo which is not a Reader.
But I can't make net.Buffers a Reader because then it would need a pointer receiver and record state, which is kinda disgusting, especially if the writev EAGAIN loop already needs to maintain that state (see the CL). I'd prefer that state be local to the loop and not required to be allocated by the user in every write call, since the whole point of this is to avoid allocations.

I think a user would care because we might have the behavior that a Conn that is closed or otherwise unusable might or might not return an error if you try to write zero bytes. That seems confusing to me. I guess it looks like we currently also don't promise what will happen if Write is called with a zero-length slice?

unusable might or might not return an error if you try to write zero bytes.

The initial validity check in the CL (if !c.ok() {) would still catch that. That's like all the methods which operate on the netFD.

I guess it looks like we currently also don't promise what will happen if Write is called with a zero-length slice?

Exactly. Zero-length reads and writes have historically been very ill-defined in Go. In general, we promise nothing.

👍 Nice writev~


你好，可以增加ppt导入功能吗？

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
Describe the solution you'd like
A clear and concise description of what you want to happen.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.

星球内有将PPT转为 json 然后导入到画布中的相关资料 有需要可以加星球了解下


When too many columns are pinned the viewport wraps around to multiple lines

view demo:
http://plnkr.co/edit/ALZWKqiqOPsiJ428hhYO?p=preview

Is this still happening @c0bra, @swalters?  I think that we've been playing with pinning and the new scrolling, and this might still be happening in some circumstances?

Still happening.  Current answer is to put a scrollbar on the pinned container, so that it doesn't break rendering.  Future enhancement is to resize the pinned v's body container.

Is this now solved with the latest sizing updates?  I can't recreate the issue in the plunker, but I'm not sure I ever could.

Closing: solution proposed that should resolve the issue, no further comments for 2 weeks, assuming solution has resolved issue.


Add support for SSH tunnels

This should be an option to connect to the database instead of connecting to the Postgres host directly.
We'll need to:

Add a new option to the "Connect Database" page
Update the connect-database endpoint to support the SSH options (server host, connection port, login username, key or password authentication)
Support SSH key authentication or password authentication

More info can be found with how Airbyte supports this type of connection: https://docs.airbyte.com/integrations/sources/postgres/#connect-via-ssh-tunnel

No description provided.

@ciminelli looking for more description.

No description provided.

@ciminelli looking for more description.

Just updated the description, let me know if this is something you'd be interested in and I can work with you on it!

Yeah, This is my understanding
current workflow, install inquery and then add details of postgres host, then connect. So now instead of connecting directly to postgres host, now we can have "Connect Database" page, so this will connect using ssh as per this doc. Am i right ? please feel free to correct 😄.

Yeah, This is my understanding: Current workflow, install inquery and then add details of postgres host, then connect. So now instead of connecting directly to postgres host, now will have "Connect Database" page, so this will connect using ssh as per this doc. Am i right ? please feel free to correct 😄.

Yes, but the SSH connection option will be right on the connect database page! We can make a dropdown to select the SSH connection option.


Added information to Audio section and created new section for credit…

… card information
Changes proposed in this pull request:

The credentials to view the Netlify deploy is the following:

username: crabigator
password: googlebegone

Added audio gif with Kyoko and Kenichi's voices displayed.


Add tenant menu to top nav if activated

The tenant menu not showing when using top navigation is a known limitation.
I don't think we should just put it next to the user menu though.
Probably we should refactor the tenant and user menus to take action objects instead of menu items. We could then make the tenant menu and item in the user menu dropdown.

The tenant menu being displayed beside the user menu makes it clear what tenant you're using without having to click anything.  This is consistent with how the tenant menu is displayed in the sidebar nav.

The sidebar has a lot of available space vertically, hence the tenant menu is shown by default. The top bar however, doesn't have the space to show the tenant menu on smaller screens. This PR would make it impossible for users to access the tenant menu on those devices.

Currently, when top navigation is enabled, all the navigation including the tenant menu is contained in the collapsible sidebar on smaller screens.  This PR only shows the tenant menu on desktop screens.  It hides the tenant menu on smaller screens since it's already contained in the collapsible sidebar nav.

On mobile with this PR

No worries.  I'll try to help with some issues as I have time.

Hah, just faced this issue... i was about to create a PR with this exact code and came in my mind to search on issues before i open it... hah...
For now i just published resources/views/vendor/filament-panels/components/topbar/index.blade.php and added this... so not a blocker to anyone, just some investigation needed to workaround it.
I think having it in there, even if it doesn't look ideal is better than just not having at all...

Also if it's possible to make it searchable :)
Thanks

Implemented in https://github.com/filamentphp/filament/commit/17f241e29e6d841631c782edcb3cde69e7da27b9.
Thanks for your patience, folks!

@zepfietje you didn't add the filament()->hasTopNavigation() && check, won't it show twice when using the sidebar nav?

No, because the parent is wrapped in that conditional.


Development environment issues, ngrok?

Anybody having issues in your development environment?
I find that messages/postbacks are frequently ignored. It will start working again if I wait a while or interact with the bot with other messages/postbacks, but this is obviously a major pain during development.
Anybody else with similar problems? Could it be Ngrok?

I believe this is more of a Facebook issue.
I develop for both kik and fb without the use of this gem, and it seems that when ever fb doesn't get a response, they put a sleep on sending you additional messages.
Likely a good feature if there's explosions happening they don't keep bombarding your webhook.
Kik doesn't care... they keep firing in the hole! LOL

@ajose01 What do you mean when FB doesn't get a response?

If you're in you're dev environment, and something goes wrong you might not respond with a 200.
So when this happens I've noticed FB puts a slight hold before they start sending you packets again.
If you look at ngrok, you won't see any traffic coming through.

Hmm, the problem I have happens even when something doesn't go wrong... meaning after a few minutes everything works when no code was changed.

Only thing I can think of at the moment is maybe you're not responding with a 200 status code right away?

Okay, maybe thats the case.
If you don't mind me asking, what are you using to return 200?

Nevermind, silly question.

Actually, it didn't fix my issue... so I guess returning 200 isn't the issue.

It happens to me too. How to make it so that facebook stop ignoring messages/postbacks?

I've been developing with ngrok, too, but I haven't experienced this. Are you sure that you're responding to all of Facebook's requests to your bot? It'd be helpful to see logs.

I'm closing this pending logs.


Delete calendar using Kotlin/Java SDK

PR Acceptance Criteria
Fixes #64
Guidelines

[ ] PR follows the Contributing guidelines?
[ ] PR Adheres to our Code of Conduct?
[ ] Have you checked to ensure there aren't other open Pull Requests for the same update/change?

Code complete

[ ] A README.md available describing how to setup and test the code?
[ ] The code sample in the correct location (i.e. calendar/java/delete-calendar)?
[ ] Link added to the repository readme (view # Code Snippets created by the Nylas Community with examples) describing what the code sample does for others to try out
[ ] The code can be run using Github Codespaces, so the maintainer can try out the code sample remotely to ensure everything is working before we accept the PR.

For PR Reviewer use

[ ] Purpose of code sample is clear
[ ] The code sample is readable
[ ] Steps try out the code are provided
[ ] Code sample runs on codespaces environment
[ ] Code works as expected
[ ] PR labels added

Done👍

@atejada Any other change required?

Thanks @NikhilTaneja98 Merging this PR 😊


[AVM Module Issue]: Metric alert with multiple scopes of type WebtestLocationAvailabilityCriteria fails with multiple scopes

Check for previous/existing GitHub issues

[X] I have checked for previous/existing GitHub issues

Issue Type?
Bug
Module Name
avm/res/insights/metric-alert
(Optional) Module Version
0.1.3
Description
I've discovered a weird issue with AVM for Metric Alerts that doesn't happen if I declare metric alert resource with Bicep explicitly, i.e. without using the AVM.
This error happens when I'm using metric alert of type 'Microsoft.Azure.Monitor.WebtestLocationAvailabilityCriteria' with multiple scopes being set, which in my case are Web Test resource ID and Application Insights resource ID. Main error message is: "Alert scope is invalid."
Bicep AVM code that FAILS looks like this:
module metricAlert 'br/public:avm/res/insights/metric-alert:0.1.3' = {
  name: 'maDeployment-${deploymentTimestamp}'
  scope: metricAlertResourceGroup
  params: {
    alertDescription: 'Response time alert'
    enabled: true
    severity: 3
    name: metricAlertName
    location: 'global'
    windowSize: 'PT5M'
    evaluationFrequency: 'PT1M'
    alertCriteriaType: 'Microsoft.Azure.Monitor.WebtestLocationAvailabilityCriteria'
    targetResourceRegion: resourceLocation
    targetResourceType: 'Microsoft.Insights/webtests'
    criterias: [
      {
        componentId: appInsightsResourceId
        failedLocationCount: 3
        webTestId: webtestResourceId
      }
    ]
    actions: [
      actionGroupResourceId
    ]
    scopes: [
      webtestResourceId
      appInsightsResourceId
    ]
    tags: union(resourceTags, {
      'hidden-link:${appInsightsResourceId}': 'Resource'
      'hidden-link:${webtestResourceId}': 'Resource'
    })
  }
}

Bicep resource WITHOUT AVM that WORKS looks like this:
resource metricAlert 'Microsoft.Insights/metricAlerts@2018-03-01' = {
  name: metricAlertName
  location: alertLocation
  tags: union(resourceTags, {
    'hidden-link:${appInsightsResourceId}': 'Resource'
    'hidden-link:${webtestResourceId}': 'Resource'
  })
  properties: {
    description: 'Response time alert'
    enabled: true
    severity: 3
    autoMitigate: true
    windowSize: 'PT5M'
    evaluationFrequency: 'PT1M'
    targetResourceRegion: resourceLocation
    targetResourceType: 'Microsoft.Insights/webtests'
    criteria: {
      'odata.type': 'Microsoft.Azure.Monitor.WebtestLocationAvailabilityCriteria'
      componentId: appInsightsResourceId
      failedLocationCount: 3
      webTestId: webtestResourceId
    }
    actions: [
      {
        actionGroupId: actionGroupResourceId
      }
    ]
    scopes: [
      webtestResourceId
      appInsightsResourceId
    ]
  }
}

This is the error message:
New-AzSubscriptionDeployment : 1:06:41 PM - The deployment 'wtSubDeploymentc6f2f9b9' failed with error(s). Showing 3 out of 3 error(s).
Status Message: Alert scope is invalid. Activity ID: 7ce8ceb2-0c71-43ef-813e-51167c918fc8. (Code:BadRequest)
Status Message: At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage
details. (Code: DeploymentFailed)
 - {
  "code": "BadRequest",
  "message": "Alert scope is invalid. Activity ID: 7ce8ceb2-0c71-43ef-813e-51167c918fc8."
} (Code:BadRequest)
Status Message: At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage
details. (Code: DeploymentFailed)
 - The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'. (Code: ResourceDeploymentFailure)
   - At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.
(Code: DeploymentFailed)
     - {
  "code": "BadRequest",
  "message": "Alert scope is invalid. Activity ID: 7ce8ceb2-0c71-43ef-813e-51167c918fc8."
} (Code:BadRequest)
CorrelationId: 0bbb1ed9-cbcb-4b61-81f4-d3b7805573d0
At line:1 char:1
+ New-AzSubscriptionDeployment -Location $subscriptionLocation -Name "w ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [New-AzDeployment], Exception
    + FullyQualifiedErrorId : Microsoft.Azure.Commands.ResourceManager.Cmdlets.Implementation.NewAzureSubscriptionDeploymentCmdlet

I followed the README for the AVM to implement the properties as shown in the examples in the file.
One thing I noted is the description for targetResourceRegion and targetResourceType properties. In AVM documentation it states that "Required if alertCriteriaType is MultipleResourceMultipleMetricCriteria.". But according to the REST API documentation it also states that these properties are also required when multiple scopes are provided: "Mandatory if the scope contains a subscription, resource group, or more than one resource.". - Which I assume would also fall under all alert types and not only MultipleResourceMultipleMetricCriteria? If that's the case, current statement in AVM documentation may be easy to misinterpret.
(Optional) Correlation Id
0bbb1ed9-cbcb-4b61-81f4-d3b7805573d0

Hey @kijunkang, I just assigned the issue to you as the module owner. Please take a look at your earliest convenice to at least triage the issue (and remove the Needs: Triage Label)

Hey @kijunkang, I just assigned the issue to you as the module owner. Please take a look at your earliest convenice to at least triage the issue (and remove the Needs: Triage Label)

Thank you @AlexanderSehr, I will start looking into this issue.

Hi @guidemetothemoon We identified the issue and currently working on it.


too big usage

Windows 10 x64

Need more information.

This is usage in started/stoped service in NON SERP SCAN mode

how many keywords in the database ? anyway it shouldn't do that. Do you have an antivirus or something ? If yes, shut it down and tell me how things are going

When MCAFEE is disabled usage is normal

I'm afraid there is little I can do about that. Only McAfee can resolve this issue I think. Maybe you can whitelist serposcope processes ? Or maybe it's just a random scan and it will stop after a while.


pe functionaility, auto cutoff

Implemented Paired end functionality for building indices and read classifier
Implemented auto cutoff (algorithm found in Cortex and mccortex) to infer threshold for k-mer filtering if no specific threshold is supplied with the -f flag.

Implemented Paired end functionality for building indices and read classifier
Implemented auto cutoff (algorithm found in Cortex and mccortex) to infer threshold for k-mer filtering if no specific threshold is supplied with the -f flag.


Microsoft.Network/stable/2016-06-01/network.json has semantic validation errors

See https://github.com/Azure/azure-rest-api-specs/pull/3785 for the failures.
[ { code: 'OBJECT_MISSING_REQUIRED_PROPERTY_DEFINITION',
message: 'Missing required property definition: nextHopType',
path: [ 'definitions', 'ExpressRouteCircuitRoutesTable' ] },
{ code: 'OBJECT_MISSING_REQUIRED_PROPERTY_DEFINITION',
message: 'Missing required property definition: nextHopType',
path: [ 'definitions', 'ExpressRouteCircuitRoutesTableSummary' ] },
{ code: 'EQUIVALENT_PATH',
message: 'Equivalent path already exists: /subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Network/connections/{virtualNetworkGatewayConnectionName}/sharedkey',
path:
[ 'paths',
'/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Network/connections/{virtualNetworkGatewayConnectionName}/sharedkey' ] } ],

foxed in Ignite release


Create zh_CN.lang

Half way done, still a working-in-progress, don't merge this for now
Essentially following the version provided by @crafteverywhere , he always gives us surprise on dictions, syntaxes and many other details.
Also use all fullwidth punctuation.
To-do:

[ ] Curiosity. I doubt it is related to the new research system, but I have no access to the beta version of TC6 so I cannot confirm.
[ ] Primordial Mote, Primordial Nodule. Same as Curiosity.
[ ] Caster's Gauntlet. Definitely related to the new Focus system.
[ ] Structure Magic. Guess there is terminology floating around...
[ ] Knowledge types. Same as Curiosity, citation needed, context needed...
[ ] The thaumonomicon.
[ ] Cleanup Westernized Chinese language occurrences

Summon @J-swl @Tartaricacid @SihenZhang @Mrkwtkr for proofreading. I did some minor changes to several of translations.
Constructive comments are welcome.

Its cool that you guys are getting going with this so long, but just a word of warning - this is definitely subject to change. There is also a lot of old stuff in the lang file that will be removed at some point.

@Azanor I believe it is easier to constantly make small update than to wait until last minute and make a huge update. That's also why I made this pull request.

“这粉末将指引我前往一个惊艳而危险的世界。”？

直觉告诉我，这粉末将引导我前往一个惊艳又危险的世界。 这样呢?! 3T你感觉如何


Install instructions

Hey, I really want to try this! But how do I install it? What about adding some install instructions to the README? :smile: :+1:

Hi, no problem, I'll add install instructions soon. For now you can use go's built in install tools.
go install github.com/rossdylan/sslcheck

Super :+1: Fairly easy! But for someone not familiar with go, hard to guess :wink:


Generate batch QueryFlags at serialize time

similar to my other cleanups.

It may be a good idea to add warn! if WITH_NAMES_FOR_VALUES would be used.

That would add a dependency of tracing to cassandra-protocol, do you still want to do that?


Copy attributes from original shapefile when exporting

An exported shapefile should have all of the attributes that the imported one did.

In the complete-pointsets branch it gets all of the numeric attributes.


CPU spikes to 100 % and no internet connection in my home router

I use the jaeles as the following command ;-
jaeles scan -c 50 -s '/home/mydirectory/jaeles-signatures/sensitive/.*' -U  example_hosts.txt -L 2 --proxy http://<IP_ADDRESS>:8080
IMMEDIATELY CPU SPIKES TO 100% after this and my pc fan running like a helicopter
jaeles scan -c 50 -s '/home/mydirectory/jaeles-signatures/sensitive/.*' -U  example_hosts.txt -L 2
It also  cause the CPU spikes to 100% later on and keep it not less than 50 % while the scan running . And further there was not any more internet connection in my home router. To have the connection again  I have to end the process of jaeles and restart my router.
First problem I need helps if possible.
For connection problem it will solve if there is an option to add resolver files of flag like -r resolvers.txt as well the rate limit hopefully. jaeles is a very nice tool and always thanks for providing it.

What is the environment and the specs that you are running jaeles in?
For CPU spikes, I would recommend configuring Jaeles in a containerized environment whose run time performance may not be necessary to "throttle". I have Jaeles running in a containerized environment and have not seen any performance issues as stated.


skip deployment steps if build is triggered by cron job

Testing
https://travis-ci.org/aws-robotics/aws-robomaker-sample-application-helloworld/jobs/521791521
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Closing as the bug this PR was fixing is no longer an issue.


add docstring for callback.

Signed-off-by: wxywb<EMAIL_ADDRESS>

Codecov Report

Merging #737 (f70a60d) into main (b26a8ab) will decrease coverage by 0.09%.
The diff coverage is 93.33%.

@@            Coverage Diff             @@
##             main     #737      +/-   ##
==========================================
- Coverage   71.19%   71.09%   -0.10%
==========================================
  Files         166      166
  Lines        7495     7497       +2
  Branches     1218     1219       +1
==========================================
- Hits         5336     5330       -6
- Misses       1799     1805       +6
- Partials      360      362       +2

Impacted Files
Coverage Δ

towhee/trainer/callback.py
84.01% <93.33%> (-0.26%)
:arrow_down:

towhee/engine/task_scheduler.py
68.65% <0.00%> (-7.47%)
:arrow_down:

towhee/engine/graph_context.py
87.71% <0.00%> (-3.51%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b26a8ab...f70a60d. Read the comment docs.

/lgtm
/approve


utourial[tool_crash] SentinelException

Command
flutter run

Steps to Reproduce

...
...
...

Logs
SentinelException

[✓] Flutter (Channel master, v1.18.0-6.0.pre.83, on Mac OS X 10.15.4 19E287, locale en-GB)
    • Flutter version 1.18.0-6.0.pre.83 at /Users/jameslatham/Developer/flutter
    • Framework revision 8e5b575174 (16 hours ago), 2020-04-19 17:20:01 -0400
    • Engine revision 2b75c6d111
    • Dart version 2.9.0 (build 2.9.0-1.0.dev 4814f000de)

[!] Android toolchain - develop for Android devices (Android SDK version 29.0.3)
    • Android SDK at /Users/jameslatham/Library/Android/sdk
    • Platform android-29, build-tools 29.0.3
    • Java binary at: /Applications/Android Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 1.8.0_212-release-1586-b4-5784211)
    ✗ Android license status unknown.
      Try re-installing or updating your Android SDK Manager.
      See https://developer.android.com/studio/#downloads or visit visit https://flutter.dev/docs/get-started/install/macos#android-setup for detailed instructions.

[✓] Xcode - develop for iOS and macOS (Xcode 11.4)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 11.4, Build version 11E146
    • CocoaPods version 1.9.1

[✓] Android Studio (version 3.6)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin version 45.1.1
    • Dart plugin version 192.7761
    • Java version OpenJDK Runtime Environment (build 1.8.0_212-release-1586-b4-5784211)

[!] IntelliJ IDEA Ultimate Edition (version 2020.1)
    • IntelliJ at /Applications/IntelliJ IDEA.app
    ✗ Flutter plugin not installed; this adds Flutter specific functionality.
    ✗ Dart plugin not installed; this adds Dart specific functionality.
    • For information about installing plugins, see
      https://flutter.dev/intellij-setup/#installing-the-plugins

[✓] VS Code (version 1.44.2)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.9.1

[✓] Connected device (1 available)
    • iPhone 11 Pro Max • 61E22532-327C-46E3-85C3-10D4DD44A0D3 • ios • com.apple.CoreSimulator.SimRuntime.iOS-13-4 (simulator)

! Doctor found issues in 2 categories.

Flutter Application Metadata
Type: null
Version: 1.0.3+8
Material: true
Android X: false
Module: false
Plugin: false
Android package: null
iOS bundle identifier: null
Creation channel: null
Creation framework version: null
Plugins
flutter_keyboard_visibility-0.7.0
flutter_secure_storage-3.3.1+1
package_info-0.4.0+16
path_provider-1.6.5
path_provider_macos-0.0.4
sqflite-1.3.0

Hi @iJamesPHP
do you have stack trace log of the exception?
can you reproduce it?

Without additional information, we are unfortunately not sure how to resolve this issue. We are therefore reluctantly going to close this bug for now.
Could everyone who still has this problem please file a new issue with the exact descriptions what happens, minimal reproducible sample code (if related), logs and the output of 'flutter doctor -v'  please.


Search pop over is buggy

1: it doesn't go away.
2: when adding a published claim, the draft before it in the same group was overwritten

Only happens when other stuff is broken. Going to close - can reopen in the future once minimum viable has been met


Drive Stats 'mi driven' total is wrong

Is there an existing issue for this?

[X] I have searched the existing issues

What happened?
I pulled stats for the past two days to review a road trip. I have two Teslas and this involves car #2 in case that matters. On the 'drives' screen it shows 8 drives with a total mileage of about 464 miles:

On the drive stats screen it shows 8 drives, but the 'mi driven' is only 271:

With 'Last 2 days' as the time setting, when I inspect the 'mi driven' query in Grafana I see this:

As you can see, 'Last 2 days' captures 6/12/2022 to 6/13/2022 in the drives dashboard, but 6/11/2022 to 6/12/2022 in the drive stats dashboard. (I am in EDT, UTC -0400, so it shows 6/10/2022 2000 hours and 6/11/2022 2000 hours)
To see if I could get it to count the part of the drive that happened after midnight local (on 6/13/2022) I put in an absolute date range of 2022-06-11 00:00:00 to 2022-06-15 23:59:59 EDT, but that still results in the wrong mileage:

The Query for the absolute date range shows as:
'''
WITH since as (
SELECT date FROM positions
WHERE car_id = '2'
ORDER BY date ASC
LIMIT 1
),
actual AS (
SELECT
date_trunc('day', date)::date AS date,
max(odometer) - min(odometer) AS distance
FROM positions
WHERE car_id = '2'
GROUP BY 1
),
base_line AS (
SELECT date_trunc('day', dd)::date AS date
FROM generate_series((select date from since) , now(), '1 day'::interval) dd
)
SELECT
base_line.date AS "time",
convert_km(COALESCE(distance, 0)::numeric, 'mi') as "distance_mi"
FROM base_line
LEFT JOIN actual ON actual.date = base_line.date
WHERE base_line.date BETWEEN '2022-06-11T04:00:00Z' AND '2022-06-16T03:59:59Z'
ORDER BY 1;
'''
I don't understand the query well enough to figure out where the discrepancy is happening.
Expected Behavior
I expected the Drive Stats -> mi driven panel to show 464 miles.
Steps To Reproduce
I'm not sure. Is it because one of the drives spans two different dates (started before midnight, ended after) in local time, or is something else wrong with the drive stats -> mi driven query?
Relevant log output
No relevant logs.

Screenshots
No response
Additional data
drives.csv
positions.csv
Type of installation
Docker
Version
v1.26.1

Try the following in "mi driven":
SELECT
	start_date AS time,
	convert_km(sum(end_km - start_km)::numeric, '$length_unit') AS distance
FROM drives
WHERE car_id = '1'AND
  $__timeFilter(start_date)
GROUP BY 1;

@adriankumpf @cwanja Do you know, why we have such a complicated query in the Drive Stats? If not, I would push that here...

@adriankumpf @cwanja Do you know, why we have such a complicated query in the Drive Stats? If not, I would push that here...

Unknown to me. If there is a similar method, I would definitely go that route. I feel like I have seen other threads where the count in drive stats, versus trips, versus drives are all different variables.

I think I see issues with the extrapolated monthly mileage and annual mileage on this dashboard also.
It appears that it is using only the days where the car was driven, not the total number of days in the time range, to determine the extrapolated mileage. For example if I select "Last 7 days" I get 607 miles total. The extrapolated monthly mileage is 3,036 and annual is 36.940.
The average daily mileage is: 607/7 ~= 87
In a 30-day month that would be 2,610 miles per month. In a 365-day year that would be: 87 * 365 = 31,755
I think this is because there was one day in the past 7 that the car was not driven. If I divide the mileage by 6 instead of 7 to get the daily mean, then I get numbers closer to what the dashboard shows.
Using only 6 days, 607/6 = 101.166. In a 30 day month that would be 101.166 * 365 = 3,035 miles per month (within 1 mi of the dashboard result). In a 52-week year that would be 101.166 * 365 = 36,925 (within 15 miles of the dashboard result).
This only gets worse if you have fewer days driven. I just tried from 2022-06-12 to 2022-06-18 (which includes future dates), resulting in 470 miles driven in that 7 day span. There are only drives for 2 of the 7 days in that span. (Actually there is a 3rd day, but the mileage is <1 mile so I think it gets filtered out). The extrapolated annual mileage is 85,834, which is what you would get using 470 / 2 * 365, but using the average of all 7 days it should be 470 / 7 = 67.14 | 67.14 * 365 = 24,506.
Shouldn't the extrapolated mileage include zero-mile days? If I drive 100 miles once a week, then my extrapolated annual mileage is 5,200 miles, not 36,500.

@DrMichael  I don't think it is using the median. For the last scenario I described (470 miles, 7-day span, 3 days driven) my median distance per day is 230 miles. (per drive is only 3.02... it definitely isn't using that).
Monthly:
230 / 7 = 32.86 | 32.86 * 30 = 986 miles per month (dashboard shows 4,703)
230 / 3 = 76.66 | 76.66 * 30 = 2,300 miles per month (dashboard shows 4,703)
Annual:
230 / 7 = 32.86 | 32.86 * 365 = 11,994 per year (dashboard shows 57,223)
230 / 3 = 76.66 | 76.66 * 365 = 27,981 per year (dashboard shows 57,233)
The only way to get the result the dashboard is showing is
Total mileage / (number of days with a drive) * (30 or 365)
It should be:
Total mileage / (number if days in the range) * (30 or 365)

Perhaps Drive Stats is more for the long term observation, Trip might be better for short term...


Tidying  init

Depends on #256 , #259 , and #260 ; please merge them first. All deps have been merged. We can go ahead with this one.
What do these changes do?
The __init__ method is very unwieldy. This PR makes it simpler by extracting some logic into properties.
This led to several improvements:

event_handler can be replaced "on-the-fly"
tls_context can be replaced "on-the-fly" and will automagically change require_tls accordingly
User can selectively overload logic in the properties; no longer have to copy > 100 lines of init code just to override one behavior 😉

Are there changes in behavior for the user?
None.
Users have to be aware though that some properties are more 'noisy' in logging.
Related issue number
Inspired partially by #253
Checklist

[x] I think the code is well written
[x] Unit tests for the changes exist
[x] tox testenvs have been executed in the following environments:

[x] Windows 10 (via PyCharm tox runner)
[x] Windows 10 (via PSCore 7.1.2)
[x] Windows 10 (via Cygwin)
[x] Ubuntu 18.04 on WSL 1.0
[x] FreeBSD 12.2 on VBox
[x] OpenSUSE Leap 15.2 on VBox

[x] Documentation reflects the changes
[x] Add a news fragment into the NEWS.rst file

Now that I think of it ... this should be merged ONLY after #260 has been merged.
That way, I can test for the behavior of the library when tls_context is being replaced totally.

Nice. I'm digging at least the idea :)

As soon as test passes on my testing systems, I'm undrafting this and let this marinate.
I might not be able to do code changes for several days ahead, due to me transitioning from "being employed" to "being a free agent" and needing to acquire a new laptop. I reckon up to a week of radio silence due to the Easter holidays. Hopefully not as long as that, depending on how soon the store can fulfill my order.

Nice. I'm digging at least the idea :)

A side effect is that SMTP instatiation is now faster 😉

@pepoluan I'm going to plan on taking a closer look at this one this week, too. Looks like there are a couple of conflicts, if you're able to tackle those.

@pepoluan I'm going to plan on taking a closer look at this one this week, too. Looks like there are a couple of conflicts, if you're able to tackle those.

Fixed the conflicts, rebase, and let's see...

 LOL

Heh seriously at this point I'll just close this PR, but grab the ideas and make a new PR based on the latest master.


Need To fix Project Edit Page

Description of bug
When we open the Projects in Portal and try to edit that using the edit button in the right top corner so it throws an error that
Attempt to read property "currency" on null means currency is null so which is to fixed .
Check the Screenshot - It Is  A Project

When we click on the edit button in file shown above it throws the following error

So create a Seeder which seeds the currency in the required table so that it do not throws an error.
Points to remember

Make sure that it do not effect other files .
Currency can be used in more that one table .
Check the flow of the code to solve the error .
Create the required files .

Created a issue related to project edit page bug


Investigate Ionic Appflow

From tenfour-archive created by mackers: ushahidi/tenfour-archive#1554
@dalezak Spend some time investigating Ionic Appflow for deployment of our Ionic app. We would like to have cloud builds of the native apps with possibly moving away from codeship at some point.
Have discussed with @tuxpiper already - he can advise and help with setting up an AppFlow trial.
https://ionicframework.com/pricing#appflow

@mackers I've sent PR to linked our repo to AppFlow project. Next step would be upgrading our Ionic plan so we can test doing native iOS and Android builds. All we'll need is the company credit card to make that happen. 💳


feat: sort interface and class members

cc @IgorMinar

lgtm

@jasonchoimtt I published 0.1.3, can you please update angular/angular