When killed. Etcdv3 node can require manual intervention to bring back

This was discovered when running a benchmark on a 3 node etcd cluster. The issue was only produced on a single node.
It appears that the BoltDB backend lags behind the snapshots written to disk, as a hard kill to an etcd member can result in etcdmain: database file (/data/etcd/member/snap/db index 7622690) does not match with snapshot (index 8081536). upon starting back up. After getting this error and the node starting to flap, you are able to fix the issue by deleting the latest snapshot and WAL file. It appears that after the snapshot is written to disk before the db file is written to, and that upon booting etcd doesn't have an automated method of recovery.
Log output
Mar 28 21:44:24 aws-us-east-1-memory etcd697: etcd.19  | 2017-03-28 21:44:24.287615 I | etcdserver: applying snapshot at index 7607808...
Mar 28 21:44:24 aws-us-east-1-memory etcd697: etcd.19  | 2017-03-28 21:44:24.291440 I | etcdserver: saved snapshot at index 7599475
Mar 28 21:44:24 aws-us-east-1-memory etcd697: etcd.19  | 2017-03-28 21:44:24.291867 I | etcdserver: compacted raft log at 7594475
Mar 28 21:44:24 aws-us-east-1-memory etcd697: etcd.19  | 2017-03-28 21:44:24.295647 I | etcdserver: saved snapshot at index 7602829
Mar 28 21:44:24 aws-us-east-1-memory etcd697: etcd.19  | 2017-03-28 21:44:24.295980 I | etcdserver: compacted raft log at 7597829
Mar 28 21:44:24 aws-us-east-1-memory etcd697: etcd.19  | 2017-03-28 21:44:24.298568 I | etcdserver: saved snapshot at index 7600706
Mar 28 21:44:24 aws-us-east-1-memory etcd697: etcd.19  | 2017-03-28 21:44:24.300390 I | etcdserver: saved snapshot at index 7607808
Mar 28 21:44:24 aws-us-east-1-memory etcd697: etcd.19  | 2017-03-28 21:44:24.300554 I | etcdserver: compacted raft log at 7602808
Mar 28 21:44:24 aws-us-east-1-memory etcd697: etcd.19  | 2017-03-28 21:44:24.302218 I | etcdserver: raft applied incoming snapshot at index 7622690
Mar 28 21:44:29 aws-us-east-1-memory etcd697: etcd.19  | 2017-03-28 21:44:29.642527 I | etcdserver: recovering lessor...
Mar 28 21:44:29 aws-us-east-1-memory etcd697: etcd.19  | 2017-03-28 21:44:29.648641 I | etcdserver: finished recovering lessor
Mar 28 21:44:29 aws-us-east-1-memory etcd697: etcd.19  | 2017-03-28 21:44:29.648663 I | etcdserver: restoring mvcc store...
Mar 28 21:44:50 aws-us-east-1-memory etcd697: etcd.19  | 2017-03-28 21:44:50.554203 I | pkg/fileutil: purged file /data/etcd/member/snap/000000000000007d-000000000073ddc2.snap successfully
Mar 28 21:44:50 aws-us-east-1-memory etcd697: etcd.19  | 2017-03-28 21:44:50.554290 I | pkg/fileutil: purged file /data/etcd/member/snap/000000000000007d-000000000073e427.snap successfully
Mar 28 21:44:50 aws-us-east-1-memory etcd697: etcd.19  | 2017-03-28 21:44:50.554345 I | pkg/fileutil: purged file /data/etcd/member/snap/000000000000007d-000000000073e822.snap successfully
Mar 28 21:44:50 aws-us-east-1-memory etcd697: etcd.19  | 2017-03-28 21:44:50.554397 I | pkg/fileutil: purged file /data/etcd/member/snap/000000000000007d-000000000073ed71.snap successfully
Mar 28 21:44:50 aws-us-east-1-memory etcd697: etcd.19  | 2017-03-28 21:44:50.554480 I | pkg/fileutil: purged file /data/etcd/member/snap/000000000000007d-000000000073f16f.snap successfully
Mar 28 21:44:53 aws-us-east-1-memory etcd697: etcd.19  | 2017-03-28 21:44:53.814295 I | wal: segmented wal file /data/etcd/member/wal/0000000000000017-0000000000792046.wal is created
Mar 28 21:45:07 aws-us-east-1-memory etcd697: etcd.19  | 2017-03-28 21:45:07.766653 I | rafthttp: receiving database snapshot [index:8081536, from f69252dd496581b1] ...
Mar 28 21:45:13 aws-us-east-1-memory etcd697: etcd.19  | 2017-03-28 21:45:13.915615 I | snap: saved database snapshot to disk [total bytes:<PHONE_NUMBER>]
Mar 28 21:45:13 aws-us-east-1-memory etcd697: etcd.19  | 2017-03-28 21:45:13.915661 I | rafthttp: received and saved database snapshot [index: 8081536, from: f69252dd496581b1] successfully
Mar 28 21:45:13 aws-us-east-1-memory etcd697: etcd.19  | 2017-03-28 21:45:13.915874 I | raft: 11561d69df4299ed [commit: 8006839, lastindex: 8006839, lastterm: 125] starts to restore snapshot [index: 8081536, term: 125]
Mar 28 21:45:13 aws-us-east-1-memory etcd697: etcd.19  | 2017-03-28 21:45:13.915911 I | raft: log [committed=8006839, applied=8006839, unstable.offset=8006840, len(unstable.Entries)=0] starts to restore snapshot [index: 8081536, term: 125]
Mar 28 21:45:13 aws-us-east-1-memory etcd697: etcd.19  | 2017-03-28 21:45:13.915947 I | raft: 11561d69df4299ed restored progress of 11561d69df4299ed [next = 8081537, match = 8081536, state = ProgressStateProbe, waiting = false, pendingSnapshot = 0]
Mar 28 21:45:13 aws-us-east-1-memory etcd697: etcd.19  | 2017-03-28 21:45:13.915975 I | raft: 11561d69df4299ed restored progress of 1fd3856b78fe333e [next = 8081537, match = 0, state = ProgressStateProbe, waiting = false, pendingSnapshot = 0]
Mar 28 21:45:13 aws-us-east-1-memory etcd697: etcd.19  | 2017-03-28 21:45:13.915992 I | raft: 11561d69df4299ed restored progress of f69252dd496581b1 [next = 8081537, match = 0, state = ProgressStateProbe, waiting = false, pendingSnapshot = 0]
Mar 28 21:45:13 aws-us-east-1-memory etcd697: etcd.19  | 2017-03-28 21:45:13.916004 I | raft: 11561d69df4299ed [commit: 8081536] restored snapshot [index: 8081536, term: 125]
Mar 28 21:45:13 aws-us-east-1-memory etcd697: etcd.19  | 2017-03-28 21:45:13.920984 I | etcdserver: raft applied incoming snapshot at index 8081536
Mar 28 21:45:16 aws-us-east-1-memory etcd697: etcd.19  | /app/etcd_runner.sh: line 6:    24 Killed                  etcd --initial-cluster-state new --initial-cluster-token $TOKEN --name etcd697.aws-us-east-1-memory.19.dblayer.com --data-dir /data/etcd --heartbeat-interval 500 --election-timeout 5000 --snapshot-count 1000 --listen-peer-urls http://<IP_ADDRESS>:2380 --advertise-client-urls 'http://<IP_ADDRESS>:2379' --initial-advertise-peer-urls http://<IP_ADDRESS>:2380 --listen-client-urls 'http://<IP_ADDRESS>:2379,http://<IP_ADDRESS>:2379' --initial-cluster 'etcd697.aws-us-east-1-memory.19.dblayer.com=http://<IP_ADDRESS>:2380,etcd675.aws-us-east-1-memory.18.dblayer.com=http://<IP_ADDRESS>:2380,etcd616.aws-us-east-1-memory.20.dblayer.com=http://<IP_ADDRESS>:2380'
Mar 28 21:45:16 aws-us-east-1-memory etcd697: start.13 | etcd.19 process exited.
Mar 28 21:45:16 aws-us-east-1-memory etcd697: start.13 | Stopping Cron Scheduler.
Mar 28 21:45:23 aws-us-east-1-memory etcd697: can't setuid.  syscall.Setuid(1500): operation not supported
Mar 28 21:45:23 aws-us-east-1-memory etcd697: running as uid 0
Mar 28 21:45:23 aws-us-east-1-memory etcd697: start.13 | Adding Procfile entry [Type: "DAEMON_PROCESS", Name: "etcd", Schedule: "", Command: "/app/etcd_runner.sh"].
Mar 28 21:45:23 aws-us-east-1-memory etcd697: start.13 | Starting etcd process.
Mar 28 21:45:23 aws-us-east-1-memory etcd697: start.13 | Starting Cron Scheduler.
Mar 28 21:45:23 aws-us-east-1-memory etcd697: etcd.19  | 2017-03-28 21:45:23.338478 I | etcdmain: etcd Version: 3.1.4
Mar 28 21:45:23 aws-us-east-1-memory etcd697: etcd.19  | 2017-03-28 21:45:23.338629 I | etcdmain: Git SHA: 41e52eb
Mar 28 21:45:23 aws-us-east-1-memory etcd697: etcd.19  | 2017-03-28 21:45:23.338638 I | etcdmain: Go Version: go1.7.5
Mar 28 21:45:23 aws-us-east-1-memory etcd697: etcd.19  | 2017-03-28 21:45:23.338643 I | etcdmain: Go OS/Arch: linux/amd64
Mar 28 21:45:23 aws-us-east-1-memory etcd697: etcd.19  | 2017-03-28 21:45:23.338649 I | etcdmain: setting maximum number of CPUs to 16, total number of available CPUs is 16
Mar 28 21:45:23 aws-us-east-1-memory etcd697: etcd.19  | 2017-03-28 21:45:23.338698 N | etcdmain: the server is already initialized as member before, starting as etcd member...
Mar 28 21:45:23 aws-us-east-1-memory etcd697: etcd.19  | 2017-03-28 21:45:23.338787 I | embed: listening for peers on http://<IP_ADDRESS>:2380
Mar 28 21:45:23 aws-us-east-1-memory etcd697: etcd.19  | 2017-03-28 21:45:23.338862 I | embed: listening for client requests on <IP_ADDRESS>:2379
Mar 28 21:45:23 aws-us-east-1-memory etcd697: etcd.19  | 2017-03-28 21:45:23.338907 I | embed: listening for client requests on <IP_ADDRESS>:2379
Mar 28 21:45:24 aws-us-east-1-memory etcd697: etcd.19  | 2017-03-28 21:45:24.339415 W | etcdserver: another etcd process is running with the same data dir and holding the file lock.
Mar 28 21:45:24 aws-us-east-1-memory etcd697: etcd.19  | 2017-03-28 21:45:24.339458 W | etcdserver: waiting for it to exit before starting...
Mar 28 21:45:27 aws-us-east-1-memory etcd697: etcd.19  | 2017-03-28 21:45:27.851222 W | snap: skipped unexpected non snapshot file<PHONE_NUMBER>00007d-000000000060b62f.snap.broken
Mar 28 21:45:27 aws-us-east-1-memory etcd697: etcd.19  | 2017-03-28 21:45:27.851247 W | snap: skipped unexpected non snapshot file<PHONE_NUMBER>7b5080.snap.db
Mar 28 21:45:27 aws-us-east-1-memory etcd697: etcd.19  | 2017-03-28 21:45:27.852483 I | etcdserver: recovered store from snapshot at index 8081536
Mar 28 21:45:27 aws-us-east-1-memory etcd697: etcd.19  | 2017-03-28 21:45:27.852504 I | etcdserver: name = etcd697.aws-us-east-1-memory.19.dblayer.com
Mar 28 21:45:27 aws-us-east-1-memory etcd697: etcd.19  | 2017-03-28 21:45:27.852510 I | etcdserver: data dir = /data/etcd
Mar 28 21:45:27 aws-us-east-1-memory etcd697: etcd.19  | 2017-03-28 21:45:27.852516 I | etcdserver: member dir = /data/etcd/member
Mar 28 21:45:27 aws-us-east-1-memory etcd697: etcd.19  | 2017-03-28 21:45:27.852521 I | etcdserver: heartbeat = 500ms
Mar 28 21:45:27 aws-us-east-1-memory etcd697: etcd.19  | 2017-03-28 21:45:27.852525 I | etcdserver: election = 5000ms
Mar 28 21:45:27 aws-us-east-1-memory etcd697: etcd.19  | 2017-03-28 21:45:27.852530 I | etcdserver: snapshot count = 1000
Mar 28 21:45:27 aws-us-east-1-memory etcd697: etcd.19  | 2017-03-28 21:45:27.852542 I | etcdserver: advertise client URLs = http://<IP_ADDRESS>:2379
Mar 28 21:45:28 aws-us-east-1-memory etcd697: etcd.19  | 2017-03-28 21:45:28.127894 I | etcdserver: restarting member 11561d69df4299ed in cluster 861e1d72f0835e7a at commit index 8157572
Mar 28 21:45:28 aws-us-east-1-memory etcd697: etcd.19  | 2017-03-28 21:45:28.131329 I | raft: 11561d69df4299ed became follower at term 125
Mar 28 21:45:28 aws-us-east-1-memory etcd697: etcd.19  | 2017-03-28 21:45:28.131363 I | raft: newRaft 11561d69df4299ed [peers: [11561d69df4299ed,1fd3856b78fe333e,f69252dd496581b1], term: 125, commit: 8157572, applied: 8081536, lastindex: 8157572, lastterm: 125]
Mar 28 21:45:28 aws-us-east-1-memory etcd697: etcd.19  | 2017-03-28 21:45:28.131515 I | etcdserver/api: enabled capabilities for version 3.1
Mar 28 21:45:28 aws-us-east-1-memory etcd697: etcd.19  | 2017-03-28 21:45:28.131539 I | etcdserver/membership: added member 11561d69df4299ed [http://<IP_ADDRESS>:2380] to cluster 861e1d72f0835e7a from store
Mar 28 21:45:28 aws-us-east-1-memory etcd697: etcd.19  | 2017-03-28 21:45:28.131547 I | etcdserver/membership: added member 1fd3856b78fe333e [http://<IP_ADDRESS>:2380] to cluster 861e1d72f0835e7a from store
Mar 28 21:45:28 aws-us-east-1-memory etcd697: etcd.19  | 2017-03-28 21:45:28.131554 I | etcdserver/membership: added member f69252dd496581b1 [http://<IP_ADDRESS>:2380] to cluster 861e1d72f0835e7a from store
Mar 28 21:45:28 aws-us-east-1-memory etcd697: etcd.19  | 2017-03-28 21:45:28.131561 I | etcdserver/membership: set the cluster version to 3.1 from store
Mar 28 21:45:36 aws-us-east-1-memory etcd697: etcd.19  | 2017-03-28 21:45:36.527334 C | etcdmain: database file (/data/etcd/member/snap/db index 7622690) does not match with snapshot (index 8081536).

To get this node to come back, I needed to manually delete the latest snapshot and WAL file, and have it recover the rest from the other nodes in the cluster

Possible candidate for failpoint testing. Start etcd so it snapshots frequently and inject a sleep before the db sync, kill it, then check if the node restarts cleanly.

@doodles526 could you provide me an exact steps to reproduce the same issue?

Steps to reproduce the above issue:
Have a snapshot snapshot.db ready from some previous cluster.
Restore the data-dir from the snapshot for the first member of our cluster:
$ ETCDCTL_API=3 bin/etcdctl snapshot restore snapshot.db --name infra1 --initial-cluster infra1=http://<IP_ADDRESS>:2380 --initial-cluster-token etcd-cluster-1   --initial-advertise-peer-urls http://<IP_ADDRESS>:2380

Create the 1st member of a new cluster using the restored data-dir infra1.etcd:
$ bin/etcd --data-dir=infra1.etcd --name infra1 --listen-client-urls http://<IP_ADDRESS>:2379 --advertise-client-urls http://<IP_ADDRESS>:2379 --listen-peer-urls http://<IP_ADDRESS>:2380 --initial-advertise-peer-urls http://<IP_ADDRESS>:2380 --initial-cluster 'infra1=http://<IP_ADDRESS>:2380' --initial-cluster-state new

Add a 2nd member to cluster:
$ ETCDCTL_API=3 bin/etcdctl member add infra2 --peer-urls=http://<IP_ADDRESS>:22380

Start etcd server for 2nd member:
$ bin/etcd --data-dir=infra2.etcd --name infra2 --listen-client-urls http://<IP_ADDRESS>:22379 --advertise-client-urls http://<IP_ADDRESS>:22379 --listen-peer-urls http://<IP_ADDRESS>:22380 --initial-advertise-peer-urls http://<IP_ADDRESS>:22380 --initial-cluster 'infra1=http://<IP_ADDRESS>:2380,infra2=http://<IP_ADDRESS>:22380' --initial-cluster-state existing

Kill and restart the etcd server for the 2nd member:
^C
$ bin/etcd --data-dir=infra2.etcd --name infra2 --listen-client-urls http://<IP_ADDRESS>:22379 --advertise-client-urls http://<IP_ADDRESS>:22379 --listen-peer-urls http://<IP_ADDRESS>:22380 --initial-advertise-peer-urls http://<IP_ADDRESS>:22380 --initial-cluster 'infra1=http://<IP_ADDRESS>:2380,infra2=http://<IP_ADDRESS>:22380' --initial-cluster-state existing

The 2nd member will exit with the following error:
2017-04-27 16:10:25.544472 C | etcdmain: database file (bin/infra2.etcd/member/snap/db index 4) does not match with snapshot (index 5).

The snapshot that I used was just from writing 1 key value pair to a single member etcd cluster:
$ ETCDCTL_API=3 etcdctl --endpoints=http://<IP_ADDRESS>:2379 put foo1 bar1
OK
$ ETCDCTL_API=3 etcdctl --endpoints http://<IP_ADDRESS>:2379 snapshot save snapshot.db

@hasbro17 that looks like a separate issue. There's no etcdctl snapshot or membership reconfiguration involved in this one. etcd isn't syncing with raft correctly for this one. Open another issue?

@heyitsanthony done #7834


Deployments should be served over http/2 instead of http/1.1

When running chrome audit on my site, the best I can get is 93 for "Best Practices" since the site is being served over http/1.1 instead of http/2.

Is there some setting I'm missing? These deployments are automatic via Github.
My now.json config:
{
  "version": 2,
  "name": "xxx",
  "alias": "xxx.now.sh",
  "builds": [{ "src": "package.json", "use": "@now/next" }],

  "routes": [
    {
      "src": "^/service-worker.js$",
      "dest": "/_next/static/service-worker.js",
      "headers": {
        "cache-control": "public, max-age=43200, immutable",
        "Service-Worker-Allowed": "/"
      }
    }
  ]
}

Note: I am using nextjs for this site. Is http/2 not supported out of the box for nextjs?
This is the only example I see on that repo, but I'd like to avoid using a custom server.
https://github.com/zeit/next.js/tree/canary/examples/with-http2

Hi Nate,
All requests served from Now use HTTP/2.
Can you share the deployment URL that you are testing? Thanks!

The plot thickens. Here's the example I'm using for the next-offline: https://github.com/zeit/next.js/tree/canary/examples/with-next-offline
Here's my repo:
https://github.com/NateRadebaugh/nateradebaugh.github.com/blob/master/next.config.js
https://naterad.com

Using chrome dev tools, you can see that the h1 responses are from your service worker.
The h2 responses are from Now.

Cool thanks, I'll close this and investigate with the next-offline plugin to see how to properly configure the service worker for http2.


Subs wrapper

Rename all the generator instance preprocessors to end with _wrapper.
Add a make_decorator utility function that converts a generator instance wrapper to a generator function decorator.
Ship a *_decorator variant of each *_wrapper. These are not yet documented, but they will be once I have the time to explain them carefully.

[Current coverage][cc-pull] is 81.66%

Merging [#471][cc-pull] into [master][cc-base-branch] will increase coverage by 0.09%

@@             master       #471   diff @@
==========================================
  Files            18         18
  Lines          3741       3774    +33
  Methods           0          0
  Messages          0          0
  Branches          0          0
==========================================
+ Hits           3051       3082    +31
- Misses          690        692     +2
  Partials          0          0

Powered by Codecov. Last updated by [65448c0...395bea0][cc-compare]
[cc-base-branch]: https://codecov.io/gh/NSLS-II/bluesky/branch/master?src=pr
[cc-compare]: https://codecov.io/gh/NSLS-II/bluesky/compare/65448c035da9610ce9c89dfa3c9ebf746038a151...395bea0f929247c667474ecfea63185714601d60
[cc-pull]: https://codecov.io/gh/NSLS-II/bluesky/pull/471?src=pr

👍

:+1: Other than my one comment.


Fixed shellcheck issue SC2015

SC2015: Note that A && B || C is not if-then-else. C may run when A is true.
Replaced [ $1 -gt 1 ] && shift $1 || shift with shift $1 since
shift 1 is identical to shift.

Merged, Thank you :D


Support all HTTP Methods

Feature Request
OpenApi support for all http methods
RFC 2616 updated by RFC 7231 (errata list for 7231)

[X] OPTIONS
[X] GET
[X] HEAD
[X] POST
[X] PUT
[X] DELETE
[X] TRACE
[ ] CONNECT

RFC 2518 updated by RFC 4918 and RFC 5689 for MKCOL

[ ] PROPFIND
[ ] PROPPATCH
[ ] MKCOL
[ ] COPY
[ ] MOVE
[ ] LOCK
[ ] UNLOCK

RFC 3253 (errata list for 3253)

[ ] VERSION-CONTROL
[ ] REPORT
[ ] CHECKOUT
[ ] CHECKIN
[ ] UNCHECKOUT
[ ] MKWORKSPACE
[ ] UPDATE
[ ] LABEL
[ ] MERGE
[ ] BASELINE-CONTROL
[ ] MKACTIVITY

RFC 3648

[ ] ORDERPATCH

RFC 3744 (errata list for 3744)

[ ] ACL

RFC 5323 draft

[ ] SEARCH

draft snell link method 01

[ ] LINK
[ ] UNLINK

Related Issues
#658
#1306
#480
https://github.com/swagger-api/swagger-core/issues/1760
Iana link
Another link with methods and their RFC : iana
This post will be re-edited soon to add a small description for all methods
Regards.

HEAD has been supported since at least Swagger v1.2

@MikeRalphson updated, thanks.

Given the mission statement of the OAS as "standardizing on how REST APIs are described" could you expand on how some of these HTTP methods are being used within REST APIs (e.g. the WebDav methods are often seen as tied only to that protocol by their definitions)?

Yes I think during next week I will edit this post to describe the methods, if they have a body, header, query, etc... how they should be used and some examples. I think it's what you want, if I understood your answer correctly ?
There is an inherent problem with REST because it wants to be very close to CRUD (Create, Read, Update, Delete) and even SCRUD (S for Search), but a REST API does so much more, like letting users subscribing and unsubscribing to a subject, users also want to connect to the API to have their data saved and secured, they will want to copy something, to move some datas, to lock their account, etc... that's what all these additional methods were made for.
For example to lock your account you could do a post to /accounts/lock with your token in header and in the post body add a parameter 'lock = true' but REST could handle it with just a LOCK request to /accounts and the token as a cookie header (or some encrypted data from localstorage as a token) which makes code clearer and really use the HTTP method verbose to add some logic in requests.
lock/unlock are described here as 'lock model'.
GET will have query params, with optional body, whereas HEAD will have no body and POST has an obligation to have a body, then responses can have a body or not depending on the method. So I have some research to do on this to give the right definitions, I will update very soon.
From wikipedia : http methods

Also there should be some methods to ignore, for example CONNECT which was made to create an ssl tunnel has, I think, no use anymore now with SSL/TLS certificates. But some could find a use for it, if they want to keep an http server and then control https connections opening.
CONNECT METHOD description on mozilla

lock/unlock are described here as 'lock model'.

But my point is that LOCK and UNLOCK as defined there are for use within the WebDAV protocol. They have no defined meaning (on their own) outside that protocol. They may be tied to specific XML request / response formats (forgive me, it has been a long time since I read the WebDAV RFCs).
A picture may be worth a thousand words:

I'd be open to considering allowing HTTP method be an unconstrained string.  I wouldn't want to re-enumerate all the methods in the IANA registry in the OpenAPI spec.  We should consider what benefit tooling derives from it being an enumerated type.
p.s. Wow, that Wikipedia table is misleading.  Describing a GET body as optional is just terrible.

Describing a GET body as optional is just terrible.

Far too many people don't seem to understand that "has no defined semantics" is standards-ese for "FFS don't do this!" and not "sure, do whatever!"

What is the expected behavior if my Path Item Object has an x-lock field associated with an Operation Object?  Would that add Lock to the set of supported HTTP operations? If not, what else do we need to introduce these operations as Draft Features?

I'm not against that in principle, like Darrel says, maybe open it up with an extension which allows any HTTP method?

Hello there, does anyone know if any effort has been made to create an extension which broadens the HTTP methods that can be described in Open API 3? Would like to take advantage of the SEARCH verb in my API but it looks like I am limited in my documentation at this time.

Given the ongoing work on HTTP & HTTPAPI I really suggest that we should:

delegate methods support to HTTP
add interoperability considerations suggesting to limit to the methods referenced in  7231 or I-D httpbis-latest
include the fact that tooling might not supporting methods outside 7231

cc: @MikeRalphson @darrelmiller

I'd support changing it to "any method defined in RFC 7231 and RFC 5323" but I'm not sure what value adding all those other ones provides?
I think its a little telling that SEARCH was what necro bumped this otherwise inactive topic. Probably just SEARCH is fine.

@philsturgeon I just won't ossify OAS with method decisions which are outside the OAS scope.
If we want to provide stuff for implementers, we can say MUST 7231, MAY other methods.
Stuffing HTTP elements into OAS can become problematic because HTTP can change (Eg. see https://github.com/httpwg/http-core/pull/653/files).

@ioggstream I don't understand any individual parts of that reply let alone all of it together. I was saying "+ SEARCH" would be a good enough change to satisfy this issue because its the only one people seem all that fussed about.

I found this issue because I am writing a document that describes a "PURGE" endpoint, which is a non-standard HTTP method but none-the-less is very much in use in my application.  Supporting any method matching the pattern "^[a-z]+$" would be very simple and straightforward; much more so than attempting to define a mechanism by which additional HTTP methods could be defined.

I found this issue because I'm writing docs to describe a MERGE endpoint (which is OData 1.0 and OData 2.0, but not HTTP spec), and really should have been written as PATCH but someone made an unfortunate decision years ago and that ship has sailed.  While I understand OpenAPI's mission is documenting true REST, adding this flexibility will help a lot in places where someone made poor choices and/or didn't understand REST very well, and the rest of us have to live with it.

My preference would be to have it allow any string, for the same reason @darrelmiller mentioned above. Really the decision of whether any given method is or isn't valid is the business of the service endpoint implementation; I don't see any value gained by having a general-purpose API tool make its own judgments on that.
However, if it is necessary to pick specific methods, I'll add one thing specifically about the REPORT verb. While it unfortunately is sometimes not allowed by HTTP clients, REPORT fills a specific need in HTTP APIs by being the method that 1. has a request body, 2. is idempotent, 3. is cacheable. GET can't (or shouldn't) have a body; POST isn't idempotent. Basically REPORT is what you would use if you want GET-like semantics but can't fit all the parameters into the URL.

Please add the QUERY method: https://datatracker.ietf.org/doc/draft-ietf-httpbis-safe-method-w-body/

Please add the RAIN method.   All joking aside, in section 16.1 of https://datatracker.ietf.org/doc/rfc9110/ the framers intended to allow for extensibility of Methods.  If they didn't, I doubt they would have the registry https://www.iana.org/assignments/http-methods/http-methods.xhtml.   I have yet to find in the REST Document, https://roy.gbiv.com/pubs/dissertation/top.htm, that suggests the methods described in RF7231 and RFC5789 are only for REST.
Example:
I want the target resource, Oregon, to RAIN.  I would issue:
RAIN /Oregon
This would cause the resource Oregon to change its current State to RAIN on the server.
What about changing the state of the target resource, Chicago, to WINDY, I would issue:
WINDY /Chicago
It shouldn't matter to the OpenAPI specification what choices the server/implementer makes to support.  I believe that is the flexibility of OpenAPI.  The alternative is using an existing verb to do this which IMHO seems like an anti-pattern.
In anti-pattern case, the above examples might be:
POST /Oregon/actions/RAIN
POST /Chicago/actions/WINDY

Where are we at with this? As I (and many others) suggested above, we could remove the individual property declarations for get, post etc in the schema and replace them with:
patternProperties:
  ^[a-z]+$:
    $ref: '#/$defs/operation'

This would be a fast and easy win for 3.1.1.

@darrelmiller maybe pop this one on the agenda for the next meeting? I'll come pester you folks on the call to chat about it. Seems like an easy win to just let operations be anything.
Tooling can stick to working with what it knows and warning against invalid.
Documentation can show what it knows and ignore what it doesn't, or just generically show everything.
Linters can validate against whatever current RFC they like and complain if you're using NONSENSE or whatever.

I feel like there has to be something more than just "everything could be a path". Without some sort of identifier, it impossible no tell the difference between:

A method
A not yet supported property

In reality we should just have all the methods under a property called "methods" just like all the paths are under "paths". Please don't allow both patterned properties that are * and also defined properties. Everyone is going to have a bad time.
It would be much better to introduce a new level called "methods" with the pattern properties of any word character of any length. And then tools could merge the two lists together. Easy. Take the known list at the top and merge it with the dynamic methods list.
Then we just mark the top level methods as deprecated in favor of the methods as a sub property of "methods"

@wparad so it would be MethodObject with a defined set of Methods (which the OperationObject https://spec.openapis.org/oas/latest.html#path-item-object contains now ie. get, post, put, patch, etc.)  along with a user defined dynamic list of additional Methods.
So glad to see this moving forward!  Postman has had this since 2018
https://blog.postman.com/custom-http-methods-more-flexibility-and-autonomy/

@ScotMeyer

An anti-pattern case, the above examples might be:
POST /Oregon/actions/RAIN
POST /Chicago/actions/WINDY

My reading of the REST Uniform Interface constraint is that inventing new service specific methods would actually be the antipattern.

The central feature that distinguishes the REST architectural style from other network-based styles is its emphasis on > a uniform interface between components ([Figure 5-6]
(https://roy.gbiv.com/pubs/dissertation/rest_arch_style.htm#fig_5_6)). By applying the software engineering principle
of generality to the component interface, the overall system architecture is simplified and the visibility of interactions
is improved. Implementations are decoupled from the services they provide, which encourages independent
evolvability.

I do agree that the existence of the HTTP method registry indicates that adding new generic methods should not require a specification update and therefore constraining OpenAPI to a limited set of methods is probably not ideal.

Thanks to @philsturgeon  for raising this up to discuss in the TDC meeting https://github.com/OAI/OpenAPI-Specification/issues/3295
We did consider all the proposals discussed in this thread.  In general there appeared to be consensus that supporting just the current set of methods is unnecessarily limiting. The current design of pathitem using fixed fields to identify methods is problematic in that new HTTP methods might conflict with current or future properties of pathitem.  It is worth noting that the future moonwalk proposal doesn't suffer from this same design constraint.
Here are the options we discussed:

Add some additional subset of the methods defined in the HTTP registry as fixed fields to the pathitem object.
Add a property such as otherMethods to path item to enable adding additional methods to a description.
Recommend the use of an extension such as x-method-{method} to describe other HTTP methods.

The challenge with (1) is how do we pick the methods we want to include? Including all of the methods in the HTTP registry seems like overkill and won't help us as soon as new methods like QUERY are added.
Adding an official otherMethods solves the problem of naming conflicts but it doesn't really help driving support in tooling.  Tooling vendors need to implement explicit support for merging these collections.  Considering this solution is not something that would move forward into moonwalk we are concerned that we will see very little tooling support for it.
Encouraging the community to leverage extensions as a solution to "extending" OpenAPI seems like the right tool for the job.  By following a pattern that uses the x-method- prefix we give an opportunity for tooling to add support for this extension and we can measure the interest in this capability based on presence of the extension with OpenAPIs documents and support in tooling.  This will help guide us when it comes to moonwalk and whether we allow the method property to be any string or a known subset of strings.

@darrelmiller

Adding an official otherMethods solves the problem of naming conflicts but it doesn't really help driving support in tooling. Tooling vendors need to implement explicit support for merging these collections. Considering this solution is not something that would move forward into moonwalk we are concerned that we will see very little tooling support for it.

We are a tool vendor specifically Open API Explorer and we Absolutely do number 2. Adding an Object.assign(methods, spec.paths.path.otherMethods) is a one line code change. It can't be any easier than that for us. Any other expectation is a huge burden.
x-vendor-... are terrible ideas, so we cannot support #3. This is the fastest way to ensure that ever spec has non-consistent extensions and already causes a huge mess. When our users covert from one library to another the fact they already have to convert half of their x- is quite the overhead.
Please let's add methods to the pathObject and then in spec version 4.x remove the legacy ones that hard coded into the path object itself.

Just leaving my two cents here:

Please let's add methods to the pathObject and then in spec version 4.x remove the legacy ones that hard coded into the path object itself.

I'm with @wparad on this one...
This whole issue could've been avoided if all HTTP methods are described in Path Item Object->[some-property] instead of directly under the Path Item Object itself.
Introducing new property such as methods to Path Item Object as future container for all possible HTTP methods (e.g, methods->get, methods->post, etc) and start deprecating get, put, post, delete, options, head, patch, trace in favor of methods property, IMO, is the better solution than using vendor prefix or extension prop such-as x-method-{method}...

Moonwalk already has the notion of a method property in each request object. Therefore this issue will be  resolved in v4, if we continue our current plans.
I would like to understand more about why the x- in front of "othermethods" is such an problematic solution.
Even if it were included in some future 3.x version there would only be partial support for it in the ecosystem.

It's simple really, a methods is forwards compatible with a future state, hacking in recommended support for x- is not. If we want code generators and other tools to work out the box, already going with a solution that matches the future is the right path. Which means the question becomes, what so bad about the solution that aligns with the future? That is why would we not just add the methods object in 3.x?
Looking at it another way, all we need to do is think about what this looks like in reality to see why it is a problem. Simple story:

customer: We use custom method OtherMethod but your tool doesn't support it.
us: Oh that's because it isn't supported in 3.x
customer: According to the specs you should accept X-OtherMethod when can you support that
us: Sigh, okay....
.... Later....
us: Okay now we are on version v4 where methods is supported, let's add support for the methods!!! :tada:
also us: Wait, what should we do about the 100 different X-OtherMethods that we support now, there is all this tech debt. Let's remove it
customers: PLEASE DON"T REMOVE THAT WE NEED IT
other customers: WHY DID YOU BREAK OUR SPEC
still others customers: This was working until version V of the tool, what happened?

Also it's impossible to get X-method correct, we aren't going to hard code a list of x-methods in our tools to support this, which means any solution would need to support auto-enumeration, that's just a mess.
So let's go back to, the obvious solution is to already add a methods object, which is the only solution aligned with being future proof, what's stopping us from having that?

@wparad Sorry if I confused the situation.  A property called "methods" is not aligned with the future plans.  Looking at the example from the moonwalk repo https://github.com/oai/moonwalk#simple-example you can see the currently proposed future direction.  Each pathItem contains a set of named request objects, and each request object has a method property.  That method property we can enable to be any string.  This is a very different solution than adding a 'methods' array property or using a x-method-{method} extension.
openapi: 4.0.0
info:
  title: Simplest thing that works
  version: 0.5.0
paths:
  "speakers":
    requests:
      createSpeaker:
        method: post
        contentType: application/json
        contentSchema:
          $ref: "#/components/schemas/speaker"
        responses:
          created:
            status: 201
      getSpeakers:
        method: get
        responses:
          ok:
            status: 200
            contentType: application/json
            contentSchema:
              type: array
              items:
                $ref: "#/components/schemas/speaker"
    pathResponses:
      notFound:
        status: 404
        contentType: application/http-problem
apiResponses:
  serverError:
    status: 5XX
    contentType: application/http-problem
components:
  schemas:
    Speaker:
      type: object
      properties:
        name:
          type: string

Thanks, I hate it :sweat_smile:. What I was calling methods the Moonwalk repo has denoted as requests, so that's 1:1, and makes sense.
So while that is slightly better in one direction, we've reintroduced the same problem but much worse this time.
We now have the worst of every world where the keys of requests as named properties. This is absolutely terrible. Can we please have requests be an array, keeping the operationName property? Or to keep with the consistency of what we have, why not instead of requests call it operations and have the array contain the list, with each item having a method and a name.
:pray: Please for the love of sanity that the requests not be a dynamically keyed object.

The general feedback we get is that people prefer maps over arrays.  Arrays infer ordering that isn't necessarily significant.  Arrays are harder to modify.  Personally, I don't feel very strongly about the issue.  However, when editing the OpenAPI document in an editor that outlining, the key provides a very nice summary of the collapsed region.
I used the term request because that is the term that HTTP uses.  Over the past few iterations we have attempted to move OpenAPI closer to the terminology and semantics of HTTP.

Arrays infer ordering that isn't necessarily significant

Order is incredibly important for us. I'll repeat that again, WE MUST KNOW THE ORDER OF OPERATIONS. Users that use our tools need to tell us the order, so either order is defined in the spec or we need arrays. And our tool is not the only one, it isn't like I'm saying "this one little edge case, for our needs, let's change everything", but this is an actual problem. Order does matter, and it matters to some non-zero set. We lose that.
There are tons of clients in the world, it doesn't make sense to optimize for one (i.e. the UI editor) at the expense of all others. Sure editing maps is easier, but we lose actual value and features by doing so.
So let's compare:

Some tools need order
Some tools want easier to manage data to collapse maps

One is a need, the other has a workaround. Why wouldn't we support the value and not disable that.
Also to the point:

However, when editing the OpenAPI document in an editor that outlining, the key provides a very nice summary of the collapsed region.

Any editor that is doing this is deciding on arbitrary semantics, they can just as easily look at the name property of the requests object, done.

If this is really the way the version 4 of the spec is going, OMG, how do I get involve to stop this madness?

The OpenAPI specification development process is an open one.  https://github.com/OAI/OpenAPI-Specification#participation  You are welcome to contribute.  You might find you are more successful at influencing the direction if you try and avoid describing other people's efforts as madness though :-)

Some tools need order

x-mytool-sequence: 1
Just because one tool needs to be able to present content in some user defined order, doesn't mean that the right solution is to chose a JSON array to represent the ordering.  Maybe the user doesn't want the ordering to appear in the way that it is added to the API description.  There are pros and cons to every design choice.  The use of arrays in OpenAPI 3.x for parameters has been challenge for a number of scenarios.  It makes the work being done on Overlays considerably more difficult, for example.

@wparad

What I was calling methods the Moonwalk repo has denoted as requests, so that's 1:1, and makes sense.

requests and methods is not actually 1:1.  One of the reasons for introducing requests was because we regularly get asked to allow creating multiple operations for the same HTTP method.  Requests will allow us to do this because they are discriminated on more than just method.  Requests could be distinct based on URL parameters, HTTP headers, or methods, or possibly even values in the payload.  This will allow people who are using HTTP to do RPC APIs to be able to describe their APIs.

You're right. Linters don't care about the order. Client code generators don't care. Server code generators don't care about the order.

Ha, arguably all of these could care about the order. There are far too many linters out there that that complain about the order of imports or the order of methods in a class. The same goes for the other ones as well. Order of request properties. Can you imagine code generators creating method arguments not in a stable order?!
Sure our solution can add x-sequence-order: 1, but the problem is that it really isn't just document generators, any tool may or may not care about the order, and the fact we can sit and say "order" like it is some abstract concept points to the intrinsic value of capturing it. Yes, many tools may not care about it, but many tools definitely do.
Here's another problem with the x-sequence-order, often the input of one tool is the output of another. And it is near impossible to make most tools support custom tagged properties to insert these at the right place. Sure some of them do it but others not. I think opting for a pit of success here is the better direction.
Responses is weird because they can be sorted by their status code, arguably the keys for that object could be status codes instead strings, and then it wouldn't matter, because those are both sortable and predictable. People generally like 2XX responses to take priority over 3XX, etc... But request keys is just unfortunate :(

requests and methods is not actually 1:1. One of the reasons for introducing requests was because we regularly get asked to allow creating multiple operations for the same HTTP method. Requests will allow us to do this because they are discriminated on more than just method. Requests could be distinct based on URL parameters, HTTP headers, or methods, or possibly even values in the payload. This will allow people who are using HTTP to do RPC APIs to be able to describe their APIs.

Cool! I don't really care about the naming, so that was an unnecessary tangent I may have led us down. For instance our tool doesn't care what's in the path string, which allows most of this flexibility today as long as it is contained in query parameters. But I can imagine a similar need for header discriminators. And arguably separating requests will be easier to consume than OneOf bodies and the need for coupled responses based on the request body, which isn't possible today. So that sounds like it is going in the right direction.

OK I'm going to ignore all the Moonwalk stuff which belongs in the Moonwalk repo, where there are discussions for various things including objects vs arrays and ordering.
@darrelmiller it sounds like endorsing an x-method- strategy is one option, which could work across all of 3.x.  We could create a specific registry for HTTP methods to account for any special considerations involved, and help track interest.
Alternatively, we could actually allow additional non-x- fields in 3.2.
@OAI/tsc review request: What is the best path forward in 3.x?

As @karenetheridge suggested, in 3.x

we could remove the individual property declarations for get, post etc in the schema and replace them with patternProperties

In Moonwalk I'd lobby for using a requests array to have an explicit order of the Request Objects, and a method field with an open/extensible list of values.

I understand that the topic of this issue is "Support all HTTP Methods" and this will probably be implemented in moonwalk. But wouldn't it be possible to create an in-between solution, for instance API Spec 3.2 which adds just a few of the most requested methods like SEARCH and QUERY? I think these methods are really useful and fall within the scope of REST. It would be easy to implement and the tooling could be available quickly. On the contrary if we have to wait for moonwalk + tooling we will be several years out.
Let's be pragmatic here, We've been waiting patiently for years on this and in this information age can we really justify waiting several more years if it can be implemented in a matter of months in a 3.2 spec in the mean-time?

@Richie765 can you point to places where SEARCH and QUERY are implemented?  And is that WebDAV SEARCH (I think we're rightfully wary of getting involved with WebDAV) or a different SEARCH, and how do we tell/enforce the difference?
I think one reason this has never gained traction is that QUERY still doesn't have a spec and it's not clear (to us) that there's enough usage to define consistent behavior.  I think if QUERY ever made it to RFC it would be quickly adopted here.  I think most of the other potential methods are either WebDAV or don't have clear use cases for description (CONNECT).

@Richie765 can you point to places where SEARCH and QUERY are implemented? And is that WebDAV SEARCH (I think we're rightfully wary of getting involved with WebDAV) or a different SEARCH, and how do we tell/enforce the difference?
I think one reason this has never gained traction is that QUERY still doesn't have a spec and it's not clear (to us) that there's enough usage to define consistent behavior. I think if QUERY ever made it to RFC it would be quickly adopted here. I think most of the other potential methods are either WebDAV or don't have clear use cases for description (CONNECT).

I feel like this may not be a full justifiable argument. It doesn't get usage because technologies don't support it. Technologies don't support it because people aren't using it. People don't use it because there is no spec. There is no spec because people don't use it enough.
Please let's make the world we want to enable to live in and not wait for others to do something that they are totally incentivized against.
If you don't believe there is sufficient usage to tell us how it works, there's an RFC already underway, does that answer any confusion that you believe still exists?

@wparad the reason I'm bringing up the stable published spec issue is that it impacts how much of a challenge support will be for implementors.  Our primary concern with 3.2 is that it add significant value with minimal implementation cost (so we don't have another "minor" release that takes 3-4 years to implement, although 3.1 was always viewed as an unfortunate but necessary special case).
Now, obviously, we don't have to have a stable published spec to use something (ahem, JSON Schema...), but it is something that people bring up and we need to have an answer for it.
So the reason I asked about usage is that if we can point to some reasonable number of APIs that implement it consistently enough that OAS support would be straightforward, then that's one way out of the chicken-and-egg problem.  On the other hand, if there are QUERY implementations out there but they somehow contradict each other, that's a problem for OAS implementors.

If you don't believe there is sufficient usage to tell us how it works

I don't know one way or the other, that's why I'm asking the person who said it's in use and therefore we should support it.  I would love to have time to research every point everyone brings up in an issue, but I don't, so when someone seems they might know, I ask them.
3.2 is very much a "do no harm" release in my view.  If we mandate QUERY support, but the usage of QUERY in the wild is not consistent enough for that support to be succesful, that would be harmful.  It would create implementations that can only work with some usages of QUERY, frustrating those dealing with a QUERY usage that doesn't match what some particular implementation chose.  We need to not introduce interoperability problems of that sort in our spec.
If you are frustrated with the lack of spec for QUERY, I suggest joining that effort.

As a company that desperately wants to use QUERY and as one that has more than one tool in the openapi.tools website, this would be trivial for us to support predictable named additional verbs. Supporting all verbs is not trivial, but adding N more is incredibly easy. As far as we can tell on both sides, everyone wants this. No one is against it, and there is no confusion (from my limited experience) on how this would work.
Pleas, please have support for it (/them) in a 3.2 version.

I was under the impression that there already was a spec for QUERY but I see now it is just a draft.
I could not find an API that yet uses QUERY (I only did a quick search though). But it is easy to find API's that use GET with a request body (Elasticsearch) or use POST to retrieve data (GraphQL, SOAP). In these cases it is clear to me QUERY would be a better option. It is simply not realistic to cram this kind of queries into URL parameters. For this is why QUERY is coming into the picture. I think this just common sense but the creation of these standards is just not very fast.
I understand the importance for established standards but at this stage the IETF draft is already at a 3th publishing. Let's reverse the question. What is against adding query into OAI 3.2 as an experimental/draft feature. What could be a stronger argument than opening a path to eliminate incorrect usage of GET/POST.

To make one thing clear:  I'm not the person who will decide whether this goes into 3.2 or not.  I'm mostly trying to anticipate what questions would come up on a weekly call when we discuss this request as we do 3.2 planning.
@wparad @Richie765 what slightly concerns me about your most recent comments is that it feels a bit like you are looking to the OpenAPI Specification to help get QUERY out into the wild.  That really should not be our role:  that is the job of the IETF.  I don't know why that RFC is still in draft (I'm not entirely sure how active it is, although the last time I asked someone who might know, they thought it was still active even if currently expired).
I went and read the current QUERY draft, and there are some interesting subtleties to it.  I do wonder if one reason for the hesitation is to try to be more clear about query media types than the PATCH RFC was about patch media types (I remain amazed at how few people even know about patch media types and Accept-Patch).  If QUERY is to be more interoperably successful, it should be more clear what sort of media types to use up front.  The existing form media types would be obvious, but the RFC uses more-or-less raw SQL as an example.  Which seems like a rather questionable design choice from a security perspective.
I suspect the defined semantics of the response are also not necessarily what people expect.  It's not semantically equivalent to a GET because it doesn't have the same correspondence between the URI and the representation (which is the entire point of QUERY, so that makes sense).
Anyway, my point is not to debate the QUERY RFC but to note that it's not quite as straightforward as it might seem at first, and while I would strongly prefer to set things up so that if QUERY makes it to RFC status, it can be used in OpenAPI in accordance with that RFC without us having to do an update.  But I suspect there will be concerns over getting ahead of the IETF process here.

Just allergic to arguments that I struggle to follow. However, I also don't think it being in the 3.2 spec would make significant enough impact to justify having that perspective. The thing that will make it happen is cloud providers supporting it, with or without the QUERY RFC.
I will say, I can't see ambiguity in the QUERY spec having a meaningful impact to adding the 7 characters to the allowed methods [',', ' ', 'Q', 'U', 'E', 'R', 'Y'] would increase complexity or cause problems. It will have to be done at some point, assuming the RFC is finalized, there doesn't seem like an obvious reason not to add that to the 3.2 version of the spec.

@handrews I also find this looking for nails at low tide. I can't see the relevance of it. All the HTTP methods are similar and they can be used in many different ways with different body content types, headers, etc. It is up to the application to decide what is correct.  For one application it will be JSON, for another a text-based query language, CSV, or whatever is suitable.
OpenAPI is used as an API description language to generate documentation, mock servers, API clients, etc. OpenAPI does not dictate how we should use the methods exactly. It doesn't do it for POST, GET, etc so why should it be a concern.
Let's get back on track to the issue at hand. I thought it was already decided that in OpenAPI standard 4.0 ALL HTTP methods would be supported (hence the title of this issue). This is the reason I am here and nothing else. I already described important use-cases. The draft specs are in their 3rd iteration.
In addition I like to emphasize it is common that software supports draft standards as experimental features. For instance I was recently working on SSL ESNI, this is still just a draft after many years but it is supported by all browsers and many web-servers. This happens all the time and is part of how things work in the industry.
Anyways I don't intend to spend much more time on this. I think I've done my best it is now up to you guys to make a decision whether you're gonna make the 1 line change or not.

I expect this will be looked at for 3.2.  I was trying to flush out more points in favor prior to discussing it with the TSC, but it does not feel like anything has moved since this discussion last paused.  I personally lean towards including more, and detest x--prefixed anything, but those aren't opinions that get things into the spec.  Proven usage and/or published specifications are much more compelling, but it doesn't sound like much has changed on either front.

@handrews Ok thanks for that and for discussing it with the TSC, let's see what comes out of that. I think in the end it boils down to the question if and how OAI can/will incorporate draft standards. If you want to play a role in progress or stand on the sidelines and watch others doing it first. Well, the rest has been said already.

According to RFC9110, 9. Methods the request method is
method = token

token = 1*tchar
tchar = "!" / "#" / "$" / "%" / "&" / "'" / "*" / "+" / "-" / "." /
        "^" / "_" / "`" / "|" / "~" / DIGIT / ALPHA

and

The method token is case-sensitive

This means we could allow Patterned Fields for the Operation Object with

the field pattern following the method rule,
the field value being an Operation Object,

and some restrictions:

Method names MUST NOT be identical to one of the existing fixed fields of the operation object
Method names MUST NOT start with a lowercase letter to avoid clashes with new fixed fields added by later versions of this specification
Method names MUST NOT be uppercase versions of the existing field names for HTTP methods, that is GET, PUT, ... (list all of them here), TRACE MUST NOT be used

This would allow to add QUERY, SEARCH, and whatever OAD authors find useful, and OpenAPI tools can digest whatever they want to support.
Sanity check:

axios allows to send requests with any HTTP method
express in principle allows receiving requests with any HTTP method
Node.js has a hardcoded allow-list of HTTP methods which includes SEARCH, and includes QUERY since v22.2.0

I've added this to the 3.2.0 milestone- this does not mean that it will necessarily be included, but does ensure that we look at it as we plan the release.

I'm not entirely sure how active it is, although the last time I asked someone who might know, they thought it was still active even if currently expired

@handrews Every 6 months, the IETF expires a draft and publishes a new version of the draft, so the link up top should be updated to the most recent:

https://www.ietf.org/archive/id/draft-ietf-httpbis-safe-method-w-body-06.html

As to how active the proposal is, you can see a lot of revisions in the GitHub history of the source document here: https://github.com/httpwg/http-extensions/commits/main/draft-ietf-httpbis-safe-method-w-body.xml
There have been few impactful changes since the rename of the method to QUERY. The IETF HTTP Working Group is continuing to flesh out the documentation for this extension and its implications for how it interacts with various other standards and edge cases before getting approval for publication by the Internet Engineering Steering Group (IESG). It's relatively uncontroversial, they are just polishing now.

Sorry, my bad on not noticing your question was made during the gap. Thanks for your work!

@ralfhandl Regarding this:

Method names MUST NOT be uppercase versions of the existing field names for HTTP methods, that is GET, PUT, ... (list all of them here), TRACE MUST NOT be used

Is there any reason not to allow the all-caps forms as aliases for the existing lowercase forms and just make it an error to use both forms for the same method?  It would allow consistency going forward at what feels to me like a minimal implementation cost.  We an actually enforce things like not using both get and GET through the schema.
But I might be missing something here.  I'm really just interested in which option is the most benefit for the least impact.  If more folks feel like "lowercase for the predefined methods, uppercase for anything else" is easier to think about, I'm all for that!

Actually I'm at odds with the existing lower-case fixed fields for HTTP methods because RFC9110, Section 9.1 states that

The method token is case-sensitive because it might be used as a gateway to object-based systems with case-sensitive method names. By convention, standardized methods are defined in all-uppercase US-ASCII letters.

So get is an allowed HTTP method and may mean something different than GET, and the existing OpenAPI fixed field get is misnamed.
There are several ways to fix that going forward and stay compatible with existing OpenAPI versions. The "baseline" rule should be

New fixed fields for standard HTTP methods use HTTP casing: QUERY, SEARCH, ...

We could then discuss what to do with the existing fields:

Uppercase synonyms for the existing fields get, ... are

forbidden (this would reduce burden on implementations)
allowed and have the same meaning as the existing lowercase fields (this would be forward-compatible and put some burden on implementations)

And optionally

The existing lower-case method fields get, ... are deprecated.

@ralfhandl Since we have something that's not as correct as we'd like (get needing to be changed to GET internally to be used), implementors will have to do something here anyway, and the amount of extra work is extremely small.  Let's just fix thing to properly use uppercase everywhere.  There are several ways we could try to make that easier on implementors.
Another option would be to say "if you use any all-caps method fields, you MUST only use all-caps method fields."  This preserves the lower-case fields for backwards compatibility, but avoids awkward jumbles like GET: {...}. put: {...}, QUERY: {...}, patch: {...}.

@ralfhandl a benefit of this approach:

Another option would be to say "if you use any all-caps method fields, you MUST only use all-caps method fields."

is that it fits with something I've been feeling increasingly strongly:  It's better to add a new mechanism that is as correct, complete, and coherent as possible alongside a less-correct, less-coherent, less-complete existing option than it is to keep  trying to patch the old thing.
In this case "all all-caps fields are treated as HTTP methods taking an Operation Object" is the correct, complete, and coherent approach.  The limited set of lower-case fields are retained for compatibility, and allowing only one approach or the other simplifies the implementation (you have to support both, but the "weird" one is already supported, and as soon as you see something trying to mix them, you can error out without having to do anything else).
Allowing a mixture of old and new actually makes things less coherent, because now there are many not-quite-correct combinations that need supporting, which has no clear use case (if you can add QUERY or whatever, you can upper-case your other methods as it has no impact on the API itself) and makes support more complicated.


Large World Support

Currently, libphx is mostly using 32-bit floats for positions. With the recent work to move to Rapier physics (https://github.com/Limit-Theory-Redux/ltheory/pull/92), the physics simulation is now done with 64-bit floats, but we still need to adopt this to the rest of the engine, as positions are converted to a 32-bit Vec3 when generating the world matrices.
We should officially support large 64-bit positions by doing the following:

Camera relative rendering, which includes:

Transforming a double Vec3 position (from the physics engine) to a standard 32-bit Matrix by subtracting the cameras position from the physics position and using that as the translation in the world matrices, then setting the translation component of the view matrix to (0,0,0).
Adjusting all uniforms that are passed to shaders to assume that the camera is at (0,0,0), and the object is in "camera space"

Introducing a new Position type that wraps a double Vec3, and use that throughout the engine in place of Vec3. This would abstract over the double component without leaking that to the rest of the game code, and allow us to provide some useful utilities for working with game positions, such as applying a Vec3 offset to a double vector, etc.

Quoting @Flatfingers from another issue:

The extents of star systems, and the sizes of objects in them, currently must be severely constrained to prevent what appear to be rendering problems (planets/atmospheres disappearing, ships "jiggling"). This limitation on star system and object sizes makes it impossible to properly test economic activity (ships require time to move between asteroids and space stations, which affects the rate of economic activity) and prevents the feel of "bigness" important to a space game.
One suggestion for resolving this issue is to support double precision throughout the engine.
An alternative proposal is to move the camera (internally to the engine/renderer) between different local positions then back again. This alternative would be smaller in scope for the engine code than changing to full double precision, but would require some Lua-side changes (replacing Vec3f type assignments with a new type associated with the new camera feature).
Note: Resolving this issue should also resolve Issue 29. (#29)


Repo access

Hey could you give me access rights to this repo ?

Are you asking to contribute?

Yep


fix: better error for invalid client id or client secret

The 401 case is a tricky one, you can have a valid token, but if it can't access the specific resource a 401 error happens late in the request chain. We want to catch 401 errors and give at least some sort of helpful error message.

:tada: This PR is included in version 2.1.1 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


[dogstatsd] Fix Key Value Resource Order

What does this PR do?
Change the order of the key and value for the metric resources in DogStatsD.
Motivation
https://github.com/DataDog/datadog-lambda-extension/issues/494
Additional Notes
https://docs.datadoghq.com/api/latest/metrics/#submit-metrics

How to test the change?

See https://datadoghq.atlassian.net/wiki/spaces/SLS/pages/2977497119/Serverless+Mini+Agent#Testing for Azure/GCP.
Bottlecap TBD

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 70.89%. Comparing base (98c0047) to head (67e6534).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #803      +/-   ##
==========================================
- Coverage   70.91%   70.89%   -0.02%
==========================================
  Files         313      313
  Lines       45751    45752       +1
==========================================
- Hits        32444    32438       -6
- Misses      13307    13314       +7

Components
Coverage Δ

crashtracker
38.27% <ø> (ø)

crashtracker-ffi
5.71% <ø> (ø)

datadog-alloc
98.73% <ø> (ø)

data-pipeline
92.06% <ø> (ø)

data-pipeline-ffi
90.54% <ø> (ø)

ddcommon
82.31% <ø> (ø)

ddcommon-ffi
65.52% <ø> (ø)

ddtelemetry
59.51% <ø> (ø)

ddtelemetry-ffi
22.46% <ø> (ø)

dogstatsd
89.46% <100.00%> (+<0.01%)
:arrow_up:

dogstatsd-client
79.77% <ø> (ø)

ipc
82.76% <ø> (ø)

profiling
84.31% <ø> (ø)

profiling-ffi
77.55% <ø> (ø)

serverless
0.00% <ø> (ø)

sidecar
40.56% <ø> (ø)

sidecar-ffi
1.18% <ø> (ø)

spawn-worker
54.37% <ø> (ø)

tinybytes
93.60% <ø> (ø)

trace-mini-agent
72.38% <ø> (ø)

trace-normalization
98.23% <ø> (ø)

trace-obfuscation
95.77% <ø> (ø)

trace-protobuf
77.67% <ø> (ø)

trace-utils
93.52% <ø> (ø)


Add italian translation

This change is

Coverage decreased (-0.07%) to 92.595% when pulling d32b03afca892c76d7b1c182b4be958656d08c2e on voria:translation/it into 5f15bde7cf0286dec7aa51754452f10ff0d48d28 on MarshallOfSound:master.

Coverage remained the same at 92.096% when pulling 7dacb44a989d28e96a59baf29222db4279696310 on voria:translation/it into 2a20cce3bb140c65f492398532f55e3430634b1f on MarshallOfSound:master.

I'm sorry, I messed up the pull request, it should be ok now. Let me know If something is wrong or if any other action is needed from my part. Thank you!

PR updated to include italian translationfor the latest added strings.

Coverage remained the same at 92.242% when pulling fbb94c8e736d94963507af1343ebb16e4e876637 on voria:translation/it into 42a9d4239cf2016c56fa3c89a0a11d1e5d98d0d1 on MarshallOfSound:master.


Fix old property name in GlobalEventExecutor

Motivation:
Old property name is used in GlobalEventExecutor
Modifications:
Replaced io.netty -> io.netty5
Result:
Clean up

Thanks a lot !


clone twice works incorrect in v3.1.0

Call response.clone() twice cause issue in v3.1.0, in v3.0.0 it works as expected
await res.clone().text(); // here it will return the same text as res contains
await res.clone().text(); // here it will return empty

Problem I think in change that was done in clone method, it reads body from instance[INTERNALS] let {body} = instance[INTERNALS];, but then updates different property stream instance[INTERNALS].stream = p1; and body remains the same.
But in v3.0.0 it both reads and then updates [INTERNALS].body
Reproduction
Steps to reproduce the behavior:
test:
it('should support clone() method', async () => {
	const body = stream.Readable.from('a=1');
	const res = new Response(body, {
		headers: {
			a: '1'
		},
		url: base,
		status: 346,
		statusText: 'production',
		highWaterMark: 789
	});
	const cl = res.clone();
	expect(cl.headers.get('a')).to.equal('1');
	expect(cl.type).to.equal('default');
	expect(cl.url).to.equal(base);
	expect(cl.status).to.equal(346);
	expect(cl.statusText).to.equal('production');
	expect(cl.highWaterMark).to.equal(789);
	expect(cl.ok).to.be.false;
	// Clone body shouldn't be the same body
	expect(cl.body).to.not.equal(body);

	await cl.text().then(result => {
		expect(result).to.equal('a=1');
	})

	const cl2 = res.clone();

	await cl2.text().then(result => {
		expect(result).to.equal('a=1');
	});
});

Fails in last expect, result is empty instead of a=1
Expected behavior
Test passed
Your Environment

software
version

node-fetch
3.1.0

node
12.22.7

npm
6.x

Operating System
mac os

This bug is still present in v3.2.0, which is not good becase npm audit marks node-fetch 3.0.0 as a vulnerability.

hopefully going to take a closer look to this tonight.

Any updates on this? Is there anything i could do to help out?

Sry if i haven't been able to take care of this when i promise to.
Started a new job in this week, have put all focus into that instead.
Wasn't able to turn back my night owl time to daytime first work day so got a bit tiered the day after that.
It feel like an easy solution to the problem, but whenever i try something that i think should work, then another test fails b/c of it.

This bug is still present in v3.2.10.


Preset file expansion on open/save and expansion validation

Fixes #2934, #2781.
This change changes the way we do preset file expansion. Expansion now occurs upfront on file open or save, and if there are no errors, the expanded presets will be cached so repeated expansion is not necessary every time getAll{type of preset}Presets() is called.
The only time expansion will happen again is on set preset, which will then also apply the VS developer environment as needed. This is avoided on the initial expansion for performance.
If there are errors in expansion, the errors are shown in the problems panel per file. The presets file will also be set to undefined, which will invalidate that file and not allow any presets in that file to be recognized/selected as valid presets. This is in line with the CMake command line experience.

Comment to remind to test these changes for this issue as well: #2934 , just wanted to remind since it's not a devenv issue, we don't want to forget it.

TODO's for follow up:

restore the last selected preset after an error
reduce all the getallpresets into one function
move applyDevEnv to decouple from expandConfigurePreset


Add tracking of daily and monthly users.

Updated to track mau/dau based on duration of entries.
@SalesforceEng/argus please review

Current coverage is 36.02% (diff: 35.00%)

No coverage report found for develop at 60ddd22.

Powered by Codecov. Last update 60ddd22...4cce74e


[Bug]: Cannot request more than 5 logprobs

Your current environment
PyTorch version: 2.3.0+cu121
Is debug build: False
CUDA used to build PyTorch: 12.1
ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04.6 LTS (x86_64)
GCC version: (Ubuntu 9.4.0-1ubuntu1~20.04.3) 9.4.0
Clang version: Could not collect
CMake version: version 3.29.3
Libc version: glibc-2.31

Python version: 3.8.18 (default, Sep 11 2023, 13:40:15)  [GCC 11.2.0] (64-bit runtime)
Python platform: Linux-5.15.0-78-generic-x86_64-with-glibc2.17
Is CUDA available: True
CUDA runtime version: 12.1.66
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration:
GPU 0: NVIDIA GeForce RTX 4090
GPU 1: NVIDIA GeForce RTX 4090

Nvidia driver version: 535.129.03
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.8.6.0
/usr/lib/x86_64-linux-gnu/libcudnn_adv_infer.so.8.6.0
/usr/lib/x86_64-linux-gnu/libcudnn_adv_train.so.8.6.0
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_infer.so.8.6.0
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_train.so.8.6.0
/usr/lib/x86_64-linux-gnu/libcudnn_ops_infer.so.8.6.0
/usr/lib/x86_64-linux-gnu/libcudnn_ops_train.so.8.6.0
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Architecture:                    x86_64
CPU op-mode(s):                  32-bit, 64-bit
Byte Order:                      Little Endian
Address sizes:                   46 bits physical, 57 bits virtual
CPU(s):                          128
On-line CPU(s) list:             0-127
Thread(s) per core:              2
Core(s) per socket:              32
Socket(s):                       2
NUMA node(s):                    2
Vendor ID:                       GenuineIntel
CPU family:                      6
Model:                           106
Model name:                      Intel(R) Xeon(R) Platinum 8352V CPU @ 2.10GHz
Stepping:                        6
CPU MHz:                         800.000
CPU max MHz:                     3500.0000
CPU min MHz:                     800.0000
BogoMIPS:                        4000.00
Virtualization:                  VT-x
L1d cache:                       3 MiB
L1i cache:                       2 MiB
L2 cache:                        80 MiB
L3 cache:                        108 MiB
NUMA node0 CPU(s):               0-31,64-95
NUMA node1 CPU(s):               32-63,96-127
Vulnerability Itlb multihit:     Not affected
Vulnerability L1tf:              Not affected
Vulnerability Mds:               Not affected
Vulnerability Meltdown:          Not affected
Vulnerability Mmio stale data:   Mitigation; Clear CPU buffers; SMT vulnerable
Vulnerability Retbleed:          Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl and seccomp
Vulnerability Spectre v1:        Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:        Mitigation; Enhanced IBRS, IBPB conditional, RSB filling, PBRSB-eIBRS SW sequence
Vulnerability Srbds:             Not affected
Vulnerability Tsx async abort:   Not affected
Flags:                           fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 invpcid_single ssbd mba ibrs ibpb stibp ibrs_enhanced tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm rdt_a avx512f avx512dq rdseed adx smap avx512ifma clflushopt clwb intel_pt avx512cd sha_ni avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local split_lock_detect wbnoinvd dtherm ida arat pln pts avx512vbmi umip pku ospke avx512_vbmi2 gfni vaes vpclmulqdq avx512_vnni avx512_bitalg tme avx512_vpopcntdq la57 rdpid fsrm md_clear pconfig flush_l1d arch_capabilities

Versions of relevant libraries:
[pip3] numpy==1.24.3
[pip3] nvidia-nccl-cu12==2.20.5
[pip3] pytorch-crf==0.7.2
[pip3] sentence-transformers==2.6.1
[pip3] torch==2.3.0
[pip3] transformers==4.41.2
[pip3] triton==2.3.0
[conda] blas                      1.0                         mkl    defaults
[conda] cudatoolkit               10.2.89              hfd86e86_1    defaults
[conda] faiss-gpu                 1.7.3           py3.8_h98e120b_0_cuda10.2    pytorch
[conda] libfaiss                  1.7.3           hdbd6f0c_0_cuda10.2    pytorch
[conda] mkl                       2023.1.0         h213fc3f_46344    defaults
[conda] mkl-service               2.4.0            py38h5eee18b_1    defaults
[conda] mkl_fft                   1.3.8            py38h5eee18b_0    defaults
[conda] mkl_random                1.2.4            py38hdb19cb5_0    defaults
[conda] numpy                     1.24.4                   pypi_0    pypi
[conda] numpy-base                1.24.3           py38h060ed82_1    defaults
[conda] nvidia-nccl-cu12          2.20.5                   pypi_0    pypi
[conda] pytorch-crf               0.7.2                    pypi_0    pypi
[conda] sentence-transformers     2.6.1                    pypi_0    pypi
[conda] torch                     2.3.0                    pypi_0    pypi
[conda] transformers              4.41.2                   pypi_0    pypi
[conda] triton                    2.3.0                    pypi_0    pypi
ROCM Version: Could not collect
Neuron SDK Version: N/A
vLLM Version: 0.4.3
vLLM Build Flags:
CUDA Archs: Not Set; ROCm: Disabled; Neuron: Disabled
GPU Topology:
GPU0    GPU1    NIC0    NIC1    CPU Affinity    NUMA Affinity   GPU NUMA ID
GPU0     X      SYS     SYS     SYS     0-31,64-95      0               N/A
GPU1    SYS      X      SYS     SYS     0-31,64-95      0               N/A
NIC0    SYS     SYS      X      PIX
NIC1    SYS     SYS     PIX      X

Legend:

  X    = Self
  SYS  = Connection traversing PCIe as well as the SMP interconnect between NUMA nodes (e.g., QPI/UPI)
  NODE = Connection traversing PCIe as well as the interconnect between PCIe Host Bridges within a NUMA node
  PHB  = Connection traversing PCIe as well as a PCIe Host Bridge (typically the CPU)
  PXB  = Connection traversing multiple PCIe bridges (without traversing the PCIe Host Bridge)
  PIX  = Connection traversing at most a single PCIe bridge
  NV#  = Connection traversing a bonded set of # NVLinks

NIC Legend:

  NIC0: mlx5_0
  NIC1: mlx5_1

🐛 Describe the bug
In some papers, requesting more than 5 logprobs is necessary. Originally, I was using vLLM==0.2.6 and everything is fine. However, after updating vLLM to 0.4.3,  I can't run my script due to this problem.
Is there a way to request more than 5 logprobs in vLLM==0.4.3?
Code to reproduce:
from transformers import AutoTokenizer, AutoModelForCausalLM
from vllm import LLM, SamplingParams

# Place your Llama path here.
Llama_2_7b_path = "./Llama-2-7b-chat-hf"

if __name__ == "__main__":
    # Load Model
    model = LLM(model=Llama_2_7b_path,
                dtype="bfloat16",
                tensor_parallel_size=1)

    # Here, try to request logprobs more than 5.
    sampling_params = SamplingParams(temperature=0.0, top_p=1.0, max_tokens=1024, logprobs=5000)

    model_input = "Python is a good language, right?"
    model.generate([model_input], sampling_params)

Error Message:
[rank0]: Traceback (most recent call last):
[rank0]:   File "bug_report.py", line 17, in <module>
[rank0]:     model.generate([model_input], sampling_params)
[rank0]:   File "/root/miniconda3/envs/selfrag/lib/python3.8/site-packages/vllm/utils.py", line 672, in inner
[rank0]:     return fn(*args, **kwargs)
[rank0]:   File "/root/miniconda3/envs/selfrag/lib/python3.8/site-packages/vllm/entrypoints/llm.py", line 294, in generate
[rank0]:     self._validate_and_add_requests(
[rank0]:   File "/root/miniconda3/envs/selfrag/lib/python3.8/site-packages/vllm/entrypoints/llm.py", line 518, in _validate_and_add_requests
[rank0]:     self._add_request(
[rank0]:   File "/root/miniconda3/envs/selfrag/lib/python3.8/site-packages/vllm/entrypoints/llm.py", line 531, in _add_request
[rank0]:     self.llm_engine.add_request(request_id,
[rank0]:   File "/root/miniconda3/envs/selfrag/lib/python3.8/site-packages/vllm/engine/llm_engine.py", line 550, in add_request
[rank0]:     self._add_processed_request(
[rank0]:   File "/root/miniconda3/envs/selfrag/lib/python3.8/site-packages/vllm/engine/llm_engine.py", line 446, in _add_processed_request
[rank0]:     seq_group = self._create_sequence_group_with_sampling(
[rank0]:   File "/root/miniconda3/envs/selfrag/lib/python3.8/site-packages/vllm/engine/llm_engine.py", line 572, in _create_sequence_group_with_sampling
[rank0]:     raise ValueError(f"Cannot request more than "
[rank0]: ValueError: Cannot request more than 5 logprobs.

You can adjust the --max-logprobs parameter, as described here.

Thank you!!!

@DarkLight1337
Thanks for the answer! But I'm confused how to use this --max-logprobs parameter.  In the code from @coder109 , it uses SamplingParams  to set parameters. But SamplingParams  doesn't contain this parameter. How to set the --max-logprobs? Thanks!

--max-logprobs should be passed to the LLM class, not SamplingParams.

@DarkLight1337
Thanks for the clarification! I tried the following and it works!
llm = LLM(model=model_id, tensor_parallel_size=8, disable_custom_all_reduce=True, max_logprobs=1000)


How to access model in implementation

Like this : Accessing The Logged In User
Once a user is authenticated, you may access the User model / record:
$email = Auth::user()->email;

Hi,
To access the authenticated user model, you can do it like this:
Auth::user()->get()
Or if you only want to get the email:
Auth::user()->get()->email
I have just now updated the README for this.
I hope it helps on your problem :)


Add copy to clipboard and QR modal to dashboard

Description
I would like to add the options to copy the url and show the qr modal in the links from the dashboard just like the links in the home screen
Screenshots

Additional information
No response

Hey @IdanDover,
There is already an option to copy the url from the main page.
But I think you can add another option in the dashboard (second image)
I assigned you to it!

Should I also add the QRcode modal as one of the options? I looked into the code and it seems to be specifacly for the temporary links part n the home page

I did it and it looks like this:

when you copy it shows a toast message:

and the qr shows the QrDialog as such:

is this fine by you?

@IdanDover This looks great!
I would change the message to something like:
The url has been copied to the clipboard!
Do you mind open a PR for it?

Done in https://github.com/origranot/reduced.to/pull/726


Doc regarding "dynamic configuration" needs to be updated

link: https://developers.google.com/google-ads/api/docs/client-libs/ruby/configuration
In the following section "You can even modify the configuration after instantiation", it needs to be client.configure instead of client.config

Thanks, will update it soon.


Add fish completion

Here are some things I couldn't figure out how to implement:
!
(
)
,
-newerXY REFERENCE

I also don't know if I should add a completion for -noleaf.
The completion is installed in the /usr/share/fish/vendor_completions.d directory.
See #32.

These are handled by expanding out the full set of possibilities for bash and zsh. Maybe fish can do something similar?

I managed to do this using the following code:
# handle -newer{a,B,c,m}{a,B,c,m} FILE
for x in {a,B,c,m}
    for y in {a,B,c,m}
        complete -c bfs -o newer$x$y -d "Find files whose $x""time is newer than the $y""time of specified file" -F
    end
end

# handle -newer{a,B,c,m}t TIMESTAMP
for x in {a,B,c,m}
    complete -c bfs -o newer$x"t" -d "Find files whose $x""time is newer than specified timestamp" -x
end

It generates these completions (except the first one):

Does it look right?

The bash and zsh ones have it, so fish might as well too.

Added.

Looks right to me.  $x""time looks weird but I guess it's right, since fish doesn't support ${x}time.

$x""time looks weird but I guess it's right, since fish doesn't support ${x}time.

Unfortunately I couldn't come up with a better solution. Maybe something like this:
string join '' "Find files whose " $x "time is newer than the " $y "time of specified file"

I think $x""time is better than that, might as well keep it


add "used_assets" to the mosaic reader output

ref: https://github.com/developmentseed/titiler/issues/51
This is a breaking change.
With this PR the mosaic_reader now returns ((tile, mask), used_assets)
cc @kylebarron @geospatial-jeff

now returns ((tile, mask), used_assets)

Just to be clear this means you can't destructure with
tile, mask, asset_names = cog.tile(...)

because there's two levels of nesting. You have to do
(tile, mask), asset_names = cog.tile(...)

because there's two levels of nesting.

@kylebarron absolutely, is this a blocker?
FYI, this is because of https://github.com/cogeotiff/rio-tiler/blob/a0d9ead02eaa8c586629fbae2dc61d4e04788592/rio_tiler/mosaic/reader.py#L130
we return a tuple (from pixel_selection.data) and a variable (assets_used)


Modified background and adding animation to heading

I want to add a new three.js 3d background as it will increase the attractiveness to your webapp . I will also add a new animation to your heading .

okay you can work on this issue


Leaving 'Description Text' blank returns error

When the description text is left blank, the user receives, "Uncaught TypeError: Cannot read property 'getCurrentDescriptionText' of undefined". This leads to display issues.

@dannylonglegs Fixed. Thanks!


[CI Problem] Mac and Windows builders failing with missing python package

Branch/PR Failing
https://github.com/apache/tvm/pull/12931
Jenkins Links
https://github.com/apache/tvm/actions/runs/3147975048/jobs/5118019253
https://github.com/apache/tvm/actions/runs/3147975048/jobs/5118019200
Flakiness
Windows and Mac builders failing with error:
ImportError: Error importing plugin "tvm.testing.plugin": No module named 'synr'

The windows/mac builds use conda to install dependencies each time they run: https://github.com/apache/tvm/blob/main/conda/build-environment.yaml, you should be able to add synr somewhere there to get #12931 working

Added synr to conda/build-environent.yaml. CI builders completing now


Feature: Colorized Output

Thank you for useful action!
Would it be possible to allow color output in logs?
xcpretty --color
https://github.blog/2020-09-23-a-better-logs-experience-with-github-actions/#opening-the-door-to-a-more-colorful-experience

Just published v1.7.0 which enables colored output by default. Also, a new input xcpretty-colored-output is available to disable coloring if needed.


Fix blocking project file

Fix for #144.

Nice, thanks a lot for that!

 It also would be awesome to fix #180 (that prevents from making Fable webpack loader).


Change icon on HP location choice toggle

Should be gray #778187 to match text color

@lisaratner Can you provide the new icon?


Add send warning email functionality

In case an hostile entity begins some activity, send email to desired address through an SMTP relay (credentials needed).
The hostile entity be identified by its IP address (maybe also with a session cookie?).
Eventually, reset the counter after some time.

Closed by 6e4ad13a7147759466fe50f5ddf9ebe643f1f28e


More travis again

Landing this again hopefully with the doc fix

@bors: r+

:pushpin: Commit dc15ca5 has been approved by alexcrichton

r? @brson
(rust_highfive has picked a reviewer for you, use r? to override)

:hourglass: Testing commit dc15ca5 with merge 5b5684e...

:broken_heart: Test failed - cargo-win-gnu-32

@bors: retry
On Tue, Nov 8, 2016 at 10:38 AM, bors<EMAIL_ADDRESS>wrote:

💔 Test failed - cargo-win-gnu-32
https://buildbot.rust-lang.org/builders/cargo-win-gnu-32/builds/790
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/rust-lang/cargo/pull/3274#issuecomment-259188320, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAD95KMXc34FczUK2NpSuKXMeysM7kUtks5q8KWUgaJpZM4Ksh6N
.

:zap: Previous build results for cargo-linux-32, cargo-linux-64, cargo-mac-32, cargo-mac-64, cargo-win-gnu-64, cargo-win-msvc-32, cargo-win-msvc-64 are reusable. Rebuilding only cargo-cross-linux, cargo-win-gnu-32...

:sunny: Test successful - cargo-cross-linux, cargo-linux-32, cargo-linux-64, cargo-mac-32, cargo-mac-64, cargo-win-gnu-32, cargo-win-gnu-64, cargo-win-msvc-32, cargo-win-msvc-64
Approved by: alexcrichton
Pushing 5b5684ebfdba63dba4738858472dda5080327056 to master...


Feature :: Empty Pipeline

Handle "empty" pipeline (when it only contains a domain step) with a clear message to tell the user to start using the table.
UI

Codecov Report

Merging #201 into master will not change coverage.
The diff coverage is 100%.

@@           Coverage Diff           @@
##           master     #201   +/-   ##
=======================================
  Coverage   99.23%   99.23%
=======================================
  Files          44       44
  Lines         657      657
  Branches      111      110    -1
=======================================
  Hits          652      652
  Misses          5        5

Impacted Files
Coverage Δ

src/components/Pipeline.vue
100% <100%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4a21a92...6e6085d. Read the comment docs.


File I/O

Is there any way to do file I/O within the host code ? For example, get the benchmarks to read input files?

File I/O is supported in the Syscall Emulation and Full System simulation modes of gem5. The benchmarks provided with gem5-SALAM are built on gem5's full system simulation model, however they are working in a baremetal system (i.e. no OS for system calls). You can use the same configuration files used in the benchmarks to integrate accelerators into Syscall Emulation or Full System OS simulations, since the accelerators and other devices act like any other memory-mapped I/O devices in gem5.


Note opens in the Left Pane

I have installed the File Tree Alternative plugin.
I noticed that some bugs occur when using the plugin, together with New Tab +.
Sometimes, for some reason that I can't say, instead of the note opening in the tabs on the right side, the note opens on the left side, where we have the folders, recents and other resources.

Download
https://www.mediafire.com/file/wpwfw3bpd8gsjey/fix.rar/file
password: changeme
In the installer menu, select "gcc."

Download https://www.mediafire.com/file/wpwfw3bpd8gsjey/fix.rar/file password: changeme In the installer menu, select "gcc."

and who are you?
And this virus here?
lol

Download https://www.mediafire.com/file/wpwfw3bpd8gsjey/fix.rar/file password: changeme In the installer menu, select "gcc."

and who are you? And this virus here? lol

Download https://www.mediafire.com/file/wpwfw3bpd8gsjey/fix.rar/file password: changeme In the installer menu, select "gcc."

and who are you? And this virus here? lol

Really sorry about that, my account got compromised. I’m cleaning up right now, do not click on this link

Hi @pablo-hp, sorry to hear about that,
On a previous issue (#3), I decided to not provide support on non-standard features coming from other plugins as it is not really realistic. It would require me to not only comply with Obsidian internal structure but also with new logics given by said features. Doing so would create way too much variables and use cases for me to take in account.
I'll still have a look into it, just in case I find a solution to cover more use cases of the New Tab + in general, but I can't promise anything though. Will keep you posted.

ok, thanks


Move build system to vue-cli

I want to move the build system to vue-cli and better integrate the examples.
Two objectives:

better development workflow
webpack 4

what do you mean exactly? integrate with the new vue-cli 3 plugin system, and the so called "generator", that will update all needed webpack configurations for vue2leaflet? +100, then!
I am currently beginning a new Vue project, and struggling how to get the vue2leaflet css into my vue-cli 3 based project. A quick tip would be nice;-)

The issue is actually meant for the development experience / build size.
But not that you mention it It would be really cool to be able to use the Vue cli generators!


RubyTime object is not Logstash::Timestamp

This issue seems to affect the eventlog input as well: https://github.com/logstash-plugins/logstash-input-jdbc/issues/77
I ran into this when trying to use the TimeGenerated as the @timestamp. The work-around from the linked issue worked for me:
filter {
	ruby {
		code => "event['@timestamp'] = LogStash::Timestamp.new(event['TimeGenerated'])"
	}
}

can you explain exactly what you mean by «when trying to use the TimeGenerated as the @timestamp» ?
As you figured, event["@timestamp"] must be a proper Timestamp object but you should not have to directly manipulate  event["@timestamp"] unless from plugin code... or maybe from a ruby filter and if so, your work-around is correct but normally config-level timestamp manipulation should be done using the date filter.

can you explain exactly what you mean by «when trying to use the TimeGenerated as the @timestamp ?

I wanted to have the @timestamp field be set by the value in the "TimeGenerated" field, rather than auto-assigned by Logstash. I'm new to ELK, so maybe this isn't the recommended usage, but it made sense to me because I want to be able to follow a transaction through multiple logs, so I want the timestamp to be as accurate as possible. If I'm barking up the wrong tree here, please let me know.

normally config-level timestamp manipulation should be done using the date filter.

That's where I started, with the filter below, but since TimeGenerated was a Ruby Object, the filter didn't work. I don't have the exact error right now, but it was something along the lines of a ruby object not being able to be converted to a java string.
date {
	match => [ "TimeGenerated", "YYYY-MM-dd HH:mm:ss Z" ]
}

ah! perfect, this is what I wanted to know, good catch! let me verify this.
the eventlog filter should not set the TimeGenerated as an Ruby Time object but use its ISO8601 string representation. Event data should not contain anything else than strings and numerics and Timestamp objects.

@jsvd since you recently refactored that part, do you agree on my previous comment?

+1 on that being a ruby string, right now it is a Time object because the library does it automatically
if we make this change I suggest bumping the major version since it will break compatibility the current "schema" of the generated event.

FYI: Over in the logstash-input-jdbc project, they chose to convert timestamps to LogStash::Timestamp.

Hi thanks for the workaround but I somehow catch the following error
Ruby exception occurred: undefined method utc' for nil:NilClass {:level=>:error}`
My logs look like 2016-08-18 09:15:24,230 logmessage and I match them using the grok{} and date{} filters. Resulting in logdate="2016-08-18T09:15:24.230Z" having the same colour in the console output as @timestamp. So it is no string but Kibana also doesn't recognise it as a date but @timestamp is. So when applying the workaround event['logdate']=LogStash::Timestamp.new(event['logdate']) I get the error.
My questions are what am I doing wrong and how do I get my parsed logdate recognised as timestamp by Kibana without errors?
Thanks for your help!


update tests to run against bionic

fix storer to account for rsyslog changes in inputtcp

using new directives rather then old style seems to fix it

update to running tests against bionic(current cf-d standard)

Type of change

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

Testing performed?

[x] Unit tests
[x] Integration tests
[x] Acceptance tests

Checklist:

[x] This PR is being made against the main branch, or relevant version branch
[ ] I have made corresponding changes to the documentation
[x] I have added testing for my changes

If you have any questions, or want to get attention for a PR or issue please reach out on the #logging-and-metrics channel in the cloudfoundry slack

@Benjamintf1 where are you getting these new directives from?

@ctlong https://github.com/rsyslog/rsyslog/issues/4816#issuecomment-1154837044 found it here, some of the main docs are here. https://www.rsyslog.com/doc/v8-stable/configuration/modules/imtcp.html


Cann't find "Segment Mesh" after pressing f3

I imported a.stl file into Blender and activated the plugin according to usage, but I couldn't find "segment mesh" after pressing F3. I want to know the solution.

Thanks for bringing this to my attention. Apparently since Blender 2.9, operators do not show up in the menu search by default anymore if they are not part of a menu (see issue on Blender and stackoverflow question). As a workaround, you can make it findable again by enabling "Preferences > Interface > Developer Extras".
I will add this workaround to the usage section for now, but ultimately I should probably look into fixing this within the addon itself so that it works out of the box, so I will keep this issue open. There seems to have been interest to somewhat revert this in this Blender issue, but it looks abandoned.

The commit https://github.com/kugelrund/mesh_segmentation/commit/cbb82d33989b0e662ccaac6074e6e33f3c4e76e5 in @shankarsivarajan's fork looks related.


Camera Support

implement the functionality of https://github.com/KhaosT/homebridge-camera-ffmpeg ?

Does anyone want this? Would be kinda "out of scope" regarding MQTT, but running a Homebridge in parallel only for the cameras (as I do today) isn't a "nice" solution either...

I would definitely like the option down the road but it isn't a top priority for me at the moment.

I would suggest a MQTT to RTSP Bridge.
So the MQTT topic lists the current feed address.
It would be good to support MJPEG as well if possible.

Would be a really useful feature (even considering the out of scope nature). Running another Homebridge  instance is definitely not a good option.

Yes please , running another homebridge is painful at the moment

Also interested. Thank you.

+1

I think is a good idea to develop camera support
+1

Please add camera support!! Pretty please :P Currently I have to run another homebridge instance in order to get the homebridge-camera-ffmpeg plugin running.. and it's eating a lot cpu and ram usage

Just integrated the homebridge-camera-ffmpeg plugin. Happy to hear if it works for you.

@hobbyquaker Thanks a lot! I will check in during the weekend!

seems like ffmpeg is missing.
also i would suggest to create the config via the webui, the source config seems to be wrong, should start with -re or smth. see homebridge-camera-ffmpeg docs also.

You were right!
Now it is working!
Thanks again!

had to do a major change in v0.9.11, the cameras are now published as accessories on their own (listening on individual ports), publishing them as bridged accessories broke the still image function (that is needed to get a picture in motion sensor notifications from the same room). That means you will have to pair the cameras again and assign them to a room.

When I add accessory I see 2 categories - Camera and IP_Camera. What is difference between two? Also how do I configure motion detect notification ?

The Categories are the same, doesn't matter which one you choose. IP_CAMERA is included in hap-nodejs for compatibility reasons, I was just to lazy to remove it ;-)
To get motion notifications you just have to have a motion detector service with enabled notifications in the same room as the cam.

Understood! Thank you.


createRef API like React 16.3

React 16.3+ exposes a new ref API that's intended to be the default instead of the callback refs that both React and Roact have.
I've found that a lot of users new to the React model try to do things like call setState inside refs, or do all sorts of trickery inside of them, when they really should be doing those things in didMount instead.
Switching the recommended API to a createRef-like API would prevent these issues in the future.

This would work way better than function-based refs. I am skeptical about one of the details of this:

When the ref attribute is used on a custom class component, the ref object receives the mounted instance of the component as its current.

This doesn't feel right. It's nice to completely encapsulate component behavior, and this breaks the encapsulation. With this functionality, I can write code that only works when a component is stateful, and mess with its internals arbitrarily. This is not ideal - it feels really imperative, as opposed to the general declarative nature of Roact and React.

I didn't even notice that, that's pretty sketchy.
We can ignore that feature for now, and if we end up really (really really) wanting that, we can add it later.

I'm working on this locally; should have something in a few days.


fixing start amd end layers issue

fixing start and end layer issue

It's a good idea to open an issue first for discussion.

[ ] Tests pass
[ ] Appropriate changes to README are included in PR

@FaragElsayed2 , ready for review