Project idea discussion thread

This repository has been created as a place for discussion and future development of rich F# experience on top of Atom editor. Inspired a lot by OmniSharp team, I strongly believe it is possible to create very powerful, cross platform, and lightweight F# code editor giving extensibility of Atom editor.
The code already in repository is just a simple proof of concept for communication between Atom package and F# Compiler Services - current iteration is using fsautocomplete.exe tool. As for now it does nothing, running plugin ( ctrl + alt + o) just run set of commands against predefined F# files and prints output to console. Also current version is using CoffeScript which might not be the best choice.
Before starting serious implementation few things should be decided and i hope for some input.
Below are some topics and possible solutions to them ( for sure there are more ;)). Feel free to give Your opinion on all or any of them.

The architecture of system

One big plugin
Set of smaller composable packages

The language of choice

Plain JavaScript
Any Compile2JS languages ( CoffeeScript / TypeScript ?)
Using F# and FunScript to compile to JS

The way of comunication with F# Compiler Services

FSharp.Autocomplete
Building Suave based local WebService hosting F# Compiler Services
Using edge.js to call F# Compiler Services directly ( probably small bridge / adapter library necessary for best experiance)

Also would want to know what is most important, what should be scope of project at start, and what fancy features would be welcomed ( Maybe support for Paket and Fake from the editor level? It should be easy to add)

I'm guessing running the F# Interactive in Atom would be part of it, too?

I'd like to see smaller components. Could always do a package of packages for an install all sort of thing.
Using FunScript could allow us to dog food it which could be cool.
Agree with @jwoood803 that the interactive would be great

Why not add F# support to Omnisharp?

@dsyme, If anywhere I would prefer to merge it with our F# bindings project - they don't have Atom support at all, this idea is in the scope they are working on, and probably this project will be using some libraries created by them. It should be another discussion with guys maintaining F# binding if it would be worth to try merge whole F# bindings with OmniSharp and I don't feel I'm competent enough to start it right now.
@jwood803, For sure Interactive should be part of it
Personally, I would also go for set of smaller components. As @reidev275 suggested it should be easy to create installer for all parts.
FunScript was an idea suggested by@tpetricek in our twitter conversation. It seems great although I personally don't have a lot experience with it and would love to hear also others opinions.

I should have mentioned that F# + Atom would be fantastic to have - it would find all sorts of uses in practice.  For example we would likely use it in a future revised http://tryfsharp.org (to replace the HTML5 editor currently used there which came from an internal Microsoft project)

I have implemented basic Autocomplete. For sure it's just basic hacky implementation but it is good proof of concept. Still plugin is implemented using CoffeeScript - I don't have enough experience in FunScript. And here goes gif ;)

That's awesome! I'm liking how this looks already.

Well, It still has some problems:

Atom has idea of grammars ( definition of language syntax highlighting) and as for now https://github.com/jbtule/language-fsharp is used. As far as I know @tpetricek was investigating building dynamic grammar using F# Compiler Services
As for now it's working just for standalone .fs files or .fsx files with #load and #r directives. It doesn't support fsproj yet.
It's written in CoffeeScript ;) And personally I'm not a big fan of this language

The usual thing with fsharpbinding is when a new editor binding comes along we make one or two people associated with that editor binding administrators on the project.  That's what happened with Emacs for example.

@dsyme, Well seems good for me. If we manage to make here something useful I would be more than happy to merge it with fsharpbinding.
As for now I would just like to get thing rolling. Just waiting for some more input from community. And Tomas' opinions since he was interested, he has some cool ideas, he investigated some topics earlier, and conversation with him pushed me to start this project

I'd love to get involved in this :-). I think I should have some time on Sunday/Monday to have a look. I think doing the early experiments here is probably a good idea. If it turns into something useful, we can move it to fsharpbindings!

It may be that moving to fsharpbindings is not needed - when that was first started there weren't many F# open projects. The F# community is now much more adept at making and sustaining new projects.
However I'd recommend moving to a github org like fsprojects (or similar) sooner or later.

Heh, we have our amazing http://fhash.org also as a github org, we could put it there ;) [just kidding]
Again, don't see any problems with moving it to some F# github org.
I have added sample error / warning highlighting implementation.
This time just a picture.

Getting to know Atom API continues - this time bottom panel with errors / warnings.

I managed to create a very simple demo using FunScript: https://github.com/tpetricek/FunScript.Atom
It actually looks pretty nice! If we manage to generate FunScript bindings for the atom package from atom.d.ts, then it should be able to do all we need. The next question if we can get F# Data to work with FunScript again and use that to parse the JSON coming from the console app!
That said, as much as I'd love to use F# for writing the F# plugin, the current story really is that if you want to use it, you need to kind of know both F# and JavaScript (and probably especially with Atom). So, just writing everything using ordinary CoffeeScript might be easier way to get it done....

I have pushed proof of concept for autocomplete created using F# and FunScript. Still not all features from CoffeScript implementation are ported but it should be done Soon™
For me development was OK and I think that advantages of F# outshines possible drawbacks ( such as bit hacky and hard-coded code generation and some other FunScript problems). Also creating tooling in F# can again be selling point not only for this project but also for F# as whole ( I would for sure mention in every F# talk I give ;) ). And people most interested in development of F# support for Atom are F# developers - so it will make easier for people to contribute. So even if I can agree that implementing it in CoffeeScript may be easier I still think it's worth doing it in F#.
Tomas, we should definitely try to do something with atom.d.ts as it would make things even easier. What's exact problem there? When I tried to generate bindings I got no unhandled exceptions as in cases of previous errors I worked around. Error log file was also empty.

I have ported last part of CoffeScript implementation to F#. There are still few problems - declaration of view is as for now very naive, I still don't use atom bindings provided by Tomas in https://github.com/Krzysztof-Cieslak/FSharp.Atom/pull/2 - integrating it will be last step for this proof of concept.

I use Atom to edit F# files some of the time, it would be nice to see semantic formatting and completion available :-)

@Krzysztof-Cieslak @tpetricek IS this buildable on non Windows?

@Krzysztof-Cieslak
Re: Using edge.js to call F# Compiler Services directly ( probably small bridge / adapter library necessary for best experience)
You could probably use edge.fs too, an all F# async path could be funky :-)

@7sharp9 I did all the work on Mac with Xamarin :-).
I think doing the calls via the console (or small HTTP server) is fine. The only thing that's not asynchronous in Atom is colorization - and that we can probably solve later (though it would be nice to use the F# compiler for that too - even in Xamarin, the colorization is really easy to break).

@tpetricek The colourisation in XS is based on two parts, a regex parser that highlights the basic syntax, and a semantic formatter that is based on the AST being available to apply some further colourisation to members, values, fields, closures etc.
As for the Compiler doing all the work, I think this might be out of scope, I see the compiler providing the capability to get all the information to do the further colourisation, like what we do in the F# adding in XS.  This is mainly done via the FSharpSymbol API thats been steadily taking shape over the last year.  There are already Symbol version of most core compiler functions rather that the older text based results that you were forced to crack apart.

@tpetricek
Re: By "all the work" I mean the little thing I contributed. Obviously @Krzysztof-Cieslak did all the work! That explains why most of this code is so bad ;)
@7sharp9
Thanks for pointing out edge.js and giving some insight about XS. As for now plugin is using static colorization created by @jbtule -  https://github.com/jbtule/language-fsharp . Actually from point of view of autocomplete and error list I just check if Grammar for currently active TextEditor has name "F#", so it's not coupled to any colorization implementation.
So, my idea right now:

Already existing code is base for core plugin - it will cover such things as autocomplete, error highlighting, error list, tooltips - all those features are in scope of FSharp.Autocomplete and we can communicate with it using standard I/O
We should make some investigation about creating better syntax highlighter which will replace or complement static highlighting we are using right now.
Creating few smaller plugins for set of tools which are commonly used in F# community (such as Paket support, FAKE runner etc) and other useful features (refactoring, navigation  etc)

@Krzysztof-Cieslak Is this all still to be done using CoffeeScript?

@jwood803, I don't think CoffeScript is necessary, F# story is OK (especially after adding generated bindings for atom), only drawback for now is that it sometimes requires updating atom.d.ts file and regenerating bindings since original atom.d.ts is bit outdated.
However every plugin is separate entity and can be developed using different stack ( tho, i don't see any reason not to use F# as for now ;) )

That's awesome! I hope to help contribute in that case :boom:
That CoffeeScript was a little intimidating, to tell the truth. :p

@jwood803, Well, I've just started using it since that's what Atom community does - all examples, tutorials and documentation is using it. As i said before - I'm not big fan of CoffeeScript, if F# was not an option I would rather go to TypeScript or plain JavaScript. Fortunately ( mostly thanks to Tomas' work) we can use F# without many problems.

@7sharp9 I might be missing something - but I think the basic colorization should be pretty straightforward using the F# Compiler Service's tokenizer. I use this to colorize things in F# Formatting and it is pretty fast & correct. So I think the regex colorization could be replaced using this.
(And the same is the case for Atom, if we can figure out how to make synchronous calls, or if we can compile the F# tokenizer into JavasScript to embed it directly :-))

I use the tokenizer in XS too, the regex is fallback for when the AST or lexing is broken.  We use the symbols to allow colorisation of AST dependent stuff like computation expressions, field declaration, field usage etc etc.

On 25 Mar 2015, at 16:11, Tomas Petricek<EMAIL_ADDRESS>wrote:
@7sharp9 I might be missing something - but I think the basic colorization should be pretty straightforward using the F# Compiler Service's tokenizer. I use this to colorize things in F# Formatting and it is pretty fast & correct. So I think the regex colorization could be replaced using this.
(And the same is the case for Atom, if we can figure out how to make synchronous calls, or if we can compile the F# tokenizer into JavasScript to embed it directly :-))
—
Reply to this email directly or view it on GitHub.

Hmm that is odd, Xamarin never worked correctly for me with e.g. """ strings or nested comments - even when my code was all correct. The lexer should not need correct F# source though (and it certainly does not need AST that can be parsed).
Is this something that has changed recently? Perhaps I just need to install the latest version...

Oh I know the lexer doesn't need an AST, semantic highlighting does though.   Field usage, field declaration have different highlight colours.   When the AST is returned async, the semantic part is applied over the basic highlighting.
I would like to improve the resilience though.
5.9 should be out soon which has fixes in that area.

On 25 Mar 2015, at 17:13, Tomas Petricek<EMAIL_ADDRESS>wrote:
Hmm that is odd, Xamarin never worked correctly for me with e.g. """ strings or nested comments - even when my code was all correct. The lexer should not need correct F# source though (and it certainly does not need AST that can be parsed).
Is this something that has changed recently? Perhaps I just need to install the latest version...
—
Reply to this email directly or view it on GitHub.

@7sharp9, @tpetricek I won't even pretend I understand what you guys are talking about ;) Can we move discussion about syntax highlighting to separate issue?
I was also planning to create separate discussion threads for core plugin and Paket  plugin ( which very basic version I should push tomorrow so maybe someone interested in contributing will have easy starting  point)

@Krzysztof-Cieslak Yeah Ill shut up :-)  @tpetricek We can discuss this more at https://github.com/fsharp/FSharpMDXS/issues/6

@7sharp9, I don't see any problems with this discussion as such, just wanted to create separate discussions for different parts of plugin :).

@Krzysztof-Cieslak Cany you answer to this: https://github.com/fsharp/fsharpbinding/issues/990
I don't know what are the implications of moving

Great project! I would like to contribute, but I don't know where to begin. I have several ideas how I could help, but I'm not sure that my opinion on what should be done coincides with community. Do you mind if I formalize the ideas as issues so we can discuss them separately? Most of them are about methodology, i.e.:

First of all, every open source project needs documentation to help new contributors start development. I already created small page "How to setup development environment", but it should be more detailed, I think. It also would be great to describe how this plugin work, its main parts and what external components are used.
What about automated testing? Should contributors write unit-tests and what framework will be used for that?
Small notes about project organization:

Is it necessary to store .dll files in lib folder under version control? Should it be installed via NuGet instead?
The same thought could be said about fsautocomplete.exe and its dlls
Should plugin core be renamed to something less general?
Should  core.fs be broken to several files to make navigation easier?

...and so on

@mcgee, Those are all good points! As for now, everything seems bit chaotic but I'm planning to do some work on those "formal" things when decisions discussed in fsharp/fsharpbinding#990 will be made.
Just to answer some of questions:

Testing is bit tricky, I'm not sure how to handle it from FunScript point of view, while testing generated JS code also can be difficult (since FunScript is generating some really messy code) ( /cc: @alfonsogarciacaro)
DLL files from lib are nowhere on nuget - those are files generated by me ( and there is option to re-generate them in our FAKE build file)
For  fsautocomplete.exe we need to wait for changes in organization of projects discussed in fsharp/fsharpbinding#990.
core for sure should and will be renamed ;)
splitting core.fs into several files seems like a good idea.

I wasn't aware of this project and started a dynamic grammar based on https://github.com/TypeStrong/atom-typescript/blob/master/CONTRIBUTING.md#dynamic-grammar
I currently only have a package that sends lines to FSharp.Compiler.Service for tokenization. This still needs a bit of work but I'd be happy to contribute my efforts back here or something more official.
My intention was to use that as research for when I get access to Nuclide so that a lot of the work there can be leveraged.

Nuclide is a set of Atom packages being developed by Facebook.
My post requesting more info about: adding more languages: https://facebook.com/groups/833105363425649?view=permalink&id=853799101356275

Hey @paulyoung.
We would love to have dynamic grammar for syntax highlighting and will be very happy with any contributions. I know @tpetricek was interested mostly in this topic and also did some investigation on it.

@paulyoung what is the dynamic grammar, is it like semantic highlighting?

@7sharp9 instead of using the TextMate grammar format for syntax highlighting it sends lines to FSharp.Compiler.Service to tokenize lines.
My initial motivation was mostly because I hate the old plist format and find that it's a pain to keep in sync as languages change.
I'm using a similar approach with a language we've been developing at work (where syntax has been changing rapidly) and wondered if doing the same for FSharp would be fast enough and open doors to using other features like the editor services.

@paulyoung This would be amazing! The current grammar is flaky in various places and I think actually calling the compiler service is the only correct way of fixing that.
I asked about how to do this at the Atom forums some time ago:

I got a couple of useful pointers in response
And there is another related thread I found

The tricky thing seems to be that Atom does this synchronously - but AFAIK, we only have asynchronous ways for calling the F# Compiler service (but perhaps there is some way to hack that?). Another option would be to get FunScript or WebSharper to compile the lexer part of F# Compiler Service into JavaScript (which would be doable, but might be non-trivial amount of work...)

Here's my question and findings.
I think support for async tokenization is being tracked here: https://github.com/atom/first-mate/issues/34
Includes interest from someone at OmniSharp, among others.

get FunScript or WebSharper to compile the lexer part of F# Compiler Service into JavaScript

This might be more performant and easier to bundle.

AFAIK the tokenization in FCS is synchronous, only retrieving symbols to assist would be async.

i.e
let tokens =
                    [| let state = ref 0L
                      for line in alllines do
                        let lineTxt = editor.GetLineText(line)
                        let tokenizer = sourceTok.CreateLineTokenizer(lineTxt)
                        let rec parseLine() =
                            [ match tokenizer.ScanToken(!state) with
                              | Some(tok), nstate ->
                                  yield tok
                                  state := nstate
                                  yield! parseLine()
                              | None, nstate -> state := nstate ]
                        yield line, parseLine() |]

It can be time consuming to call though as the state of the previous line is required, otherwise information such as an active #if will be lost etc

It's definitely looking like it's going to be too slow from what I could see last night.

Tokenisation only, without any async symbol retrial should be around 30-50ms for normal sized files.

Admittedly my focus was on mapping tokens to what Atom expects. I had hoped that some optimization could come later.

Discussions about project to be continued in #96


Makes sure that the property of a style is  not an url

Hello,
just had this issue that made me sweat for a while today.
Please let me know if I need to do anything else to get this PR merged.
Commit msg
I actually got a bug with background-image property
pointing to cloudfront with this url:
https://d3r0d0kb2vhuxd.cloudfront.net
The viewportUnitExpression regex matched the url
and the url got rewritten.
This is a fix to avoid that.

interesting. thanks for figuring this out!
if you could rename url to urlExpression (to fit with the others) and rename the commit message to fix(polyfill): do not consider URLs viable values for running the polyfill.

ping! not sure why there's a conflict now.

the file was reindented. I resolved the conflict usign the web editor (did not know this github feature existed...) thx!


tests: test upgrade from luminous to luminous

since<EMAIL_ADDRESS>isn't able to deploy releases prior to
luminous, we can't really test upgrade from jewel to luminous at the
moment.
Waiting for a solution, let's at least test a rolling upgrade from
luminous to luminous (at least to make the job passing the ci...)
Signed-off-by: Guillaume Abrioux<EMAIL_ADDRESS>

jenkins test luminous-centos-non_container-update

jenkins test luminous-centos-container-update

thanks to #3685 I see there's a solution.


moving site and using Nginx - session error

I am moving a site from a server that was using apache, mysql and php 5.x.  Seems that a server update broke the system, and then admin login would keep coming up even after a correct password.  But I am moving all sites off that server to VMs so this was an opportune time.
The move is to Nginx, Mariadb, php-fpm, and php 7.1.
I have all http redirected to https.
I have defined sumaserver and suma as two sites, one server and one client.  The document roots are
/var/www/html/suma2/sumaserver/
and
/var/www/html/suma2/client
won't do analytics site until these work.
If I remove sumaserver from document root the /sumaserver/ in the URL does not work.  But removing client from the document root works using /client/ although the page is not the one we created but a default client page.
the paths are symbolic links to /var/www/app/suma/service/web and /var/www/app/suma/web respectively.  I have edited the config.yaml and spaceassessConfig.js files to match the document root path.  I have edited Globals.php and BaseController.php since /sumaserver/ is not in the path because of the document root, but that didn't help but I left them edited anyway.
Using Nginx .htaccess files are not used so the rewrite rule is rewritten to the ssl.conf file.

Please refer to the Suma documentation for more information: http://cazzerson.github.io/Suma/
By the way, that github link goes to a 404 page
I go to https://sumaserver.library.appstate.edu/admin/login I get:
500 Application error
Zend_Session_Exception: Zend_Session::start() - /var/www/app/suma/service/lib/zend/library/Zend/Session.php(Line:482): Error #2 session_start(): Failed to read session data: files (path: /var/opt/remi/php71/lib/php/session) in /var/www/app/suma/service/lib/zend/library/Zend/Session.php:495
Stack trace:
#0 /var/www/app/suma/service/lib/zend/library/Zend/Session/Namespace.php(143): Zend_Session::start(true)
#1 /var/www/app/suma/service/lib/zend/library/Zend/Auth/Storage/Session.php(86): Zend_Session_Namespace->__construct('Zend_Auth')
#2 /var/www/app/suma/service/lib/zend/library/Zend/Auth.php(91): Zend_Auth_Storage_Session->__construct()
#3 /var/www/app/suma/service/lib/zend/library/Zend/Auth.php(141): Zend_Auth->getStorage()
#4 /var/www/app/suma/service/controllers/AdminController.php(414): Zend_Auth->hasIdentity()
#5 /var/www/app/suma/service/lib/zend/library/Zend/Controller/Action.php(502): AdminController->preDispatch()
#6 /var/www/app/suma/service/lib/zend/library/Zend/Controller/Dispatcher/Standard.php(308): Zend_Controller_Action->dispatch('loginAction')
#7 /var/www/app/suma/service/lib/zend/library/Zend/Controller/Front.php(954): Zend_Controller_Dispatcher_Standard->dispatch(Object(Zend_Controller_Request_Http), Object(Zend_Controller_Response_Http))
#8 /var/www/app/suma/service/web/index.php(96): Zend_Controller_Front->dispatch()
#9 {main}->getMessage()
Zend_Session_Exception: Zend_Session::start() - /var/www/app/suma/service/lib/zend/library/Zend/Session.php(Line:482): Error #2 session_start(): Failed to read session data: files (path: /var/opt/remi/php71/lib/php/session) in /var/www/app/suma/service/lib/zend/library/Zend/Session.php:495
Stack trace:
#0 /var/www/app/suma/service/lib/zend/library/Zend/Session/Namespace.php(143): Zend_Session::start(true)
#1 /var/www/app/suma/service/lib/zend/library/Zend/Auth/Storage/Session.php(86): Zend_Session_Namespace->__construct('Zend_Auth')
#2 /var/www/app/suma/service/lib/zend/library/Zend/Auth.php(91): Zend_Auth_Storage_Session->__construct()
#3 /var/www/app/suma/service/lib/zend/library/Zend/Auth.php(141): Zend_Auth->getStorage()
#4 /var/www/app/suma/service/controllers/AdminController.php(414): Zend_Auth->hasIdentity()
#5 /var/www/app/suma/service/lib/zend/library/Zend/Controller/Action.php(502): AdminController->preDispatch()
#6 /var/www/app/suma/service/lib/zend/library/Zend/Controller/Dispatcher/Standard.php(308): Zend_Controller_Action->dispatch('loginAction')
#7 /var/www/app/suma/service/lib/zend/library/Zend/Controller/Front.php(954): Zend_Controller_Dispatcher_Standard->dispatch(Object(Zend_Controller_Request_Http), Object(Zend_Controller_Response_Http))
#8 /var/www/app/suma/service/web/index.php(96): Zend_Controller_Front->dispatch()
#9 {main}->getTraceAsString()
I suspect it may be a path issue maybe with where to write the session data.  PHP71 from the Remi Repository is installed in /var/opt/remi/php71 path.
I've checked that path and there are not files in the session directory to read and there is no error about not being able to write the files.    My server is in a vlan not accessible from off campus, sorry.
My Nginx ssl.conf file is below if anyone wants to try it.
Any help would be appreciated,
Thomas Bennett
Systems & Operations Analyst
Appalachian State University Library
ssl.conf (nginx.conf uses an include for the directory that ssl.conf is in):
SUMASERVER
server {
    listen       443 ssl http2 ;
    server_name  sumaserver.library.appstate.edu;
    root         /var/www/html/suma2/sumaserver/;
    access_log      /var/log/nginx/sumaserver_access.log;
    error_log       /var/log/nginx/sumaserver_error.log debug;
    # rewrite logs to error log at notice or higher
    rewrite_log on;

    ssl_certificate "/etc/pki/tls/certs/star.library.appstate.edu.pem";
    ssl_certificate_key "/etc/pki/tls/private/star.library.appstate.edu.key";
    ssl_session_cache shared:SSL:1m;
    ssl_session_timeout  10m;
    ssl_ciphers HIGH:!aNULL:!MD5;
    ssl_prefer_server_ciphers on;
    ssl_dhparam /etc/pki/tls/certs/dh2048.pem;

    index   index.html index.php;

    location / {
            autoindex on;
            disable_symlinks off;
	# Literal translation of .htaccess file
	# see http://www.anilcetin.com/convert-apache-htaccess-to-nginx/
            if (-e $request_filename){
                    set $rule_0 1;
            }
            if ($request_filename ~ "-l"){
                    set $rule_0 1;
            }
            if (-d $request_filename){
                    set $rule_0 1;
            }
            if ($rule_0 = "1"){

            #ignored: "-" thing used or unknown variable in regex/rew
            }
            rewrite ^/.*$ /index.php last;

    }

    location = /favicon.ico {
            log_not_found off;
            access_log off;
    }

    location = /robots.txt {
            allow all;
            log_not_found off;
            access_log off;
    }

    # PHP-FPM PARAMETERS
    #NOTE: You should have "cgi.fix_pathinfo = 0;" in php.ini
    location ~ \.php$ {
            try_files $uri =404;
            fastcgi_split_path_info ^(.+\.php)(/.+)$;
            fastcgi_pass unix:/var/run/php-fpm.sock;
            fastcgi_param SCRIPT_FILENAME  $document_root$fastcgi_script_name;
            fastcgi_param SCRIPT_NAME $fastcgi_script_name;
            fastcgi_index index.php;
            include fastcgi_params;
            fastcgi_intercept_errors on;
            fastcgi_read_timeout    256s;
    }
}

SUMA Client
server {
    listen       443 ssl http2 ;
    server_name  suma.library.appstate.edu;
    root         /var/www/html/suma2/client/;
    access_log      /var/log/nginx/suma_access.log;
    error_log       /var/log/nginx/suma_error.log;
    # rewrite logs to error_log at notice or higher
    rewrite_log on;

    ssl_certificate "/etc/pki/tls/certs/star.library.appstate.edu.pem";
    ssl_certificate_key "/etc/pki/tls/private/star.library.appstate.edu.key";
    ssl_session_cache shared:SSL:1m;
    ssl_session_timeout  10m;
    ssl_ciphers HIGH:!aNULL:!MD5;
    ssl_prefer_server_ciphers on;
    ssl_dhparam /etc/pki/tls/certs/dh2048.pem;

    index   index.html index.php;

    location / {
	# should turn off autoindex in production
            autoindex on;
            disable_symlinks off;
    }

# Don’t log missing ico file
    location = /favicon.ico {
            log_not_found off;
            access_log off;
    }

    location = /robots.txt {
            allow all;
            log_not_found off;
            access_log off;
    }

    # PHP-FPM PARAMETERS
    # pass the PHP scripts to FastCGI server listening on socket
    #NOTE: You should have "cgi.fix_pathinfo = 0;" in php.ini
    location ~ \.php$ {
            try_files $uri =404;
            fastcgi_split_path_info ^(.+\.php)(/.+)$;
            fastcgi_pass unix:/var/run/php-fpm.sock;
            fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
            fastcgi_param SCRIPT_NAME $fastcgi_script_name;
            fastcgi_index index.php;
            include fastcgi_params;
            fastcgi_intercept_errors on;
            fastcgi_read_timeout    256s;
    }

# Don’t log missing image files in web log
   location ~* \.(js|css|png|jpg|jpeg|gif|ico)$ {
           expires max;
           log_not_found off;
   }

    autoindex      on;
}

/var/opt/remi/php71/lib/php/session was owned by root:apache.  I changed that to root:nginx and now get a login page.  YEAH! ;-)  Now to see if I can log in.  I'm still not getting the correct client page, looks like a default page.

Logged in as admin but the page is a list of tables each giving a similar error for Global.php and BaseController.php.  Had to return to the original version of those files ;-).

Under Suma Administrative Tools, the drop down appears to be populating but clicking on Display doesn't show anything.
Client still does bring up the page we should have, still looks like a default page and there doesn't seem to be any errors logging to indicate why.  Nothing under select an initiative  dropdown.

Hi Thomas,
I'm not terribly familiar with Nginx, but this sounds like an issue with
rewrite rules. Maybe something like this will help?
https://www.nginx.com/blog/converting-apache-to-nginx-rewrite-rules/
Bret
On Thu, Aug 10, 2017 at 3:59 PM, Thomas Bennett<EMAIL_ADDRESS>wrote:

Under Suma Administrative Tools, the drop down appears to be populating
but clicking on Display doesn't show anything.
Client still does bring up the page we should have, still looks like a
default page and there doesn't seem to be any errors logging to indicate
why. Nothing under select an initiative dropdown.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/suma-project/Suma/issues/69#issuecomment-321657384,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AANiVVyAKGhSIKak7fX0N1FqiFJrObsHks5sW2EYgaJpZM4OzvMQ
.

Thanks  for the suggestion, thats actually one of the first URLs I found for translating rewrite rules.  At the bottom of that post is almost the apache rewrite rules suma uses but no one replied to that guy on line on how to rewrite it.  The best I found was the converter at http://www.anilcetin.com/convert-apache-htaccess-to-nginx/ which gave me the one I used but I still don't understand why it doesn't work when I set document root to where it should be and /sumaserver/ does not follow the symlink even though I turned disable symlinks off.  I am going to look again at the if rules in the rewrite and to see exactly what they mean, could be I just assume they were the same as the .htaccess rules.
Thanks again
Thomas

Well, I have installed apache but still using PHP71.  I had to rename the session.yaml so it would not be used so I could get admin logged in.  I don't know if something has changed in the call to start session except that it seems now start session can accept parameters to override php.ini settings.


replace deprecated docker-compose with docker compose

Due to Docker’s official announcement that the docker-compose command will be deprecated as a standalone tool and will no longer be maintained or updated in the future, Docker recommends using the new docker compose command.
This new command is part of the Docker CLI and is designed to integrate the functionality of docker-compose for better support and consistency.

When running the yarn script command in a project initialized with subql init, you may encounter an error indicating that the docker-compose command does not exist.


KML import error after exporting/importing as query

The process described below it's a bit messy, but it's happening regularly when users play and filter some KMLs

Create an empty dataset, add a row with coordinates and random data
Export it as KML
Reimport it
Perform a query:
`SELECT *, 1 from tablename``
Export dataset from query as KML
Reimport the KML (or find it here).
Export
Reimport (find it here

2015-05-19 09:42:02 UTC: ogr2ogr output: Warning 1: Field 'timestamp' already exists. Renaming it as 'timestamp3'
Warning 1: Field 'begin' already exists. Renaming it as 'begin3'
Warning 1: Field 'altitudemode' already exists. Renaming it as 'altitudemode3'
Warning 1: Field 'tessellate' already exists. Renaming it as 'tessellate3'
Warning 1: Field 'extrude' already exists. Renaming it as 'extrude3'
Warning 1: Field 'visibility' already exists. Renaming it as 'visibility3'
 Warning 1: Field 'draworder' already exists. Renaming it as 'draworder3'
Warning 1: Field 'icon' already exists. Renaming it as 'icon3' 2015-05-19 09:42:02 UTC: ogr2ogr exit code: 0
2015-05-19 09:42:02 UTC: Creating the_geom from wkb_geometry column
2015-05-19 09:42:02 UTC: replace empty strings by nulls?
2015-05-19 09:42:02 UTC: no string column found, nothing replaced
2015-05-19 09:42:02 UTC: geometrifying
2015-05-19 09:42:02 UTC: casting wkb_geometry to geometry
2015-05-19 09:42:02 UTC: Converting to 2D point
2015-05-19 09:42:02 UTC: Total processed: 1
2015-05-19 09:42:02 UTC: Total processed: 1
2015-05-19 09:42:02 UTC: FINISHED: Converting to 2D point
2015-05-19 09:42:02 UTC: geometrified
2015-05-19 09:42:02 UTC: Renaming column wkb_geometry TO the_geom
2015-05-19 09:42:02 UTC: Enabling autovacuum for "cdb_importer"."importer_1e0930d6fe0b11e486aa0e8dde98a187"
2015-05-19 09:42:02 UTC: Georeferenced
2015-05-19 09:42:02 UTC: Finished importing data from /tmp/imports/20150519-15185-gmpg5m/untitled_table_339.kml
2015-05-19 09:42:02 UTC: Proceeding to register
2015-05-19 09:42:02 UTC: Before renaming from importer_1e0930d6fe0b11e486aa0e8dde98a187 to untitled_table_339
2015-05-19 09:42:02 UTC: Before moving schema 'untitled_table_340' from cdb_importer to iriberri
2015-05-19 09:42:02 UTC: Before persisting metadata 'untitled_table_340' data_import_id: bda4474f-dde5-4027-a1b3-b551ba8b4841
2015-05-19 09:42:05 UTC: Exception: PG::Error: ERROR: column "_end_1" of relation "untitled_table_340" already exists

Basically, it seems that KML columns get duplicated: 	begin2, timestamp2...

According to GDAL, this behavior will not be corrected.
Link: https://trac.osgeo.org/gdal/ticket/5212


allow fetch to receive a ref option

It would be 💯 if we could allow fetch to receive a ref option
One example of why it's useful:
When interacting with GitHub pull requests, you need to specify a ref:
git fetch origin pull/20/head
This fetches a branch associated with a PR

It's been a busy few months, and I've forked the repo so we can continue moving forward. I'll try to update this soon

Merged pr


Bump chef-zero dep to 4.0

:+1:
/cc @chef/client-core

Hey @jkeiser - what were the changes for 0.4.0?  Is there somewhere I can go read them?

:+1:


Docs website is empty

The readthedocs website for discord-components  is empty.
It is supposed to show the docs for discord-components. In bing, it still shows what the site used to show, as in the below screenshot

When going to the website, clicking on "Edit on github" tries to go to https://github.com/kiki7000/discord.py-components/blob/master/index.rst but gives a 404 error.
Edit:
I looked in the commit history and found that it was commit f0767071e0d1d409245c76cbbdd4b8f681a94a40 that removed the docs folder.

You can use this for now


some questions about the code

Hi, what is the meaning of Underlined code in the picture?

Hi,
These are learnable weights which represent importance of the corresponding branch.

Hi,
These are learnable weights which represent importance of the corresponding branch.

Thanks! I have another question about code. I find that you use deform_conv2d to achieve the CycleFC. we use deform_conv2d API,  value in offset must be int? or the type of value in offset can be Float?

Currently, we implement the CycleFC by hacking deform_conv2d, i.e., fixing the offset pattern in our desired format.
As introduced in our paper, the offset should always be int value, which is memory-efficient. However, the offsets of the original deform conv are float by default.
Furthermore, we have a plan to implement CycleFC  with an integrated operator through CUDA instead of haking deforable_conv.
This kernel optimization for Cycle FC has a great potential for faster speed.

Hi, I have a new question about the code. As shown in the picture, If I delete the Underlined code, the code will raise error which is shown in the second picture, what is the reason? if I do not use register_buffer API, How can I achieve the deform_conv2d?

Using register_buffer , the offset can be moved to gpu when calling model.cuda().
If you don't use it, the offset is on the cpu.

Using register_buffer , the offset can be moved to gpu when calling model.cuda().
If you don't use it, the offset is on the cpu.
Ref: https://discuss.pytorch.org/t/why-model-to-device-wouldnt-put-tensors-on-a-custom-layer-to-the-same-device/17964/8?u=alpha

Thanks! problem is solved!

Closed as no response for a long time.
Feel free to reopen or open a new issue if you have further questions.


Or 5 of any sapling sin a pyre = 5 dead bush


Adding error tracking for untracked local files, and being on the wrong branch

Pull Request instructions:

Please follow the update protocol
Answer the questions below in detail. Your responses will be emailed to experimenters.
If the experimenters must do anything new, provide detailed step by step instructions on the wiki
Use markdown syntax in order for your comments to be rendered reliably in the email: "1." instead of "1)", use four spaces for indents.
If you use the keyword "skip email" in the title, it will skip the email updates
Merges from "develop" into "production_testing" should use the keyword "production merge" in the title for reliable indexing of updates
Merges from "production_testing" into "main" should use the keyword "update main"

Describe changes:

Right now, if a user has untracked local changes to the repository, or is running the GUI off the main branch we do not alert the user, or log that information as an error.
Now we add a note to the log on startup if there are untracked changes
We log an error if the python version is not 3.9
If a user starts a session AND the mouse is not "0", then:

we check for branch and untracked local changes.
If the branch is not main or production testing, then the user is prompted. If they continue, an error is logged.
If the branch has untracked local changes, then the user is prompted. If they continue, an error is logged.

What issues or discussions does this update address?

resolves #383

Describe the expected change in behavior from the perspective of the experimenter

If you are running the code off pipeline, you will be alerted.
If you are running the code with untracked changes to the code, you will be alerted

Describe the outcome of testing this update on a rig in 447

[ ] tested in 447

@XX-Yin @hanhou This is ready for review

Do you why is Foraging.py recognized as a dirty file even though I haven't modified it?

A weird case happened on my side. If the foraging.bonsai or foraging.bonsai.layout is changed, the Foraging.py file will be labeled as dirty even no changes. It's normal if we change other python files.


Repost: targetUpdatedProperties do not reflect old and new values

Hi,
do you have any updates when the Updated Properties will be correct?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: b651a95f-7c34-a979-f97e-e4feedb47487
Version Independent ID: ff679d7a-0bc5-692f-8ef8-72e24d402f23
Content: Interpret the Azure Active Directory audit log schema in Azure Monitor
Content Source: articles/active-directory/reports-monitoring/reference-azure-monitor-audit-log-schema.md
Service: active-directory
Sub-service: report-monitor
GitHub Login: @MarkusVi
Microsoft Alias: markvi

@crmhh Thanks for your feedback! We will investigate and update as appropriate.

@MarkusVi do you have any idea on this ?

The article is under investigation.
#please-close

Also seeing it still happening so I cant see exactly when a company details was changed, production impact across SharePoint and OneDrive.
Activity
Set company information
ModifiedProperties
[
{
"Name": "Included Updated Properties",
"NewValue": "",
"OldValue": ""
}
]


Add type annotations

Feature request
I would like to use type annotations with numba (no advanced features intended, such as reading annotations runtime: just plain annotations for mypy).
Are there any plans for working on that? If not, is it OK to submit PRs for this feature?

To clarify, are you wanting to add type annotations to numba's code base for mypy to work?

For exported items such as @numba.njit or TypedDict for now (while annotating internal members may help numba developer...)

Yes, it makes sense to add type annotations to the public facing APIs. But be warned, I think the Numba decorators are a bit complicated to add annotations just because they can be used in multiple ways.
If you would like to contribute, it is best to keep the PR small so it is easier for maintainers to review.


new year new formatting!

Okay, back at it again with another formatting discussion. I closed #74 since it was causing several merge conflicts and we haven't formally agreed on what we want to use as a formatting standard.
Reading around, I've formed a tentative opinion that the black formatter is becoming a 'standard' and I've made some attempts to test it out. If you're using Visual Studio Code, you can apply the black formatter to a file using [alt + shift + F] (probably similar on windows?). And, I was able to set up a git pre-commit hook that applies the black formatter to your files upon each commit, letting you review the results.
What are your thoughts?

Included in #102


Validation of non required fields

When validating a form, a non required empty field gets a validation error.
The schema for that field that is passed to the validation function is reduced to the part of the corresponding property.
There is no way the validator could know if this field is required or not.
As far as I have seen in the code of the ZSchemaValidatorFactory the validation errors of required fields are set up by the surrounding parent in the function denormalizeRequiredPropertyPaths.
So when you have a field which is optional but has to have a minLength if given,
this can't be validated right at the moment.
Regarding to this discussion at Z-Schema https://github.com/zaggino/z-schema/issues/49
the validation of non required empty fields can be skipped by extending the property type with "null" and then passing a null value instead of an empty string.
e.g.
  "name":{
        "type":["string", "null"]
   }

so this could be defined for all non required fields and let keep all minLength , pattern etc. validation rules.

Hello
I ran into the same issue, IMHO an optional property should not be validated against any constraint, the problem should be fixed at the validator. That said, to workaround the ZSchema issue, we can avoid validation process on non required field that have no values. Then the question is what is an empty field in terms of Form.
Schema validation should be skipped in some condition in the following piece of code :
extract from 'fromproperty.ts'
`  public _runValidation(): any {
let errors = this.schemaValidator(this._value) || [];
let customValidator = this.validatorRegistry.get(this.path);
if (customValidator) {
let customErrors = customValidator(this.value, this, this.findRoot());
errors = this.mergeErrors(errors, customErrors);
}
if (errors.length === 0) {
errors = null;
}
this._errors = errors;
this.setErrors(this._errors);

}
`
I guess I'll have to provide more information to the formproperty like a 'required' boolean to implement validation skip mechanism.
I'll give it a try.

Just did a simple test

In form property factory, check if field is required by analyzing 'required' property of parent schema, and add required property in formproperty class constructor

let required = parent != null && parent.schema['required'] && parent.schema['required'].indexOf(propertyId) != -1;

In formproperty _runValidation, skip validation when formproperty is not required and has no value

// Skip validation for optional fields that have no value
if (!this._required && !this._hasValue()) {
return;
}

And it seems to work as expected. Not need to add "null" type in schema.
Any opinion from makinacorpus team ?

have example for "Validation handled by z-schema "? @daniele-pecora  thanks.

@JianhuisHuang
We worked around this by using a custom validator factory, which does take in count if an optional field is empty.
Herenis how we use it:
https://github.com/daniele-pecora/ngx-schema-form-widgets-material/blob/master/projects/ngx-schema-form-widgets-material/src/lib/fix-optional-empty-fields-z-schema-validator-factory.ts


Add solidproject.org development and maintenance task force

Adding the development and maintenance of the solidproject.org website as a ongoing task force of the Team.
I am proposing @csarven  and myself as co-leads of this task force, but I also want to invite other team members who might be interested to add their name as a commit suggestion.
That aside, anyone can contribute to the website at any time without being formally a task force member.

What is the relationship between this task force and the Creators?
I suggest making it clear that this task force applies to the content of the website, not to the website itself, which is maintained by the Admins,

What is the relationship between this task force and the Creators?

Good question. I think we can have a discussion as to whether roles are needed still now that we're working on tasks which might have a mix of different roles.
I'll merge this as is as we can make improvements later on after that discussion.


After firmware update application can't detect peripherals

Hello, I'm using this library for updating firmware to my device. Update process goes well, problem occurs when it finishes application can't detect Bluetooth peripherals. Do you have any suggestions ?

I have had the same issue.
My issue is that the device connects, but the phone can't see the device.
Currently I have found a workaround, either restart your phone or in settings turn bluetooth off and on again. (Does not work if not done from settings)

I'm restarting application, only that helps me.

Hello,
Sorry, I'm on a leave until mid January. I'll look into that when I'm back at the office.

Try making your device send a Service Changed indication after updating its firmware. It may help.

Hi,
Did you manage to solve the problem? Can we close this issue?

Closing due to inactivity. Please, reopen if needed.


Update/Add new SmilingWolf interrogator models.

SmilingWolf Just released new interrogator models with updated datasets, can you add them to BooruDatasetTagManager interrogator?
https://huggingface.co/SmilingWolf/wd-vit-tagger-v3
https://huggingface.co/SmilingWolf/wd-swinv2-tagger-v3
https://huggingface.co/SmilingWolf/wd-convnext-tagger-v3

Checking your commit I guess that now the v2 of the swin tagger is broken.

Checking your commit I guess that now the v2 of the swin tagger is broken.

Have you updated the dependencies from requirements.txt? Post the log with the error, since I don’t have a similar problem.

I did, that's strange

Oh, I didn't see the part where you said "post the log with the error". Well there's no error, just at the moment of tagging the tagger process is killed when using swinv2, tried redownloading but is still the same

Looks like I found another dependent package, the wrong version of which was causing the crash. Try reinstalling the dependencies from requirements.txt again with a new commit.

but onnxruntime is the runtime for CPU only, does work but iy defaults to CPU

but onnxruntime is the runtime for CPU only, does work but iy defaults to CPU, also it's not needed with onnxruntime-gpu

Initially, I thought so too, but an experiment on another computer showed the opposite. The following packages were installed on the computer:
onnx==1.15.0
onnxruntime==1.15.0
onnxruntime-gpu=1.17.1
After trying to start generating tags, interrogator_rpc simply completed without any errors. Then I ran the command pip install --upgrade onnxruntime which updated onnxruntime to version 1.17.1 and tag generation worked without any problems.

It works but installing onnxruntime will override the providers. I used this piece of code:
import onnxruntime as ort
print(ort.get_available_providers())

Output was ["AzureExecutionProvider", "CPUExecutionProvider"]. After uninstalling onnxruntime I ran the program again and the output now was ["CUDAExecutionProvider", "CPUExecutionProvider"]

It works but installing onnxruntime will override the providers. I used this piece of code:
import onnxruntime as ort
print(ort.get_available_providers())

Output was ["AzureExecutionProvider", "CPUExecutionProvider"]. After uninstalling onnxruntime I ran the program again and the output now was ["CUDAExecutionProvider", "CPUExecutionProvider"]

Yes, unfortunately this is a problem that I don’t know how to solve. The onnxruntime update was needed to support models with ONNX ML opset version 4. Onnxruntime 1.17.1 works with both version 3 and version 4, but onnxruntime-gpu 1.17.1 crashes when trying to use models with version 3. Perhaps this is a bug in onnxruntime-gpu itself. I couldn't find any WD Tagger implementations that support the new models. If you know repositories with working code, please write links to them.

in my case onxxruntime-gpu only crashes when using swin v2 tagger, others seems to work fine

I would do it in other way but i guess that will work

I would do it in other way but i guess that will work

It seems to me that this is a bug in onnxruntime-gpu, which is why I made this exception. I hope this will be fixed in future releases and I will remove this exception.

Yea, I understand that and its a good idea. What I was thinking of, its since v2 taggers can be ran using onnx or keras you could use keras but it isn't worth for just one tagger, I too hope a future release of onnxruntime-gpu fixes it!


TS types: Array member can be null on ModelConnection when it should not (bookings: [Booking!] )

Before opening, please confirm:

[X] I have installed the latest version of the Amplify CLI (see above), and confirmed that the issue still persists.
[X] I have searched for duplicate or closed issues.
[X] I have read the guide for submitting bug reports.
[X] I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.

How did you install the Amplify CLI?
npm i -g @aws-amplify/cli
If applicable, what version of Node.js are you using?
v14.17.6
Amplify CLI Version
7.6.12
What operating system are you using?
Pop!_OS 21.10
Amplify Codegen Command
codegen
Describe the bug
I'm not 100% sure, but this may have started to happen after upgrading CLI to 7.6.11, I also just tried 7.6.12. Not sure what was the version last time before this I made schema changes.
In my schema I have:
type Company @model(subscriptions: { level: off }) @auth(rules: [{allow: public, operations: [read]}, {allow: groups, groupsField: "groups"}, {allow: private} ]) {
  ...
  bookings: [Booking!] @hasMany(indexName: "byCompany", fields: ["id"])
}

Which on amplify codegen --maxDepth 4 generates TS types:
export type Company = {
  ...
  bookings?: ModelBookingConnection | null,
};

export type ModelBookingConnection = {
  ...
  items:  Array<Booking | null >,

};

I don't think I have changes anything in the schema, it just started to generate that null yesterday.
Expected behavior
Member of the array should not be null, as I have bookings: [Booking!] in the schema.
}

Log output

# Put your logs below this line
2022-01-24T07:26:23.638Z|info : amplify codegen codegen  {"maxDepth":"4","yes":false}
2022-01-24T07:26:23.674Z|info : amplify-cli-core.banner-message/index.ts.fetch banner messages from https://aws-amplify.github.io/amplify-cli/banner-message.json({}

Additional information
No response

Hey @parvusville :wave: I was also able to confirm the output of the compilation is generating the ModelBookingConnection type without the required field as shown in @phani-srikar 's comment above. Unfortunately there does not appear to be a workaround; marking as a bug 🙂

create a GraphQL API
use the following schematype Company @model {
  id: ID!
  bookings: [Booking!]! @hasMany(indexName: "byCompany", fields: ["id"])
}

type Booking @model {
  id: ID!
  companyID: ID! @index(name: "byCompany")
}

run amplify api gql-compile
observe malformed output typetype ModelBookingConnection {
  items: [Booking]!
  nextToken: String
}

This looks like behavior carried over from the v1 Transformer (I was able to reproduce the issue there as well).

So, this has nothing to do with whether nullability is specified or not in the input schema. The connection type is generated for the list operation of the @model.
The code change to make the list entries non-nullable seems pretty straightforward:
diff --git a/packages/amplify-graphql-model-transformer/src/graphql-types/query.ts b/packages/amplify-graphql-model-transformer/src/graphql-types/query.ts
index b36b1ba17..ae5b8b779 100644
--- a/packages/amplify-graphql-model-transformer/src/graphql-types/query.ts
+++ b/packages/amplify-graphql-model-transformer/src/graphql-types/query.ts
@@ -13,7 +13,7 @@ export const makeListQueryFilterInput = (
 export const makeListQueryModel = (type: ObjectTypeDefinitionNode, modelName: string, isSyncEnabled: boolean): ObjectTypeDefinitionNode => {
   const outputType = ObjectDefinitionWrapper.create(modelName);

-  outputType.addField(FieldWrapper.create('items', type.name.value, true, true));
+  outputType.addField(FieldWrapper.create('items', type.name.value, false, true));
   outputType.addField(FieldWrapper.create('nextToken', 'String', true, false));

   if (isSyncEnabled) {

My biggest concern would be breaking existing code, but I'm not sure that the list operations ever send back null entries anyway.
However, if you would later change your schema to bookings: [Booking], the generated types would not start allowing nulls (and like I said, I'm not sure if we ever return those in lists anyway).
@yuth @lazpavel do you have any thoughts here?

Based on this PR description, having nullable elements is intentional


question: should authorize forcibly close websocket connections with expired jwt token?

During testing I discovered that websocket connections proxied by Caddy and authorized by caddy-authorize, established before token expires, are not closed by caddy upon token expiration. I suspect this could be exploited by an attacker that is able to steal a token, to establish a long-living connection to any back-office service protected by Caddy/caddy-authorize.
I got around this by enabling JWT verification in the back-office service, and not use caddy-authorize for that specific service, making the back-office service handling websocket connection termination. But for other use-cases where the service does not have this option, this might be a problem.

that websocket connections proxied by Caddy and authorized by caddy-authorize, established before token expires, are not closed by caddy upon token expiration.

@andoks , I think it is how Caddy works. The authorize plugin (any authorization plugins) do not keep track of the open connections. Thus, will not be able to terminate one.
However, are you doing the following in your configs? This will make sure that the hijacked token will not be accepted if the requester has ip address different from the one in the token.
auth portal:
authp {
  enable source ip tracking
}

authorize:
    authorize {
      validate source address
    }

There is "unconventional" way to terminate websocket connections when the token expires.

At the beginning of connection, we can get  source IP address and port
The plugin maintain authorized connection tables: src-ip-port => time
When an entry in the table has expired time, the plugin makes a kernel call to terminate the connection.

The above is doable in Linux.

@greenpau,

I think it is how Caddy works. The authorize plugin (any authorization plugins) do not keep track of the open connections. Thus, will not be able to terminate one.

That makes sense, especially since websockets probably has some connection state (and definitely could be tied to state in the applications' business logic), meaning both client and server should be involved in closing such connections (at least the server).

However, are you doing the following in your configs? This will make sure that the hijacked token will not be accepted if the requester has ip address different from the one in the token.

No, not currently, but I should definitely consider doing this.

There is "unconventional" way to terminate websocket connections when the token expires.

That seems like an interesting avenue for a possible solution.
I just want to point out that I do not actually need caddy-authorize to handle this, I was posting the question for documentation purposes mostly :+1:.

@andoks , good question though! 👍 Will re-open it when a need arrives.


Failures with macro functions

I'm back
glsl.parse(`void main(){
#define thing(a, b, c) \
{ \
	a *= b+c; \
}
	uint a = 2u;
}`)

Error:
	uint a = 2u;
             ^
Expected "(" but "a" found.

It also fails with single line macros without { }, although the error is different I think.

This is a fun one: three problems in one.
First, indeed multi-line macros fail to parse and also the code generator did not generate parameter lists for macros. The above example unfortunately does not produce a multi-line macro, but it seems to cause a different issue where parsing fails if a variable declaration after a define has leading indentation.
I have to admit, it's a bit of a head scratcher, so I will have to look at this with rested eyes.


improve speed of HTML processing

We've been getting reports of performance hits when using critters for per-request inlining of styles using SSR. See for example:
https://github.com/nuxt-modules/critters/issues/21
https://github.com/nuxt/nuxt/issues/8569#issuecomment-873419661
https://github.com/vercel/next.js/pull/16539#issuecomment-877409549
reproduction repository: danielroe/critters-test

remake of https://github.com/GoogleChromeLabs/critters/issues/78

Any updates?

we've shipped a range of performance improvements
have you tried it?

No, not yet. Can you share link to new options?

there are no new options

I have one css file (~600kb/100kb gzip), should i split it into chunks for better performance?
Can beasties search needed css files using the path option provided in new Beasties({})?


feat: biggroup handles points at infinity

Please read contributing guidelines and remove this line.

Benchmark results
Metrics with a significant change:

app_circuit_proving_time_in_ms (Test:emit_nullifier): 1,780 (-32%)
app_circuit_proving_time_in_ms (FPC:fee_entrypoint_public): 9,235 (-43%)
protocol_circuit_witness_generation_time_in_ms (base-parity): 345 (-36%)
protocol_circuit_witness_generation_time_in_ms (public-kernel-app-logic): 297 (-32%)
protocol_circuit_witness_generation_time_in_ms (public-kernel-tail): 899 (+22%)
protocol_circuit_witness_generation_time_in_ms (merge-rollup): 93.0 (-41%)
protocol_circuit_proving_time_in_ms (public-kernel-app-logic): 715 (-34%)
protocol_circuit_proving_time_in_ms (merge-rollup): 1,638 (-30%)

Detailed results
All benchmarks are run on txs on the Benchmarking contract on the repository. Each tx consists of a batch call  to create_note and increment_balance, which guarantees that each tx has a private call, a nested private call, a public call, and a nested public call, as well as an emitted private note, an unencrypted log, and public storage read and write.
This benchmark source data is available in JSON format on S3 here.
L2 block published to L1
Each column represents the number of txs on an L2 block published to L1.

Metric
8 txs
32 txs
64 txs

l1_rollup_calldata_size_in_bytes
740
740
740

l1_rollup_calldata_gas
6,680
6,692
6,692

l1_rollup_execution_gas
586,865
586,877
586,877

l2_block_processing_time_in_ms
1,599
6,024
12,063

l2_block_building_time_in_ms
35,268
137,519
275,391

l2_block_rollup_simulation_time_in_ms
35,053
136,748
273,859

l2_block_public_tx_process_time_in_ms
16,658
69,924 (-1%)
141,966 (-1%)

L2 chain processing
Each column represents the number of blocks on the L2 chain where each block has 16 txs.

Metric
5 blocks
10 blocks

node_history_sync_time_in_ms
18,011 (+1%)
33,188 (-1%)

node_database_size_in_bytes
21,368,912
38,031,440

pxe_database_size_in_bytes
29,868
59,425

Circuits stats
Stats on running time and I/O sizes collected for every kernel circuit run across all benchmarks.

Circuit
protocol_circuit_simulation_time_in_ms
protocol_circuit_witness_generation_time_in_ms
protocol_circuit_proving_time_in_ms
protocol_circuit_input_size_in_bytes
protocol_circuit_output_size_in_bytes
protocol_circuit_proof_size_in_bytes
protocol_circuit_num_public_inputs
protocol_circuit_size_in_gates

private-kernel-init
161 (-1%)
3,603 (+1%)
22,822 (-2%)
20,412
62,374
85,376
2,601
1,048,576

private-kernel-inner
533
4,536 (+6%)
47,259 (+4%)
89,855
62,374
85,376
2,601
2,097,152

private-kernel-reset
992
6,497 (+1%)
89,047 (+1%)
330,808
62,374
85,376
2,601
4,194,304

private-kernel-tail
532
5,038 (+1%)
39,201
87,224
79,333
10,656
266
2,097,152

base-parity
8.16
:warning: 345 (-36%)
4,143 (+5%)
128
64.0
2,208
2.00
131,072

root-parity
51.5 (+1%)
128 (-16%)
40,333 (+5%)
27,064
64.0
2,720
18.0
2,097,152

base-rollup
910
4,720 (+3%)
43,449
111,126
925
3,104
30.0
2,097,152

root-rollup
72.4
168 (-1%)
8,483 (+1%)
11,454
789
3,424
40.0
524,288

public-kernel-app-logic
192 (-1%)
:warning: 297 (-32%)
:warning: 715 (-34%)
96,791
84,967
113,344
3,475
4,096

public-kernel-tail
883 (-1%)
:warning: 899 (+22%)
1,181 (+7%)
388,047
7,723
10,144
250
512

public-kernel-setup
164 (+3%)
386 (-2%)
968 (+2%)
137,024
84,967
113,344
3,475
4,096

public-kernel-teardown
170 (+1%)
440 (-5%)
1,126
141,902
84,967
113,344
3,475
4,096

merge-rollup
6.73 (-2%)
:warning: 93.0 (-41%)
:warning: 1,638 (-30%)
2,696
925
3,104
30.0
65,536

private-kernel-tail-to-public
N/A
9,470 (-12%)
68,730 (+1%)
N/A
N/A
113,856
3,491
4,194,304

Stats on running time collected for app circuits

Function
app_circuit_proof_size_in_bytes
app_circuit_proving_time_in_ms
app_circuit_size_in_gates
app_circuit_num_public_inputs

SchnorrAccount:entrypoint
15,456
47,337 (-2%)
2,097,152
416

Test:emit_nullifier
15,456
:warning: 1,780 (-32%)
65,536
416

FPC:fee_entrypoint_public
15,456
:warning: 9,235 (-43%)
524,288
416

Token:transfer
15,456
37,860 (-5%)
2,097,152
416

FPC:fee_entrypoint_private
15,456
9,033 (+1%)
524,288
416

Token:unshield
15,456
37,187 (+1%)
2,097,152
416

SchnorrAccount:spend_private_authwit
15,456
2,682 (+1%)
131,072
416

Tree insertion stats
The duration to insert a fixed batch of leaves into each tree type.

Metric
1 leaves
16 leaves
64 leaves
128 leaves
512 leaves
1024 leaves
2048 leaves
4096 leaves
32 leaves

batch_insert_into_append_only_tree_16_depth_ms
13.1
21.4
N/A
N/A
N/A
N/A
N/A
N/A
N/A

batch_insert_into_append_only_tree_16_depth_hash_count
16.7
31.8
N/A
N/A
N/A
N/A
N/A
N/A
N/A

batch_insert_into_append_only_tree_16_depth_hash_ms
0.767
0.660
N/A
N/A
N/A
N/A
N/A
N/A
N/A

batch_insert_into_append_only_tree_32_depth_ms
N/A
N/A
61.5
96.9
314
607
1,189
2,369
N/A

batch_insert_into_append_only_tree_32_depth_hash_count
N/A
N/A
95.9
159
543
1,055
2,079
4,127
N/A

batch_insert_into_append_only_tree_32_depth_hash_ms
N/A
N/A
0.632
0.599
0.571
0.568
0.565
0.566
N/A

batch_insert_into_indexed_tree_20_depth_ms
N/A
N/A
73.8
143
453
889
1,768
3,547
N/A

batch_insert_into_indexed_tree_20_depth_hash_count
N/A
N/A
106
208
692
1,363
2,707
5,395
N/A

batch_insert_into_indexed_tree_20_depth_hash_ms
N/A
N/A
0.645
0.642
0.617
0.614
0.613
0.617
N/A

batch_insert_into_indexed_tree_40_depth_ms
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
80.5

batch_insert_into_indexed_tree_40_depth_hash_count
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
108

batch_insert_into_indexed_tree_40_depth_hash_ms
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
0.709

Miscellaneous
Transaction sizes based on how many contract classes are registered in the tx.

Metric
0 registered classes
1 registered classes

tx_size_in_bytes
84,315
673,143

Transaction size based on fee payment method
| Metric |  |
| - |  |
Transaction processing duration by data writes.

Metric
0 new note hashes
1 new note hashes
2 new note hashes

tx_pxe_processing_time_ms
37,638 (+1%)
3,641
113,941 (+1%)

Metric
0 public data writes
1 public data writes
2 public data writes
4 public data writes
8 public data writes

tx_sequencer_processing_time_ms
1,306 (-1%)
2,242
1,894
2,813
1,808


Type inference broken for aggregation queries when extension is turned on.

Bug description
When an extension is turned on (have attempted this with multiple extensions) it causes aggregation queries to show types that are not fully "computed" or "reduced" as compared to when an extension is off.
How to reproduce
Instally any extension, e.g. prisma-kysely
Attempt to run any aggregation query:
await snapshotDb.snapshotRecord.aggregate({_count:true})
Type shown by VSCode:

Expected behavior

Prisma Schema
model SnapshotRecord {
  id String @default(cuid()) @id
}

Environment & setup
Node.js v20.11.0
Prisma Version
"@prisma/client": "5.13.0"
"prisma": "5.13.0",
"prisma-extension-kysely": "2.1.0",
"prisma-kysely": "1.8.0",

Checking in here - anything further needed from me to replicate?

bump

Please don't bump issues like this. We have more than enough of them, and get to them when we get to them. I now honestly almost didn't reproduce this when it popped up on the todo list, because there are other issues where users are just patiently waiting for someone to help them and look into their problem.

I think I can reproduce this:

But I have similar behavior with other non-aggregation functions:

extension used
const prisma = basePrisma.$extends({
  name: "Soft Delete",
  query: {
    $allModels: {
      findUnique: ({ args, query }) => {
        return query({ ...args, where: { /* deletedAt: null, */ ...args.where } });
      },
    },
  },
});

Doesn't really do anything in this state as well.

Can you confirm this as well?


Update anticheat-compatibility.md

Updated most anti-cheats to the latest update as of 11th February 2023. Verus update found on their discord.
Reflex was removed from Spigot, nowhere else to be found. More info here: https://www.spigotmc.org/threads/reflex-anticheat.572425

Apologies for the lack of action on this. Is this still up to date?

Needs a rebase regardless

This became stale unfortunately - will be reopened if updated


Repeated "resolved" notifications firing

I'm seeing repeating "resolved" notification firing into notification endpoints from the same alert rule, with two different Prometheus installations. The alerts seem to repeat 5 times after the initial notification and then stop. It seems to affect every version of Alertmanager right back to 0.5.0 - the latest version that doesn't exhibit the problem is 0.4.2.
I'm assuming this relates to #815.
I've also tried with an alertmanager built from master and observe the same behaviour. The offending alert, once resolved, seems to notify every repeat_interval. The following example config below was an attempt to replicate it with much smaller timeframes.
Once the alert really resolves in Prometheus, I see nothing in either Prometheus or Alertmanager UIs.
The following is the alert manager log in debug mode for the repeating notifications.
time="2017-05-31T06:40:37Z" level=debug msg="Received alert" alert=JobDown[d36f98e][resolved] component=dispatcher source="dispatch.go:183"
time="2017-05-31T06:40:37Z" level=debug msg="flushing [JobDown[d36f98e][resolved]]" aggrGroup={}:{alertname="JobDown"} source="dispatch.go:425"
time="2017-05-31T06:40:37Z" level=debug msg="Received alert" alert=JobDown[d36f98e][resolved] component=dispatcher source="dispatch.go:183"
time="2017-05-31T06:40:38Z" level=debug msg="flushing [JobDown[d36f98e][resolved]]" aggrGroup={}:{alertname="JobDown"} source="dispatch.go:425"
time="2017-05-31T06:41:07Z" level=debug msg="Received alert" alert=JobDown[d36f98e][resolved] component=dispatcher source="dispatch.go:183"
time="2017-05-31T06:41:07Z" level=debug msg="flushing [JobDown[d36f98e][resolved]]" aggrGroup={}:{alertname="JobDown"} source="dispatch.go:425"
time="2017-05-31T06:41:07Z" level=debug msg="Received alert" alert=JobDown[d36f98e][resolved] component=dispatcher source="dispatch.go:183"
time="2017-05-31T06:41:07Z" level=debug msg="flushing [JobDown[d36f98e][resolved]]" aggrGroup={}:{alertname="JobDown"} source="dispatch.go:425"
time="2017-05-31T06:41:37Z" level=debug msg="Received alert" alert=JobDown[d36f98e][resolved] component=dispatcher source="dispatch.go:183"
time="2017-05-31T06:41:37Z" level=debug msg="Received alert" alert=JobDown[d36f98e][resolved] component=dispatcher source="dispatch.go:183"
time="2017-05-31T06:41:37Z" level=debug msg="flushing [JobDown[d36f98e][resolved]]" aggrGroup={}:{alertname="JobDown"} source="dispatch.go:425"
time="2017-05-31T06:42:07Z" level=debug msg="Received alert" alert=JobDown[d36f98e][resolved] component=dispatcher source="dispatch.go:183"
time="2017-05-31T06:42:07Z" level=debug msg="flushing [JobDown[d36f98e][resolved]]" aggrGroup={}:{alertname="JobDown"} source="dispatch.go:425"
time="2017-05-31T06:42:07Z" level=debug msg="Received alert" alert=JobDown[d36f98e][resolved] component=dispatcher source="dispatch.go:183"
time="2017-05-31T06:42:08Z" level=debug msg="flushing [JobDown[d36f98e][resolved]]" aggrGroup={}:{alertname="JobDown"} source="dispatch.go:425"
time="2017-05-31T06:42:37Z" level=debug msg="Received alert" alert=JobDown[d36f98e][resolved] component=dispatcher source="dispatch.go:183"
time="2017-05-31T06:42:37Z" level=debug msg="flushing [JobDown[d36f98e][resolved]]" aggrGroup={}:{alertname="JobDown"} source="dispatch.go:425"
time="2017-05-31T06:42:37Z" level=debug msg="Received alert" alert=JobDown[d36f98e][resolved] component=dispatcher source="dispatch.go:183"
time="2017-05-31T06:42:37Z" level=debug msg="flushing [JobDown[d36f98e][resolved]]" aggrGroup={}:{alertname="JobDown"} source="dispatch.go:425"
time="2017-05-31T06:43:07Z" level=debug msg="Received alert" alert=JobDown[d36f98e][resolved] component=dispatcher source="dispatch.go:183"
time="2017-05-31T06:43:07Z" level=debug msg="flushing [JobDown[d36f98e][resolved]]" aggrGroup={}:{alertname="JobDown"} source="dispatch.go:425"
time="2017-05-31T06:43:07Z" level=debug msg="Received alert" alert=JobDown[d36f98e][resolved] component=dispatcher source="dispatch.go:183"
time="2017-05-31T06:43:37Z" level=debug msg="Received alert" alert=JobDown[d36f98e][resolved] component=dispatcher source="dispatch.go:183"
time="2017-05-31T06:43:37Z" level=debug msg="Received alert" alert=JobDown[d36f98e][resolved] component=dispatcher source="dispatch.go:183"

Route and Receivers config:
route:
  group_by: ['account', 'region', 'environment', 'alertname']
  group_wait: 30s
  group_interval: 1m
  repeat_interval: 2m
  receiver: infra

receivers:
- name: 'infra'
  slack_configs:
  - send_resolved: true
    channel: "channel"

A note on the configuration in general. Repeat interval should be generally really long in a sane setup – at the order of 3 to 48 hours. You will be notified in the group interval should anything change about the state of alerts. The repeat interval is only about unchanged state – anything in the minute range is typically way too noisy for this.
Can you provide your Prometheus configuration as well. In particular the rule evaluation interval and alerting rule in question?

You are sending from two Prometheus servers, right? Do they have different external labels?

@fabxc - Sorry, I wasn't very clear on a couple of those things.
The alert manager config above was just so that I could replicate the problem without waiting and waiting for hours. The normal config I run with is:
 group_wait: 30s
  group_interval: 5m
  repeat_interval: 3h

In reference to two servers I was highlighting that I have two completely independent, separate configurations of Prometheus/Alertmanager that end up triggering the same scenario. There's no transmission of metrics between them or sharing of an alertmamanger or anything like that. The common factor between them is that they're using the same exporter and alert rules.
I'm not sure how it's relevant, but the exporter in question that triggers the scenario is https://github.com/Braedon/prometheus-es-exporter (version 0.3.0).
I'm running with Prometheus configs of:
  scrape_interval:     15s
  evaluation_interval: 30s (or 15s) on the second server

The target is scraped via a standard file_sd_config target, that just configures a few labels.
The alert rule is like so:
ALERT ElasticSearchClusterStatus
  IF es_cluster_health_status{job="elasticsearch"} != 0
  FOR 1m
  ANNOTATIONS {
    description = "Elasticsearch cluster {{ $labels.account }} - {{ $labels.region }} - {{ $labels.environment }} health is no longer Green. {{ $value }}",
    account = "{{ $labels.account }}",
    region = "{{ $labels.region }}",
    environment = "{{ $labels.environment }}"
  }

I hope that's useful information.

So you have two Prometheus servers but they are not sending to the same Alertmanager cluster?
The logs indicate the same alert coming in twice in short sequence. Do they have any external labels set?
Really having trouble reproducing this in any way.

I'll do some significant digging into establishing whether there are two sources of the alerts, but I can't see how. If not, I'll reproduce and include all of the config and logs.

@damomurf Thanks for your help! I will move this issue to milestone v0.8. As we are having difficulties reproducing this bug, and as it is not a mission critical issue we will release v0.7 without it for now. Don't get me wrong. We are still very much interested in fixing this issue.

I am also experiencing repeated "resolved" notifications with version=0.5.1, branch=master, revision=0ea1cac51e6a620ec09d053f0484b97932b5c902.
Anonymized configuration snippets follow.
Alertmanager configuration:
route:
  group_by: ['alertname', 'service']
  group_wait: 1m
  group_interval: 5m
  repeat_interval: 3h
  receiver: 'slack-devops'

receivers:
- name: 'slack-devops'
  slack_configs:
  - api_url: "xxx"
    channel: "#devops"
    send_resolved: true
    text: "{{ range .Alerts }}{{ .Labels.job }} {{ end }}\n{{ range .CommonAnnotations.SortedPairs }}{{ .Name }} = {{ .Value }}\n{{ end }}"

Prometheus alerts configuration:
ALERT web_endpoint_is_down
    IF probe_success < 1.0
    FOR 5m
    ANNOTATIONS {
        description = "xxx"
    }

Prometheus monitoring configuration:
global:
  scrape_interval: 1m
  scrape_timeout: 10s

scrape_configs:
  - job_name: 'xxx'
    metrics_path: /probe
    params:
      module: ['basic_http_get']
      target: ['https://xxx']
    static_configs:
      - targets: ['zzz:9115']

  - job_name: 'yyy'
    metrics_path: /probe
    params:
      module: ['basic_http_get']
      target: ['https://yyy']
    static_configs:
      - targets: ['zzz:9115']

rule_files:
  - "/etc/prometheus/alerts"

Logs only seem to have basic "performing maintenance" style info, no warnings/errors or other interesting entries of any kind.

@sandersaares Are you able to test out v0.7.1 in your environment and try to reproduce the issue?

Yes, doing that right now.

Meanwhile, one thing I noticed was that the "job" label collection was different in each of the repeated "resolved" notifications. Here are the ones it sent (together with all the firing notifications). You can see some sort of pattern here but I can't quite understand what it means.
[FIRING:1] web_endpoint_is_down (zzz:9115 website1)
website1
description = xxx

[FIRING:3] web_endpoint_is_down (zzz:9115)
website1 website3 website2
description = xxx

[FIRING:2] web_endpoint_is_down (zzz:9115)
website4 website5 website2 website1 website3
description = xxx

[13:47]
[RESOLVED] web_endpoint_is_down (zzz:9115)
website2 website1 website3 website4 website5
description = xxx

[13:48]
[RESOLVED] web_endpoint_is_down (zzz:9115)
website3 website4 website5 website2
description = xxx

[13:48]
[RESOLVED] web_endpoint_is_down (zzz:9115 website1)
website1
description = xxx

[13:49]
[RESOLVED] web_endpoint_is_down (zzz:9115)
website4 website5 website2 website1 website3
description = xxx

[13:50]
[RESOLVED] web_endpoint_is_down (zzz:9115)
website2 website1 website3
description = xxx

[13:50]
[RESOLVED] web_endpoint_is_down (zzz:9115)
website4 website5
description = xxx

[RESOLVED] web_endpoint_is_down (zzz:9115)
website3 website4 website5 website2 website1
description = xxx

[RESOLVED] web_endpoint_is_down (zzz:9115)
website3 website4 website5 website2
description = xxx

I am unable to reproduce with 0.7.1.
However, the behavior was also not consistently problematic with 0.5.1 - I first saw it yesterday and have been using the software for quite some time now. Will monitor behavior and report back if I have any more info to add - perhaps there are some hidden factors that affect the situation.

@sandersaares Thanks for reporting. Let us know if you are facing any further issues!
@damomurf Any news  from your side? Are you still facing these issues with v0.7.1?

@mxinden I'll deploy v0.7.1 first thing in my morning and definitely report back.

I'll need to leave it for at least 24 hours to get a read on some regular (ugh, yes - regular) alerts that were triggering the issue. However, initial indications are good.

After running this for several days under v0.7.1, it definitely seems like the repeat resolved notification situation has been largely solved. I recall seeing one instance of a repeat, but believe there could have been additional reasons for that.
v0.7.1 shows "Error: Timeout exceeded" on the Silences page in the UI for me, but I'll raise that as another issue.

@damomurf Thanks for testing this out on v0.7.1! I will close here for now, as it seems you issue has been resolved. Please feel free to reopen if it occurs again.

Hi all.
I have the same problem with repeating "resolved" notifications firing.
Tried on v 0.8.0 and also 0.7.1
Testing with simple config
global:
  slack_api_url: "https://hooks.slack.com/services/qqqwwweee"

route:
  receiver: slack-alert
  group_wait: 30s
  group_interval: 2m
  repeat_interval: 3m

  # All alerts in a notification have the same value for these labels.
#  group_by: [alertname, datacenter, app]

receivers:
- name: "slack-alert"
  slack_configs:
  - channel: "#alerts"
    text: "<!channel> \ndescription: {{ .CommonAnnotations.description }}\nsummary: {{ .CommonAnnotations.summary }}"
    send_resolved: true

and alert.rules
# Alert Rules

ALERT disk_used_percent
  IF disk_used_percent > 20
  FOR 5s
  LABELS { severity="critical" }
  ANNOTATIONS {
   summary = "Disk",
   description = "{{ $labels.instance }} has critical disk usage (current value: {{ $value }})",
   }

@dIoMeDs according to @damomurf this has been fixed since v0.7.1. Please open up a new issue with more details on how we can reproduce your issue. Preferably with the help of the webhook and the send-alerts script.


Improved search in "Go to file" panel

Would love to see a more accurate search for what I'm actually searching for. For example when I'm searching for "archive" and I'am in mid sentence i get these results:

I think, at least for me that the index.js inside the archive folder file is much more relevant than the .babelrc and yarn.lock. Sublime on the other hand does exactly what I want.

This change would make the workflow with VS Code even better than it already is 😄

We use the path only as tiebreaker, that makes filenames with non-consecutive matches win against paths with consecutive matches here.

Also https://github.com/Microsoft/vscode/issues/35572#issuecomment-339282965

Thank you @bpasero.
I like to add that lots of web frameworks like symfony have lots of files with identical names in different directories (indexSuccess.php, etc).

@grothkopp this is only an issue if your query happens to fully match on the file name. once your query has enough significant characters from the parent folders it should not happen anymore.

@bpasero yes, but as my screenshot shows its easy to fully match the individual characters in long file names.
And partial matches in filesnames still have precedence (see my screeshot: oneshotRecurringTask.class.php in lib/task ranks before _betrag.php in lioshop)
There's also the issue with "greedyness" (the last p in php matching instead of the p in the path).

I just noticed that adding a trainling slash after the directory name ( "lioshop/") solves the problem in the insider build.
But I think thats more a workaround than a solution.

@grothkopp yes good point, if you include a path separator, the matching on the file name alone will no longer happen.

@chrmarti
Possible to make this a priority? locate a file using a relative path is much needed than only being able to find by just the name. Especially for people working with large projects with lots of modules/files.
Thanks

Yes, https://github.com/microsoft/vscode/issues/25925#issuecomment-357261679 should be covered with https://github.com/microsoft/vscode/issues/30404

Any progress on this? It's been 3 years and Sublime still outclasses VSCode when it comes to fuzzy search matching in a intuitive way. Would be super awesome if more focus could be made on improving this. I think a lot of developers would appreciate it 👍

Is it possible to write an extension to replace the file search provider?  I agree with many previous commenters that Sublime's gets it right so much more often than VS Code's... i really miss it.  One piece of the algorithm that I think is sorely missing is that a search like hw should match hello_world.py better than it matches howto.py, by recognizing that underscores (or camelcase, etc) are natural word breaks and first letters of words are more important than others...

Another example screenshot showing that path names should not be used just as tie breakers. Path name exact string matches should appear higher than simply finding all the letters somewhere in the filename. This is just confusing:

Is there a separate issue about sorting case-sensitive matches higher?  Let me know.
For example, I'd like to see HelpAppletClientActionTranslatorTest bubble up to the top here:

Since users are most likely jumping to files related to the code they are currently editing, we could borrow some ideas from https://github.com/microsoft/vscode/issues/80444, in particular, order by proximity to currently editing file would seem like a low hanging fruit
I've had great success using fzf with proximity-sort

I'm having similar needs as @pforhan .

I want to access my files with their initial letters, not just any matching files

Bringing my comments from the closed as duplicate issue above so they can add to the conversation.
It's not only Sublime Text that gets it right. GitHub does a way better job and basically does pretty much exactly what I expect every time I use it.
Surely it should not be too hard for MS to copy the implementation from GH, which has VS Code baked into it, which they also own, and which runs in a web browser without performance issues?
It is disheartening to see that this issue is still open after nearly 7 and a half years.
Original comments below:
I always have terrible results with the VS Code Quick Open feature, and excellent results with the GitHub "go to file" feature.
For example, I want to jump to a file djangosite/urls/default.py so I Quick Open and type urldef, but I get this soup in VS Code:

In GitHub with "go to file", I get exactly what I want:

If I try to separate the two fragments with a space url def (and even try reversing them to def url thinking the first fragment might better match the filename and then be narrowed by path), I get better results, but still the best match is 4th down on the list below what appear to be lower quality matches:

Why does a 3 consecutive character match from the start of the parent directory and covering 75% of the parent directory name (urls) not trump 3x single character matches from anywhere inside separate path fragments? The individually matching characters make up a tiny percentage of the overall length of each fragment, and some are not even matching anywhere near the start of the fragment.
This is my experience often with VS Code where I have to ignore and visually scan past a long list of garbage matches to find the one I actually want. Which is infuriating with the GitHub implementation nails it pretty much every time.

This is a good example, thanks for sharing. In both cases our algorithm prefers a better performing match in the file name over a match in the full path and that is the root cause of many issues. Addressing this would mean to give up on the separation between file name matches and path matches and focus only on path matches I think.

I would also suggest that in response to your comment about the algorithm preferring a better match in the filename, that these "better" filename matches shown in my earlier screenshots are not actually better at all considering that pretty much ALL of the top matches include single isolated character matches in the middle of a word in the filename which is extremely low value.


Add an extension mangement page

Follow up to #6975, #7009, and #7010.

Following up on @magdmartin's comment in https://github.com/OpenRefine/OpenRefine/issues/7010, perhaps it would be worth having this logic in a separate function, which would be used both by the "open workspace directory" and the "open extensions directory" features. This way we could refine the support for each platform in a single place. (For instance a utility class with a static method on it.) I'm not suggesting that refinement should be done in this PR, it can be done separately.

I see a lot of potential follow ups, beyond small things like what you propose I see two medium ones as rather interesting:

Exposing more module metadata through Butterfly and render it in the table it (I think @tfmorris and I had a brief exchange about this somewhere in the past)
Making the two open directories feature stoggable in configuration(rather than exposing them and just throw errors).

Great, looking forward to those! Don't you think it would make sense to introduce this common function in this PR already?

Great, looking forward to those! Don't you think it would make sense to introduce this common function in this PR already?

Personally I would much rather get this on to master ASAP as it exposes it to more users/feedback.

@wetneb OpenDirectoryUtilities.OpenDirectory() might not be the best way to name things, feel free to change it during review.
I decided not to touch the if (!"<IP_ADDRESS>".equals(serverName) && !"localhost".equals(serverName)) logic as it seems to be broken, given it doesn't actually say anything about from where the content is served, looking at the Host header might be better but it's likely not preserved in most remote setups. It should be solved wither by a startup flag or something like #7014, I do however, consider both approaches out of scope for this PR.

@sunilnatraj would you be able to confirm that this works as expected under Windows?

@wetneb I am able to launch the application without any issues. I am using a Mac system. I do not have a Windows system to check this out.

My bad, I had misunderstood #7009.


CAD-792 mainnet-benchmark.sh:  work-limited, local chainsync benchmark

this adds a locally-served chainsync benchmark script, with integrated profiling & whistles.

this has been in intensive use over the past couple of weeks, in the course of monitoring performance investigations.

This PR does not result in breaking changes to upstream dependencies.

Checklist

[x] This PR contains all the work required to resolve the linked issue.

[ ] The work contained has sufficient documentation to describe what it does and how to do it.

[x] The work has sufficient tests and/or testing.

[x] I have committed clear and descriptive commits. Be considerate as somebody else will have to read these.

[x] I have added the appropriate labels to this PR.

I would prefer something like this, that lets you easily connect to an external server.

I see -- I'll add this functionality in my spare time, as this work is not currently planned in our sprint.

Changing status to WIP, to reflect necessity of feature change, as suggested by Karl.

outdated? what is the status of this PR?

No, not outdated, likely to be resurrected.

Remove the hydra-dont-build label once work resumes on the PR.  This will reduce eval load on hydra until then.  Thanks.

Closing this. If this is still relevant please re-open a new PR.


No AMD GPUs specified

Hi can you please help me with this? I am remotly connected to a miner rig with 12 AMD GPUs. The OS is Ubuntu 18.04.6 LTS.
 After following official documentation I have managed to install every single version of rocm from 3.0 to 5.1. Everytime I had the same problem:
 ======================= ROCm System Management Interface =======================
WARNING: No AMD GPUs specified
================================= Concise Info =================================
GPU  Temp  AvgPwr  SCLK  MCLK  Fan  Perf  PwrCap  VRAM%  GPU%
================================================================================
============================= End of ROCm SMI Log ==============================

No devices were recognized. I need rocm for tensorflow AI project. But have no clue why it does not work. Please help me. If any additional information is needed please tell me what to provide.
One more piece of info that maybe important. There is/was a cuda installed on this ubuntu.

Looks like there is no ePCI3 AtomicOp supports.
You can check by dmesg |grep kfd.
If you find something like this, it said that your motherboard or cpu cannot support PCI atomics, so we cannot use gfx803 on ROCm.
kfd: skipped device 1002:7300, PCI rejects atomics
Here is the requirement :
https://github.com/RadeonOpenCompute/ROCm/tree/rocm-4.5.2#supported-cpus

Is there any workaround? Or this hardware is not cooparable with tensorflow at all?

If it is the PCIe-AtomicsOp issue. It is said your hardware cannot support ROCm with gfx803.
Not only tf, but also all of ROCm components. The only way is upgrade your hardware, e.g. vega56/64, radeon vii, mi100, mi200, rx6800xt.

@krzysztofMajchrzak-GIT  Hi, is your issue resolved on the latest ROCm? If so can we close this ticket?

Closing the ticket.
@krzysztofMajchrzak-GIT, ROCm 6.0.2 supported configuration can be found at https://rocm.docs.amd.com/projects/install-on-linux/en/latest/reference/system-requirements.html.


Add callback to run

This makes it much simpler to hook standard CLI behavior with custom behavior, as detailed in the docs: https://db-migrate.readthedocs.io/en/latest/API/programable/#run
Related: https://github.com/db-migrate/node-db-migrate/issues/313

As stated in the issue. Not gonna happen, run is explicitly not returning a callback and never will. It will exit with proper exit codes and is CLI only.
I have however still reviewed your changes, so you know for your next PR what to look out for.


Upgrade to Webpack 5

Webpack v4 is old and outdated and should be upgraded to Webpack v5. Mend already raised a PR (#545) to do this, however that PR simply upgrades the package number and doesn't actually do any migration work. It's possible that migration isn't extremely necessary, however it would be good to make sure we're not using anything that's deprecated.
This issue will be somewhat a copy of the content on Webpack's migration guide with added context for some steps and with some steps removed that have already been done: https://webpack.js.org/migrate/5/
This migration should take place after #578 is merged.
When performing this migration, please make a draft PR and put each top-level step in a separate commit. This will make it easier to see what's going on while the migration is happening, catch any mistakes during the process, and diagnose issues if they arrise.
Steps:

[x] Upgrade webpack-cli to ^5.0.1 in package.json and run a yarn install
[x] Update plugins and loaders to their latest versions

[x] Upgrade babel-loader to ^9.1.2
[x] Upgrade style-loader to ^3.3.1
[x] Upgrade css-loader to ^6.7.3
[x] Upgrade postcss-loader to ^7.0.2
[x] Upgrade sass-loader to ^13.2.0
[x] Upgrade file-loader to ^6.2.0
[x] Upgrade url-loader to ^4.1.1
[x] Upgrade html-webpack-plugin to ^5.5.0
[x] Upgrade circular-dependency-plugin to ^5.2.2
[x] Upgrade terser-webpack-plugin to ^5.3.7
[x] Upgrade fork-ts-checker-webpack-plugin to ^8.0.0

[x] Run yarn install then both yarn build and yarn build-docs to make sure there are no build warnings or errors

If there are warnings or errors in this step, consult help here

[x] Change Webpack configuration option optimization.noEmitOnErrors: true to optimization.emitOnErrors: false in src/webpack.config.js
[x] Set the following Webpack config options in both Webpack configs (src/webpack.config.js and src-docs/webpack.config.js):

[x] node.Buffer: false
[x] node.process: false

[x] Run yarn build and yarn build-docs again to see if there are any errors

If there are warnings or errors in this step, consult help here

[x] Remove the Webpack config options from the previous step (node.Buffer and node.process)
[x] Upgrade webpack to ^5.76.0 and run yarn install
[x] Clean up configuration

[x] Replace node.fs: 'empty' with resolve.fallback.fs: false in src-docs/webpack.config.js
[x] Change [hash] to [contenthash] in src-docs/webpack.config.js
[x] Follow up: replace url-loader and file-loader with Webpack asset loaders

[ ] Run the following commands again to see if there are any errors

[x] yarn build
[ ] yarn build-docs
[ ] yarn start
[x] yarn test-unit
[x] yarn lint
If there are warnings or errors in this step, consult help here

If the builds succeeded in the last step, the migration is complete! Mark the PR as ready for review.

This issue is currently at a stand-still and I don't have the bandwidth to finish it. The checklist above is up to date on the work that has been done in #587. For anyone who wants to pick this up: currently, the only thing left is fixing an error when running yarn start or yarn build-docs. Webpack can't seem to find a specific file for some reason. I have no clue why but it might have something to do with a Babel transformation or some sort of dead code elimination, or something related. Once that is figured out, this can be merged.
To pick this up, I recommend adding AbhishekReddy1127/oui as a remote, pulling Upgrade-webpack and pushing it to your own fork to maintain Git history. From there, you should be able to create a new PR as you normally would.

@andreymyssak @SergeyMyssak Any interest in picking this up? I know you have some other issues you're working on, so it's no problem if not.


postgres does not work with different hostname

https://github.com/alpinelinux/alpine-docker-gitlab/blob/3f3227eb1c5aedeb38616084bb1e5b5839e37a86/overlay/usr/local/bin/entrypoint.sh#L23

I can't use the alpinelinux/gitlab container with a postgesql container if the hostname for postgresql is not postgres. Otherwise this line will run infinite and the gitlab container will not start.