[FEATURE REQUEST] Parallel execution of the scan

Is there a way to perform different scan tasks in parallel?

See #55, #100, and #131 for more info on this. I think Horodov will end up being the solution we adopt for this.

+1 on Horovod, I am currently trying to figure out how to use ti in hyperparameter search, will report back if I have something worth sharing.

Wonderful. This would be a great contribution!

https://stackoverflow.com/questions/57027924/talos-multi-gpu-feature

Parallelism is currently the most common label in the open issues, so I think there will be some action in this front soon. Maybe for 0.6.4(ish) something already.
Related: #100 #131

@mikkokotila I'm willing to help test new features if we get a branch going. I have multiple GPUs available to run model-data parallelism.

@awilliamson perfect and thanks. I will update here once I get started.

Have you had time to work on this yet?

@renatobellotti not yet, but I'm getting a new machine in the next week or so, which will be a great time to get started :)

Excellent, glad to hear that! :)
I'd like to make a suggestion: Not everybody has access to GPU clusters and/or cloud infrastructure (privacy, independence of research etc.). Therefore, it'd be very valuable to go for a flexibel approach, allowing parallel execution on both CPU and GPU clusters.
It seems to be possible to run Tensorflow sessions in parallel, perhaps this can be used in combination with a task queue library or so...

Another thought that came to me:
Probably, the most flexible solution could be to implement the model training/evaluation tasks as jobs in a scheduling system, e. g. slurm. Then the user can provide information whether CPU/GPU nodes should be used etc.

Has there been any progress?

Another thought that came to me:
Probably, the most flexible solution could be to implement the model training/evaluation tasks as jobs in a scheduling system, e. g. slurm. Then the user can provide information whether CPU/GPU nodes should be used etc.

I think connecting to something like slurm would be super useful!

Merging with #566


pydantic stops video_understanding task

my pydantic version is 2.2, and when run the video_understanding task, it shows the warning and terminates the run.
UserWarning: Field "model_id" has conflict with protected namespace "model_".

You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ()`.
  warnings.warn(
[1]    63213 illegal hardware instruction  python run.py

I can see solutions such as from pydantic import ConfigDict is added in omagent_core/llm/gpt.py and
class Config:

        protected_namespaces = ()
        extra = "allow"

is added in omagent_core/llm/openai_gpt.py.
But it doesn't resolve the issue.

It is caused by the OVD tool in 'engine/tools/ovd_tool.py'. I fixed it in the most recent commit.


Minor fix to templating

In a previous PR, I had failed to test that my template for adding a listener actually worked. This is a minor change to fix the issue. I have verified the helm chart installs successfully with this change.

I'm going to be naughty and merge this without approvals as it's a small change, and I've verified that it works locally with building the helm chart.


Decisions around few-shot, 0-shot, fine-tuning

Quick research and decisions to be made about:

Which models need to be fine-tuned on which tasks?
Which models need to be evaluated few-shot?
Which models should be evaluated 0-shot?

Zero-shot JAT, GPT-4, and OpenVLA for v0
Few-shot and fine-tuning in later versions


Builds Failing- Jcenter is down, replace with mavenCentral

Bintray support has ended a while back (official announcement: https://jfrog.com/blog/into-the-sunset-bintray-jcenter-gocenter-and-chartcenter/).Today, Jcenter went down and so installing from this location is causing builds to fail (See https://github.com/facebook/react-native/issues/35136)
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':app:lintVitalAlphaRelease'.
> Could not resolve all artifacts for configuration ':raygun4reactnative:releaseRuntimeClasspath'.
   > Could not resolve com.facebook.react:react-native:+.
     Required by:
         project :raygun4reactnative
      > Failed to list versions for com.facebook.react:react-native.
         > Unable to load Maven meta-data from https://jcenter.bintray.com/com/facebook/react/react-native/maven-metadata.xml.
            > Could not get resource '[https://jcenter.bintray.com/com/facebook/react/react-native/maven-metadata.xml'](https://jcenter.bintray.com/com/facebook/react/react-native/maven-metadata.xml).
               > Could not GET '[https://jcenter.bintray.com/com/facebook/react/react-native/maven-metadata.xml'](https://jcenter.bintray.com/com/facebook/react/react-native/maven-metadata.xml).
                  > Read timed out

In your build.gradle file you need to remove reference to jcenter and replace it the more reliable mavenCentral. Example:
diff --git a/node_modules/raygun4reactnative/android/build.gradle b/node_modules/raygun4reactnative/android/build.gradle
index 9b2b2c2..5215439 100644
--- a/node_modules/raygun4reactnative/android/build.gradle
+++ b/node_modules/raygun4reactnative/android/build.gradle
@@ -30,7 +30,7 @@ buildscript {
     if (project == rootProject) {
         repositories {
             google()
-            jcenter()
+            mavenCentral()
         }
         dependencies {
             classpath 'com.android.tools.build:gradle:3.4.1'
@@ -71,7 +71,7 @@ repositories {
         url "$rootDir/../lib"
     }
     google()
-    jcenter()
+    mavenCentral()
 }

 dependencies {

Closing as this was already fixed


added instruction for go server start with recipe in Readme

go:server installs and tries to start the go server.
If hostname is not setup on the ci-server , go server start fails.
Added steps in read me to do before running chef-client with go:server recipe to help users.

Hmm.  I've not seen this before.
Does your machine not have DNS?  Could you share a snippet of the log where it fails?

Closing since there has been no activity on this issue.


Problems with masking in dynamic topology.

The masking tool has an unexpected behaviour with dynamic topology, I tried to show you in this video.

Thanks for the report.
Fixed by https://github.com/stephomi/sculptgl/commit/e08b44a4dfb0d593005f9681e12e480d5a080746
Decimation/subdivision is now dependent of masking.

It works pretty well now. Thank you.


Edges of SaaS hover styles not clickable

There is a dead area (in purple below) that's highlighted, but can't be clicked in the SaaS list.

cc @cshinn @rhamilto

This is an easy fix.  Line 26 needs to move to line 52.  I'd do it, but I'm still having build issues.  :(

Fixed by #141


sticky: extensions

:exclamation: please try to NOT start discussions in here, start a new issue instead. ONLY use this thread to report extensions - thank you
Use this issue for extension announcements: new, gone-to-sh*t, recommendations for adding or dropping in the wiki list 4.1: Extensions. Stick to privacy and security related items, and do not mention legacy extensions
:small_orange_diamond: Added Web Extensions

... go look on the Wiki page - not listing things twice

:small_orange_diamond: Pending Web Extensions

ClearURLs | GitHub
Negotiator | GitHub

doesn't work (yet) full of bugs and errors - ask earthlng - worth revisiting

Google search link fix | GitHub

google/yandex. Personally, I use a user script called "google privacy" which has been working for years - I should find a link for it, see if it's maintained, stick in the wiki for user scripts? -Thorin

Searchonymous2 | GitHub
Modify Header Value
Referer Control
eCleaner (Forget Button) | GitHub

NOTE: bulk clean per storage type: eg indexedDB, dom storage etc
NOTE: Is using wrong Storage API for settings??? Requires cookies in order to access local dom.

mHeaderControl | GitHub
Shape Shifter | GitHub
EditThisCookie
History AutoDelete
Canvas Defender
FoxyProxy

Basic
Standard

WebAPI Manager | GitHub
Forget Me Not | GitHub

:small_orange_diamond: Rejected If you strongly disagree, then by all means, bring it up

... nada yet

...

FacebookTrackingRemoval barely tested here (no FB), but I trust the dev.

Not interested in Extensions that pander to a single site. The adblocking and cosmetics of hiding elements can already be done (uM, uBo .. heck, even Stylus). The link clicking/tracking can also be done not that I have uber investigated it, but rules can be created I'm sure. So unless I'm missing something.. nah

Please, I would like to submit to be added to the list, Peter Snyder add-on (WebApiManager): https://addons.mozilla.org/en-US/firefox/addon/webapi-manager/

@Decopi It is already listed above in the pending section

User Agent Switcher

AMO

FAQ

I know current policy for spoofing, so I'll silently keep an eye on this webext.

Another suggestion, ClearURLs, Kevin Roebert:
https://addons.mozilla.org/en-US/firefox/addon/clearurls/

The main difference of this add-on, first is an open source, and second it has 65 rules against average of 13 rules of other similar adds-on.

In fact : https://addons.mozilla.org/en-US/firefox/addon/clearurls/

added * Header Editor | GitHub
moved GM to legacy recommendations - the thing is a mess right now

ClearURLs has a couple of issues

you need to set an extension permission exception for cookies in order to be able to log. It doesn't use IDB, but somehow the cookie setting blocks storage of log data
it has web_accessible_resources [1] which on the face of it seems ok. The log is viewed internally, not sure about the alert part. Can someone look into this?
bit weird: the counter keeps going up and up per page - eg on reddit it blocks 2 items, then go next page and it climbs to 4 etc. It starts from scratch on a new tab+reddit etc. I'd rather have it PER page, not a running total. That way you can see if anything is blocked or not.

[1]
  "web_accessible_resources": [
      "siteBlockedAlert.html",
      "log.html"
  ]

Hi, I'am the owner of ClearURLs. I want to answere your questions:

you need to set an extension permission exception for cookies in order to be able to log. It doesn't use IDB, but somehow the cookie setting blocks storage of log data

We're saving the log with the following two functions:
/**
* Function to log all activities from ClearUrls.
* Only logging when activated.
* The log is only temporary saved in the cache and will
* permanently saved with the saveLogOnClose function.
*
* @param beforeProcessing  the url before the clear process
* @param afterProcessing   the url after the clear process
* @param rule              the rule that triggered the process
*/
function pushToLog(beforeProcessing, afterProcessing, rule)
{
    if(logging)
    {
        log.log.push(
            {
                "before": beforeProcessing,
                "after": afterProcessing,
                "rule": rule,
                "timestamp": Date.now()
            }
        );
    }
}

/**
* This function is triggered by the event windows.onRemoved and tabs.onCreated
* and will save the log permanently to the local storage.
* We only save the log anticyclically based on performance.
*/
function saveLog()
{
    if(logging)
    {
        browser.storage.local.get('resetLog', function(data) {
            if(data.resetLog)
            {
                log = {"log": []}; // Delete the old log
                browser.storage.local.set({"resetLog": false});
            }
            else
            {
                browser.storage.local.set({"log": JSON.stringify(log)});
            }
        });
    }
}

We're only save a log when the user activate the logging function. From default the logging is disabled, because it is only a feature for debugging and to make suggestions for new rules.

it has web_accessible_resources [1] which on the face of it seems ok. The log is viewed internally, not sure about the alert part. Can someone look into this?

The altert part is only a html site, that inform the user, that a whole domain was blocked, because of ad-tracking, like doubleclick{dot}net.

bit weird: the counter keeps going up and up per page - eg on reddit it blocks 2 items, then go next page and it climbs to 4 etc. It starts from scratch on a new tab+reddit etc. I'd rather have it PER page, not a running total. That way you can see if anything is blocked or not.

Yeah, this is a feature-bug :P. We will change this in the next update. So the counter will counts only the blocked fields on a page, not on a tab like now.
I am available for any questions.

Thanks @KevinRoebert I am not a developer, so excuse my ignorance.

Are you using the right storage API, because a cookie permission should not be needed AFAIK. other web extensions do not have this issue storing some local non-IDB data - I guess it depends where. Yes you would need one if it were IDB - see 1406675 and the work around ( https://github.com/ghacksuserjs/ghacks-user.js/wiki/4.1.1-Setting-Extension-Permission-Exceptions ). It's not critical, because I can easily set a site permission exception on the UUID
FYI: WEs can access their own storage regardless of prefs using the storage API (+permission).

Although this API is similar to Window.localStorage it is recommended that you don't use Window.localStorage in extension code. Firefox will clear data stored by extensions using the localStorage API in various scenarios where users clear their browsing history and data for privacy reasons, while data saved using the storage.local API will be correctly persisted in these scenarios.

Web Accessible Resources - see this issue for more info. The UUID which is unique to each install of a web extension, can be exploited in some cases. Other extensions do not require this for their UI popups, etc. Is it possible to do away with them. AFAICT, you're not injecting or doing anything to web content, just blocking requests - right? So we should be safe (told you I am not an expert!).

Good to know about the counter

The altert part is only a html site, that inform the user, that a whole domain was blocked, because of ad-tracking, like doubleclick{dot}net.

Yeah, not a fan of this. Why do we need to access an external site to be told a domain was blocked

@KevinRoebert Here is a comment by gorhill (the dev of uBo and uMatrix) that shows you how to bypass using web accessible resources - all over my head, but there ya go :)

Edit: OK, looks like you're injecting that alert into web pages. This could be a problem. Not your fault - it's Mozilla's fault for using a UUID per install and not masking/hiding it properly

I do not directly injecting the alert into web pages. The progress is the following.

ClearURLs detects a blocked domain.
ClearURLs rewrite the asked url to  var siteBlockedAlert = browser.extension.getURL('./siteBlockedAlert.html'); (the addon intern alert page).
Firefox do not visited the asked domain, firefox visited the rewritet addon intern domain.
So the displayed url is something like that: moz-extension://9b334143-726d-4afb-8aa4-930b025ae4b5/siteBlockedAlert.html

This is the code for the rewrite:
/*
 * Cancel the Request and redirect to the site blocked alert page,
* to inform the user about the full url blocking.
*/
if(result.cancel){
     return {
          redirectUrl: siteBlockedAlert
      };
}

And will be evaluate here on line 7:
/**
* Call by each Request and checking the url.
*
* @type {Array}
*/
browser.webRequest.onBeforeRequest.addListener(
    clearUrl,
    {urls: ["<all_urls>"]},
    ["blocking"]
);

Thanks for the info :+1:  I saw an image on AMO (the third one) and to me it looked like the alert was placed in the web page in lieu of a blocked element or something - it doesn't reach the edges of the page. Sorry if the term "inject" is not correct (told ya I was a dummy) :)

Added Forget Me Not | GitHub to the list above to keep an eye on etc.
This may get closer to the all-in-one persistent storage extension I am dreaming of. Blacklist all (delete after domain no longer open), whitelist (keep), greylist (session only) etc. But it has issues due to FF limitations such as FPI, containers, PB mode, localStorage by host (FF58+) being the main ones.
It does have some extra items such as history, form data, service workers, certs, IDB - but they're listed under clear all. So far the ONLY thing cleaned by domain is cookies (and localStorage in FF58+), and you can do that with C-AD. Still, worth keeping an eye on maybe.

Nano Adblocker Beta is out, I hope someone will test it as I do.

I would like to recommend adding the extension Redirect AMP to HTML. Google is trying to gather as much information about everyones browsing habits as possible, and to eat up as many web services as possible. Now they've gone so far as to host other peoples content for them. This is very destructive to the web, and your privacy. Read more about it here: https://danielmiessler.com/blog/google-amp-not-good-thing/
This extension redirects you away from AMP, so you don't use it yourself and don't link others to it when sharing the link. It's very important to send the message to newspapers and everyone that we don't want Google to own the web, we don't want AMP.
Thanks!

CSS Exfil Protection
Based on reading about this on Bleeping Computer earlier today. Anyone already familiar with this BS?

With the recent upgrades to the CSS language, CSS code has become a powerful tool that could be abused to track users on websites, extract and steal data from a web page, collect data entered inside form fields (including passwords), and even deanonymize Dark Web users in some scenarios.
Over the past month, three interesting research projects have tackled CSS as an attack vector, showing how this once benign language could be turned against users.

Article on Bleeping:
CSS Code Can Be Abused to Collect Sensitive User Data

^^ continue it here: https://github.com/ghacksuserjs/ghacks-user.js/issues/343#issuecomment-363645884

css exfil protection added to wiki (extensions & test pages) - my FF without the extension enabled was vulnerable only in same-origin, cross-origin seemed to be already covered - no idea what it is in a nilla FF

FYI:
If you block access to storage\temporary folder then you can't manage rules in "Header Editor" addon.
Cheers

Can you elaborate - what is blocking temp storage - is css exfil doing that?

No, no, you got me wrong. CSS Exfil doesn't have nothing with that.
Its general. The sub folder "storage\temporary" in profile folder.
I had for some reason (testing) for a long time now, deleted this folder and created a file with the same name, to block its usage. And in this case I can't manage rules in "Header Editor" addon.
The same I had done long time ago with "storage\default", but now, for obvious reasons how WEs storage works (, this is not a case anymore.

that's an IDB folder (but IDB can also be used by web workers - eg think of uBO's web worker when searching for the block via logger in the 3rd party filters). I can't find the doc (I swear it was in MDN docs) about temporary folder use. It was an entry about permanent/chrome/temporary - and from what I remember, the temp was nothing to worry about - I think it HAS to clean up after itself when finished. I should find that doc again.

For those who manage cookies:  CookieBro
https://addons.mozilla.org/en-US/firefox/addon/cookiebro/
I am only 'testing' Quantum in portable mode [mainly testing CSS replacements for CTR etc]
as my daily browser is Waterfox 56 - yet I have also replaced my beloved [but now legacy] CookieKeeper with this addon in Waterfox as well.
Since yesterday (v2.0.1) individual cookies can now be whitelisted and protected from deletion - this was my one proviso before changing cookie managers.
It has many other features but critically (for me) it is very easy to use in addition to having individual cookie protection.
I save a few logins & prefs and auto-delete the rest.

For those who manage cookies: CookieBro
I discovered CookieBro yesterday, had a look, not bad, but incompatible with FPI First Party Isolation, and this is of course specific to FPI at this stage of Firefox (I think FF59 will make it possible, not sure).
Unfortunately CookieBro doesn't handle setting a site's cookie authorization to Session only.
Take the scenario of a user who wishes to have all cookies blocked with exceptions for a few sites. The developer of CookieBro says to include . in the extension's Blacklist option and then to set a whitelist exception for a given site : but what happens if my wish is not to have a site whitelisted but only accepted for the session? i.e. userstyles.org : the site requires a cookie just to display, otherwise white page! I'm not going to whitelist userstyles.org just to be able to access it -> a sesssion-only cookie is the solution.
Be it reminded that with CookieBro, whitelist = allow, blacklist=block ... but session exception is missing.
That doesn't do the job completely.

Cookiebro 2.2.0 - 2018-03-17

Feature: you can now allow blacklisted sites to set session cookies.
Off by default. (enable in Options)

I would like to suggest the "Temporary Containers" add-on by stoically:
https://addons.mozilla.org/en-US/firefox/addon/temporary-containers/
I was a container-user without add-ons. I generally believe that the less add-ons, the better.
However, Container in Firefox is too basic/primitive in the sense that it has layout options but not functionality options. Even the Firefox' container add-on is very limited in functions. Everything related to containers in Firefox has to be manual, each website is handled manually (a nightmare).
I hope to be wrong, but with this lack of functions, average users are going to use Firefox' containers in the wrong way.
For example, containers in Firefox have not the option to open every website in new separate different container. Neither have the option to clean the container when closing (after "x" minutes).
In terms of privacy, this two functionalities are a must! Otherwise, is non sense to use only one container, the same container, for all websites. And the second function deleting everything, it avoids the need for other add-ons like cookie add-ons etc.
The "Temporary Containers" add-on by stoically has this two functions and many other functions.
Also, is an Open Source:
https://github.com/stoically/temporary-containers/issues
And "Temporary Containers" is just 369.5 KiB (only 100 KiB more than Firefox add-on).

Neither have the option to clean the container when closing (after "x" minutes)

I need to investigate this. PB mode (which does not allow IDB) can destroy all PB mode related data (well, it's supposed to - cookies, localStorage, appCache, cache, service workers cache, asm.js cache etc).
Can a container window (which does allow IDB) actually clear only it's IDB stuff? I do not think so!!! How a container clears selective data depends on how it's doing it. How does PB mode know what to clear. I am thinking that extensions such as Temporary Containers are using the same APIs etc that other cleaners are using - eg Cookie Auto-Delete. And many persistent storage items do not have sufficient APIs to clear by host or time-range.

Containers : I've never used them, I even block all privacy.userContext.xxx
Private Browsing (PB) : I never use it
FPI : yes, yes and yes. Very strong, very.
Nice resume, @Thorin-Oakenpants , clear enough (especially for a Sunday morning!)

^^ TL;DR
This is all you need IMO

Use FPI
Use a one off PB window for logging into different accounts on the same service (added bonus, no IDB), as this guarantees no persistent data

But...

Use containers (FF shipped setup) if you want, on top of FPI, if can see any extra benefit in it (I can't)
For sites that cross-domain logins are an issue, use a portable Chrome or a secondary Firefox profile. I am not sure of how widespread this issue is. Or consider not logging in or dumping the service. If you take privacy seriously, then this sort of sh*t shouldn't be encouraged

And...

extensions that claim to clear (all) data can be misleading, and give a false sense of privacy by leaving behind (orphaned) tracking data
That said, FF are working hard on Storage Management v2, Quota Manager, a Site Data UI, improved sanitizing, better integration with OAs, and even better wording in the UI and a dozen other aspects
Hopefully one day, EVERY single type of persistent data can be cleared by a single API call by host or time-range and bulk cleared by OA = the holy grail IMO.

clear enough (especially for a Sunday morning!)

f*k .. I ended up writing a book

Nice resume

The word is summary.. a résumé is personal and outlines said person's background and skills. Maybe I should have done that instead, it'd be shorter :grin:

Hi @Thorin-Oakenpants ! Thanks for your comments.
I am sure that the author of "Temporary-Container" (@stoically) is the right person to address your comments. His add-on' page also includes illustrative texts. But please, allow me to share my opinion:
FPI is great, and the intention here never was to use Containers instead FPI!!!!!!!!!!!
Both (FPI/Container) are two different things, created for two different needs.
In fact, both are complementary, and in my opinion both should be used together for better (not best) privacy practices/benefits.
I agree with you about "containers not mature". But that is the reason I recommended @stoically add-on. At present time is the only add-on that today expands container functions. I am not saying that this is the holy grail. I am just suggesting this add-on because increase privacy/security, and because is the best option out there for containers.
I understand that you don't see a real need for container. I was in contact with Mozilla' employees and other experts in FPI/Containers, and I am convinced about the utility. But as I said, the author of the add-on (@stoically) is the right person to address your arguments.
I also understand your general idea to replace containers by using PB tabs in normal mode. The problem is that for average users is unpractical. I know that your approach regarding privacy/security always is the best and the right to do. But sometimes we need to think in average users, and we need to think about alternatives (not solutions) for average users. Even if these alternatives are not solutions, are not perfect, are best and not better practices, if these alternatives increase privacy/security... will should use it.

My problems with English. I did mean nice summary ...
It's not a book. Modern trend to squeeze and compress has its limit; when the topic is complex compressing exaggeratedly would inevitably omit the necessary info.
The topic is complex, the summary clarifies the ingredients, for me anyway.

Lets just clarify a couple of statements here

FPI is great, and the intention here never was to use Containers instead FPI

But if you're using FPI, then Containers often NOTHING you haven't already got, just stating the facts

In fact, both are complementary

There's nothing complimentary about them - FPI effectively overrides containers in the sense that the "container" is now useless as a data bucket, because everything is already isolated on a more granular level.
If you are talking about data cleaning, that's nothing other extensions can't already do (eg Cookie Auto-Delete), and already do in normal mode (I am not 100% certain that some ContextId data types that can't be cleared by host, can't be cleared by OA, so correct me if I'm wrong).

Yup, I know not everyone has the same strategy. For example, I block all cookies except for about 8 I keep of which 4 are for logins, and another 4 or 5 which are session only (to make some sites work). Those 4 logins, I do not care about 1st party tracking (i.e linking previous visits) because I logged in, they already know me by a unique id (an exception here would be multiple accounts on the same service, which I do not have = I would use the PB window trick).
Because cookies control 1st party localStorage and IDB (and because 3rd party cookies are blocked as well it blocks 3rd party access to almost all persistent data), because appCache has to ask for permission, because Service Workers are disabled .. I do not really have to worry about clearing data when I close a domain. That is, I do not have to really worry about 1st party tracking. And I clear almost all data on close (eg session cookies, IDB etc)
Until sanitizing gets it's cr*p together, this has been my strategy and that of the user.js since inception - do not collect anything unless it is absolutely required. But that's ME.
I have no issues with Containers, or extensions that enhance that in some fashion. Because not everyone will want to enable FPI, not everyone blocks all cookies by default.
At some stage, maybe I'll add a recommendation for those who do not use FPI, to consider TC - but first I want (and do not have the time) to investigate what it says it clears etc. In the meantime its added to the first post

^^ don't get me wrong, I just had a quick look at the AMO page. Its a great extension (and the idea is great), but its also incomplete (FF lacking APIs). I just need to "vet" it first .. so for now, its on the list, not on the wiki

Hi @Thorin-Oakenpants ,
I read carefully all your comments. Again, thank you!
I never will say you are wrong about FPI/Containers. But I will just say that your info doesn't match with the info I received from Mozilla's employee & other experts.
My understanding still is that FPI and Containers are two different things, with two different functions, for different needs etc... one is not better than the other... one is not replacing the other... and both are complementary. More important, my understanding is that the combination of FPI and Containers increases privacy/security, not in a perfect way, but in a very useful way for average users (not for expert users like you).
However, my original intention here was just to suggest an add-on. Period.
I am not here to debate the concept of FPI or Containers. This is a very different conversation that should be kept only between you and expert tech guys.
Thank you again!

But I will just say that your info doesn't match with the info I received from Mozilla's employee & other experts

In what way? Without wanting to misquote anyone, what I have said on OAs is based, among other things, on chatting with Arthur Edelstein (key lead on the Tor Uplift) and he assures me, that while he's not 100% knowledgeable on all aspects of the coding, he was assured, that what he told me was correct, directly from one of the guys responsible for it at Mozilla (i.e the coder) - horse's mouth and all that.
As to their purpose and limitations - well of course they're different. One is superior/more-hardened for TBB, the other is something that grew out of an old add-on idea. The fact they use OAs is about the only thing in common.

and both are complementary. More important, my understanding is that the combination of FPI and Containers increases privacy/security

Please stop saying they're complimentary. In almost everything I have said, I am talking about containers. I think you might be saying containers but thinking about "Temporary Containers". Containers on their own do NOTHING more than FPI already does (and a PB mode window when needed). I also wouldn't include the word security - this is all about privacy.
I get where you coming from. I too am excited by new privacy measures. Having now taken a break and come back to it (new topic, yet to read), something crossed my mind. And this comes back to the 1st party tracking I was talking about earlier (i.e when you revisit a site in the same session), I would have to check, but if Temporary Containers assigns a new ContextId, then that would mean each instance of a same-domain visit would be a new isolation (obviously no existing tab of that domain should be open). So yes, if leveraged that way, then there is one example I can think of that this exceeds what FPI does, and since FPI also isolates other things, then in this ONE case so far, they're complimentary.
Not knocking anyone or anything here. Appreciate everyone's time and input :kiss:

note to self: https://addons.mozilla.org/firefox/addon/https-by-default/ is now a web extension

I would like to submit to be added to the list, @Cowlicks add-on (Privacy Possum): https://addons.mozilla.org/en-US/firefox/addon/privacy-possum
The GitHub page (https://github.com/cowlicks/privacypossum) offers a nice description, also comparing PP with other similar privacy add-ons/extensions, showing PP advantages, added values etc.

Just some quick notes:

cookie tracking - we already block 3rd party cookies. Also FPI isolates 3rd party cookies if you allow them, same with containers (per container). And uMatrix allows cookies blocking with whitelisting on a per scope level, and then there's Cookie AutoDelete and a hundred other cookie extensions

ETags - we can already do this with a rule for Header Editor - or users can disable both disk+memory cache. The first solution is the best. The second can be a tad extreme (nothing in memory will cause extra traffic etc). A third solution is clearing cache eg say every 15 minutes via an extension but is not a perfect solution (time factors leave holes for repeat visits).

Referers - Smart Referer is a dedicated referer extension that allows a per origin-destination level of control, with whitelists/greylists/blacklists etc (not actually using it myself, so slightly hazy on it)

"fingerprinting" - uM (and uBO) can handle 3rd parties if used in "medium" or hardened mode or whatever they call it. Yes, not everyone uses it that way. And some 3rd parties are unavoidable no doubt (for the site to work). And there are 1st party FP scripts. Been subscribed to FPjs2 for years. Mozilla's RFP pref defeats most of this (fonts, multiple window metrics, hardware, timing, etc etc). The key to FP'ng is that it is a worst case scenario (always do your tests allowing all JS etc), but in really you control most if it by limiting JS and 3rd parties.

Not entirely sure what this extension brings to the table, except duplication and possibly interference with other extensions/methodologies (eg is talks about faking FP'ng stuff). Note: I have not installed it or checked it out (eg options).

Maybe add Privacy Settings (@schomery) to https://github.com/ghacksuserjs/ghacks-user.js/wiki/4.1-Extensions …

… what the user.js already covers …

I assumed that the 'Extensions' page includes extensions that might be used in isolation (not necessarily with the user.js).

I get your point. Most extensions that do things FF can't eg in legacy FF there is no way to hide resource URIs, or in FF57+ how do you handle CSS Exfil, how do you do granular control of referers with a pref (you can't), or how do you build smart rule sets for HTTPS. etc
Since we're focusing on non-legacy extensions, Privacy Settings offers 6(?) pref tweaks (probably at least two more with FPI and RFP), and in the overall picture does almost nothing TBH - not knocking what Schomery has done at all, and AFAIK it's been abandoned (due to limitations in Web Ext). To me it's a false sense of privacy (but not due to the author)
The user.js on the other hand, offers 250+ settings that differ from default, and enforce another 90+ at default values. While it might not have a pretty interface, I do not wish to recommend an extension that may not be maintained anymore, and can confuse users (eg if its in the user.js at true, but they keep setting it to false in the extension, and so on)

Thanks for clarification.

… 26 items on the AMO page's first image, but the description only lists 16 "items" …

At a glance: the first image is probably of a legacy version, something less than 0.2.6.

A handy one for security was updated to WE three days ago: VTzilla. I only barely skimmed the source code, but the privacy policy is fine.
 I would only enable the option to ask to scan the file on VirusTotal before downloading, and only use it to scan executables that I don't mind the VirusTotal community getting their hands on.
I realise this probably doesn't belong in the wiki, since this doesn't bring anything new to the table (nothing other security solutions can't cover), no real hardening going on either, and even Firefox itself has its own built-in Safe Browsing for this purpose. I just wanted to let folks know.

^^ without having read it fully... so context menu: scans the file you point at on the server .. and THEN you can download it if you want after the VT result. Seems like a reasonable extension.  Maybe the dev(s) can be persuaded to change that
That said, I agree this probably outside our scope. Not my mandate to babysit people's DL and web habits

Just to make clear why I still wanted to share it here (for anyone wondering):
I personally find VirusTotal, HybridAnalysis and similar services really useful because I normally prefer to tighten security where data comes in, instead of continuously scanning memory and file system for malicious code (the latter being what resident antiviruses do). Why? The same reason I wash my hands often instead of leaving everything to my immune system (I don't have OCD... not THAT often).
When you mix these services with healthy practices and tools such as a decent firewall, keeping the whole system up to date, being cautious with what you plug in all your ports (and tweaking the system to disable autorun and such), and running in a virtual machine the shit you still just can't trust, then you don't need resident antiviruses. They become mostly just a waste of system resources.
It is probably not an optimal mindset for the average computer user (some of said practices require a fair bit of know-how, albeit nothing is really difficult), but I suspect I'm not the only one with that philosophy around here. :)

@claustromaniac nice one...
Show 'Send to VirusTotal' prompt when downloading files and Pause downloads when sending to VirusTotal options doesn't seem to work (haven't tried much now, but maybe containers), otherwise unchecked all options, having just a short-cut from context menu is nice.

but I suspect I'm not the only one with that philosophy around here

I think among users here you would be in the majority. Any decent strategy would cover all layers/levels/architecture and have multiple fallbacks. As long as you're informed, you can do what you like based on your threat model / risk assessment.
Like you, I prefer to be proactive. AV is massively overrated IMO (although it's mandate has changed a lot over time). To me, it's something that comes too late. I'd rather plug all the holes that could let it get on my system in the first place. Not for everyone, but I actually don't even bother having AV at all, I feel that secure in my setup. In the 20 years or so I had an AV, I never once had anything except false positives.
The biggest problem in security is ... humans.  This is why OpSec is so hard to master/teach

I tried Forget Me Not and it is not removing cookies in the current version.  Waiting on Lusito/forget-me-not#59.

@bberberov
Try C-AD.
Is with C-AD the same problem?
If it is, I might have one idea.

I'm not using any cleaning extensions (I just don't let anything get stored really) - might take a while for everything in that regard to mature due to FF web extension limitations etc - look at all the persistent storage types (cookies, localStorage, IDB, app cache, SW cache) and mix that with  OA's (FPI & PBMode & Containers) and between all the combos, there are still issues.

https://github.com/Lusito/forget-me-not works if set up correctly. There is also a nice update coming soon. I tried C-AD, but stopped using it, because it doesn't clean Local Storage and Indexed DB well. The developer keeps saying it cannot be done, but somehow ForgetMeNot not is doing it.

Quite a lot pages elder people are using does not work when cookies are denied, specially 1st party.
There are a few that doesn't work if 3rd party cookies are denied (for example vimeo embedded video), funny is that those are still working when you just accept 3rd party cookie and do a outgoing deny with uM,
So what I do:
FF accept all cookies... dealing with C-AD for first party and deny all 3rd party with uM.
What I have found out that C-AD does not work as intended when uM->Settings->Privacy the option Delete blocked cookies is ticked/enabled. That is what I assumed @bberberov might have the problem.
@Joel889 in the past "Forget Me Not" didn't clean LS as well (this might changed lately, but I haven't keeping an eye on it... will give today another look). What does exaclty mean "if set up correctly"?
For sure there is another WE... "Temporary Containers" which is really great, but for real it has to polish some quirks to be really set and forget and I hope author will find some time to do those.
Cheers

Show 'Send to VirusTotal' prompt when downloading files and Pause downloads when sending to VirusTotal options doesn't seem to work (haven't tried much now, but maybe containers)

True. I didn't test much either, but it could be just a bug. It is tagged as experimental, after all.

AV is massively overrated IMO

I mostly agree with you. If you're ahead enough in the learning curve you can perfectly ditch AV entirely, in that sense AV is overrated. Not to mention that hash-based detection is very easy to fool, and heuristics has many pitfalls.
I still think they're necessary (and good) tools because not everyone can afford the time (or brains) to learn what's needed to feel that safe without one. Besides, there is a learning curve (no one is born knowing shit), so even people with the time and brains to learn are better off using one while they learn.
Anyway, I better drop this before going off topic much more lol. Sorry about that.
By the way, I suggested the VTZilla devs to change the defaults. Let's see if that argument is good enough for them.

@crssi I don't have Delete blocked cookies on.
@Joel889 I just went Quantum from being on ESR ahead of ESR changing at 60.  I used CookieKeeper etc.  I didn't use to keep cookies, but they I started using Bugzillas more, and it's very convenient to configure the view once and have it persist, even when not logged in.  I think that's a sensible use case that I shouldn't have to give up on.  There are others.
@Thorin-Oakenpants I'm not a dev, but I looked at some of the WE API, and there does not seem to be a lot to work with there.  I can see now why devs are not porting extensions and functionality is lost.

@bberberov I didn't mean it that it is worse now than it was before.
I have never used a cleaning extension, so I'm not 100% sure - and due to my lack of allowing 99.9% of persistent data in the past, I have no anecdotal evidence (eg until FF56 IDB was disabled, until 59 appcache was disabled, until 60 workers were disabled - cookies remain so - I've never seen an IDB entry in my life!) ... but I believe with IDB that there has NEVER been the ability for sanitizing it by host or time range. Also FPI (FF51) & cookies are an issue (fixed in 58). So currently ESR52 users with FPI have issues (i.e I think even legacy extensions can't handle OA's). If anything, the new Web Extensions API will allow all of this to be properly sorted out once and for all

@Joel889

I initially used C-AD, but stopped using it, because it doesn't clean Local Storage and Indexed DB well. The developer keeps saying it cannot be done, but somehow ForgetMeNot is able to clean everything. If you read through my posts, I've tested this extensively with both addons.

CAD already clears localstorage if you enable the setting.
As for as I know, there's no API to clear IndexedDB per individual site. Right now, you can clear IndexedDB for every site as well as various other data from the browsingData API. I could implement it with all those other data but it would be copying every other simple cleaning extensions that do the exact same thing. One example
I rather use APIs that can take advantage of individual site cleaning rather than an all or nothing approach (since you can just use various other simple cleaning extensions that are easy to make or use the Clear history when Firefox closes option).

@Joel889
Forget Me Not does not clean IndexedDB (only manually for all sites as @mrdokenny say) and for cookies it work very unreliable, where CAD does the job very good.
CAD wins over FMN big time.

@Thorin-Oakenpants This is slightly off topic, but in your opinion, can Places be re-implemented as a Web Extension?  Is there enough access through the API to record the same complete history and a storage capability to keep a bookmark hierarchy?  I looked a little at the documentation, but I'm new at this.

@bberberov I am not an extension developer, so I have no idea. There should be storage capacity for web extensions (eg IDB) but size limits may play a factor. I'm not sure why you would want to reinvent the wheel - eg FF already collects all this info - is it for display purposes? Maybe an extension can hook into places.sqlite and display/search it in a shiny new interface. I seem to remember a legacy extension or trial (but may be confused with the browser involved) that showed (based on history) things like frequency of visits, number of visits, etc with little graphs and stuff on a dashboard.

@Thorin-Oakenpants Annotations are going away, which includes descriptions:
https://bugzilla.mozilla.org/show_bug.cgi?id=1460577
The WE API for bookmarks is limited.  There are new regressions in bookmark handling in 59/60 vs. 52.  I'm looking for a solution to these issues.

No one has said anything about my comment above. IMO it's a very important privacy extension, so let me quote myself:

I would like to recommend adding the extension Redirect AMP to HTML. Google is trying to gather as much information about everyones browsing habits as possible, and to eat up as many web services as possible. Now they've gone so far as to host other peoples content for them. This is very destructive to the web, and your privacy. Read more about it here: https://danielmiessler.com/blog/google-amp-not-good-thing/
This extension redirects you away from AMP, so you don't use it yourself and don't link others to it when sharing the link. It's very important to send the message to newspapers and everyone that we don't want Google to own the web, we don't want AMP.
Thanks!

^^ not forgotten .. here's my inbox (somewhat sanitized!)

The top line just came in. I still have, marked as unread (but I read it) the one from 3 months ago yet to action. That is, it's still actionable because I haven't written it off, so to speak. I agree it's important to stop this BS, but at the same time, this repo is predominantly about desktop - as in I do not wish to end up dealing with Android BS if I can help it.
That said, what does everyone else think? It's definitely a privacy issue, but does it really give any added privacy on Android? Note: I haven't looked at the actual extension. Maybe start a new issue, IDK.

@forteller thanks for mentioning it again, I totally missed your original comment.
AMP sounds absolutely appalling but what else can you expect from Google. I completely agree with everything written in your linked article. Thanks for the link

This extension redirects you away from AMP

unfortunately only after you visit the AMP site because it has to get the non-AMP link from the source code. Still useful when you bookmark a site though, for example.

and don't link others to it when sharing the link.

this is probably the main reason to use the extension. It's not just about mobile either, Pants.

github: https://github.com/da2x/amp2html
it's a single contentscript injected into all http(s) sites and runs at document-end.
There are 3 parts to it:

redirect away from AMP sites
modify links on news.google.*
modify links on google search

I never use anything google and don't share a lot of links so I don't have much use for the extension but if it can help that potentially fewer people link me or anyone else to AMP and thus google servers, I'm all for recommending it. Anything we can do to fight back against the evil G gets a :+1: from me

@mrdokenny

CAD already clears localstorage if you enable the setting.

Not always as confirmed here:
https://github.com/Cookie-AutoDelete/Cookie-AutoDelete/issues/355
Both localstorage values (https://html5demos.com/storage/) disappear automatically in ForgetMeNot, but one of the values remain in C-AD.
Yes, IndexDB cannot be cleaned on a per site basis, but having an on demand button beats having to close the browser in order to clear it.

Good news for SSleuth (the only legacy extension I missed) :
github.com/sibiantony/ssleuth/issues/78

Quoting myself:

I suggested the VTZilla devs to change the defaults. Let's see if that argument is good enough for them.

Welp, it clearly wasn't. They changed its privacy policy instead :man_facepalming:. More precisely, they outright removed the one that was on AMO and lumped the bits relevant to the extension with the general VT privacy policy. Not only that, but they also added a new opt-out exclusively for collecting information.
Moreover, quoting the relevant part of the policy:

Existing users of a VirusTotal extension will need to opt-in to share pDNS data with the Community. Users downloading the VT extension for the first time may opt-out of this collection in the  extension’s settings.

...the funny thing is that's not even true. Existing users have to opt-out too. This move has EvilCorp written all over it. I very much doubt something like this would've happened back when VT was not owned by EvilCorp.
I guess I'lll go back to bookmarking VT and manually scanning files when I want to. I don't want to trust them and I don't wan't to dissect the extension on every update just to see if I keep it or not. Not worth my time.
Feel free to move this under Rejected in the OP, if you want.

^^ But you can still uncheck all and have a nice shortcut with context menu to virus scan on demand.

@crssi sure, I can do that, but did you see what the policy looks like now? They don't make very clear what parts of the policy apply to the use of the extension and what parts don't. The policy on AMO used to be very specific on that. As it looks now, they could push an update any day adding data collection under the hood and I won't be sure they aren't doing that if I don't look at the source code first.
I just don't want to trust them, and I don't want to invest that kind of time just for a context menu entry. Even writing my own extension for that would be a better option than revisiting the source code every now and then lol.

Virus Total's good for Google to collect info, while the protection/warning issued don't cover 100% the VT user. Malware creators use different services that are similar to VT but do not share (at least not openly) the results of file scanning.
I'd use Kaspersky instead, while keeping in mind that you're covered on 90% of online malware.

Blend In?
https://addons.mozilla.org/en-US/firefox/addon/blend-in/
This is the only user agent spoofer I found that passes platform and oscpu at https://browserleaks.com/javascript.
I don't know if the fact that there's only one setting is desirable, though.

^^ Thats OK. It's interesting from a technical perspective. Just been reading about it - https://github.com/haqer1 is the owner so we could always @ him/her. I was surprised to see of the few reviews that they go back 7 years! 1000 users. Never heard of this before. I'm semi-impressed by the buildIDs to be honest. Interested to know how/who/what maintains the various spoofs, but its a nice touch that its plausible.
A decent UA spoofer could be handy, if used in moderation. AFAIK, an extension can override RFP's spoofing (since its the last one to modify the header?, NFI about JS stuff) - so I can see edge cases where you might spoof as a different version. It kinda needs a global pref to spoof-or-not and then a white or black list. And it kinda needs to allow end users to define the UA.
 So IDK, a OS+version spoofer seems fairly unimportant except to unbreak sites, not as a mechanism to defeat FP'ng.

^^ Since the RFP spoof doesn't spoof the OS (except to 4 platforms), this means Linux tends to stand out. And since "advanced" users (think uMatrix, uBO etc in hardened mode) can really limit exposure, this could a handy tool (i.e a decent UA spoofer) to handle the case of OS/platform spoof
https://hardware.metrics.mozilla.com/ - Win 7 43%, Win 10 38% ... Linux 2.7%

containerise on AMO automatically open websites in a container.

Containerise on AMO automatically opens websites in a container.

Blend In?
https://addons.mozilla.org/en-US/firefox/addon/blend-in/

This is the only user agent spoofer I found that passes platform and oscpu at https://browserleaks.com/javascript.

Yes, but it's still not perfect: on http://ip-check.info/?lang=en the User-Agent is reported as Windows NT 6.1 and the Browser Type as Mozilla 5.0 (Windows). However, the System is (correctly) reported as Linux. So this confirms Pants' remarks: you cannot reliably hide your OS.

@Atavic it looks Containerise is better alternative to Multi-Account Containers, at least permanent containers created within is not deleted by TC... at least from what I see.

@crssi

it looks Containerise is better alternative to Multi-Account Containers, for example permanent containers created within is not deleted by TC

I've never seen this on my system. Permanent containers in MAC have never collided with TC so far.

If permanent container is created in FF settings then not, but if it created over MAC, then after TC cleansing those are gone... see https://github.com/stoically/temporary-containers/issues/157
When using Containerise instead of MAC, those problems are gone.

If permanent container is created in FF settings then not, but if they are created over MAC, then after TC cleansing those are gone

Well, not here. I always create them in MAC.

Which FF version do you use? I am on 60.0.1.

So am I.

Can you reproduce it on a clean profile?

No, I haven't tried that yet. I'm afraid we're getting a bit OT here ;-)

Sure we are... sorry, I will delete my clutter now.
Cheers

Hello. I just wanted to say that ClearURLs moved to GitLab. The repository can now be accessed via the following link: https://gitlab.com/KevinRoebert/ClearUrls

@KevinRoebert  Thanks. I've fixed our link. PS: in your readme notice your misspelled "Official" (missing the second i )

https://addons.mozilla.org/firefox/addon/universal-bypass/ - if anyone wants to check it out (excuse my ignorance on link shorteners) does this bypass giving away your browsing to the relevant parties (like the EvilCorp AMP bullshit)?

does this bypass giving away your browsing to the relevant parties

In short: yes.
AFAIK It's merely a convenience thing. It helps speed things up a bit, and it might help avoid unwanted side effects of clicking scripted buttons on those sites (like redirections and whatnot). The extension still needs you to allow scripts on most (if not all) of those URL shortening sites, though, so that last part might not be of much relevance anyway.

(like the EvilCorp AMP bullshit)

If you're refering to the Convert AMP to HTML extension, I'd say that extension is just not meant to protect one's privacy in any way. It's a useful extension, just not privacy-wise.

I never click on shortened links anyway

Going a little OT, you might want to add those domains to your uBO filters, just in case. Before checking out that extension I wasn't even aware there were that many link-shortening sites out there...

(like the EvilCorp AMP bullshit)

If you're refering to the Redirect AMP to HTML extension

Nope. By bullshit, I mean 3rd parties sitting between me and the content... yet another asshole clipping the ticket

I was going to make my own extension for this but I found one that looks good: Detect Cloudflare | GitHub
I was a little bit surprised when I found the Block Cloudflare MITM extension is not listed here. Should I have posted this in #310 instead?
I also found Claire, but I like the visibility of Detect Cloudflare and the fact that it lists all third party domains that go through Cloudflare as well.

detect cloudflare .. so 50% of sites I visit and growing - I think it's a losing battle TBH. Block Cloudflare is too extreme IMO - i think i've covered this before, but you can just block cloudflare in uM (right?, or is it not apparent and gets loaded via a redirect or something? sorry for using the correct technical terms for shit).
Claire looks kinda cool - shows HTTP2 and IPv6. Maybe we should discuss all this in a new topic

since we block HTTP2 and will do so for IPv6 (and NFI what a ray thingy is) .. I kinda like Detect Cloudflare especially the distinction tween orange and red

I think it's a losing battle TBH. Block Cloudflare is too extreme IMO

I agree. Blocking is not for everyone. Not for many, even.

you can just block cloudflare in uM (right?, or is it not apparent and gets loaded via a redirect or something?

CF in the broadest sense is a CDN. It offers part of its services by delivering content as a third-party (Decentraleyes covers some or all of these), and that part can be blocked with uM, uBO, etc. The worrying part from a security/privacy perspective is their service as a reverse proxy. That's what people refer to when they say CF is MITM'ing users. CF stands between the server and you, and some or all HTTPS traffic gets decrypted and re-encrypted along the way without your consent (but with the server's consent, since they are the ones using CF).
CF uses a number of custom headers, including the CF-Ray header, that can be used to detect its presence as a reverse proxy. That's what these extensions do.
 Sure, I could just check the headers myself, but that gets tedious pretty fast. The extension not only saves me that trouble, but also works as a constant reminder that I shouldn't blindly trust the padlock next to my URL.

Am thinking we should add Bitwarden to the wiki - personally, I plan on wiping all my FF saved passwords and disabling it, and using Bitwarden instead. Haven't really looked at the whole thing, but FF's encryption of passwords isn't meant to be the greatest (but I do not consider my PC to be under any threat). No other password manager extension comes close to BW, right? Anyone got any opinions?
might need to retire this issue and start a clean new one, and review what's been covered in the hundred comments

No other password manager extension comes close to BW, right?

Hm. There have been several concerns (example) about the security of LastPass. What makes you sure that Bitworden does it better?

My understanding is that the data is encrypted locally and no-one but you can decrypt it, so the cloud versions for syncing cannot be compromised. Maybe I need to actually read up on the exact features. Plus it's open source AFAIK

Yes, this is exactly how Lastpass does it, too - see, e.g., https://www.lastpass.com/how-lastpass-works . And yet there seem to exist - or, at least, existed - implementation errors. They attracted attention because Lastpass is the most popular password manager. Who knows if Bitwarden is affected by similar flaws.
I am not sure if you should recommend any cloud-based password manager. There are many users who refrain from using them because of security concerns.

Hmmm .. I thought LastPass had all the keys, so to speak. Something I did for a client a few weeks ago really led me to believe they could see it all (password info rendered into an https page - I think it was either the print output or an online vault page or something). I guess not. BTW, a quick search unearthed this
https://hackernoon.com/psa-lastpass-does-not-encrypt-everything-in-your-vault-8722d69b2032
Not that BW couldn't have issues as well. BTW, BW allows you to use any cloud, including your own (but yeah, most people couldn't do that and would be even less able to make it secure)

Lately, I've been trying out MasterPassword (GitHub). I like the concept a lot, but I wish it were possible to create longer passwords. It can at most generate 20-char-long passwords AFAICT. I wouldn't use it for anything super important, for that reason.

That concept is usually called "password generators". The adblockplus guy reviewed the security details of all of the ones he could find and ended up writing his own.

Thanks for the info, @fmarier !

Just FYI, regarding the Detect Cloudflare extension that I mentioned previously, I released a fork of it with some minor changes here.

I prefer toolbar icons, so everything is visually in one place

I prefer a Page Action for this one because I don't need to see the extension's icon when CF is not detected. It just wastes valuable space on smaller screens.
I opened a pull request to commit the other significant difference to the original extension. If it is merged, the behavior of the icon will be the only difference left between the original and my fork. I considered making more changes but maybe some other time :)


Add community guides, changelog and contribution guide

Closes #6 and adds:

PR template
Issue template
Contributing guide
Releasing guide
Changelog

Codecov Report

Merging #9 into master will not change coverage.
The diff coverage is n/a.

@@          Coverage Diff          @@
##           master     #9   +/-   ##
=====================================
  Coverage     100%   100%
=====================================
  Files           1      1
  Lines          47     47
=====================================
  Hits           47     47

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7d1b89a...069ff34. Read the comment docs.


Module  has no exported member SetDataOptions and QueryFilters

Describe the bug
How can I import types for SetDataOptions and QueryFilters?
To Reproduce
Steps to reproduce the behavior:
import { QueryFilters, SetDataOptions } from 'react-query';

Expected behavior
I should be able to import SetDataOptions and QueryFilters
Screenshots

Not sure why we don't export it. I guess you could extract it from the params of the functions that have it, but I guess we can also expose it. Care to open a PR?

A bit convoluted, but you can also do:
type QueryFilters = NonNullable<Parameters<QueryClient['getQueryData']>[1]>
type SetDataOptions = NonNullable<Parameters<QueryClient['setQueryData']>[2]>


feat(E2E): add e2e modes - allow running protractor in parallel proce…

Add test:ci modes in order to allow running more than one process of test:ci in parallel

please rebase your branch

I'm a bit confused about this, there is some code here which was copied from master or something, will probably make more sense after rebase. also, what's the plan here? are we going to add some code that reports performance so we can compare options? Who's invoking those tasks?


Example server fails when accessing /admin with Error: Cannot read properties of undefined (reading 'createdAt')

Link to reproduction
No response
Payload Version
3.0.0-beta.60
Node Version
v20.12.2
Next.js Version
15.0.0-rc.0
Describe the Bug
 ⨯ TypeError: Cannot read properties of undefined (reading 'createdAt')
    at Array.reduce (<anonymous>)
digest: "4292446021"
   8 |         } else {
   9 |             columns = [
> 10 |                 table[name]
     | ^
  11 |             ];
  12 |         }
  13 |         if (unique) return uniqueIndex(`${tableName}_${columnName}_idx`).on(columns[0], ...columns.slice(1));
 ⨯ ../../node_modules/.pnpm/@payloadcms+db-postgres@3.0.0-beta.60_@opentelemetry+api@1.9.0_@types+pg@8.11.6_encoding<EMAIL_ADDRESS>(10:1) @ eval

I tried looking around a bit on whether I could find a fix to this, but I have hard time understanding where to even start. It's expecting some createdAt column that's not defined in the object, but the table creation logic is kind of... complex to say the least so no idea where it's supposed to come from.
Reproduction Steps
Using pnpm, run pnpx create-payload-app@beta cms
Enter information, most relevantly blank, postgresql and postgres://[username]@localhost:5432/cms (no password, not sure if relevant)
Then start the application with pnpm run dev, and access http://localhost:3000/admin.
Adapters and Plugins
@payloadcms/db-postgres 3.0.0-beta.60

I also had this issue yesterday locally when updating to beta 60 but not because of payload but because of update drizzle-orm and drizzle-kit.
These versions worked for me:
    "drizzle-kit": "0.20.18",
    "drizzle-orm": "0.30.10",

I need these explicit dependencies because of some customs scripts i wrote to check that the migrations added in the PR will apply cleanly against the target branch.

I am running into the same issue, after trying to install the payload 3.0 beta.
npx create-payload-app@beta

Choosing the website theme, using postgres + password on an empty database.
During installation, running into the same issue, as reported here: https://github.com/payloadcms/payload/issues/6752
Drizzle was not installed or mentioned in my package.json, using npm.
Trying to install Drizzle manually: npm i drizzle-orm
Ending up: with  "drizzle-orm": "0.32.10",
Running into a new issue:
`✓ Starting...
✓ Ready in 2.4s
○ Compiling /admin/[[...segments]] ...
✓ Compiled /admin/[[...segments]] in 3.1s (4086 modules)
⨯<EMAIL_ADDRESS>(10:1) @ eval
⨯ TypeError: Cannot read properties of undefined (reading 'createdAt')
at Array.reduce ()
digest: "787331360"
8 |         } else {
9 |             columns = [

10 |                 table[name]
| ^
11 |             ];
12 |         }
13 |         if (unique) return uniqueIndex(${tableName}_${columnName}_idx).on(columns[0], ...columns.slice(1));
GET /admin 500 in 5183ms
`

Using @yobottehg mention, of changing "drizzle-orm": "0.30.10" via npm install --legacy-peer-deps`
PostGres Connection does work now.

Hey you all
If you want to use drizzle dependency directly, ensure that it has the same version as our drizzle package uses so you won't end up with 2 drizzle instances. https://github.com/payloadcms/payload/blob/ee3d5856e3749ea72372430142f00f6fd5657096/packages/drizzle/package.json#L49 - currently 0.32.1
That applies to other dependencies like pg https://github.com/payloadcms/payload/blob/ee3d5856e3749ea72372430142f00f6fd5657096/packages/db-postgres/package.json#L54
So this in your package.json should work.
"drizzle-orm": "0.32.1",

If using npm, generally when using beta you always want to use --legacy-peer-deps, or .npmrc file:
legacy-peer-deps=true

pnpm should work fine, to generate a project with it use:
pnpx create-payload-app@beta --use-pnpm
Let me know here if something isn't clear, closing.

I'm on Beta-96 and I added 2 date fields as following:
    {
      name: 'ccpp_consent_date',
      type: 'date',
      admin: {
        date: {
          displayFormat: 'dd/MM/yyyy H:mm:ss',
          pickerAppearance: 'dayAndTime',
        },
        position: 'sidebar',
      },
      label: 'City Contact Privacy Policy Consent Date',
    },
    {
      name: 'tc_consent_date',
      type: 'date',
      admin: {
        date: {
          displayFormat: 'dd/MM/yyyy H:mm:ss',
          pickerAppearance: 'dayAndTime',
        },
        position: 'sidebar',
      },
      label: 'Terms and Conditions Consent Date',
    }

I can set save the values only the first time, but on update it always give me this error:
error: invalid input syntax for type timestamp with time zone: "{}"
    at eval (webpack-internal:///(rsc)/./node_modules/pg-pool/index.js:45:11)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async eval (webpack-internal:///(rsc)/./node_modules/drizzle-orm/node-postgres/session.js:60:22)

the error appears even when I'm trying to update from the admin UI and from the API! here's my API Update:
      await payload.update({
        id: selectedUser.id,
        collection: 'users',
        data: {
          ccpp_consent_date: new Date().toISOString(),
          tc_consent_date:  new Date().toISOString()
        },
      })

this issue appeared to me first when I moved to postgres Database specifically with the Dates fields like CreatedAt and UpdatedAt, I did a work around to fix it by adding a hook into all collections like this:
import type { CollectionBeforeChangeHook } from 'payload'

export const setTimestamps: CollectionBeforeChangeHook<any> = ({ data, collection }) => {
  const currentDate = new Date().toISOString()
  if (!data.createdAt) {
    data.createdAt = currentDate
  } else if (typeof data.createdAt !== 'string' && !(data.createdAt instanceof Date)) {
    data.createdAt = currentDate
  }

  if (!data.updatedAt) {
    data.updatedAt = currentDate
  } else if (typeof data.updatedAt !== 'string' && !(data.updatedAt instanceof Date)) {
    data.updatedAt = currentDate
  }
}

However Now it's happening with me when a user reset his password and there's a value in the resetPasswordExpiration field!

there's

Looks like a different issue! Please, open a new one with reproduction.


ci: add CSI_UPGRADE_VERSION var to build.env

Describe what this PR does
Currently, upgrade version for upgrade tests
need to be set in ci/centos branch.
This commit adds a variable in build.env,
so that we have the flexibility to use
this value instead.
Upgrade tests are failing in https://github.com/ceph/ceph-csi/pull/4005.
Depends-on: #4007

Show available bot commands
These commands are normally not required, but in case of issues, leave any of
the following bot commands in an otherwise empty comment in this PR:

/retest ci/centos/<job-name>: retest the <job-name> after unrelated
failure (please report the failure too!)

@mergifyio rebase


Audio is distorted

When recording audio at 22050 sample rate Sound is totally distored , sounds like an Alien. Tested on chrome 44.0.2403.155 ,  Linux Kernel 3.16 .

Have you tried a different version of videojs-record or chrome and does it produce the same results?

Master and 0.8.14 tested.
I tested on several different linux pc. All same Alien-sounding distorting :laughing:
Increasing Samplerate to 440000 solve that problem tho.

videojs-record uses recordrtc.js for the recording. Can you reproduce the problem with a standalone recordrtc example? If so it's a problem in that library...

We will just use 440000 for now. it works fine.

Modified the title and removed the version nr reference because this is not related to a specific version of the plugin.

I've changed the default rate to 44100. Closing this ticket, feel free to reopen with a solution..


Installing rancher on bare-metal with 1 gb RAM

RancherOS Version: (ros os version)
Latest 1.5.5
Where are you running RancherOS? (docker-machine, AWS, GCE, baremetal, etc.)
Baremetal (laptops)
Greetings. I know that this is not recommended/supported; but I do have about 6 machines that I need to get to live with just 1 gb of ram. I've found a lot of similar github issues in here; and read them all; but neither of them tells you how to actually achieve this. I know that might be achievable with running a network/PXE boot server; but I am actually not sure that I want to get into that.
Is there some way (maybe with image repacking? or something?) that will allow me to install and run Rancher on 1GB machines? (on 2gb machines it works fine).
Thanks for the hints and help

One thing that is mentioned in the docs is downgrading docker version that has a smaller size.
Check it out here.


Validation run on string instead of object

Currently, i am experimenting with project swagger-node, which utilizes this project - so thats why I report here.
I am having troubles validate payload responses against defined schemas. The source of the problem is in the test file, where the response body (json string) is not parsed into an object.
For illustration, excerpt from file in /test/api/client/...
...
function(error, res, body) {
  if (error) return done(error);

  res.statusCode.should.equal(200);

  // body = '{"message": "here i am"}'
  validator.validate(body, schema).should.be.true;
  done();
});
...

Why the response payload does not get converted to an object? When I manually adjust the code the validation succeeds...
validator.validate(JSON.parse(body), schema).should.be.true;

Shouldnt the template be adjust to reflect this?

@samekmichal you're right, the body should be parsed out to a JSON object like you've provided in your example.
What needs to happen here is the templates should be adjusted to include the change and all of the diff files for testing must be updated to reflect the change...would you submit a PR for this? it's a simple change, but lots of files to change

OK, I will have a try.

@Remco75 have you noticed this at all? It seems weird to me that this looks to be a bug but no one else has reported having trouble with it.
I dug around the request docs a bit and confirmed that when the data must be accessed they do indeed JSON.parse() the body param of the callback.

@noahdietz I will look into it tomorrow. I did not notice it yet... strange indeed. Let's investigate shortly, and decide

Also, this only pertains to use of request as the http agent. Using supertest references the res.body value for validation, as it doesn't have a third body param.

Could this be a backend implementation detail? I'm looking at the source code for request and it looks like if json is being transferred it will parse the JSON object, store in the response object and even return it as the extra parameter (request source reference). I'm kind of shooting from the hip here, just getting thoughts down.

Dug into this a bit, and found 2 things:
According to the docs we should set json to true and then the body will be parsed, and the response is JSON, and it will set the correct header.
We do it slightly different, we set the body inside the json and set the application/json header ourselves.
To at least better catch these errors, I can make a PR for that. I you still mangage to get a JSON string after this you are doing something weird ;-)
So, to sum it up: I don't think we should fix this, it's not a bug, but wel should improve the code: I will make a PR if we want this
(so let me know)

@Remco75 so what kind of change are you proposing? Just adding json: true to request options?
Even if what we do currently "works", if there is a way to make it more reliable/idiomatic, I'm ok with that, so go ahead with a PR if you've got time.

Ok, will try to find time to make a PR tomorrow or Wednesday. But again, this does not seem like a true bug to me.

@Remco75 no pressure, if this doesn't make the cut for a new release, I'm not too worried.

hmm, this is getting some serious work for a bug that might not even be a bug.
Im skipping this one, at least for the release.

I am happy to do this work as this is a big problem for me. I would just like some guidance on what you were both imagining. json: true definitely fixes my issues, but I want to understand if that is what you had in mind?

As you can see if you run this it definitely does not return an object by default:-
var request = require("request");

request({
        url: 'https://api.github.com/users/baynezy',
        method: 'GET',
        headers: {
            'User-Agent': 'baynezy'
        }
    },
    function(error, res, body) {
        if (error) return done(error);

        console.log(typeof body);
        console.log(body);
    });

In looking at the request docs more and the templates, I'm wondering how what we do currently even works.
If request data is given for the body, it populates the object in the request.options.json property...this should just be a boolean flag that toggles the parsing of JSON by request. The data should really be put in the request.options.body.
@baynezy if you want to make the changes, I'll accept a PR.

set json: true, this should only happen when the produces/consumes is application/json
put the body parameter objects in the body property. This should be changing just the property name in the templates and diff files.

@noahdietz - I am not sure you can limit it to just application\json. Vendor specific media types are quite prevalent e.g. application/vnd.github.v3+json.
I can go with just implementing application\json for now, but there will be more work needed.

@baynezy good point
would it be too hacky to right a handlebars helper to check the consume/produces strings for json? If that sounds like a reasonable approach, go for it, then we can support json in general.


Outdated package on PyPI: 0.5.6 (latest is 0.5.7)

https://pypi.org/project/ppci/ :

ppci 0.5.6

Would be nice to upload the latest version there.

Actually, looking at the package release dates https://pypi.org/project/ppci/#history vs 93e955969a2c86fe2e87b329a6215fdcfa05c503, it could be that PyPI has 0.5.7, uploaded before the release version bump.
Still confusing situation and would be nice to fix it.

Actually, maybe just release 0.5.8 ;-).

I uploaded the latest version to pypi. Can this issue be closed?

Actually, looking at the package release dates https://pypi.org/project/ppci/#history vs 93e9559, it could be that PyPI has 0.5.7, uploaded before the release version bump.

No, I just reserved a version number for future release. The next release number is already known in advance.

Posted ann on /r/Compilers: https://old.reddit.com/r/Compilers/comments/eihcuc/ppci_057_pure_python_compiler_infrastructure/ . @windelbouwman, could you queue up sending an ann to python-announce mailing list: https://mail.python.org/mailman3/lists/python-announce-list.python.org/ ?
Closing otherwise, thanks for the release!

@pfalcon I was unaware of the python announce list, what does it mean?

@windelbouwman: Well, it's a list used to announce Python-related conferences, but much more frequently, to announce releases of Python software (as in: software in Python). It's just another channel to spread a word about the project. While community can help with posts on discussion sites like HN/Reddit, I'd think that making "official" announcement (and posting to python-announce seems as "official" as it can get ;-) ) is a prerogative of project maintainers.
You can get a sample of how people do it in the archives<EMAIL_ADDRESS>. The only suggestion I'd personally have is to always start with (at least) a short description - many people fancy starting with "foo version X.Y has following changes: ...", and that's pretty boring to anyone who doesn't know what the heck "foo" is, and that's the majority of readers. Anyway, for first time, just copying a few paragraphs from README into email should work.
Thanks!

I has been posted<EMAIL_ADDRESS>

Thanks!


TextInput multiline = { false } not working in iOS

Description
TextInput Multiline={ false } is not working for iOS. After adding textAlign:'left' and writingDirection: 'ltr',
<TextInput
defaultValue='Password'
placeholder='Password'
underlineColorAndroid='transparent'
placeholderTextColor="#C0C0C0"
secureTextEntry={true}
style={ styles.rtlTextStyle }
multiline={false}
/>
rtlTextStyle : {
writingDirection: 'ltr',
textAlign: 'left',
height :40
}
Version
0.66.3
Output of npx react-native info
System:
OS: macOS 11.5.2
CPU: (8) x64 Apple M1
Memory: 34.84 MB / 8.00 GB
Shell: 5.8 - /bin/zsh
Binaries:
Node: 12.22.7 - ~/.nvm/versions/node/v12.22.7/bin/node
Yarn: Not Found
npm: 6.14.15 - ~/.nvm/versions/node/v12.22.7/bin/npm
Watchman: 2<IP_ADDRESS> - /opt/homebrew/bin/watchman
Managers:
CocoaPods: 1.11.2 - /usr/local/bin/pod
SDKs:
iOS SDK:
Platforms: iOS 14.5, DriverKit 20.4, macOS 11.3, tvOS 14.5, watchOS 7.4
Android SDK: Not Found
IDEs:
Android Studio: 4.2 AI-202.76<IP_ADDRESS>86908
Xcode: 12.5.1/12E507 - /usr/bin/xcodebuild
Languages:
Java: 11.0.12 - /usr/bin/javac
npmPackages:
@react-native-community/cli: Not Found
react: 17.0.2 => 17.0.2
react-native: 0.66.3 => 0.66.3
react-native-macos: Not Found
npmGlobalPackages:
react-native: Not Found
Steps to reproduce
Create a new Component with Text Input and add following Style Sheet for textInput
textStyle : {
writingDirection: 'ltr',
textAlign: 'left',
height :40
}
Snack, code example, screenshot, or link to a repository
No response

I am also facing this issue


ConcurrentDictionary

Hi,
When I use autofixture to generate a Dictionary<T, U> I get values populated in it by default, but with a ConcurrentDictionary<T, U> I don't.
Is this by design & I should use a specimen builder to alter that behaviour myself, or is it a missing feature or bug?

If anyone else comes accross the same issue, I've written a specimen builder for this:
public class ConcurrentDictionaryGenerator : ISpecimenBuilder
{
	public object Create(object request, ISpecimenContext context)
	{
		Type type = request as Type;
		if (type == null)
		{
			return new NoSpecimen();
		}

		Type[] typeArguments = type.GetTypeInfo().GetGenericArguments();

		if (typeArguments.Length != 2 || typeof(ConcurrentDictionary<,>).MakeGenericType(typeArguments) != type)
		{
			return new NoSpecimen();
		}

		Type nonConcurrentDictionaryType = typeof(Dictionary<,>).MakeGenericType(typeArguments);
		IDictionary nonConcurrentDictionary = (IDictionary) context.Resolve(nonConcurrentDictionaryType);

		IDictionary concurrentDictionary = (IDictionary) Activator.CreateInstance(type);

		foreach (object key in nonConcurrentDictionary.Keys)
		{
			object value = nonConcurrentDictionary[key];
			concurrentDictionary.Add(key, value);
		}

		return concurrentDictionary;
	}
}

Hello,
I cannot tell the reasons behind this, but it looking at the source code the supported collections are hard coded.
So it is not something like "does type implement IDictionary", it is "is type Dictionary<,>". Therefore ConcurrentDictionary does not work.
You dont need to create your own specimenbuilder, you can use what is already there:
      var fixture = new Fixture();

      var query = new ModestConstructorQuery();
      var methodInvoker = new MethodInvoker(query);
      var dictionaryFiller = new DictionaryFiller();
      var postProcessor = new Postprocessor(methodInvoker, dictionaryFiller);
      var exactTypeSpecification = new ExactTypeSpecification(typeof(ConcurrentDictionary<,>));
      var specimenBuilder = new FilteringSpecimenBuilder(postProcessor, exactTypeSpecification);

      fixture.Customizations.Add(specimenBuilder);

The more I learn about the internals of this library, the more I love it

Interesting to know it's only on Dictionary<,> currently & thanks for the built-in way of doing it!
I guess the only remaining question for someone would then be: would this be a good feature to add? Perhaps not for ConcurrentDictionary<,> specifically, but all IDictionary<,> implementations?

Just that I dont spread wrong information, it is is not only Dictionary<,>. I just wanted to point out that the list of types can be found directly in Fixture.cs. E.g. it is also SortedDictionary<,>, but as you already figured out, it is not ConcurrentDictionary<,>.
By all means I would say that a future implementation should check for the interface.

In reality what's hardcoded it's not the support rather the fact that interfaces are relayed to some concrete type. Like IDictionary and IReadOnlyDictionary are relayed to Dictionary, and so on.


update and consolidate guides

Besides addressing the currently open issues…

[ ] #335
[ ] #318
[ ] #317
[ ] #302
[ ] #297
[x] #249
[x] #200
[ ] #180
[ ] #172
[ ] #104
[ ] #96
[ ] #112

…we should try to improve the overall UX of the guides.
Ideas:

increase crossreferencing between guides where appropriate
rename Workflows by Workflow Guide in the TOC
add another top-level hierarchy for the TOC on the left (navigation bar) for the sister guides
link to Quiver from the Workflow Guide
link the Eval spec prominently, perhaps as a separate guide?

Here is a first draft on the Section User Guides & Info
How to OCR
Quick Start
The Quickstart guide includes:

An installation with Docker
setting up a workspace
an ocrd process command with a recommended workflow (as an alternative, I am thinking about using standard .nf file here) and calling nextflow)
how to view the results
then a link to our GitHub Wiki and/or to the more detailed instruction and/or maybe the current workflow guide?

Setup
The more detailed instructions would basically be like in https://ocr-d.de/en/setup
Usage

The current "User Guide" starting with preparing a workspace. The rest of it has to be revised, e.g. https://github.com/OCR-D/ocrd-website/issues/96
then a link to our GitHub Wiki and/or and/or the current workflow guide

OCR steps [how to use them and how to choose a workflow]
-the current Workflow Guide with a link to recommended workflows instead of having them at the buttom of the very long page
Recommended Workflows
cf. current Workflow Guide
maybe a link to Quiver?
Models
Glossary
Link to Github wiki/advanced tutorials

link to Quiver from the Workflow Guide

It seems Quiver is linked nowhere on the website (not even the sitemap). Should I open a separate issue for this @kba?


Calculator to check if a number is Armstrong number or not

Enter calculator name -
Amrstrong Number Calculator
Describe calculator functionality -
Checks whether a given number is armstrong number or not.
A number is thought of as an Armstrong number if the sum of its own digits raised to the power number of digits gives the number itself. For example, 0, 1, 153, 370, 371, 407
Under which category you wish to place it -

[] Computer Science
[x] Mathematics
[] Physics
[] Chemistry
[] Biology
[] Finance
[] Scientific
[] Health
[] Sports
[x] Let project owner decide
[] Suggest a new category - (Answer here)

@SarthakKeshari I would like to work on this issue.


User Deletion

Create a link to allow the user to delete their information
https://github.com/jamesmonks/Chat/blob/5f2c732e68bc3e715ceba58f4cd94c2326691a95/__ProjectOverview__#L5-L5

firebase_fx.js =>  delete_user()

This shouldn't have been closed, comments on commit closed this issue automatically via git.


Github Actions

Hi,
there isn't probably a real value in that,
but we could introduce github actions so that we could help keepings things all in github.
We could create a workflow:

lint
test
build

and set it as CI check in prs
__
this would probably mean decomissioning github

I can give a hand here :)
@hitmands what about adding a semantic-release step too?
PS: I think you meant "decomissioning travis"

it'd be interesting to hear @CrossEye here

I really like the idea of having everything in one place. I don't know anything about our current Travis setup though.

The Travis setup was a long time ago, but I have no interest in maintaining it.  If we can do it all in GitHub, I think that would be great.  At work these days, we do everything in GitLab, and I love how little friction there is!


render on client side only

Hi, please describe what configuration do we need to render on the client side only.
I mean without isomorphic server side rendering.
Thanks in advance.

Hello mate, you need to get rid of server related code and webpack configuration 😅
However, I'd suggest you to take your time and write a basic boilerplate for yourself instead of shredding this one.
Or you can take a look at these issues:

https://github.com/barbar/vortigern/issues/54
https://github.com/barbar/vortigern/issues/57

Have a nice day.

OK. But please describe the meaning of client.tsx file

just

replace the get part in server.tsx
=>

app.get('*', (res) => {
  res.sendFile(path.join(__dirname, 'public/index.html'));
});

so that it serves simple index.html page

Add an html page, e.g at the root of src:

<!DOCTYPE html>
<html lang="en">
  <head>
    <title>Some title</title>
  </head>
  <body>
  	<main id="app"></main>
  	<script src='/public/js/app.js'></script>
  </body>
</html>

Import that index.html page from client.tsx

or, instead of 2) and 3), use html-webpack-plugin to generate an html like above from a template (cleaner).

I'm getting res.sendfile is not a function
`const appConfig = require('../config/main');
import * as e6p from 'es6-promise';
(e6p as any).polyfill();
import 'isomorphic-fetch';
const express = require('express');
const path = require('path');
const compression = require('compression');
const Chalk = require('chalk');
const favicon = require('serve-favicon');
const app = express();
app.use(compression());
if (process.env.NODE_ENV !== 'production') {
const webpack = require('webpack');
const webpackConfig = require('../config/webpack/dev');
const webpackCompiler = webpack(webpackConfig);
app.use(require('webpack-dev-middleware')(webpackCompiler, {
publicPath: webpackConfig.output.publicPath,
stats: { colors: true },
noInfo: true,
hot: true,
inline: true,
lazy: false,
historyApiFallback: true,
quiet: true,
}));
app.use(require('webpack-hot-middleware')(webpackCompiler));
}
app.use(favicon(path.join(__dirname, 'public/favicon.ico')));
app.use('/public', express.static(path.join(__dirname, 'public')));
app.get('*', (res) => {
res.sendFile(path.join(__dirname, 'public/index.html'));
});
app.listen(appConfig.port, appConfig.host, (err) => {
if (err) {
console.error(Chalk.bgRed(err));
} else {
console.info(Chalk.black.bgGreen(
\n\n💂  Listening at http://${appConfig.host}:${appConfig.port}\n,
));
}
});`
any one ?


Clean up Result and Option modules

@zth No API changes, cleaning up some comments, removing unused code, improving compiled code


adding test for actions!

This PR will add the main workflow to test Github actions. Specifically, we want to parse metadata from the Dockerfile in the repo, and then extract software dependencies with container-diff, and finally, deploy the index.html back to Github pages. I'll likely need to do testing and debugging before this is working fully.

It worked!! Yaaash!


Fluid spreading check fails at corners

Description
Fluid spreading checks for impassable collision groups works except for corners
Reproduction
Have the same setup as the following video
https://user-images.githubusercontent.com/60196617/180125596-857b99ac-cf05-4627-9cb9-715641feaeeb.mp4
Additional context
The solution for this is probably tweaking the CanExpand() method in Content.Server/Fluids/EntitySystems/FluidSpreaderSystem.cs
Permalink
https://github.com/space-wizards/space-station-14/blob/e82e171ce642347c7a3d987404b17c1e5905e8ae/Content.Server/Fluids/EntitySystems/FluidSpreaderSystem.cs#L191

Original author of FluidSpreadSystem the line picking next direction is actually this
https://github.com/space-wizards/space-station-14/blob/e82e171ce642347c7a3d987404b17c1e5905e8ae/Content.Server/Fluids/EntitySystems/FluidSpreaderSystem.cs#L178
Replacing 8 ordinal with 4 cardinal directions should do the trick. CanExpand just checks if at given coordinate there is a tile, and that the field is unobstructed. I think I accounted for any number of fields when dividing solution.
That said, it does seem to be spilling a bit too much. No idea why?


How to use table name different from the default?

How can I manually specify a table name for a Queryable item? I've looked around the documentation and github and it looks like this feature was added a few years ago in https://github.com/diesel-rs/diesel/pull/1233? Unfortunately, I'm unable to find any documentation on how to actually use this feature.
Checklist

[x] I have already looked over the issue tracker for similar issues.
[x] This issue can be reproduced on Rust's stable channel. (Your issue will be
closed if this is not the case)

Queryable is not related to an table in any way. Queryable is a trait that maps the results of an query to a user provided type, where a query does not necessarily contain exactly one table. Therefore there is no way to "use a different table name" as this does not even make sense for what Queryable stand for.
Closed as not an issue.

@weiznich Ok, let me rephrase using the getting started example:
Given these contents of models.rs:
#[derive(Queryable)]
pub struct Post {
    pub id: i32,
    pub title: String,
    pub body: String,
    pub published: bool,
}

Diesel generates this schema.rs
table! {
    posts (id) {
        id -> Nullable<Integer>,
        title -> Text,
        body -> Text,
        published -> Bool,
    }
}

My question is: how is the table name posts determined and how can I configure it to be something else instead?


[orf:fm4:story] RemoteDisconnected

Checklist

[X] I'm reporting a broken site
[X] I've verified that I'm running yt-dlp version 2022.01.21. (update instructions)
[X] I've checked that all provided URLs are alive and playable in a browser
[X] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[X] I've searched the bugtracker for similar issues including closed ones. DO NOT post duplicates
[X] I've read the guidelines for opening an issue
[ ] I've read about sharing account credentials and I'm willing to share it if required (no account required)

Region
Austria
Description
As discussed, the orf:fm4:story extractor doesn't seem to work anymore.
error: "Remote end closed connection without response"
example URL: https://fm4.orf.at/stories/3021295
I looked into orf.py but I am of no help yet, sorry.
Verbose log
[debug] Command-line config: ['https://fm4.orf.at/stories/3021295', '--ignore-config', '-v']
[debug] Encodings: locale cp1252, fs utf-8, out utf-8, err utf-8, pref cp1252
[debug] yt-dlp version 2022.01.21 [f20d607] (win_exe)
[debug] Python version 3.8.10 (CPython 64bit) - Windows-10-10.0.19043-SP0
[debug] exe versions: ffmpeg N-105208-gb24f0c82b3-20220108 (setts), ffprobe N-105208-gb24f0c82b3-20220108
[debug] Optional libraries: Cryptodome, mutagen, sqlite, websockets
[debug] Proxy map: {}
[debug] [orf:fm4:story] Extracting URL: https://fm4.orf.at/stories/3021295
[orf:fm4:story] 3021295: Downloading webpage
[orf:fm4:story] 885473: Downloading JSON metadata
[orf:fm4:story] 885473: Downloading JSON metadata
ERROR: [orf:fm4:story] 3021295: Unable to download JSON metadata: Remote end closed connection without response (caused by RemoteDisconnected('Remote end closed connection without response')); please report this issue on  https://github.com/yt-dlp/yt-dlp . Make sure you are using the latest version; type  yt-dlp -U  to update. Be sure to call yt-dlp with the --verbose flag and include its complete output. (caused by RemoteDisconnected('Remote end closed connection without response')); please report this issue on  https://github.com/yt-dlp/yt-dlp . Make sure you are using the latest version; type  yt-dlp -U  to update. Be sure to call yt-dlp with the --verbose flag and include its complete output.
  File "yt_dlp\extractor\common.py", line 730, in _request_webpage
  File "yt_dlp\YoutubeDL.py", line 3526, in urlopen
  File "urllib\request.py", line 525, in open
  File "urllib\request.py", line 542, in _open
  File "urllib\request.py", line 502, in _call_chain
  File "yt_dlp\utils.py", line 1348, in http_open
  File "urllib\request.py", line 1358, in do_open
  File "http\client.py", line 1344, in getresponse
  File "http\client.py", line 307, in begin
  File "http\client.py", line 276, in _read_status

With 2<IP_ADDRESS>, I get a 502:
$ yt-dlp https://fm4.orf.at/stories/3021929/
[orf:fm4:story] 3021929: Downloading webpage
[orf:fm4:story] 890753: Downloading JSON metadata
[orf:fm4:story] 890753: Downloading JSON metadata
Traceback (most recent call last):
  File "/home/martin/bin/yt-dlp/yt_dlp/extractor/common.py", line 735, in _request_webpage
  File "/home/martin/bin/yt-dlp/yt_dlp/YoutubeDL.py", line 3591, in urlopen
  File "/usr/lib/python3.9/urllib/request.py", line 523, in open
    response = meth(req, response)
  File "/usr/lib/python3.9/urllib/request.py", line 632, in http_response
    response = self.parent.error(
  File "/usr/lib/python3.9/urllib/request.py", line 561, in error
    return self._call_chain(*args)
  File "/usr/lib/python3.9/urllib/request.py", line 494, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.9/urllib/request.py", line 641, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 502: Bad Gateway

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/martin/bin/yt-dlp/yt_dlp/extractor/common.py", line 617, in extract
  File "/home/martin/bin/yt-dlp/yt_dlp/extractor/orf.py", line 565, in _real_extract
  File "/home/martin/bin/yt-dlp/yt_dlp/extractor/common.py", line 997, in _download_json
  File "/home/martin/bin/yt-dlp/yt_dlp/extractor/common.py", line 976, in _download_json_handle
  File "/home/martin/bin/yt-dlp/yt_dlp/extractor/common.py", line 768, in _download_webpage_handle
  File "/home/martin/bin/yt-dlp/yt_dlp/extractor/common.py", line 753, in _request_webpage
yt_dlp.utils.ExtractorError: Unable to download JSON metadata: HTTP Error 502: Bad Gateway (caused by <HTTPError 502: 'Bad Gateway'>); please report this issue on  https://github.com/yt-dlp/yt-dlp , filling out the "Broken site" issue template properly. Confirm you are on the latest version using  yt-dlp -U

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/martin/bin/yt-dlp/yt_dlp/YoutubeDL.py", line 1389, in wrapper
  File "/home/martin/bin/yt-dlp/yt_dlp/YoutubeDL.py", line 1459, in __extract_info
  File "/home/martin/bin/yt-dlp/yt_dlp/extractor/common.py", line 643, in extract
yt_dlp.utils.ExtractorError: [orf:fm4:story] 3021929: Unable to download JSON metadata: HTTP Error 502: Bad Gateway (caused by <HTTPError 502: 'Bad Gateway'>); please report this issue on  https://github.com/yt-dlp/yt-dlp , filling out the "Broken site" issue template properly. Confirm you are on the latest version using  yt-dlp -U

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/martin/bin/yt-dlp/__main__.py", line 19, in <module>
  File "/home/martin/bin/yt-dlp/yt_dlp/__init__.py", line 864, in main
  File "/home/martin/bin/yt-dlp/yt_dlp/__init__.py", line 854, in _real_main
  File "/home/martin/bin/yt-dlp/yt_dlp/YoutubeDL.py", line 3254, in download
  File "/home/martin/bin/yt-dlp/yt_dlp/YoutubeDL.py", line 3227, in wrapper
  File "/home/martin/bin/yt-dlp/yt_dlp/YoutubeDL.py", line 1380, in extract_info
  File "/home/martin/bin/yt-dlp/yt_dlp/YoutubeDL.py", line 1407, in wrapper
  File "/home/martin/bin/yt-dlp/yt_dlp/utils.py", line 1088, in format_traceback
TypeError: format_exception() missing 2 required positional arguments: 'value' and 'tb'

In case someone needs a quick work-around: Visit the page in your browser, open the Network monitor, search for the manifest.mpd URL, and pass that to yt-dlp, e.g.
yt-dlp https://apasfiis.sf.apa.at/filehandler_nas/_definst_/nas/filehandler/online/890753_fh_20220212-psc-komplett-v1.smil/manifest.mpd