[CT-2270] [Feature] Submaterializations

Is this your first time submitting a feature request?

[X] I have read the expectations for open source contributors
[X] I have searched the existing issues, and I could not find an existing issue for this feature
[X] I am requesting a straightforward extension of existing dbt functionality, rather than a Big Idea better suited to a discussion

Describe the feature
In my company, we have a need to run operations on our materialized tables such as user deletion requests.  The dbt run-operation is single threaded and doesn't take advantage of the node yml structure.  Other submaterializations might be things like 'clone' which just implements snowflake's clone feature.
I was thinking that an improvement would be to (for a particular model)
materialized: incremental
submaterializations:
  - user_delete

Another model might be
materialized: table
submaterializations:
  - user_delete

Materializations named incremental_user_delete and table_user_delete could be created to match above
And it could be run using
dbt run --submaterialization user_delete ...
or
dbt ls --submaterialization user_delete ...
(also compile, parse)
Any model which has the submaterialization would respond to the above run and it would use the dag order / threads to run the operation.
https://getdbt.slack.com/archives/C50NEBJGG/p1677938569851409
Describe alternatives you've considered
I've considered altering the base materializations, but this has the downside of increasing their complexity for an alternate use case.
Who will this benefit?
Anyone who wants to run operations on nodes
Are you interested in contributing this feature?
yes
Anything else?
No response

Blue/Green deploys could work like (simplified)
dbt run --submaterialization clone --exclude state:modified+
dbt run  --select state:modified+

A materialization for clone might look like (abbreviated a bit to make readable)
{% macro clone_table(source_schema) %}
    {% if source_schema %}
        {% call statement('main') %}
            {% if config.get('source_database') %}
                {% set query %}
                    CREATE OR REPLACE TRANSIENT TABLE {{ this }} CLONE
                       {{ config.get('source_database') }}.{{ source_schema }}.{{ this.identifier }}
                {% endset %}
            {% else %}
                {% set query %}
                    CREATE OR REPLACE TRANSIENT TABLE {{ this }} CLONE
                       {{ this.database }}.{{ source_schema }}.{{ this.identifier }}
                {% endset %}
            {% endif %}
        {% endcall %}
    {% else %}
        {{ store_raw_result('main', message='clone_table: no source schema', code=None, rows_affected=None, agate_table=None) }}
    {% endif %}
{% endmacro %}

{% materialization table_clone, default -%}
    {% do clone_table(var('source_schema')) %}
    {{ return({'relations': [this.incorporate(type='table')]}) }}
{% endmaterialization %}

{% materialization incremental_clone, default -%}
    {% do clone_table(var('source_schema')) %}
    {{ return({'relations': [this.incorporate(type='table')]}) }}
{% endmaterialization %}

I made a mistake on 7122 and rebuilt the branch using 7135.  Added support for --select submaterialization:

I appreciate it @jtcohen6!  My concern with many of the approaches that I'm seeing to solving the problem that this solves is that a second system or complex macros are setup.
dbt run-operation

single threaded
doesn't have select logic
all graph work is very custom

another solution

puts DDL operations in another non-dbt system such as airflow or snowflake tasks
the other system needs to understand the same data that dbt understands

updating the materialization

the materialization could have a mode that can cause it to do different things
this makes all models which use the materialization have the ability to do those different things.  There might be cases where you want some models to only do X ddl and others to some other DDL

proposed solution

modularizes the DDL into something that reusable and selectable
the DDL has access to DBT's internal config/compiled_code
can use the dbt graph to order things and takes advantage of multiple threads

I realize I'm biased :), but I think this extends DBT in the way that it was meant to be extended.

In our workflow, models which change are rebuilt.  The various DDL operations are also applied such as CCPA and GDPR.  What is nice is to be able to test only those models which have changed with respect to the DDL operations.

My reasoning for the submaterialization is that it creates extra complexity on the materialization to have to know when to clone, when to create, etc.  It is also the case that there may be materializations of different types where you want to run a common action such as clone.  In this case, it would be nice to have a unified approach such as having materializations suffixed with _clone.

We also have almost an identical use case.  In our case, we might have older data which has been curated through a series of expensive transformations.  Ideally, we could clone this table before continuing.  There are times when it makes sense to add to the materialization and times when it is really nice to have submaterializations.

The materialization could use the statement CREATE TABLE <relation> CLONE <clone_relation> as you said, but then it would also need some kind of control (if logic) to know when to clone vs create.  As you point out incremental can be done in a few ways.  However, to clone incremental, it looks like cloning the table model.  In the slightly incomplete example above, I reuse the same logic for both table_clone and incremental_clone and don't touch the incremental or table materializations.
There are different ways to know which tables to clone.  We could use tags, some other config parameter, etc.  Cloning will always require checking for existence of the source, specifying a source, etc.  The PR associated with this makes submaterialization: a selector allowing for --select submaterialization:clone.
I agree that there are more ways to slice this onion than what I proposed.  I mostly liked the fact that I can separate the logic cleanly for tasks that are clearly in the DDL / maintenance category.   I'm certainly not trying to get you to agree that this is best, just given how close I am to the solution, felt like a bit more explanation might be helpful.  The case for using this for other things like CCPA imo is very strong.


Only list files in migration tool project option that actually can be migrated

Describe the bug
Atm we're listing all files of the project which is not a great user experience, as it's uncertain to the user if all their files will be affected by the upgrade.
It would be better to just list the files that actually contain the components we're planning to upgrade. That would also make the list shorter, make it easier to filter files and give a better overview before pressing the migration button.
To reproduce
Try the migrate project option on the MRTK project - there's a long list of files shown but actually only one scene gets migrated.
Expected behavior
Only show upgradeable files in that list.

similar to what is done on deciding if a file is saved or not on migration in this PR https://github.com/microsoft/MixedRealityToolkit-Unity/pull/7688

Resolved on #7707


implement FlareSolverr to solve some CF captchas

Is your feature request related to a problem? Please describe.
One of the sites I am scraping suddenly setup through Cloudflare and I keep getting presented with a CAPTCHA challenge on changedetection, but not on e.g. Jackett, which does have FlareSolverr implemented.
Describe the solution you'd like
Please bake in support for FlareSolverr add the ability to specify FlareSolverr API URL in the [protocol:-http]://[fqdn or ip:-localhost]:[port:-8191] format.
Thank you!

Have been rejected from Cloudflare and Cloudfront a lot in the last week or two. Seems like their browser fingerprinting has caught up with playwright / chrome docker containers. Would appreciate some solution, as the number of sites which I can actually access has reduced by about 20%

Interestingly, just came across https://github.com/lwthiker/curl-impersonate
This seems to bypass Cloudfront blocking, if used with a correct User-Agent

I wonder if you update the HOSTS file on the machine that's running Change Detection, will it go directly to the IP of the host server and bypass CloudFlare...?

Interesting idea. However that would depend on you knowing the webserver direct IP address, and half the reason people hide behind Cloudflare or Cloudfront is to obscure that address, and to take advantage of the DDos protection afforded by those platforms.

For what it's worth I managed to use FlareSolverr to fetch a page protected by CloudFlare.
It's not very straightforward and involves using regex filtering feature from jq which could be a bit tricky to extract data from HTML.
I don't know if this will work for every cases but it should help in most simple ones.
First of all, I'm on a Synology NAS and have both FlareSolverr and changedetection.io installed as docker containers.
You have to setup FlareSolverr and make sure you can access it from changedetection.io.
To ease the configuration my FlareSolver docker has a 'hostname: flaresolverr' setting and it runs on port 8090.
If both are on the same network and setup in the same docker instance that should work.
On changedetecton.io, for the page you want to fetch here is the configuration:

In 'Request' tab

Fetch Method > Basic fast Plaintext/HTTP Client (doesn't seem to work with Playwright, not sure why)
Proxy > No proxy (might work with a proxy, I couldn't test as I don't use one)
Click on Show advanced options:
Request method > POST
Request body >
{
  "cmd": "request.get",
  "url":"URL TO BE FETCHED",
  "maxTimeout": 60000
}

Request header >
Content-Type: application/json

In 'Filters & Triggers' tab

CSS/JSONPath/JQ/XPath Filters >
jq:.solution.response | capture("REGEX TO EXTRACT CONTENT FROM HTML">(?<name>[^<]*)</b>"; "gm")
FlareSolver returns a json with the HTML in the 'solution.response' attribute, then I added the 'capture' filter which in this case lists the items I want to fetch from the page based on a regex.
Here it really depends on your page and on what you want to extract.

@wpigoury thanks for the info! I was able to get some responses using FlareSolverr, super interesting project, looks like undetected-chromedriver is actually binary patched!
Trying to think of a workflow here

Site gets 403, goes into flare-solverr mode
on next request, it asks flare-solverr for the headers (cookies etc)
those cookies are stored in some kind of database so any other watch for the same domain name could re-use those credentials (would have to add some extra API perhaps to flare-solverr with a mini-db, store in-memory with python dict or something)
add those cookies/headers to the next request of the watch

Should the site get 403 again, then I think it can just repeat the above steps...

I have to add - on the sites where I needed cloudflare, simply moving to a better residential IP pretty much solved it and I didnt need flaresolverr...

I'm selfhosting changedetection at home via docker/selfhosted browserless chrome instance, and I'm running into the cloudflare captcha for sites like www.bhphotovideo.com when trying to monitor for restock. There's a few other sites that do the same thing consistently as well.

OK yeah, i'll add this to my next list of tasks :) lots of requests coming in

Hi guys! First of all, thanks @dgtlmoon for this great project, you rock! I've just made this adapter https://github.com/mimnix/FlareProxy  You can deploy it alongside FlareSolverr, add it as a proxy in the Changedetection proxy settings, and everytime you need to demonstrate you're just a human watching a web page protected by Cloudflare, the transparent proxy is right there for you :wink:


SEGV on DecodeConfig

reported by @Gaillard in #36:
"""
I am also getting another seg fault going through DecodeConfig, maybe unrelated. I don't have a test case for that one though, just the stack.
fatal error: unexpected signal during runtime execution
[signal SIGSEGV: segmentation violation code=0x2 addr=0x7f6f88000388 pc=0x7f6f88000388]

runtime stack:
runtime.throw(0x1a7eb53, 0x2a)
    /usr/local/go/src/runtime/panic.go:566 +0x95
runtime.sigpanic()
    /usr/local/go/src/runtime/sigpanic_unix.go:12 +0x2cc

goroutine 49 [syscall, locked to thread]:
runtime.cgocall(0xbea500, 0xc420349638, 0xc400000000)
    /usr/local/go/src/runtime/cgocall.go:131 +0x110 fp=0xc420349608 sp=0xc4203495c8
NAME/vendor/github.com/pixiv/go-libjpeg/jpeg._Cfunc_new_decompress(0x0)
    ??:0 +0x4a fp=0xc420349638 sp=0xc420349608
NAME/vendor/github.com/pixiv/go-libjpeg/jpeg.DecodeConfig(0x25127e0, 0xc4202a20e0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
    /home/gaillard/go/src/NAME/vendor/github.com/pixiv/go-libjpeg/jpeg/decompress.go:323 +0xa4 fp=0xc420349688 sp=0xc420349638

"""

Issue in DecodeIntoRGB.
While running openai's universe's first agent, got unexpected crashes (I'm using amd64). I reported the full issue there. I've tried the Go versions 1.6, 1.7 and 1.8, but the error remains. Possibly, some issue in DecodeIntoRGB.
fatal error: unexpected signal during runtime execution
[signal 0xb code=0x1 addr=0x2c pc=0x7f8f432bad1c]

runtime stack:
runtime.throw(0x7f8f4420b740, 0x2a)
	/usr/lib/go/src/runtime/panic.go:530 +0x92
runtime.sigpanic()
	/usr/lib/go/src/runtime/sigpanic_unix.go:12 +0x5e
runtime.reimburseSweepCredit(0x7f8eec000b60)
	/usr/lib/go/src/runtime/mgcsweep.go:399 +0x52

goroutine 35 [syscall, locked to thread]:
runtime.cgocall(0x7f8f43a803d9, 0xc820045440, 0xc800000000)
	/usr/lib/go/src/runtime/cgocall.go:123 +0x121 fp=0xc820045418 sp=0xc8200453e8
github.com/openai/go-vncdriver/vendor/github.com/pixiv/go-libjpeg/jpeg._Cfunc_new_decompress(0x0)
	??:0 +0x49 fp=0xc820045440 sp=0xc820045418
github.com/openai/go-vncdriver/vendor/github.com/pixiv/go-libjpeg/jpeg.DecodeIntoRGB(0x7f8f3ed89138, 0xc8200e66c0, 0xc820045618, 0x0, 0x0, 0x0)
	/home/manan/data-science-machine-learning-code/openai/go-vncdriver/.build/src/github.com/openai/go-vncdriver/vendor/github.com/pixiv/go-libjpeg/jpeg/decompress.go:256 +0x93 fp=0xc820045560 sp=0xc820045440
github.com/openai/go-vncdriver/vncclient.(*TightEncoding).Read(0xc8205a2000, 0xc8200fc000, 0xc8200a4000, 0x7f8f2c7ba168, 0xc8200e8018, 0x0, 0x0, 0x0, 0x0)

@mananSingh Thank you for your report. I'm investigating this issue but could not reproduce yet.
Could you tell me some environments?

linked libjpeg-trubo version (or libjpeg version)
running on mac, Linux or Windows ?

libjpeg-turbo8-dev
linux (ubuntu 14.04)
amd64

This issue may fixed in #53.


Release Google.Cloud.Connectors.V1 version 1.0.0

Changes in this release:
No API surface changes; just dependency updates and promotion to GA.

Created release for Google.Cloud.Connectors.V1-1.0.0


iOS 7.1.2 Safari Error

I'm loading the promises polyfilled version (localforage.js) but I am getting an error when using on this browser. The error is as follows:
TypeError: 'undefined' is not an object (evaluating 'adbinfo.db.transaction')
Any ideas?

What version of localForage are you using? Any chance you’re using Private Browsing mode? I think we fixed bugs with it but I haven’t pushed the version yet; I should do that soon.

tofumatt

On 1 October 2015 at 16:29:58, alexstanbury<EMAIL_ADDRESS>wrote:
I'm loading the promises polyfilled version (localforage.js) but I am getting an error when using on this browser. The error is as follows:
TypeError: 'undefined' is not an object (evaluating 'adbinfo.db.transaction')
Any ideas?
—
Reply to this email directly or view it on GitHub.

Well this is strange, it has suddenly started working without me changing anything.
For what it's worth, I'm using 1.2.4, not using private browsing mode and cookies are set to block from third parties.
It's my friend's device I am testing with, I opened Safari and opened a new tab (it wasn't in private mode as the browser colouring was light grey instead of dark) and browsed to my site and got the error. I got it every time upon refresh. I went to download Chrome to see if it was the same issue but before it completed I went back to safari and it worked...
I can now recreate it, if I open up private mode, select 'keep all' tabs and load the site, I get the aforementioned error as expected, the interesting thing is if I switch private mode off and again select 'keep all' tabs then the error persists even though I'm not in private mode any more.
I guess this is an iOS bug then, not sure what happened originally but I presume something funky with private mode interfering...I will update if I can provide any more info.
Cheers

Glad to hear it's working, though yeah that sounds like a super-weird bug.
Let me know if you run into anything reproducible or into this issue again. I'll close this for now, but don't be shy about issues.
Thanks for you detailed response!


Input functions improperly returning directory

I think I have discovered a bug when using directories and checkpoints. I have a checkpoint which creates a variable number of files to a certain output directory. I am using an 'aggregate'-esque rule here called merge_hiC which takes all files created from previous rule processing of the checkpoint output.
The error is that when I use the input function findMerge, it returns only the directory. I even place gibberish after what should be the directory path from the checkpoint and only get the directory path returned as part of the input function.
In other words, no matter what I put in to the input function, I only get the directory returning from the checkpoint.
Snakemake version
This is in Snakemake 5.8.1 and 5.4.4
#!python

def findMerge(wildcards):
	checkpoint_output = checkpoints.splitting.get(read='2').output.split_parts
	return checkpoint_output+'HSkjdfhk'
	# PARTS, = glob_wildcards(os.path.join(checkpoint_output, "hiC_{part}.fastq"))
	# return expand(rules.repair_hiC.output.repaired_bam, part=PARTS)

def findMergeTwo(wildcards):
	checkpoint_output = checkpoints.splitting.get(read='1').output.split_parts
	return checkpoint_output+'HSkjdfhk'
	# PARTS, = glob_wildcards(os.path.join(checkpoint_output, "hiC_{part}.fastq"))
	# return expand(rules.repair_hiC.output.repaired_bam, part=PARTS)

rule all:
	input:
		'alignment/hic/hic.fix.sort.bam',
		findMerge,
		findMergeTwo

rule make_flags:
	output:
		touch(expand('.hiC_{read}', read=['1','2']))

checkpoint splitting:
	input:
		flag = '.hiC_{read}'
	output:
		split_parts=directory("split_hiC/{read}/"),
		combined = temp('split_hiC/hiCAll_{read}.fastq'),
		sort_hi = temp('split_hiC/hiCAll_{read}_sorted.fastq')
	shell:
		'''
		N=$(( $RANDOM % 10))
		for j in $(seq 1 $N); do echo -n $j > {output.split_parts}/hiC_$j.fastq; done
		touch {output.combined}
		touch {output.sort_hi}
		'''

rule map_hiC:
	input:
		split_hiC = 'split_hiC/{read}/hiC_{part}.fastq',
		flag = '.hiC_{read}'
	output:
		sorted_bam = 'alignment/hic/chunks{read, \d+}/{part, \d+}.nsort.bam'
	shell:
		'''
		touch {output.sorted.bam}
		'''

rule repair_hiC:
	input:
		sorted_bam = expand('alignment/hic/chunks{read}/{{part}}.nsort.bam', read=['1','2']),
	output:
		repaired_bam = 'alignment/hic/repaired/{part, \d+}.bam'
	params:
		sge_opts = '-l mfree=4G'
	shell:
		'''
		touch {output.repaired_bam}
		'''

rule merge_hiC:
	input:
		findMerge,
	output:
		sorted_bam = 'alignment/hic/hic.fix.sort.bam',
	shell:
		'''
		file {input}
		touch {output.sorted_bam}
		'''

Multiple checkpoints seem to cause issues with the DAG updating properly.  This sounds similar to (this)[https://bitbucket.org/snakemake/snakemake/issues/1284/requesting-checkpoint-output-in-rule-all] but that should be fixed...
Some things to try that will help troubleshoot:

Removing split_parts from the checkpoint call
The two input functions are identical except for the read wildcard.  Can you incorporate that into your input directives to just use one function?  Limiting the number of 'simultaneous' checkpoints may solve a race condition.
What happens if you remove the directory modifier?  Change the output of your checkpoint to be the first file that would be produced, then use your glob to find all the parts.  This will determine if it's directory related or just the structure of the snakefile.

Let's see where that gets us!

Thank you for the quick reply!
I ended up making a couple of changes, and was able to achieve the desired result.
As a note, changing the checkpoint output call (split_parts) had no effect. Same for the two input functions. The directory modifier was part of a solution, and I am listing the additional changes below.

I removed the directory modifier and changed the output of the checkpoint to the first file made in the rule
Added a touch() file to be created after each checkpoint iteration which is required by the merge_hiC step.
Changed findMerge to the following:
def findMerge(wildcards):  PARTS, = glob_wildcards(os.path.join("split_hiC/1/hiC_{part}.fastq"))  return expand(rules.repair_hiC.output.repaired_bam, part=PARTS)

This triggers proper evaluation of the DAG after checkpoints. However, it does not illuminate what the bug is. I am happy to test additionally, but for now have found a satisfying conclusion.

That's interesting, does your input function, findMerge, not directly call the checkpoints object anymore?
If you want to test more, you can refine and simplify your original example to make it a test case. In particular, is it the directory, multiple checkpoint outputs, or their combination that's causing issues?
It would be great to get more test cases with checkpoints to prevent regression!

@wharvey31 This looks a lot like the behaviour I also encountered, i.e. input functions returning the directory instead of the files after checkpoint evaluation, see #55 (I just put this here for future reference...)


TS conversion effort for representations related modules

I open this PR to discuss and make suggestions about these conversions and how to best align them with NGL standards!
Comments welcome!

Have found out where the error was. It's now functional!

More conversions, related to surfaces

Hi @ppillot is this ready to be merged?

I think it was. My main concern was about the style and rigourness of the conversion.

I've updated it with the latest changes from master and all checks have passed. I think it is ready for merging!

This is the continuation of the TS conversion effort: this PR includes all the parsers.

All the code has now been converted to TS (except for the polyfills)

These last commits make NGL able to generate a d.ts file.
I have a concern with the code as I had to add imports in modules when a public function in it returns a member that is not defined in that module.
For example, a getAtom() returns an AtomProxy object, then TS needs to have it imported in the module even if it implicitly "knows" the type of the return. The side effect of this import is that it is then considered as an unused import! To prevent from writing the declaration for these functions output (which may be very complex and error prone), I've added dummy exports of types that include those imports.

Firstly, this is really great and much appreciated!

For example, a getAtom() returns an AtomProxy object, then TS needs to have it imported in the module even if it implicitly "knows" the type of the return. The side effect of this import is that it is then considered as an unused import! To prevent from writing the declaration for these functions output (which may be very complex and error prone), I've added dummy exports of types that include those imports.

Do you get the unused import warning if you use AtomProxy as an explicit return type for the function that's returning the AtomProxy?

Do you get the unused import warning if you use AtomProxy as an explicit return type for the function that's returning the AtomProxy?
No, you're correct and as long as the function return type is as simple as that, this is the strategy I've used which is clean.

I've used it when the return type was complex. For example, in cylinder-buffer, one of the returned objects has the definition:
new CylinderGeometryBuffer(data: CylinderBufferData, params?: Partial<{
    radialSegments: number;
    openEnded: boolean;
} & {
    opaqueBack: boolean;
    side: BufferSide;
    opacity: number;
    depthWrite: boolean;
    clipNear: number;
    clipRadius: number;
    clipCenter: Vector3;      // <--- required import
    flatShaded: boolean;
    wireframe: boolean;
    roughness: number;
    metalness: number;
    diffuse: number;
    diffuseInterior: boolean;
    useInteriorColor: boolean;
    interiorColor: number;
    interiorDarkening: number;
    forceTransparent: boolean;
    matrix: Matrix4;             // <--- required import
    disablePicking: boolean;
    sortParticles: boolean;
    background: boolean;
}>): CylinderGeometryBuffer

It required me to import Vector3 and Matrix4 in the module and I did not want to replicate the whole structure as it is (tedious) error prone and not easily extendable if modifications are made to the code.
Maybe the best would be to add and interface in CylinderGeometryBuffer to name the type of this params object

I've just tried it and got the following ts error: error TS4023: Exported variable 'CylinderBufferDefaultParameters' has or is using name 'Vector3' from external module "/Users/... .../ngl/node_modules/@types/three/three-core" but cannot be named.

Hey, thank you so much, this is a great effort!
I merged it and made some changes to the build process to get a bundle. Also, I removed to dummy variables as I did not quite got why the where there (but that might be my ignorance, its in an easily reversible commit), at least I did not get any unused import errors (but I also updated to ts 3.0.1).

You're most welcome!
Have you been able to get a (or a list of) proper declaration file? The dummy imports were there for this reason.
I've tried several options for the declaration(s) file(s) and could not find something entirely satisfactory.
Here is what I've tried

in tsconfig, adding a declaration: true and a declarationDir: "types"
This outputs a declaration file for each module in NGL. I added a types property to package.json. When I consume these declarations file in my own projects, I can import the regular modules from NGL (e.g. Selection, ColorMaker,...) and I can import declarations for other modules returned by these modules such as BitArray or AtomProxy with a syntax such as import BitArray from "../../npm_modules/ngl/utils/bitarray
in a specific ts config file, adding a declaration: true, emitDeclarationOnly: true, outFile: "ngl.d.ts", module: AMD
This outputs a single declaration file containing all the modules from NGL. When I consume this d.ts in my own projects, I can import the regular modules from NGL (e.g. Selection, ColorMaker,...) and I can import declarations for other modules with a syntax such as import BitArray from "utils/bitarray" which seems like a nonsense to me as this doesn't exist. I'd prefer have a reference to this module under ngl namespace.
I've tried the library dts-bundle which should concat the declarations, with no success
I've tried the library dts-generator which under the hood uses the output from the TS compiler. It generates a single file with declarations under a prefix. The problem is that I have to change my import declarations in my project to import {Selection} from "ngl/src/ngl" (I thing I could get rid of the src part, but not of the double ngl...) which means breaking with every existing code relying on this kind of import.

So, it seems to me that option 1 is the best of these, even if it's still not perfect (it means shipping hundreds of declaration file with the npm package) as the imports would have a correct syntax.
What is your opinion?

I've tried to output a d.ts file from the latest code on github (tsc -d --outFile "build/types/ngl.js" --module "system" --emitDeclarationOnly) and I get the same errors as before the dummy imports trick, plus a couple due to webvr (easily fixable):
$ tsc -d --outFile "build/types/ngl.js" --module "system" --emitDeclarationOnly
src/buffer/cylinder-buffer.ts(19,14): error TS4023: Exported variable 'CylinderBufferDefaultParameters' has or is using name 'Matrix4' from external module "/Users/paul0/Sites/ngl-ts2/ngl/node_modules/@types/three/three-core" but cannot be named.
src/buffer/cylinder-buffer.ts(19,14): error TS4023: Exported variable 'CylinderBufferDefaultParameters' has or is using name 'Vector3' from external module "/Users/paul0/Sites/ngl-ts2/ngl/node_modules/@types/three/three-core" but cannot be named.
src/buffer/cylinderimpostor-buffer.ts(16,14): error TS4023: Exported variable 'CylinderImpostorBufferDefaultParameters' has or is using name 'Vector3' from external module "/Users/paul0/Sites/ngl-ts2/ngl/node_modules/@types/three/three-core" but cannot be named.
src/buffer/cylinderimpostor-buffer.ts(39,7): error TS4041: Return type of public getter 'defaultParameters' from exported class has or is using name 'Vector3' from external module "/Users/paul0/Sites/ngl-ts2/ngl/node_modules/@types/three/three-core" but cannot be named.
src/buffer/doublesided-buffer.ts(95,7): error TS4036: Parameter type of public setter 'matrix' from exported class has or is using name 'Matrix4' from private module '"/Users/paul0/Sites/ngl-ts2/ngl/node_modules/@types/three/three-core"'.
src/buffer/doublesided-buffer.ts(106,7): error TS4041: Return type of public getter 'parameters' from exported class has or is using name 'Matrix4' from external module "/Users/paul0/Sites/ngl-ts2/ngl/node_modules/@types/three/three-core" but cannot be named.
src/buffer/doublesided-buffer.ts(106,7): error TS4041: Return type of public getter 'parameters' from exported class has or is using name 'Vector3' from external module "/Users/paul0/Sites/ngl-ts2/ngl/node_modules/@types/three/three-core" but cannot be named.
src/buffer/doublesided-buffer.ts(110,3): error TS4053: Return type of public method from exported class has or is using name 'Matrix4' from external module "/Users/paul0/Sites/ngl-ts2/ngl/node_modules/@types/three/three-core" but cannot be named.
src/buffer/doublesided-buffer.ts(110,3): error TS4053: Return type of public method from exported class has or is using name 'Vector3' from external module "/Users/paul0/Sites/ngl-ts2/ngl/node_modules/@types/three/three-core" but cannot be named.
src/buffer/geometry-buffer.ts(78,34): error TS4061: Parameter 'params' of constructor from exported class has or is using name 'Vector3' from external module "/Users/paul0/Sites/ngl-ts2/ngl/node_modules/@types/three/three-core" but cannot be named.
src/buffer/hyperballstick-buffer.ts(21,14): error TS4023: Exported variable 'HyperballStickBufferDefaultParameters' has or isusing name 'Matrix4' from external module "/Users/paul0/Sites/ngl-ts2/ngl/node_modules/@types/three/three-core" but cannot benamed.
src/buffer/hyperballstick-buffer.ts(21,14): error TS4023: Exported variable 'HyperballStickBufferDefaultParameters' has or isusing name 'Vector3' from external module "/Users/paul0/Sites/ngl-ts2/ngl/node_modules/@types/three/three-core" but cannot benamed.
src/buffer/hyperballstickimpostor-buffer.ts(23,14): error TS4023: Exported variable 'HyperballStickImpostorBufferDefaultParameters' has or is using name 'Vector3' from external module "/Users/paul0/Sites/ngl-ts2/ngl/node_modules/@types/three/three-core" but cannot be named.
src/buffer/hyperballstickimpostor-buffer.ts(47,7): error TS4041: Return type of public getter 'defaultParameters' from exported class has or is using name 'Vector3' from external module "/Users/paul0/Sites/ngl-ts2/ngl/node_modules/@types/three/three-core" but cannot be named.
src/buffer/image-buffer.ts(41,14): error TS4023: Exported variable 'ImageBufferDefaultParameters' has or is using name 'Matrix4' from external module "/Users/paul0/Sites/ngl-ts2/ngl/node_modules/@types/three/three-core" but cannot be named.
src/buffer/image-buffer.ts(41,14): error TS4023: Exported variable 'ImageBufferDefaultParameters' has or is using name 'Vector3' from external module "/Users/paul0/Sites/ngl-ts2/ngl/node_modules/@types/three/three-core" but cannot be named.
src/buffer/image-buffer.ts(56,7): error TS4041: Return type of public getter 'defaultParameters' from exported class has or is using name 'Matrix4' from external module "/Users/paul0/Sites/ngl-ts2/ngl/node_modules/@types/three/three-core" but cannot be named.
src/buffer/image-buffer.ts(56,7): error TS4041: Return type of public getter 'defaultParameters' from exported class has or is using name 'Vector3' from external module "/Users/paul0/Sites/ngl-ts2/ngl/node_modules/@types/three/three-core" but cannot be named.
src/buffer/point-buffer.ts(63,14): error TS4023: Exported variable 'PointBufferDefaultParameters' has or is using name 'Matrix4' from external module "/Users/paul0/Sites/ngl-ts2/ngl/node_modules/@types/three/three-core" but cannot be named.
src/buffer/point-buffer.ts(63,14): error TS4023: Exported variable 'PointBufferDefaultParameters' has or is using name 'Vector3' from external module "/Users/paul0/Sites/ngl-ts2/ngl/node_modules/@types/three/three-core" but cannot be named.
src/buffer/point-buffer.ts(95,7): error TS4041: Return type of public getter 'defaultParameters' from exported class has or is using name 'Matrix4' from external module "/Users/paul0/Sites/ngl-ts2/ngl/node_modules/@types/three/three-core" but cannot be named.
src/buffer/point-buffer.ts(95,7): error TS4041: Return type of public getter 'defaultParameters' from exported class has or is using name 'Vector3' from external module "/Users/paul0/Sites/ngl-ts2/ngl/node_modules/@types/three/three-core" but cannot be named.
src/buffer/sphere-buffer.ts(16,14): error TS4023: Exported variable 'SphereBufferDefaultParameters' has or is using name 'Matrix4' from external module "/Users/paul0/Sites/ngl-ts2/ngl/node_modules/@types/three/three-core" but cannot be named.
src/buffer/sphere-buffer.ts(16,14): error TS4023: Exported variable 'SphereBufferDefaultParameters' has or is using name 'Vector3' from external module "/Users/paul0/Sites/ngl-ts2/ngl/node_modules/@types/three/three-core" but cannot be named.
src/buffer/text-buffer.ts(288,14): error TS4023: Exported variable 'TextBufferDefaultParameters' has or is using name 'Matrix4' from external module "/Users/paul0/Sites/ngl-ts2/ngl/node_modules/@types/three/three-core" but cannot be named.
src/buffer/text-buffer.ts(288,14): error TS4023: Exported variable 'TextBufferDefaultParameters' has or is using name 'Vector3' from external module "/Users/paul0/Sites/ngl-ts2/ngl/node_modules/@types/three/three-core" but cannot be named.
src/buffer/text-buffer.ts(349,7): error TS4041: Return type of public getter 'defaultParameters' from exported class has or is using name 'Matrix4' from external module "/Users/paul0/Sites/ngl-ts2/ngl/node_modules/@types/three/three-core" but cannot be named.
src/buffer/text-buffer.ts(349,7): error TS4041: Return type of public getter 'defaultParameters' from exported class has or is using name 'Vector3' from external module "/Users/paul0/Sites/ngl-ts2/ngl/node_modules/@types/three/three-core" but cannot be named.
src/buffer/tubemesh-buffer.ts(23,14): error TS4023: Exported variable 'TubeMeshBufferDefaultParameters' has or is using name 'Matrix4' from external module "/Users/paul0/Sites/ngl-ts2/ngl/node_modules/@types/three/three-core" but cannot be named.
src/buffer/tubemesh-buffer.ts(55,7): error TS4041: Return type of public getter 'defaultParameters' from exported class has or is using name 'Matrix4' from external module "/Users/paul0/Sites/ngl-ts2/ngl/node_modules/@types/three/three-core" but cannot be named.
src/buffer/vector-buffer.ts(24,14): error TS4023: Exported variable 'VectorBufferDefaultParameters' has or is using name 'Matrix4' from external module "/Users/paul0/Sites/ngl-ts2/ngl/node_modules/@types/three/three-core" but cannot be named.
src/buffer/vector-buffer.ts(24,14): error TS4023: Exported variable 'VectorBufferDefaultParameters' has or is using name 'Vector3' from external module "/Users/paul0/Sites/ngl-ts2/ngl/node_modules/@types/three/three-core" but cannot be named.
src/buffer/vector-buffer.ts(34,7): error TS4041: Return type of public getter 'defaultParameters' from exported class has or is using name 'Matrix4' from external module "/Users/paul0/Sites/ngl-ts2/ngl/node_modules/@types/three/three-core" but cannot be named.
src/buffer/vector-buffer.ts(34,7): error TS4041: Return type of public getter 'defaultParameters' from exported class has or is using name 'Vector3' from external module "/Users/paul0/Sites/ngl-ts2/ngl/node_modules/@types/three/three-core" but cannot be named.
src/buffer/wideline-buffer.ts(22,14): error TS4023: Exported variable 'WideLineBufferDefaultParameters' has or is using name 'Vector3' from external module "/Users/paul0/Sites/ngl-ts2/ngl/node_modules/@types/three/three-core" but cannot be named.
src/buffer/wideline-buffer.ts(44,7): error TS4041: Return type of public getter 'defaultParameters' from exported class has or is using name 'Vector3' from external module "/Users/paul0/Sites/ngl-ts2/ngl/node_modules/@types/three/three-core" but cannot be named.
src/geometry/shape.ts(28,14): error TS4023: Exported variable 'ShapeDefaultParameters' has or is using name 'Matrix4' from external module "/Users/paul0/Sites/ngl-ts2/ngl/node_modules/@types/three/three-core" but cannot be named.
src/parser/xml-parser.ts(41,3): error TS4053: Return type of public method from exported class has or is using name 'XMLNode'from external module "/Users/paul0/Sites/ngl-ts2/ngl/src/utils/parse-xml" but cannot be named.
src/representation/ballandstick-representation.ts(152,3): error TS4053: Return type of public method from exported class has or is using name 'Surface' from external module "/Users/paul0/Sites/ngl-ts2/ngl/src/surface/surface" but cannot be named.
src/representation/ballandstick-representation.ts(152,3): error TS4053: Return type of public method from exported class has or is using name 'Volume' from external module "/Users/paul0/Sites/ngl-ts2/ngl/src/surface/volume" but cannot be named.
src/representation/ballandstick-representation.ts(163,3): error TS4053: Return type of public method from exported class has or is using name 'Surface' from external module "/Users/paul0/Sites/ngl-ts2/ngl/src/surface/surface" but cannot be named.
src/representation/ballandstick-representation.ts(163,3): error TS4053: Return type of public method from exported class has or is using name 'Volume' from external module "/Users/paul0/Sites/ngl-ts2/ngl/src/surface/volume" but cannot be named.
src/representation/contact-representation.ts(238,3): error TS4053: Return type of public method from exported class has or isusing name 'ContactPicker' from external module "/Users/paul0/Sites/ngl-ts2/ngl/src/utils/picker" but cannot be named.
src/representation/hyperball-representation.ts(62,3): error TS4053: Return type of public method from exported class has or is using name 'Surface' from external module "/Users/paul0/Sites/ngl-ts2/ngl/src/surface/surface" but cannot be named.
src/representation/hyperball-representation.ts(62,3): error TS4053: Return type of public method from exported class has or is using name 'Volume' from external module "/Users/paul0/Sites/ngl-ts2/ngl/src/surface/volume" but cannot be named.
src/representation/line-representation.ts(152,3): error TS4053: Return type of public method from exported class has or is using name 'BondDataFields' from external module "/Users/paul0/Sites/ngl-ts2/ngl/src/structure/structure-data" but cannot be named.
src/representation/line-representation.ts(152,3): error TS4053: Return type of public method from exported class has or is using name 'BondDataParams' from external module "/Users/paul0/Sites/ngl-ts2/ngl/src/structure/structure-data" but cannot be named.
src/representation/line-representation.ts(152,3): error TS4053: Return type of public method from exported class has or is using name 'Surface' from external module "/Users/paul0/Sites/ngl-ts2/ngl/src/surface/surface" but cannot be named.
src/representation/line-representation.ts(152,3): error TS4053: Return type of public method from exported class has or is using name 'Volume' from external module "/Users/paul0/Sites/ngl-ts2/ngl/src/surface/volume" but cannot be named.
src/representation/measurement-representation.ts(290,3): error TS4053: Return type of public method from exported class has or is using name 'Matrix4' from external module "/Users/paul0/Sites/ngl-ts2/ngl/node_modules/@types/three/three-core" but cannot be named.
src/representation/measurement-representation.ts(290,3): error TS4053: Return type of public method from exported class has or is using name 'Vector3' from external module "/Users/paul0/Sites/ngl-ts2/ngl/node_modules/@types/three/three-core" but cannot be named.
src/representation/molecularsurface-representation.ts(341,3): error TS4053: Return type of public method from exported class has or is using name 'Volume' from external module "/Users/paul0/Sites/ngl-ts2/ngl/src/surface/volume" but cannot be named.
src/representation/structure-representation.ts(269,3): error TS4053: Return type of public method from exported class has or is using name 'Surface' from external module "/Users/paul0/Sites/ngl-ts2/ngl/src/surface/surface" but cannot be named.
src/representation/structure-representation.ts(269,3): error TS4053: Return type of public method from exported class has or is using name 'Volume' from external module "/Users/paul0/Sites/ngl-ts2/ngl/src/surface/volume" but cannot be named.
src/representation/structure-representation.ts(285,3): error TS4053: Return type of public method from exported class has or is using name 'Surface' from external module "/Users/paul0/Sites/ngl-ts2/ngl/src/surface/surface" but cannot be named.
src/representation/structure-representation.ts(285,3): error TS4053: Return type of public method from exported class has or is using name 'Volume' from external module "/Users/paul0/Sites/ngl-ts2/ngl/src/surface/volume" but cannot be named.
src/representation/structure-representation.ts(293,3): error TS4053: Return type of public method from exported class has or is using name 'Surface' from external module "/Users/paul0/Sites/ngl-ts2/ngl/src/surface/surface" but cannot be named.
src/representation/structure-representation.ts(293,3): error TS4053: Return type of public method from exported class has or is using name 'Volume' from external module "/Users/paul0/Sites/ngl-ts2/ngl/src/surface/volume" but cannot be named.
src/representation/surface-representation.ts(387,3): error TS4053: Return type of public method from exported class has or isusing name 'ColormakerParameters' from external module "/Users/paul0/Sites/ngl-ts2/ngl/src/color/colormaker" but cannot be named.
src/representation/unitcell-representation.ts(69,3): error TS4053: Return type of public method from exported class has or isusing name 'UnitcellPicker' from external module "/Users/paul0/Sites/ngl-ts2/ngl/src/utils/picker" but cannot be named.
types/webvr-api/index.d.ts(136,5): error TS2717: Subsequent property declarations must have the same type.  Property 'leftBounds' must be of type 'number[] | null | undefined', but here has type 'Float32Array | number[] | null | undefined'.
types/webvr-api/index.d.ts(137,5): error TS2717: Subsequent property declarations must have the same type.  Property 'rightBounds' must be of type 'number[] | null | undefined', but here has type 'Float32Array | number[] | null | undefined'.

I'm interested in this as well. I tried tsc -d --outfile ngl.esm.js --module system -emitDeclarationOnly and it does generate a .d.ts (with some errors), but it has some problems when used, such as relative import names which are not allowed in an ambient module declaration. I then tried @ppillot 's option 1 (as best I could) and got something that looks OK (I copied the generated types folder into dist), but it does have many errors. @ppillot do you have a PR or a patch with where you've gotten to with your option 1? That seems like the best way to go. I'd be happy to collaborate on this.
Also I'm integrating with Three.js, so I will probably need to build my own NGL with the same version of Three I'm using anyway (it's under active development and I can already see some errors due to recent changes).

Hi @garyo , I have not touched this for a few months now... I think the further I got was with my branch dts3 back in September https://github.com/ppillot/ngl/tree/dts3
I now think there is a concern with the way the module is compiled: the system parameter used in the command line does not produce a regular ES6 module (but a SystemJS compliant one instead), but it was the only way I had found to produce a single file module. Declarations files instead are output in separate files on a per module basis as you had noticed.
I think that ultimately the goal would be to output proper ES6 modules so that they can be loaded directly in compliant browsers and also so that one can do some tree shaking when using a bundler such as WebPack. I am not sure about how that would go with your objective of perusing a specific three.js version....

OK, thanks -- I'm trying your dts3 branch now.
One small weirdness: when running a regular build on your branch, I had to hand-modify<EMAIL_ADDRESS>(I know, not even part of this package) to fix typescript errors. Not sure what's up with that. Although when I installed the latest three 0.103.0 (which has full type decls) I could uninstall @types/three, so we don't need @types/webvr-api anymore -- so that's good. But that introduces some incompatibilities I'll look into fixing. Nothing too serious...

I have a version that works for me now, based on three 0.103.0, with complete types and no errors in my app! You can browse it at https://github.com/garyo/ngl/commit/a462b2e -- I'd appreciate any feedback.
Also I agree with you about creating ES6 modules -- right now built ngl includes a whole copy of three.js for instance. I'm using webpack so modules would be way better for me.

@ppillot, have you submitted a PR for your https://github.com/ppillot/ngl/tree/dts3 branch? I need it (since I'm using typescript) and as noted above, it is working fine for me, generating full type decls.
I have a PR pending to update NGL to three.js r103 (which I need for other reasons, but NGL and my code have to agree on the three.js version -- I'll probably update it to support the most recent r106 before submitting it here), but it depends on that dts3 branch.


iOS: ArgumentError (Invalid argument(s): Failed to load dynamic library 'sqlite3.framework/sqlite3': dlopen(sqlite3.framework/sqlite3, 0x0001)

I don't have a reproductible example (yet) but here is the issue:
I'm using drift 2.0.1 on a simulator iOS I get the error
ArgumentError (Invalid argument(s): Failed to load dynamic library 'sqlite3.framework/sqlite3': dlopen(sqlite3.framework/sqlite3, 0x0001): tried: '/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Library/Developer/CoreSimulator/Profiles/Runtimes/iOS.simruntime/Contents/Resources/RuntimeRootsqlite3.framework/sqlite3' (no such file), '/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Library/Developer/CoreSimulator/Profiles/Runtimes/iOS.simruntime/Contents/Resources/RuntimeRoot/usr/lib/swift/sqlite3.framework/sqlite3' (no such file), '/usr/lib/swift/sqlite3.framework/sqlite3' (no such file), '/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Library/Developer/CoreSimulator/Profiles/Runtimes/iOS.simruntime/Contents/Resources/RuntimeRoot/usr/lib/swift/sqlite3.framework/sqlite3' (no such file), '/usr/lib/swift/sqlite3.framework/sqlite3' (no such file), '/Users/valentin/Library/Developer/CoreSimulator/Devices/6D1E9DF8-540C-4307-89E7-FC2D14CF27B8/data/Containers/Bundle/Application/7645EBEB-B99A-49D7-9FE3-BD0280DE5A0D/Runner.app/Frameworks/sqlite3.framework/sqlite3' (no such file), '/Users/valentin/Library/Developer/CoreSimulator/Devices/6D1E9DF8-540C-4307-89E7-FC2D14CF27B8/data/Containers/Bundle/Application/7645EBEB-B99A-49D7-9FE3-BD0280DE5A0D/Runner.app/Frameworks/sqlite3.framework/sqlite3' (no such file), '/Users/valentin/Library/Developer/CoreSimulator/Devices/6D1E9DF8-540C-4307-89E7-FC2D14CF27B8/data/Containers/Bundle/Application/7645EBEB-B99A-49D7-9FE3-BD0280DE5A0D/Runner.app/Frameworks/sqlite3.framework/sqlite3' (no such file), 'sqlite3.framework/sqlite3' (no such file), '/System/Library/Frameworks/sqlite3.framework/sqlite3' (no such file))

The calls tack:
_open (dart:ffi-patch/ffi_dynamic_library_patch.dart:12)
new DynamicLibrary.open (dart:ffi-patch/ffi_dynamic_library_patch.dart:23)
_defaultOpen (/Users/valentin/.pub-cache/hosted/pub.dartlang.org/sqlite3-1.7.2/lib/src/ffi/load_library.dart:55)
OpenDynamicLibrary.openSqlite (/Users/valentin/.pub-cache/hosted/pub.dartlang.org/sqlite3-1.7.2/lib/src/ffi/load_library.dart:127)
sqlite3 (/Users/valentin/.pub-cache/hosted/pub.dartlang.org/sqlite3-1.7.2/lib/src/ffi/api/sqlite3.dart:11)
_NativeDelegate.openDatabase (/Users/valentin/.pub-cache/hosted/pub.dartlang.org/drift-2.0.1/lib/native.dart:155)
Sqlite3Delegate._createDatabase (/Users/valentin/.pub-cache/hosted/pub.dartlang.org/drift-2.0.1/lib/src/sqlite3/database.dart:73)
Sqlite3Delegate.open (/Users/valentin/.pub-cache/hosted/pub.dartlang.org/drift-2.0.1/lib/src/sqlite3/database.dart:61)
DelegatedDatabase.ensureOpen.<anonymous closure> (/Users/valentin/.pub-cache/hosted/pub.dartlang.org/drift-2.0.1/lib/src/runtime/executor/helpers/engines.dart:415)
<asynchronous gap> (Unknown Source:0)
ServerImplementation._handleEnsureOpen (/Users/valentin/.pub-cache/hosted/pub.dartlang.org/drift-2.0.1/lib/src/remote/server_impl.dart:114)
<asynchronous gap> (Unknown Source:0)
DriftCommunication.setRequestHandler.<anonymous closure>.<anonymous closure> (/Users/valentin/.pub-cache/hosted/pub.dartlang.org/drift-2.0.1/lib/src/remote/communication.dart:0)
<asynchronous gap> (Unknown Source:0)

And just to be sure, you have a dependency on sqlite3_flutter_libs? Can you post the versions of sqlite3_flutter_libs and sqlite3 that are present in your pubspec.lock?

I have

sqilte3 : 1.7.2

But I don't have sqlite3_flutter_libs
Should I add sqlite3_flutter_libs manually in my pubspecs?

sigh I see it is said in the doc to do so https://drift.simonbinder.eu/docs/getting-started/ ...
I upgraded from 1.5.0 to 2.0.1 and it was working on 1.5.0 without sqlite3_flutter_libs. I didn't
And I didn't see anything about it in the changelogs and the migration documentation

Any luck after adding that package? I'm surprised to hear it ever worked without sqlite3_flutter_libs, from a quick check I don't see anything that could have changed here.

FWIW running flutter test integration_test in this directory works for me. If adding sqlite3_flutter_libs doesn't fix the problem for you, it would be interesting to know whether those tests pass for you or if they're broken as well.

I looks like it works now, thank you ! I'm closing this. Thank you for the support!


Snapcamera not getting web lenses

Does it work with a snapcamera clean install(from your proton drive with no previous cache)?
I followed your steps. I have the server up and runnig with certificates, edited hosts file, patched exe to access web lens but the main lenses panel remains empty with a permanent loading spinner like it would without all this.
On the other hand, in the local server database, I can see like two hundred something lenses. Is there any easy way to troubleshoot this?

If you don't see any snap lenses at all then you probably have blocked the domain studio-app.snapchat.com in your firewall.

Nope. Firewall is off.
PS C:\WINDOWS\system32> Get-NetFirewallProfile

Name                            : Domain
Enabled                         : False
DefaultInboundAction            : NotConfigured
DefaultOutboundAction           : NotConfigured
AllowInboundRules               : NotConfigured
AllowLocalFirewallRules         : NotConfigured
AllowLocalIPsecRules            : NotConfigured
AllowUserApps                   : NotConfigured
AllowUserPorts                  : NotConfigured
AllowUnicastResponseToMulticast : NotConfigured
NotifyOnListen                  : True
EnableStealthModeForIPsec       : NotConfigured
LogFileName                     : %systemroot%\system32\LogFiles\Firewall\pfirewall.log
LogMaxSizeKilobytes             : 4096
LogAllowed                      : False
LogBlocked                      : False
LogIgnored                      : NotConfigured
DisabledInterfaceAliases        : {NotConfigured}

Name                            : Private
Enabled                         : False
DefaultInboundAction            : NotConfigured
DefaultOutboundAction           : NotConfigured
AllowInboundRules               : NotConfigured
AllowLocalFirewallRules         : NotConfigured
AllowLocalIPsecRules            : NotConfigured
AllowUserApps                   : NotConfigured
AllowUserPorts                  : NotConfigured
AllowUnicastResponseToMulticast : NotConfigured
NotifyOnListen                  : True
EnableStealthModeForIPsec       : NotConfigured
LogFileName                     : %systemroot%\system32\LogFiles\Firewall\pfirewall.log
LogMaxSizeKilobytes             : 4096
LogAllowed                      : False
LogBlocked                      : False
LogIgnored                      : NotConfigured
DisabledInterfaceAliases        : {NotConfigured}

Name                            : Public
Enabled                         : False
DefaultInboundAction            : NotConfigured
DefaultOutboundAction           : NotConfigured
AllowInboundRules               : NotConfigured
AllowLocalFirewallRules         : NotConfigured
AllowLocalIPsecRules            : NotConfigured
AllowUserApps                   : NotConfigured
AllowUserPorts                  : NotConfigured
AllowUnicastResponseToMulticast : NotConfigured
NotifyOnListen                  : True
EnableStealthModeForIPsec       : NotConfigured
LogFileName                     : %systemroot%\system32\LogFiles\Firewall\pfirewall.log
LogMaxSizeKilobytes             : 4096
LogAllowed                      : False
LogBlocked                      : False
LogIgnored                      : NotConfigured
DisabledInterfaceAliases        : {NotConfigured}

and hosts file is redirected to localhost
PS C:\WINDOWS\system32> ping studio-app.snapchat.com

Haciendo ping a studio-app.snapchat.com [<IP_ADDRESS>] con 32 bytes de datos:
Respuesta desde <IP_ADDRESS>: bytes=32 tiempo<1m TTL=128
Respuesta desde <IP_ADDRESS>: bytes=32 tiempo<1m TTL=128
Respuesta desde <IP_ADDRESS>: bytes=32 tiempo<1m TTL=128
Respuesta desde <IP_ADDRESS>: bytes=32 tiempo<1m TTL=128

Estadísticas de ping para <IP_ADDRESS>:
    Paquetes: enviados = 4, recibidos = 4, perdidos = 0
    (0% perdidos),
Tiempos aproximados de ida y vuelta en milisegundos:
    Mínimo = 0ms, Máximo = 0ms, Media = 0ms
PS C:\WINDOWS\system32>

Did you check Docker Desktop if all 4 services are running?
There might be another application/server blocking a port.

And you are not seeing any lenses at all?
Did you try to search for anything?
And can you visit http://localhost:80 in your browser?
If you click on snap3 you should be able to see all log entries.

as you can see, no search option whatsoever. Maybe the search option only appears once the lenses are loaded
snap3-webproxy-1  | /docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
snap3-adminer-1   | [Wed Mar  8 20:19:58 2023] PHP 7.4.33 Development Server (http://[::]:8080) started
snap3-webproxy-1  | /docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
snap3-webproxy-1  | /docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
snap3-webproxy-1  | 10-listen-on-ipv6-by-default.sh: info: Getting the checksum of /etc/nginx/conf.d/default.conf
snap3-webproxy-1  | 10-listen-on-ipv6-by-default.sh: info: /etc/nginx/conf.d/default.conf differs from the packaged version
snap3-webproxy-1  | /docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
snap3-webproxy-1  | 20-envsubst-on-templates.sh: Running envsubst on /etc/nginx/templates/default.conf.template to /etc/nginx/conf.d/default.conf
snap3-webproxy-1  | /docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh
snap3-webproxy-1  | /docker-entrypoint.sh: Configuration complete; ready for start up
snap3-webproxy-1  | 2023/03/08 20:19:58 [notice] 1#1: using the "epoll" event method
snap3-webproxy-1  | 2023/03/08 20:19:58 [notice] 1#1: nginx/1.23.3
snap3-webproxy-1  | 2023/03/08 20:19:58 [notice] 1#1: built by gcc 10.2.1 20210110 (Debian 10.2.1-6)
snap3-webproxy-1  | 2023/03/08 20:19:58 [notice] 1#1: OS: Linux <IP_ADDRESS>-microsoft-standard-WSL2
snap3-webproxy-1  | 2023/03/08 20:19:58 [notice] 1#1: getrlimit(RLIMIT_NOFILE): 1048576:1048576
snap3-webproxy-1  | 2023/03/08 20:19:58 [notice] 1#1: start worker processes
snap3-webproxy-1  | 2023/03/08 20:19:58 [notice] 1#1: start worker process 33
snap3-webproxy-1  | 2023/03/08 20:19:58 [notice] 1#1: start worker process 34
snap3-webproxy-1  | 2023/03/08 20:19:58 [notice] 1#1: start worker process 35
snap3-webproxy-1  | 2023/03/08 20:19:58 [notice] 1#1: start worker process 36
snap3-webproxy-1  | 2023/03/08 20:19:58 [notice] 1#1: start worker process 37
snap3-webproxy-1  | 2023/03/08 20:19:58 [notice] 1#1: start worker process 38
snap3-webproxy-1  | 2023/03/08 20:19:58 [notice] 1#1: start worker process 39
snap3-webproxy-1  | 2023/03/08 20:19:58 [notice] 1#1: start worker process 40
snap3-mysql-1     | 2023-03-08 20:19:57+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 5.7.41-1.el7 started.
snap3-mysql-1     | 2023-03-08 20:19:57+00:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql'
snap3-mysql-1     | 2023-03-08 20:19:57+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 5.7.41-1.el7 started.
snap3-mysql-1     | '/var/lib/mysql/mysql.sock' -> '/var/run/mysqld/mysqld.sock'
snap3-mysql-1     | 2023-03-08T20:19:57.876486Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
snap3-mysql-1     | 2023-03-08T20:19:57.878281Z 0 [Note] mysqld (mysqld 5.7.41) starting as process 1 ...
snap3-mysql-1     | 2023-03-08T20:19:57.883445Z 0 [Note] InnoDB: PUNCH HOLE support available
snap3-mysql-1     | 2023-03-08T20:19:57.883494Z 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
snap3-mysql-1     | 2023-03-08T20:19:57.883504Z 0 [Note] InnoDB: Uses event mutexes
snap3-mysql-1     | 2023-03-08T20:19:57.883513Z 0 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
snap3-mysql-1     | 2023-03-08T20:19:57.883519Z 0 [Note] InnoDB: Compressed tables use zlib 1.2.12
snap3-mysql-1     | 2023-03-08T20:19:57.883524Z 0 [Note] InnoDB: Using Linux native AIO
snap3-mysql-1     | 2023-03-08T20:19:57.883819Z 0 [Note] InnoDB: Number of pools: 1
snap3-mysql-1     | 2023-03-08T20:19:57.883983Z 0 [Note] InnoDB: Using CPU crc32 instructions
snap3-mysql-1     | 2023-03-08T20:19:57.885841Z 0 [Note] InnoDB: Initializing buffer pool, total size = 128M, instances = 1, chunk size = 128M
snap3-mysql-1     | 2023-03-08T20:19:57.899938Z 0 [Note] InnoDB: Completed initialization of buffer pool
snap3-mysql-1     | 2023-03-08T20:19:57.902968Z 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
snap3-mysql-1     | 2023-03-08T20:19:57.914818Z 0 [Note] InnoDB: Highest supported file format is Barracuda.
snap3-mysql-1     | 2023-03-08T20:19:57.930744Z 0 [Note] InnoDB: Creating shared tablespace for temporary tables
snap3-mysql-1     | 2023-03-08T20:19:57.930834Z 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
snap3-mysql-1     | 2023-03-08T20:19:57.968528Z 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
snap3-mysql-1     | 2023-03-08T20:19:57.969558Z 0 [Note] InnoDB: 96 redo rollback segment(s) found. 96 redo rollback segment(s) are active.
snap3-mysql-1     | 2023-03-08T20:19:57.969600Z 0 [Note] InnoDB: 32 non-redo rollback segment(s) are active.
snap3-mysql-1     | 2023-03-08T20:19:57.971024Z 0 [Note] InnoDB: 5.7.41 started; log sequence number 15516138
snap3-mysql-1     | 2023-03-08T20:19:57.971386Z 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
snap3-mysql-1     | 2023-03-08T20:19:57.971751Z 0 [Note] Plugin 'FEDERATED' is disabled.
snap3-mysql-1     | 2023-03-08T20:19:57.974183Z 0 [Note] InnoDB: Buffer pool(s) load completed at 230308 20:19:57
snap3-mysql-1     | 2023-03-08T20:19:57.977571Z 0 [Note] Found ca.pem, server-cert.pem and server-key.pem in data directory. Trying to enable SSL support using them.
snap3-mysql-1     | 2023-03-08T20:19:57.977626Z 0 [Note] Skipping generation of SSL certificates as certificate files are present in data directory.
snap3-mysql-1     | 2023-03-08T20:19:57.977636Z 0 [Warning] A deprecated TLS version TLSv1 is enabled. Please use TLSv1.2 or higher.
snap3-mysql-1     | 2023-03-08T20:19:57.977640Z 0 [Warning] A deprecated TLS version TLSv1.1 is enabled. Please use TLSv1.2 or higher.
snap3-mysql-1     | 2023-03-08T20:19:57.979157Z 0 [Warning] CA certificate ca.pem is self signed.
snap3-mysql-1     | 2023-03-08T20:19:57.979628Z 0 [Note] Skipping generation of RSA key pair as key files are present in data directory.
snap3-mysql-1     | 2023-03-08T20:19:57.980412Z 0 [Note] Server hostname (bind-address): '*'; port: 3306
snap3-mysql-1     | 2023-03-08T20:19:57.980689Z 0 [Note] IPv6 is available.
snap3-mysql-1     | 2023-03-08T20:19:57.980812Z 0 [Note]   - '::' resolves to '::';
snap3-mysql-1     | 2023-03-08T20:19:57.980863Z 0 [Note] Server socket created on IP: '::'.
snap3-mysql-1     | 2023-03-08T20:19:57.984143Z 0 [Warning] Insecure configuration for --pid-file: Location '/var/run/mysqld' in the path is accessible to all OS users. Consider choosing a different directory.
snap3-mysql-1     | 2023-03-08T20:19:57.996830Z 0 [Note] Event Scheduler: Loaded 0 events
snap3-mysql-1     | 2023-03-08T20:19:57.997693Z 0 [Note] mysqld: ready for connections.
snap3-mysql-1     | Version: '5.7.41'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306  MySQL Community Server (GPL)
snap3-webapp-1    | Snap Camera Server is running on port 5645
snap3-webapp-1    | Initialization delayed by 30 seconds.
snap3-webapp-1    | Ignoring invalid configuration option passed to Connection: driver. This is currently a warning, but in future versions of MySQL2, an error will be thrown if you pass an invalid configuration option to a Connection
snap3-webapp-1    | [INFO] [migration] Nothing to run
snap3-webapp-1    | [INFO] Done
snap3-webapp-1    | Database migration complete
snap3-webapp-1    | Initialization complete! 🎉

I've restarted the server just now to have a clean log.
I can access localhost:80 alright. I get the docker "getting started" landing page

Getting started landing page?
What do you mean? That doesn't sound right...
You should see connections incomming in your logs if you visit http://localhost:80
And http://localhost:80 should look like this

yes, sorry. That was cached by the browser from docker's example container.
This is what I get:

Do you see incoming requests in your log?
Also close Snap Camera and restart the application.
You should see incomming requests in your log.
If there is no incomming logs you need to check your /etc/hosts file

  TCP    [::1]:80               mycomputer:55650         ESTABLISHED
  TCP    [::1]:80               mycomputer:55651         ESTABLISHED
  TCP    [::1]:55650            mycomputer:http          ESTABLISHED
  TCP    [::1]:55651            mycomputer:http          ESTABLISHED

those are the only ones I found. And my hosts file as follows:
# Copyright (c) 1993-2009 Microsoft Corp.
#
# This is a sample HOSTS file used by Microsoft TCP/IP for Windows.
#
# This file contains the mappings of IP addresses to host names. Each
# entry should be kept on an individual line. The IP address should
# be placed in the first column followed by the corresponding host name.
# The IP address and the host name should be separated by at least one
# space.
#
# Additionally, comments (such as these) may be inserted on individual
# lines or following the machine name denoted by a '#' symbol.
#
# For example:
#
#      <IP_ADDRESS>     rhino.acme.com          # source server
#       <IP_ADDRESS>     x.acme.com              # x client host

# localhost name resolution is handled within DNS itself.
#	<IP_ADDRESS>       localhost
#	::1             localhost

# Added by Snap Camera Server Auto Config
<IP_ADDRESS>       studio-app.snapchat.com
# Added by Docker Desktop
<IP_ADDRESS> host.docker.internal
<IP_ADDRESS> gateway.docker.internal
# To allow the same kube context to work on the host and the container:
<IP_ADDRESS> kubernetes.docker.internal
# End of section

Do you see incoming requests in your log?

I've tried several times but no.
snap3-webproxy-1  | /docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
snap3-mysql-1     | 2023-03-08 21:31:34+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 5.7.41-1.el7 started.
snap3-adminer-1   | [Wed Mar  8 21:31:35 2023] PHP 7.4.33 Development Server (http://[::]:8080) started
snap3-webproxy-1  | /docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
snap3-mysql-1     | 2023-03-08 21:31:35+00:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql'
snap3-webproxy-1  | /docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
snap3-webproxy-1  | 10-listen-on-ipv6-by-default.sh: info: Getting the checksum of /etc/nginx/conf.d/default.conf
snap3-mysql-1     | 2023-03-08 21:31:35+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 5.7.41-1.el7 started.
snap3-webproxy-1  | 10-listen-on-ipv6-by-default.sh: info: /etc/nginx/conf.d/default.conf differs from the packaged version
snap3-webproxy-1  | /docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
snap3-mysql-1     | '/var/lib/mysql/mysql.sock' -> '/var/run/mysqld/mysqld.sock'
snap3-mysql-1     | 2023-03-08T21:31:35.540839Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
snap3-webproxy-1  | 20-envsubst-on-templates.sh: Running envsubst on /etc/nginx/templates/default.conf.template to /etc/nginx/conf.d/default.conf
snap3-webproxy-1  | /docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh
snap3-webproxy-1  | /docker-entrypoint.sh: Configuration complete; ready for start up
snap3-webproxy-1  | 2023/03/08 21:31:36 [notice] 1#1: using the "epoll" event method
snap3-mysql-1     | 2023-03-08T21:31:35.542550Z 0 [Note] mysqld (mysqld 5.7.41) starting as process 1 ...
snap3-webproxy-1  | 2023/03/08 21:31:36 [notice] 1#1: nginx/1.23.3
snap3-mysql-1     | 2023-03-08T21:31:35.548486Z 0 [Note] InnoDB: PUNCH HOLE support available
snap3-webproxy-1  | 2023/03/08 21:31:36 [notice] 1#1: built by gcc 10.2.1 20210110 (Debian 10.2.1-6)
snap3-webproxy-1  | 2023/03/08 21:31:36 [notice] 1#1: OS: Linux <IP_ADDRESS>-microsoft-standard-WSL2
snap3-webproxy-1  | 2023/03/08 21:31:36 [notice] 1#1: getrlimit(RLIMIT_NOFILE): 1048576:1048576
snap3-mysql-1     | 2023-03-08T21:31:35.548555Z 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
snap3-mysql-1     | 2023-03-08T21:31:35.548564Z 0 [Note] InnoDB: Uses event mutexes
snap3-mysql-1     | 2023-03-08T21:31:35.548569Z 0 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
snap3-webproxy-1  | 2023/03/08 21:31:36 [notice] 1#1: start worker processes
snap3-webproxy-1  | 2023/03/08 21:31:36 [notice] 1#1: start worker process 33
snap3-webproxy-1  | 2023/03/08 21:31:36 [notice] 1#1: start worker process 34
snap3-mysql-1     | 2023-03-08T21:31:35.548574Z 0 [Note] InnoDB: Compressed tables use zlib 1.2.12
snap3-mysql-1     | 2023-03-08T21:31:35.548579Z 0 [Note] InnoDB: Using Linux native AIO
snap3-mysql-1     | 2023-03-08T21:31:35.548930Z 0 [Note] InnoDB: Number of pools: 1
snap3-mysql-1     | 2023-03-08T21:31:35.549198Z 0 [Note] InnoDB: Using CPU crc32 instructions
snap3-webproxy-1  | 2023/03/08 21:31:36 [notice] 1#1: start worker process 35
snap3-webproxy-1  | 2023/03/08 21:31:36 [notice] 1#1: start worker process 36
snap3-webproxy-1  | 2023/03/08 21:31:36 [notice] 1#1: start worker process 37
snap3-mysql-1     | 2023-03-08T21:31:35.552261Z 0 [Note] InnoDB: Initializing buffer pool, total size = 128M, instances = 1, chunk size = 128M
snap3-webproxy-1  | 2023/03/08 21:31:36 [notice] 1#1: start worker process 38
snap3-mysql-1     | 2023-03-08T21:31:35.578706Z 0 [Note] InnoDB: Completed initialization of buffer pool
snap3-mysql-1     | 2023-03-08T21:31:35.584954Z 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
snap3-webproxy-1  | 2023/03/08 21:31:36 [notice] 1#1: start worker process 39
snap3-webproxy-1  | 2023/03/08 21:31:36 [notice] 1#1: start worker process 40
snap3-mysql-1     | 2023-03-08T21:31:35.597220Z 0 [Note] InnoDB: Highest supported file format is Barracuda.
snap3-mysql-1     | 2023-03-08T21:31:35.615502Z 0 [Note] InnoDB: Creating shared tablespace for temporary tables
snap3-mysql-1     | 2023-03-08T21:31:35.615615Z 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
snap3-mysql-1     | 2023-03-08T21:31:35.692145Z 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
snap3-mysql-1     | 2023-03-08T21:31:35.692824Z 0 [Note] InnoDB: 96 redo rollback segment(s) found. 96 redo rollback segment(s) are active.
snap3-mysql-1     | 2023-03-08T21:31:35.692855Z 0 [Note] InnoDB: 32 non-redo rollback segment(s) are active.
snap3-mysql-1     | 2023-03-08T21:31:35.693839Z 0 [Note] InnoDB: 5.7.41 started; log sequence number 15711950
snap3-mysql-1     | 2023-03-08T21:31:35.694220Z 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
snap3-mysql-1     | 2023-03-08T21:31:35.694517Z 0 [Note] Plugin 'FEDERATED' is disabled.
snap3-mysql-1     | 2023-03-08T21:31:35.696655Z 0 [Note] InnoDB: Buffer pool(s) load completed at 230308 21:31:35
snap3-mysql-1     | 2023-03-08T21:31:35.700784Z 0 [Note] Found ca.pem, server-cert.pem and server-key.pem in data directory. Trying to enable SSL support using them.
snap3-mysql-1     | 2023-03-08T21:31:35.700831Z 0 [Note] Skipping generation of SSL certificates as certificate files are present in data directory.
snap3-mysql-1     | 2023-03-08T21:31:35.700839Z 0 [Warning] A deprecated TLS version TLSv1 is enabled. Please use TLSv1.2 or higher.
snap3-mysql-1     | 2023-03-08T21:31:35.700842Z 0 [Warning] A deprecated TLS version TLSv1.1 is enabled. Please use TLSv1.2 or higher.
snap3-mysql-1     | 2023-03-08T21:31:35.701575Z 0 [Warning] CA certificate ca.pem is self signed.
snap3-mysql-1     | 2023-03-08T21:31:35.701650Z 0 [Note] Skipping generation of RSA key pair as key files are present in data directory.
snap3-mysql-1     | 2023-03-08T21:31:35.702012Z 0 [Note] Server hostname (bind-address): '*'; port: 3306
snap3-mysql-1     | 2023-03-08T21:31:35.702072Z 0 [Note] IPv6 is available.
snap3-mysql-1     | 2023-03-08T21:31:35.702089Z 0 [Note]   - '::' resolves to '::';
snap3-mysql-1     | 2023-03-08T21:31:35.702111Z 0 [Note] Server socket created on IP: '::'.
snap3-mysql-1     | 2023-03-08T21:31:35.707195Z 0 [Warning] Insecure configuration for --pid-file: Location '/var/run/mysqld' in the path is accessible to all OS users. Consider choosing a different directory.
snap3-mysql-1     | 2023-03-08T21:31:35.716500Z 0 [Note] Event Scheduler: Loaded 0 events
snap3-mysql-1     | 2023-03-08T21:31:35.716787Z 0 [Note] mysqld: ready for connections.
snap3-mysql-1     | Version: '5.7.41'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306  MySQL Community Server (GPL)
snap3-webapp-1    | Snap Camera Server is running on port 5645
snap3-webapp-1    | Initialization delayed by 30 seconds.
snap3-webapp-1    | Ignoring invalid configuration option passed to Connection: driver. This is currently a warning, but in future versions of MySQL2, an error will be thrown if you pass an invalid configuration option to a Connection
snap3-webapp-1    | [INFO] [migration] Nothing to run
snap3-webapp-1    | [INFO] Done
snap3-webapp-1    | Database migration complete
snap3-webapp-1    | Initialization complete! 🎉

This is all I get after restarting snapcamera. I guess I should see some here right?

Ok. It might be a docker issue. I'll look into it and get back to you tomorrow. Thanks for your help!

Also check your application Log file, it should contain information about failed connections:
%USERPROFILE%\AppData\Local\Snap\Snap Camera\Log.txt

I've got it sorted out. Problem was with an existing self-signed localhost certificate from VMware Workstation (also installed on my machine). Snapcamera application kept refusing handshake because of that.
That certificate didn't show up in the management console. Disabling VMware server sharing did the trick.
After that, Docker crapped out on me for some reason. Had to reinstall. Now I see those incoming requests.

It works:) Nice job you've done here. Thanks a lot.


fix(graphcache): Restore stale being set to true on blocked (but in-flight) operations

Note: I wasn't reasonably able to write tests for the looping protection yet that hit this particular case. Manual testing is needed!

Summary
For partial cache results, stale: false was being set when the looping protection kicks in. In simple words, if an operation is triggered twice, once as part of an unrelated query updating, causing a partial result, then again, the second trigger will create a cache result with stale: false, even if the operation would be in flight.
This PR restores stale: true being set, if we reasonably believe the operation is still in flight.
Set of changes

Add separate stale: true override if operation is in flight

Seems to address the issue I was having (mentioned here: https://twitter.com/andrewingram/status/1752787499053723667), but I also haven't been able to make a simplified reproduction :/

The test I added should sufficiently reproduce this scenario. It doesn't quite match the manner in which this case is triggered, but it should represent both the overall conditions that triggered this bug/issue and the fix itself


prevent truncation

is there a way to prevent output truncation? Set the truncation character limit to a custom value?

Hi
Log messages are truncated by Logger, so you should configure it. Check Elixir docs.

found it!
config :logger,
  truncate: :infinity


[WIP] Unboxing for C++ backend

This change set adds limited unboxing support for the C++ backend. Calling interface methods on boxed value types is functional, as is calling methods such as Convert.ToInt32 with a boxed value type. Enums (as in #2884) still fails, I need to look at what is going on with FindInterfaceMethodImplementationTarget.
@jkotas
Working examples:
using System;

internal class Program
{
    interface IMyInterface
    {
        void MyMethod();
    }
    struct MyValueType : IMyInterface
    {
        void IMyInterface.MyMethod()
        {
            Console.WriteLine("IMyInterface.MyMethod");
        }
    }
    private static void Main()
    {
        object o = new MyValueType();
        ((IMyInterface)o).MyMethod();
    }
}

using System;

internal class Program
{
    private static void Main()
    {
        Console.WriteLine(Convert.ToInt32((object)1));
    }
}

@benpye,
Thanks for having already signed the Contribution License Agreement. Your agreement was validated by .NET Foundation. We will now review your pull request.
Thanks,
.NET Foundation Pull Request Bot

Will address the two comments.
Is it possible to trace through the RyuJIT compiled code with any symbols. In the struct case it seems that FindImplSlotForCurrentType (at https://github.com/dotnet/corert/blob/master/src/Runtime.Base/src/System/Runtime/DispatchResolve.cs#L51 ) fails until pCur is NULL, and we return 0. Would be useful to see the expected behaviour to compare.

@benpye Do you plan to do more changes for this? I think this change is improvement as is - I would be happy to merge it. The cleanup around the unboxing stub node define can be done separately.

@jkotas Happy to merge this as is, as you say, already an improvement. Have addressed your comments and squashed. Can work on cleaning up and what is going on with the enum case.

Thanks!


Remove support for labelName query param in /apis/{apiId}/swagger devportal endpoint

Description:
Need to remove support for [1].
[1]. https://github.com/wso2/carbon-apimgt/blob/master/components/apimgt/org.wso2.carbon.apimgt.rest.api.store.v1/src/main/resources/devportal-api.yaml#L244
Steps to reproduce:
Affected Product Version:

Environment details (with versions):

OS:
Client:
Env (Docker/K8s):

Optional Fields
Related Issues:

Suggested Labels:

Suggested Assignees:

fixed with https://github.com/wso2/carbon-apimgt/pull/10010
related issue: https://github.com/wso2/product-apim/issues/10253


ERROR: Service 'prebuiltsdk' failed to build: The command '/bin/sh -c ./src/mkli nd -q glibc' returned a non-zero code: 1

If I try to build Lind with docker-compose build, I get this error:
In file included from /home/lind/lind_project/src/native_client/tools/SRC/gcc/gcc/tree-dump.h:    26,
                 from /home/lind/lind_project/src/native_client/tools/SRC/gcc/gcc/tree-mudflap.c:40:
/home/lind/lind_project/src/native_client/tools/SRC/gcc/gcc/tree-pass.h:102:5: note: enum constant defined here
  102 |     GIMPLE_PASS,
      |     ^~~~~~~~~~~
make[3]: *** [Makefile:949: tree-mudflap.o] Error 1
make[3]: *** Waiting for unfinished jobs....
make[2]: *** [Makefile:4858: all-gcc] Error 2
make[1]: *** [Makefile:565: BUILD/stamp-x86_64-nacl-pregcc-standalone] Error 2
make: *** [Makefile:957: build-with-glibc] Error 2

[./src/mklind:552] Error: [function: build_glibc()] [arguments: (nil)].

It took 274 seconds

All done.
ERROR: Service 'prebuiltsdk' failed to build: The command '/bin/sh -c ./src/mklind -q glibc' returned a non-zero code: 1

I will get the whole log of the error. I had this error on Docker on both Ubuntu and Fedora.
Also, is docker-compose the right way to build Lind? Do I need to run ./mklind first?

The error is attached here: https://gist.github.com/neelchauhan/db1bec84c63fb5545968ca26dfcdc808
I am running Lind/Docker on Fedora 30 in a VM on a FreeBSD 12.0 server (I am using FreeBSD's native bhyve hypervisor, not VirtualBox).


Possible Bug - IFC model not loading correctly.

Describe the bug 📝
Hey,
After loading this particular model (ifc file), I see that not all the items are loaded correctly. Some elements may be missing. I am testing directly with the latest version of the official template.
In the following screenshot you can see the original model loaded correctly using usBIM.

And this is the result of having loaded it using the official template:

As you can see, most of the roof has not been loaded. Some parts are loaded but the geometry is not correct.

I am aware that you need the IFC file to replicate the problem. But in this case, the IFC I am using is a corporate one, which I cannot publish. Is there a way to send you the IFC without it being public?
Thank you very much.
Reproduction ▶
No response
Steps to reproduce 🔢
No response
System Info 💻
System:
    OS: Windows 11 10.0.22631
    CPU: (20) x64 13th Gen Intel(R) Core(TM) i7-13700H
    Memory: 2.64 GB / 15.64 GB
  Binaries:
    Node: 20.11.1 - C:\Program Files\nodejs\node.EXE
    npm: 10.2.4 - C:\Program Files\nodejs\npm.CMD
  Browsers:
    Edge: Chromium (126.0.2592.113)
    Internet Explorer: 11.0.22621.3527

Used Package Manager 📦
npm
Error Trace/Logs 📃
No response
Validations ✅

[X] Read the docs.
[X] Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
[X] Make sure this is a repository issue and not a framework-specific issue. For example, if it's a THREE.js related bug, it should likely be reported to mrdoob/threejs instead.
[X] Check that this is a concrete bug. For Q&A join our Community.
[X] The provided reproduction is a minimal reproducible example of the bug.

Hi, yes! You can send it to<EMAIL_ADDRESS>Can you try to load the IFC here and see if the issue persists? Thanks!

Hello, thanks for your quick response.
I Have tried to upload the model, and the result is different.
Here the roof is not even loaded:

I have sent you the email with the IFC model.
Thank you.


TR2 - Fan freeze seen in Opera House

Freeze seen in Opera House with the fans not working and unable to go through some doors. Settings file provided below.
TombRaiderIIPC.gz

TRModelTransporter Checks/Updates
Export

[ ] Check each TRAnimation's FrameOffset and FrameSize values for every model. TRosettaStone section 6.2.2 details potential for animations to use frames not necessarily "owned" by the corresponding model. These frames may not have been included during cross-level if the "other" model is not also imported (or if its frames aren't beside the next model's frames after import).

Import

[ ] Remove all redundant models i.e. enemies no longer required.
[ ] Reorganise TRModels[] so model IDs are in order.
[ ] Reorganise Frames[] so offset pointers in models are in order.
[ ] Reorganise TRAnimation[] based on TRModel[] reorganisation.
[ ] Check/remove duplicated TRSoundDetails objects.

Unit testing for Export/Import

[ ] Test 1

For each level, serialize original level file.
Export all models in the level.
Delete all animations, frames, animation dispatches, state changes, commands, sounds and models.
Import the default models for the level.
Serialize the level again and confirm the result matches step 1.


Addition of Project maintainer in readme.md

Hi there,
I want to add Project maintainer feature in readme.md file
please assign me this issue.

ok @PrathmeshMutke. I will assign you this issue. But first complete the previous issue assigned to you. Get that successfully merged and then I'll assign you this issue.

@PrathmeshMutke Assigning you the issue.

@PrathmeshMutke Any updates?

I am interested, kindly assign this task to me

I am interested, kindly assign this task to me

Assigning it to you @deed-pooL. Go ahead. You have 24 hours to complete and submit the PR.

Please provide me with any specific details if you have. I will complete
the task by 15th july.
On Wed, 14 Jul 2021 at 13:56, sameer srivastava @.***>
wrote:

Assigned #29
https://github.com/sameersrivastava13/Vaccine-Tracker-App/issues/29 to
@deed-pooL https://github.com/deed-pooL.
—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub
https://github.com/sameersrivastava13/Vaccine-Tracker-App/issues/29#event-5018255208,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ANOKZIB6VUH6RT7JNUX4OQ3TXVCZ3ANCNFSM5AHDDUYA
.

Please provide me with any specific details if you have. I will complete the task by 15th July.

For now you can just add heading (Project maintainer), my name, and photograph. All these you can find in my profile. Take reference from other readme files.

Sure, will do that.
On Wed, 14 Jul 2021 at 16:55, sameer srivastava @.***>
wrote:

Please provide me with any specific details if you have. I will complete
the task by 15th July.
For now you can just add heading (Project maintainer), my name, and
photograph. All these you can find in my profile. Take reference from other
readme files.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/sameersrivastava13/Vaccine-Tracker-App/issues/29#issuecomment-879812143,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ANOKZIF6EK2PO5B5BZHOFUDTXVX2LANCNFSM5AHDDUYA
.


RedisSentinelCache not connecting after upgrade to 3.0

I had an existing setup fo Superset 2.1 which I now upgraded to 3.0.  Upgrade was smooth.  Internally I had configured for the RedisSentinelCache.
The config for CACHE_CONFIG and DATA_CACHE_CONFIG are similar:
CACHE_CONFIG = {
    'CACHE_TYPE': 'RedisSentinelCache',
    'CACHE_DEFAULT_TIMEOUT': 3600,
    'CACHE_KEY_PREFIX': 'ss_',
    'CACHE_REDIS_PASSWORD' : 'xxxxx',
    'CACHE_REDIS_DB' : 15,
    'CACHE_REDIS_SENTINELS': [('ip1',26379), ('ip2', 26379), ('ip3',26379)],
    'CACHE_REDIS_SENTINEL_MASTER' : 'mastername'
}

The above configuration was working perfectly in Superset 2.1.  However in Superset 3.0 I always get the error : redis.exceptions.ConnectionError: Error 111 connecting to localhost:6379. Connection refused.  Looks like Flask-caching is trying to connect to RedisCache on default localhost 6379 instead of RedisSentinelCache.
Exception:
Exception possibly due to cache backend.
Traceback (most recent call last):
  File "/home/jenkinsci/.pyenv/versions/3.9.18/envs/superset3/lib/python3.9/site-packages/redis/connection.py", line 707, in connect
    sock = self.retry.call_with_retry(
  File "/home/jenkinsci/.pyenv/versions/3.9.18/envs/superset3/lib/python3.9/site-packages/redis/retry.py", line 46, in call_with_retry
    return do()
  File "/home/jenkinsci/.pyenv/versions/3.9.18/envs/superset3/lib/python3.9/site-packages/redis/connection.py", line 708, in <lambda>
    lambda: self._connect(), lambda error: self.disconnect(error)
  File "/home/jenkinsci/.pyenv/versions/3.9.18/envs/superset3/lib/python3.9/site-packages/redis/connection.py", line 1006, in _connect
    raise err
  File "/home/jenkinsci/.pyenv/versions/3.9.18/envs/superset3/lib/python3.9/site-packages/redis/connection.py", line 994, in _connect
    sock.connect(socket_address)
  File "/home/jenkinsci/.pyenv/versions/3.9.18/envs/superset3/lib/python3.9/site-packages/gevent/_socketcommon.py", line 607, in connect
    raise _SocketError(err, strerror(err))
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/jenkinsci/.pyenv/versions/3.9.18/envs/superset3/lib/python3.9/site-packages/flask_caching/__init__.py", line 834, in decorated_function
    cache_key = decorated_function.make_cache_key(f, *args, **kwargs)
  File "/home/jenkinsci/.pyenv/versions/3.9.18/envs/superset3/lib/python3.9/site-packages/flask_caching/__init__.py", line 591, in make_cache_key
    fname, version_data = self._memoize_version(
  File "/home/jenkinsci/.pyenv/versions/3.9.18/envs/superset3/lib/python3.9/site-packages/flask_caching/__init__.py", line 541, in _memoize_version
    version_data_list = list(self.cache.get_many(*fetch_keys))
  File "/home/jenkinsci/.pyenv/versions/3.9.18/envs/superset3/lib/python3.9/site-packages/cachelib/redis.py", line 95, in get_many
    return [self.load_object(x) for x in self._client.mget(prefixed_keys)]
  File "/home/jenkinsci/.pyenv/versions/3.9.18/envs/superset3/lib/python3.9/site-packages/redis/commands/core.py", line 2002, in mget
    return self.execute_command("MGET", *args, **options)
  File "/home/jenkinsci/.pyenv/versions/3.9.18/envs/superset3/lib/python3.9/site-packages/redis/client.py", line 1266, in execute_command
    conn = self.connection or pool.get_connection(command_name, **options)
  File "/home/jenkinsci/.pyenv/versions/3.9.18/envs/superset3/lib/python3.9/site-packages/redis/connection.py", line 1461, in get_connection
    connection.connect()
  File "/home/jenkinsci/.pyenv/versions/3.9.18/envs/superset3/lib/python3.9/site-packages/redis/connection.py", line 713, in connect
    raise ConnectionError(self._error_message(e))
redis.exceptions.ConnectionError: Error 111 connecting to localhost:6379. Connection refused.

How to reproduce the bug

Enable RedisSentinelCache
Click on any dashboard
Error displayed

Expected results
RedisSentinelCache should work
Environment
(please complete the following information):

browser type and version: Chrome latest
superset version: 3.0 via pip
python version: 3.9.18 installed via pyenv
node.js version: node -v
any feature flags active: DRILL_BY, DRILL_TO_DETAIL

Checklist
Make sure to follow these steps before submitting your issue - thank you!

[x ] I have checked the superset logs for python stacktraces and included it here as text if there are any.
[ x] I have reproduced the issue with at least the latest released version of superset.
[ x] I have checked the issue tracker for the same issue and I haven't found one similar.

Additional context

flask-caching was upgraded.


Language Server appears not to start up

Describe the bug
I've followed the installation instructions, but the language server doesn't seem to start up. No .d.ts file is created.
I'm using VSCode.
Reproduction
https://github.com/mattpocock/gql-tada-testing
gql.tada version
^1.2.1
Validations

[X] I can confirm that this is a bug report, and not a feature request, RFC, question, or discussion, for which GitHub Discussions should be used
[X] Read the docs.
[X] Follow our Code of Conduct

I suppose this does add some question around debugging, but I think, the answer here is quite simple: gql.tada should have a CLI that allows doing all the operations we tell people out of the box.
I've also been thinking that for some cases a doctor or check subcommand would be useful that checks several pre-conditions 🤔

We can't surface it no, I have tried a lot of things but I have only managed to surface it in tsserver. We're working on https://github.com/0no-co/gql.tada/pull/58 to be able to surface these differently

OK, makes sense! I'll close this issue as it feels unrelated to the real problem, which you guys have in hand.

just want to add that folders won't be automatically created so no .d.ts file either


can socket.io-redis can be configured with redis-cluster?

The above example seems to be of using a node cluster with a single redis server instead of a cluster of redis servers. Unless I am mistaken and the process for both is mostly the same except providing some sort of special url!
Please help!

I am also trying to figure out how socket.io-redis can be connected to a redis cluster.

@iojohnso An alternate could be to use socket.io-ioredis, available here:
https://github.com/GannettDigital/socket.io-ioredis

We also need a working socket.io adapter using a redis cluster. Did anyone solve this problem yet?
The suggestion of @chaudhryjunaid sounds promising, but it seems that prebuilt ioredis clients can only be provided for pub/sub (?)

FYI, https://github.com/GannettDigital/socket.io-ioredis/issues/8

By providing an ioredis pub/sub client (implemented in https://github.com/socketio/socket.io-redis/pull/95), you should now be able to connect to a redis-cluster.
Doc: https://github.com/luin/ioredis#cluster
Closing this now :angel:


use XDG_CACHE_HOME on Linux

[x] Have you followed the guidelines in our Contributing document?
[x] Have you checked to ensure there aren't other open Pull Requests for the same change?
[x] Have you added an explanation of what your changes do and why you'd like us to include them?
[ ] Have you written new tests for your changes? Here's an example.
[ ] Have you successfully run brew style with your changes locally?
[ ] Have you successfully run brew typecheck with your changes locally?
[ ] Have you successfully run brew tests with your changes locally?

Closes #16156
This variable was getting filtered out before so it wasn't possible for us to use it as the default for HOMEBREW_DEFAULT_CACHE and HOMEBREW_DEFAULT_LOGS. I'm no bash expert so if there's a better way to handle this let me know.

You can just add it to USED_BY_HOMEBREW_VARS I think

Makes sense, thanks @apainintheneck!


[ZEPPELIN-3931] Redisplay angularObjectBind when the note is reopened

What is this PR for?
At present, Bind's angularObject in note is only valid in the current operation web page.
When the note is reopened, or the zeppelin service is restarted, the angularObject of Bind in the note cannot be displayed, and the bind operation must be repeated.
The submarine has a lot of startup commands and parameters. In order to provide the best experience for the user, we provide the parameters to the user through the WEB control.
Zeppelin's own dynamic form is more suitable for parameter input in the query, but it does not meet the needs of the submarine interpreter, so the submarine interpreter uses the angular template to generate a richer input interface.
The controls in the interface are saved to the note through Bind's angularObject, so that when the user reopens the note, there is no need to re-enter it.
What type of PR is it?
[Improvement]
Todos

[x] Save angularObject to Note in NotebookServer::angularObjectClientBind(...) function.
[x] Delete angularObject to Note in NotebookServer::angularObjectClientUnbind(...) function.
[x] Load angularObject update to Note in NotebookServer::getNote(...) function.
[x] Add test case in NotebookServerTest::testAngularObjectSaveToNote(...).
[x] Add test case in NotebookServerTest::testLoadAngularObjectFromNote(...).

What is the Jira issue?

https://issues.apache.org/jira/browse/ZEPPELIN-3931

How should this be tested?
CI pass
Screenshots (if appropriate)

Questions:

Does the licenses files need update? No
Is there breaking changes for older versions? No
Does this needs documentation? No

CI Pass, https://travis-ci.org/liuxunorg/zeppelin/builds/473897099

@felixcheung , @jongyoul , @zjffdu,  Please help me review code. Thanks! :-)