Should  our package be added to conda-forge?

Our repo is currently dependent on geopandas. In Windows, it is easier to install geopandas using conda. If we add our package and required packages (as needed) to conda-forge, our installation instructions could be simplified to: conda install -c conda-forge openpolicedata
https://conda-forge.org/docs/user/introduction.html
https://conda-forge.org/docs/maintainer/adding_pkgs.html#the-staging-process
Some of our packages (at least Socrata) aren't available through conda, but it looks like we could add them to conda-forge: "You don’t have to be the upstream maintainer of a package in order to contribute it to conda-forge. "
Example installation using conda-forge package: https://github.com/cenpy-devs/cenpy

We talked about this and decided we should add our package to conda-forge. We will first see if we can just have OPD on pypi and link to conda-forge or we need to have our package on conda. First investigate what needs to be done to get package on conda, then decide steps on what to do for package placement (PyPi, conda etc)

I'm investigating if we need all our dependences on conda-forge based on:
"Avoid external dependencies¶
As a general rule: all dependencies have to be packaged by conda-forge as well. This is necessary to assure ABI compatibility for all our packages."
https://conda-forge.org/docs/maintainer/adding_pkgs.html#the-staging-process.

Going to take a step back and first get the package installed on PyPi. This will allow us to resolve the issue if GeoPanadas can be installed under Windows or needs to be an optional package. Once we have a working PyPi installation we can use grayskull to generate the meta.yaml file for conda-forge.

Updates have now been pushed that make GeoPandas optional. Should be ready to attempt to add package to pip

Able to create the meta.yaml file for conda using our latest code packaged on pypi test using the command:
grayskull pypi --pypi-url https://test.pypi.org/pypi openpolicedata-sowdm
Did you want to create a package without the -sowdm?
Also we need a version number

Does it need a -{something} at the end or can it just be openpolicedata?
Thoughts on the version #? You probably have more experience with that than me.

It should be able to just be called openpolicedata. If you create the account on test pypi I can add the package. I would recommend starting with version 0.1.0 where for major.minor.patch we follow
major => change when adding features that breaks backwards compatibility of something
minor => change when adding features
patch => bug fixes

0.1.0 sounds good

I now seem to recall that on TestPyPI, they recommend the -username after your package name so maybe we should use openpolicedata-sowdm for now. I'm trying to find the documentation but haven't been able to yet.

Need to create openpolicedata on pypi
Test install from pypi-test
Upload version 0.1.0 to pypi
Create Grayskull script for conda-forge
Submit to conda-forge

@potto216 Additional question for testing wheel install: You've trying installing from pypi-test. Can you install directly from a wheel file saved locally? Here are directions for installing opd from wheel. This might avoid having to change versions every time on pypi-test and save a few steps when testing if new installer versions work

Removed geopandas from setup.cfg, generated a wheel file, and successfully pip installled from the local wheel file in dist folder

Still unable to to install on ubuntu using 0.1.3 getting errors like
ERROR: Could not find a version that satisfies the requirement pandas (from openpolicedata-sowdm) (from versions: none)
ERROR: No matching distribution found for pandas (from openpolicedata-sowdm)

and
Collecting sodapy
  Downloading https://test-files.pythonhosted.org/packages/49/30/15d83581870d4e6f71a0a77e1c7bdc368ab6113775fa5977e9cca6101e77/sodapy-0.1.7.tar.gz (13 kB)
    ERROR: Command errored out with exit status 1:
     command: /home/user/tmp/.venv/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-xkibm1sg/sodapy/setup.py'"'"'; __file__='"'"'/tmp/pip-install-xkibm1sg/sodapy/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-install-xkibm1sg/sodapy/pip-egg-info
         cwd: /tmp/pip-install-xkibm1sg/sodapy/
    Complete output (5 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-xkibm1sg/sodapy/setup.py", line 4, in <module>
        from past.builtins import execfile
    ModuleNotFoundError: No module named 'past'
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

At a minimum found the following packages need to be installed before a pip download
sudo apt-get install libkrb5-dev build-essential libssl-dev libffi-dev python3-dev

pip install wheel
pip install gssapi

Still working through the bugs.

I just tried installing from Windows and got an error finding the dependencies too.
The first line of the install says:
Looking in indexes: https://test.pypi.org/simple/
Maybe it's only looking for the dependencies in test.pypi?
Then, I got the same error as you
Collecting sodapy
  Downloading https://test-files.pythonhosted.org/packages/49/30/15d83581870d4e6f71a0a77e1c7bdc368ab6113775fa5977e9cca6101e77/sodapy-0.1.7.tar.gz (13 kB)
    ERROR: Command errored out with exit status 1:
     command: 'C:\Users\matth\.conda\envs\opd-test\python.exe' -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\matth\\AppData\\Local\\Temp\\pip-install-r29w7x1q\\sodapy_cd28b27f15154e559b9a5ecf09aa2b2d\\setup.py'"'"'; __file__='"'"'C:\\Users\\matth\\AppData\\Local\\Temp\\pip-install-r29w7x1q\\sodapy_cd28b27f15154e559b9a5ecf09aa2b2d\\setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base 'C:\Users\matth\AppData\Local\Temp\pip-pip-egg-info-_tvfyqmh'
         cwd: C:\Users\matth\AppData\Local\Temp\pip-install-r29w7x1q\sodapy_cd28b27f15154e559b9a5ecf09aa2b2d\
    Complete output (5 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "C:\Users\matth\AppData\Local\Temp\pip-install-r29w7x1q\sodapy_cd28b27f15154e559b9a5ecf09aa2b2d\setup.py", line 4, in <module>
        from past.builtins import execfile
    ModuleNotFoundError: No module named 'past'

Note that it downloaded the file from https://test-files.pythonhosted.org/packages/49/30/15d83581870d4e6f71a0a77e1c7bdc368ab6113775fa5977e9cca6101e77/sodapy-0.1.7.tar.gz (13 kB)
I'm going to test 2 things:

Downloading the wheel from testpypi and then installing it from file
Checking if there is any way to tell pip where to look for dependencies

@potto216
I found the answer here: https://stackoverflow.com/questions/34514703/pip-install-from-pypi-works-but-from-testpypi-fails-cannot-find-requirements
It pointed me to: https://packaging.python.org/en/latest/guides/using-testpypi/
which suggested running
pip install --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ your-package
which works!

I just created a new Ubuntu VM, followed your instructions
sudo apt-get install libkrb5-dev build-essential libssl-dev libffi-dev python3-dev
and then ran
pip install --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ openpolicedata-sowdm
This successfully installed, and I was able to import and load some data.
I think the next step is for me to upload to regular PyPI?

@potto216 We are live on PyPI!
https://pypi.org/project/openpolicedata/

What's your username on PyPI so I can invite you to be an owner?

Awesome! I got it to install with  pip install openpolicedata . I'll work on the conda-forge script and send you my PyPI info

Submitted a PR to have our package added on conda-forge https://github.com/conda-forge/staged-recipes/pull/19327

Find out if need re-approval from conda-forge when updates are made to our package

Did not get a response from the PR or using the suggested bot command @conda-forge-admin. Trying their glitter channel

Was able to get a response from the conda team on glitter and I would also need to submit requests and be the maintainer for arcgis, sodapy. I created recipes for these, but they failed the automatic checks. I felt the amount of effort to get these to pass exceeded the benefit so I'm going to close the branch. I also deleted the PR on conda


Modded weapons that don't use a vanilla class have their attack damage set to 1

Shown here with multiple mods

However Chest Cavity's Cleavers work fine because they use the vanilla SwordItem class

Fixed in version 1.3.3, thank you for the bug report!


JS & JSX Spread operators

I can't make spread operator to work in both JSX (in arguments) or some component declaration objects. We don't even need to polyfill it (for example with Object.assign), as we don't transpile output to es5, so it is fine to keep spread operator as it is in compiled file. But for some reason Babel/Babylon throw errors on it.
@bencompton can you take a look please when you have time?)

I went into the accordion-content component and tried some Spread operator examples, and here is what happened:
    const propsObj = {
      id: this.props.id,
      style: this.props.style,
      className: this.classes,
    };

    //Array spread works fine
    const someArray = [1, 2, 3, 4];
    const arrayWithSpread = [...someArray, 5, 6];

    //Object spread fails
    const someObject = { foo: 'test', bar: 'test2' };
    const objectWithSpread = { ...someObject, baz: 'test3' };

   //JSX spread works fine
    return (
      <div {...propsObj}>
        <slot></slot>
      </div>
    );

Rest/Spread are still considered experimental and are slated for ES2018. It appears that Babel currently only supports it in this plugin. I installed the plug-in and hard-coded codeToAst so it would use that plug-in. The Babel compiler stopped erroring on the object Spread, but I had to upgrade @babel/core to the latest version to make the plug-in work. Unfortunately, it seems that the latest version of @babel/core is causing problems. After calling babel.transform in codeToAst, the ast property is now coming back as null for some reason.
Ultimately, we will need to provide the ability to pass in Babel compiler options as you said in #9. That will then allow us to add in the above plug-in to support object Rest/Spread. It appears that we first need to figure out why the latest version is returning a null AST.


Warnigs in Druid node logs after upgrade Java to 11

Hi guys,
Just to let you know (probably nothing, but just in case) that I've found some warnings being logged down each time Druid nodes are started:
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.google.inject.internal.cglib.core.$ReflectUtils$1 (file:/usr/local/apache-druid-24.0.0/lib/guice-4.1.0.jar) to method java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.securit
y.ProtectionDomain)
WARNING: Please consider reporting this to the maintainers of com.google.inject.internal.cglib.core.$ReflectUtils$1
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release

Thanks and keep up the great job!

It's a known problem. See: https://github.com/apache/druid/pull/12232#issuecomment-1059917494
This won't cause any problem as I know. I don't know there is somebody working on this problem because it's a little bit tricky.

This is due to JDK encapsulation: https://druid.apache.org/docs/latest/operations/java.html#strong-encapsulation
You can avoid them, if you like, by adding the mentioned --add-exports and --add-opens to your jvm parameters. Although as mentioned, the warnings are harmless, so this isn't required. Patch #12839 adds some logic to the startup scripts to set these automatically on Java 17, where they are required.

Closing since this is not an issue on Java 11, and on Java 17 we set the required parameters in the startup scripts. Please let us know if you run into problems.


Add Ppx_deriving.register_external

This PR adds a new function allowing one to declare a deriver that is interpret by another ppx rewriter:
(** [register_external name] registers [name] as an external deriver that is
    not interpreted by ppx_deriving. Ppx_deriving will simply ignore [name]
    when interpreting [[\@\@deriving]] annotations and leave it to be
    expanded by another rewriter. *)
val register_external : string -> unit

The aim is to allow ppx_deriver and ppx_type_conv to live in harmony when linked as part of the same ocaml-migrate-parsetree driver once #123 is merged.
Current situation
Currently ppx_type_conv does the following in order to interoperate with ppx_deriving:

it exports its own derivers using the Ppx_deriving API
it imports ppx_deriving derivers using Ppx_deriving.{add_register_hook,derivers}

If ppx_deriving and ppx_type_conv were linked as part of the same omp driver, the result might not be what one would expect. In practice, since ppx_type_conv uses the 4.03 AST and ppx_deriving (with #123) uses the 4.05 one, ppx_type_conv would be executed first and because it removes [@@deriving ...] annotations, ppx_deriving would see nothing. This would be a bit surprising for users since as soon as they start using a ppx_type_conv based rewriter such as ppx_sexp_conv, they would have to change the option syntax of their [@@deriving ...] annotations. Moreover it is a bit fragile.
Goal
With this PR and #123, ppx_type_conv will register its own deriviers with Ppx_deriving.register_external and ignore ppx_deriving's derivers.
This is better than the current situation for two reasons:

ppx_type_conv and ppx_deriving don't work exactly the same way and with this change they'll both get to interpret their own derivers in the way they were supposed to be interpreted
the import/export code in ppx_type_conv is big and tedious and this change will allow us to eventually get rid of it

Thanks!


ARM64 docker image?

I used the instructions to pull and run almond-server with podman, and it wouldn't start because of architecture related error. Is there a docker image for AArch64 architecture? Or is there a configuration change I can make somewhere? This is for an orange pi zero plus2 with an allwinner H5 64bit cpu.

Hi! Thanks for your report! We don't have multiarch images at the moment, primarily because we don't have an arm64 machine to build on. You might be able to install Almond from source though, following the instructions in the README under "Development Setup".


Support afterRender hook for custom elements/attributes

I have a custom element
<x-foo bar.bind="[1,2]"></x-foo>

which has a template:
<template>

${f}

</template>

and resulting HTML:

1

2

I have a fooChanged event which allows me to hook in before the DOM has been updated with the new
's for the updated foo:
### something sets foo = [1,2,3] ###
---> fooChanged(newValue,oldValue)
---> HTML updated:

1

2

3

Is there a hook for after the DOM has been updated instead of before?
e.g.
### something sets foo = [1,2,3] ###
---> fooChanged(newValue,oldValue)
---> HTML updated:

1

2

3

--> afterRender(...)

Please let me know if I can provide any other info or if there's already a way to do this in Aurelia.  Thanks!

Note that I use the attached handler to do the for the first render...

Can you explain what it is you want to accomplish with this? We might have a way to do it already, or if not, your use case can help use determine specific details for our implementation.

Absolutely.   I'm building a data table using the jQuery Data Tables plugin.   I'm currently trying to generate much of the markup using templating ( configs via data- and columns via thead.tr.th's and  and then, once the dom is rendered, instantiate $('table').DataTable(...)  When the model changes, the changed handlers are executing before the underlying HTML is updated so if I'm in the columnsChanged(newValue,oldValue) handler, the thead.tr.th's represent oldValue vs. newValue.
I've come up with a few workarounds for this, so this isn't a big issue for me but I would use the functionality if present.   Thanks!

Can you share a bit about what workarounds you are using currently?

Well, workarounds are really just a single workaround:  move to fully JS-based rendering.  Use a single config property from the parent view model which is bound to the custom element e.g. <x-data-table value.bind="tableConfig">and use thevalueChanged` to re-render the content using JS entirely.
I'd prefer to do it with discretely bound objects so that I can more clearly factor my custom element class, however this method removes the hard requirement for an afterRender-like hook.

You might want to look at using the micro task queue. Perhaps this will be
of use?
https://github.com/aurelia/templating-resources/blob/master/src/compose.js
On Sun, May 24, 2015 at 2:10 PM, Shanon Levenherz<EMAIL_ADDRESS>wrote:

Well, workarounds are really just a single workaround: move to fully
JS-based rendering. Use a single config property from the parent view model
which is bound to the custom element e.g. and use thevalueChanged` to re-render the
content using JS entirely.
I'd prefer to do it with discretely bound objects so that I can more
clearly factor my custom element class, however this method removes the
hard requirement for an afterRender-like hook.
—
Reply to this email directly or view it on GitHub
https://github.com/aurelia/templating/issues/79#issuecomment-105007460.

--
Rob Eisenberg,
President - Blue Spire
www.durandaljs.com

Thanks @EisenbergEffect - queueMicroTask is working well for me.   Really appreciate the assistance here.

You're welcome. I'll keep this open so we can evaluate if we need to add something else in the future or at least do some education around this scenario.

Great -- please let me know if I can provide any other info/examples; I'd be glad to help however possible.

I had the same needs for building Google MDL components, and I guess everyone who needs to call some wrapper after the DOM re-rendering will come through this
When I had this problems, people suggested me some workarounds ultil Rob went to the Gitter channel and also told me to use the queueMicroTask, and it worked smoothly.
But since this is something that's highly needed to integrate with most of the 3rd-party UIs, I think there should be an global event for that, like there is "bind", "activate", "attached". The OP suggested "afterRender", and on Gitter I have suggested "domUpdated". Not sure about what name would be the best, but for sure it would be nice to have such a feature.

As I wrote on gitter https://gitter.im/Aurelia/Discuss?at=55ba9cbb7978296537a2bd8f. You actually need to queue it on the macro task queue since the UI rendering is happening there. That's basically doing a setTimeout. That said I would be very careful adding that kind of event for every view since eventually it could be a real performance killer. So if you don't want that event you don't want that penalty. The framework could perhaps detect if that event is defined and queue it on the macro task queue?

Yep, you can use the micro task queue for this :)

Hi all
sorry for bothering you, but can you point out a comprehensive article/example on how to use this so much talked about micro task queue in Aurelia? I can't find anything complete on the web.
I'm trying to solve some problems with third-party jquery components (please please Aurelia Interface soooon ;)), and an "after render" hook would come in handy.
Thanks!

import {inject, TaskQueue} from 'aurelia-framework';

@inject(TaskQueue)
export class MyElement {
  constructor(taskQueue) {
    this.taskQueue = taskQueue;
  }

  scheduleSomething() {
    this.taskQueue.queueMicroTask(() => doSomething());
  }
}

Without more details, I don't know if this will solve your specific problem, but that is how you use the Micro Task Queue. You can see the api here: http://aurelia.io/docs.html#/aurelia/task-queue/1.0.0-beta.1.0.1/doc/api/class/TaskQueue

@EisenbergEffect , that was sufficient..
I supposed there were much more code and trickery involved, but now I've just read the task-queue code and I know exactly when/where using it. ;) Indeed, I was already using mutation observers in other modules, but didn't realize they could be used to schedule tasks after DOM updates (but it makes a lot of sense now!).
A side question: is it possible/advisable to extend the default view lifecycle to provide custom extension points, like the aforementioned afterRender hook?
Thank you very much for pointing me in the right direction!
P.

I don't see that there's an issue for you to standardize on that sort of hook in your app, if you need it. However, I'm reluctant to extend Aurelia's lifecycle beyond what it is for now.

My story: Generating s from an array and wanting to use imagesLoaded+masonry - so I need to invoke a javascript snippet only after I'm sure the s are in the DOM. queueMicroTask did not work for me, but queueTask seems to.


Refactor Process and ProcessState

This is some cleanup as part of #1457.
This PR has 2 commits:

takes the Host object out of Process

This was causing some borrow checker problems with ProcessState, when we need a &Process to create a ProcessState to pass it to a Host method. The borrow-checker prevents us from doing that, since there's already a &mut Process reference to get the Host (i.e. process.host.borrow_mut().host_method(&process_state)).
This also forced many methods to be generic of a Host when they really didn't need to, such as processor::trace::finalize_trace().

Converts the ProcessState trait into a struct

This is cleaner IMO, where all advice provider methods were generic over a ProcessState, where actually there's only ever going to be a single implementation (i.e. the VM's Process). I believe the intention with the trait was to clearly indicate the parts of the Process that the AdviceProvider is allowed to view - the current design achieves this in a simpler way.

I checked in miden-base, and I believe this change won't cause problems there.

takes the Host object out of Process

This was causing some borrow checker problems with ProcessState, when we need a &Process to create a ProcessState to pass it to a Host method. The borrow-checker prevents us from doing that, since there's already a &mut Process reference to get the Host (i.e. process.host.borrow_mut().host_method(&process_state)).
This also forced many methods to be generic of a Host when they really didn't need to, such as processor::trace::finalize_trace().

I haven't looked at the code in too much detail yet, but https://github.com/0xPolygonMiden/miden-vm/issues/1100 describes another potential approach to this. Not sure about pros/cons of these approaches yet - but one thing the approach in #1100 tries to avoid is having to pass host to all internal operation handlers.

I'm not sure how #1100 is a solution to the borrow-checker problem. I see the same problem of needing to do e.g.
// Process::op_emit
fn op_emit(&mut self, event_id) -> ... {
  // doesn't work, since `&mut self` takes a mutable reference to the entire `Process` struct
  // (including the inner `self.state`), such that `&self.state` is not allowed.
  self.host.on_event(&self.state)?;
}

The larger problem IMO stems from the circular dependency
Process -> Host -> AdviceProvider
  ^                      |
  |______________________|

where a A -> B means "A requires reference to B". In the current design, Process is a master struct that contains everything that is needed to execute the union of all instructions. Therefore, AdviceProvider methods need a &mut self to modify the advice provider, while also having a read-only reference to Process state (which doesn't work with the borrow-checker as described above).
I find it to be a cleaner design to have Process be only the state that will be encoded into the trace, while the Host is a separate entity that is the "outside environment" in service of trace generation (i.e. advice provider, MastForest dependencies, event handlers, and debug/trace handling). i.e. it's a better separation of concerns, and is also borrow-checker friendly.
As an aside, I would also move Decoder.debug_info into the Host, so that Process really only pertains to trace generation. It would also allow us to have Process::execute_decorator(&self, ...) instead of &mut self, clearly encoding that decorators do not modify the trace. It would also make Host more consistent, since it currently is responsible for handling trace events, but not debug infos (which are similar in my mind).

but one thing the approach in https://github.com/0xPolygonMiden/miden-vm/issues/1100 tries to avoid is having to pass host to all internal operation handlers.

Given the above interpretation, I actually like to visually see which method also mutate the Host, and which don't. For example, we can tell from the signature op_emit(&mut self, event_id, host) that it mutates the external environment, but op_caller(&mut self) doesn't.


Add section on devnet setup

See: https://gist.github.com/blockpane/6223e76971bd3e15666793e016eb938d

https://github.com/dapixio/fio-devnet/blob/master/README.md


Fix links for issue report

changing url to html_url parameter. See example response:
https://docs.github.com/en/rest/issues/issues#list-repository-issues
r: @damccorm

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Choose reviewer(s) and mention them in a comment (R: @username).
[ ] Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
[ ] Update CHANGES.md with noteworthy changes.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.
To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md
GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

hah nice - thanks!


PathBuf file tests

Ensure DataType::File flow works properly and PathBufs given are well-formed

Confirmed suspicions, its always returning anything but DataType::File

This is the blocker apart from final polish for final 2.1.0 release

Closing as old branch is no longer being worked upon


⚠ Bofrost.de has degraded performance

In 528f296, Bofrost.de (https://www.bofrost.de) experienced degraded performance:

HTTP code: 200
Response time: 1363 ms

Resolved: Bofrost.de performance has improved in f2ddb13 after 45 minutes.


⚠ Bofrost.be has degraded performance

In 6dd1a01, Bofrost.be (https://www.bofrost.be) experienced degraded performance:

HTTP code: 200
Response time: 1745 ms

Resolved: Bofrost.be performance has improved in e82ba5d after 20 minutes.


Add deepcopy plugin

It is required by https://github.com/cilium/cilium/pull/26646, so I could include the FlowFilter proto in the generated CRD.
After this change is merged I will use the updated builder image for code generation.

I guess I don't have sufficient permissions to run images/scripts/update-cilium-builder-image.sh or am I doing something wrong?

/test

/test

/test


FR: Domain name variable but in a reversed format for more convenient sorting

Would you please add a variable like {domain} but with the domain URL parts reversed? I suggest the variable name {rdomain}.
Example:
{domain}: subdomain.domain.tld
{rdomain}: tld.domain.subdomain
I would find this useful because it would allow me to more easily locate content from same websites with differnet subdomains when viewing an alphabetically sorted list of domain folders.
Example:
With {domain}
en.wikipedia.com/article.md
www.google.com/article.md
zh.wikipedia.com/article.md
With {rdomain}
com.google.www/article.md
com.wikipedia.en/article.md
com.wikipedia.zh/article.md
Thank you for your hard work and consideration!

In august last, this extension didn't add  domain in reverse order.
I add this feature in my open source chrome extension: markClipper
Enable "domain" properties from extension from chrome web store

Example:

github link : mark-clipper - obsidian web clipper"
Chrome web store: markClip - chrome web store


[UNDERTOW-1934] invoke onClose in case of network failure

Issue: https://issues.redhat.com/browse/UNDERTOW-1934
Ref: https://github.com/undertow-io/undertow/pull/1292

I'm closing this one in favor of #1292 as I find that other solution cleaner. Thanks a lot for having a look at this @baranowb !


is tia compatible with Mac?  'modules not found' error after intalling in Python 27

re installed Python 2.7, re installed tia via $ pip install tia, also logged into bloomberg on the same pc, but still getting the same error message that there is 'No Module named tia.bbg etc'...look around the site and try the tia.dataanalysis, but get the same error....
Any way that I can check that tia is installed correcty. or any further steps - I am working on a Max OSX. any help much appreciated.

Found that the issue was that I was trying to access Bloomberg data remotely


Improve the text at the top of the home page.

Received the latest home page text from the product team.

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: jasonmontgomery sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.


Option to disable webpack inline loaders

Is there anyway one can completely disable inline loaders when running webpack? I couldn't find anything in the docs, or in the source code itself.
If there isn't, is it something planned? What is the recommended way to deal with this?

What is use case?

consider building a tool that process code as user input in the server by a webpack process and returns a bundle to be executed in the user browser
even though webpack configuration is not accessible to the user, one can easily write a malicious custom loader and get remote command execution right away in the server building that input through webpack inline loaders

@evilebottnawi any thoughts on your side?

Many loaders rely on inline loader syntax, so you would break a lot when this could be disabled.
But anyway. webpack doesn't provide security you want to have. The source code is considered as trusted code. Inline loaders is only one way of many more ways to execute code from source code.
If you want to run webpack on untrusted code, put the whole process into an isolated environment. e. g. via docker containers.

the thing is that even docker containers are not that secure, there is also plenty of room in which one with some understanding about cgroups and namespaces can simply jailbreak out of the container and even get root access (of course it depends on a lot of variants here, but you got the point)
one can try to make it secure by deploying workloads in a container orchestrator and even like that it can be quite vulnerable
my point is that it is quite sad to see there is simply no way to disable this feature even if you might not even use it at all. I understand it can break a lot but one could at least choose to have this enabled or not.

You can create eslint rule and do not run code where people will be use inline loaders

For In addition to inline loaders you can also execute code at webpacking time using magic comments. See this Stack Overflow question: https://stackoverflow.com/a/66300450/315936


feat(#64): Auto convert pile-type-specification inputs to strings

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 87.005%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
1319

Relevant Lines:
1516

💛  - Coveralls


Moving over different windows' popups automatically switches focus

Checklist

[X] I'm reporting a problem with Chatterino
[X] I've verified that I'm running the most recent nightly build or stable release
[X] I've looked for my problem on the wiki
[X] I've searched the issues and pull requests for similar looking reports

Describe your issue
Describe the bug
When multiple Chatterino windows are opened and overlaid on top of each other, moving the cursor over another one's popups (hovers) will automatically move the focus over to the other window.
To reproduce

Open two windows of Chatterino
Overlay them on top of each other partially
Mouse over different popups (e.g. stream title, link preview, emote detail, badge detail, etc.)
Observe windows switching focus
Likely only observable on Windows

Screenshots
Video preview

https://github.com/Chatterino/chatterino2/assets/7684273/c7934cb3-607b-4e17-88ef-25888b31e47e
In this video preview, I do not press any keys or any mouse buttons

OS and Chatterino Version
Chatterino Nightly 2.4.6 (commit fd4cac2c) built on 2023-12-10 with Qt 6.5.0, MSVC 193732826 Running on Windows 10 Version 22H2, kernel: 10.0.19045

Can't reproduce on Linux with X11 or Wayland - macOS TBD

https://github.com/Chatterino/chatterino2/pull/4998 introduced this by setting a parent on the tooltip:

If parent is nullptr, the new widget becomes a window. If parent is another widget, this widget becomes a child window inside parent.
docs

The PR fixing this should remember to delete the window if no parent is set (deleteLater).

I suspect we'll need to do platform specific code here and leave the tooltip parentless on Windows, while leaving macOS & Linux the same.
Would be good to test the bug on Linux with a non-tiling window manager too

My guess is that the bug is some result of bad window flags, and we're patching around that.


Configurations for using a custom JSON Formatter

Current behavior
JSON Formatter is required to be installed by anyone who wishes to use the reporting functionality. This adds an additional installation step / dependency to any machine running tests. It also seems to tightly couple JSON generation with the preprocessor.
Desired behavior
A way to add a custom JSON formatter other than the required Standalone executable one.
This could be a configuration option that would allow the user to only generate the .ndjson file before running their own formatting process once the preprocessor has finished?
After reading the docs I did try to achieve this with json.args: ["custom-formatter.js"] but couldn't find supporting documentation for that option. I also ran into the ENOENT issue from the FAQ (where the Standalone Cucumber JSON Formatter is being searched for, even with the formatter variable set in package.json).
I used many variations of the config "json.enabled" "json.args", "json.formatter", "json.output" options but was unable to get a .ndjson without triggering the search for the standalone formatter executable.
EG:
   "json": {       "enabled": true,       "args": ["cucumber-report-json-formatter.js"],       "output": "cucumber-report.json"     }
   "json": {       "enabled": true,       "args": ["cucumber-report-json-formatter.js"],       "formatter": "/formatter/",       "output": "cucumber-report.json"     }
etc..
Versions

Cypress version: 10.3.0
Preprocessor version: 11.4.0
Node version: 14.19.3

Checklist

[x] I've read the FAQ.
[x] I've read Instructions for logging issues.
[x] I'm not using<EMAIL_ADDRESS>(package name has changed and it is no longer the most recent version, see #689).

There's a lot of things to address here..
If you only require the NDJSON output, you can simply configure { "messages": { "enabled": true } } and that's it. Then you have a NDJSON file, without the need of external dependencies.
Configuring { "json": { "formatter": "/formatter/" } } is never correct, because you have no executable named /formatter/ on your system (I checked).
If you want to process the NDJSON using a custom JS-script, you can configure { "json": { "formatter": "node", "args": ["cucumber-report-json-formatter.js"] } }.

You can make it as a new .js file. If you put the new file in your root directory then the cucumber-preprocessor config in package.json would be
 "cypress-cucumber-preprocessor": {
    "stepDefinitions": [
      "cypress/stepDefinitions/*.steps.js"
    ],
    "json": {
      "enabled": true,
      "formatter": "node",
      "args": ["fileName.js"],
    }
  }

Replace "fileName.js" with whatever you called your file

I think you should update your outputFile location. When you run test, it creates a cucumber-report.json file that just has general information about process not an actual json file. content of that file should look something like this.

then your actual .json report should be in your outputFile location.

@oguzhantx  I could not generate the report if i am adding this in my index.js on('after:run', (results) => {
if (results) {
fs.mkdirSync("cypress/.run", { recursive: true });
fs.writeFile("cypress/.run/results.json", JSON.stringify(results));
}
})


Can't get navigation working on comma2

Describe the bug
When enabling navigation I can see the web page at http://comma_ip:8082 and enter my API keys. On the device I have loaded my local region's map (northeast) and it seems to download it on reboot. I am not seeing a significant disk usage change after thus supposed download. My disk is at about 22 out of 25GB used before and after. Once everything is installed I tap on the screen to bring up the map on the right and it just says "Map loading" with a black screen behind it.
Provide a route where the issue occurs
a2b1d650214053d7
openpilot version
release2_e2e
Additional info
No response

Offline database is only for mapd. You need active connections for navigation. Your database also doesn’t seem to be downloaded. You need minimum 50gb free storage in /data/media/0/

The /data mount only has 24G total space. That's all there is on a C2, right?

So Nav and Map are 2 different thing, Nav is the map you see in the driving screen, it "ONLY" show the map info (and the navigation info).
Nav requires 4G/WIFI to work.
Map however is running at the background and will update the cruise speed based on your current position.
Map can work both online (need 4G/WIFI) and offline (Storage Space), depends on your location you may need a huge storage space that may not be available on your C2. :(

I have tried using it tethered to my phone and I still just get a blank map.

Is there a list of the toggles to set for getting Nav and Map on a C2? I have it tethered and I can put a destination in via the web interface, but I just get a black "Map Loading" box and never get any nav. I am getting valid street names on the bottom as  I drive so it's getting something. I am also getting speed limits although they are almost never correct.

navigation does not work on the c2. we disabled it due to performance reason. Will close for now since its is intended behavior.


Read only is a valid member type for project user

About this change—what it does

In the GUI and through the Terraform provider you can use read only as a valid member type, so the documentation should reflect that.
https://aiven.io/docs/platform/reference/project-member-privileges

Why this way

It looks like the documentation is generated from this Go-file, so hopefully it will populate where it is supposed to be.

Hey. Thanks for the contribution!
Unfortunately, it looks like this resources is not the only one. I have made a PR that solves the issue automatically https://github.com/aiven/terraform-provider-aiven/pull/1807/files

No worries, great catch making it automated.


Resolve dependency on custom osm-python-tools package

For interacting with Nominatim and Overpass, we use the package OSMPythonTools, but we made some changes that have not been merged into the upstream. See if we can get them merged; otherwise we will need to fork the package.

As of the new version of OSMPythonTools (0.3.2), the necessary changes are present in the library, so we can upgrade the dependency and this is resolved.


chores: add error message in response for request failure instances

Add message property in error response for failed requests.

It is closed via this MR.


When installing gvmd, get a /run/gvmd Permission Denied

Back at it again with the permissions issue again. Trying to figure on my side and probably on your side too?
--Installing /opt/gvm/var/lib/gvm/gvmd
-- Installing /run/gvmd
CMake Error at cmake_install.cmake directory "/run/gvmd": Permission denied.
make: *** [Makefile:119: install] Error 1

I'll have a look into it.

Ok they issue is, that they again changed the pathing, so it requires to have the socket in /run/gvmd/. My setup ran from /run/gvm/ fine. Maybe this is a local thing, I'll try to find a proper way to handle this. BUT, greenbone also split gsad from gsa, and did some minor changes like branching schema on openvas-smb too. I'll have to figure out how to build this whole thing again.
By the way, how do you think about a move to ansible?

PATHing has been a big proponent of them which is getting annoying. And to answer your question, Ansible is great. It would be great if we can have a bash script with and Ansible being used to install GVM.

I think, I've sorted it out. Could you review the changes in #14 to be working for you?
There is one leftover that might cause issues in the future. The install of gvm-libs requires the /run/gvm directory to be present, but there is apparently no runtime needing this thing. I'm not sure, whether it's actually used somewhere, but if so we will have to put an entry into the tempfiles config to assure the directory to exist upon reboot.

Will check this out tomorrow!

So as for the systemd gvmd.service, it is not working. It times out trying to get itself built. Will have to dig into "why" it is posing that error.

Could you briefly describe what changes you did apply to the script, and what the error looks like?

Sorry it took some time, here is the error for the gvmd.service. I ran journalctl -xe to grab the error message:
Dec 04 22:59:56 ip-172-31-60-47 sudo[29907]:     root : TTY=unknown ; PWD=/home/ubuntu ; USER=root ; ENV=GVM_INSTALL_PREFIX=/opt/gvm ; COMMAND=/usr/bin/bash -c create_gvmd_service ()
                                             {
                                                 set -e;
                                                 cat <<EOF > /etc/systemd/system/gvmd.service
                                             [Unit]
                                             Description=Open Vulnerability Assessment System Manager Daemon
                                             Documentation=man:gvmd(8) https://www.greenbone.net
                                             Wants=postgresql.service ospd-openvas.service
                                             After=postgresql.service ospd-openvas.service network.target networking.service
                                             [Service]
                                             Type=forking
                                             User=gvm
                                             Group=gvm
                                             PIDFile=/run/gvmd/gvmd.pid
                                             RuntimeDirectory=gvmd
                                             RuntimeDirectoryMode=2775
                                             EnvironmentFile=$GVM_INSTALL_PREFIX/etc/default/gvmd
                                             ExecStart=$GVM_INSTALL_PREFIX/sbin/gvmd --osp-vt-update=/run/ospd/ospd-openvas.sock -c /run/gvmd/gvmd.sock --listen-group=gvm
                                             Restart=always
                                             TimeoutStopSec=10
                                             PrivateTmp=true
                                             [Install]
                                             WantedBy=multi-user.target
                                             EOF

                                                 systemctl daemon-reload;
                                                 systemctl enable --now gvmd.service;
                                                 systemctl --no-pager status gvmd.service
                                             };
Dec 04 22:59:56 ip-172-31-60-47 sudo[29907]:     root : (command continued) create_gvmd_service
Dec 04 22:59:56 ip-172-31-60-47 sudo[29907]: pam_unix(sudo:session): session opened for user root by (uid=0)
Dec 04 22:59:56 ip-172-31-60-47 systemd[1]: Reloading.
Dec 04 22:59:57 ip-172-31-60-47 systemd[1]: Reloading.
Dec 04 22:59:57 ip-172-31-60-47 systemd[1]: gvmd.service: Failed to load environment files: No such file or directory
Dec 04 22:59:57 ip-172-31-60-47 systemd[1]: gvmd.service: Failed to run 'start' task: No such file or directory
Dec 04 22:59:57 ip-172-31-60-47 systemd[1]: gvmd.service: Failed with result 'resources'.
-- Subject: Unit failed
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support

My installation of gvmd and the gvmd.service are located here in my code: https://github.com/romelBen/gvm_bash_script

The issue seems to be this one: EnvironmentFile=$GVM_INSTALL_PREFIX/etc/default/gvmd
If this file does not exist, the service fails - I get the very same error like you.
The environment file is not created directly from my script. I need to find out, where this thing comes from - probably from makeing gvmd.

Works on my machine.


[Bug] Android phone vibration not working

Platform

Device: Blackshark 5
OS: android
Browser: Kiwi
Browser Version: 124..0.6327.2
Better xCloud Version: 3.5.1

...
After xcloud interface update, vibration of phone no longer working

Still working fine for me.
Make sure this setting is on


Get device name on pairing request impossible

For showing requesting info on display with code i require device name.
Actual implement show unknown. I try modify and ask device name gap , but reply from iphone in pairing state isnt handled or sended.
Who know. But on moment when pair request window on iphone end timeout, name is received.
a2dp_sink.get_connected_source_name()
Ofcourse with my modified code. Calling in case ESP_BT_GAP_CFM_REQ_EVT:
esp_bt_gap_read_remote_name(peer_bd_addr)

I cat't see any bug in my code here


Banner overlaps, can't dismiss, if user has set Safari accessibility font size

On first login, there is a green banner with the message "What's better than Zulip in your browser? The Zulip desktop app!" With an accessibility font size of 18 or 24 points, It extends into both the left and right sidebars and can't be dismissed because the close button is under the text "USERS".  At 24 points, the left end of the banner is under the "Home" label in the left sidebar. (Presumably any similar banners would have the same problem.)
These examples are from Safari 9, with minimum font size set in the Advanced preferences pane: "Never use font sizes smaller than".
24 point

18 point

Thanks for reporting @feorlen!  It seems like we might be able to fix that by just changing the z-index to put that banner on top...

Fix merged, thanks to @vikas-parashar.


corrects caps in landing page

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app


Spec out vocabulary meta-model

We want to have uniform way of representing vocabulary which should include support for description, synonym and relationships.

not sure who should be the reviewer. either karl or carl.

the best place to look at this is in the data-commons project where the isa common model designs are committed.

My two concerns:

Not sure the purpose of the vocabulary level table nor having vocabulary_id foreign keys in the concept level table. I expect there to be a separate concept table for each distinct vocabulary, with all rows of the table being from the same vocabulary. Thus, I'd turn the vocabulary entity content into an annotation on a particular concept table rather than allowing individual rows to associate to vocabularies.
I think synonyms need to be present in the same concept table under the same uniqueness constraint so that there cannot be collisions between base terms and synonym terms due to invalid table content and so that foreign key references to terms are possible since not all referencing tables will be fully normalized. So, I'd make a nullable foreign key reference similar to the parent reference to have a synonym reference its own canonical concept.

Chiming in here on Karl's comment. I agree with the first point. For the second point, how about just making the synonym table an association table (mapping concept-to-concept within the same concept table).

I factored out the vocabulary meta-model into its own diagram and made updates per the earlier suggestions in this thread.
https://github.com/informatics-isi-edu/data-commons/blob/master/models/vocab-metamodel.pdf

The vocab table is very complicated. Most of the fields will be null in
many cases. Can we simplify it or have 2 different versions? Or start with
a simple version and if needed add more columns later?
On Tue, May 10, 2016 at 2:22 PM, robes<EMAIL_ADDRESS>wrote:

I factored out the vocabulary meta-model into its own diagram and made
updates per the earlier suggestions in this thread.
https://github.com/informatics-isi-edu/data-commons/blob/master/models/vocab-metamodel.pdf
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub
https://github.com/informatics-isi-edu/ermrest/issues/42#issuecomment-218295655[image:
Web Bug from
https://github.com/notifications/beacon/AAZUDA1xmEzqrn8OAqiWPBcTzZP9wbfMks5qAPcBgaJpZM4GNlp3.gif]

I would prefer not.  We already have use cases with RBK where we want subclassing and synonyms.  Also Facebase anatomy.  We really need to push on this.  I would not suggest two versions, as we never start working out the UI issues that we have to.
Carl

Dr. Carl Kesselman
Dean’s Professor, Epstein Department of Industrial and Systems Engineering
Fellow, Information Sciences Institute
Viterbi School of Engineering
Professor, Preventive Medicine
Keck School of Medicine
University of Southern California
4676 Admiralty Way, Suite 1001, Marina del Rey, CA  90292-6695
Phone:  +1<PHONE_NUMBER>
Email:  carl<EMAIL_ADDRESS>Web: http://www.isi.edu/~carl
On May 10, 2016, at 6:55 PM, Hongsuda <notifications<EMAIL_ADDRESS>wrote:
The vocab table is very complicated. Most of the fields will be null in
many cases. Can we simplify it or have 2 different versions? Or start with
a simple version and if needed add more columns later?
On Tue, May 10, 2016 at 2:22 PM, robes <notifications<EMAIL_ADDRESS>wrote:

I factored out the vocabulary meta-model into its own diagram and made
updates per the earlier suggestions in this thread.
https://github.com/informatics-isi-edu/data-commons/blob/master/models/vocab-metamodel.pdf
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub
https://github.com/informatics-isi-edu/ermrest/issues/42#issuecomment-218295655[image:
Web Bug from
https://github.com/notifications/beacon/AAZUDA1xmEzqrn8OAqiWPBcTzZP9wbfMks5qAPcBgaJpZM4GNlp3.gif]

—
You are receiving this because you were assigned.
Reply to this email directly or view it on GitHubhttps://github.com/informatics-isi-edu/ermrest/issues/42#issuecomment-218342266

As per some email discussions and my latest review of the PDF mentioned by @robes:

I think parent_id and synonym_id should both be dropped as predefined columns (see next item).
I think annotations are needed to mark two kinds of hierarchical column, where each one can be instantiated zero or more times on a single vocabulary. A further generalization would be an annotation to identify relations encoded in external tables, e.g. for non-hierarchical relations or for relations between different vocabularies or subsets.

Hierarchical parentage: the generalization of parent_id, i.e. direct ancestor in some named hierarchy
Hierarchical top: the generalization of synonym_id, i.e. transitive ancestor in some named hierarchy

Not every vocabulary has/needs all of id, code, uri.  What subsets and/or default rules can be defined to consistently define or fake these?
It keeps bugging me that we don't know whether particular projects/apps would want all terms in one vocabulary table or not.  How can we juggle different subset domains to allow appropriate foreign key constraints to be defined for each use case?
Very few vocabularies have validity periods and invalidation reasons. I'm not sure why we'd put such esoterica in our systems when there is very little useful UX we can build around them.

Some comments on the PDF above:

Assumes zero or one parents term concept. Most ontologies are graphs (with labeled edges)
'codes' is terminology that is specific to some medical systems. There are a variety of identifier schemes and meta-schemes. We find that CURIEs provide a nice unifying mechanism for relating shortform IDs/codes to the official semweb URI
@robes and I discussed the use cases for making a separate table for each vocabulary source. I am skeptical about the utility of using foreign keys as constraints here, semantic constraints may prove more useful here

For an example of a relational schema that supports a variety of ontology metamodels, see the controlled vocabulary module in Chado: http://gmod.org/wiki/Chado_CV_Module
In particular the two tables:

cv
cvterm

It would be easy to map the SQL DDL to the ermrest schema definition language, and plop in these with every ermrest instance
Also worth looking at metamodels such as LexGrid. If you want to get really meta there is a UML OWL metamodel out there but that may not be a useful level of abstraction for you.

I realize now this issue is sort of in the wrong repo... Its sort of misleading to call this an ERMrest issue other than we want to define an annotation in the ERMrest guide so that applications can understand how to interpret parts of an ERMrest schema as vocabulary. We never intended to imply that we would bake in a vocabulary meta-model into the ERMrest protocol. My mistake.
That aside, I like what I see in the Chado CV module and it is probably better for many of our use cases than OMOP, so it may be a better template to follow or borrow from. @cmungall, Is there a way to get the SQL DDL for the Chado CV Module? I looked across the gmod.org site and didn't find it although I do see instructions for getting the whole stack of software (I believe). Even better would be to also get an exemplar vocabulary to populate it with so we could play around with ERMrest queries over it. Any pointers would be appreciated.

Schema:
https://github.com/GMOD/Chado/blob/master/chado/modules/cv/cv.sql
@scottcain may be able to help with some of the other questions. There is a truly contorted loader I wrote years ago that goes obo->xml, xml->db via a custom protocol called xort/stag, I think @kimrutherford has something much better (is that in the gmod stack yet?)
It should be possible to get some example dumps or open postgres ports to play with. See for example: http://flybase.org/static_pages/feature/previous/articles/2013_12/precomp_files_sql_querying.html
Any of the vocabs you use should work, though it will only support the obo subset, which is fine for your purposes (unless you plan on implementing reasoning within sql...)

I think @kimrutherford has something much better (is that in the gmod stack yet?)

Hi.  We have a prototype OBO -> Chado loader here: https://github.com/pombase/chobo/
It's not in GMOD yet and we haven't announced it.  It works for us but isn't general enough to help
others yet.  For example it doesn't cope with every OBO file yet.  There are a lot of corner cases that the code doesn't handle yet, but we're working on it.
It's in CPAN so can be installed with:
cpanm PomBase::Chobo
The latest version is 0.005, released a few minutes ago.

It should be possible to get some example dumps or open postgres ports to play with.

PomBase is much smaller than FlyBase so our Chado Postgres dumps are much smaller if anyone is just looking for something to play with:
https://curation.pombase.org/releases/pombase-chado-v60-2016-05-12/pombase-chado-v60-2016-05-12.dump.gz
That dump was made from Postgres 9.5 but we aren't using any fancy new features so it should work with older versions.

I copied some notes on Chado and ERMrest from another location so it is easier to share and discuss...

@robes  I think this old issue is obsoleted by the slimmed down vocabulary table convention we've started to follow with the model-management helpers in deriva-py.  Shall we close this?

yeah
From: Karl Czajkowski<EMAIL_ADDRESS>Reply-To: informatics-isi-edu/ermrest<EMAIL_ADDRESS>Date: Friday, July 27, 2018 at 11:44 AM
To: informatics-isi-edu/ermrest<EMAIL_ADDRESS>Cc: Robert Schuler<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Subject: Re: [informatics-isi-edu/ermrest] Spec out vocabulary meta-model (#42)
@robeshttps://github.com/robes I think this old issue is obsoleted by the slimmed down vocabulary table convention we've started to follow with the model-management helpers in deriva-py. Shall we close this?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/informatics-isi-edu/ermrest/issues/42#issuecomment-408506115, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AEAhWphW72rc1Yu1OWvZG-3zL8tCxdpQks5uK19sgaJpZM4GNlp3.


Error pulling submodule

I am having trouble with the git submodule update --init command.
First issue was<EMAIL_ADDRESS>Permission denied (publickey), which was due to my not having set up SSH access to github. I set that up following github instructions, and that part now works. (It is cloning  git@github.com:paulfitz/grist-core.)
The next error is this:
fatal: remote error: upload-pack: not our ref bdaad4c26e761aa6c4705dc28b9d4afe8e4d69c5
Fetched in submodule path 'core', but it did not contain bdaad4c26e761aa6c4705dc28b9d4afe8e4d69c5. Direct fetching of that commit failed.

I don't see any mention of this ref anywhere in the checkout, and I am at a loss at why it's looking for it or how to fix it.

Sorry @dsagal, I somehow never noticed this thread. Do you still have the clone issue?
For the access thing, I remember running into this (but the other way around) and wondering whether there was a friendly way to set this up. Lots of long threads like https://stackoverflow.com/questions/40841882/automatically-access-git-submodules-via-ssh-or-https. If I upstream some of my patches than a relative path might work.


create search for admin index

to show all items in repository, published or unpublished (dark archive)

filter by published/unpublished

can filter unpublished objects by collection

ask about if they want all unpublished vs published by entire index

Resolved. #205


Expose dandiarchive as webdav service

To facilitate integration in various external projects (e.g., OSDF with Pelican underneath) which can interface to webdav services.  Treat it as unification of our API to a standard API for file access.
TODOs:

[x] decide on how to map our versions for dandisets within the path or some webdav supported mechanism for versioning (if there is any)
[x] double-check if webdav supports "redirection" so we do not need to tunnel all bytes from S3 through the webdav service but rather just redirect.
[x] implement mapping/endpoint: I wonder even if it somehow could be just the /paths endpoint, but likely not since that one is dandiset specific, but may be it could be at the top level of API
[x] decide on implementation

could be internal to dandi-archive.  pros:   more efficient since would operate directly on DB/within Python, likely minimizing flood of the logs; it would come "included" by default with any dandi-archive installation; would not require another web server to provide it;  as our API still in 0.x and adds breakages from time to time -- would be easier to keep consistent; cons: it would be easier/faster to develop/fix as independent service likely
could be some additional external service. pros: faster/easier to develop; would stress test our API more to see what it is missing;  cons: solution would need to provide yet another webserver

[ ] deploy as webdav.dandiarchive.org or at some other end-point (discussion below)

may be it could be implemented as independent service, not part of API.
attn @jwodder -- how much did you play with webdav?
edits:

https://github.com/mar10/wsgidav looks very promising and already has (non-production) backends for various things, e.g. for mercurial: https://wsgidav.readthedocs.io/en/latest/addons-mercurial.html .
re structure - I think we at large should mimic our S3, just adding a little more of consistency and have dandisets/ and zarrs/ (not zarr/) folders (problem for zarr/ though -- heavy folder listing).
on the best way to expose versions, possibilities, assuming that we do have some global prefix which ends with {dandiset_id}.

following mercurial example above -- have draft/, released/ (or latest/ - most recent release), releases/{VERSION}.  It kinda makes it to require draft/ or other prefix folder to get to content but it is consistent so I like it most.
have draft/, released/ (or latest/) and then all versions at the same level. kinda ok, but since numbered releases would most likely sort first and oldest first -- I think it would be not that convenient of a default view...
"smarty pants" version -- kinda making versioning optional and reacting to the regex for version at the top level and if not - giving the tree of the draft or most recent version... too smarty - don't like it
following our URL schema dandi://INSTANCE/DANDISET_ID[@VERSION][/PATH] and thus incorporating version into the folder name for dandiset  -- IMHO ugly

FWIW: filed https://github.com/mar10/wsgidav/issues/303

@satra @waxlamp I would like to proceed with moving dandidav deployment into the "official" dandiarchive.org space from its temporary https://dandi.centerforopenneuroscience.org/ .
Please guide us with @jwodder through on what we need to do to accomplish the drill.

create a new instance in the aws account or heroku account and add a route53 cname alias for it. ideally, there are a few considerations with respect to horizontal scalability, but before we do that get a basic setup running. also estimate the costs of this service based on the infrastructure you choose. pinging @aaronkanzer who may be able to help with some considerations depending on choices.
perhaps a devops doc could help you and others in the future as to how to deploy new services. note that once we move over k8s to 2i2c, we will want to use that substrate for future services.

am I correct that heroku would be better target since it would hard limit us on resources so we do not break the bank?

you can limit things in aws as well (fixed instance, no load balancer, etc.,.). but it may be quicker/easier in heroku.


Users mongodb atlas

Connect users service to MongoDB Atlas, hook up querying a user's authorization level from the db, and integrate tests of the database.

The CI checks are failing because Travis is hiding the database password, an encrypted environment, from the test build on this PR since it's coming from my fork of this repo--see https://docs.travis-ci.com/user/pull-requests/#pull-requests-and-security-restrictions. Is there a better way to handle this? The Travis build passes on my fork: https://travis-ci.com/mukobi/large-events

Testing/Travis-CI update: I went ahead and mocked away the db in testing using the mongomock Python module. This module closely emulates the behaviour of the pymongo module so the tests should remain safe and robust. I know mocking is not always ideal, but now the tests all pass (and run faster).

@evankanderson I resolved all your comments and linked the corresponding commits. Also tested in a container and using gunicorn on Cloud Run. Good to merge?

Yes, good to merge.


[TW#15931] No custom log output on UART1 GPIO16,17 after boot

Expected Behavior
When selecting custom log output UART1, RX: GPIO16, TX: GPIO17, all stuff loged via ESP_LOG macros shall be visible via serial.
Observed behavior
Since a couple of weeks (sorry, can't pinpoint it), after pulling latest idf: All boatloader messages are visible, then when the app starts, no log output is visible. There are three ways to mitigate this right now:

choose UART0 (not an option)
On native esp-idf run

void app_main(void)
{
    DPORT_SET_PERI_REG_MASK(DPORT_PERIP_CLK_EN_REG, DPORT_UART1_CLK_EN);
    DPORT_CLEAR_PERI_REG_MASK(DPORT_PERIP_RST_EN_REG, DPORT_UART1_RST);
    // ....
}

On Arduino run

    HardwareSerial serialUART1(1);
    serialUART1.begin(115200, SERIAL_8N1, 16, 17);
    serialUART1.println("Hello on UART1");
    serialUART1.flush();

Both of the last hacks work, but it would be nice to find the root cause of this. IMHO ESP_LOG should always work when the correct settings in sdkconfig are set.
Steps to Reproduce

In sdconfig set:

CONFIG_CONSOLE_UART_CUSTOM=y
CONFIG_CONSOLE_UART_CUSTOM_NUM_1=y
CONFIG_CONSOLE_UART_NUM=1
CONFIG_CONSOLE_UART_TX_GPIO=17
CONFIG_CONSOLE_UART_RX_GPIO=16
CONFIG_CONSOLE_UART_BAUDRATE=115200

Compile, flash an reboot device
No log output from app visible
Try hacks from Observed behavior
No it works again

Notes
This is also mentioned in https://www.esp32.com/viewtopic.php?f=13&t=3166

https://github.com/espressif/esp-idf/commit/4776f73ca4651a5bb65b4db2407205c0dd39925e#diff-b1df72eee4bf4584db9a57c5b0489832

This looks indeed like the root cause here. Where should this be fixed? I mean: when selecting UART1 as log output, this peripheral clock should either not be disabled or enabled again right in place, right?

Closing as fixed.


Filtering, Ordering and Pagination Support on nested Edges

Opening a new Issue to consolidate these closely related features.
Summary 💡
Currently it is not possible to filter, order and/or paginate on nested edges in any combination possible (Related: #1329).
This is also true when using the graphql integration. Following queries are currently not possible:
query QueryAllZoosWithAnimalsNamed($name: String!) {
   queryZoo {
      name
      animals(filter: {name: $name}) { // filtering on nested edges is not possible
         name
       }
   }
}

query QueryAllZoosWithOrderedAnimals {
   queryZoo {
      name
      animals(orderBy: {direction: DESC, field: NAME}) { // ordering on nested edges is not possible
         name
       }
   }
}

query QueryFoldersWithPreviewThumbnail {
   queryFolder {
      name
      images(first: 1) { // pagination on nested edges currently not possible
         thumbnail
       }
   }
}

All of these queries (and of course combinations of these do not work currently. Of course we could always find workarounds, but they are often not desirable because they introduce additional maintenance effort (e.g. extra fields) or force the (web)client to do extra structuring work as we loose our top-down graph structure.
Motivation 🔦
This functionality should be a top priority for a graph-based entity framework brings ent one step closer to become the one-stop-solution for a GraphQL database.
For reference: @yonidavidson

This would be very important! Kind of a dealbreaker in my eyes.
Being able to pass parameters down to fields for pagination is a must have when working with graphs.

@a8m What priority do you have for this?

Hey 👋🏻
All the three were added in the latest version of entgql. Thanks for the reminder, I think we can close thus 🙌

Are there any examples, documentation how to use/create a support for something like this
uery QueryAllZoosWithOrderedAnimals {
   queryZoo {
      name
      animals(orderBy: {direction: DESC, field: NAME}) { // ordering on nested edges is not possible
         name
       }
   }
}

on the server side?

Hey @marcinztipsera,
If you’re enabling the RelayConnection annotation to your edges/nodes, it should work. See https://entgo.io/docs/tutorial-todo-gql-paginate#add-pagination-support-for-query


Awesome Thingsboard

Hi
Could I add a link to this repo to my Awesome Thingsboard collection?

Yes, you can do this.
Thanks @devaskim


chore(deps): update common-dev-assets digest to e137274

This PR contains the following updates:

Package
Update
Change

common-dev-assets
digest
827deac -> e137274

Configuration
📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
👻 Immortal: This PR will be recreated if closed unmerged. Get config help if that's undesired.

[ ] If you want to rebase/retry this PR, check this box

This PR has been generated by Renovate Bot.

:tada: This PR is included in version 2.1.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Hiding a unique data column

I have a situation where I have data with a unique key, but I don't want that data to be displayed in the table since it is as meaningless ID number. None of the other data in the table is unique by itself. I am forced to set a key with the current version, and I can't specify a unique combination of key columns. Would it be an easy improvement to allow a column to be declared as a key, but hide it from the table? Or to allow for more than one key where the combination is unique?

Hi @holmancts
I think the "hidden column on table" is better solution for your situation.
I will start to plan and implement in recent, but it need some time :)

Hi @holmancts
I've release v0.8.2 for supporting hidden column, just use hidden={true} in TableHeaderColumn will hide that column.
Thanks your suggestion ;)


Unable to access function PostGIS_Lib_Version

Hello,
I'm trying to implement PostGIS in my project, using your image mdillon/postgis:10-alpine and jsor/doctrine-postgis PHP library. It tries to use PostGIS_Lib_Version() function, but I have the following error:

Here is a part of my docker configuration:
version: '3'

services:
  # ...
  db:
    image: mdillon/postgis:10-alpine
    healthcheck:
      test: chmod +x /usr/local/bin/healthcheck && /usr/local/bin/healthcheck
    environment:
      - POSTGRES_DB=api_platform
      - POSTGRES_PASSWORD=api_platform
    volumes:
      - ./docker/postgres/healthcheck.sh:/usr/local/bin/healthcheck:rw

I also tried to execute update-postgis.sh script, but everything seems ok:
docker exec -e POSTGRES_USER=postgres db update-postgis.sh

Is there anything I missed?

My bad, extensions are installed by database, and I'm using a different database for the test environment :-)


Rescue worker class sync_workers exceptions and move on

Related to #13958
In the above issue, if ManageIQ::Providers::StorageManager::CinderManager::EventCatcher.sync_workers
raises an exception, the server process exits fatally and
all workers exit Error heartbeating to MiqServer because DRb::DRbConnError: Connection reset by peer Worker exiting.
We now rescue any exceptions here, log it and move on to other worker
classes.

@jrafanie Is there a BZ for this?  Can you please create one if it doesn't exist?

@simaishi thanks!  Here:  https://bugzilla.redhat.com/show_bug.cgi?id=1429020

Euwe backport details:
$ git log -1
commit 15181e4b246aa2620ff82adfe9f35346652d8435
Author: Gregg Tanzillo<EMAIL_ADDRESS>Date:   Fri Feb 17 16:37:37 2017 -0500

    Merge pull request #13976 from jrafanie/rescue_worker_sync_workers

    Rescue worker class sync_workers exceptions and move on
    (cherry picked from commit aaf74065d514f9f3362265f303468d297ac84fdb)

    https://bugzilla.redhat.com/show_bug.cgi?id=1429648


init storing data in mongodb

I moved this into its own folder (called db) to make sure theres no dependencies with our old code. The folder should be renamed to something better and the good code from our previous folder should be migrated over.

Handing this branch off to @AndrewYeeYee, our Database Lead.

I am going to do some more research on Document DB and indexing. Until then I think Andrew is better to review the DB changes. I do think the two scraping methods look good though and like the snippet format change.


Conversion error for parameters when pipelineref.bundle is used

Webhook error messages show:
{"severity":"error","timestamp":"2022-09-15T14:38:48.420Z","logger":"tekton-pipelines-webhook","caller":"conversion/conversion.go:59","message":"Conversion failed","commit":"698f8a7-dirty","uid":"1c56f711-20c2-454d-90d1-0210280ee8f5","desiredAPIVersion":"tekton.dev/v1","error":"unable to marshal output: json: error calling MarshalJSON for type v1.ParamValue: impossible ParamValues.Type: \"\"","stacktrace":"knative.dev/pkg/webhook/resourcesemantics/conversion.(*reconciler).Convert\n\tknative.dev/pkg@v0.0.0-20220818004048-4a03844c0b15/webhook/resourcesemantics/conversion/conversion.go:59\nknative.dev/pkg/webhook.conversionHandler.func1\n\tknative.dev/pkg@v0.0.0-20220818004048-4a03844c0b15/webhook/conversion.go:64\nnet/http.HandlerFunc.ServeHTTP\n\tnet/http/server.go:2109\nnet/http.(*ServeMux).ServeHTTP\n\tnet/http/server.go:2487\nknative.dev/pkg/webhook.(*Webhook).ServeHTTP\n\tknative.dev/pkg@v0.0.0-20220818004048-4a03844c0b15/webhook/webhook.go:262\nknative.dev/pkg/network/handlers.(*Drainer).ServeHTTP\n\tknative.dev/pkg@v0.0.0-20220818004048-4a03844c0b15/network/handlers/drain.go:113\nnet/http.serverHandler.ServeHTTP\n\tnet/http/server.go:2947\nnet/http.(*conn).serve\n\tnet/http/server.go:1991"}

I believe the issue is that param value conversion doesn't handle param type "" (i.e. param type not set-- corresponds to string type)

@lbernick what version does it affect ?

@lbernick what version does it affect ?

Looks like it's the stored version as of v1beta1 according to https://github.com/tektoncd/pipeline/pull/5506#discussion_r972979172.

I think @vdemeester might be asking about the software version not the api version (to know if we need to backport)-- is that right? I think this change was introduced in 0.39.0 so at least we only have to do one backport 😬

@lbernick ah yeah I should have been more clear. And yes, which release, to know if I need to make the PR fixing this in 0.40.0 milestone or not 👼🏼

Yes let's include it in the 0.40.0 milestone and backport to 0.39.0

Well, 0.40.1.. 😅 for some reason I thought it was merged when I chose the commit to be released on.. 🤦🏼

@vdemeester
Release 0.40.2
still see the same issue for some reason
{"severity":"error","timestamp":"2022-10-07T05:09:46.799Z","logger":"tekton-pipelines-webhook","caller":"conversion/conversion.go:59","message":"Conversion failed","commit":"40a438a","uid":"722ad1f6-0b60-4105-aa49-04966ee6ecaf","desiredAPIVersion":"tekton.dev/v1","error":"unable to marshal output: json: error calling MarshalJSON for type v1.ParamValue: impossible ParamValues.Type: \"\"","stacktrace":"knative.dev/pkg/webhook/resourcesemantics/conversion.(*reconciler).Convert\n\tknative.dev/pkg@v0.0.0-20220818004048-4a03844c0b15/webhook/resourcesemantics/conversion/conversion.go:59\nknative.dev/pkg/webhook.conversionHandler.func1\n\tknative.dev/pkg@v0.0.0-20220818004048-4a03844c0b15/webhook/conversion.go:64\nnet/http.HandlerFunc.ServeHTTP\n\tnet/http/server.go:2084\nnet/http.(*ServeMux).ServeHTTP\n\tnet/http/server.go:2462\nknative.dev/pkg/webhook.(*Webhook).ServeHTTP\n\tknative.dev/pkg@v0.0.0-20220818004048-4a03844c0b15/webhook/webhook.go:262\nknative.dev/pkg/network/handlers.(*Drainer).ServeHTTP\n\tknative.dev/pkg@v0.0.0-20220818004048-4a03844c0b15/network/handlers/drain.go:113\nnet/http.serverHandler.ServeHTTP\n\tnet/http/server.go:2916\nnet/http.(*conn).serve\n\tnet/http/server.go:1966"}

Installed
https://github.com/tektoncd/pipeline/releases/download/v0.40.2/release.notags.yaml
https://github.com/tektoncd/triggers/releases/download/v0.21.0/release.notags.yaml
https://github.com/tektoncd/triggers/releases/download/v0.21.0/interceptors.notags.yaml

hum 🤔 #5506 defenitely got in 0.40.1 though.
@mike-serchenia can you share the task/pipeline used to reproduce this ?

@vdemeester Please let me know how if I can help to debug it
Client version: 0.26.0
Pipeline version: v0.40.2
Triggers version: v0.21.0
Dashboard version: v0.29.2

PR
apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
  annotations:
    github.tekton.dev/log-server: https://tekton-log.mydomain.com/logs
    github.tekton.dev/name: '{{ with index .Annotations "tekton.ec.ai/memberOf" }}{{.}}
      / {{ end }}{{ with index .Annotations "tekton.ec.ai/pipeline" }}{{.}}{{ else
      }}{{ index .Labels "tekton.dev/pipeline" }}{{ end }} / {{ index .Labels "tekton.dev/pipelineTask"
      }}'
    github.tekton.dev/owner: mikes
    github.tekton.dev/ref: 7b4347c3e6a2caa2c90d1a50b09c3f19c9a72122
    github.tekton.dev/repo: git-yamls
    github.tekton.dev/url: https://tekton.mydomain.com/#/namespaces/{{ .Namespace }}/pipelineruns/{{
      index .Labels "tekton.dev/pipelineRun" }}?pipelineTask={{ index .Labels "tekton.dev/pipelineTask"
      }}
  creationTimestamp: "2022-10-07T16:46:23Z"
  generateName: kubernetes-sanity-run-
  generation: 1
  labels:
    ec.ai/component: tekton-pipelines
    ec.ai/component_group: tekton
    ec.ai/contract: ec
    ec.ai/data_classification: internal
    ec.ai/environment: src
    ec.ai/managed_by: gitops
    ec.ai/sub_component: pipelinerun
    ec.ai/team: platform
    ec.ai/technical_owner: rluckie
    tekton.dev/pipeline: kubernetes-sanity
  name: kubernetes-sanity-run-bg6h2
  namespace: tekton
  resourceVersion: "59144069"
  uid: a7e67d00-744e-41dc-abf5-1630e662be47
spec:
  params:
  - name: clone-url
    value: https://github.com/mikes/git-yamls.git
  - name: clone-ref
    value: 7b4347c3e6a2caa2c90d1a50b09c3f19c9a72122
  - name: repo-name
    value: git-yamls
  - name: repo-owner-name
    value: mikes
  - name: whitesource-api-key
    value: apiKeyDev
  - name: whitesource-project-version
    value: 1b1b1b1b
  - name: sonar-opts
    value:
    - -Dsonar.pullrequest.key=2701
    - -Dsonar.pullrequest.branch=mike-serchenia-patch-1
    - -Dsonar.pullrequest.base=main
  pipelineRef:
    bundle: us-east4-docker.pkg.dev/mikes-app-source/tekton-catalog/pipeline/kubernetes:0.1
    name: kubernetes-sanity
  podTemplate:
    nodeSelector:
      node_pool: tekton-pipelines
    tolerations:
    - effect: NoSchedule
      key: tekton-pipelines
      operator: Equal
      value: "true"
  serviceAccountName: tekton-build-bot
  timeouts:
    finally: 10m0s
    pipeline: 1h0m0s
    tasks: 30m0s
  workspaces:
  - name: source
    volumeClaimTemplate:
      metadata:
        creationTimestamp: null
      spec:
        accessModes:
        - ReadWriteOnce
        resources:
          requests:
            storage: 5Gi
        storageClassName: rook-ceph-block
      status: {}
  - name: cache
    persistentVolumeClaim:
      claimName: cephfs-pvc
status:
  conditions:
  - lastTransitionTime: "2022-10-07T16:46:53Z"
    message: 'Tasks Completed: 1 (Failed: 0, Cancelled 0), Incomplete: 1, Skipped:
      0'
    reason: Running
    status: Unknown
    type: Succeeded
  pipelineSpec:
    params:
    - name: clone-url
      type: string
    - name: clone-ref
      type: string
    tasks:
    - name: checkout
      params:
      - name: url
        value: https://github.com/mikes/git-yamls.git
      - name: revision
        value: 7b4347c3e6a2caa2c90d1a50b09c3f19c9a72122
      taskRef:
        bundle: gcr.io/tekton-releases/catalog/upstream/git-clone:0.4
        kind: Task
        name: git-clone
      workspaces:
      - name: output
        workspace: source
    - name: lint
      runAfter:
      - checkout
      taskRef:
        bundle: us-east4-docker.pkg.dev/mikes-app-source/tekton-catalog/task/kustomize-lint:0.1
        kind: Task
        name: kustomize-lint
      workspaces:
      - name: source
        workspace: source
    workspaces:
    - name: source
  startTime: "2022-10-07T16:46:24Z"
  taskRuns:
    kubernetes-sanity-run-bg6h2-checkout:
      pipelineTaskName: checkout
      status:
        completionTime: "2022-10-07T16:46:52Z"
        conditions:
        - lastTransitionTime: "2022-10-07T16:46:52Z"
          message: All Steps have completed executing
          reason: Succeeded
          status: "True"
          type: Succeeded
        podName: kubernetes-sanity-run-bg6h2-checkout-pod
        startTime: "2022-10-07T16:46:26Z"
        steps:
        - container: step-clone
          imageID: gcr.io/tekton-releases/github.com/tektoncd/pipeline/cmd/git-init@sha256:db18a9c1607c8cbbcd72f61d0c4d795b9ff528669deacd5f8a1672e4ef198ffd
          name: clone
          terminated:
            containerID: containerd://1ae406984c89765f4c33af71526faee500f0428e2b795a4ac1ee56fe5930e22e
            exitCode: 0
            finishedAt: "2022-10-07T16:46:52Z"
            message: '[{"key":"commit","value":"7b4347c3e6a2caa2c90d1a50b09c3f19c9a72122","type":1},{"key":"url","value":"https://github.com/mikes/git-yamls.git","type":1}]'
            reason: Completed
            startedAt: "2022-10-07T16:46:50Z"
        taskResults:
        - name: commit
          type: string
          value: 7b4347c3e6a2caa2c90d1a50b09c3f19c9a72122
        - name: url
          type: string
          value: https://github.com/mikes/git-yamls.git
        taskSpec:
          description: |-
            These Tasks are Git tasks to work with repositories used by other tasks in your Pipeline.
            The git-clone Task will clone a repo from the provided url into the output Workspace. By default the repo will be cloned into the root of your Workspace. You can clone into a subdirectory by setting this Task's subdirectory param. This Task also supports sparse checkouts. To perform a sparse checkout, pass a list of comma separated directory patterns to this Task's sparseCheckoutDirectories param.
          params:
          - description: Repository URL to clone from.
            name: url
            type: string
          - default: ""
            description: Revision to checkout. (branch, tag, sha, ref, etc...)
            name: revision
            type: string
          - default: ""
            description: Refspec to fetch before checking out revision.
            name: refspec
            type: string
          - default: "true"
            description: Initialize and fetch git submodules.
            name: submodules
            type: string
          - default: "1"
            description: Perform a shallow clone, fetching only the most recent N
              commits.
            name: depth
            type: string
          - default: "true"
            description: Set the `http.sslVerify` global git config. Setting this
              to `false` is not advised unless you are sure that you trust your git
              remote.
            name: sslVerify
            type: string
          - default: ""
            description: Subdirectory inside the `output` Workspace to clone the repo
              into.
            name: subdirectory
            type: string
          - default: ""
            description: Define the directory patterns to match or exclude when performing
              a sparse checkout.
            name: sparseCheckoutDirectories
            type: string
          - default: "true"
            description: Clean out the contents of the destination directory if it
              already exists before cloning.
            name: deleteExisting
            type: string
          - default: ""
            description: HTTP proxy server for non-SSL requests.
            name: httpProxy
            type: string
          - default: ""
            description: HTTPS proxy server for SSL requests.
            name: httpsProxy
            type: string
          - default: ""
            description: Opt out of proxying HTTP/HTTPS requests.
            name: noProxy
            type: string
          - default: "true"
            description: Log the commands that are executed during `git-clone`'s operation.
            name: verbose
            type: string
          - default: gcr.io/tekton-releases/github.com/tektoncd/pipeline/cmd/git-init:v0.21.0
            description: The image providing the git-init binary that this Task runs.
            name: gitInitImage
            type: string
          - default: /tekton/home
            description: |
              Absolute path to the user's home directory. Set this explicitly if you are running the image as a non-root user or have overridden
              the gitInitImage param with an image containing custom user configuration.
            name: userHome
            type: string
          results:
          - description: The precise commit SHA that was fetched by this Task.
            name: commit
            type: string
          - description: The precise URL that was fetched by this Task.
            name: url
            type: string
          steps:
          - env:
            - name: HOME
              value: /tekton/home
            - name: PARAM_URL
              value: https://github.com/mikes/git-yamls.git
            - name: PARAM_REVISION
              value: 7b4347c3e6a2caa2c90d1a50b09c3f19c9a72122
            - name: PARAM_REFSPEC
            - name: PARAM_SUBMODULES
              value: "true"
            - name: PARAM_DEPTH
              value: "1"
            - name: PARAM_SSL_VERIFY
              value: "true"
            - name: PARAM_SUBDIRECTORY
            - name: PARAM_DELETE_EXISTING
              value: "true"
            - name: PARAM_HTTP_PROXY
            - name: PARAM_HTTPS_PROXY
            - name: PARAM_NO_PROXY
            - name: PARAM_VERBOSE
              value: "true"
            - name: PARAM_SPARSE_CHECKOUT_DIRECTORIES
            - name: PARAM_USER_HOME
              value: /tekton/home
            - name: WORKSPACE_OUTPUT_PATH
              value: /workspace/output
            - name: WORKSPACE_SSH_DIRECTORY_BOUND
              value: "false"
            - name: WORKSPACE_SSH_DIRECTORY_PATH
            - name: WORKSPACE_BASIC_AUTH_DIRECTORY_BOUND
              value: "false"
            - name: WORKSPACE_BASIC_AUTH_DIRECTORY_PATH
            image: gcr.io/tekton-releases/github.com/tektoncd/pipeline/cmd/git-init:v0.21.0
            name: clone
            resources: {}
            script: |
              #!/usr/bin/env sh
              set -eu
                  # Script was cut
          workspaces:
          - description: The git repo will be cloned onto the volume backing this
              Workspace.
            name: output
          - description: |
              A .ssh directory with private key, known_hosts, config, etc. Copied to
              the user's home before git commands are executed. Used to authenticate
              with the git remote when performing the clone. Binding a Secret to this
              Workspace is strongly recommended over other volume types.
            name: ssh-directory
            optional: true
          - description: |
              A Workspace containing a .gitconfig and .git-credentials file. These
              will be copied to the user's home before any git commands are run. Any
              other files in this Workspace are ignored. It is strongly recommended
              to use ssh-directory over basic-auth whenever possible and to bind a
              Secret to this Workspace over other volume types.
            name: basic-auth
            optional: true
    kubernetes-sanity-run-bg6h2-lint:
      pipelineTaskName: lint
      status:
        conditions:
        - lastTransitionTime: "2022-10-07T16:47:11Z"
          message: Not all Steps in the Task have finished executing
          reason: Running
          status: Unknown
          type: Succeeded
        podName: kubernetes-sanity-run-bg6h2-lint-pod
        startTime: "2022-10-07T16:46:53Z"
        steps:
        - container: step-lint
          imageID: k8s.gcr.io/kustomize/kustomize@sha256:01c5882c5cb98dee17e1f90ff756e5e4696cf058e671ead9077815333533911d
          name: lint
          running:
            startedAt: "2022-10-07T16:47:10Z"
        taskSpec:
          params:
          - default: |
              check_ignore () { # this function checks if argument contains substring from "ignore" array
              ignore="scripts static-sites"
              for folder in $ignore
                  # Script was cut
            name: SCRIPT
            type: string
          - default: 4.1.3
            name: KUSTOMIZE_VERSION
            type: string
          steps:
          - env:
            - name: SCRIPT_CONTENTS
              value: |
                check_ignore () { # this function checks if argument contains substring from "ignore" array
                ignore="scripts static-sites"
                for folder in $ignore
                  # Script was cut
            image: k8s.gcr.io/kustomize/kustomize:v4.1.3
            name: lint
            resources: {}
            script: |
              #!/usr/bin/env sh
              apk add --no-cache bash
              printf '%s' "${SCRIPT_CONTENTS}" > /tmp/kustomize-lint.sh
              /bin/bash /tmp/kustomize-lint.sh
            workingDir: /workspace/source
          workspaces:
          - name: source

pipeline
---
apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
  name: kubernetes-sanity
spec:
  params:
    - name: clone-url
      type: string
    - name: clone-ref
      type: string
  workspaces:
    - name: source
  tasks:
    - name: checkout
      taskRef:
        name: git-clone
        bundle: gcr.io/tekton-releases/catalog/upstream/git-clone:0.4
      params:
        - name: url
          value: $(params.clone-url)
        - name: revision
          value: $(params.clone-ref)
      workspaces:
        - name: output
          workspace: source
    - name: lint
      runAfter:
        - checkout
      taskRef:
        name: kustomize-lint
        bundle: us-east4-docker.pkg.dev/elementalcognition-app-source/tekton-catalog/task/kustomize-lint:0.1
      workspaces:
        - name: source
          workspace: source
---
apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
  name: kubernetes-sanity
spec:
  params:
    - name: clone-url
      type: string
    - name: clone-ref
      type: string
  workspaces:
    - name: source
  tasks:
    - name: checkout
      taskRef:
        name: git-clone
        bundle: gcr.io/tekton-releases/catalog/upstream/git-clone:0.4
      params:
        - name: url
          value: $(params.clone-url)
        - name: revision
          value: $(params.clone-ref)
      workspaces:
        - name: output
          workspace: source
    - name: lint
      runAfter:
        - checkout
      taskRef:
        name: kustomize-lint
        bundle: us-east4-docker.pkg.dev/my-project-080808/tekton-catalog/task/kustomize-lint:0.1
      workspaces:
        - name: source
          workspace: source

I think the issue caused by bundle: gcr.io/tekton-releases/catalog/upstream/git-clone:0.4
using bundle: gcr.io/tekton-releases/catalog/upstream/git-clone:0.7 seems to be working fine

Thanks @mike-serchenia , but now it feels to me that even not with your case the webhook has been trying to convert to v1.

Ah ok, it's using the bundle, which are not going through the conversion webhook or anything. We should already be running .SetDefaults on what the bundle fetched, so there might be a regression or something overlooked.
cc @JeromeJu

@mike-serchenia but indeed, your bug is still there, I'll re-open and update the title.

(We also need to validate if this is also happening for spec coming out of resolvers)

i was checking requirements to use resolvers
https://tekton.dev/docs/pipelines/bundle-resolver/
A cluster with the alpha feature gate enabled could be an issue since GKE Alpha has the following limitations
https://cloud.google.com/kubernetes-engine/docs/concepts/alpha-clusters#limitations
to be more specific:

Automatically deleted after 30 days

the same problem is still exist in v.0.50.2; Tekton can not be used because of this problem.

the same problem still exists in v.0.50.2; Tekton can not be used because of this problem.

Hi @tirelibirefe , would you mind provide more details on the Pipeline/Task that you are running that we might be able take a look into it?

Hello @JeromeJu
Thank you for your attention.
Unfortunately the customer took Tekton out of scope because of the error; so I won't be able to provide any additional information regarding to the problem.
Thanks & Regards


Use miekg dns library for A record lookups

Instead of using two different libraries for DNS record lookups the test will use only miekg/dns library for both A and CNAME record lookups.

/retest

/retest

/lgtm

/lgtm


Lambda 418 responses trigger AWS/Lambda Errors alarm

Describe the bug
Alarm triggered at 2020-12-07T15:12 (see lamba log on prod) for 418 responses.
We do no expect any errors to be reported upon 418s, so something is behaving unexpectedly.
According to https://docs.aws.amazon.com/lambda/latest/dg/monitoring-metrics.html:

Errors – The number of invocations that result in a function error. Function errors include exceptions thrown by your code and exceptions thrown by the Lambda runtime. The runtime returns errors for issues such as timeouts and configuration errors. To calculate the error rate, divide the value of Errors by the value of Invocations. Note that the timestamp on an error metric reflects when the function was invoked, not when the error occurred.

Probably something in the lambda code is causing the runtime to raise an exception. Should verify that all lambda callbacks don't return values in the error fields unless we really want the alarm to go off.
Alarm: https://github.com/dockstore/dockstore-deploy/blob/aa3ada86318fc1a2150f964df7762cafabe421b4/dockstore.yml.template#L882
Lambda code: https://github.com/dockstore/lambda/blob/develop/upsertGitHubTag-f79c4f36-3e61-43f4-8f6c-0b2e0e9774d7/deployment/index.js
Additional context

Webservice version: 1.10.0
host location: Prod

┆Issue is synchronized with this Jira Story
┆Issue Number: DOCK-1666
┆Sprint: Backlog
┆Issue Type: Story

See SEAB-1692 for the motivation for this alarm, i.e., the alarm in particular should fire for that case.


OldNew Mashup Deprecated and Reverted

I've deprecated the old OldNew Mashup theme and archived it for posterity.  I also reverted that archive to the last release before I started hacking up a storm two weeks ago, and have created a new repo (cshoredaniel/new-oldnew-mashup) for the new version I am working on.  I think the old theme should probably be removed from the directory as there a few warts I'm not happy with, and I'm not planning on fixing them.
I'm not quite ready to add the new theme yet, but I'm working on doing making it work better with this theme repo, as well as in general.

@cshoredaniel Ok. Thanks for the heads up. I will remove your theme shortly.


Implement google-c2p resolver.

As part of this, I've changed the Resolver API such that the base class no longer takes ownership of the WorkSerializer or ResultHandler.  That will require internal changes, so I'll put together a cherry-pick for that once we're happy with this PR.
I don't really have a good way to test the C2P resolver, but I'm hoping that you can put together an interop test that will cover it.

The test failures both look like infrastructure issues.

The test failures both look like infrastructure issues.


docs: misc

see it lively here

use CLIPLoss in get started section for CLIP documentation
add auto-doc generated developer reference back
add Jina echosystem section
documentation style fix.
add back the documentation release CD, not tested (it will replace the current OSS finetuner docs after release).

wired, let me see


proxy settings in sublime server

Hi,
The only thing missing from sublimeserver are proxy settings. Is there any plan to put them there?

+1 for proxy settings and I'd like to suggest including Server Side Includes(shtml)