forceSelfClosingTags may return empty data

Hello,
While testing get-operationnal queries on a juniper device that uses netconf 1.0, I was getting timeouts... so I looked around in the debugger and found out that the forceSelfClosingTags (which needs to be used with juniper because of a bug), erases the query alltogether.
See the following MWP (I copied the forceSelfClosingTags function and hard-coded the pattern):
package main

import (
	"fmt"
	"regexp"
)

func main() {
	req := `
	<?xml version="1.0" encoding="UTF-8"?><rpc xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="101"><get><filter type="subtree">
	<routing-policy xmlns="http://openconfig.net/yang/routing-policy"></routing-policy>
	</filter></get></rpc>]]>]]>
	`
	print("filter")
	print(req)
	print("self-close")
	print(string(forceSelfClosingTags([]byte(req))))
}

func forceSelfClosingTags(b []byte) []byte {

	emptyTagIdxs := regexp.MustCompile(`<(\w+)></\w+>`).FindAllSubmatchIndex(b, -1)

	var nb []byte

	for _, idx := range emptyTagIdxs {
		// get everything in b up till the first of the submatch indexes (this is the start of an
		// "empty" <thing></thing> tag), then get the name of the tag and put it in a self-closing
		// tag.
		nb = append(b[0:idx[0]], fmt.Sprintf("<%s/>", b[idx[2]:idx[3]])...) //nolint: gocritic

		// finally, append everything *after* the submatch indexes
		nb = append(nb, b[len(b)-(len(b)-idx[1]):]...)
	}

	return nb
}

and it's output:
filter
        <?xml version="1.0" encoding="UTF-8"?><rpc xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="101"><get><filter type="subtree">
        <routing-policy xmlns="http://openconfig.net/yang/routing-policy"></routing-policy>
        </filter></get></rpc>]]>]]>
        self-close

It seems to me that the pattern is too restrictive. A self-closing tag may contain any valid attributes.
I've experimented with this replace function instead (with some success):
func forceSelfClosingTags(b []byte) []byte {

	cleanStr := regexp.MustCompile(`<([^>/]+?)>\s*</[\w-]+>`).ReplaceAllString(string(b), "<$1/>")

	return []byte(cleanStr)
}

@netixx when you say with some success, was this replace function failing at some requests still?

It seems to me that the pattern is too restrictive. A self-closing tag may contain any valid attributes.

Yeah, cool, makes sense to me.

@netixx when you say with some success, was this replace function failing at some requests still?

Defo curious the answer to this :D I'm happy to take a look and get a PR merged for this if ya come up with a solution that you are happy with and is working 100%! It would be great if we could get some basic unit tests around this too just so there isn't a regression here as this feels like one I would 100000% forget about and accidentally change at some point :)

Just bumping this one back up! I think it makes sense and I'm happy to get this pattern updated, but I'm not sure all the cases we need to cover. Maybe even just having some example outputs we can dump into regex101 and mess about with would be enough for me to take it from here if you're not able to get a pr together.
Thanks ya'll!
Carl

Hi @carlmontanari
I wanted to revive this, and I first started by creating a few regex101 examples comparing the two approaches. It is indeed valid to have a self closing tag for an element which has attributes (as verified with a validator), e.g.
<elem some=value />

1 Original approach
In the original approach we have a trivial regexp that is indeed only capable of force self-closing on tags that do not have any attributes - https://regex101.com/r/wKlysd/1
2 Proposed approach
@netixx The proposed approach didn't work out for a simple set if XML blobs I used, therefore I had to tweak it a little bit as shown here https://regex101.com/r/rmsS2E/2 (pay attention to the substitution pane where the original XML blob gets transformed)
@carlmontanari I can write some tests to see if that re works, but have a look at it first.

Awesome, thanks for picking this up @hellt !! Yeah the proposed approach sounds good to me, let's do it :D

Hello, sorry for being so late to the party...
By with some success, I meant that I tested it on the one device that was giving me trouble originally and that's it :)
It's a go for @hellt better regex for me as well!

Hah cool, thanks @netixx ! I think Roman mentioned getting this sorted out is somewhere on his todo list so we should have this sorted in a little bit!

Boom, Roman is the man, closing and wrapping this up in #114!


KYLIN-4370 Fix "Table or view not found" error when using JDBC source

Fix "Table or view not found" error when build cube with spark for JDBC source at step "Build cube by Layer"

Codecov Report

Merging #1112 into master will decrease coverage by 0.02%.
The diff coverage is 0%.

@@             Coverage Diff             @@
##             master   #1112      +/-   ##
===========================================
- Coverage     25.02%     25%   -0.03%
- Complexity     6226    6227       +1
===========================================
  Files          1442    1443       +1
  Lines         88023   88143     +120
  Branches      12321   12337      +16
===========================================
+ Hits          22031   22040       +9
- Misses        63832   63936     +104
- Partials       2160    2167       +7

Impacted Files
Coverage Δ
Complexity Δ

.../java/org/apache/kylin/engine/spark/SparkUtil.java
0% <0%> (ø)
0 <0> (ø)
:arrow_down:

...va/org/apache/kylin/cube/model/DictionaryDesc.java
65.38% <0%> (-34.62%)
8% <0%> (+1%)

...he/kylin/dict/lookup/cache/RocksDBLookupTable.java
72.97% <0%> (-5.41%)
6% <0%> (-1%)

...lin/cube/model/validation/rule/DictionaryRule.java
86.84% <0%> (-4.59%)
16% <0%> (+1%)

...kylin/common/persistence/RootPersistentEntity.java
18.6% <0%> (-1.85%)
4% <0%> (ø)

...org/apache/kylin/dimension/AbstractDateDimEnc.java
34.88% <0%> (-1.71%)
6% <0%> (ø)

...org/apache/kylin/metadata/model/DataModelDesc.java
59.21% <0%> (-0.58%)
72% <0%> (-1%)

...ain/java/org/apache/kylin/cube/model/CubeDesc.java
63.56% <0%> (-0.57%)
182% <0%> (+1%)

...ache/kylin/engine/mr/common/AbstractHadoopJob.java
15.21% <0%> (-0.04%)
2% <0%> (ø)

...torage/gtrecord/SortedIteratorMergerWithLimit.java
78.72% <0%> (ø)
2% <0%> (ø)
:arrow_down:

... and 17 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 78deee8...2444690. Read the comment docs.

Pull Request Test Coverage Report for Build 5652

0 of 2 (0.0%)  changed or added relevant lines in 1 file are covered.
1168 unchanged lines in 25 files lost coverage.
Overall coverage decreased (-0.02%) to 27.467%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

engine-spark/src/main/java/org/apache/kylin/engine/spark/SparkUtil.java
0
2
0.0%

Files with Coverage Reduction
New Missed Lines
%

core-dictionary/src/main/java/org/apache/kylin/dict/lookup/cache/RocksDBLookupTable.java
1
81.08%

core-cube/src/main/java/org/apache/kylin/cube/model/validation/rule/DictionaryRule.java
2
94.74%

core-cube/src/main/java/org/apache/kylin/cube/inmemcubing/MemDiskStore.java
3
78.42%

stream-core/src/main/java/org/apache/kylin/stream/core/storage/columnar/compress/FSInputLZ4CompressedColumnReader.java
3
89.47%

core-cube/src/main/java/org/apache/kylin/cube/inmemcubing/RecordConsumeBlockingQueueController.java
4
85.71%

core-metrics/src/main/java/org/apache/kylin/metrics/lib/impl/BlockingReservoir.java
4
0%

core-storage/src/main/java/org/apache/kylin/storage/gtrecord/SortedIteratorMergerWithLimit.java
5
87.23%

stream-core/src/main/java/org/apache/kylin/stream/core/storage/columnar/FSInputGeneralColumnDataReader.java
7
0%

core-cube/src/main/java/org/apache/kylin/cube/model/DictionaryDesc.java
8
69.23%

engine-mr/src/main/java/org/apache/kylin/engine/mr/ByteArrayWritable.java
12
0%

Totals

Change from base Build 5638:
-0.02%

Covered Lines:
24210

Relevant Lines:
88143

💛  - Coveralls


GitPad doesn't work from an elevated command prompt

When attempting to perform a "git commit" operation, it fails with the following message:
---------------------------
App is Elevated
---------------------------
Run this application as a normal user (not as Elevated Administrator)
---------------------------
OK
---------------------------

I didn't launch the command session as an administrator (and it doesn't say "- Administrator" in the title), but it still says this, so that's got to be a bug.  Not sure what else to try here.
Additionally: why should it matter? (normally, I would like to run my command sessions as an administrator.) -- I guess this is more of a feature request: please remove this requirement.  (I'm pretty sure in the past I've used GitPad from an administrative command prompt.)

I usually make the following two settings changes on a new machine -- I had not yet made them on this machine -- once I made the following two setup changes the issue goes away.  But still very odd.

Win+R always run as administrator
Link Administrator Profile with Regular Profile

Running windows 8.1.

This should have been fixed: https://github.com/github/GitPad/commit/15d0a9a9c1d90266ea5556abe2145e02f5a6840d


bad wifi connection - constantly new accessory

Hello @ebaauw,
i have a little problem with homebridge-zp. I have two Sonos Play:1 speakers, one in the living room and one in the bathroom. The bathroom has often times disconnects, due to weak Wi-Fi. 👎
{
"platform": "ZP",
 "service": "switch",
 "brightness": true,
 "speakers": true
},

I think, always, if the WLAN causes problems, HomeKit always get two new characteristics for the bathroom (Sonos + Multiroom) - the old characteristics are deleted. Room assignments are lost, settings in favorites and status also.
Can the problem be solved? So, Sonos Play: 1 in the bathroom is not always recognized as a new speaker?
Cheers 👋
Nico

I think, always, if the WLAN causes problems ... Room assignments are lost, settings in favorites and status also.

This cannot be caused by WLAN problems alone.  Losing room assignments, etc, only happens when the accessory is removed from HomeKit and then re-added, causing HomeKit to treat it as a new accessory.  To remove the accessory from HomeKit, you would have to restart homebridge while the bathroom sonos cannot be reached, so homebridge-zp starts only exposing the living room sonos.  Then you would have to restart homebridge again, and this time homebridge-zp must find the bathroom sonos, so it's exposed again (and seen as a new accessory by HomeKit).
You might try and set "searchTimeout": 15 in config.json, to give homebridge-zp some more time to find the sonos players on startup. The default of 5 seconds is sometimes not enough in my network, so I use this setting.

It looks good - since 2 days no problems. I'll get back in 5 days. 👍

Closing this issue.  Please re-open if there's still problems.

Unfortunately, the problem with searchTimeout has not been resolved. I have now added a new wlan repeater to solve the wireless problemes in the bathroom. Today, was the Play:1 in the livingroom removed and new added in HomeKit - a other wlan repeater is 3 meters away. 😢
Everything is quite strange.
I have now set journald logging to persistent and will report again. Whether at the time of restarting Homebridge or in case of a crash, the sonos player is really unreachable / disconnected.
Kind regards,
Nico

Do you connect each Sonos player to your WiFi network separately, or do you use the Sonos WiFi mesh network?  I find that in my case, using the Sonos mesh is far more reliable.

Hi @ebaauw,
i use my own WiFI network. I needed, for my music library on my Samba- / Windows-File-Share. 👎

Hi @ebaauw,
you're right, if homebridge crashes (shit logitech harmony plugin) or reboots and the sonos player was not found, it will be added next time as a new accessory.
Homebridge crash, automatically restart without sonos player at bathroom (Bad):
Oct 27 09:04:10 rpi-smarthome homebridge[28888]: [2018-10-27 09:04:10] [ZP] warning: lastest version: homebridge-zp v0.2.35
Oct 27 09:04:10 rpi-smarthome homebridge[28888]: [2018-10-27 09:04:10] [ZP] Wohnzimmer: setup ZPS12 v9.2 player RINCON_949F3E027C8E01400 at <IP_ADDRESS>:1400
Oct 27 09:04:10 rpi-smarthome homebridge[28888]: [2018-10-27 09:04:10] [Staubsauger EG Flur] Firmwareversion: 3.3.9_001518
...
Oct 27 09:04:12 rpi-smarthome homebridge[28888]: [2018-10-27 09:04:12] [Harmony Hub] Found activities:
Oct 27 09:04:12 rpi-smarthome homebridge[28888]:         PowerOff
Oct 27 09:04:12 rpi-smarthome homebridge[28888]:         Media Center
Oct 27 09:04:12 rpi-smarthome homebridge[28888]:         PlayStation 4
Oct 27 09:04:23 rpi-smarthome homebridge[28888]: [2018-10-27 09:04:23] [ZP] listening on http://<IP_ADDRESS>:44589/notify
Oct 27 09:04:23 rpi-smarthome homebridge[28888]: [2018-10-27 09:04:23] [ZP] Wohnzimmer Sonos: SUBSCRIBE http://<IP_ADDRESS>:1400/GroupManagement/Event
Oct 27 09:04:23 rpi-smarthome homebridge[28888]: [2018-10-27 09:04:23] [ZP] Initializing platform accessory 'Wohnzimmer Sonos'...
Oct 27 09:04:23 rpi-smarthome homebridge[28888]: Setup Payload:
...

Next homebridge restart with both sonos player (Bad, Wohnzimmer):
...
Oct 27 09:32:27 rpi-smarthome homebridge[3558]: [2018-10-27 09:32:27] [ZP] warning: lastest version: homebridge-zp v0.2.35
Oct 27 09:32:27 rpi-smarthome homebridge[3558]: [2018-10-27 09:32:27] [Staubsauger EG Flur] Firmwareversion: 3.3.9_001518
Oct 27 09:32:27 rpi-smarthome homebridge[3558]: [2018-10-27 09:32:27] [ZP] Bad: setup ZPS12 v9.2 player RINCON_949F3E027CE401400 at <IP_ADDRESS>:1400
Oct 27 09:32:27 rpi-smarthome homebridge[3558]: [2018-10-27 09:32:27] [ZP] Wohnzimmer: setup ZPS12 v9.2 player RINCON_949F3E027C8E01400 at <IP_ADDRESS>:1400
...
Oct 27 09:32:30 rpi-smarthome homebridge[3558]: [2018-10-27 09:32:30] [Harmony Hub] Found activities:
Oct 27 09:32:30 rpi-smarthome homebridge[3558]:         PowerOff
Oct 27 09:32:30 rpi-smarthome homebridge[3558]:         Media Center
Oct 27 09:32:30 rpi-smarthome homebridge[3558]:         PlayStation 4
Oct 27 09:32:40 rpi-smarthome homebridge[3558]: [2018-10-27 09:32:40] [ZP] listening on http://<IP_ADDRESS>:42707/notify
Oct 27 09:32:40 rpi-smarthome homebridge[3558]: [2018-10-27 09:32:40] [ZP] Bad Sonos: SUBSCRIBE http://<IP_ADDRESS>:1400/GroupManagement/Event
Oct 27 09:32:40 rpi-smarthome homebridge[3558]: [2018-10-27 09:32:40] [ZP] Wohnzimmer Sonos: SUBSCRIBE http://<IP_ADDRESS>:1400/GroupManagement/Event
Oct 27 09:32:40 rpi-smarthome homebridge[3558]: [2018-10-27 09:32:40] [ZP] Initializing platform accessory 'Bad Sonos'...
Oct 27 09:32:40 rpi-smarthome homebridge[3558]: [2018-10-27 09:32:40] [ZP] Initializing platform accessory 'Wohnzimmer Sonos'...
Oct 27 09:32:40 rpi-smarthome homebridge[3558]: Setup Payload:
...

The bad Wi-Fi connection should not be there, the problem I have fixed. 👍
I set now the searchTimeout to 30.

Hi @ebaauw,
searchTimeout to 30 brought no improvement. 😢
According to my nagios monitoring the sonos speaker in the bathroom was still hard to reach. (ping) I restart my two sonos speaker 3 days ago, since then no disconnections and no new accessory in HomeKit.
I've written a bash script that automatically reboots my speakers once a week.
https://github.com/nicoh88/cron_sonos-reboot


Override location of project environments

anaconda-project offers the environment variable ANACONDA_PROJECT_ENVS_PATH as a means to override the location where environments are created. By default of course they go in ./envs/<name>, but we may have reason to home the environments outside of the project itself.
This variable actually accepts a comma-separated list of values. New environments are created in the first writable directory on the path.
You can see how anaconda-project does this here:
https://github.com/Anaconda-Platform/anaconda-project/blob/master/anaconda_project/env_spec.py#L318-L348
More information here: https://anaconda-project.readthedocs.io/en/latest/config.html

PR #159 adds support to change the location of envs


[OpenAPI Hub] feature request: automatically maintain readme.test.md file as well when adding new api version

Currently OpenAPIHub automates most steps to create OpenAPI Specifications (a.k.a Swaggers) required when adding a new API version: doc
It mainly copies swagger/examples/scenarios to new api-version folder, and update the api-version as well as readme.md tags. However, tags in readme.test.md file currently are not updated automatically. The request is to support it as well.
See PR: https://github.com/Azure/azure-rest-api-specs/pull/22557

While we're not doing new features in OpenAPI Hub (and it will eventually be retired), we should absolutely take this into account when implementing Resource Planner.

@JonathanCrd is this being consider as part of the work on https://github.com/Azure/azure-rest-api-specs/pull/27932 ?

@JonathanCrd is this being consider as part of the work on Azure/azure-rest-api-specs#27932 ?

It was not, but I'll add this to the project backlog to include it as an enhancement for the script.

It was not, but I'll add this to the project backlog to include it as an enhancement for the script.

Before you add it, make sure to reach out to the person that created the issue and see if this is still needed.

As we keep removing the support for OpenAPI, this feature request will not happen either


custom tools for agent

I come from the langchain python docs, where it was quite simple to initialize an agent based on another function. What would be the equivalent of making a custom tool in langchainjs?
Thank you in advance! 🦜
python code i'm trying to replicate into langchainjs:
def extract_youtube_transcript(youtube_url: str) -> str:
    """
    Given the YouTube Url
    1. Get the raw transcript
    2. Summarize it for us into a ChromaDB
    """
    loader = YoutubeLoader.from_youtube_url(youtube_url, add_video_info=True, language="en-US")
    summary = loader.load()
    return summary

def summarize_youtube_transcript(url: str) -> str:
    # Initialize with tools, llm, template
    agent_tools = [
        Tool(
            name="Crawl Google",
            func=extract_youtube_transcript,
            description="useful for when you need to get a YouTube video url",
        ),
    ]

I think you're looking for the DynamicTool. You can find the documentation here.

The DynamicTool class takes as input a name, a description, and a function. Importantly, the name and the description will be used by the language model to determine when to call this function and with what parameters!

  const tools = [
    new DynamicTool({
      name: "FOO",
      description:
        "call this to get the value of foo. input should be an empty string.",
      func: () => "baz",
    }),
    new DynamicTool({
      name: "BAR",
      description:
        "call this to get the value of bar. input should be an empty string.",
      func: () => "baz1",
    }),
  ];


URLErrors on Windows for conda:

Snakemake version: 5.7.4
Using conda: in a rule results in URLError on Windows
$ snakemake -k  -j --use-conda
The flag 'directory' used in rule all is only valid for outputs, not inputs.
Building DAG of jobs...
WorkflowError:
Failed to open environment file Z:\Kimberly\Projects\SBEM_analysis\src\sbem_analysis\snakemake_genes\envs/snakemake_genes.yaml:
URLError:
Example rule that errors:
rule QC_graphs:
input:
QC_stats = "tables/after_QC_{gene_assignment}.csv"
output:
    QC = directory("QC_{gene_assignment}")

conda:
    "envs/snakemake_genes.yaml"

log:
    "logs/QC_graphs_{gene_assignment}.log"

threads: 8

script:
    "scripts/QC_graphs.py"

I am seeing the same with snakemake 5.12.3 on Windows 10:
> snakemake -j 8 -R dummy --use-conda
Building DAG of jobs...
WorkflowError:
Failed to open environment file C:\Users\angelo.peronio\Documents\lavoro-Angelo\muetec\build\2020-03-05-snakemake-test\build.condaenv.yaml:
URLError: <urlopen error unknown url type: c>
  File "C:\tools\miniconda3\envs\muepy-dev\lib\site-packages\snakemake\deployment\conda.py", line 198, in create
  File "C:\tools\miniconda3\envs\muepy-dev\lib\site-packages\snakemake\deployment\conda.py", line 70, in content
  File "C:\tools\miniconda3\envs\muepy-dev\lib\site-packages\snakemake\deployment\conda.py", line 37, in content

The Snakefile:
rule dummy:
    output:
        "dummy.file"
    conda:
        "build.condaenv.yaml"
    shell:
        "echo dummy > dummy.file"

I guess it is how urllib works.
If I understand correctly, the problem is here:
https://github.com/snakemake/snakemake/blob/1e2e234e24b0473fe669124e5d4422385eb5cf12/snakemake/deployment/conda.py#L32-L38
If interested, I could put together a pull request.

Any movement on this? Was there a PR created?

@svHatch I think I fixed this issue. The tests pass on windows now, but please help test it in real projects.


Display local city names labels in translated labels.

By default, Baremaps displays city names and labels in the local language only.
For countries that use a non Latin alphabet (e.g. cyrillic, arabic or east Asian scripts), that impedes usuabiliy as it is much harder to navigate on the map without prior knowledge of the local language.
I'd like the ability to display the map in a given language, or better, the ability to display both the labels in the local language and the translation below. Ideally, the default translation would be English but it could be chosen by the user, as to not make it Western-centric only.

I just added a language selector in the web site that enable the selection of the base language.
https://github.com/apache/incubator-baremaps-site/pull/43
Unfortunately, this solution only work at low zoom level where we select all the attributes. We should keep this issue open until the basemap gets modified accordingly.


Automate tests for DNS Entries

Description

Automate tests for DNS Entries:

create view
list view
details view

Please note that this task depends on #552 - DNS Provider must exist to create a DNS Entry.


Double click + shift click should select all words in range

On 1.25.1, with no extensions.
If I have text Hello World, and I do:

Double click on H, so Hello is selected
Shift-Click on o

In Word, Notepad, TextEdit, Chrome and many other apps: This would select over Hello World.
In Code, this selects Hello Wo.

I personally prefer Code to select Hello Wo in your example rather than the suggested behavior of selecting the words in range.
If this functionality were to be implemented, I'd like to see it as an option rather than the only available behavior of shift+click.


must be python

I had python 3 installed as python3
So, it didn't work as python.
Without running in a container with complete control over which python it is, seems like it should be flexible to a python or python3 call.
ubuntu@actively-champion-shrimp:~/analysis/n_t$ snakemake
Building DAG of jobs...
Using shell: /bin/bash
Provided cores: 1
Rules claiming more threads will be scaled down.
Job counts:
        count   jobs
        1       all
        1       simulation
        1       stairwayplot
        3

[Fri Mar  8 00:38:18 2019]
rule simulation:
    output: G3OOA.trees
    jobid: 1

/bin/bash: python: command not found
[Fri Mar  8 00:38:18 2019]
Error in rule simulation:
    jobid: 1
    output: G3OOA.trees

RuleException:
CalledProcessError in line 11 of /home/ubuntu/analysis/n_t/Snakefile:
Command 'set -euo pipefail;  python GutenkunstThreePopOutOfAfrica.py' returned non-zero exit status 127.
  File "/home/ubuntu/analysis/n_t/Snakefile", line 11, in __rule_simulation
  File "/usr/lib/python3.6/concurrent/futures/thread.py", line 56, in run
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Complete log: /home/ubuntu/analysis/n_t/.snakemake/log/2019-03-08T003818.632251.snakemake.log

Although... maybe that wasn't the problem.
I added alias python=python3 to my ~/.bashrc, and now python works.
But, when I run snakemake I'm getting the same error.
This relates back to https://github.com/andrewkern/analysis/issues/2

think this was a hard coded path issue that should now be fixed. will close this but feel free to reopen when you pull the new version and test it

I'm not entirely sure if that fixed the python3 issue. Now I'm getting a new error.
I'm going to keep this closed for now, because I think this error has more to do with https://github.com/andrewkern/analysis/issues/2


ARROW-11610: [C++] Download boost from sourceforge instead of bintray

This also removes some bintray URLs where we had mirrored dependency versions for redundancy (cf. ARROW-11611); they're not active anymore anyway because we've bumped the dependency versions and didn't update our mirroring.
For what it's worth, our builds won't touch the updated boost on sourceforge because we still have our trimmed boost bundle in front of it (updating that is ARROW-11612), so we'll have to confirm this by other means.

The description looks good. However, actual changes may include unrelated commits...

@github-actions rebase

Thanks @kiszk, looks like that was a side effect of the accidental force-push to master. I've rebased so it should be cleaned up now.

Looks good to me.

Doesn't work:
$ ./build-support/trim-boost.sh
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  108k  100  108k    0     0  81607      0  0:00:01  0:00:01 --:--:-- 81607

gzip: stdin: not in gzip format
tar: Child returned status 1
tar: Error is not recoverable: exiting now

The URL (https://sourceforge.net/projects/boost/files/boost/1.71.0/boost_1_71_0.tar.gz) is a 404. Did you test it before submitting this PR?

I did check sourceforge but apparently not with the version of boost we currently are pinned to. For some reason it looks like every other version is present there (https://sourceforge.net/projects/boost/files/boost/) but 1.71.0 is empty (https://sourceforge.net/projects/boost/files/boost/1.71.0/).
I think to move forward then, we can (1) rebuild and publish the trimmed boost bundle with the latest version of boost (alongside the 1.71.0 version that's there now) and then (2) bump the boost version in versions.txt.

Oh, I checked the directory exists, but I did not see files under the directory. Sorry.

I tried bumping boost to the latest (1.75.0), and that does successfully download from sourceforge. However, the build fails because of Thrift 0.12's use of deprecated boost endian headers (cf. https://issues.apache.org/jira/browse/THRIFT-4836), which were removed in boost 1.73.0. This was apparently fixed in Thrift 0.13.
I see three options:

Upgrade to thrift 0.13 to allow newer boost. This was stalled before because thrift 0.13 apparently requires newer cmake than we do. https://issues.apache.org/jira/browse/THRIFT-5137 is the issue we created about that. It looks like an easy fix, which we could then apply as a patch step in our cmake build to allow 0.13.
Apply the patch from THRIFT-4861 in our build_thrift cmake macro and keep going with thrift 0.12 while allowing us to upgrade to newer boost.
Bump boost only to 1.72, the last version before the deprecated headers were removed, and don't bother with thrift.

IMO option 1 is best; I think thrift 0.13 also uses even less boost than 0.12, so we could prune our boost bundle further. And I suspect it only requires a very small change. Option 3 is the least effort but just defers the maintenance burden.

I like 1. I think that we can require CMake 3.10 or later only when an user wants to use bundled Thrift.
We already requires CMake 3.7 or later only when an user wants to use bundled Zstandard.

FYI: We'll drop support for Ubuntu 16.04 LTS soon (2021-04 is EOL of Ubuntu 16.04 LTS) and Ubuntu 18.04 LTS or later ship CMake 3.10 or later. We can use CMake 3.17.5 from EPEL on CentOS 7.

Rebased after upgrading thrift to 0.13 in #9658

@github-actions crossbow submit -g nightly


Performance improvements for Hashtables, ForEach, and Where-Object

This issue is change the way certain functions work in Pode, to aid in improving its performance. All the following changes are quicker, and although in small cases they're negligible it could still matter for large files or hundreds of rapid requests.
The following changes are to be applied to code that gets called on web requests; general logic on initial server start isn't mandatory as it's called once.

All references to ForEach-Object to be replaced with a normal foreach loop.
All references to Where-Object to be replaced with a foreach loop with an if statement.
All hashtable property references to be changed to indexing: $ht[$key] = $value

Another speed improvement:

Change all $array | Select-Object -First 1 to $array[0] - but protect again nulls.

And also -Last 1 to $array[$array.Length - 1]

Ran some performance tests over the branch with the changes, and the current develop branch; and looks quite good.
develop:
checks.....................: 100.00% ✓ 13016 ✗ 0
http_req_duration..........: avg=91.88ms min=54.99ms med=88.01ms max=222.05ms p(90)=106.01ms p(95)=119.03ms
http_reqs..................: 6508    108.464655/s
iteration_duration.........: avg=92.07ms min=57ms    med=88.02ms max=222.05ms p(90)=106.17ms p(95)=119.52ms
iterations.................: 6508    108.464655/s

Issue-258:
checks.....................: 100.00% ✓ 15608 ✗ 0
http_req_duration..........: avg=76.62ms min=16ms    med=76.01ms max=196.02ms p(90)=82.01ms p(95)=84.01ms
http_reqs..................: 7804    130.065655/s
iteration_duration.........: avg=76.8ms  min=18.99ms med=76.01ms max=197.02ms p(90)=82.01ms p(95)=84.02ms
iterations.................: 7804    130.065655/s

Awesome! Good job!

Continuing with the changes I've also:

Modified Test-Empty to used -is rather than .GetType(). (doubles the functions performance)

Ie: Checking a string is down from 20ms to 10ms

Updated any string checks to use [string]::IsNullOrWhiteSpace(..) (drastically increases performance)
Updated array checks to directly check $null and .Length (same increase as strings)

With this, here's the new stats:
checks.....................: 100.00% ✓ 20252 ✗ 0
http_req_duration..........: avg=58.94ms min=10ms    med=57ms max=134ms  p(90)=66.99ms p(95)=71ms
http_reqs..................: 10126   168.761103/s
iteration_duration.........: avg=59.12ms min=11.97ms med=57ms max=134ms  p(90)=67ms    p(95)=71.99ms
iterations.................: 10126   168.761103/s


on itemSelected requires  stopPropagation in order to prevent clicking another item/button behind

I am having issue with (itemSelect) while I am select an item, it also triggers a click behind the list. Is there a way to event.stopPropagation()?

Unfortunately not in the current version, but I will add a support for that in the next release.

fixed in 64063f45dba0652088b0adc6686653f96d631cf1

@kadoshms the issue seems to occur again, I am testing it on Android and it does occur. Can you please return the event somehow?

@kadoshms confirmed, the issue is still outstanding, at least on Android platform..

@kadoshms Also seeing this bug. Is there a fix? This almost makes the plugin unusable.

@kadoshms  Really still occurs


policy handle capabilities validation

I'll merge this now and make changes discussed yesterday in follow up PR, then follow with Kotlin implementation as well.

FTR, this PR is a good example where I have to read all of the changed files to try to determine exactly what we are doing, since the commit message itself doesn't provide sufficient detail. See also the note I just sent in chat/email.

FTR, this PR is a good example where I have to read all of the changed files to try to determine exactly what we are doing, since the commit message itself doesn't provide sufficient detail. See also the note I just sent in chat/email.

Yeah, i totally understand, i'll write better descriptions next time, sorry for the inconvenience.


doctests

Hey there, thanks for the great plugin!
Bit in a hurry right now, so just wanted to let you know that it seems that doctests are not fully supported (not listed in test summary, no indicator in signs?)
Let me know if I can help with this!

Hey!
Yeah, support for doctests was missing, I just hadn't gotten around to add it.
It was rather simple, so just added it. Let me know what you think.
For now, the doctests appear as a single test, the same way dynamic tests are handled.
I'm working improving that for dynamic tests, and the change will apply to doctests too.

Thanks, freat work!


Translate empty struct types as object instead of {}

Currently, cuetsy translates an empty struct:
f: {...} @cuetsy(@kind="type")

To an empty object:
export type f = {};

It appears that {} is subtly different than the primitive type object in TypeScript's type system, as indicated by this playground). I think object is what we want here, but will probably have to research a bit further before committing to the approach.

It should probably be Record<string, unknown> if we want to express the equivalent of "any struct", see https://www.reddit.com/r/typescript/comments/tq3m4f/the_difference_between_object_and_recordstring/

Interesting! Yes, object clearly does still allow array types, which is a possibility we want to eliminate.


Picture is not taken in full resolution but scaled

Bug Report
Problem
What is expected to happen?
Picture should be taken in full resolution
What does actually happen?
Picture is taken not in full resolution but rather scaled.
Information
CameraPreview.m
Line 704 seems to produce this error. When the same Resolution as the picture is given as a parameter, the image gets scaled by the factor 0.75.
Command or Code

Environment, Platform, Device
IOS
Version information

Checklist

[ X] I searched for existing GitHub issues
[X] I updated all Cordova tooling to most recent version
[X] I included all the necessary information above

CameraPreview.m
Line 704 seems to produce this error. When the same Resolution as the picture is given as a parameter, the image gets scaled by the factor 0.75.

This plugin does not have a CameraPreview class.
https://github.com/apache/cordova-plugin-camera/tree/master/src/ios

Youre totally right, had multiple tabs with almost the same name. Moved it to the right project (cordova-plugin-camera-preview).

Youre totally right, had multiple tabs with almost the same name. Moved it to the right project (cordova-plugin-camera-preview).


Adds extra options for spo page remove command

Closes #6294

In the issue was asked to be able to remove pages that were locked for shared use. In the meantime it looked a nice enhancement to also include a --recycle flag to recycle pages instead of permanently deleting them.

We'll do our best to get this in ASAP @paul-gladoon. If you are really blocked by this, you can reach out to me to make it working using the m365 request command.

Thanks @milanholemans for this addition. Great work.


Is there a way to retry the job after the job is failed in Azkaban?

Is there a way to retry the job after the job is failed in Azkaban automatically? Like add new config of the retry times for jobs on Azkaban server.

Take a look at this: http://azkaban.github.io/azkaban/docs/latest/#configuration
You can set retry attempts


Delete button on AliasDetail screen doesn't work

Describe the bug
The delete button on the AliasDetail screen doesn't work in the latest version - seems to be independent of the DB backend.
To Reproduce
Open an alias, try to delete, receive spinning wheel of death.
Logs show the request is never submitted to the API
Expected behavior
Delete request is submitted to the api and alias is deleted.

Looks like line 463 in api.js never actually fires it does everything before it, but the api never gets the request.
I'm confused.....
        Api.AliasDetail.delete({id: $routeParams.id}, $scope.alias).$promise.then(function (response) {

Changing the above line to Api.AliasDetail.save does save the capcode, so there is nothing wrong with the code that i can tell.
Something about the delete method isn't being liked.
Anyone have any ideas?

::1 - - [07/May/2020:06:38:54 +0000] "DELETE /api/capcodes/104 HTTP/1.1" - - "http://localhost:3000/admin/aliases/104"

Maybe i was wrong, it does seem to be submitted, but nothing happens to it.

Never used Angular 1 only 2+, but is here is missing delete option
https://github.com/pagermon/pagermon/blob/5aaf1f0e8655d66f3a522a4af5f97785a2bbc63d/server/themes/default/public/javascripts/admin/admin.main.js#L9
?
Called from here https://github.com/pagermon/pagermon/blob/5aaf1f0e8655d66f3a522a4af5f97785a2bbc63d/server/themes/default/public/javascripts/admin/admin.main.js#L462

The documentation i found seemed to suggest delete was an inherent option. Save is also missing but works...
Either way i did try creating it there with no luck

Spent a few hrs messing around with this again last night, couldn't get it going. Code hasn't changed since original version, any ideas @davidmckenzie ?

My prod's on 0.3.5 - deleting from list view works, deleting from detail page times out. It's definitely firing the request, just getting a timeout, so don't think anything wrong on frontend side, it's likely that API route is just broken.
It never hits logger.main.info('Deleting '+id); so it's either a problem with router.delete calls in general, or something weird with var id = parseInt(req.params.id, 10);

i think it's with the router.delete.
I actually put a console log above that and it never gets triggered :(


setRemoteDescription OperationError: Failed to set remote offer

Hi, I'm trying to set up collider and coturn on my google vm instance. My file constants.py looks like that, with leia.tureex.com forwarding to my external ip:
ICE_SERVER_OVERRIDE  = [
{
"urls": [
"turn:leia.tureex.com:19305?transport=udp",
"turn:leia.tureex.com:19305?transport=tcp"
],
"username": "arnaud",
"credential": "leia"
},
{
"urls": [
"stun:leia.tureex.com:19302"
]
}
]
#ICE_SERVER_OVERRIDE = None
ICE_SERVER_BASE_URL = 'https://networktraversal.googleapis.com'
ICE_SERVER_URL_TEMPLATE = '%s/v1alpha/iceconfig?key=%s'
ICE_SERVER_API_KEY = os.environ.get('ICE_SERVER_API_KEY')
WSS_INSTANCE_HOST_KEY = 'host_port_pair'
WSS_INSTANCE_NAME_KEY = 'vm_name'
WSS_INSTANCE_ZONE_KEY = 'zone'
WSS_INSTANCES = [{
WSS_INSTANCE_HOST_KEY: 'leia.tureex.com:443',
WSS_INSTANCE_NAME_KEY: 'holo-apprtc-1',
WSS_INSTANCE_ZONE_KEY: 'us-west1-c'
}, {
WSS_INSTANCE_HOST_KEY: 'leia.tureex.com:443',
WSS_INSTANCE_NAME_KEY: 'holo-apprtc-1',
WSS_INSTANCE_ZONE_KEY: 'us-west1-c'
}]
My main.go file looks like that:
var tls = flag.Bool("tls", true, "whether TLS is used")
var port = flag.Int("port", 443, "The TCP port that the server listens on")
var roomSrv = flag.String("room-server", "https://leia.tureex.com", "The origin of the room server")
My turn server.conf looks like that:
listening-port=3478
tls-listening-port=5349
alt-listening-port=0
alt-tls-listening-port=0
cert=/cert/cert.pem
pkey=/cert/key.pem
verbose
fingerprint
lt-cred-mech
use-auth-secret
static-auth-secret=arnaud
user=arnaud:leia
stale-nonce
no-loopback-peers
no-multicast-peers
The coturn log (after running sudo turnserver) looks like that:
0: log file opened: /var/log/turn_25436_2018-06-22.log
0:
RFC 3489/5389/5766/5780/6062/6156 STUN/TURN Server
Version Coturn-<IP_ADDRESS> 'dan Eider'
0:
Max number of open files/sockets allowed for this process: 1048576
0:
Due to the open files/sockets limitation,
max supported number of TURN Sessions possible is: 524000 (approximately)
0:
==== Show him the instruments, Practical Frost: ====
0: TLS supported
0: DTLS supported
0: DTLS 1.2 supported
0: TURN/STUN ALPN supported
0: Third-party authorization (oAuth) supported
0: GCM (AEAD) supported
0: OpenSSL compile-time version: OpenSSL 1.0.2g  1 Mar 2016 (0x1000207f)
0:
0: SQLite supported, default database location is /usr/local/var/db/turndb
0: Redis supported
0: PostgreSQL supported
0: MySQL supported
0: MongoDB is not supported
0:
0: Default Net Engine version: 3 (UDP thread per CPU core)
=====================================================
0: Config file found: /usr/local/etc/turnserver.conf
0: Config file found: /usr/local/etc/turnserver.conf
0: Domain name:
0: Default realm:
0:
CONFIGURATION ALERT: you did specify the long-term credentials usage
but you did not specify the default realm option (-r option).
 TLS/SCTP listener opened on : <IP_ADDRESS>:3478
0: IPv4. TLS/TCP listener opened on : <IP_ADDRESS>:3478
0: IPv4. TLS/SCTP listener opened on : <IP_ADDRESS>:3479
0: IPv4. TLS/TCP listener opened on : <IP_ADDRESS>:3479
0: IPv4. TLS/TCP listener opened on : <IP_ADDRESS>:5349
0: IPv4. TLS/TCP listener opened on : <IP_ADDRESS>:5350
0: IPv4. TLS/SCTP listener opened on : <IP_ADDRESS>:3478
0: IPv4. TLS/TCP listener opened on : <IP_ADDRESS>:3478
0: IPv4. TLS/SCTP listener opened on : <IP_ADDRESS>:3479
0: IPv4. TLS/TCP listener opened on : <IP_ADDRESS>:3479
0: IPv4. TLS/TCP listener opened on : <IP_ADDRESS>:5349
0: IPv4. TLS/TCP listener opened on : <IP_ADDRESS>:5350
0: IPv6. TLS/SCTP listener opened on : ::1:3478
0: IPv6. TLS/TCP listener opened on : ::1:3478
0: IPv6. TLS/SCTP listener opened on : ::1:3479
0: IPv6. TLS/TCP listener opened on : ::1:3479
0: IPv6. TLS/TCP listener opened on : ::1:5349
0: IPv6. TLS/TCP listener opened on : ::1:5350
0: IO method (general relay thread): epoll (with changelist)
0: turn server id=1 created
0: IPv4. TLS/TCP listener opened on : <IP_ADDRESS>:3478
0: IPv4. TLS/TCP listener opened on : <IP_ADDRESS>:3479
0: IPv4. TLS/TCP listener opened on : <IP_ADDRESS>:5349
0: IPv4. TLS/TCP listener opened on : <IP_ADDRESS>:5350
0: IPv4. TLS/TCP listener opened on : <IP_ADDRESS>:3478
0: IPv4. TLS/TCP listener opened on : <IP_ADDRESS>:3479
0: IPv4. TLS/TCP listener opened on : <IP_ADDRESS>:5349
0: IPv4. TLS/TCP listener opened on : <IP_ADDRESS>:5350
0: IPv6. TLS/TCP listener opened on : ::1:3478
0: IPv6. TLS/TCP listener opened on : ::1:3479
0: IPv4. DTLS/UDP listener opened on: <IP_ADDRESS>:3478
0: IPv4. DTLS/UDP listener opened on: <IP_ADDRESS>:3479
0: IPv4. DTLS/UDP listener opened on: <IP_ADDRESS>:5349
0: IPv4. DTLS/UDP listener opened on: <IP_ADDRESS>:5350
0: IPv4. DTLS/UDP listener opened on: <IP_ADDRESS>:3478
0: IPv4. DTLS/UDP listener opened on: <IP_ADDRESS>:3479
0: IPv4. DTLS/UDP listener opened on: <IP_ADDRESS>:5349
0: IPv4. DTLS/UDP listener opened on: <IP_ADDRESS>:5350
0: IPv6. DTLS/UDP listener opened on: ::1:3478
0: IPv6. DTLS/UDP listener opened on: ::1:3479
0: IPv6. DTLS/UDP listener opened on: ::1:5349
0: IPv6. DTLS/UDP listener opened on: ::1:5350
0: Total General servers: 2
0: IPv6. TLS/TCP listener opened on : ::1:5349
0: IPv6. TLS/TCP listener opened on : ::1:5350
0: IO method (admin thread): epoll (with changelist)
0: IO method (auth thread): epoll (with changelist)
0: IO method (auth thread): epoll (with changelist)
0: SQLite DB connection success: /usr/local/var/db/turndb
0: IPv4. CLI listener opened on : <IP_ADDRESS>:5766
The corider log looks like that (after running sudo /collider/collidermain -port=443 -tls=true):
018/06/22 23:41:52 Starting collider: tls = true, port = 443, room-server=https://leia.tureex.com
2018/06/22 23:42:04 Created room 20000
2018/06/22 23:42:04 Added client 82663979 to room 20000
2018/06/22 23:42:04 Client 82663979 registered in room 20000
2018/06/22 23:43:48 Added client 34371593 to room 20000
2018/06/22 23:43:48 Client 34371593 registered in room 20000
2018/06/22 23:43:48 Sent queued messages from 82663979 to 34371593
2018/06/22 23:52:00 Deregistered client 34371593 from room 20000
2018/06/22 23:52:00 Removed client 34371593 from room 20000
2018/06/22 23:52:10 Removing client 34371593 from room 20000 due to timeout
2018/06/22 23:54:06 Deregistered client 82663979 from room 20000
2018/06/22 23:54:06 Removed client 82663979 from room 20000
2018/06/22 23:54:06 Removed room 20000
2018/06/22 23:54:14 Created room 206000
2018/06/22 23:54:14 Added client 03566176 to room 206000
2018/06/22 23:54:14 Client 03566176 registered in room 206000
2018/06/22 23:54:16 Removing client 82663979 from room 20000 due to timeout
2018/06/22 23:58:21 Deregistered client 03566176 from room 206000
2018/06/22 23:58:31 Removing client 03566176 from room 206000 due to timeout
2018/06/22 23:58:31 Removed client 03566176 from room 206000
2018/06/22 23:58:31 Removed room 206000
And this is what I see when I deploy the app (gcloud app deploy) and trying to join a room with a second device:

Do you see where the problem is?

Problem solved, I didn't open my ports on the google cloud plateform


ngOptions removeUnknownOption() doesn't work right in mobile browsers

I'm submitting
This may be an issue caused by a new browser version
Current behavior:
When using Select as in the ngOptions expression for an array of objects (a collection), the first empty option is not removed after the first selection of an option occurs.  Subsequent selections also do not remove the empty option.  This lingering empty option causes the indexing of the options to be off by 1.  For example, if the user selects option 3, then option 4 is selected in the model and option 4 is displayed in the select element.  If the user selects the empty option, then the first object in the collection is selected (instead of the lingering empty option).  If the user selects the last option, then no option is selected (presumably because the indexing error tries to select an object after the last element in the array).
As a workaround I edited the anguar.js file in version 1.8.2 to comment out line 35304 which removes the unkownOption.
  self.removeUnknownOption = function() {
    if (self.unknownOption.parent()) self.unknownOption.remove();  // I commented out this line
  };

Removing this line keeps the unknown option in the options list, and this precludes the indexing problem.  Of course, it also causes the unknown option to never be removed.
Expected / new behavior:
I expect the unknown option to be removed after the user selects an option for the first time.  Until a few months ago this worked.
Minimal reproduction of the problem with instructions:
Unfortunately this may be difficult to reproduce, because it only occurs on SOME mobile devices.  My iPhone 12 uses the same version of Chrome and does not have this behavior.  But my iPad Pro on the same version of Chrome does have the behavior.  The behavior occurs in our production app on many iPads in the field using Safari and Mobile Chrome.  Users report this did not always occur until several months ago.
This never seems to occur on the Windows Chrome browser of any version tested.  Only mobile.
AngularJS version: 1.8.2
Browser: Mobile Chrome 96.0.4664.116 and Safari on iOS v15.1
Anything else:
I suppose it is possible the browser is running out of memory, and maybe the lack of resources on the mobile device could cause the unkownOption to never be removed.

I realize AngularJS v1.8.2 is ending support soon, but we submitted this request a few days before the EOL deadline.  :)  Seriously though, any advice on hos to handle this would be great as we will not be able to upgrade to Angular for a little while.

Thx for reporting this, @jeremyml. As you said, AngularJS is End-Of-Life now, so no more fixes will be published.
Unfortunately, I don't what might be causing the behavior you described (as I don't have access to an iPad).
I am going to close the issue, but if you find the root cause or a work-around, feel free to post it below to help others that might also run into the same problem.


SSR support example and docs

Closes #21

Codecov Report

Merging #23 into master will increase coverage by 2.45%.
The diff coverage is 96.42%.

@@            Coverage Diff             @@
##           master      #23      +/-   ##
==========================================
+ Coverage   96.59%   99.04%   +2.45%
==========================================
  Files           7        9       +2
  Lines          88      105      +17
  Branches        7       10       +3
==========================================
+ Hits           85      104      +19
+ Misses          3        1       -2

Impacted Files
Coverage Δ

src/client/client.ts
100% <100%> (+13.63%)
:arrow_up:

src/hooks/useCachedResponse/useCachedResponse.ts
100% <100%> (ø)

src/context/clientContextProvider.tsx
100% <100%> (ø)
:arrow_up:

src/hooks/useQuery/useQuery.ts
100% <100%> (ø)
:arrow_up:

src/cache/cache.ts
100% <100%> (ø)
:arrow_up:

src/hooks/useClient/useClient.ts
75% <75%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update db2e8f7...0d4ed00. Read the comment docs.


[BUG] Anvil search is broken on 1.21.1

Describe the bug
The anvil search menu is broken on 1.21.1,
To Reproduce
Steps to reproduce the behavior:

Run /ah
Click on search button
Error is thrown

Expected behavior
Search menu should come up
Screenshots

Environment (please complete the following information):

OS - Ubuntu
Server Type - paper
Server Version - 1.21.1
Plugin Version - 2.5.0

Additional context
It looks like the dependency is outdated and it should be fixed on the newest version at least when looking at the commits on the anvil gui repo.

i will update this dependancy next update dont worry


Create continuous deploy pipeline for the staging environment

Why
As a developer for the project I want to be able to test changes I merged onto the staging branch as soon as possible so that I minimise the time required to fix potential bugs.
What

[ ] create CI server agnostic deploy script for deploying RP images to the AWS ECS alternative staging cluster (https://console.aws.amazon.com/ecs/home?region=us-east-1#/clusters/nrgi-alternative/services)
[ ] update the Shippable config for the project to deploy the staging branch using the new script and environment variables encrypted with Shippable
[ ] encrypt env variables required for staging deploy using Shippable

Notes
We're aiming for a bash script for deploy here even though Shippable advertises for having an integration that can deploy to ECS for 2 reasons:

the ECS integration Shippable provides does not seem to be able to define a volume with the source on the host which can be seen in the container - which is required for sharing the DB cert
we may switch to a different CI server if Shippable does not prove to be suitable

Moved this to the nrgi-dcos repo.


Project's Wiki

For the discussion related to the Wiki of the project.

The wiki has now a specific page for the resources we used or found interesting.


Meet and Greet!

Hi @AguaClara/aide
So nice to meet all of you the other day! I'm sorry about the technical difficulties. Luckily, we have alternative ways to contact one another (such as through GitHub issues) that aren't quite as bandwidth demanding 😄 .
On the right, you'll notice issues can be in projects, have assignees, labels, and milestones. It's most important to assign yourself issues as you work. This is how everyone on the team can keep track of what you're working on and provide suggestions. Issues will be the main way of contacting me (or anyone else, for that matter) to get help. To reach out to me, just use a mention by typing an "at sign" followed by a username @avtrigg , for instance.
In an effort to get everyone used to using GitHub, let's try introducing ourselves through this GitHub issue! Try to use as many GitHub Flavored Markdown (GFM) things as possible using that linked cheatsheet. For fun, let's each aim to each use at least 5 GFM things (links, italics, tables, code, etc) in our answers.
Also, be sure to include a list of your coding expertise - such as particular languages you like to code in and/or particular types of work you prefer (UI/UX or databases or server-side or front-end logic, security, algorithms).
I look forward to learning more about y'all!

Introduction
I'm Ethan! I will be providing technical assistance and advising to the AIDE team. I worked a little bit over the summer to develop an application architecture for AIDE that we'll be following (more or less). Below are some fun facts that attempt to use as much GFM as possible.
Coding Expertise
I like to dabble in any language I need to adequately architect software systems. I like to create a coding environment, using development tools and good coding practices, to incentivize high-quality code. I'm a big fan of Git and GitHub.

Places
I currently live in Honduras. Specifically, I live here.
I grew up in a suburb of Boston in MA.
Quote
A good quote by Ernst F. Schumacher:

Any intelligent fool can make things bigger, more complex, and more violent. It takes a touch of
genius — and a lot of courage to move in the opposite direction.

Fruits
When you come to Honduras on the AguaClara trip this winter, you might find the following flavor guide useful for some fruits I haven't seen in the US:

Fruit Name
Flavor
Type of Plant

Maracuyá
Tart with a fantastically fruity smell about the size of an apple
Tree

Lichi
Like a large sweet grape with a hairy cover
Vine

Tamarindo
Astringent gooey seeds in a bean pod
Tree

Code Snippet
Lastly, here's a short snippet of code I came across recently that counts source lines of code. Use it to count the AIDE lines of python code - or whatever type you like, just edit it slightly!
## prints recursive count of lines of python source code from current directory
## includes an ignore_list. Also prints total sloc

import os
cur_path = os.getcwd()
ignore_set = set(["count_sourcelines.py"])

loclist = []

for dir, _, files in os.walk(cur_path):
    for file in files:
        if file.endswith(".js") and file not in ignore_set:
            totalpath = os.path.join(dir, file)
            loclist.append( ( len(open(totalpath, "r").read().splitlines()),
                               totalpath.split(cur_path)[1]) )

for linenumbercount, filename in loclist:
    print "%05d lines in %s" % (linenumbercount, filename)

print "\nTotal: %s lines (%s)" %(sum([x[0] for x in loclist]), cur_path)

Can't wait to work on AIDE with y'all!

Let's try this again next year!


Notebooks: Font for TEXT (not grid) Results

In the Notebooks, when the results of a query are text (not grid), the font is proportional and can't be changed.
There should be a configurable setting to let users define the font they would like for all the query input and outputs including the Notebook results when the output is text and not a grid.

Love this request. We can totally do this.


Add logging.

Add some logging.

This pull request introduces 1 alert when merging f00da4b79559cbb9f3d414fd44027b056556f8b5 into 1413cc0e23260c0aba349271b52b81749473f1ee - view on lgtm.com
new alerts:

1 for Dereferenced variable is always null

Comment posted by lgtm.com


SECURITY ISSUE template instance user mail is replaced by password

Bug Report
System Information

Operating System (Including distribution name and version number)
Debian GNU/Linux 12 on targets & Debian GNU/Linux 11 on Controller
AMP version and build date (Always use the version number, 'latest' is not valid!)
<IP_ADDRESS> - 20230830.1
Which AMP release stream you're using (Mainline, Nightly or FastTrack)
Mainline

I confirm:

[x] that I have searched for an existing bug report for this issue.
[x] that I am using the latest available version of AMP.
[x] that my operating system is up-to-date.

Symptoms

What are you trying to do?
We have tried to set an email address on an automatically created user on an instance template.
What are you expecting to happen?
I expect that the user gets the email address set which is given or that the email address remains empty if none is given.
What is actually happening? ('Nothing' is not an acceptable answer!)
No matter if you enter an email address or not the given password will be written into the "email field". This is a breach of privacy.

Reproduction

After installing the game panel, we activated it with our Enterprise license key. This Enterprise version allows us to create "instance templates" which make it easier to install a game server. There you can set among other things that a user should be created which gets the rights to the game server. If you want to create this user then the password will be entered as the email address of the user and that always no matter if you leave the email address field empty or not. It is always the password

Fixed, this was caused by parameters being in the wrong order when creating a new user as opposed to using an existing user.


Support different storage such as Redis, memcached

I'd like to check if you like the idea of supporting different storage. If so, I'll create a few PRs for this.

not adding new features at this time.


Configura AZ Pipeline

Configura canalizacion de compilacion para aplicaicon y realease

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: martincal61 sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.


Updated quiver dependency version as incompatible with latest flutter

Hi, I was getting an incompatible version constraint error when trying to get this package, so have updated the quiver version to match flutter ^0.28.0:
Incompatible version constraints on quiver:

flutter_test 0.0.0 depends on version 0.28.0
orderable_stack 0.1.2 depends on version ^0.25.0
pub get failed (1)

Thanks

@marchampson thanks :)


Prevent long email addresses + names from getting hidden

There's a bunch of space on the right of this for the overlap with the
overlaying content. This shrinks the name and email to fit within that
area.
It also tweaks the bottom constraint for the email address so it
provides more even vertical spacing on extra long email addresses.

@mikekatz please review

:+1:


Added localtunnel alternative

Fixes connection limits due to ngrok  #23

Localtunnel should be installed as a dependency but otherwise I can confirm this works.


[App Request] How would you feel about an org-mode app?

Advantages of todo.txt:

Simplicity by design. One line.
Text-file based (editable, future-proof, etc).

Disadvantages of todo.txt:

Not everything can be reasonably done in one line.

Current apps for todo.txt:

Desktop: good number, although maintenance and features vary.
Mobile: reasonable number.

Advantages of org-mode:

Supports an arbitrary number of lines per node/task.
Text-file based.
Can be as simple as todo.txt or deeply customized and extended.

Disadvantages of org-mode:

Not easily accessible outside Emcas (see below), which isn't for everyone and has a steep learning curve to say the least.

Current apps for org-mode:

Desktop: Barely any apps outside the Emacs ecosystem. I looked. For a LONG time.

Extensions for VS Code, Atom, Sublime, etc:

Potential is limited by the API of the parent app.
Text-based support. No bars, panels, buttons, good visual manipulation, etc.

Mobile: Only a couple of apps. For example Orgzly.

If you look at the discussions of enthusiasts online, I think there is a real need for a good, accessible GUI to org-mode:

Get started with Org mode without Emacs
https://www.reddit.com/r/orgmode/comments/lcizik/does_orgmode_outside_of_emacs_make_sense/
And see the discussions on multiple Hacker News posts. It gives a good idea about how the org-mode situation is seen by many technical people. 1, 2, 3, 4, 5.

My suggestion
Even with the better situation of todo.txt, Sleek seems to have struck a chord with users, especially considering that it is not a year-old in public yet. Many reports (of bugs) and specific requests suggest active usage rather than just "try this app and wait for maturity". It satisfies a need that wasn't really met before. I have been using todo.txt for years now and struggling to go beyond simple usage until Sleek.
Org-mode needs an app that enables that usage where the open, text-based format is the robust foundation instead of the whole experience. You get ease of use AND advantages of text and an open standard.
There are JS parsers and web apps for org-mode, but not a desktop app. The situation is ripe for a good one. I think an app that covers even the very basics of org-mode visually will be received with welcome

TLDR

todo.txt is good. It has apps already. But people like Sleek. Good GUI/UX.
org-mode is good. It has no apps.
You understand the benefit of text formats.
You are good with JavaScript. JavaScript base for org-mode exists.
Basic org-mode app would be good. You make Slook. People like Slook. You help people! 😃

To be clear, I am NOT necessarily talking about a "pretty" UI like this. That would be a bonus but it can be achieved with many markup languages and with apps/extension like those for VS Code.
I meant grouping and managing of bits of information/todos (tagging/filtering/searching, etc). Like what todo.txt and Sleek provide, but with the power of org-mode (multi-line notes, formatting)!
If you see Orgzly, you would understand how it can be used for anything!
CintaNotes is the closed-source equivalent of the organizational aspect of org-mode.

@9z0 , I think this is a great idea. Last year I gave org-mode a try but didn't stick to it, so I don't have a lot of experience with it.
If there are JS parsers already, the process of getting org-mode text files displayed in sleek or a fork of sleek, should not be an issue. If sleek would be able to identify a file as being a todo.txt or org mode file, it could use different parsers to continue. So one app would be able to handle both.
The challenge in general would be to figure out what org-mode adds on the feature side and what would need to be done to make sleek handle those org-mode features properly.
I will take a closer look at org mode soon and try to figure out, what would need to be done.

Glad you liked it.
You can look into these projects (under various licenses):
https://github.com/DanielDe/org-web
https://github.com/200ok-ch/organice
https://github.com/daitangio/org-mode-parser
https://github.com/borablanca/orgmodeweb
https://github.com/fusupo/org-parse
https://github.com/orgapp/orgajs (TypeScript)
https://github.com/rasendubi/uniorg (TypeScript)
https://github.com/k2052/org-to-markdown
https://github.com/mooz/org-js (A bit old)
https://github.com/Patrickskiba/weborg
https://github.com/keenan691/org-mode-connection
https://github.com/jsorg/jsorg (old)
There are also these VS Code extensions:
https://github.com/vscode-org-mode/vscode-org-mode
https://github.com/aboukirev/vscode-orgmode
https://github.com/robaboyd/vs-org
https://github.com/crz33/vscode-markdown-agenda
I think i

Glad you liked it.
You can look into these projects (under various licenses):
https://github.com/DanielDe/org-web
https://github.com/200ok-ch/organice
https://github.com/daitangio/org-mode-parser
https://github.com/borablanca/orgmodeweb
https://github.com/fusupo/org-parse
https://github.com/maxminoS/markup-parser (TypeScript)
https://github.com/orgapp/orgajs (TypeScript)
https://github.com/rasendubi/uniorg (TypeScript)
https://github.com/k2052/org-to-markdown
https://github.com/mooz/org-js (A bit old)
https://github.com/Patrickskiba/weborg
https://github.com/keenan691/org-mode-connection
https://github.com/jsorg/jsorg (old)
https://github.com/jsns/gorg-mode (TypeScript)
There are also these VS Code extensions:
https://github.com/vscode-org-mode/vscode-org-mode
https://github.com/aboukirev/vscode-orgmode
https://github.com/robaboyd/vs-org
https://github.com/crz33/vscode-markdown-agenda

in sleek or a fork of sleek

I think it may be better not to use the same app for both. The formats (todo.txt and org-mode) are different enough that a single UI model (or even options) may not work for both.
Using as much as possible of the code base for both should help with maintenance, of course.
Good luck!

Before returning to todo.txt a few weeks ago, I had been using orgzly and org-mode for about a year. And before that I was using todo.txt with topydo, QTodoTxt, and simpletask-android for a couple of years.  So I can compare the two systems with a fair amount of context.
The TL;DR is that I don't think sleek should try to handle both formats.  I certainly didn't use all of org-mode, nor do I claim to understand every option that it offers.  It is no doubt very useful for some folks.  But many of us just want to have the minimum amount of complexity while keeping track of all of our tasks in a simple, reliable way.   Sleek and todo.txt seem to do this well. Here are some more observations.

Orgzly is a very nice Android app.  Attractive and works well, although I found the synchronization behavior was a little tricky.
I didn't find a good desktop app, partly because true org-mode fans will usually use emacs where org-mode was created.  So there is a open niche for a nice GUI app, but it probably wouldn't get those emacs users to leave their favorite tool.
one of the big todo.txt advantages is that line-oriented unix/linux shell tools can easily process tasks one per line, and you lose that simplicity with org-mode.
I found that I didn't need most of the extra complexity that org-mode format offers.  While it is nice to have multi-line task descriptions or notes, you still need to keep a short one-line heading description that you can recognize in a task list.  I'm willing to work around the single-line limitation of todo.txt (and it can be a LONG line!) in order to keep the simplicity.
org-mode is intended to be a general outliner, note-taker, and Markdown alternative, as well as a task manager.  When I took notes in orgzly, I would eventually wish that I could get them out of there and into my note-taking system, which is a wiki-like web app.
Task states like "TODO" and "DONE" are just defaults in org-mode which can be customized to anything the user wants.  Org-mode fans will build quite elaborate workflows around this with many more than two states for a task.
There are multiple dates that can be associated with an item: SCHEDULED, DEADLINE, COMPLETED, etc. Each date can have times, time ranges, etc.  One use of org-mode is to track the time you spent working on each task for time-accounting and billing.
One thing I did like about org-mode was the ability to have tasks with sub-tasks (a general tree of tasks).  But in practice I found that I hardly ever used this feature.  In todo.txt, the +project tag is a limited substitute for this, but with the advantage that a subtask can have multiple "parent" projects because they are just tags.  Org-mode has tags too, and I actually used those more than I used subtasks.

Nice insights. I generally agree.

I didn't find a good desktop app, partly because true org-mode fans will usually use emacs where org-mode was created. So there is a open niche for a nice GUI app, but it probably wouldn't get those emacs users to leave their favorite tool.

This is the point. I don't see them are a significant target audience. It is for us non-Emacs plebs!

I didn't need most of the extra complexity that org-mode format offers.

I personally want the organization ability more than anything. It represents for me a text-based relational database!
Because it inherently supports nesting/outlining, tags (with inheritance), multilines, custom attributes, date/time, markup it has the potential to be whatever the user wants and as simple/complex as they need.
Searchable, readable universal text files that can organize all. In a grep-accessible place. That can have one or multiple graphical interfaces to tackle any/multiple of these features. Accessible on all your devices. In a future-proof way. Without depending on third-party services.
I have been looking for years and through many many solutions. It doesn't have to be org-mode, but I mean what are the alternatives with that description?

@9z0  I think we are on the same page.  I looked for an org-mode desktop app, didn't find one, started to write one, and then sleek lured me back to todo.txt.  I do like outliners (check out Vimflowy for instance).   Org-mode seemed attractive, but it is hard to implement fully, and I'm not sure I need all of my todos and notes combined into one document.  So I have instead built separate solutions for note-taking and just want something clean and simple for keeping track of tasks.  It sounds like you want the two more integrated together, maybe something like TaskPaper – Plain text to-do lists for Mac.  This type of thing does appeal to me, but I haven't ever found just the right thing. Definitely want a plain-text system, or at least one where I control my own data and don't need the cloud.
Also there is the platform issue.  For me, it needs to be an open-source linux app, preferably with a compatible open-source Android app that I can keep in sync.  With sleek and simpletask-android, I have that combination for todo.txt.

@9z0 I think we are on the same page.

👍

I do like outliners (check out Vimflowy for instance).

TaskPaper – Plain text to-do lists for Mac

Thanks. Yeah, although they both seem limited to one line notes.

and I'm not sure I need all of my todos and notes combined into one document.

It sounds like you want the two more integrated together

Not necessarily! I don't like cramming everything together either. You can have a multiple GUI apps or just one or two, but at the end, your todo file(s) and note file(s) are accessible in text files. You get the benefits of visual organization and the benefits of text formats.
It is about applying the benefit of todo.txt + Sleek to actual notes rather than one-line todos.
Sometimes notes about a topic can have multiple aspects and so if you want organization using plain text, notes tend to scatter around in multiple places, and you end up with duplicates/old information, trying to remember all the time (where did I put it? What words did I use so I can search? etc) . One the other hand, a large text file in it self is difficult to manage and keep clear.
I always find myself trying to organize the mess going into circles of:
splitting things into folders and files for clarity > "where should I put this?" > "where did I put this?" > search notes folders using different terms > "I need to organize things better in one place" > A relational database > difficult and slow to search and filter without a custom-made UI > don't know how to make one > "Nothing developed by others fits my needs" > "This one is very good and I can adapt." > "I doesn't have a mobile app!" > Try org-mode or something similar > Difficult to filter and manage in plain text and Orgzly doesn't have that important feature > Try a non-standard mobile app for a portion of notes/todos important to have on-the-go. > The app doesn't have a desktop equivalent!
And now I have my notes in 6 different places and 8 different formats!

I always find myself trying to organize the mess going into circles of:

splitting things into folders and files for clarity > "where should I put this?" > "where did I put this?" > search notes folders using different terms > "I need to organize things better in one place" > A relational database > difficult and slow to search and filter without a custom-made UI > don't know how to make one > "Nothing developed by others fits my needs" > "This one is very good and I can adapt." > "I doesn't have a mobile app!" > Try org-mode or something similar > Difficult to filter and manage in plain text and Orgzly doesn't have that important feature > Try a non-standard mobile app for a portion of notes/todos important to have on-the-go. > The app doesn't have a desktop equivalent!

And now I have my notes in 6 different places and 8 different formats!

Yes, you have described my history perfectly.  I have used most every todo system out there, and many many note-taking systems, and never could find one to commit to for more than a year or two.  The cloud solutions always failed me at some point due to unexpected code changes and instability, and of course, there is the privacy issue as well.  So I have committed to self-hosted and local solutions, synchronized with syncthing, hosted on my own network.  But still there are too many choices and not enough solid functionality that is supported on all of my platforms.  Currently I use a wiki-like system for notes in Markdown, which I edit in my editor and then generate a locally hosted searchable static site with hugo or similar static site generator.  I'm hoping that Markdown will prove to be a format that I can continue to use for a very long time, because of the diverse software options, like zettlr, stackedit, dillinger, bear on the mac, etc etc.  I don't actually use any of these at the moment, but having the option to move my data to them without format changes is very attractive.  Editing notes in my code editor is a good thing because of all the muscle memory I have for the shortcuts and features of that editor (vscodium).
So bringing this back to sleek, if someone were to write a good desktop GUI app that supported the same org-mode subset as orgzly does on Android, I would be interested.  But we seem to agree that org-mode would be difficult to support alongside todo.txt in sleek itself.


Make it clear that IAM does not support regex, convert: .* => *

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

I revised the text. Thanks for taking the time to submit a pull request.

So I don't love that you added the * back into the statement. The new update looks like this:

then allow the action only if the key value begins with "t1.*", "t2.*", or "m3.*"

But the key never begins with t1.*. It begins with t1. so saying "if it begins with t1.*" is the confusing thing that I wanted to have fixed. You explicitly added that part back.
Collectively these changes are better than the original state, but I wish that * wasn't adedd in there. Alternatively, changing the whole sentence would have been better to say:

then allow the action only if the key matches "t1.*", "t2.*", or "m3.*" ... something something about how t1. followed by anything.

But honestly whatever, still better than it was. Thanks for getting this in.

Hi @wparad, I appreciate the explanation. I removed the wildcard * from the statement for added clarity.
Thanks for the feedback.


DependencyCallStacks loading error

I'm getting an error when adding DependencyCallStacks to my ApplicationInsights.config. It looks like AI wont load the DLL, but the Fusion assembly binding log indicates that the Microsoft.AI.DependencyCallstacks.dll was loaded successfully.
The Windows event log contains this stack trace:
Service cannot be started. System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.InvalidOperationException: Type 'Microsoft.ApplicationInsights.DependencyCallstacksTelemetryInitializer, Microsoft.AI.DependencyCallstacks' could not be loaded.
   at Microsoft.ApplicationInsights.Extensibility.Implementation.TelemetryConfigurationFactory.CreateInstance(Type interfaceType, String typeName)
   at Microsoft.ApplicationInsights.Extensibility.Implementation.TelemetryConfigurationFactory.LoadInstance(XElement definition, Type expectedType, Object instance)
   at Microsoft.ApplicationInsights.Extensibility.Implementation.TelemetryConfigurationFactory.LoadInstances[T](XElement definition, ICollection`1 instances)
   --- End of inner exception stack trace ---
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor)
   at System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] a...

How can I trace what's going on? This doesn't happen when running the code in VS, only on the server running the software I want to track.

My bad. There is an assembly binding error. It looks to me like the AI SDK <IP_ADDRESS>0 doesn't play well with Microsoft.AI.DependencyCallstacks version <IP_ADDRESS>54 which is the latest available at MyGet.
LOG: This bind starts in default load context.
LOG: Using application configuration file: C:\Program Files\BPCloud\MSS\StatusServerService.exe.Config
LOG: Using host configuration file:
LOG: Using machine configuration file from C:\Windows\Microsoft.NET\Framework64\v4.0.30319\config\machine.config.
LOG: Post-policy reference: Microsoft.ApplicationInsights, Version=<IP_ADDRESS>39, Culture=neutral, PublicKeyToken=31bf3856ad364e35
LOG: GAC Lookup was unsuccessful.
LOG: Attempting download of new URL file:///C:/Program Files/BPCloud/MSS/Microsoft.ApplicationInsights.DLL.
LOG: Assembly download was successful. Attempting setup of file: C:\Program Files\BPCloud\MSS\Microsoft.ApplicationInsights.dll
LOG: Entering run-from-source setup phase.
LOG: Assembly Name is: Microsoft.ApplicationInsights, Version=<IP_ADDRESS>0, Culture=neutral, PublicKeyToken=31bf3856ad364e35
WRN: Comparing the assembly name resulted in the mismatch: Revision Number
ERR: The assembly reference did not match the assembly definition found.
ERR: Run-from-source setup phase failed with hr = 0x80131040.
ERR: Failed to complete setup of assembly (hr = 0x80131040). Probing terminated.

I'll build the Telemetry Initializer from source and use that instead.


Public repo not found

Are you experiencing an issue with...

[x] shields.io
[ ] My own instance
[ ] badge-maker NPM package

:beetle: Description
Public Github repository that had its visibility previously set to private is not being found by shields. After changing the name of the repository the repository is found under its new name by shields, but changing it back to its original name makes shields unable to find it again.
:link: Link to the badge
https://img.shields.io/github/license/yooksi/pz-zmod
:bulb: Possible Solution
I suspect this might be a caching issue that will probably work itself out in time.

The issue was resolved quickly after I opened this issue, it was a caching issue.


Apparently required to set ANDROID_HOME

Error executing simctl: 117 Usage: simctl list [devices|devicetypes|runtimes] Android SDK not found. The ANDROID_HOME variable must be set. Failure building APK. Unable to find components. Starting lib/main.dart on Nexus 9... "build/app.apk" does not exist.

On master branch, we're seeing the above error. Didn't find instructions in our Getting Started that sets ANDROID_HOME.
Did we miss a step? (if so, we'll need to document it)
Did we hit a known bug?

cc @mrasz
cc @devoncarew do you know why we're seeing the above?

cc @qchong as this might require an update to "Getting Started" ?

We should first try and locate adb on the path, and then fall back to something like ANDROID_HOME. In the passed I've used the which package (https://pub.dartlang.org/packages/which).

Are we responsible for saying "The ANDROID_HOME variable must be set." ?
If so, let's try: "Please visit https://flutter.io/getting-started/ for instructions on how to install the Android SDK." instead.
Talking about ANDROID_HOME here in this error message is confusing, since we've never asked the user to set that.
Thoughts?

@sethladd I've just run into this and found it confusing. I kinda expected to be able to follow just the getting started instructions from a vanilla mac and have Flutter running.
I don't mind installing the full Android SDK if I need to, or adding an extra variable after doing the adb install.

I kinda expected to be able to follow just the getting started instructions from a vanilla mac and have Flutter running.

Agreed. Following the Getting Started guide should get you to a working "hello world"

The "flutter doctor" command could help here. See https://github.com/flutter/flutter/issues/1466

See https://github.com/flutter/flutter/issues/1692

Is there a workaround while this check gets implemented?

Updated the Getting Started per Seth's comment. (pr: https://github.com/flutter/website/pull/160)

I suspect this is a new requirement as part of the dynamic services work?  @mpcomplete?  Previously we would always use a prefab apk from the cloud for flutter start, but such an apk only works in the zero-optional-services case which is likely to decrease in frequency soon.

I think this might be caused by my change to 'flutter start' to make it call 'flutter apk'. That should only happen if the app contains an apk/AndroidManifest.xml.
I think we should change the Getting Started guide. We're going to eventually require the android SDK in all cases.

I really would rather we not require any environment variables be set to develop in flutter. Is there no way for us to autodetect the path in the common case? What do we need ANDROID_HOME for?

To expand on why: Ideally, I'd like us to get to a state where you just download the Flutter SDK, and you're done. You can start coding right away. Everything just works. This won't be feasible if we require that you go and hack at your .bashprofile or AUTOEXEC.BAT or whatever.

We no longer need ANDROID_HOME.  We autodetect the Android SDK if its in the normal location.


Diagnostics on type

This PR adds getting diagnostics on type capability. Currently, the compiler doesn't support flawless workflow for this feature, so it is implemented with some drawbacks.
What we lack from purs-ide:

In an ideal situation, purs-ide should allow getting diagnostics without codegen and without touching externs/code-db.
In a more ideal situation it should also allow do it without involving the file system (copying sources to temporary location).

This PR fixes it in an experimental way (and it works ok).
Current implementation:
Settings:

diagnosticsOnType: true - to enable type check while typing
diagnosticsOnOpen: true - to get type check for just opened files (by default library files excluded, not to polute full build)

More advanced:

diagnosticsCodegen: true - produces codegen on each check (allows to check foreign, because without JS codegen target enabled it purs-ide won't check it)
revertExternsAndCacheDb: true - advanced mode which will enable some hacky tricks for polluting full build less (makes no sense if codegen enabled).

Some details of internal changes:

Build related functions moved to IdePurescript.Build
Rebuild state is introduced to exclude simultaneously not allowed build routines.
removed buildOpenedFiles option (relaced with disagnosticsOnOpen
In Build in diagnostic info related functions Foregin.Object replaced with Map

Questions:

Should diagnosticsCodegen be enabled by default? (this is bad when file watches are involved).
Should fastRebuild=true be effective with fullBuildOnSave=true? (currently, it is)

On possible implementation strategies of the feature:

Do not touch externs/cache-db.
We send purs source by copying it into temp directory. In this case purs ide generate updated externs and places in cache-db entries with dummy 1800 dates, wich will cause a rebuild be performed for updated modules during next full rebuild.

modules see online changes in other modules.

if the code was edited but not updated eventually it will still cause the rebuild
disagnostics on open (if enabled) will cause the rebuild
foreign modules/declarations are not checked

To reduce the full build polution, we exlude library files from diagnostics on open.

foreign modules/declarations are are checked when diagnosticsCodegen enabled.

This is a default strategy enabled with: diagnosticsOnType and diagnosticsOnOpen settings.

Do save/revert externs/cache-db
Before sending files to purs ide we save the state of cache-db and externs of the module, after we get diagnostics we revert changes back. Purs ide, in this case, returns to the old state: as purs-ide checks cache-db file modified state (which will be changed externally by revert operation) it will reload all the modules (externs) from a disk.

modules DON'T see online (not saved) changes in other modules
additional hacky fs (save/revert) operations

full build is not polluted (if no eventual changes made)

Modification of this strategy: to revert externs only for documents with version 1 (and if the content is the same as on disk), this allows to avoid polluting of full build on file open and allow:

modules see online changes of other modules
diagnostics of opened files doesn't pollute full build

polluting rebuild (in case of any unintentional update of file content)

Use fake module names. Do not touch externs/cache-db (not implemented).
We replace the module name in the source with some fake one, and send it to purs ide, so output and cache-db get populated with those fake entries, but it should not have an effect on real modules.

modules DON'T see online changes in other modules
modules (their externs) are built into a fake directory in the output, so they pollute the output directory and cache db with those fake entries.

full build should not be polluted
foreign modules/declarations will be checked

@nwolverson I've updated the PR to the current verson and would like it to be merged eventually. So it would be nice if you review and check it. This PR also involves some refactoring to state management api (setter/getter) but nothing significant.
I use diagnostics on type feature for a long time though though with update compiler version (without involving FS).
Maybe @i-am-the-slime could check this too.

@wclr @nwolverson What's the status on this? How can we/do we still want to move this forward?

@i-am-the-slime can you try this branch for yourself?

I'm testing this at the moment.
For better or worse the project I'm in has some files that take a significant time to rebuild, I'm not sure what the best way to make that better - perhaps just visibility of what's happening, a way to configure excluded files (simpler than the heuristic version included in the other timeline), I don't know. Not a blocker, the feature can be turned off.
The compiler PR - that needs to evolve but that can happen after this is merged.
Revert externs/cachedb hack - I need to think about that. But it looks easy to remove so don't care too much.

Revert externs/cachedb hack - I need to think about that.

I don't really like those hacks, so I implemented to see how this works out even with thouse hacks there some flaws and speed is definetly not the best. But I don't case about them, because I use updated compiler.
I think we really should move the sugessted compiler PR forward after some additional considerations, it should not break anthing, but will allow this feature be much more robust.
At the same time there are still some issues in how purs ide works that should be fixed to make rebuild process smoother (esp.  it refers to the interaction of ide process with full build results).

@nwolverson

For better or worse the project I'm in has some files that take a significant time to rebuild

I've recently been working with big files 2000-4000 lines and they really take significant time to be rebuilt by purs ide. Also, this refers to modules that depend on large files (this was a surprise for me). I have files ~100 line that use a module of 4000 lines (which also has a lot of exports ~500) and those files take long to be rebuilt too.
By long, I mean more than one sec: 1,5~4 sec. This visual feedback delay also happens only when the module is built without errors, when there are errors even on big files usually it is quick to bring the error feedback.
So this large delay is more about compler speed, not file system usage .
Are you talking about large files? What time do diagnostics take in your case?

We have one file that is taking about 15s to rebuild (and it's not huge, if we can make a standalone repro we'll raise an issue). And yeah even in that case, if there's a parse error the build is lightning fast, it's just when typechecking goes bad that problems arise.
So my experience just running on this branch for a while - I got used to it, normally don't notice it and grow to expect it. Couple of negatives:

Sometimes it kicks off when I start typing something, where I have the instinct to type just a few tokens and hit save to rebuild; the build taking a second or two means my normal experience of rebuilding is delayed, so it takes double the time to see feedback. Not sure if there's a debouncing model that makes this better

I got the feeling at first that the rebuild as I type blocks autosuggest, which is a big negative to me - suggestions are higher priority. If this is purs ide blocking on the rebuild request while getting a subsequent completion request, I guess that could be a hard problem.

Anyway not reasons not to proceed, only not to be default.

Go ahead and do them against master, I'm not ready to do a release anyway - need changes in vscode extension and finish updating build for 0.15.0/esmodules things.

Just got notified about the activity here. Sorry for not getting back earlier.
One bad bug I had in my version of this was related to #177 . I don't know if @wclr 's version is also affected by this.
So when I would be renaming a module it would produce lots of folders in output:
Module.Before
Module.Befor
Module.Befo
Module.Bef
...
Module.Afte
Module.After
The autocomplete would then lead to suggesting 20 different modules (19 of which wrong) for an import from that module.

@i-am-the-slime I observed the same thing with another LS, I think possibly Haskell?

So when I would be renaming a module it would produce lots of folders in output:

Right,  when updating the module name with FS involved it creates all those intermediate names in the output, which is not nice of course. Also, the compiler keeps each such (intermediate/stale) module name in cache-db.json and actually never(!) it got never removed from there, not even while on a full build.
I will write up my proposal in the corresponding issue.

@i-am-the-slime
I've updated compiler's PR, reflecting in the description the latest version of the actual changes.
@nwolverson maybe now you could review it more closely and say what you think about the proposed solution.

@i-am-the-slime
I've opened new PR https://github.com/purescript/purescript/pull/4362, reflecting in the description the latest version of actual changes.
@nwolverson maybe now you could review it more closely and say what you think about the proposed solution.

@wclr Very nice. I guess I also have to build my own version of the language server to make use of this, then?

@i-am-the-slime certanly, from this branch (or the current master as it was merged), and set purescript settings diagnosticsOnType, diagnosticsOnOpen to true.


Files on item pages don't use pagination

If you have more than 20 files in the ORIGINAL bundle of an item, only the first 20 will be shown. There is no pagination support as of yet.
Perhaps full blown pagination is too much for the simple item page, a simple "more" link seems more appropriate

I'd add pagination with "more" links on the simple item page, and a complete pagination component on the full item page. This will take an estimated 12h


macOS smoketest failure: unable to open file

https://dev.azure.com/monacotools/Monaco/_build/results?buildId=123076&view=logs&j=3792f238-f35e-5f82-0dbc-272432d9a0fb&t=b4255493-5d6b-5e5c-80d7-ebab0307fbf4
  1) VSCode Smoke Tests (Web)
       VSCode Smoke Tests (Web)
         Editor
           shows correct quick outline:
     Error: Timeout: get elements '.quick-input-widget .quick-input-list .monaco-list-row .label-name' after 20 seconds.
      at poll (/Users/runner/work/1/s/test/automation/out/code.js:176:19)
      at async Code.waitForElements (/Users/runner/work/1/s/test/automation/out/code.js:253:16)
      at async QuickInput.waitForQuickInputElements (/Users/runner/work/1/s/test/automation/out/quickinput.js:25:9)
      at async QuickAccess.openFile (/Users/runner/work/1/s/test/automation/out/quickaccess.js:42:9)

@joaomoreno any hints what we would do here to strengthen the test?
Some thoughts:

can we attach a screenshot of the running app when this happens as build artefact to help diagnose what is going on in the test (didn't we explore this once?)
should we print the current active element (I think we do this in the native smoke tests but not browser based ones?), this could hint if maybe a dialog has focus at that time

Oh wait, this is not the quick outline test failing, this is actually failing to open a file:
at async QuickAccess.openFile (/Users/runner/work/1/s/test/automation/out/quickaccess.js:42:9)

This is a very basic operation that needs to succeed for smoke tests to function...

No, the failure is in Quick Open to show the results:
      at async QuickInput.waitForQuickInputElements (/Users/runner/work/1/s/test/automation/out/quickinput.js:25:9)

The smoke test opens quick input, types some text and waits for results. They never come. My suspicion is always the same: some other UI piece steals focus away from Quick Open and thus Quick Open hides.
My suggestion for the entire smoke tests: we disable hide on blur for Quick Open, for the entire duration of the smoke test.

True 👍
Oh no, this is not good, the getting started page is back:

It is disabled in native tests but not web. This could be an explanation...

I love it. I actually have a branch now in which I run the smoke tests on Windows and I keep hitting some issue. That might be it as well!

This is just web though, in desktop we have a CLI arg to disable welcome --skip-welcome

Yeah they only fail on web 😆


Won't install any mods

debug.log
Once I've selected either of the two BG3 exes (the standard or the DX11 one), I can add mods to the list but the buttons "Install selected mod" and "Toggle enabled" do nothing. In addition, clicking on Settings in Candor does nothing. If I remove a mod from the list and try to add it again, I get an error message saying " could not be loaded. Please try again." Closing and reopening fixes that. Mods I've tried to use are Tiefling Racials and Devilish Pack, both of which are on the BG3 Nexus. My PC is running Windows 10. The folders seem to be set up correctly since "Open " options in Candor do indeed open the corrent folder.

I have published a hotfix on the Nexus to fix this, If in future an issue like this arises, it also now won't silently error, but instead show a pop up about what went wrong. Hope this helps!


Cannot test fastlin due to error "get_barcodes.rs:45:30"

Hi! I would love to test fastlin, but I am getting the following error with standard MTBC barcodes.
fastlin -d . -b barcodes-fastlin/MTBC_barcodes.tsv
fastlin thread 'main' panicked at 'byte index 88 is out of bounds of ATCGCCAGCAGTATCGAACGCAATATCCGTGAACTCGAGGGCGCGCTGAT', src/get_barcodes.rs:45:30
note: run with RUST_BACKTRACE=1 environment variable to display a backtrace . get barcodes and genome size`
Note: running conda version

Hi,
my apologies, I modified the length of kmers in the MTBC barcode file but forgot to update fastlin's code accordingly. It will work fine with version 0.2.0.
nb: not yet available in Conda as I'm trying to add fastlin to the Bioconda channel
best wishes,
Romain

It works now, thanks!!


Asset pre-compilation fails when using Devise and named scope with lambda in User model

Apologies if this is not in fact a problem with Devise, but  lib/devise.rb, lib/devise/mapping.rb and lib/devise/rails/routes.rb appear in the backtrace, so I thought it was worth letting you know.
The problem appeared when I added a named scope incorporating a lambda function to the User model that uses devise:
  joins(:skills).where("user_skills.offered = ? AND skills.id = ?", :true, skill_id)
}````

Now calling `rake assets:precompile` fails, both locally and on a development server, with the error `Could not find table 'users'`.

Full backtrace of the rake error here: https://gist.github.com/1276348

I have also asked about this on StackOverflow, as I can see that this may not be a bug in Devise, but I have tried a few workarounds elsewhere relating to other rake tasks failing in the same way, and a similar error relating to factory_girl, but none of them seem to work in this case.

http://stackoverflow.com/questions/7716900/rails-capistrano-deploy-error-fails-on-rake-assetprecompile-in-named-scope-wit

Can you please push to Github a small application that reproduces the error? Also, please include Devise and Rails versions in your reports and please ensure that you are running on latest Devise and Rails 3.1.

Apologies for the omissions José, I am relatively new to GitHub.
I have spent the past hour or so looking into this, and I have a few more pieces of information.
The application you wanted is here: https://github.com/ljfc/devise_error
Recreating the error should be just a matter of running rake assets:precompile
Indeed, the example will not even permit the initial rake db:migrate.
A few notes:
Adding the Heroku fix config.assets.initialize_on_precompile = false to config/application.rb, does not appear to help.
The problem was apparent in Devise 1.4.5, and upgrading to 1.4.8 has also not helped. The example app uses 1.4.8.
The example app uses Rails 3.1.1, and also displays the known Rack 1.3.4 warning already initialized constant WFKV_, which appears to be unrelated.
My original app is still using Rails 3.1.0 for the moment, but this error seems to occur with both versions.
I have included the same scope code in a similar small application using a simple has_secure_password approach rather than Devise, and it doesn't give the same errors, so it does seem to be something to do with the code Devise adds.
A final note:
I have fixed this in my main app by replacing the scope-with-lambda with a class method, which I believe is now best practice anyway. So I am not under pressure to fix this personally, although it would still be good to know if there is something obvious wrong.
Let me know if there is anything else I can provide, but please bear in mind I am on UK time, so will not respond until tomorrow now.

Ok, thank you very much for the app and the debugging you have done so far. Please keep me posted if you find anything else, I will take a look at your app hopefully later this week.

hi,
https://github.com/ljfc/devise_error/blob/master/app/models/user.rb#L3
my recommendation: don't use ":find_by_..." as name for a scope, because scopes always find something. :)
choose a meaningful name like ":by_something" or ":by_skill" (your question on stackoverflow).
if you rename your scope ":find_by_something" in ":by_something", then it works.
hope this helps.

Hi Matt,
Thanks for the tip. I guess I just thought it was functionally equivalent to the generated find_by_... methods, so it ought to be named the same. I suppose it's more similar to a find_by_all_... because it returns an empty array if nothing is found.
Could this actually be causing the error? The lambda function does work, it just causes certain rake tasks to bomb.
Leo

Sorry, I can't explain the exact technical cause of the error in your particular case.
This issue can be closed (not a devise issue).
Best, Matt

Okay, cheers Matt.


chore: dereference whole API on importing to avoid extra definitions

My solution to https://github.com/anymaniax/orval/issues/863  I'm not quite sure if this is a proper solution for the problem, but it works for me.
Basically, I'm dereferencing the whole API at the beginning so there are no extra schemas that could generate types.

We can add the deferencing but as an option in the input options of orval.