Implement the signal action

Signal for core plugins will be implemented separately. The website will also be updated as part of that work.
Supports #548.

I tested this by adding signal support to the mock_docker plugin. Here's what the output looks like for docs (note I used AWS plugin root's docs to represent a typical description):
wash . ❯ docs mock_docker/mock_containers_dir/chalice
This is the AWS plugin root. The AWS plugin reads the AWS_SHARED_CREDENTIALS_FILE
environment variable or $HOME/.aws/credentials and AWS_CONFIG_FILE environment
variable or $HOME/.aws/config to find profiles and configure the SDK. The profiles
it lists can be limited by adding

aws:
  profiles: [profile_1, profile_2]

to Wash’s config file.

The AWS plugin currently supports EC2 and S3. IAM roles are supported when configured
as described here. Note that currently region will also need to be specified with the
profile.

If using MFA, Wash will prompt for it on standard input. Credentials are valid for 1 hour.
They are cached under wash/aws-credentials in your user cache directory so they can be
re-used across server restarts. Wash may have to re-prompt for a new MFA token in response
to navigating the Wash environment to authorize a new session.

SUPPORTED SIGNALS
* stop
    Stop the container
* start
    Start the container

As for testing signal, signal for the mock_docker plugin's containers writes the passed-in signal to a file. For example,
wash . ❯ signal foo mock_docker/mock_containers_dir/chalice
puppetlabs.wash/bad-action-request: Bad request for signal action on /Users/enis.inan/Library/Caches/wash/mnt568356764/mock_docker/mock_containers_dir/chalice: invalid signal foo. Valid signals are start, stop
wash . ❯ signal start mock_docker/mock_containers_dir/chalice
wash . ❯ exit
Goodbye!
➜  wash git:(signal) ✗ cat container_signal
start

Ok, updated to support signal groups.


No logs when deis used with k8s

Hi all,
We have a setup with a stateless deis 1.12.2 using k8s schedular, for now  we used the in memory logger, but the problem is we do not have any logs with deis logs command:
# deis logs -a alfred

<< empty response

we do have log when we use kubectl though and on the instance level docker log works as well,
kubectl --namespace=alfred logs alfred-v2-cmd-33h04
Starting server... done.
time="2016-01-05T22:50:47Z" level=info msg="(377662431b14f424e58a14dde72fdd46) getImage" time=25.104065198s
time="2016-01-05T22:50:47Z" level=info msg="(377662431b14f424e58a14dde72fdd46) hips.Process" time=25.104211844s
time="2016-01-05T22:51:39Z" level=info msg="(377662431b14f424e58a14dde72fdd46) getImage" time=9.979237052s

core@ip-10-0-101-124 ~ $ docker  logs a1e8e540cecc
Starting server... done.
time="2016-01-05T22:50:47Z" level=info msg="(377662431b14f424e58a14dde72fdd46) getImage" time=25.104065198s
time="2016-01-05T22:50:47Z" level=info msg="(377662431b14f424e58a14dde72fdd46) hips.Process" time=25.104211844s
time="2016-01-05T22:51:39Z" level=info msg="(377662431b14f424e58a14dde72fdd46) getImage" time=9.979237052s

I know there is some work going on with logging on deis 2.0, but I was wondering what's the status of logging for 1.12.2 with k8s ?

This should be working.  Can you look at the values of the following two etcd keys and then check whether they seem to accurately point to the logger?

/deis/logs/host
/deis/logs/port

Generally speaking, logger publishes those and then the logspout on each node looks at those at startup to discover the logger.  In the past, people running stateless have sometimes overridden those values to bypass the logger and send logs straight from logspout to some external log archive.  (There's no longer any good reason to bypass the logger now that the in-memory log storage is available.  Logger can drain to an external archive instead if that's still desired.)
The salient point is... check that those etcd keys accurately point the logspout instances at where the logger is actually running.
One thing to note is issue #4293.  Because of that, logspout instances really need to be restarted if/when logger is rescheduled from one node to another.
Apart from all this, the only other possibility is that logspout isn't running or has "seized up" for some reason-- an issue we've seen in the past that we believe has been corrected.

Thanks for the answer, here for the etcd keys,
core@ip-10-0-101-124 ~ $ etcdctl get /deis/logs/host
<IP_ADDRESS>
core@ip-10-0-101-124 ~ $ etcdctl get /deis/logs/port
514

It seems that the host is correct: <IP_ADDRESS>
core@ip-10-0-101-124 ~ $ fleetctl list-units
UNIT					MACHINE				ACTIVE	SUB
deis-builder.service			27cf4e1e.../<IP_ADDRESS>	active	running
deis-controller.service			374e5749.../<IP_ADDRESS>		active	running
deis-kube-apiserver.service		27cf4e1e.../<IP_ADDRESS>	active	running
deis-kube-controller-manager.service	27cf4e1e.../<IP_ADDRESS>	active	running
deis-kube-kubelet.service		bea10648.../<IP_ADDRESS>	active	running
deis-kube-kubelet.service		de6ac0cb.../<IP_ADDRESS>	active	running
deis-kube-proxy.service			50416cee.../<IP_ADDRESS>	active	running
deis-kube-proxy.service			665c0c4b.../<IP_ADDRESS>	active	running
deis-kube-scheduler.service		27cf4e1e.../<IP_ADDRESS>	active	running
deis-logger.service			27cf4e1e.../<IP_ADDRESS>	active	running
deis-logspout.service			27cf4e1e.../<IP_ADDRESS>	active	running
deis-logspout.service			374e5749.../<IP_ADDRESS>		active	running
deis-logspout.service			50416cee.../<IP_ADDRESS>	active	running
deis-logspout.service			665c0c4b.../<IP_ADDRESS>	active	running
deis-logspout.service			bea10648.../<IP_ADDRESS>	active	running
deis-logspout.service			de6ac0cb.../<IP_ADDRESS>	active	running
deis-publisher.service			bea10648.../<IP_ADDRESS>	active	running
deis-publisher.service			de6ac0cb.../<IP_ADDRESS>	active	running
<EMAIL_ADDRESS>	374e5749.../<IP_ADDRESS>		active	running
<EMAIL_ADDRESS>	50416cee.../<IP_ADDRESS>	active	running
<EMAIL_ADDRESS>	665c0c4b.../<IP_ADDRESS>	active	running

The port is correct as well: 514
core@ip-10-0-140-126 ~ $ docker ps
CONTAINER ID        IMAGE                   COMMAND                CREATED             STATUS              PORTS                                                   NAMES
dcde37b27106        deis/builder:v1.12.1    "/bin/entry /bin/boo   2 days ago          Up 2 days           <IP_ADDRESS>:2223->2223/tcp                                  deis-builder
f304d7aef77e        deis/logspout:v1.12.1   "/bin/logspout"        2 days ago          Up 2 days                                                                   deis-logspout
7e3f01a4ceae        deis/logger:v1.12.1     "/bin/logger --enabl   2 days ago          Up 2 days           <IP_ADDRESS>:514->514/udp, 514/tcp, <IP_ADDRESS>:8088->8088/tcp   deis-logger
a1226c6b8978        sysdig/agent            "/docker-entrypoint.   2 days ago          Up 2 days                                                                   sysdig-agent

Logger seems to be up, but has no data regarding my app (alfred)
core@ip-10-0-140-126 ~ $ docker logs --tail=all deis-logger
2016/01/05 20:57:32 configurer running
2016/01/05 20:57:42 configurer: Activated new storage adapter: memory:2000
2016/01/05 20:57:43 syslogish server running
2016/01/05 20:57:43 weblog server running
2016/01/05 20:57:43 publisher running
2016/01/05 20:57:43 deis-logger running
2016/01/07 18:28:19 Could not find logs for 'alfred'
2016/01/07 18:28:20 Could not find logs for 'alfred'

The logsprout on the instance containing the pod seems to be running.
core@ip-10-0-101-124 ~ $ docker logs --tail=all deis-logspout
2016/01/07 21:25:53 attach: e3e2f607a78a deis-logspout success
2016/01/07 21:25:53 attach: a1e8e540cecc k8s_alfred-cmd.a21bfbc0_alfred-v2-cmd-33h04_alfred_7f0140d1-b3fe-11e5-b730-0a4263846359_21f5274f success
2016/01/07 21:25:53 attach: ae2b0847b46e k8s_POD.e4cc795_alfred-v2-cmd-33h04_alfred_7f0140d1-b3fe-11e5-b730-0a4263846359_9806dce9 success
2016/01/07 21:25:53 attach: 3a199a5d1638 deis-publisher success
2016/01/07 21:25:53 attach: bb7e0364fcf6 sysdig-agent success
2016/01/07 21:25:53 etcd: http://<IP_ADDRESS>:4001
2016/01/07 21:25:53 routing all to udp://<IP_ADDRESS>:514
2016/01/07 21:25:53 loading and persisting routes in /tmp
2016/01/07 21:25:53 logspout serving http on :8000

I don't know how logs should looks like when logsrpout send data to logger,
but it seems there is something missing here, either data are not sent or logger does not store the data ?
any idea would be more then welcome, and thanks again for your help.

Use docker ps on the node hosting alfred to find the container id.  Then you can grep the logspout logs on that same host to see if logspout has "attached" to that container.

Ok. here with another application:
here just to check app logs, and get node on which a pod of my app is running:
(hearst)nassims-MacBook-Pro:devops nacim$ deis logs -a rover

(hearst)nassims-MacBook-Pro:devops nacim$ kubectl --namespace=rover get pods -o wide
NAME                 READY     STATUS    RESTARTS   AGE       NODE
rover-v8-cmd-z6wzs   1/1       Running   0          11m       <IP_ADDRESS>

I ssh to the node:
(hearst)nassims-MacBook-Pro:devops nacim$ ssh <IP_ADDRESS> -F provisioning/envs/devops/ssh.config
Warning: Permanently added '<IP_ADDRESS>' (RSA) to the list of known hosts.
Last login: Sun Jan 10 17:59:01 2016 from <IP_ADDRESS>
CoreOS stable (766.4.0)
Update Strategy: No Reboots
Failed Units: 1
  update-engine.service
core@ip-10-0-135-161 ~ $ docker ps
CONTAINER ID        IMAGE                                  COMMAND                CREATED             STATUS              PORTS               NAMES
48db454fc867        <IP_ADDRESS>:5000/rover:v8             "./bin/boot"           11 minutes ago      Up 11 minutes                           k8s_rover-cmd.9005fbc7_rover-v8-cmd-z6wzs_rover_82f14b08-b7c2-11e5-946e-12d891e80f69_3919bd5f
8a607b8367c4        gcr.io/google_containers/pause:0.8.0   "/pause"               11 minutes ago      Up 11 minutes                           k8s_POD.e4cc795_rover-v8-cmd-z6wzs_rover_82f14b08-b7c2-11e5-946e-12d891e80f69_9fa41800
f63811bb51a6        <IP_ADDRESS>:5000/alfred:v2            "/go/bin/alfred"       18 hours ago        Up 18 hours                             k8s_alfred-cmd.d533fc33_alfred-v2-cmd-c4z55_alfred_da12cd0d-b727-11e5-946e-12d891e80f69_1925fec2
959ef0761ff2        gcr.io/google_containers/pause:0.8.0   "/pause"               18 hours ago        Up 18 hours                             k8s_POD.e4cc795_alfred-v2-cmd-c4z55_alfred_da12cd0d-b727-11e5-946e-12d891e80f69_db1f8733
be8efcac2133        deis/logspout:v1.12.1                  "/bin/logspout"        42 hours ago        Up 42 hours                             deis-logspout
de91b56b1db7        deis/publisher:v1.12.1                 "/usr/local/bin/publ   46 hours ago        Up 46 hours                             deis-publisher
7204d2b0a937        sysdig/agent                           "/docker-entrypoint.   47 hours ago        Up 47 hours                             sysdig-agent

and here I get logsprout logs, I can see something like "attach" for my container.
core@ip-10-0-135-161 ~ $ docker logs be8efcac2133 &> /tmp/logs && cat /tmp/logs | grep 48db454fc867
2016/01/10 17:49:42 event: 48db454fc867 create
2016/01/10 17:49:42 event: 48db454fc867 start
2016/01/10 17:49:42 attach: 48db454fc867 k8s_rover-cmd.9005fbc7_rover-v8-cmd-z6wzs_rover_82f14b08-b7c2-11e5-946e-12d891e80f69_3919bd5f success
2016/01/10 17:50:04 event: 48db454fc867 exec_create: env
2016/01/10 17:50:04 event: 48db454fc867 exec_start: env
core@ip-10-0-135-161 ~ $

and here I check if docker logs on my container actually contains data:
core@ip-10-0-135-161 ~ $ docker logs 48db454fc867
/app/rover/urls.py:87: RemovedInDjango110Warning: Support for string view arguments to url() is deprecated and will be removed in Django 1.10 (got content.views.healthcheck). Pass the callable instead.
  url(r'^healthcheck/?$', 'content.views.healthcheck'),

System check identified some issues:

WARNINGS:
?: (urls.W002) Your URL pattern '^/?$' has a regex beginning with a '/'. Remove this slash as it is unnecessary.
Operations to perform:
  Apply all migrations: content, contenttypes, auth
Running migrations:
  No migrations to apply.
[2016-01-10 17:49:43 +0000] [11] [INFO] Starting gunicorn 19.4.1
[2016-01-10 17:49:43 +0000] [11] [INFO] Listening at: http://<IP_ADDRESS>:80 (11)
[2016-01-10 17:49:43 +0000] [11] [INFO] Using worker: sync
[2016-01-10 17:49:43 +0000] [14] [INFO] Booting worker with pid: 14
core@ip-10-0-135-161 ~ $

So to be honest, I suspect logsprout not being reading the logs from the right source of data ? could that be linked to the fact we use k8s schedular ?

Just a quick FYI that the kubernetes scheduler in v1 is a technical preview and is not to be used in production (see the note here). It's just a proof of concept and there are likely going to be bugs.
That being said, this is likely an issue with the kubernetes scheduler as the pod names are distinctly different than what a fleet unit name looks like, though I'd have to reproduce the issue to get a better view of the problem.

@Grindizer, I'm at a loss in terms of what's going on here.  @bacongobbler mentioned the difference in how things are named being a potential culprit.  I've checked up on those regexes and decided to double-check them.
TL;DR-- nothing wrong with the container names or the regular expressions that try to match them.
With a container named k8s_rover-cmd.9005fbc7_rover-v8-cmd-z6wzs_rover_82f14b08-b7c2-11e5-946e-12d891e80f69_3919bd5f, this line in the logspout code:
https://github.com/deis/deis/blob/master/logspout/logspout.go#L109
Should match it and extract the tokens:

rover
cmd-z6wzs

This line incorporates those into a log message:
https://github.com/deis/deis/blob/master/logspout/logspout.go#L87-L92
The format of that message will be:
<timestamp> rover[cmd-z6wzs]: <message>

On the logger end, the app name is extracted from the message using this regex:
https://github.com/deis/deis/blob/master/logger/syslogish/server.go#L21
Given a message formatted as indicated above and given that regular expression, it seems that rover is correctly identified as the application that the log message originated from.  It should be slotted into the ringbuffer for that application.
fwiw, I use this little online tool for testing regular expressions:
https://regex-golang.appspot.com/assets/html/index.html

The only remaining thing I can think of is to ask you to check the value of the /deis/logs/storageAdapterType key in etcd.

@krancour
core@ip-10-x ~ $ etcdctl get /deis/logs/storageAdapterType
memory:2000

@krancour It looks like this was our fault. After your last response about how it "should" work, I went back over everything and noticed we didn't have the correct udp ports open to allow the syslog traffic back and forth - we had tcp instead. Sorry to waste your time but thanks for the clear explanations and quick responses.

I've made that same mistake before, so don't feel too bad.
@Grindizer that may be something for you to check as well.

@krancour I'm working with @Grindizer . It was the same issue.

I suspected, but didn't know that for sure.  Thanks.

@krancour really thank you again for your time, if it can help we at least learn better how to debug logging in Deis (I know, it's a kinda thin :o)).

It was no trouble.  You won't be the only ones to encounter this difficulty.  The actual root cause and all of the hypothesizing as to other possible causes will certainly help  others in the community eventually, and I'm sure the greater insight you now have into Deis will pay dividends as well.  Pay it forward.  :)


Refactor comment box

Description of change
Small suggested refactor of comment box on Assessment view using Stimulus to conditionally display only one div instead of two and to attach the correct IDs when yes and no are clicked.

This looks good, by refactoring it to use stimulus it would be great if we have one comment field in the db! What do you think?

Closing in favour of #106


embed pipy as service lb and service mesh  engine.

add v0.4.2 pipy operator pr https://github.com/xiaods/k8e/pull/144

done


Public response when no response type defined

Fixes #1348. We want access to the internal response in all cases. It was skipped when a response type wasn't defined in the schema.
Now the generated code includes the following impl Response when there are no headers:
    // azure-sdk-for-rust/services/mgmt/compute/src/package_2022_11_01/mod.rs
    pub mod power_off {
        use super::models;
        pub struct Response(azure_core::Response);
        impl Response {
            pub fn into_raw_response(self) -> azure_core::Response {
                self.0
            }
            pub fn as_raw_response(&self) -> &azure_core::Response {
                &self.0
            }
        }
        impl From<Response> for azure_core::Response {
            fn from(rsp: Response) -> Self {
                rsp.into_raw_response()
            }
        }
        impl AsRef<azure_core::Response> for Response {
            fn as_ref(&self) -> &azure_core::Response {
                self.as_raw_response()
            }
        }
    }

and if there are headers, they are included as well:
    // azure-sdk-for-rust/services/mgmt/alertsmanagement/src/package_preview_2021_08/mod.rs
    pub mod delete {
        use super::models;
        pub struct Response(azure_core::Response);
        impl Response {
            pub fn into_raw_response(self) -> azure_core::Response {
                self.0
            }
            pub fn as_raw_response(&self) -> &azure_core::Response {
                &self.0
            }
            pub fn headers(&self) -> Headers {
                Headers(self.0.headers())
            }
        }
        impl From<Response> for azure_core::Response {
            fn from(rsp: Response) -> Self {
                rsp.into_raw_response()
            }
        }
        impl AsRef<azure_core::Response> for Response {
            fn as_ref(&self) -> &azure_core::Response {
                self.as_raw_response()
            }
        }
        pub struct Headers<'a>(&'a azure_core::headers::Headers);
        impl<'a> Headers<'a> {
            #[doc = "Service generated Request ID."]
            pub fn x_ms_request_id(&self) -> azure_core::Result<&str> {
                self.0.get_str(&azure_core::headers::HeaderName::from_static("x-ms-request-id"))
            }
        }
    }

I updated the changelog, even though I'm not generating a new version. I was able to generate updated SDKs locally using the fork of the docs but the docs themselves crashed and I didn't investigate (a union code generation issue I believe), and wasn't sure which branch to use.
Let me know if there's something I can do to help with that. I plan to generate the compute for myself and use a patched version.

@microsoft-github-policy-service agree


added cluster for ext-pusher

Added missing string for https://github.com/flarum/flarum-ext-pusher/pull/6

Alphabetical order please :)

I just did it on master.
Sorry, @Luceos ;)

@franzliedke no problem, glad you do this in fact. It keeps the pace on the project ;)


aws wrangler - s3.to_csv stripes 00:00:00 in timestamps

Describe the bug
when exporting to csv in s3 using aws wrangler, when it's a datetime column and time is 00:00:00, for example 2024-04-04 00:00:00, the csv will only have 2024-04-04.
In the same column, if you have another row that is 2024-04-04 11:11:11 (non 00:00), then in the same column but different rows, some will be date, and some will be datetime
How to Reproduce

create a dataframe with a column for date time
Have a few rows with different datetime, including at least 1 row with 00:00:00
Export to s3 using aws wrangler (import awswrangler as wr) with: wr.s3.to_csv
download and open the csv in your computer
notice that the row with 00:00:00 is only exporting the date

Expected behavior
datetime that have 00:00:00 should still be exported into csv.
Your project
No response
Screenshots
No response
OS
Lambda x86_64
Python version
Python 3.10
AWS SDK for pandas version
AWSSDKPandas-Python310	layer 13
Additional context
No response

Hi - thank you for your help on this!
This is within a lambda function:
        cur.execute("select * from test")
        offer_reports = cur.fetchall()
        data = pd.DataFrame(test)
        if not data.empty:
            wr.s3.to_csv(data, f's3://cdc-target/test.csv',index=False)

I tried using df.to_csv() but getting an write error within the lambda function when trying to use that function.

Your code feeds a test variable to pd.DataFrame but it is referenced nowhere else so hard to infer anything from it. In any case, here are two suggestions:

Ensure that the column is indeed of type datetime64
pandas to_csv has a date_format argument that you can leverage to enforce how datetime objects are formatted (e.g. '%d/%m/%Y %H:%M:%S')

Hi @jaidisido - apologies, I was trying to remove any syntax from my code, my code actually is the following:
        cur.execute("select * from reports")
        reports = cur.fetchall()
        data = pd.DataFrame(reports)
        if not data.empty:
             wr.s3.to_csv(data, f's3://cdc/reports.csv',index=False)

with this code, for some reason it is stripping out 00:00:00, if i add 1 second to it, then I have no issues exporting out datetime.
I tried using to_csv in the lambda function but running into permission issues and haven't been able to resolve the permission issues yet.


Add more template fields to DbtBaseOperator

This partially addresses #754 via allowing for built-in templating support for the DbtBaseOperator.
I also noticed --full-refresh was not documented so I added that in.
Still missing
Manual run pattern is not documented; the fact that these fields are templated is not documented. I don't really know where in the docs to put this. The docs are very API-focused more than narrative-based or suggestive, and Cosmos's maintainers prefer this style of documentation so it's hard to find a spot for this. It's possible that that's fine and we just keep this as a feature for more advanced users who dig into source code to discover for themselves? 🤷

@dwreeves upps I missed documenting it. Thanks for taking care of it 🙇

I think the thing that would make the most sense for now is to just add a section under /operator-args.html that is just about the templated variables. With the exception of --full-refresh, all of these variables are either (1) potentially used for rendering if DBT_LS is true (vars and env), or (2) used for model selection (select, selector, models, exclude) and are thus typically out of scope for operator_args inside of the dbt->Airflow converter context. A note should be included that clarifies this, although it's a little confusing because --full-refresh is essentially the only variable where it would make sense to do something like: operator_args={"full_refresh": "{{ params.full_refresh }}"}.

(2) Highlight the Cosmos arguments that are template fields where we expose them. Eg. In the case of select, exclude and selection, this could be in:
https://astronomer.github.io/astronomer-cosmos/configuration/render-config.html

So this is a bit weird because templating does not work when rendering a DAG, only when executing the task. I will keep in mind that a note should be made about this.

This also does beg a question though about the recent treatment of the dbt_vars and dbt_env. Right now in validate_initial_user_config(), an error is raised when both operator_args={"vars": ...} and project_config.dbt_vars are set. Since both rendering and execution use --vars and environment variables, this means that rendering and execution are strongly coupled and it's not possible to have these differ.
This was something I was thinking about irrespective of this particular PR. I think it's possible that it should be possible to set both operator_args={"vars": ...} and project_config.dbt_vars, and that the behavior would be that project_config.dbt_vars is used for rendering and operator_args={"vars": ...} could either merge or overwrite. Alternatively, render config and execution config could each have their own notion of dbt vars that can get set to the project configs' vars if either is not defined at the execution or rendering level.
In general the render and execution configs share a lot of configuration in the case that DBT_LS needs to be run, and maybe a separate discussion but I think there should be some sort of way to make the configurations more decoupled at the user level, but more DRY in their under-the-hood implementations. If that makes sense.

The reason why this discussion is potentially relevant to this PR is because of my note above about how the only operator args you can seemingly reliably utilize templating for at Cosmos's "highest level" API (i.e. DbtDag + DbtTaskGroup) is full_refresh.
I mean, you can access vars and env technically, but these will either:

if set in operator_args, they will raise deprecation warnings as of 1.3
if set in project_config and LoadMode.DBT_LS is used, then it will not properly render at all during LoadMode.DBT_LS rendering, i.e. the string literal will be used for either the dbt var or env var, since it never gets passed through Jinja and the scheduler wouldn't have potentially relevant context variables anyway.

This all feels both somewhat suboptimal, and also extremely confusing, to users. It's strange that ProjectConfig(dbt_vars={'logical_date': '{{ context["logical_date"] }}') is valid code, and works perfectly fine and as-intended if the user is using LoadMode.DBT_MANIFEST, but does not behave so well when using LoadMode.DBT_LS. It's tricky to convey this in documentation as a general statement. "Templating does not work with ProjectConfig(dbt_vars=...)" is not a true statement, but it is not a false statement either. The truth is templating works only in execution, and may or may not fail in rendering. Here are 2 example cases when it fails in rendering:

Since executing dbt ls does involve dbt parsing all the Jinja, and if your SQL query has something like {{ modules.datetime.datetime.fromisoformat(var("logical_date")) }} then the code will fail when rendering even though it would succeed when executing. In this case I think the user would want to either not have logical_date set at all (and rely on a default instead), or alternatively they should be allowed to set logical_date differently in rendering and execution.

Env vars can be a little more realistic a problem if, say, an env var is being used as part of a connection or to enable or disable models. For example, imagine the user sets a dbt profile with something like role: {{ env_var("DBT_ROLE") }}, and in their DAG they want to set DAG(..., params={"dbt_role": Param(...)}) and inside a DbtTaskGroup they want to set DbtTaskGroup(..., project_config=ProjectConfig(..., dbt_vars={"dbt_role": "{{ params.dbt_role }}"})). Here, again, it works fine in execution, but it does not work in the case where the load method is LoadMode.DBT_LS specifically.

That said, I am aware these are potentially extreme edge cases-- for example in most cases passing the string literal {{ logical_date }} into the dbt rendering will have zero impact on parsing with dbt ls and everything will render fine. So this adds to the confusion! The fully qualified correct statement is: Templating works in execution but does not work in rendering when the load method is LoadMode.DBT_LS; it may or may not matter depending on your dbt project. And that's just way more confusing than saying it doesn't work with dbt ls fully unqualified, and instead pushing users toward a separate layer of abstraction in the event that they want access to templating.

OK, this is all very confusing and in the weeds so I hope what I am saying makes sense. TLDR is:

Probably just stick this in operator-args.html as a separate section.
I would not mention this at all in ProjectConfig; and if we do, we may want to revisit some of the 1.3 changes regarding deprecation of vars= and env= in the operator_args?
The use of templating is actually really complicated and isn't fully addressed by the abstractions. It may be best to revisit this.
For now, without any further code changes, the only real thing that can be templated via DbtDag or DbtTaskGroup is full_refresh. (Maybe also warn users that render_template_as_native_obj=True should be true for that to work, since right now it cannot handle strings; alternatively make it so full_refresh can handle strings??).

@tatiana:

I have updated the docs
I have made another minor change to handle type-checking for templating of full_refresh.
I added full_refresh to the DbtBuildMixin

Sorry about the above long post, but it is still relevant to the use of env and vars in templating and DAG execution. Specifically the case of passing {{ logical_date }} as an environment variable or dbt var is far from being a stretch and the current API is not super friendly to this.
To partially address it I added the following note in the docs:

.. note::
    Using Jinja templating for ``env`` and ``vars`` may cause problems when using ``LoadMode.DBT_LS`` to render your DAG.

I would love it if this PR made it into Cosmos 1.4. I feel like this is a pretty straightforward PR and shouldn't be hard to merge or have any unintended side effects.
I discussed with @jlaneve that I have a massive blog post with multiple optimizations (including one I've never seen discussed anywhere that shaves an average of 3.3 seconds off each task execution; that really adds up). In that blog post, one of the "quality of life" improvements I wrote about assumes this PR is merged into Cosmos 1.4: 😄👉👈

It looks like the only reason this is failing is due to the strict coverage requirement. Hrm. I really really do want this available for 1.4.0, but I don't know when I'll have time to address that. Probably this weekend at the earliest.

Cov requirements are still failing. There is no universe where I get this done before the weekend, just as an FYI.

Thanks, @dwreeves , I just managed to get some time to add the missing coverage bit - it will be released as part of the alpha today!

Yes! Thank you! Can't wait for this 😁


Add the permission validity check when modifying broker read/write permissions

To avoid misoperations, we need to check whether the permission values entered by users are valid

You can submit a pull request to solve it. Thank you.


OCTO-1748: Make number of iterations as a method argument in _bayes_sampling

Number of iterations for tests is decreased to 2000, tests assertions are adopted accordingly.

Coverage decreased (-0.5%) to 78.93% when pulling e377f0c65e76e60a87f534b8ef510894478ba08f on darya_expan into 79006b473c89114df860a7494386524a7cfe37b1 on dev.

Good job Dasha! Our build pipeline seems to be reduced by ~5 mins.


Update Composer dependencies (2020-08-05-00-08)

Visual regression test passed!
View the visual regression test report


Create height slider transition class / animation / mixin

Something that can be applied to components to give an easy way of minimizing them.
Absolutely positoined + / - toggle, appears on hover

Changed the navigation - don't need to worry about sliders / accordions


[WIP] Improve installation instructions

My plan with this PR is to move the documentation of supported deployment methods into the repository and out of the wiki. I think that the installation and update documentation should be tied to the revision one is using.
Suggested official (read supported) installation methods:

The new 'manual' installation
Heroku when paying for the 'Hobby' tier
Docker to check huginn out locally (I dont think we should suggest running it on production, if anyone wants to do it he probably knows what she is doing? /cc @ianblenke )

Wiki pages which I plan to integrate and/or delete after this is done:

Heroku Revise and move to the repository
OpenShift I suggest removing the Link to it but to keep the page in the wiki for the case they fully support rails 4.2
Deployment with Capistrano, Unicorn, nginx, Foreman, and Upstart More or less replaced with the new guide, capistrano is unneeded for most users and developers probably have their own preferred way to set up capistrano and the server
Deploying huginn on any server or virtualbox using chef solo and/or vagrant not maintained for a long time and broken?
Using Huginn on predeployed Docker container Revise and move to the repository
Try Huginn Free on Terminal.com does this still work, has anyone tried it?
Running Many Job Workers Will be covered in the repository

Most of the documentation is heavily insprired by the awesome guides of GitLab.
It is probably easier to read the docs by opening the docs folder of my fork than reading the markdown in the PR.
TODOs:

[x] Check installation on vagrant using the ubuntu/trusty64 base box
[x] Check installation on vagrant using the ubuntu/precise64 base box
[x] Check installation on vagrant using the debian/jessie64 base box
[x] Check installation on vagrant using the debian/wheezy64 base box
[x] Verify the installation works on DigitalOcean with Ubuntu 14.04
[x] Verify the installation works on DigitalOcean with Ubuntu 12.04
[x] Verify the installation works on DigitalOcean with Debian 8.1
[x] Verify the installation works on DigitalOcean with Debian 7
[x] Write the update documentation
[x] Document how to change the Procfile to run multiple DelayedJob workers
[x] Move Docker documentation to the repository
[ ] Move/revise Heroku documentation to the repository
[ ] Remove old/unused files from the repository

Thanks to @lmarien and @enfop for doing an initial proof reading.
/cc @virtadpt @robbielj

Is there a substantive difference between a database set up for production mode, and one set up for development mode?

When this is up, I can contribute instructions for installing RVM and Ruby v2.x for installing Huginn.

Configuring .env - using DATABASE_ENCODING=utf8 isn't always a good idea because, if you use any of the Twitter agents they'll eventually run into four-byte characters (i.e., emoji).  On my instance it causes weird crashes.  Consider changing this to DATABASE_ENCODING=utf8mb4?

Is there a substantive difference between a database set up for production mode, and one set up for development mode?

None that I know of.

When this is up, I can contribute instructions for installing RVM and Ruby v2.x for installing Huginn.

Sorry the whole idea behind this is to have one, known, proven and working solution which makes it easier for everyone to debug issues. Supporting rvm complicates things and gives almost no benefits for production setups.

Configuring .env - using DATABASE_ENCODING=utf8 isn't always a good idea

Thanks, I will include the next lines out of the .env which explain the utf8mb4 encoding.

Awesome!
Unless Vagrant is working well, I'd be supportive of removing it and Chef, unless we can find someone who wants to work on them.

They do mostly work, nowadays - after fixing cookbook dependencies a while
back, I did find bugs in the roles (particularly the development one) a
couple weeks ago that I haven't had time to fix yet.  The production role
seems to be just fine - not bitrotted, works with current versions of
various required cookbooks, etc..
Please don't rip them out - it raises the complexity bar quite a bit on
what it takes to fiddle with huginn.  :)  Or maybe we just need to have A
Plan for how better to do this?
I suppose what I would suggest, if folks really want to separate concerns,
is to rip all the chef bits out into a cookbook, hosted on github, and then
have a separate vagrant-huginn repo that hosts a Vagrantfile,
Cheffile/Berksfile, and maybe a little bit of documentation.  That would be
the entrypoint for infrastructure hackers -- clone vagrant-huginn, have it
pull down the cookbook and other environmental doodads, and then have the
cookbook install and boot your huginn instance.
What changes, if any, would it take to make the above stuff all be tested
with travis-ci?  I am really just about totally unfamiliar with travis,
having spent more time in jenkins shops - if the Chef stuff isn't tested
anyway, then perhaps the above split would have little-to-no impact on
huginn, and ripping it out would clean up the huginn repo significantly?
I could probably do the cookbook-and-vagrant-repo required with not much
trouble - I guess I'm polling for other ideas before I just go messing with
stuff that perhaps no one else will ever use....
best,
--e
On Sun, Jun 21, 2015 at 8:59 PM, Andrew Cantino<EMAIL_ADDRESS>wrote:

Awesome!
Unless Vagrant is working well, I'd be supportive of removing it and Chef,
unless we can find someone who wants to work on them.
—
Reply to this email directly or view it on GitHub
https://github.com/cantino/huginn/pull/872#issuecomment-113981229.

@elijah To be honest I did not know that there is a development recipe. I get your point, that it makes setting up a development environment relatively easy. However one also needs to install a lot (knife-solo, librarian-chef and the recipe dependencies) to run the recipe which also could be done by a simple shell script. How about adding either something like rails-dev-box or a Dockerfile to boot a development box?

What changes, if any, would it take to make the above stuff all be tested with travis-ci?

The issue #416 to improve the chef deployment is almost a year old and even has quite a high bounty on it. So I don't think anyone is motivated to update the recipes and maintain it afterwards, which is understandable because it's a lot of work.
Even if it is working now, it clearly has problems that would needed to be fixed (ruby 1.9 is not supported, no SSL support, outdated .env, no documentation on how to update).
My feelings about the production recipe are about the same as for the development recipe, You need to install a few tools, maybe even on a VM if you are on windows, to run the recipe and afterwards need to configure it anyways.
The installation only needs to be done once, and doing it manually probably does not take longer then 30 minutes.

Personally, I think it's a shame to loose support / documentation on how to use Capistrano, since it makes updating really easy.  I'm looking forward to seeing your update process in this guide :)

On Mon, Jun 22, 2015 at 2:47 PM, Dominik Sander<EMAIL_ADDRESS>wrote:

@elijah https://github.com/elijah To be honest I did not know that
there is a development recipe. I get your point, that it makes setting up a
development environment relatively easy. However one also needs to install
a lot (knife-solo, librarian-chef and the recipe dependencies) to run the
recipe which also could be done by a simple shell script. How about adding
either something like rails-dev-box
https://github.com/rails/rails-dev-box or a Dockerfile to boot a
development box?

I took some time yesterday to rip out the site-cookbooks from the deploy
folder, make them a first class cookbook (elijah/cookbook-huginn), and
rework the Vagrant and Cheffile bits into another repo
(elijah/vagrant-huginn) that's been tweaked to call the cookbook-huginn
repo instead of using what's in the site-cookbooks folder.  I tried pretty
hard to make sure I included back-links to where the bits really came from,
so that someone stumbling across those repos would know where to dig in for
more.
This should make it possible to work on the chef cookbook bits without
disturbing any part of the huginn source base, and without needing it to
keep on living there.
I need to add a Makefile in the same dir that one can use to quickly
install the Vagrant-y dependencies, so that one can quickly get started
just by cloning that repo.... I have one somewhere, from another project,
just need to copy bits over.
I think the development recipe/role probably needs to die, and just be made
switchable with an attribute applied to what used to be the 'production'
role's recipes.
What changes, if any, would it take to make the above stuff all be tested

with travis-ci?
The issue #416 https://github.com/cantino/huginn/issues/416 to improve
the chef deployment is almost a year old and even has quite a high bounty
on it. So I don't think anyone is motivated to update the recipes and
maintain it afterwards, which is understandable because it's a lot of work.
Even if it is working now, it clearly has problems that would needed to be
fixed (ruby 1.9 is not supported, no SSL support, outdated .env, no
documentation on how to update).
My feelings about the production recipe are about the same as for the
development recipe, You need to install a few tools, maybe even on a VM if
you are on windows, to run the recipe and afterwards need to configure it
anyways.
The installation only needs to be done once, and doing it manually
probably does not take longer then 30 minutes

I actually was completely unaware that there was a bounty for this stuff -
just scratching my own itch pretty hard.
It's not too much work - and it's worthwhile work.  It needs to be done.
;-)  I'd very much like to just work my way down the list of wanna-haves
that other people have had, and make it better iteratively.
My interest in all of this is, as always, having additional duct tape to
glue together my own data - the fact that huginn allows for people to
actually develop their own bits and scratch their own itches is intensely
valuable, in an era where Yahoo Pipes is dead, IFTTT seems to only really
be workable if you are a large company who can get their attention (though
I understand they have plans for an API later...), and things like hubot
and logstash and other chat-opsy tools are immensely powerful but
discoodinated....
--e

@elijah That sounds great! Do you think we should keep a minimal Vagrantfile to set up a development environment for huginn in this repository or do you plan to integrate it into your cookbook repo?

Personally, I think it's a shame to loose support / documentation on how to use Capistrano, since it makes updating really easy. I'm looking forward to seeing your update process in this guide :)

That is true, and I am planning to keep my Capistrano setup, but don't think it is a great option to upgrade huginn when you are not following the development process, it is too easy to miss new/changed defaults in the .env or Procfile
Maybe we can change to "User" and "Administrator" sections to "User" and "Developer" and add information about Capistrano to the Developer section?

Awesome @elijah!  It's great to have you involved!  Would you like to make a PR to Huginn to remove chef and link to your repo?

Hi ! Thank you for the good work. I've read the suggested documentation section.
Overall I'm still very stunned of the complexity of Ruby installation and Ruby-based software.
Not your fault of course, I'm just realizing...
For example why would I have to install a compiler on a production server ?
The next step might be to write an installation script or something but it would be distribution-dependent :( Do you have ideas about that ?
About the doc " Deploying huginn on any server or virtualbox using chef solo and/or vagrant", I can say it works but is incomplete. For example it doesn't say a word about foreman. But it works and most of all it is quick as It somehow do all the manual job for you :)

@mhow2 Thanks!

For example why would I have to install a compiler on a production server ?

gems (ruby libraries) sometimes include c extension for speed improvements which need to be compiled. Thats why most ruby/rails application require a compile and quite a few c libraries.

The next step might be to write an installation script or something but it would be distribution-dependent :( Do you have ideas about that ?

@elijah Is working on improving the chef recipes in a separate repository, I don't know if he has plans to support multiple distributions.
I do not have plans to support more distributions with the guide because I don't know them well enough. If anyone else wants to add and maintain support for centos/*bsd/etc we probably need to find a way to separate the distribution specific instructions from the general huginn tasks.
The biggest issue with that would probably be finding a foreman export format (init system) which is supported on all distributions.

About the doc " Deploying huginn on any server or virtualbox using chef solo and/or vagrant", I can say it works but is incomplete. For example it doesn't say a word about foreman. But it works and most of all it is quick as It somehow do all the manual job for you :)

I have heard from a few people that the Vagrant/chef setup is working relatively well and hope brining it up to date will not be too much work.
The target audience I have in mind for the manual installation guide are users who have their own server but not necessarily experience with runy and/or a configuration management software. By the time you installed VirtualBox, vagrant, chef, and the needed cookbook dependencies, are at at least half way through the manual installation, plus you know what happened on your server.
To just try out huginn I think nothing can beat the docker container. These days docker runs everywhere and huginn boots after running one command.

Hi @dsander,
I tried docker build and and it works very well. However that's the first time I run a docker image :)
About running Huginn in production environment in a docker container (that would run on a VM as s3 or any cloud provider), I would like to know if there are best practices to both:

upgrade Huginn.
backup agents configuration.

The goal being the ability to restore a Huginn instance with its configuration in no time.
Maybe a combination of backup then destroy/pull new docker image / restore could be ?
(Am I out of topic or those best practices could  be included in Huginn documentation ?)
Thanks !

@mhow2 I agree a guide section on Docker would be useful.  (That might be something @icblenke could help with. :wink:)  I believe upgrading is as simple as deploying an updated Docker image, which builds automatically from Huginn master.  Backing up is a great question.  I wrote this wiki page a while ago, which is what I use, but I don't know how it'd work with Docker.

@mhow2 I am using docker quite a bit to test out stuff locally (mostly as a vagrant replacement) and for test runners but don't run any of my applications in production with docker. Mostly because I don't know either what's the best practice to upgrade.
@cantino @mhow2 I think you would use a linked database container and just update the huginn application to the latest version. You don't need to update back up the configuration when running the docker container because it's passed via ENV variables. The backup of the database container would then probably work like it does with a non-container database.

@dsander That's correct. The only reason there is a mysql built-in to the docker container is so that it may be run stand-alone.
Ideally, a container only runs one process. This typically means running puma/unicorn in one container, sidekiq in another, and mysql in another.
The only reason why the Huginn docker image has a built-in mysql is to allow it to be self-contained for easy adoption.
In a production environment, it is best to separate your persistence tiers from your immutable infrastructure application tiers.
Persistence layers, convergence tools, and DBaaS offerings.
This typically means using a DBaaS offering like AWS RDS, or Google Compute Cloud MySQL, OpenStack Trove, or something similar to assist in managing your persistence layer snapshots, backups, failovers, and other operational details.
There are also specific persistence layer hybrid-cloud management harnesses like Vitess.io that have the singular goal of managing a specific persistence layer like MySQL.
All implementations use some form of convergence tool like Chef, Puppet, Ansible, or SALT to help keep the "bits on disk" for configuration and software on these long-running nodes the same. You can always do this yourself if you don't wish to depend on the ShadowOps of DBaaS offerings. This requires some level of awareness on your part as to how to use these tools and maintain these components yourself.
Cloud resources as a persistence tier
Whichever way you chose, do yourself a favor and avoid provisioning cloud resources without a cloud convergence tool like AWS CloudFormation, OpenStack HEAT, Hashicorp Terraform, or even Chef/Puppet/Ansible and Foreman.
Cloud resources themselves are a persistence tier that needs convergence.
If you don't, you will quickly find yourself with a cloud account full of a mess of hand-edited cloud resources using the web console with no consistency imposed upon them.
Immutable Infrastructure layers
Your application containers shouldn't contain any persistence information themselves. Because of this they can easily be created and destroyed as the load demands it.
Immutable Infrastructure is something Docker is ideal for.  Docker has base "golden" images as built from Dockerfile diretion, and a container volume layer that is discarded if the container is removed. By treating containers as temporary, a convergence tool to keep the "bits on disk" the same over time isn't necessary. In effet, containers become immutable, and the infrastructure they provide will not diverge into "bit rot" over time as a normal host would.
To configure these containers, it is possible to share container volumes of config files, but that is a bit messy and generally requires another orchestration mechanism to generate them. Better still is to adopt the Heroku 12-factor approach of using environments for all container configuration.
As Huginn was built to be deployed in this 12-factor way using environment variables, it containerizes nicely with docker.
That leaves the "how do I deploy containers in production"?
This is where a Platform-as-a-Service (PaaS) is ideal.
This PaaS could be a public SaaS platform like Heroku or AWS ElasticBeanstalk / ECS.
This PaaS can also be a private or hybrid cloud hosting something like docker-compose on docker-swarm on docker-machine, Kubernetes,  Deis.io or one of the many others like it out there.
The key factors for picking a PaaS are really fair scheduling of containers to available compute resources and how it horizontally scales up and down on that load, as well as how deployment rolls and rollbacks are done so as to be as fault-tolerant as possible.
What do I do today?
Having fought the good fight with Deis for a long while, I've thrown in the towel on ever hoping to run Ceph reliably, nor maintain etcd quorum on anything but a dedicated persistence tier.
For development, I run docker-compose locally on docker-machine.
For production, I use AWS CloudFormation for the VPC and other cloud resource convergence, AWS RDS for database persistence tiers, AWS ElasticCache for redis, and AWS ElasticBeanstalk to deploy docker based immutable infrastructure micro-service application tiers.
YMMV. There is no one single solution to The Right Way to deploy docker containers. It depends as much on which cloud provider(s) you use as it does anything else.

Thank you everyone for all the provided information.
@dsander

but don't run any of my applications in production with docker, mostly because I don't know either what's the best practice to upgrade.

Isn't the solution for this scenario is what you write in the rest of your post ? (eg, ENV variables to attach an external DB). So to upgrade you just deploy the new docker image, and huginn would upgrade the database schema on startup. @cantino, is the DB schema upgrade process works from any to any version ?
Currently we are just testing Huginn but I want to make sure I would be able to maintain the software if it is adopted and would run in production. We are using either openstack or cloudstack own private cloud provider. As now I haven't tested docker with attached database as ENV. But I have the following issues with what I tested until now:

deploy with chef : works well but unable to start/stop all huginn process from init scripts (because the init script doesn't do anything about foreman). Upgrading is known to be done just with deploying again. (not 100% sure of this ?).
deploy with docker : lightening fast. Easy to manage. By the way I was unable to do a commit of the running Huginn container (which could be used as a from of backup), and start a new image based on this commit because of a stalling pid file foreman stdout | 07:06:35 web.1  | A server is already running. Check /app/tmp/pids/server.pid.. Maybe I should halt the container before commiting but just don't know how to do that for now ;).

And for backup, I've been surprised not to see something oriented to the end user ; as a simple "backup agent(s)" button so any user can backup his agent configuration for the UI.

Yes, @mhow2 Rails migrations are almost always designed to go from any past version to any future version.

@ianblenke Thanks for the detailed information! This is consistent with my experience, there is some great 'enterprise' PaaS support for docker but no complete and easy to use solution for small deployments.

Isn't the solution for this scenario is what you write in the rest of your post ?

Yes that's the theory about how to do it. In practice I don't know how one would to it to achieve near zero-downtime deployments. (Which I don't need for my huginn instance, but a few other rails applications)

By the way I was unable to do a commit of the running Huginn container (which could be used as a from of backup), and start a new image based on this commit

You don't need to back up anything in the huginn container if you are using a linked database. Every data huginn uses is stored in the mysql or postgresql database.

You don't need to back up anything in the huginn container if you are using a linked database. Every data huginn uses is stored in the mysql or postgresql database.

What if the linked database runs in another container like suggested in the docker setup and being destroyed for any reason ? Do you mean that I could link the Huginn container to a official 'production-safe' mysql instance instead of a docker mysql container ?

Yes!
General "best practice" at the moment is to maintain your persistence tiers with a convergence tool or automated harness, and that you do so separately from your application layer tier.
So long as DATABASE_URL points at a mysql database with correct credentials, Rails really doesn't mind where it is hosted.

There's a separate repo that has the vagrantfile and bits inside;  the
huginn project itself is a cookbook, and would be the place to land 'tests'
of chef-cookbook functionality, etc.
I haven't had time to get back to this in the last couple of days; I did
get as far as putting the wishlist from the bounty into one of the readme
files, and plan to just crank down it and make all those things work.
--e
On Mon, Jun 29, 2015 at 3:01 PM, Dominik Sander<EMAIL_ADDRESS>wrote:

@elijah https://github.com/elijah That sounds great! Do you think we
should keep a minimal Vagrantfile to set up a development environment for
huginn in this repository or do you plan to integrate it into your cookbook
repo?
—
Reply to this email directly or view it on GitHub
https://github.com/cantino/huginn/pull/872#issuecomment-116820449.

On Thu, Jul 2, 2015 at 3:55 PM, Dominik Sander<EMAIL_ADDRESS>wrote:

@elijah https://github.com/elijah Is working on improving the chef
recipes in a separate repository
https://github.com/elijah/cookbook-huginn, I don't know if he has plans
to support multiple distributions.
I do not have plans to support more distributions with the guide because I
don't know them well enough. If anyone else wants to add and maintain
support for centos/*bsd/etc we probably need to find a way to separate the
distribution specific instructions from the general huginn tasks.

I'm planning to make it work under test-kitchen;  that should get me pretty
far toward being able to test on both CentOS/RHEL and Ubuntu, etc.  [Most
of the cookbooks that huginn was already using are pretty flexible / well
tested, and so this should be a fairly well-bounded task....
--e

I can sure do this.
Sorry for the lag in replying to mail; I got busy with an infrastructure
project, and so this kind of got dropped on the floor for a few days.  :)
--e
On Mon, Jun 29, 2015 at 8:20 PM, Andrew Cantino<EMAIL_ADDRESS>wrote:

Awesome @elijah https://github.com/elijah! It's great to have you
involved! Would you like to make a PR to Huginn to remove chef and link to
your repo?
—
Reply to this email directly or view it on GitHub
https://github.com/cantino/huginn/pull/872#issuecomment-116900723.

I think I am finally done! The change from upstart to runit made the installation on debian a lot easier, because runit does not replace the default init system, on top of that the guide should work on non debian based distributions as long as the dependencies are installed (namely ruby, mysql, runit and nginx).
Using capistrano to deploy updates is now very easy, just set a few .env variables and run cap production deploy

WOW.
I have some reading to do.  I'll give this a read and get back to you soon.  Do you want to link it from the wiki / README now and ask people for beta feedback?

Yeah that's a good idea, but I will wait until you are done with initial proof reading.
The branch then also needs to be merged into my master branch to make manual upgrade guide work.

Very good!  Clearly a huge effort.  It might be worth recommending that people setup iptables or ufw (on Ubuntu) as a basic security along with SSL too.

I concur.  UFW would be easier than a set of IPtables commands (though they can be pasted into /etc/rc.local pretty easily).

Thanks for your feedback!

It might be worth recommending that people setup iptables or ufw (on Ubuntu) as a basic security along with SSL too.

I think we should stick with huginn in our guides, when we start adding recommendations about the general server setup it isn't really a point where we should stop.

The only counter argument is that this guide does make some server recommendations, so I think it'd be worth at least suggesting that installing a firewall, along with SSL, would be a good idea.

The only counter argument is that this guide does make some server recommendations, so I think it'd be worth at least suggesting that installing a firewall, along with SSL, would be a good idea.

Would you just recommend installing a firewall like ufwor also make configuration suggestions? For the latter I do not know what config one would use, mine is pretty simple: reject everything and allow port 22, 80 and 443

I'd probably just have a sentence suggesting "we also recommend applying standard security practices to  your server, including installing a firewall (ufw is good on Ubuntu)."

Gotcha, that makes sense.

@cantino How do you want to proceed with this PR, should we keep it in my branch and add link from the official README or merge it?
I covered all the cased I could think of in the troubleshooting section, but am pretty sure we will have to expand it over time :smile:

I think we should definitely merge it and update the wiki.  Do you want to do that now?

I will update the wiki after merging it with links to the new documentation to avoid having dead pages in the wiki.

Oh, I understand the "per default" line now, sorry for the confusion.
I'd be fine with you either using or removing the deployment/ directory.  I
find lib/ a bit odd, though, since it has a typical Rails usage that's for
Ruby code.  Not a strong opinion, however.
On Wednesday, August 19, 2015, Dominik Sander<EMAIL_ADDRESS>wrote:

I will update the wiki after merging it with links to the new
documentation to avoid having dead pages in the wiki.
—
Reply to this email directly or view it on GitHub
https://github.com/cantino/huginn/pull/872#issuecomment-132679314.

Merged and updated the wiki pages, thanks for all the feedback! I hope I did not change/remove too much information from the wiki.

Thank you for the huge effort! I'll try to find a time to convert this into
an Ansible script since that's what I will likely use for my personal
deployment.  It is time to upgrade my Capistrano usage too so this is all
very helpful!
On Wednesday, August 19, 2015, Dominik Sander<EMAIL_ADDRESS>wrote:

Merged and updated the wiki pages, thanks for all the feedback! I hope I
did not change/remove too much information from the wiki.
—
Reply to this email directly or view it on GitHub
https://github.com/cantino/huginn/pull/872#issuecomment-132796282.


Batch Data Import needs to check Field Level Security

Included in beta release version 3.48 (Beta 2) and higher

Included in production release version 3.48


Fix for #345 - headless fails on windows

String passed was being re-allocated, use std:string to create a copy instead
Fixes #345

LGTM


Twitter API returned a 401 (Unauthorized) when trying to get users's friends on twitter

I'm using the following code which for a while worked like charm, but recently in 9 of the 10 attempts I'm getting error from twitter api
from twython import Twython, TwythonError
from pymongo import Connection
import pika, time

connection = Connection("host")
connection.admin.authenticate('admin', 'passwords')
db = connection['tweets']

consumer_key="key"
consumer_secret="secret"

access_token="token"
access_token_secret="secret_token"

t = Twython(app_key=consumer_key,
            app_secret=consumer_secret,
            oauth_token=access_token,
            oauth_token_secret=access_token_secret
           )

Q ='download_friends'

r_connection = pika.BlockingConnection(pika.ConnectionParameters(
               'host'))
channel = r_connection.channel()
channel.queue_declare(queue= Q,
                arguments={'x-message-ttl':600000})

while 1:
    method_frame, header_frame, id = channel.basic_get(Q)
    if db.friends.find_one({"_id": id}) != None:
        print "Panic, user already exists"
        continue
    try:
        r = t.get_friends_list(user_id = id)
    except Exception as ex:
        print ex, id
    else:
        print id
        r['_id'] = id
        r['time'] = time.time()
        db.friends.save(r)

    time.sleep(121)

Twitter API returned a 401 (Unauthorized), An error occurred
processing your request.

Here is a stacktrace
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/twython/endpoints.py", line 290, in get_friends_list
    return self.get('friends/list', params=params)
  File "/usr/local/lib/python2.7/dist-packages/twython/api.py", line 230, in get
    return self.request(endpoint, params=params, version=version)
  File "/usr/local/lib/python2.7/dist-packages/twython/api.py", line 224, in request
    content = self._request(url, method=method, params=params, api_call=url)
  File "/usr/local/lib/python2.7/dist-packages/twython/api.py", line 194, in _request
    retry_after=response.headers.get('retry-after'))
twython.exceptions.TwythonAuthError: Twitter API returned a 401 (Unauthorized), An error occurred processing your request.

I the remaining 1 attempt I'm actually getting a user's friends.
I've googled a bit and corrected time on the machine (as they say one of the most common causes of this error), yet error still persists.
Also there is minimal not working example:
twitter = Twython('API key', 'API secret', oauth_version=2)
ACCESS_TOKEN = twitter.obtain_access_token()

print ACCESS_TOKEN

t = Twython('API key', access_token=ACCESS_TOKEN)
r = t.get_friends_list(user_id = 'id')

It is able to get ACCESS_TOKEN but nothing more.

See Twitter’s API details on this: has your token expired?
API request contains invalid bearer token
Using an incorrect or revoked bearer token to make API requests will result in:
HTTP/1.1 401 Unauthorized
Content-Type: application/json; charset=utf-8
Content-Length: 61
...
{"errors":[{"message":"Invalid or expired token","code":89}]}

I'm using
APP_KEY = '****'
ACCESS_TOKEN = '****'
twitter = Twython(APP_KEY, access_token=ACCESS_TOKEN)
twitter.search(q='python')

Getting 401. Here's the trace:
File "get_user_data.py", line 8, in <module>
  twitter.search(q='python')
File "/usr/local/lib/python2.7/dist-packages/twython/endpoints.py", line 169, in search
  return self.get('search/tweets', params=params)
File "/usr/local/lib/python2.7/dist-packages/twython/api.py", line 265, in get
  return self.request(endpoint, params=params, version=version)
File "/usr/local/lib/python2.7/dist-packages/twython/api.py", line 259, in request
  api_call=url)
File "/usr/local/lib/python2.7/dist-packages/twython/api.py", line 198, in _request
  retry_after=response.headers.get('X-Rate-Limit-Reset'))
twython.exceptions.TwythonAuthError: Twitter API returned a 401 (Unauthorized), Invalid or expired token.

Any help would be appreciated.


Making changes to the models does not change the chatbot.

I must be missing something here.
When trying to add a model to ext/django_chatterbot/models.py file, nothing happens.  Migrating states that there are no changes detected.  Is this not where to change, add, or delete files in Chatterbot 0.5.5?

Did you tried making migrations
Bash python manage.py migrate
https://docs.djangoproject.com/en/1.10/topics/migrations/#workflow


fix Conds bug on Find and add test for FindAndCount

Codecov Report

Merging #652 into master will decrease coverage by 0.02%.
The diff coverage is 84.61%.

@@            Coverage Diff             @@
##           master     #652      +/-   ##
==========================================
- Coverage   48.06%   48.03%   -0.03%
==========================================
  Files          34       34
  Lines        8359     8362       +3
==========================================
- Hits         4018     4017       -1
- Misses       3856     3858       +2
- Partials      485      487       +2

Impacted Files
Coverage Δ

session_cond.go
81.48% <ø> (+7.4%)
:arrow_up:

session_find.go
56.21% <100%> (+0.11%)
:arrow_up:

statement.go
61.45% <81.81%> (-0.62%)
:arrow_down:

xorm.go
67.14% <0%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1aa1846...2288de7. Read the comment docs.


Rob: Watch cleantalk

Rob requests the watch of the watch_keyword cleantalk. See the MS search here and the Stack Exchange search here.

cleantalk has been seen in 3 true positives, 0 false positives, and 0 NAAs.

K.Dᴀᴠɪs approved this PR in Charcoal HQ

Recent bunch of spam related to 'cleantalk'. Posts do not include links.
Examples:
https://stackoverflow.com/a/53272370/563532
https://stackoverflow.com/a/53271903/563532


Allowing disabling of pixel limit

As discussed in #250 allowing boolean values for limitInputPixels this partially removes the limit. Unfortunately there are other limits based upon the int32 for width/height that I stumbled on shortly after so I'd like to take a stab at that but this gets you significantly larger images. Let me know your thoughts

Hello, thanks for adding this feature. I'll add comments inline.
Are you able to update the API docs to mention the acceptance of a boolean value?
I'm happy for any always-positive parameters to change from signed to unsigned integer if that helps too.

@lovell should be able to handle the noted things and docs, would you be opposed to adding a really large image to the test suite to test these things better?

"would you be opposed to adding a really large image to the test suite"

Great idea. As long as the compressed file size is not "really large", this would also provide a good example of libvip's low memory usage.
A single-colour, non-interlaced PNG (with row filtering disabled?) should compress the best. Decompression memory usage relates to image width rather than height, so perhaps try something like 2000px wide and whatever-you-need px high.

just an update my holidays were a little hectic but I do plan on getting back to wrapping this up in January

@kentongray No worries, thanks for the update. Let me know if you need any guidance.

The master branch is now using the vips8 C++ bindings so you'll notice a conflict or two when rebasing.

This would be a great feature.  Is anyone currently working on @lovell comments?
How is the current permitted value calculated for limitInputPixels?
I get a exception telling me the permitted range but assume it varies depending on the machine because I noticed others using higher values than I (current: 268402689).  Is this correct?

@webbrandon Please go ahead and work on this if you're willing and able. limitInputPixels is currently set here to the maximum value allowed for WebP, which is the lowest common denominator when it comes to limits in supported formats.

Hey guys sorry for dropping off the planet on this one, but nothing will make progress like needed a feature (can't wait to use the padding). OK so I made the following changes.

Update docs
Added a large image  (20,000x14,000) and crushed it so it is 43k.
Updated tests to use the large image with true and false values

The only problem I potentially see is the large image test takes a bit (16267ms) on my machine. Not sure if that is acceptable or not.

Coverage increased (+0.01%) to 98.832% when pulling 3c3b3dabaf3d79064dbbadfa3d65dc9cd9392b1c on iOffice:master into b2d7d4c4a9a37d5ab7c85eccdcd056fbfaf27506 on lovell:master.

i'm guessing the failing test is due to the large image... let me know your thoughts on how to proceed blarg really was trying to wrap this up today

Loving the cyan, thanks for the updates @kentongray.
It looks like the new test is timing out on Windows CI. Are you able to double the test-win limit to 60000ms?
Please can you squash to a single commit (I've recently enabled PR squashing as the default behaviour but this PR was created before that, so you'll have to do it manually). More importantly, please add your good self as a fully-fledged contributor.

Coverage increased (+0.01%) to 98.832% when pulling 3e88428c75ebaede7cd10b529e23f5dc7349789c on iOffice:master into b2d7d4c4a9a37d5ab7c85eccdcd056fbfaf27506 on lovell:master.

Coverage increased (+0.01%) to 98.832% when pulling ed5b39388754141fd0ba101a3bf90f91b07efa0a on iOffice:master into b2d7d4c4a9a37d5ab7c85eccdcd056fbfaf27506 on lovell:master.

Coverage increased (+0.01%) to 98.832% when pulling ed5b39388754141fd0ba101a3bf90f91b07efa0a on iOffice:master into b2d7d4c4a9a37d5ab7c85eccdcd056fbfaf27506 on lovell:master.

Coverage increased (+0.01%) to 98.832% when pulling 8241884fb43541e10144d8ea2a7bc09f4b13a3bf on iOffice:master into b2d7d4c4a9a37d5ab7c85eccdcd056fbfaf27506 on lovell:master.

unfortunately still failed it seems :sob:

No worries, I'll look after the failing test (either add more time or skip on Windows).
Thanks again Kenton for all your work on this feature - this will be included in v0.14.1.

@kentongray your test is now <50s on Windows CI with https://github.com/lovell/sharp/commit/71fb839e2ba46327f8fc6a69eb811f6f72b79205

yay thanks @lovell

no problem @kentongray .  thank you very much for your contribution.


bug(COMPONENT): Angular Material Select adds 22px at bottom

Is this a regression?

[ ] Yes, this behavior used to work in the previous version

The previous version in which this bug was not present was
No response
Description
The Angular Material Select dropdown adds a div that has a height of 22px with the following CSS classes.
mat-mdc-form-field-subscript-wrapper mat-mdc-form-field-bottom-align ng-tns-c4048155087-35

This causes selects to get "Pushed Up" 22px when using for example in a material toolbar, causing it to be out of alignment with everything else.
Is this div necessary?
Reproduction
Steps to reproduce:

Open developer tools and look for the div
2.https://material.angular.io/components/select/examples

Expected Behavior
No additional bottom padding / height / margin.
Actual Behavior
The select adds 22px at the bottom.
Environment

Angular:
CDK/Material:
Browser(s):
Operating System (e.g. Windows, macOS, Ubuntu):

This div is for extra text like a hint or an error
.
I would recommend using the menu component instead. Our docs site shows an example of using the menu in a toolbar in the upper right hand side. (Github link)


New version: HopTB v0.8.3

Registering package: HopTB
Repository: https://github.com/HopTB/HopTB.jl
Created by: @atbug
Version: v0.8.3
Commit: 463728945e7187df613699f0d4966c5bfb351fe8
Reviewed by: @atbug
Reference: https://github.com/HopTB/HopTB.jl/commit/463728945e7187df613699f0d4966c5bfb351fe8#commitcomment-132944837
Description: Tight-binding package written in Julia

I would like to close the pull request because I opened another one: https://github.com/JuliaRegistries/General/pull/95581


Add an explicit step to a range

Without this, the line emits a warning when compiled with Elixir 1.17

CI is failing. Can you take a look? I’m without laptop til Monday so I can't look til then.

Pull Request Test Coverage Report for Build be6b6c105d49dc4262624254509f9a0b66de1ea3-PR-15
Details

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 94.667%

Totals

Change from base Build 08f57019018b380321d1cbdbd58a5505254bdfb0:
0.0%

Covered Lines:
142

Relevant Lines:
150

💛  - Coveralls

@whatyouhide Updated. Should work on any version of Elixir now.

Thanks @alco 💟

Can we have this pushed to a new version to silence the warning? :)

Release v1.0.0.


Migration: Refactored the way network connections are modeled per issue 126

Tasks for migration - write queries to do the following:

[x] networkConnection renamed to links
[x] hasFromNode renamed to linksFrom
[x] hasToNode renamed to linksTo
[ ] Verify/warn if new restriction (on NetworkLink that it must have exactly 2 links) fails
[ ] Verify/warn if new restrictions (on NetworkLink and NetworkNode that they must be memberOf a Network) fail

@Jamie-SA @rjyounes Can this be closed now?

Out-of-date.


[bug] SFPEGRecordListDisplay- Flow: The componenent shows 'null' values

Describe the bug
in SFPEG Record List Display component, when a field is not filled a null value is shown (cf; Image)
Expected behavior
Nothing is shown
Screenshots

@Mchaouche I shall have a look at this ASAP.

@Mchaouche Just had a look at your issue and cannot reproduce it.
Could you provide additional details ?
By the way, you opened the issue on the wrong repository (PEG_LIST instead of PEG_FLW)

Closed issue... not reproduced


Backend + Frontend implementation for UC: Layer Alternatives

Tasks

[ ]  implement API endpoint
[ ] implement in frontend

pr #696 contains a function to copy a layer and all its associated plantings.
Use case
Layer Alternatives
Related Pull request
#696

Thank you for creating the issue! Let us discuss today the prioritization.


⚠ Bofrost.es has degraded performance

In 2e5ce2d, Bofrost.es (https://www.bofrost.es) experienced degraded performance:

HTTP code: 200
Response time: 1378 ms

Resolved: Bofrost.es performance has improved in cda08c2 after 20 minutes.


Runtime Warning due to functions named like modules

Errors like this
RuntimeWarning: 'rasa_nlu.test' found in sys.modules after import of package 'rasa_nlu', but prior to execution of 'rasa_nlu.test'; this may result in unpredictable behaviour

are apparently due to functions being named like modules.
This has occured with:

rasa_nlu.train
rasa_core.train
rasa_core.run
rasa_nlu.test
rasa_core.test
rasa_core.visualize

!python -m rasa_nlu.test \
    --data export_data_format_train.json \
    --model model/ner_model \
    --mode evaluation

When I run evaluation script I am also getting the same error is it fixed?
/home/user/anaconda2/envs/jd_matcher/lib/python3.6/runpy.py:125: RuntimeWarning: 'rasa_nlu.test' found in sys.modules after import of package 'rasa_nlu', but prior to execution of 'rasa_nlu.test'; this may result in unpredictable behaviour
  warn(RuntimeWarning(msg))
2019-04-30 11:55:12 INFO     rasa_nlu.components  - Added 'SpacyNLP' to component cache. Key 'SpacyNLP-en'.
2019-04-30 11:55:12 INFO     rasa_nlu.training_data.loading  - Training data format of export_data_format_train.json is rasa_nlu
2019-04-30 11:55:12 INFO     rasa_nlu.training_data.training_data  - Training data stats:
	- intent examples: 0 (0 distinct intents)
	- Found intents:
	- entity examples: 3 (9 distinct entities)
	- found entities: 'soft_skills', 'technical_skills', 'degree', 'salary', 'job_position', 'degree_major', 'job_location', 'overall_work_experience', 'technology_work_experience'

Traceback (most recent call last):
  File "/home/user/anaconda2/envs/jd_matcher/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/user/anaconda2/envs/jd_matcher/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/user/anaconda2/envs/jd_matcher/lib/python3.6/site-packages/rasa_nlu/test.py", line 1001, in <module>
    main()
  File "/home/user/anaconda2/envs/jd_matcher/lib/python3.6/site-packages/rasa_nlu/test.py", line 995, in main
    cmdline_args.histogram)
  File "/home/user/anaconda2/envs/jd_matcher/lib/python3.6/site-packages/rasa_nlu/test.py", line 744, in run_evaluation
    intent_targets = [None] * test_data.training_examples
TypeError: can't multiply sequence by non-int of type 'list'

HI @Nomiluks, the runtime warning is just a warning and shouldn't have anything to do with your error. However you do seem to have run into a bug! I believe that line 744 should say
intent_targets = [None] * len(test_data.training_examples)

Thanks for catching -- we don't have many people running the test script without an intent classifier in their pipeline, so it was a pretty sneaky bug. I'll make sure to fix it.

HI @Nomiluks, the runtime warning is just a warning and shouldn't have anything to do with your error. However you do seem to have run into a bug! I believe that line 744 should say
intent_targets = [None] * len(test_data.training_examples)

Thanks for catching -- we don't have many people running the NLU test script without an intent classifier in their pipeline, so it was a pretty sneaky bug. I'll make sure to fix it.

thanks looking forward.

intent_targets = [None] * len(test_data.training_examples)

@erohmensing  I have added following changes in rasa_nlu/test.py code:
        # intent_targets = [None] * test_data.training_examples         intent_targets = [None] * len(test_data.training_examples)
But getting another error can you help me to fix this one?
`/home/user/anaconda2/envs/jd_matcher/lib/python3.6/runpy.py:125: RuntimeWarning: 'rasa_nlu.test' found in sys.modules after import of package 'rasa_nlu', but prior to execution of 'rasa_nlu.test'; this may result in unpredictable behaviour
warn(RuntimeWarning(msg))
2019-05-02 14:29:13 INFO     rasa_nlu.components  - Added 'SpacyNLP' to component cache. Key 'SpacyNLP-en'.
2019-05-02 14:29:13 INFO     rasa_nlu.training_data.loading  - Training data format of export_data_format_test.json is rasa_nlu
2019-05-02 14:29:13 INFO     rasa_nlu.training_data.training_data  - Training data stats:
- intent examples: 0 (0 distinct intents)
- Found intents:
- entity examples: 0 (0 distinct entities)
- found entities:
2019-05-02 14:29:13 INFO     main  - Running model for predictions:
0it [00:00, ?it/s]
2019-05-02 14:29:13 INFO     main  - Entity evaluation results:
2019-05-02 14:29:13 INFO     main  - Evaluation for entity extractor: CRFEntityExtractor
Traceback (most recent call last):
File "/home/user/anaconda2/envs/jd_matcher/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/home/user/anaconda2/envs/jd_matcher/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/user/anaconda2/envs/jd_matcher/lib/python3.6/site-packages/rasa_nlu/test.py", line 1002, in
main()
File "/home/user/anaconda2/envs/jd_matcher/lib/python3.6/site-packages/rasa_nlu/test.py", line 996, in main
cmdline_args.histogram)
File "/home/user/anaconda2/envs/jd_matcher/lib/python3.6/site-packages/rasa_nlu/test.py", line 780, in run_evaluation
report_folder)
File "/home/user/anaconda2/envs/jd_matcher/lib/python3.6/site-packages/rasa_nlu/test.py", line 421, in evaluate_entities
merged_targets, merged_predictions)
File "/home/user/anaconda2/envs/jd_matcher/lib/python3.6/site-packages/rasa_nlu/test.py", line 181, in get_evaluation_metrics
output_dict=output_dict)
File "/home/user/.local/lib/python3.6/site-packages/sklearn/metrics/classification.py", line 1568, in classification_report
name_width = max(len(cn) for cn in target_names)
ValueError: max() arg is an empty sequence`

Hi @erohmensing
When running the following command
python3 -m rasa_nlu.train -c nlu_config.yml --data data/nlu.md -o model --fixed_model_name nlu --project current --verbose
I'm getting the following errors at my prompt
RuntimeWarning: 'rasa_nlu.train' found in sys.modules after import of package 'rasa_nlu', but prior to execution of 'rasa_nlu.train'; this may result in unpredictable behaviour   warn(RuntimeWarning(msg)) 2019-05-08 22:27:36.044846: F tensorflow/core/platform/cpu_feature_guard.cc:37] The TensorFlow library was compiled to use AVX instructions, but these aren't available on your machine.

Hey @math77 , these are just warnings that you can ignore. We haven't found any "unpredictable behaviour" as a result of that first warning so far. 🙂

Hey @erohmensing
When executing the command I believe that it is not creating the models directory.

When you run the following code
from rasa_nlu.model import Interpreter import json interpreter = Interpreter.load("./models/current/nlu") message = "let's see some italian restaurants" result = interpreter.parse(message) print(json.dumps(result, indent=2))
My prompt returns the following message:
 File "run.py", line 3, in <module>     interpreter = Interpreter.load("./models/current/nlu")   File "/home/matheus/anaconda3/envs/robot/lib/python3.6/site-packages/rasa_nlu/model.py", line 299, in load     model_metadata = Metadata.load(model_dir)   File "/home/matheus/anaconda3/envs/robot/lib/python3.6/site-packages/rasa_nlu/model.py", line 73, in load     "from '{}'. {}".format(abspath, e)) rasa_nlu.model.InvalidProjectError: Failed to load model metadata from '/home/matheus/Documentos/projects/estudos/studies-rasa-lib/hall/models/current/nlu/metadata.json'. [Errno 2] No such file or directory: './models/current/nlu/metadata.json'

This is because you're using -o model , so it's outputting in the model directory, but you're looking for it in the models directory.
Your usage problem is not relevant to this github issue, please ask usage questions like these in the forum. Also, in a case like this, it's best to take a look into your actual directories to see if it has created the directory if you don't believe it did -- this will show you that it created a directory by another name due to a typo.


HighCal Sausage/Muckshake misslabeled

The High Cal Sausage and Muckshake variants are labeled wrong. The normal versions unlock with Cauldron lvl 2. The HighCal ones unlock with Cauldron lvl 1.

All the high-cal recipes only require the level 1 cauldron, so this is by design.
I might add the cauldron requirements at some point, but I don't really care about them as I've unlocked lvl 5 cauldron.


Allow user to tweak the parameters of the simualtion

The user should be able to change sliders which change the parameters of the simulation.
Estimated time: 1 h.

Closed by 3134612.


Update ingest-json-formats.md

Wait for review from @vladikbr

#sign-off

@ohadbitt, @shsagir
Do we still need a review from @vladikbr?

@ohadbitt, @shsagir
Do we still need a review from @vladikbr?

Vladik has already reviewed. Thx for checking. Please go and and merge.
#sign-off


Exercise to read-in and do something with geojson file

[ ] Read in the file
[ ] Do something with it, e.g. create a new feature
[ ] Print the output

And print the name and area in m^2 of each country, and maybe sort? Idea from Dustin Carlino :pray:


get reactive data.

Package does not return reactive data.  I've forked your example app, searched for a particular package, edited the 'packageName' in mongo.  The data in the app will not update until I re-query the search.
Please advise.

+1 to this issue.
The readme says "You can get the reactive data source with the PackageSearch.getData api" but when I use that, the data on the client side doesn't automatically reflect my updated documents.

+1

I'm getting reactive data with this package. Make sure your event handler is properly configured so your search works. For me, it was as simple as changing change input to keyup input.

I'm having the same issue, it's not updating when it's changed. If I search for the document, it shows me the updated name, however if I press escape or backspace, it shows me the name it used to be.

+1

+1

same solution as @petermikitsh. once changed from onChange to onKeyUp, all is reactive now. this is a non-issue.

i had fun getting this to work in react. while it was very simple, it was tough to figure out as im still new to reactive programming. hope this helps someone:
  getInitialState() {
    return {
      name: "",
      items: []
    };
  },
  handleChange(e) {
    var name = e.target.value;

    this.setState({name: name});

    if (name.length >=3) ItemSearch.search(name);
  },
  handleKeyUp() {
    this.setState({items: ItemSearch.getData()});
  },

ok -- this seems to only work locally, but not with any latency involved. actually, if you watch state and add a record to the database, getData is NOT reactive. perhaps this can help simulate the root of the issue.

This is in fact a non-issue. I was not setup to listen to reactive data sources in React. Here is the correct config if anyone wants to see it:
...
  mixins: [ReactMeteorData],
  getMeteorData() {
    return {
      items: ItemSearch.getData()
    };
  },
  getInitialState() {
    return {
      name: ""
    }
  },
  handleChange(e) {
    var name = e.target.value;

    if (name.length >= 3) ItemSearch.search(name);

    this.setState({name: name});
  },
...

Then you listen to this.data.items instead of this.state.items.

+1
Data not really reactive or I use this package wrong.
I have a search result after page loaded and if add item in database it will not appear on the page. If use search field and type item name - it appear on the page.
For trigger search active I use keyup event.
Below are parts of my code.
template.js
Template.tpl.onRendered(function() {
    ExercisesSearch.search(' ');
});

Template.tpl.events({
    'keyup #searchExercises': _.throttle(function(event) {
         var text = $(event.target).val().trim();
         ExercisesSearch.search(text);
     }, 200)
});

search.js server
SearchSource.defineSource('exercises', function(searchText, options) {
     //var options = {sort: {isoScore: -1}, limit: 20};

     if(searchText) {
         var regExp = buildRegExp(searchText);
         //var selector = {exerciseName: regExp, createdBy: Meteor.userId(), isDeleted: false};
         var selector = {
             exerciseName: regExp,
             $or: [{
                 createdBy: Meteor.userId(),
                 isPrivate: true
             }, {
                 isPrivate: false
             }],
             $and: [{
                  isDeleted: false
             }]};
             return Exercises.find(selector, {sort: {isoScore: -1}, limit: 20}).fetch();
     } else {
         return Exercises.find({
             $or: [{
                 createdBy: Meteor.userId(),
                 isPrivate: true
             }, {
                 isPrivate: false
             }],
             $and: [{
                 isDeleted: false
             }]
         }).fetch();
     }
 });

search.js client
var options = {
     keepHistory: 1000 * 60 * 5,
     localSearch: true
 };
 var exerciseFields = ['exerciseName'];
 ExercisesSearch = new SearchSource('exercises', exerciseFields, options);

page.html

                    <input class="form-control" id="searchExercises" type="text" placeholder="Search..">

...

{{#each exercisesList}}

{{exerciseName}}

{{{substrVar exerciseDesc "500"}}}

{{/each}}

tpl_helper.js
Template.tpl.helpers({
exercisesList: function() {
         return ExercisesSearch.getData({
             transform: function(matchText, regExp) {
                 return matchText;
             },
             sort: {isoScore: -1}
         });
     }
});

I can repeat search query in browser console and "hidden" on the page item present in result.

Having the same issue here also, updated values in an entry on the db are not reactively rendered

+1

+1

"Having the same issue here also, updating values on an entry to the db are not reactively rendered" I am experiencing the same as you

Definitely is not a "real reactive" package. It only takes a "snapshot" from the results returned by the query... I hope in next update it will be solved because the package working great.

+1

Tried to implement one approach to this problem. Take a look #54 .

@bompi88 well patch, but the performance is an issue
+1

@bompi88 Thanks for sharing your patch.  I like your idea.  Also, your patch may clarify things for those contributing confusion by saying it's a "non-issue" etc.
Looking closely at your patch, it appears to me that while it does react to changes to documents in the search results, it doesn't react to documents which may have been inserted (added), 'externally'.  Can you please confirm that?

@steinitz That's true. If that should be supported, I think this package should be more module-based, mongodb and rest-based or something similar. Each module have separate implementation of the local store. I think this package tries to provide a way of searching dependent of which search engine you are using, and therefore the reactivity gets put on the sideline. Only mongodb searches would utilize this kind of reactivity, am I right?

@bompi88
Thanks for your thoughtful reply.  You wrote:

If that should be supported, I think this package should be more module-based, mongodb and rest-based or something similar.

Maybe so.  Rest-based might be a step backwards though.

I think this package tries to provide a way of searching dependent of which search engine you are using, and therefore the reactivity gets put on the sideline.

That might be true in theory, but, for example, getData() currently uses hard-coded Mongo selectors.

Only mongodb searches would utilize this kind of reactivity, am I right?

Good question.  I'll be interested to see how Meteor handles reactivity with other databases.
In any case, your clever solution brings us closer to having a truly reactive fast search.  I'd be interested to hear about any inspiration you might have about reacting to newly-created docs.
My only idea at present, a crude one, is a way to force SearchSource to 'start over' - MyCollectionSearchSource.reset(), maybe, which the app calls when it creates a new document.  Ugh.  You can see why I'm poking around for a real solution.
Thanks again for your thoughts and for sharing your work.


is there a bot.on("reaction_add")

is there something so i can wait until someone reacts to my message and then edit my message when someone reacted?

Yes, look in source code!

i tried finding it, i found bot.py def on()
but i cant find the what the coroutines for on() are.

check documentation

For once, mini amin is right!

nope


Block Blast blocks moved up

Device:
Acer r752t-c3m5 Chromebook
Issue:
Block Blast blocks are moved up from cursor, thus meaning bottom part of play area is not usable.
Steps to Reproduce:
Head to https://ultimateunblocked1.onrender.com/games/block-blast/index.html
Pick up a block
Block will be higher up than cursor
Comments:
I know that this is based on somebody else's project, but maybe something is broken with how it sees the position of the cursor on your side? Maybe finding another project could be the fix, not sure if this is an issue y'all can fix on your end.

This problem is present on every device I've tested it on. This is most likely a problem with the project that we got Block Blast from. I'll try to find another project without this issue.

Interestingly enough, this problem is not present when using a touchscreen. I fear that this is an issue with the HTML5 version of Block Blast and how it handles mouse input. This problem is present on every project I've tried, including futzumi/block-blast which is based on a different version of the game, and it's also present on projects outside of GitHub. I'm going to leave this issue open for a few more days in case I find a working version.

For me, this problem is present even when using a touchscreen, maybe different computers have different ways of handling touchscreen inputs?

That's definitely possible. I've tried on several mobile devices and the blocks land exactly where you drag them, and I've gotten a friend to try on their laptop touchscreen and the blocks work as intended.
Also, if you un-maximize the window and resize it smaller, the game becomes slightly more playable.

Works, but maybe we should update the Block Blast page to have the hotfix there? (until we can find a way to either get this version working or to find a new version)

Wanted to add on: this is much more playable on screens that have higher resolutions or are just bigger (for some reason?)
Tested on my Macbook Air M2 2022, works just fine even though blocks are still not lined up.

Works, but maybe we should update the Block Blast page to have the hotfix there?
That's a great idea, I'm going to fork the repository onto this organization.

this is much more playable on screens that have higher resolutions or are just bigger
I'm finding this to be true too


Wrong spelling of favourite

"You can develop in your favorite language, be it .NET, .NET Core, Java, Ruby, Node.js, PHP, or Python"

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 53dd65b1-6f9d-9431-5850-9cee636aeabd
Version Independent ID: 0012e819-bb2b-2955-d87b-d6f145fb6af7
Content: Overview - Azure App Service
Content Source: articles/app-service/overview.md
Service: app-service
GitHub Login: @cephalin
Microsoft Alias: cephalin

@brian-zhou Thanks for the feedback! We are currently investigating and will update you shortly.

@brian-zhou Thanks again for feedback! Both favourite and favorite is correct spelling. Favorite without the "u" is the spelling used in the United States.
We will now proceed to close this thread. If there are further questions regarding this matter, please let us know.


Why performance is inconsistent with that shown in paper?

It seems that after running Structack, the accuracy doesn't see significant drop. I reproduced the code with parameter "--model gcn --dataset cora". Accuracy of "clean" mode is 0.82+ while accuracy of "structack" mode is no less than 0.81. Is there any problem?

Hi,
I just realized that the perturbation rate was set to 0.01 for Structack. I've updated it to 0.05 as mentioned in the paper just now.
Could you check if the problem is solved, and close the issue if that's the case?
Best,
Hussain

Thanks for your reply, it has been solved.


Change Inner Text "Copied!" when click copy button

Hi! I'm Chilean and my native language is Spanish. How can I change text "Copied!" to "Copiado!".
Thanks

@Llank84 can you take a look at #7 before I merge to see if this resolves your issue?

@arronhunt it does not work😔. Thanks anyway!