Use code metric config files from Artifactory

This PR removes the configuration files for the code metrics tools PMD, FindBugs and CheckStyle.
Instead, gradle's new TextResource class is used to resolve and read the files from Artifactory.
This allows for reusing the config files across different projects, versioning and easier maintenance.

Refer to this link for build results (access rights to CI server needed):
http://jenkins.terasology.org/job/TerasologyPRs/32/
Build Log
last 5 lines
[...truncated 2020 lines...]
   at hudson.model.Executor.run(Executor.java:240)
[ANALYSIS-COLLECTOR] Skipping publisher since build result is FAILURE
Notifying upstream projects of job completion
Setting status of 97df66b215b023e5354bfe35da235ae49173bf81 to FAILURE with url http://jenkins.terasology.org/job/TerasologyPRs/32/ and message: Merged build finished.

Uh oh, something went wrong with the build. Need to check on that

Refer to this link for build results (access rights to CI server needed):
http://jenkins.terasology.org/job/TerasologyPRs/34/
Hooray Jenkins reported success with all tests good!

After spending a couple hours looking at this more closely I'm conflicted. The nerd in me wants to gleefully embrace the shiny new text resource, but its use seems a bit quirky.
Extracting the config file into each (sub)project's build dir, then cheating about the timing for that so a clean wouldn't hit before it gets used ... eep. Quirky Gradle looks quirky. Works though, and is still neat.
It also makes it trickier (though probably not unsolvable) to configure IDE analytics. Does it work like this in Eclipse without manual config? In IntelliJ for instance config/gradle/ide.gradle customizes the Checkstyle config to use the config/checkstyle stuff, which I guess now would be missing (did you not delete the actual checkstyle.xml there for some reason?)
I puzzled at it for a bit and realized there might be a far less geeky and way more boring way to do it. Here's a quick (well, okay, not so quick, am apparently not on top of my Gradle-fu tonight, this took way too long and could be done better) proof of concept:
https://github.com/MovingBlocks/Terasology/commit/b59211a208f1d7eed70c95389582be5c7056d59e
That task isn't tied to anything yet, and I don't know if it would hit similar timing issues. Could we add it as a dependency to the idea task and the code analytics config tasks? Would it need a doFirst? Etc
It is based off the extractNatives task, which admittedly is only needed at game runtime, so this might not work after all :/
In a similar vein I noticed Gradle 2.2 adds support for doing something like idea.project.cvs = "Git" instead of the hack we use in ide.gradle, but since that's a project level property it doesn't work for our nested Git roots so can't use that either :(
Going to take a break from the Gradle for now, let me know what you think. Maybe we can just copy via text resource into a single specific directory and configure to hit that both in Gradle land and via IDE? Rather than goofy unzip of dependency. Not sure either option is "better" - one is just geekier and new, me want :-)
In other news: It takes approximately 17 minutes to run "gradlew check" with near 70 modules in source form locally.

I really like your solution - I actually prefer it over mine, since it is consistent with the previous file structure and thus still supports IDE integration nicely.
A similar question was asked in the Gradle forums with the following answer:
task extractApi(type: Copy) {
    from { // use of closure defers evaluation until execution time
        configurations.api.collect { zipTree(it) }
    }
    into "$buildDir/api/"
}

I suggest making the checkStyle* tasks depend on this copy task. Not sure how to do it for all source sets. My solution sets it manually for main and test and once more in engine for dev.

Ah, so that's the right syntax! I was hunting for a clean collect/ziptree version but just couldn't find it last night. Thanks for digging that up.
I can try to hook it up tonight and see how IntelliJ and such deals with it.


Overlay media cannot be removed

Describe the bug
Added media does not being removed
To Reproduce
Steps to reproduce the behavior:

Add media overlay
Click on button to remove it

Expected behavior
Media overlay should be removed
Screenshots
If applicable, add screenshots to help explain your problem.
 I'll fix that, thanks.

Fixed!


dart2native does not produce deterministic results

With Dart VM version: 2.7.2 (Mon Mar 23 22:11:27 2020 +0100) on "windows_x64"
I'm getting non-deterministic results after running dart2native on the same (small) .dart file.
given this (and ignore if wrong, I'm still learning the language) example (boots.dart):
import 'dart:io';
import 'dart:core';

int main(List<String> args) {
  int exitCode = -1;
  if( args.isEmpty ) {
    print("Expected one or more arguments");
    return exitCode;
  }
  var stopWatch = Stopwatch()..start();
  var process = Process.start(args[0], args.sublist(1));
  process.then((process) {
    stdout.addStream(process.stdout);
    stderr.addStream(process.stderr);
    process.exitCode.then((processExitCode) {
        exitCode = processExitCode;
        var elapsedTime = stopWatch.elapsedMicroseconds;
        print("${elapsedTime/1e6}: $args");
    });
  });
  return exitCode;
}

I'm getting this:
D:\p\shinyBoots>dart2native boots.dart && md5sum boots.exe
Generated: d:\p\shinyboots\boots.exe
8392b3c328ff9cb9da26071df1e9ffed *boots.exe

D:\p\shinyBoots>dart2native boots.dart && md5sum boots.exe
Generated: d:\p\shinyboots\boots.exe
e44de3450ceac0c3b3db52c9cec16e14 *boots.exe

D:\p\shinyBoots>dart2native boots.dart && md5sum boots.exe
Generated: d:\p\shinyboots\boots.exe
6213497d0d167731fabc7dc2ea9880ec *boots.exe

Seems like related to this -> https://github.com/dart-lang/sdk/issues/33264

/cc @rmacnak-google

We suspect this issue has been fixed some time ago. I've verified on Linux that the above example compiles deterministically to the same executable.
So I assume this issue can be closed.
@malkia Please re-open if you still have this issue.


Data irregularities cause epacems_to_parquet to fail

After (apparently) successfully running the new data package based ETL process on the full EPA CEMS dataset (all years, all states), I tried to run the epacems_to_parquet script, but it encountered errors, and ultimately failed.  Several errors were of the type:
sys:1: DtypeWarning: Columns (8,10,12,14) have mixed types. Specify dtype option on import or set low_memory=False.

But the thing that crashed it eventually was:
Traceback (most recent call last):
  File "/home/zane/miniconda3/envs/pudl-dev/bin/epacems_to_parquet", line 11, in <module>
    load_entry_point('catalystcoop.pudl', 'console_scripts', 'epacems_to_parquet')()
  File "/home/zane/pudl/src/pudl/convert/epacems_to_parquet.py", line 319, in main
    clobber=args.clobber)
  File "/home/zane/pudl/src/pudl/convert/epacems_to_parquet.py", line 205, in epacems_to_parquet
    df = year_from_operating_datetime(df).astype(IN_DTYPES)
  File "/home/zane/pudl/src/pudl/convert/epacems_to_parquet.py", line 123, in year_from_operating_datetime
    df['year'] = df.operating_datetime_utc.dt.year
  File "/home/zane/miniconda3/envs/pudl-dev/lib/python3.7/site-packages/pandas/core/generic.py", line 5175, in __getattr__
    return object.__getattribute__(self, name)
  File "/home/zane/miniconda3/envs/pudl-dev/lib/python3.7/site-packages/pandas/core/accessor.py", line 175, in __get__
    accessor_obj = self._accessor(obj)
  File "/home/zane/miniconda3/envs/pudl-dev/lib/python3.7/site-packages/pandas/core/indexes/accessors.py", line 343, in __new__
    raise AttributeError("Can only use .dt accessor with datetimelike " "values")
AttributeError: Can only use .dt accessor with datetimelike values

Hey @roll, after we got off the call this morning I realized that this seems to be something happening in the datapackage.save() step. When we're trying to put all the data in the EPA CEMS "resource group" into a single table in SQLite, for some reason it only seems to be loading the last year of several years which are part of the data package.  It may or may not be loading all of the different states (I've only done the ETL locally trying to load all the years of Colorado data, but instead I only get the 2018 Colorado data in the database). When I was trying to load the full 25 year / 50 state dataset, it finished very quickly, which suggests to me that it wasn't even trying to load all those other years and/or states, and instead just did a small subset of the work.
Is there any obvious reason you can think of why this might happen?  Here's what one of the resource descriptors looks like:
        {
            "profile": "tabular-data-resource",
            "name": "hourly_emissions_epacems_2002_co",
            "path": "data/hourly_emissions_epacems_2002_co.csv.gz",
            "title": "hourly_emissions_epacems",
            "encoding": "utf-8",
            "mediatype": "text/csv",
            "format": "csv",
            "dialect": {
                "delimiter": ",",
                "header": true,
                "quoteChar": "\"",
                "doubleQuote": true,
                "lineTerminator": "\r\n",
                "skipInitialSpace": true,
                "caseSensitiveHeader": false
            },
            "schema": {
                "fields": [
                    {
                        "name": "state",
                        "type": "string",
                        "constraints": {
                            "enum": [
                                "AL",
                                "AR",
                                "AZ",
                                "CA",
                                "CO",
                                "CT",
                                "DC",
                                "DE",
                                "FL",
                                "GA",
                                "IA",
                                "ID",
                                "IL",
                                "IN",
                                "KS",
                                "KY",
                                "LA",
                                "MA",
                                "MD",
                                "ME",
                                "MI",
                                "MN",
                                "MO",
                                "MS",
                                "MT",
                                "NC",
                                "ND",
                                "NE",
                                "NH",
                                "NJ",
                                "NM",
                                "NV",
                                "NY",
                                "OH",
                                "OK",
                                "OR",
                                "PA",
                                "RI",
                                "SC",
                                "SD",
                                "TN",
                                "TX",
                                "UT",
                                "VA",
                                "VT",
                                "WA",
                                "WI",
                                "WV",
                                "WY"
                            ]
                        },
                        "description": "State the plant is located in.",
                        "format": "default"
                    },
                    {
                        "name": "plant_id_eia",
                        "type": "integer",
                        "description": "The unique six-digit facility identification number, also called an ORISPL, assigned by the Energy Information Administration.",
                        "format": "default"
                    },
                    {
                        "name": "unitid",
                        "type": "string",
                        "description": "Facility-specific unit id (e.g. Unit 4)",
                        "format": "default"
                    },
                    {
                        "name": "operating_datetime_utc",
                        "type": "datetime",
                        "description": "Date and time measurement began (UTC).",
                        "format": "default"
                    },
                    {
                        "name": "operating_time_hours",
                        "type": "number",
                        "description": "Length of time interval measured.",
                        "format": "default"
                    },
                    {
                        "name": "gross_load_mw",
                        "type": "number",
                        "description": "Average power in megawatts produced during time interval measured.",
                        "format": "default"
                    },
                    {
                        "name": "steam_load_1000_lbs",
                        "type": "number",
                        "description": "Total steam pressure produced by a unit or source in any calendar year (or other specified time period) produced by combusting a given heat input of fuel.",
                        "format": "default"
                    },
                    {
                        "name": "so2_mass_lbs",
                        "type": "number",
                        "description": "Sulfur dioxide emissions in pounds.",
                        "format": "default"
                    },
                    {
                        "name": "so2_mass_measurement_code",
                        "type": "string",
                        "constraints": {
                            "enum": [
                                "LME",
                                "Measured",
                                "Measured and Substitute",
                                "Other",
                                "Substitute",
                                "Undetermined",
                                "Unknown Code",
                                ""
                            ]
                        },
                        "description": "Identifies whether the reported value of emissions was measured, calculated, or measured and substitute.",
                        "format": "default"
                    },
                    {
                        "name": "nox_rate_lbs_mmbtu",
                        "type": "number",
                        "description": "The average rate at which NOx was emitted during a given time period.",
                        "format": "default"
                    },
                    {
                        "name": "nox_rate_measurement_code",
                        "type": "string",
                        "constraints": {
                            "enum": [
                                "Calculated",
                                "LME",
                                "Measured",
                                "Measured and Substitute",
                                "Not Applicable",
                                "Other",
                                "Substitute",
                                "Undetermined",
                                "Unknown Code",
                                ""
                            ]
                        },
                        "description": "Identifies whether the reported value of emissions was measured, calculated, or measured and substitute.",
                        "format": "default"
                    },
                    {
                        "name": "nox_mass_lbs",
                        "type": "number",
                        "description": "NOx emissions in pounds.",
                        "format": "default"
                    },
                    {
                        "name": "nox_mass_measurement_code",
                        "type": "string",
                        "constraints": {
                            "enum": [
                                "Calculated",
                                "LME",
                                "Measured",
                                "Measured and Substitute",
                                "Not Applicable",
                                "Other",
                                "Substitute",
                                "Undetermined",
                                "Unknown Code",
                                ""
                            ]
                        },
                        "description": "Identifies whether the reported value of emissions was measured, calculated, or measured and substitute.",
                        "format": "default"
                    },
                    {
                        "name": "co2_mass_tons",
                        "type": "number",
                        "description": "Carbon dioxide emissions in short tons.",
                        "format": "default"
                    },
                    {
                        "name": "co2_mass_measurement_code",
                        "type": "string",
                        "constraints": {
                            "enum": [
                                "LME",
                                "Measured",
                                "Measured and Substitute",
                                "Other",
                                "Substitute",
                                "Undetermined",
                                "Unknown Code",
                                ""
                            ]
                        },
                        "description": "Identifies whether the reported value of emissions was measured, calculated, or measured and substitute.",
                        "format": "default"
                    },
                    {
                        "name": "heat_content_mmbtu",
                        "type": "number",
                        "description": "The energy contained in fuel burned, measured in million BTU.",
                        "format": "default"
                    },
                    {
                        "name": "facility_id",
                        "type": "integer",
                        "description": "Plant ID, assigned by the Energy Information Administration.",
                        "format": "default"
                    },
                    {
                        "name": "unit_id_epa",
                        "type": "integer",
                        "description": "Unit ID for each unit at a facility, assigned by the Energy Information Administration.",
                        "format": "default"
                    }
                ],
                "missingValues": [
                    ""
                ]
            },
            "licenses": [
                {
                    "name": "CC-BY-4.0",
                    "title": "Creative Commons Attribution 4.0",
                    "path": "https://creativecommons.org/licenses/by/4.0/"
                }
            ],
            "sources": [
                {
                    "title": "Environmental Protection Agency Continuous Emission Monitoring Systems",
                    "path": "ftp://newftp.epa.gov/dmdnload/emissions/hourly/monthly",
                    "source_code": "epacems",
                    "parameters_pudl": {
                        "epacems_years": [
                            1995,
                            1996,
                            1997,
                            1998,
                            1999,
                            2000,
                            2001,
                            2002,
                            2003,
                            2004,
                            2005,
                            2006,
                            2007,
                            2008,
                            2009,
                            2010,
                            2011,
                            2012,
                            2013,
                            2014,
                            2015,
                            2016,
                            2017,
                            2018
                        ],
                        "epacems_states": [
                            "CO"
                        ],
                        "partition": {
                            "hourly_emissions_epacems": [
                                "epacems_years",
                                "epacems_states"
                            ]
                        }
                    }
                }
            ],
            "bytes": 5551312,
            "hash": "sha256:5402de2870c8833947680b86e41f42797a58c9170b846ffc6d8aff69be4fbdd4",
            "created": "2019-10-12T05:04:39Z",
            "start_date": "1995-01-01",
            "end_date": "2018-21-31",
            "group": "hourly_emissions_epacems"
        },

Hey @karldw have you run into any of these data issues with the EPA CEMS to parquet conversions? Seems like it doesn't affect all of the years/states, only some.  Do the errors above ring a bell?

Hmm, I didn't have this problem when the path was zipfile → parquet (that is, before #385), but I haven't tried the zipfile → datapackage → parquet path yet.

I tried the zipfile → datapackage → parquet path, using the development version 73c9dc2652d59e3741a9a0d64fbea1593bda0f99. I get the same error you do  for the state-years that have no data (e.g. Arkansas in 1995).
I get a different error in other cases:
$ epacems_to_parquet --pkg_dir datapackage/pudl-epacems/epacems --year 2018 -c
2019-10-18 10:45:45 [    INFO] pudl:302 Checking for required EPA CEMS input files...
2019-10-18 10:45:48 [    INFO] pudl.convert.epacems_to_parquet:203 Converted 854160 records for 2018 and al.
2019-10-18 10:45:50 [    INFO] pudl.convert.epacems_to_parquet:203 Converted 359160 records for 2018 and ar.
2019-10-18 10:45:52 [    INFO] pudl.convert.epacems_to_parquet:203 Converted 779640 records for 2018 and az.
2019-10-18 10:45:59 [    INFO] pudl.convert.epacems_to_parquet:203 Converted 2165976 records for 2018 and ca.
2019-10-18 10:46:02 [    INFO] pudl.convert.epacems_to_parquet:203 Converted 621960 records for 2018 and co.
2019-10-18 10:46:04 [    INFO] pudl.convert.epacems_to_parquet:203 Converted 530328 records for 2018 and ct.
2019-10-18 10:46:05 [    INFO] pudl.convert.epacems_to_parquet:203 Converted 26280 records for 2018 and dc.
Traceback (most recent call last):
  File ".../bin/epacems_to_parquet", line 11, in <module>
    load_entry_point('catalystcoop.pudl', 'console_scripts', 'epacems_to_parquet')()
  File ".../src/pudl/convert/epacems_to_parquet.py", line 319, in main
    clobber=args.clobber)
  File ".../src/pudl/convert/epacems_to_parquet.py", line 208, in epacems_to_parquet
    df, preserve_index=False, schema=schema),
  File "pyarrow/table.pxi", line 1065, in pyarrow.lib.Table.from_pandas
  File "pyarrow/table.pxi", line 1093, in pyarrow.lib.Table.from_arrays
  File "pyarrow/table.pxi", line 452, in pyarrow.lib._sanitize_arrays
  File "pyarrow/array.pxi", line 233, in pyarrow.lib.asarray
  File "pyarrow/array.pxi", line 570, in pyarrow.lib.Array.cast
  File "pyarrow/error.pxi", line 86, in pyarrow.lib.check_status
pyarrow.lib.ArrowNotImplementedError: No cast implemented from dictionary<values=double, indices=int8, ordered=0> to dictionary<values=string, indices=int8, ordered=0>

This one is also new to me, but looks like a pyarrow schema issue.

By the "same error" do you mean both the Dtype warning and the datetime failure? Are those both related to year/state combos with no data?  (in which case I guess we just need to check whether the data exists for that year before attempting the conversion?).
What makes you think it's a schema issue, rather than a data issue -- i.e. are there some strings in there that need to be converted to floats, but they aren't convertible for some reason (leading zero, junk characters, etc.).
@cmgosnell just created dictionaries of data type by column for the EIA data, which explicitly cast every field to the required type after it's been cleaned up and before it's loaded into the CSVs.  Maybe doing something similar for the CEMS before it's written out would help? How can we get some specific examples of the records with apparent type mismatches that are breaking things?

I haven't seen the dtype warning. For the combination of states and years I've run, I hit one of the two errors first.
I think the schema issue is coming up for DC in 2018 because the so2_mass_measurement_code and co2_mass_measurement_code are all NaN. In the explicit schema epacems_to_parquet provides, these are (correctly) encoded as dictionaries of strings.
Debugging, omitting some output:
$ python3 -m pdb $(which epacems_to_parquet) --pkg_dir datapackage/pudl-epacems/epacems --year 2017 -c
(Pdb++) continue
...
Traceback (most recent call last):
pyarrow.lib.ArrowNotImplementedError: No cast implemented from dictionary<values=double, indices=int8, ordered=0> to dictionary<values=string, indices=int8, ordered=0>
Uncaught exception. Entering post mortem debugging
(Pdb++) up 6
(Pdb++) schema
year: int32 not null
state: dictionary<values=string, indices=int8, ordered=0>
plant_id_eia: int32 not null
unitid: string not null
operating_datetime_utc: timestamp[ms, tz=utc] not null
operating_time_hours: float
gross_load_mw: float not null
steam_load_1000_lbs: float
so2_mass_lbs: float
so2_mass_measurement_code: dictionary<values=string, indices=int8, ordered=0>
nox_rate_lbs_mmbtu: float
nox_rate_measurement_code: dictionary<values=string, indices=int8, ordered=0>
nox_mass_lbs: float
nox_mass_measurement_code: dictionary<values=string, indices=int8, ordered=0>
co2_mass_tons: float
co2_mass_measurement_code: dictionary<values=string, indices=int8, ordered=0>
heat_content_mmbtu: float not null
facility_id: int32
unit_id_epa: int32
(Pdb++) df.dtypes
state                                   category
plant_id_eia                               int64
unitid                                    object
operating_datetime_utc       datetime64[ns, UTC]
operating_time_hours                     float64
gross_load_mw                            float64
steam_load_1000_lbs                      float64
so2_mass_lbs                             float64
so2_mass_measurement_code               category
nox_rate_lbs_mmbtu                       float64
nox_rate_measurement_code               category
nox_mass_lbs                             float64
nox_mass_measurement_code               category
co2_mass_tons                            float64
co2_mass_measurement_code               category
heat_content_mmbtu                       float64
facility_id                              float32
unit_id_epa                              float32
year                                       int64

(Pdb++) import pyarrow
(Pdb++) pyarrow.lib.Table.from_pandas(df, schema=schema)
*** pyarrow.lib.ArrowNotImplementedError: No cast implemented from dictionary<values=double, indices=int8, ordered=0> to dictionary<values=string, indices=int8, ordered=0>
(Pdb++) dummy_cat = pd.CategoricalDtype(categories=['a'], ordered=False)
(Pdb++) df2 = df.astype({'so2_mass_measurement_code': dummy_cat, 'co2_mass_measurement_code': dummy_cat})
(Pdb++) pyarrow.lib.Table.from_pandas(df2, schema=schema) # now it works

It seems like pyarrow is unhappy converting a zero-category categorical column into a string categorical column. This is something pyarrow should handle, but since it doesn't (as of v0.15.0), we can get around the problem with more extensive IN_DTYPES.
Here's one (untested) option:
IN_DTYPES = {
    "co2_mass_measurement_code": pd.CategoricalDtype(
        categories=[
            "LME",
            "Measured",
            "Measured and Substitute",
            "Other",
            "Substitute",
            "Undetermined",
            "Unknown Code",
            "",
        ],
        ordered=False,
    ),
    "nox_mass_measurement_code": pd.CategoricalDtype(
        categories=[
            "Calculated",
            "LME",
            "Measured",
            "Measured and Substitute",
            "Not Applicable",
            "Other",
            "Substitute",
            "Undetermined",
            "Unknown Code",
            "",
        ],
        ordered=False,
    ),
    "nox_rate_measurement_code": pd.CategoricalDtype(
        categories=[
            "Calculated",
            "LME",
            "Measured",
            "Measured and Substitute",
            "Not Applicable",
            "Other",
            "Substitute",
            "Undetermined",
            "Unknown Code",
            "",
        ],
        ordered=False,
    ),
    "so2_mass_measurement_code": pd.CategoricalDtype(
        categories=[
            "LME",
            "Measured",
            "Measured and Substitute",
            "Other",
            "Substitute",
            "Undetermined",
            "Unknown Code",
            "",
        ],
        ordered=False,
    ),
    # We know state is never missing, so we can use default category inference
    "state": "category",
    "unitid": "str",
    # Note: it'd be better to use pandas' nullable integers once this issue is
    # resolved: https://issues.apache.org/jira/browse/ARROW-5379
    # 'facility_id': "Int32",
    # 'unit_id_epa': "Int32",
    "facility_id": "float32",
    "unit_id_epa": "float32",
}

I wrote some code that should address this, but I'll test it later this week before creating a PR.

Thank you for looking into all this @karldw!  I'm looking forward to finally playing around with the CEMS data soon and getting familiar with dask.

@zaneselvans
Is it something that's not yet confirmed to be a problem with Frictonless libs?

@roll I don't think we know whether this would be an issue in the Frictonless libs -- right now the compressed CSV files are being read directly using pands.read_csv which isn't aware of the schema which is specified by datapackage.json When one reads a pandas DataFrame from a datapackge, does it automatically set the column data types based on the schema?  If so, and we can tell it to only read in a subset of the resources which are part of the big epacems_hourly table, then maybe reading the data into pandas using datapackage, before writing it out to parquet would get around this?

@zaneselvans
If using tableschema-pandas-py it should translate data types

@roll thanks! we are not right now, but this is obviously what we should be doing. I'll spend some time converting this script to using it and see if that solves this problem

@roll poking around at the push_datapackage going off of this example like this:
datapkg_path = str(pathlib.Path(pudl_settings['datapackage_dir'], 'pudl_eia_only/eia-test/datapackage.json'))
storage = datapackage.push_datapackage(datapkg_path, 'pandas')

or even the example and I was getting this error:
~/miniconda3/envs/pudl-dev/lib/python3.7/site-packages/datapackage/pushpull.py in push_datapackage(descriptor, backend, **backend_options)
     49
     50     # Get storage
---> 51     plugin = import_module('jsontableschema.plugins.%s' % backend)
     52     storage = plugin.Storage(**backend_options)
     53

~/miniconda3/envs/pudl-dev/lib/python3.7/importlib/__init__.py in import_module(name, package)
    125                 break
    126             level += 1
--> 127     return _bootstrap._gcd_import(name[level:], package, level)
    128
    129
ModuleNotFoundError: No module named 'jsontableschema'

I have tableschema-pandas version 1.0.0 installed.
(pudl-dev) %0 ~/code/pudl/devtools% conda list tableschema-pandas                                                             christinagosnell@New-Shiny-Thing

# packages in environment at /Users/christinagosnell/miniconda3/envs/pudl-dev:
#
# Name                    Version                   Build  Channel
tableschema-pandas        1.0.0                    pypi_0    pypi

So I tried pip install jsontableschema, which then got me to this error with both the example package and the pudl package:
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-11-741ad89e97a3> in <module>
      1 data_url = 'http://data.okfn.org/data/core/country-list/datapackage.json'
----> 2 storage = datapackage.push_datapackage(data_url, 'pandas')

~/miniconda3/envs/pudl-dev/lib/python3.7/site-packages/datapackage/pushpull.py in push_datapackage(descriptor, backend, **backend_options)
     46
     47     # Init model
---> 48     model = Package(descriptor)
     49
     50     # Get storage

~/miniconda3/envs/pudl-dev/lib/python3.7/site-packages/datapackage/package.py in __init__(self, descriptor, base_path, strict, storage, schema, default_base_path, **options)
    104
    105         # Build package
--> 106         self.__build()
    107
    108     def __del__(self):

~/miniconda3/envs/pudl-dev/lib/python3.7/site-packages/datapackage/package.py in __build(self)
    316
    317         # Instantiate profile
--> 318         self.__profile = Profile(self.__current_descriptor.get('profile'))
    319
    320         # Validate descriptor

~/miniconda3/envs/pudl-dev/lib/python3.7/site-packages/datapackage/profile.py in __init__(self, profile)
     27         self._registry = self._load_registry()
     28         self._schema = self._load_schema(profile, self._registry)
---> 29         self._validator = self._load_validator(self._schema, self._registry)
     30         self._check_schema()
     31

~/miniconda3/envs/pudl-dev/lib/python3.7/site-packages/datapackage/profile.py in _load_validator(self, schema, registry)
    104
    105     def _load_validator(self, schema, registry):
--> 106         validator_class = jsonschema.validators.validator_for(schema)
    107         return validator_class(schema)
    108

AttributeError: module 'jsonschema' has no attribute 'validators'

@cmgosnell
Sorry for the confusion but the example is out-dated. It works the same way as all other drivers:
https://github.com/frictionlessdata/testsuite-extended/blob/master/tests/test_datapackage.py#L59

Hey @roll. thanks for the pointer! this got me through the Package() definition step.. and sorry to keep bugging you with errors, but now on package.save(storage=storage) I'm getting this error:
StorageError: Multi-column primary keys are not supported

we are definitely using multi-column PKs in many of our tables. Do you know if that is this an expected error for this type of application or am I doing something strange?
The full error is here:
---------------------------------------------------------------------------
StorageError                              Traceback (most recent call last)
<ipython-input-14-ba9d352b9895> in <module>
      6                                'pudl_cems',
      7                                'pudl-all', 'datapackage.json'))))
----> 8 package.save(storage=storage)
      9
     10 # Load

~/miniconda3/envs/pudl-dev/lib/python3.7/site-packages/datapackage/package.py in save(self, target, storage, merge_groups, **options)
    269             for bucket in storage.buckets:
    270                 source = sources[buckets.index(bucket)]
--> 271                 storage.write(bucket, source())
    272
    273         # Save descriptor to json

~/miniconda3/envs/pudl-dev/lib/python3.7/site-packages/tableschema_pandas/storage.py in write(self, bucket, rows)
    147         # Prepare
    148         descriptor = self.describe(bucket)
--> 149         new_data_frame = self.__mapper.convert_descriptor_and_rows(descriptor, rows)
    150
    151         # Just set new DataFrame if current is empty

~/miniconda3/envs/pudl-dev/lib/python3.7/site-packages/tableschema_pandas/mapper.py in convert_descriptor_and_rows(self, descriptor, rows)
     38         elif len(schema.primary_key) > 1:
     39             message = 'Multi-column primary keys are not supported'
---> 40             raise tableschema.exceptions.StorageError(message)
     41
     42         # Get data/index

StorageError: Multi-column primary keys are not supported

@cmgosnell
Please try<EMAIL_ADDRESS>I have added support for composite primary keys


Version 2.1.0 does not identify images

I used to work with previous versions of retina display (v1.3), I just had to install the script and run it, every image was detected and the scripts tried to find the @2x version to replace it.
Now I am creating a new website and after loading the script, nothing happens. well reading carefully I found now the user has to add a data-rjs="3" to be able to handle the retina image but after doing it as in the examples shown it doesn't work for me.
I had to go back to v1.3 to finish my project, but I want to know if there is something wrong with the procedure, maybe something is not clear enough for me, or if maybe there is a kind of jQuery conflict.

Same issue, couldn't get 2.1.0 to work at all.

There is not a dependency on jQuery in order to use Retinajs. Include the script either in your footer or bundled with your other assets. Mark your images that you want Retinajs to swap out with data-rjs=2 or data-rjs=3 and go. This is assuming you're using the standard @2x or @3x suffixes on your images.
ie  and if you have an image in<EMAIL_ADDRESS>retinajs will swap it.

I came across retinajs this morning and gave v2.1.0 a try. Like @chrisgrande and @whgandalf, I couldn't get it to work. I then tried v2.0.0 and it worked.

Same here, v2.1.0 did not work with the rjs=2 method, but v2.0.0 did. Thanks @DevFountain .

+1

Same problem with 2.1.0. Had to switch to 2.0.0

I also had to downgrade to 2.0.0 for it to work.

Same. 2.0.0 works but not 2.1.0
In case the additional information is useful, I am building a site Wordpress, and was using the "traditional" method of including the script.

I was having trouble with 2.1.0 (after switching from 1.*), but managed to get it working by:

Adding data-rjs="2" to my image tags
Running window.retinajs(); at the very end of the page

Hope this helps :)


Workspace mode not finding package dependency

I'm trying to port a project to UV but I can't get it to work. The project is basically the described use case in the workspaces docs: a core library, and a FastAPI server depending on it, in another library project in the same repo.
Goal: be able to run commands like uv run python3 -m pytest -q . from the root (works great in conda, can't find the workspace dependency with UV)
I've included snippets of the files in question below, but the full setup is in this branch: https://github.com/Kiln-AI/Kiln/tree/uv
Note: it works fine when I'm not using workspace mode, and just have 2 independent projects with a local path dependency (can run the tests of each independently). When I move it to workspace mode, I'm getting ModuleNotFoundError: No module named 'kiln_ai' (where kiln_ai is the workspace project dependency).
Root pyproject.toml
[project]
name = "kiln-root"
version = "0.1.0"
description = "uv workspace project for Kiln AI. See kiln-ai package on pypi for our librabrary (libs/core)"
readme = "README.md"
requires-python = ">=3.9"
dependencies = [
    "kiln-ai",
]

[tool.uv.workspace]
members = ["libs/core", "libs/server"]

[tool.uv.sources]
kiln-ai = { workspace = true }

The dependency project: /libs/core/pyproject.toml - https://github.com/Kiln-AI/Kiln/blob/uv/libs/core/pyproject.toml
The dependent project:  /libs/server/pyproject.toml - https://github.com/Kiln-AI/Kiln/blob/uv/libs/server/pyproject.toml
# Only relevant sections in the issue, full file here: https://github.com/Kiln-AI/Kiln/blob/uv/libs/server/pyproject.toml

[tool.uv.sources]
# This works fine, if we don't use UV workspace mode (top level pyproject.toml), but errors if we do.
# kiln-ai = { path = "../core" }

# This returns "ModuleNotFoundError: No module named 'kiln_ai'" in workspace mode, if included here, or not included at all.
kiln-ai = { workspace = true }

Version Info
uv --version
uv 0.4.28 (debe67ffd 2024-10-28)

Command
# /libs/server or root have same issue. the same issue happens for tasks that don't need the central conftest.py like just running the server.
uv run python3 -m pytest -q .
ImportError while loading conftest 'xx/kiln/conftest.py'.
../../conftest.py:5: in <module>
    from kiln_ai.utils.config import Config
E   ModuleNotFoundError: No module named 'kiln_ai'

@zanieb  that's here: https://github.com/Kiln-AI/Kiln/blob/uv/libs/core/pyproject.toml
Pretty stock I think. Full files in the links above.
Thanks!

Sorry I see it at https://github.com/Kiln-AI/Kiln/blob/uv/libs/core/pyproject.toml now
You'll want to mark the project as "packaged" by defining a build system https://docs.astral.sh/uv/concepts/projects/#build-systems

Yeah we should definitely add some sort of warning or hint here.


feat: background start for connector plugins

There seems to be flaky test I introduced - will fix that

Codecov Report

Merging #1315 (e63dc71) into main (56b8d2f) will not change coverage.
The diff coverage is 100.00%.

@@            Coverage Diff            @@
##              main     #1315   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files          310       310
  Lines        20960     21040   +80
=========================================
+ Hits         20960     21040   +80

Impacted Files
Coverage Δ

internal/blockchain/ethereum/config.go
100.00% <100.00%> (ø)

internal/blockchain/ethereum/ethereum.go
100.00% <100.00%> (ø)

internal/blockchain/ethereum/eventstream.go
100.00% <100.00%> (ø)

internal/blockchain/fabric/config.go
100.00% <100.00%> (ø)

internal/blockchain/fabric/eventstream.go
100.00% <100.00%> (ø)

internal/blockchain/fabric/fabric.go
100.00% <100.00%> (ø)

internal/dataexchange/ffdx/config.go
100.00% <100.00%> (ø)

internal/dataexchange/ffdx/ffdx.go
100.00% <100.00%> (ø)

internal/namespace/manager.go
100.00% <100.00%> (ø)

internal/tokens/fftokens/config.go
100.00% <100.00%> (ø)

... and 1 more


Order rule payload by rule_condition position

1. Why is this change necessary?
At some point, rules get quite complex. To have an "early return" the position of a condition can be useful. In the administration, a user can add a condition before or after another. The indexer don't care and will add rules random to the payload.
Example
Adminitration order:
And-Rule with

SalesChannel check
Or-Condition with many sub-conditions

Order in Rule payload

SalesChannel check
Or-Condition with many sub-conditions

OR

Or-Condition with many sub-conditions
SalesChannel check

In this example the Or-Condition doesn't need to be validated, if the sales channel is not correct
Without this order by, it's not clear, which condition is checked first.
2. What does this change do, exactly?
Adds a second order parameter for rule payload generation
3. Describe each step to reproduce the issue or behaviour.
Create a rule with

And-Rule
First small condition (e.g. SalesChannel check, Customer is logged in)
Second computationally intensive conditions (e.g. complex delivery address conditions)

4. Please link to the relevant issues (if any).
5. Checklist

[x] I have rebased my changes to remove merge conflicts
[ ] I have written tests and verified that they fail without my change
[x] I have created a changelog file with all necessary information about my changes
[ ] I have written or adjusted the documentation according to my changes
[ ] This change has comments for package types, values, functions, and non-obvious lines of code
[x] I have read the contribution requirements and fulfil them.

@shopware/st-business-ops can you take a look at this one?

Hello,
thank you for creating this pull request.
I have opened an issue on our Issue Tracker for you. See the issue link: https://issues.shopware.com/issues/NEXT-25782
Please use this issue to track the state of your pull request.

Thank you for your contribution! 🎉


Notero 0.4.10 does not work on MacOS

The new update for Notero which is v0.4.10 does not work on MacOS.
When I updated it, the Notero Preferences under 'Tools' disappeared and it stopped syncing.
My system:
Apple MBPro M2 Max
MacOS Ventura 13.5 with Zotero 6.0.26
Note: everything was fixed when I rolled back to v0.4.9 and I am submitting this issue just to flag it for consideration.

I have the same issue on Windows.

I have the same issue on Windows

I am having the same issue, Mac OS Ventura 13.4.1, Zotero 6.0.26

Same for me. Please fix this!!

Same here, Mac OS Ventura 13.4.1, Zotero 6.0.26

Hi y'all, sorry about this! Thanks for reporting it.
My latest upgrade of dependencies introduced a change that ended up crashing Notero as it started up. (See #317 if you're curious.)
This should be fixed in version 0.4.11.


ER Diagram background using Darkest Dark theme

System information:

Operating system (distribution) and version: Mageia 7 x86_64
DBeaver version: 6.1.4 CE
Additional extensions: Darkest Dark theme

Describe the problem you're observing:
When i generate ER Diagram, background is black but, if i click on each other place in DBeaver, background change to white, also if i try to export an image for example.

to restore background color to black i need to use Refresh Diagram button

still reproducible in 7.1.2

Related request - it would be nice to have the option to export with a white background and white tables even when in dark mode, so that you can get a nice printable image without changing color themes for the whole application.

Still reproducible in 7.1.2

Still reproducible in 7.1.2

not reproducible against 21.0.3

Still reproducible in 21.0.3

Still reproducible in 21.2.4

Still reproducible in 21.2.4

22.1.0 still reproducible

<IP_ADDRESS>303040621 - Still reproducible


Build Reports

Critical Changes
Great changes again
Changes
Issues Closed

This functionality was added to the Master branch in a different pull request, so closing this one as no longer needed

This functionality was added to the Master branch in a different pull request, so closing this one as no longer needed


UTF8 support?

a little related to #16 - I tried typing in some double byte characters to the telnet client, and the data was mangled. Since my terminal is setup for unicode, some googling said this is a problem at socket setup, in that what I am seeing as i type input is being echoed back from the server?
I added this line line but no effect so wonder if its something deeper in the engine that is locked to ascii?
  var newSocket = function (socket) {
    socket.setEncoding('utf8');   // no effect

eg typing in 日本語 shows like:

This is now fixed. I removed double byte characters in at the normalizer layer.

For reference - https://github.com/silentrob/normalizer/blob/master/test/basic.js#L63-L67


XOR fails if RUN_KLAYOUT_STREAMOUT is set to 0

Issue:
XOR fails if RUN_KLAYOUT_STREAMOUT is set to 0 and the flow fails.
Suggestion:
XOR should not run if RUN_KLAYOUT_STREAMOUT is set to 0.

No. This is a misconfigured flow on the part of the designer using a step that requires a KLayout GDS input without generating a KLayout GDS output earlier in the step.

I understand; but this wasn't the case in OpenLane 1. XOR was skipped in OpenLane 1 when Klayout GDS is not generated.

I'm very comfortable breaking compatibility here. Adding a billion exceptions to solve a minor problem is not why I started this rewrite. :slightly_smiling_face:


Allow empty protected branch path

gitolite complains once per involved user when you set a R rule for a path different from the empty one, because if you can read you can read everything.
My log is full thousands of messages like the following
....
2017-05-25.16:22:27     12200   warn    doesn't make sense to supply a ref ('refs/heads/.*') for 'R' rule
2017-05-25.16:22:27     12200   warn    doesn't make sense to supply a ref ('refs/heads/.*') for 'R' rule
2017-05-25.16:22:27     12200   warn    doesn't make sense to supply a ref ('refs/heads/.*') for 'R' rule
2017-05-25.16:22:27     12200   warn    doesn't make sense to supply a ref ('refs/heads/.*') for 'R' rule
2017-05-25.16:22:27     12200   warn    doesn't make sense to supply a ref ('refs/heads/.*') for 'R' rule
2017-05-25.16:22:27     12200   warn    doesn't make sense to supply a ref ('refs/heads/.*') for 'R' rule
2017-05-25.16:22:27     12200   warn    doesn't make sense to supply a ref ('refs/heads/.*') for 'R' rule
2017-05-25.16:22:27     12200   warn    doesn't make sense to supply a ref ('refs/heads/.*') for 'R' rule
2017-05-25.16:22:27     12200   warn    doesn't make sense to supply a ref ('refs/heads/.*') for 'R' rule
...

I've tried setting the path to .* but it is not sufficient.  The only way to silence it is to set an empty path, which was not allowed.
With this patch you can set an empty path that will match everything.

This old PR is very unlikely to be merged as is. I'll close it for now. If you feel it is still relevant please either rebase on master or create a new PR.


[docs] How to use Schemabuilder

Is there any documentation on how to use the schemabuilder?
I'm trying to understand the orchardcore code and how it works.
For example: https://github.com/OrchardCMS/OrchardCore/blob/32612bef1f017db7961b0fb6ad28a3a61f260b9a/src/OrchardCore.Modules/OrchardCore.ContentFields/Indexing/SQL/Migrations.cs
Why the need for AlterIndexTable after calling CreateMapIndex?
From looking at the source code, isn't AlterIndexTable table called already in the CreateMapIndex function?
The documentation in the wiki only shows how to use IndexProvider<>.

SchemaBuilder allow you to apply DDL in your database, in Orchard Core, the SchemaBuilder is injected into a Migration so you can use a property to manipulate the database schema

Yes I understand that, but why the double call to altertableIndex?

@deanmarcussen any idea?

The CreateMapIndex call makes the table, the subsequent AlterIndexTable calls create sql key indexes (not related to YesSql indexes) on the columns in the table.

Thanks @deanmarcussen  and @hishamco


Add wikibase:1.31

Bug: T197212

The reason it fails is that the bundle can't find the base 1.31 image because it's not yet pushed to docker hub. I can split this PR to make sure first the base gets merged and then the bundle one.

Hmm,, but the first image should be tagged with wikibase/wikibase:1.31 when built? No?

I think as @addshore suggested we should probably remove all the --pulls


[MNT] revert recent CI changes

Update: CI does not fail on this, compared to current PR where almost always sth fails. This indicates that we have probably identified a change condition, even if not the root cause.
This indicates that revert will fix the issues, until the point where we understand the root cause better.

This is a diagnostic PR to investigate https://github.com/alan-turing-institute/sktime/issues/3103
Purpose: see whether the tests complete when these recent changes to CI are reverted:
https://github.com/alan-turing-institute/sktime/pull/2896
https://github.com/alan-turing-institute/sktime/pull/3107
If the theory is correct thatsaid changes are the culprit, this should pass. Otherwise, it would fail.

restarting for good measure, see if it breaks on a second run

1st and 2nd time: win and unix runs
1st time: mac breaks with same issue

Has served its purpose - we know that the CI changes are not at fault since reverting them leaves us with the same issues.


Please check all of the platforms you are having the issue on (if platform is not listed, it is not supported)

[x] iOS
[ ] Android
[ ] UWP
[ ] .NET Standard

Version of Library
6.2.6 (vanilla, not Mvvmcross)
Version of OS(s) listed above with issue
iOS 13
Steps to Reproduce

Start a scan in a APP in debug mode
Close abruptly trough the IDE (Visual studio for mac)
Restart

Expected Behavior
The scan start without problem
Actual Behavior
The library crash with "there is already another scan"
I start the scanner in the MainViewController inside ViewwillAppear with ScanExtra.
My code
if(CrossBleAdapter.Current.IsScanning)
            {
                CrossBleAdapter.Current.StopScan();
            }

            scanner = CrossBleAdapter.Current.ScanExtra(new ScanConfig {
                ScanType = BleScanType.LowLatency,

            }).Subscribe(ManageScanResult);

Note the CrossBleAdapter.Current.IsScanning return always false in my test.
In ViewWillDisappear
if(scanner != null)
            {
                scanner.Dispose();
            }

ManageScanResult just add the device to a List
I already read #161 but it does not resolve my problem. I use ScanExtra as you can see. My app is in Xamarin.iOS (not Xamarin.Forms)

I can’t reproduce this.  IsScanning is checking directly against the cbcentralmanager so the fact that it is returning false and two lines later is returning true is odd.
Note that you can call StopScanning without checking the flag yourself.  Scanextra also has a restart scan parameter you can use.

Hi,
I test on another iOS device (iPhone 6 with iOS 12.4.3) it happened as well, but not always (iPhone 7 iOS 13 happened everytime I abrutly close the app from the IDE).
However, I resolved with your suggestion about the restart flag:
scanner = CrossBleAdapter.Current.ScanExtra(new ScanConfig {
                ScanType = BleScanType.LowLatency,

            }, true).Subscribe(ManageScanResult);

I do not know if the flag restart cause overhead, but in my case it doesn't matter. My app will scan for about 20-30 second (I need to test it against the bluetooth device I use)
I put the Dispose only in ViewWillDisapper for testing purpose.
Thanks 👍


Relicense to LGPLv3

Relicense the project to LGPLv3, to allow use in more projects without worry of GPL contagion.

I give my explicit permission to relicense my contributed code under the terms GNU Lesser General Public License, version 3.0.


[ch3751] Fix wording in my donation page

https://app.clubhouse.io/enmarche/story/3751/empêcher-les-personnes-qui-font-déjà-un-don-mensuel-d-en-faire-un-second

don't forget to test your dev 😈


fix: commit the type piracy at runtime

"fixes": https://github.com/oolong-dev/OpenTelemetry.jl/issues/93
This moves the type piracy of the Base.schedule at runtime, to provide compatibility with Julia 1.10.
Sysimages will break!
The final fix will probably be something else, but I'm adding that here for raising awareness.
Tests pass, but I'm probably missing something (I'm learning about OpenTelemetry as we speak! super nice talk at JuliaCon btw 🤗)

This is also a bit problematic: sysimage-type issue (with this PR)
ERROR: LoadError: InitError: Evaluation into the closed module `OpenTelemetrySDK` breaks incremental compilation because the side effects will not be permanent. This is likely due to some other module mutating `OpenTelemetrySDK` with `eval` during precompilation - don't do this.


[DROOLS-1200] fix antlr lexer for recognition of multibyte unicode chars

Replaced by https://github.com/droolsjbpm/drools/pull/810


Small fixes

Small fixes in docs/core/preview3/tools/csproj.md.

Hi @kendaleiv, I'm your friendly neighborhood .NET Foundation Pull Request Bot (You can call me DNFBOT). Thanks for your contribution!
You've already signed the contribution license agreement. Thanks!
The agreement was validated by .NET Foundation and real humans are currently evaluating your PR.
TTYL, DNFBOT;

@kendaleiv Thank you for spotting these and submitting a fix. We really appreciate it.  This looks great. I'm :shipit: now.


feat: bitcoin regtest network option

Try out this version of Leather — download extension builds.

We have added a Bitcoin regtest option to the 'Add Network' function. This new feature allows you to add Bitcoin network alongside the Stacks networks.
This feature allows the user to add and use Bitcoin regtest networks, which can be much faster Testnet.
Currently when you add a Bitcoin network, the associated Stacks network will be Stacks Testnet.

@fabric-8 @mica000 are you able to provide to general design guidance for this feature we'd like to get into production?
The gist here is that @Polybius93, and many other developers, need to be able to target different network types, such as regtest. Our current UI doesn't support this, and a solution would valuable for many devs. As this is a developer-only feature, we don't need to finesse this particular part of the app, but think some feedback would still go a long way.
@Polybius93 for the designer to best help, can you please describe the problem this solves, and what the minimum requirements to fix the problem. I find the buttons added here a little confusing, plus I also don't understand why you don't need to fill in all the fields. Wouldn't a network type select here work too?

A segmented control would be pretty good here! (figma mockup)
While doing a quick mockup I also noticed that we aren't "containerizing" the contents in fullpage view. Do I remember correctly that this is something @pete-watters is looking into as part of the broader brand refinement work? 😅

@kyranjamie @fabric-8 @mica000
Hello!
In the current dev branch, the "add network" feature only allows users to add a new Stacks network. Afterwards, it associates a Bitcoin network constant with it (either Mainnet or Testnet). However, the network slice and its PersistedNetworkConfiguration are not prepared to handle a custom Bitcoin network.
This PR addresses the issue by enabling users to choose whether they want to add a new Stacks or Bitcoin network. If the user selects a Bitcoin network, the key field is disabled and automatically set to "bitcoin-regtest" for differentiation in the network converting logic.
The logic for adding Stacks networks remains unchanged, as it will still select a corresponding Bitcoin network. Additionally, when adding a Bitcoin network, the Stacks Testnet network is automatically made available alongside it.
Ideally, users should be able to set both Stacks and Bitcoin networks at the same time. However, I have not yet found an easy solution for this. There may be possible ways to modify the objects and functions involved to handle both scenarios effectively.

Thanks for the summary @Polybius93! Here's a design proposal (Figma), WDYT?

Thanks for the summary @Polybius93! Here's a design proposal (Figma), WDYT?

Hello! It looks really great, but before I proceed with making the changes to align with your design, I would prefer to wait for input from other developers, as the UI might undergo changes afterward (if any changes to the logic are required).
What do you think @kyranjamie ?
Thank you!

@Polybius93 Discussed this with @kyranjamie yesterday based upon which the design proposal has been updated.
Link to Design on Figma
We simplified things as much as possible, effectively merging both "tabs" into one unified view as that represents things more accurately.

At it's core, we porpose a dropdown ("Bitcoin API") where any selection will populate the text inputs below with defaults we assume would be useful.
But since those are just text inputs, the end user can obv. change this to whatever they need.

Propose selection / applied defaults:

Mainnet
BTC Network: blockstream.info/api
STX Network: https://api.hiro.so
Testnet
BTC Network: https://blockstream.info/testnet/api
STX Network: https://api.testnet.hiro.so
Signet
BTC Network: https://mempool.space/signet/api
STX Network: https://api.testnet.hiro.so
Mainnet
BTC Network: https://blockstream.info/testnet/api
STX Network: https://api.testnet.hiro.so

Happy to hear any thoughts or questions you might have on this and thanks for your support!

👋 Feel free to ping me once you feel the implementation is ready for a design review — or if there are any design-questions.

👋 Feel free to ping me once you feel the implementation is ready for a design review — or if there are any design-questions.

👋 Feel free to ping me once you feel the implementation is ready for a design review — or if there are any design-questions.

👋 Feel free to ping me once you feel the implementation is ready for a design review — or if there are any design-questions.

@fabric-8 Hey! Finished the code and all the checks are passing now, so the PR is ready to be reviewed! Thank you!

All good from my side for now, we'll probably do some design updates here in the aftermath as part of a broader design/brand-alignment initiative, but that shouldn't block the merge here!
Maybe just one minor thing: In fullscreen/tab view, the form contents are very narrow - any chance we can fix this? I think usually the max-width here should sit at around 400px, not quite sure why that doesn't work here. The width worked correctly in the previous add network view

All good from my side for now, we'll probably do some design updates here in the aftermath as part of a broader design/brand-alignment initiative, but that shouldn't block the merge here!
Maybe just one minor thing: In fullscreen/tab view, the form contents are very narrow - any chance we can fix this? I think usually the max-width here should sit at around 400px, not quite sure why that doesn't work here. The width worked correctly in the previous add network view

Thank you @fabric-8 ! Yeah, now that you have wrote this, I noticed that there is an extra container in the layout, that's why it was narrow, I've fixed and pushed it up.

@kyranjamie to do final code review

Sorry @Polybius93, reviewing this I discovered that there's a breaking change for users with existing networks saved.
Because we changed the name of the networks from stacks.url --> stacks.stacksUrl and bitcoin.url to bitcoin.bitcoinUrl, users with existing networks persisted will try and read bitcoinUrl which will be undefined.
Is it possible to leave the network schema as it was before?

Sure, I'll make it work in the afternoon! Thank you!

@Polybius93 any updates here? It'd be great to get this merged in soon

@Polybius93 any updates here? It'd be great to get this merged in soon

We were having some issues migrating the existing networks into this new structure, @kyranjamie is helping me out in in this issue currently.


Brew tap floc 1611

Fixes FLOC-1611

Tidy up the section in the release process documentation to remove the consecutive code extracts, and this is OK to merge (now that I understand what is happening with latest-installable


Customize Div Error

Hello,
Is there any method to customize error message ?
Somthing based on a template/div like this might be usefull :
{
errorTemplate:'#error-cam',
onError:function(){}
}

Which error message are you referring to?


Added Integration Foregoing 1.12 CraftTweaker documentation

Added documentation for Washing Factory, Fermentation Station and Fluid Sieving Machine.

You commit didn't fix the issue, right now a user will just copy that code block and paste it into the script, and it isn't going to work.
Ideally it should look like:
// FermentationStation.add(ILiquidStack fluidInput, ILiquidStack fluidOutput);

FermentationStation.add(<liquid:lava> * 1, <liquid:pyrotheum> * 1);

So users can copy it, and know what param is what.

Sorry about that, should be fixed now.

Thanks!


Create issue-templates

Description:
To streamline and standardize the process of creating new issues in the "remsfal" project, I propose the creation of issue templates. These templates will guide contributors on providing necessary details for different types of issues (e.g., bug reports, feature requests, documentation updates).
Tasks:

[ ] Identify common issue types (e.g., bug reports, feature requests, and documentation improvements).
[ ] Create a markdown template for each type of issue.
[ ] Add the templates to the .github/ISSUE_TEMPLATE/ directory.
[ ] Test the templates to ensure they render correctly when creating a new issue.

Acceptance Criteria:

Templates for bug reports, feature requests, and documentation updates are available.
Templates are pre-populated with the required fields and guidelines.
Templates are easy to use and improve the clarity of submitted issues.

This issue equally concerns remsfal-frontend.

A very good example can be found here: https://github.com/keycloak/keycloak/tree/main/.github/ISSUE_TEMPLATE
The GitHub documentation about creating such templates can be found here: https://docs.github.com/en/communities/using-templates-to-encourage-useful-issues-and-pull-requests/configuring-issue-templates-for-your-repository

@TheoThmn Would you like to implement this issue?

This issue is related to https://github.com/remsfal/remsfal-frontend/issues/209

@TheoThmn Would you like to implement this issue?

Yes, please @astanik. I'd propose to implement the issue 222 as well. That way the templates stay the same throughout the entire project.

Excellent! Can you please write a comment in issue https://github.com/remsfal/remsfal-frontend/issues/209 in order that i can assign you the task?

I propose for this to be implemented in a new repository named ".github" in the remsfal-organization. That way, the templates will be set centrally for frontend and backend at the same time.
Further information on how to organization-wide community health files
What are your thoughts on this, @astanik?


"flutter packages get" gets stuck

Both "flutter packages get" and "pub get "get stuck and fail to complete.

looks like a dup of #56

yea, that is definitely possible if you are trying to use the latest versions of built_value and build. If that is the case, i just published a dev branch that works with the latest versions, so pub should resolve with that version.
@kostaa will you post your pubspec.yaml if that dev branch doesn't work for you?

I think it should have been published as 7.5.0-dev instead.
Nothing serious, but built_redux: ^7.4.0-dev fetches 7.4.0 because it's newer

Thanks for that all is good now.

@zoechi pub would still fetch 7.4.0 if it was made as 7.5.0-dev. From pub_semver:

when pub is choosing a version, it uses priority order which is different from strict comparison ordering. Any stable version is considered higher priority than any unstable version. The above versions, in priority order, are:
1.2.0-alpha
1.3.0-experimental
1.0.0
1.2.0

Essentially the only way pub will resolve to the dev tag is if you have it pinned or if it is forced to resolve to the dev tag due to dependency constraints.

If xou published it as 7.5.0-dev then yes, I only saw 7.4.0-dev.
Perhaps I checked to early


Bump react-navigation/native and react-navigation/stack version

Details
Minor bump of react-navigation packages: stack and native.
There is one particular bugfix in the newest version especially important for the navigation refactor.
There is the commit with fix https://github.com/react-navigation/react-navigation/commit/ab66777c092fae3d28c9c8381aeb5b92945ca256
Fixed Issues

$ GH_LINK
PROPOSAL: GH_LINK_ISSUE(COMMENT)
Tests

[x] Verify that no errors appear in the JS console

Offline tests

QA Steps

[x] Verify that no errors appear in the JS console

PR Author Checklist

[x] I linked the correct issue in the ### Fixed Issues section above
[x] I wrote clear testing steps that cover the changes made in this PR

[x] I added steps for local testing in the Tests section
[x] I added steps for the expected offline behavior in the Offline steps section
[x] I added steps for Staging and/or Production testing in the QA steps section
[x] I added steps to cover failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[x] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)
[x] I tested this PR with a High Traffic account against the staging or production API to ensure there are no regressions (e.g. long loading states that impact usability).

[x] I included screenshots or videos for tests on all platforms
[x] I ran the tests on all platforms & verified they passed on:

[x] Android / native
[x] Android / Chrome
[x] iOS / native
[x] iOS / Safari
[x] MacOS / Chrome / Safari
[x] MacOS / Desktop

[x] I verified there are no console errors (if there's a console error not related to the PR, report it or open an issue for it to be fixed)
[x] I followed proper code patterns (see Reviewing the code)

[x] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick)
[x] I verified that comments were added to code that is not self explanatory
[x] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[x] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method

[x] If any non-english text was added/modified, I verified the translation was requested/reviewed in #expensify-open-source and it was approved by an internal Expensify engineer. Link to Slack message:

[x] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[x] I verified any copy / text that was added to the app is correct English and approved by marketing by adding the Waiting for Copy label for a copy review on the original GH to get the correct copy.
[x] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[x] I verified the JSDocs style guidelines (in STYLE.md) were followed

[x] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[x] I followed the guidelines as stated in the Review Guidelines
[x] I tested other components that can be impacted by my changes (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar are working as expected)
[x] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[x] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[x] I verified that if a function's arguments changed that all usages have also been updated correctly
[x] If a new component is created I verified that:

[x] A similar component doesn't exist in the codebase
[x] All props are defined accurately and each prop has a /** comment above it */
[x] The file is named correctly
[x] The component has a clear name that is non-ambiguous and the purpose of the component can be inferred from the name alone
[x] The only data being stored in the state is data necessary for rendering and nothing else
[x] For Class Components, any internal methods passed to components event handlers are bound to this properly so there are no scoping issues (i.e. for onClick={this.submit} the method this.submit should be bound to this in the constructor)
[x] Any internal methods bound to this are necessary to be bound (i.e. avoid this.submit = this.submit.bind(this); if this.submit is never passed to a component event handler like onClick)
[x] All JSX used for rendering exists in the render method
[x] The component has the minimum amount of code necessary for its purpose, and it is broken down into smaller components in order to separate concerns and functions

[x] If any new file was added I verified that:

[x] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[x] If a new CSS style is added I verified that:

[x] A similar style doesn't already exist
[x] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(themeColors.componentBG)

[x] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[x] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[x] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[x] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.
[x] I have checked off every checkbox in the PR author checklist, including those that don't apply to this PR.

Screenshots/Videos

Web

Mobile Web - Chrome

Mobile Web - Safari

Desktop

iOS

Android

@cristipaval Please copy/paste the Reviewer Checklist from here into a new comment on this PR and complete it. If you have the K2 extension, you can simply click: [this button]

I have read the CLA Document and I hereby sign the CLA

recheck

Hey @adamgrzybowski ! You have to correctly link the gh issue in the Fixed Issue section in order to get the right people assigned as reviewers to the PR. I edited it for now and manually assigned the right reviewers.

What should we test here?
@adamgrzybowski Can you run pod install? To commit changes made to pod.lock (if any)

@s77rt lets test core navigation things, some internal deeplinks (a link in the message), link from outside of the app, just in case you wont notice any regression
@adamgrzybowski lets get the commits signed, you can also create a new branch and force push if you cant get the old commits verified.

Reviewer Checklist

[ ] I have verified the author checklist is complete (all boxes are checked off).
[ ] I verified the correct issue is linked in the ### Fixed Issues section above
[ ] I verified testing steps are clear and they cover the changes made in this PR

[ ] I verified the steps for local testing are in the Tests section
[ ] I verified the steps for Staging and/or Production testing are in the QA steps section
[ ] I verified the steps cover any possible failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[ ] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)

[ ] I checked that screenshots or videos are included for tests on all platforms
[ ] I included screenshots or videos for tests on all platforms
[ ] I verified tests pass on all platforms & I tested again on:

[ ] Android / native
[ ] Android / Chrome
[ ] iOS / native
[ ] iOS / Safari
[ ] MacOS / Chrome / Safari
[ ] MacOS / Desktop

[ ] If there are any errors in the console that are unrelated to this PR, I either fixed them (preferred) or linked to where I reported them in Slack
[ ] I verified proper code patterns were followed (see Reviewing the code)

[ ] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick).
[ ] I verified that comments were added to code that is not self explanatory
[ ] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[ ] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method
[ ] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[ ] I verified any copy / text that was added to the app is correct English and approved by marketing by adding the Waiting for Copy label for a copy review on the original GH to get the correct copy.
[ ] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[ ] I verified the JSDocs style guidelines (in STYLE.md) were followed

[ ] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[ ] I verified that this PR follows the guidelines as stated in the Review Guidelines
[ ] I verified other components that can be impacted by these changes have been tested, and I retested again (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar have been tested & I retested again)
[ ] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[ ] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[ ] If a new component is created I verified that:

[ ] A similar component doesn't exist in the codebase
[ ] All props are defined accurately and each prop has a /** comment above it */
[ ] The file is named correctly
[ ] The component has a clear name that is non-ambiguous and the purpose of the component can be inferred from the name alone
[ ] The only data being stored in the state is data necessary for rendering and nothing else
[ ] For Class Components, any internal methods passed to components event handlers are bound to this properly so there are no scoping issues (i.e. for onClick={this.submit} the method this.submit should be bound to this in the constructor)
[ ] Any internal methods bound to this are necessary to be bound (i.e. avoid this.submit = this.submit.bind(this); if this.submit is never passed to a component event handler like onClick)
[ ] All JSX used for rendering exists in the render method
[ ] The component has the minimum amount of code necessary for its purpose, and it is broken down into smaller components in order to separate concerns and functions

[ ] If any new file was added I verified that:

[ ] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[ ] If a new CSS style is added I verified that:

[ ] A similar style doesn't already exist
[ ] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(themeColors.componentBG)

[ ] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[ ] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[ ] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[ ] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.
[ ] I have checked off every checkbox in the PR reviewer checklist, including those that don't apply to this PR.

Screenshots/Videos

Web
https://user-images.githubusercontent.com/16493223/225161758-6aa0bd5f-53f9-4c73-8c2b-b96f3f627f10.mp4

Mobile Web - Chrome
https://user-images.githubusercontent.com/16493223/225161471-6b848f2c-c8da-4083-9f58-905eed92cf2e.mp4

Mobile Web - Safari
https://user-images.githubusercontent.com/16493223/225161289-5d88e823-71c3-4572-8825-d4644923ab40.mp4

Desktop
https://user-images.githubusercontent.com/16493223/225162501-714ecd4f-5f55-48b3-9308-075abf5225e7.mp4

iOS
https://user-images.githubusercontent.com/16493223/225162737-d2a1188b-78eb-42cf-9933-6d7273a9e887.mp4

Android
https://user-images.githubusercontent.com/16493223/225162944-f5ecb3f0-bc50-4866-8393-8cbc01019be2.mp4

I have uploaded testing videos. Initial tests looks promising, all seems normal. Will perform more tests asap.
@adamgrzybowski Please tag me once you add the platforms videos.

Web
https://user-images.githubusercontent.com/67908363/225168407-9a098db3-7cd8-4cab-a562-b6d8c98e034f.mov
Android
https://user-images.githubusercontent.com/67908363/225168565-e4f662a2-9228-4e01-ab39-7bcfa385b26a.mov
iOS
https://user-images.githubusercontent.com/67908363/225168672-ded3a330-f19d-4a81-8421-c952243fbd66.mov
Desktop
https://user-images.githubusercontent.com/67908363/225168724-e10a935e-376b-4728-8d55-0151e5ff4908.mov
@s77rt

@adamgrzybowski Please add those to the PR description (follow the template) and add mWeb as well.

Just waiting for testing videos....

@s77rt Just added. Thanks for your patience and help! 🙇 All that procedures are still new to me

@mountiny Let's merge this :rocket:

Is this eligible for C+ compensation?

sorted in the linked issue

🚀 Deployed to staging by https://github.com/mountiny in version: 1.3.25-1 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

🚀 Deployed to production by https://github.com/roryabraham in version: 1.3.25-8 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅


Error: magicksave: libMagick error: NoEncodeDelegateForThisImageFormat `gif'

Did you see the documentation relating to installation?
Yes
Have you ensured the architecture and platform of Node.js used for npm install is the same as the architecture and platform of Node.js used at runtime?
Yes
Are you using the latest version? Is the version currently in use as reported by npm ls sharp the same as the latest version as reported by npm view sharp dist-tags.latest?
Custom build
If you are installing as a root or sudo user, have you tried with the npm install --unsafe-perm flag?
build/lin.sh
If you are using the ignore-scripts feature of npm, have you tried with the npm install --ignore-scripts=false flag?
N/A
What is the complete output of running npm install --verbose sharp? Have you checked this output for useful error messages?
N/A
What is the output of running npx envinfo --binaries --system?
npx envinfo --binaries --system
npx: installed 1 in 1.352s
System:
OS: Linux 4.19 Amazon Linux 2
CPU: (4) x64 Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
Memory: 1.08 GB / 1.94 GB
Container: Yes
Shell: 4.2.46 - /bin/bash
Binaries:
Node: 10.21.0 - /var/lang/bin/node
npm: 6.14.4 - /var/lang/bin/npm
ISSUE
I'm trying to use the new gif animation functionality and it is failing for GIFs. It is working great for WEBP.
The error (looks like a libmagick error) is:
Error: magicksave: libMagick error: NoEncodeDelegateForThisImageFormat `gif'
Here is what I'm using to install libmagcik
mkdir ${DEPS}/imagemagick
cd ${DEPS}/imagemagick
curl -O https://imagemagick.org/download/ImageMagick.tar.gz
tar xzf ImageMagick.tar.gz
rm ImageMagick.tar.gz

./configure \
  LDFLAGS=${LDFLAGS/\$/} \
  --host=${CHOST} \
  --prefix=${TARGET} \
  --disable-static --enable-shared \
  --with-modules \
  --without-bzlib \
  --without-dps \
  --without-freetype \
  --without-jbig \
  --without-jpeg \
  --without-jp2 \
  --without-lcms \
  --without-lzma \
  --without-png \
  --without-tiff \
  --without-wmf \
  --without-xml \
  --without-zlib \
  --without-perl \
  --with-webp \
  --without-x \
  --without-magick-plus-plus \
  --enable-delegate-build \
  --disable-dependency-tracking \
  --disable-docs \
  --disable-openmp \
  --disable-installed

make install-strip

I have libMagickCore-7.Q16HDRI.so.7 and libgif.so.7 in my vendors/8.10.1/lib directory, but I noticed at the end of the libmagick build I get:
Libraries have been installed in:
   /target/lib/ImageMagick-7.0.10/modules-Q16HDRI/coders

If you ever happen to want to link against installed libraries
in a given directory, LIBDIR, you must either use libtool, and
specify the full pathname of the library, or use the '-LLIBDIR'
flag during linking and do at least one of the following:
   - add LIBDIR to the 'LD_LIBRARY_PATH' environment variable
     during execution
   - add LIBDIR to the 'LD_RUN_PATH' environment variable
     during linking
   - use the '-Wl,-rpath -Wl,LIBDIR' linker flag
   - have your system administrator add LIBDIR to '/etc/ld.so.conf'

See any operating system documentation about shared libraries for
more information, such as the ld(1) and ld.so(8) manual pages.

I'm getting that libmagick is compiled with GIF:
GIF* GIF       rw+   CompuServe graphics interchange format
GIF87* GIF       rw-   CompuServe graphics interchange format (version 87a)
but those "coders" (are these the plugins libmagick refers to?) are nowhere to be seen in the final bundle unless they are par of libMagickCore-7.Q16HDRI.so.7.
For VIPS I'm getting:
checking for giflib... libraries -lgif, headers in default path
...
file import/export with giflib:		yes (found by search)

My question is:
Do I need to manually copy those "coders" in my build script? Or use copydeps in the build/lin.sh script?

I tried using copydeps for the  libMagickCore-7.Q16HDRI.so.7 and copied over the ImageMagick folder in lib to lib-filtered like we do with glib2, but it did not help. Not sure what to try next.

If you've made additions to https://github.com/lovell/sharp-libvips/blob/master/build/lin.sh then please provide the complete modification, perhaps as a forked repo/branch.
My best guess would be that you need to make the following changes:
- --disable-static --enable-shared --with-modules
+ --enable-static --disable-shared --without-modules

Thanks! Hopefully that's it!
I was following this: https://github.com/lovell/sharp/pull/2012#issuecomment-695995503 but having issues with the LDFLAGS, so I think in my journey down the rabbit hole I must have changed those settings. I posted a question about it in #2012 as well a while back, but didn't get very specific so I figured I'd create a new question for it.
I need to get better at my copy/pasta. Btw, love this project! I'll post back if it works.

It worked! It's slow, but it works. I think this is back to where I started, so now I don't know what I did wrong the first time. 🤷 GIFs are so much fun. Thanks for the help!

@cthorner mind if I ask you the complete lin.sh you used? or at least the portion of the fiel you modified? I'm trying to compile a version with gif support and I'm getting the same error you got in the title


Take me to the notication for the push

A push notification should take you to the channel that they were created in.
(I had a few push notifications for new content - but didn't know which channel they were in so never replied)

This seems to be working better now. I'll close it off and reopen if I see it again.


Is there or could there be a good workflow for using the CMS to do translation?

- Do you want to request a feature or report a bug?
feature
- What is the current behavior?
missing feature :)
- If the current behavior is a bug, please provide the steps to reproduce.
n/a
- What is the expected behavior?
the hoped-for behavior is that the CMS would support a workflow that allows for translation of a set of fixed strings.  It may already do this, so the solution to this issue may be "here's a way to set up that workflow" but @dkjess and I were talking and I didn't have one at hand.
Suppose that your app uses strings for all of the text in the site.  The site is built from some arbitrary-format files (e.g. ERB files).  Is there a way that this and only this could be handled by the CMS?  My suggestion was something like:

use a markdown file to contain the original strings and placeholders for their translations
use the CMS to edit that markdown file (already supported by the CMS) and
pull the content from the markdown file into your app using a script (something smarter than, but resembling "TOKEN1_SPANISH=grep TOKEN1_SPANISH input.md | cut -d= -f2 sed s/TOKEN1_SPANISH/$TOKEN1_SPANISH/g file.erb" and cycling through the tokens to repeat.

I guess the big hangup is, this wouldn't be a category, doesn't need media files, etc - just wants a git-backed way to expose just this file for editing to the content team, whose commit could still trigger a site build.  Clearly there would need to be some sanity checking in the script, but besides that, any advice on achieving this workflow?
- Please mention your node.js, and operating system version.
shouldn't matter :)

@fool it's an interesting idea - but I think this is more the job of static site generators like Jekyll or Hugo.
There is a plugin for Jekyll that can do this https://github.com/Anthony-Gaudino/jekyll-multiple-languages-plugin - and you would be able to edit those files using Netlify CMS by defining collections for the localization files.

@fool closing, I agree that this is probably out of scope for the cms.


Minimized AF2 predictions

Hello,
I am doing some testing of the code with several hundreds of designs of 7-15 residues peptidic binders targeting a GPCR I used hotspots to define the binding site during the RFDiffusion prediction.
During the AF2 step, I found major clashes in almost all of them. The lowest PAE_interaction value I have until now is 12.4. Are the AF2 predictions minimized/relaxed?
Thanks in advance.

The AF2 predictions are not minimized but we find that when pAE is good (<10) the structures can be minimized to not have clashes. AF2 is known to generate clashing side chains before minimization since it is operating in a constrained torsional space for side chains but these side chains can often be quickly minimized to have no clashes.
I would just filter by pAE unless you are trying to get a Rosetta ddG value, in which case you will need to minimize to have a meaningful ddG.

Thanks


add getUser function

see https://github.com/layerhealth/distill/pull/3599#discussion_r1854571272

actually i wanna give this branch a better name


Kafka lag emitter - Kafka Indexing Service

Fixes https://github.com/druid-io/druid/issues/4101

Unrelated test failure -
Tests run: 34, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 10.079 sec <<< FAILURE! - in io.druid.granularity.QueryGranularityTest
testDeadLock(io.druid.granularity.QueryGranularityTest)  Time elapsed: 10.034 sec  <<< ERROR!
java.lang.Exception: test timed out after 10000 milliseconds
	at sun.misc.URLClassPath$JarLoader.parseClassPath(URLClassPath.java:1018)
	at sun.misc.URLClassPath$JarLoader.getClassPath(URLClassPath.java:992)
	at sun.misc.URLClassPath.getLoader(URLClassPath.java:335)
	at sun.misc.URLClassPath.getResource(URLClassPath.java:198)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:364)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findClass(URLClassLoader.java:360)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:344)
	at io.druid.granularity.QueryGranularityTest.testDeadLock(QueryGranularityTest.java:799)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:483)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
Tests run: 2592, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 90.134 sec - in io.druid.segment.IndexMergerTest
Results :
Tests in error:
  QueryGranularityTest.testDeadLock:799 »  test timed out after 10000 millisecon...

👍

Very late here...but is there a particular monitor that needs to be configured in order to get these metrics? I have kafka indexing extension loaded and tasks running but do not see these lag metrics being emitted, nor any complaints about missing dimensions when the emitter runs.
Doc here does not say which monitor is required http://druid.incubator.apache.org/docs/latest/operations/metrics.html
Also would i expect the middlemanager node to be emitting these since that is the node running the kafka tasks?

@smokemonster99 you don't need to include any monitor, you just need to enable emitter (http://druid.incubator.apache.org/docs/latest/configuration/index.html#emitting-metrics), then you can see the metrics depending on emitter. The metrics would be emitted from tasks running on middle manager nodes.

I have the emitter configured and it is sending out jvm and other metrics but I do not see Kafka lag. Maybe because there are no remaining unread Kafka messages? Also I am using the stated-emitter but that shouldn’t matter.
I can send some new messages to Kafka and see if it sends them then. I’ll get back, thanks.
Sent from my iPad

On Oct 12, 2019, at 4:54 AM, Parag Jain<EMAIL_ADDRESS>wrote:
@smokemonster99 you don't need to include any monitor, you just need to enable emitter (http://druid.incubator.apache.org/docs/latest/configuration/index.html#emitting-metrics), then you can see the metrics depending on emitter. The metrics would be emitted from tasks running on middle manager nodes.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.

@pjain1 thanks! That was it, I just needed to actually send something to kafka to get the metrics to show up in datadog via dogstatsd.
Maybe you can help with this somewhat related question...My configured monitors such as jvm and sys work fine but I see there is a realtime monitor for ingestion, (org.apache.druid.segment.realtime.RealtimeMetricsMonitor) but when I configure it as a monitor on middlemanager, I get errors in the logs such as "No implementation for java.util.List<org.apache.druid.segment.realtime.FireDepartment> was bound."
Any ideas here? I am just looking for ingest metrics and thought that one was the best to use...

RealtimeMetricsMonitor is only used by RealtimeIndexTask so you cannot use this monitor with Kafka Indexing service you can can use Supervisor Ingestion stats report for similar metrics - https://druid.apache.org/docs/latest/development/extensions-core/kafka-ingestion.html#getting-supervisor-ingestion-stats-report. Also see https://druid.apache.org/docs/latest/development/extensions-core/kafka-ingestion.html#operations for different types of stats you can get for Supervisor.

Thanks @pjain1 !
Back to the Kafka lag, I have the emitter set to the statsd-emitter sending to dogstatsd (Datadog). I am getting many metrics related to sys, jvm etc. As far as Kafka lag, I only saw druid.coordinator.ingest.kafka.lag, druid.coordinator.ingest.kafka.avglag and druid.coordinator.ingest.kafka.maxlag and since i saw them in datadog metrics explorer it means druid tried to post it at some point, yet there is never any values associated with THESE metrics. You said to expect them as middlemgr metrics but they are not being posted by middlemgr, just coordinator apparently and these are all no value (not even 0). I am running 0.15.1. Any further help would be appreciated.

Hi, @smokemonster99
Maybe you can check you emitter configuration file. For example, graphite emitter is graphite_whitelist.json.  After adding the follow pattern, you can get the metrics.

  "ingest/kafka/lag" : ["dataSource"],
  "ingest/kafka/maxLag" : ["dataSource"],
  "ingest/kafka/avgLag" : ["dataSource"]

Thanks guys, I rebuilt my cluster from scratch (terraform deployment) and today the 3 lag metrics started posting to datadog! Not sure exactly what changed but I am not complaining.

Actually @wankunde my config file has
"ingest/kafka/lag" : { "dimensions" : ["dataSource"], "type" : "gauge" },
"ingest/kafka/maxLag" : { "dimensions" : ["dataSource"], "type" : "gauge" },
"ingest/kafka/avgLag" : { "dimensions" : ["dataSource"], "type" : "gauge" },
And I notice that my metrics do not have my datasource in the name like yours do. My metric posts like .druid.coordinator.ingest.kafka.lag
I plan to add more datasources shortly so will need that differentiation. Do I need to change ["datasource"] to an actual list of my datasources?


VS2017 Error A server with the specified hostname could not be found while adding an Apple Account

Trying to add an Apple Developer account [Tools > Options > Xamarin > Apple Accounts] to Visual Studio Professional 2017 Version 15.9.16
will result in a dialog saying:
Error adding account
There was an error while trying to log in: A server with the specified hostname could not be found.
At the paired mac's Keychain Access app, we can see in the Passwords section every attempt to try to add the account with a corresponding entry like:
<EMAIL_ADDRESS>Please advise

Closing this Duplicate in order to leave another at the specific doc


local deployment of llama-box models without gguf suffix has incorrect computed_resource_claim

Describe the bug

Steps to reproduce

Deploy a model qwen2.5 from Ollama.
Deploy a model qwen2.5 from local downloaded from Ollama. The file path is '/opt/models/ollama/qwen2_5'
Check the computed_resource_claim of the models.

Result

The computed_resource_claim for the model deployed from Ollama is correct.

The predicted vram is incorrect for the model deployed from local.

Expected behavior

Environment

GPUStack version: main 27fa7fe
OS: Ubuntu 22.04
GPU: NVIDIA GeForce RTX 4080 SUPER * 2

Additional context
Add any other context about the problem here.

Verified on main e4a7bf3


Incorrect Parameter type - Update nf-wdfdevice-wdfdevicequeryinterfaceproperty.md

Just to follow up -- the fix here is still coming!

Still waiting on update to https://docs.microsoft.com/en-us/windows-hardware/drivers/ddi/content/wdfdevice/nf-wdfdevice-wdfdevicequeryinterfaceproperty

Fixed!  Thanks @pankajbharti