Hive + databricks ingestion

succesfully deployed datahub on docker via quickstart guide, managed to ingest multiple data sources.
Attempted to ingest the hive_metastore on databricks following the recipe as per official documentation.
attempted ingestion via user interface gives the following error log:
Execution finished with errors.
{'exec_id': 'ab520d1f-10b1-4432-bb3f-df3ac1b2ba9c',
 'infos': ['2024-03-22 12:15:40.380167 INFO: Starting execution for task with name=RUN_INGEST',
           "2024-03-22 12:15:46.476249 INFO: Failed to execute 'datahub ingest', exit code 1",
           '2024-03-22 12:15:46.476440 INFO: Caught exception EXECUTING task_id=ab520d1f-10b1-4432-bb3f-df3ac1b2ba9c, name=RUN_INGEST, '
           'stacktrace=Traceback (most recent call last):\n'
           '  File "/usr/local/lib/python3.10/site-packages/acryl/executor/execution/default_executor.py", line 140, in execute_task\n'
           '    task_event_loop.run_until_complete(task_future)\n'
           '  File "/usr/local/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete\n'
           '    return future.result()\n'
           '  File "/usr/local/lib/python3.10/site-packages/acryl/executor/execution/sub_process_ingestion_task.py", line 282, in execute\n'
           '    raise TaskError("Failed to execute \'datahub ingest\'")\n'
           "acryl.executor.execution.task.TaskError: Failed to execute 'datahub ingest'\n"],
 'errors': []}

~~~~ Ingestion Report ~~~~
{
  "cli": {
    "cli_version": "0.13.0",
    "cli_entry_location": "/tmp/datahub/ingest/venv-hive-79357ad588bf7a10/lib/python3.10/site-packages/datahub/__init__.py",
    "py_version": "3.10.13 (main, Jan 17 2024, 06:53:56) [GCC 12.2.0]",
    "py_exec_path": "/tmp/datahub/ingest/venv-hive-79357ad588bf7a10/bin/python3",
    "os_details": "Linux-<IP_ADDRESS>-microsoft-standard-WSL2-x86_64-with-glibc2.36",
    "mem_info": "93.61 MB",
    "peak_memory_usage": "93.61 MB",
    "disk_info": {
      "total": "1.08 TB",
      "used": "20.03 GB",
      "used_initally": "20.03 GB",
      "free": "1.01 TB"
    },
    "peak_disk_usage": "20.03 GB",
    "thread_count": 1,
    "peak_thread_count": 1
  },
  "source": {
    "type": "hive",
    "report": {
      "events_produced": 0,
      "events_produced_per_sec": 0,
      "entities": {},
      "aspects": {},
      "warnings": {},
      "failures": {},
      "soft_deleted_stale_entities": [],
      "tables_scanned": 0,
      "views_scanned": 0,
      "entities_profiled": 0,
      "filtered": [],
      "num_view_definitions_parsed": 0,
      "num_view_definitions_failed_parsing": 0,
      "num_view_definitions_failed_column_parsing": 0,
      "view_definitions_parsing_failures": [],
      "start_time": "2024-03-22 12:15:43.143248 (1.02 seconds ago)",
      "running_time": "1.02 seconds"
    }
  },
  "sink": {
    "type": "datahub-rest",
    "report": {
      "total_records_written": 0,
      "records_written_per_second": 0,
      "warnings": [],
      "failures": [],
      "start_time": "2024-03-22 12:15:42.818081 (1.34 seconds ago)",
      "current_time": "2024-03-22 12:15:44.162189 (now)",
      "total_duration_in_seconds": 1.34,
      "max_threads": 15,
      "gms_version": "v0.13.0",
      "pending_requests": 0
    }
  }
}

~~~~ Ingestion Logs ~~~~
Obtaining venv creation lock...
Acquired venv creation lock
venv is already set up
venv setup time = 0 sec
This version of datahub supports report-to functionality
+ exec datahub ingest run -c /tmp/datahub/ingest/ab520d1f-10b1-4432-bb3f-df3ac1b2ba9c/recipe.yml --report-to /tmp/datahub/ingest/ab520d1f-10b1-4432-bb3f-df3ac1b2ba9c/ingestion_report.json
[2024-03-22 12:15:42,793] INFO     {datahub.cli.ingest_cli:147} - DataHub CLI version: 0.13.0
[2024-03-22 12:15:42,821] INFO     {datahub.ingestion.run.pipeline:238} - Sink configured successfully. DataHubRestEmitter: configured to talk to http://datahub-gms:8080
[2024-03-22 12:15:43,731] INFO     {datahub.ingestion.run.pipeline:255} - Source configured successfully.
[2024-03-22 12:15:43,732] INFO     {datahub.cli.ingest_cli:128} - Starting metadata ingestion
[2024-03-22 12:15:44,162] INFO     {datahub.ingestion.reporting.file_reporter:52} - Wrote UNKNOWN report successfully to <_io.TextIOWrapper name='/tmp/datahub/ingest/ab520d1f-10b1-4432-bb3f-df3ac1b2ba9c/ingestion_report.json' mode='w' encoding='UTF-8'>
[2024-03-22 12:15:44,163] INFO     {datahub.cli.ingest_cli:133} - Source (hive) report:
{'events_produced': 0,
 'events_produced_per_sec': 0,
 'entities': {},
 'aspects': {},
 'warnings': {},
 'failures': {},
 'soft_deleted_stale_entities': [],
 'tables_scanned': 0,
 'views_scanned': 0,
 'entities_profiled': 0,
 'filtered': [],
 'num_view_definitions_parsed': 0,
 'num_view_definitions_failed_parsing': 0,
 'num_view_definitions_failed_column_parsing': 0,
 'view_definitions_parsing_failures': [],
 'start_time': '2024-03-22 12:15:43.143248 (1.02 seconds ago)',
 'running_time': '1.02 seconds'}
[2024-03-22 12:15:44,163] INFO     {datahub.cli.ingest_cli:136} - Sink (datahub-rest) report:
{'total_records_written': 0,
 'records_written_per_second': 0,
 'warnings': [],
 'failures': [],
 'start_time': '2024-03-22 12:15:42.818081 (1.35 seconds ago)',
 'current_time': '2024-03-22 12:15:44.163248 (now)',
 'total_duration_in_seconds': 1.35,
 'max_threads': 15,
 'gms_version': 'v0.13.0',
 'pending_requests': 0}

below the recipe template used

source:
    type: hive
    config:
        host_port: '<databricks workspace URL>:443'
        username: token
        password: API token
        scheme: databricks+pyhive
        options:
            connect_args:
                http_path:
        stateful_ingestion:
            enabled: false
        profiling:
            enabled: false
            profile_table_level_only: true
`

figured it out, turns out they gave me the warehouse URL rather than the cluster.
It won't give you a clear error in this case just a failed execution so might be hard to troubleshoot.


DOC-3679 correct at least typo

Addresses: DOC-3679

Fixed minor typo in instances where both include and its introductory copy included text 'at least'

This change is

Approved by Stella out of band.


[MISC] Use the NO CHANGELOG label to optimize the changelog detection logic in CI.

Currently, we have implemented changelog checks for all PRs, but this can be inconvenient for PRs involving minor changes that don't require a changelog update. Therefore, we could introduce a GitHub label to indicate whether a PR can skip the changelog CI checks.
jobs:
  # ---------------------------------
  # Unit Testing
  # ---------------------------------
  unit-testing:
    runs-on: ${{ matrix.os }}
    strategy:
      fail-fast: false
      matrix:
        os: [ "ubuntu-latest" ]
        python-version: ["3.10", "3.11"]
    steps:
    - name: Checkout repository
      uses: actions/checkout@v4

    - name: Get PR Labels
      id: labels
      uses: actions/github-script@v6
      with:
        github-token: ${{secrets.GITHUB_TOKEN}}
        script: |
          const { data: pullRequest } = await github.rest.pulls.get({
            owner: context.repo.owner,
            repo: context.repo.repo,
            pull_number: context.issue.number
          });
          return pullRequest.labels.map(label => label.name);

    - name: Validate Changelog Update
      if: "!contains(steps.labels.outputs.result, 'no changelog')"
      uses: tarides/changelog-check-action@v2
      with:
        changelog: CHANGELOG.md

https://github.com/SuperDuperDB/superduperdb/pull/2034#issuecomment-2095290084


Adding packed-with-huan.yml

New rule that identifies files "packed" (encrypted) with Huan (https://github.com/frkngksl/Huan).

not sure why the thorough lint fails here

I think it's because we grep for "name: " in the CI script and that's part of the rule text. So merging here.


Text area Resize

The text area for the code can be dragged outside of div#form (dragging right, ontop of the div#sheet/div#outputcode. Once it leaves it can never come back!

Fixed in eddbb6c189335e3b7f8533bd195d1f3fec8ceaab. Thanks for the bug report!


AMD support

The title says it all :dancer:

https://github.com/daniel-lundin/snabbt.js a reference for module head and foot

Done.


fix issues with backspace not recognizing on Mac OS. 10.15.7

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 4 (0.0%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.6%) to 70.241%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

src/tuzue/ui/curses.py
0
4
0.0%

Totals

Change from base Build<PHONE_NUMBER>:
-0.6%

Covered Lines:
321

Relevant Lines:
457

💛  - Coveralls

I was able to reproduce this in an isolated environment, and I'm pretty sure this is solved in release 0.1.2 - can you try it out?

update on main.

I was able to reproduce this in an isolated environment, and I'm pretty sure this is solved in release 0.1.2 - can you try it out?

Yes. Its working.


Use new modal for confirm dataset rename on editor

Replaces the existing confirmation modal:

With:

Not completely done, created PR to get early feedback @saleiva @javierarce I have some questions

Is the text OK? Used what already existed, let me know what you want instead.
From notes from a week or two an "interrogation" glyph is supposed to be used as icon, do you have one prepared? :)
This modal can be be opened in two use-cases, either through the inline-edit (like in the gif above) or through "edit metadata". Considering this modal will be changed later and the post-save feedback still has the small thing "Saving.." label in the mid-top I think it's better to skip the typical full "Saving…" modal until we refactor the edit-metadata part. Ok with you?

Frontend tests were OK :+1: (details)

I propose this shorter version that also reduces the repetition of 'name':

Renaming dataset will affect your API calls
If you are accessing this dataset via API don't forget to use the new name in your API calls afterwards.

current text:

Renaming dataset will affect your API calls
If you are accessing your dataset via API don't forget to update the name in the API calls after changing the name

:+1:

Frontend tests were OK :+1: (details)

colors are a bit weird. Can we use the orange that we use on the small "geo" tag on the table??

Sure, I used this color because it was the one defined for "alert' color, and already used in other places, so if you find this color weird you might want to revisit those too. :)

ok?

@javierarce I'm done so please code review too

Frontend tests were OK :+1: (details)

:+1:

Added an icon using, based on Lato bold:

Frontend tests were OK :+1: (details)

awesome!


Add me!

Obspogon
https://obspogon.neocities.org/
https://obspogon.neocities.org/updates.xml and https://obspogon.neocities.org/blog/feed.xml

The webring code only allows a single RSS feed. Which one would you prefer?

Go with https://obspogon.neocities.org/blog/feed.xml to match the others.


Add method to pass any setting to DuckDB config

Most settings from https://duckdb.org/docs/sql/configuration.html currently can't be configured since the set method is private. This PR adds a new with method that makes it easy to compose a config with arbitrary settings.

Thanks for the PR! Could we add a test with an invalid setting as well?

Thanks for the PR! Could we add a test with an invalid setting as well?

Absolutely! Done.

Thanks!


wrangler dev fails if HTTP/2 is unavailable

Blocked on #864
wrangler dev fails if HTTP/2 is unavailable, rather than falling back to HTTP/1. This causes problems in environments where HTTP/2 is unsupported, such as in StackBlitz.

This should not be blocked by #864, as it would work with API token - allowing the full experience, except the login part.

Following up on this, it looks like Stackblitz doesn't currently support the http2 module, which Wrangler depends on

Wait, so what's the current status of this? Is the "quick win" to gracefully fallback to http/1.1 when http/2 is unsupported?

This isn't really a quick win—the problem is that we rely on the http2 module for remote mode. Fixing it would require rewriting most of remote mode

Following up to track status on this: this involves investigating whether we can use fetch or similar APIs rather than the http2 module


problem with the frequency of ansys calculated and scipy calculated are different

Hello, everyone,
I have a problem with the frequency calculated.
when I use ansys calculated I can get [2871,6168,8689,11468]
When I try to get the mass and stiffness matrix from a .full file with the scipy.sparse.linalg.eigsh module to calculate, Python give me
[15.811 15.812 15.813 15.814].
I think they should be the same.
Here is the code I use for testing:
import pyansys
import os
import scipy
from scipy.sparse import linalg
import numpy as np

fullfile = os.path.join('G:\STIFF', 'CURVE.full')
full = pyansys.read_binary(fullfile)
dof_ref, k, m = full.load_km()   # returns upper triangle only

k += scipy.sparse.triu(k, 1).T
m += scipy.sparse.triu(m, 1).T

k += scipy.sparse.diags(np.random.random(k.shape[0])/1E20, shape=k.shape)

w, v = linalg.eigsh(k, k=20, M=m, sigma=10000)

f = np.real(w)**0.5/(2*np.pi)
print('First four natural frequencies')
for i in range(4):
    print('{:.3f} Hz'.format(f[i]))

here my full file and result file. which get from ansys calculated. Hope someone can help me.
file.zip

I'm running into the same issue as you in regards to the python solve.  However, when solving this within MAPDL using MAPDL Math, I'm ending up with a different set of eigenvalues than you:

! read in matrices
*SMAT, stiff, D, import, full, file.full, STIFF
*SMAT, mass, D, import, full, file.full, mass

! solve
/SOLU
ANTYPE,MODAL
MODOPT,LANB,4,1
*VEC, MYVEC, D, ALLOC, 0
*eig,STIFF,MASS,,MYVEC

Which returns:
  FREQUENCIES AT CURRENT LANCZOS CYCLE
     1  0.85280110E+01     2  0.12485649E+02     3  0.14800782E+02
     4  0.17033644E+02     5  0.21613667E+02     6  0.22647486E+02
     7  0.26119110E+02     8  0.29797630E+02     9  0.30706920E+02
    10  0.31378644E+02    11  0.33062955E+02    12  0.34077783E+02
    13  0.36827381E+02    14  0.42039170E+02

     number of steps      :      2
     eigenvalues found    :     14
     total no. eigenvalues:     14

  LANCZOS CYCLE NUMBER =   2

     new shift:  1.4948D+04     modes still needed:        0
  Allocate a [4] Vector : MYVEC

 *** ANSYS - ENGINEERING ANALYSIS SYSTEM  RELEASE Release 18.2.2   18.2.2   ***
 ANSYS Mechanical Enterprise
 88888888  VERSION=LINUX x64     12:22:41  OCT 15, 2020 CP=      0.752

 *** FREQUENCIES FROM BLOCK LANCZOS ITERATION ***

  MODE    FREQUENCY (HERTZ)

   FREQUENCY RANGE REQUESTED=    4 MODES ABOVE   1.00000     HERTZ

    1     8.528010965666
    2     12.48564886186
    3     14.80078189537
    4     17.03364386520

Please note that I'm running this on ANSYS v18.2, and there might be a fundamental difference in full files between v17 and v18.2.  Can you try running the above on v17?
Pinging @FredAns.  Is there something we're potentially missing here?

I'm running into the same issue as you in regards to the python solve. However, when solving this within MAPDL using MAPDL Math, I'm ending up with a different set of eigenvalues than you:

! read in matrices
*SMAT, stiff, D, import, full, file.full, STIFF
*SMAT, mass, D, import, full, file.full, mass

! solve
/SOLU
ANTYPE,MODAL
MODOPT,LANB,4,1
*VEC, MYVEC, D, ALLOC, 0
*eig,STIFF,MASS,,MYVEC

Which returns:
  FREQUENCIES AT CURRENT LANCZOS CYCLE
     1  0.85280110E+01     2  0.12485649E+02     3  0.14800782E+02
     4  0.17033644E+02     5  0.21613667E+02     6  0.22647486E+02
     7  0.26119110E+02     8  0.29797630E+02     9  0.30706920E+02
    10  0.31378644E+02    11  0.33062955E+02    12  0.34077783E+02
    13  0.36827381E+02    14  0.42039170E+02

     number of steps      :      2
     eigenvalues found    :     14
     total no. eigenvalues:     14

  LANCZOS CYCLE NUMBER =   2

     new shift:  1.4948D+04     modes still needed:        0
  Allocate a [4] Vector : MYVEC

 *** ANSYS - ENGINEERING ANALYSIS SYSTEM  RELEASE Release 18.2.2   18.2.2   ***
 ANSYS Mechanical Enterprise
 88888888  VERSION=LINUX x64     12:22:41  OCT 15, 2020 CP=      0.752

 *** FREQUENCIES FROM BLOCK LANCZOS ITERATION ***

  MODE    FREQUENCY (HERTZ)

   FREQUENCY RANGE REQUESTED=    4 MODES ABOVE   1.00000     HERTZ

    1     8.528010965666
    2     12.48564886186
    3     14.80078189537
    4     17.03364386520

Please note that I'm running this on ANSYS v18.2, and there might be a fundamental difference in full files between v17 and v18.2. Can you try running the above on v17?
Pinging @FredAns. Is there something we're potentially missing here?

Thanks for your help, I can get the frequencies same as you. I think the frequencies your write is good although it's a little different with scipy.sparse.linalg.eigsh. And maybe what the eigsh calculated is not right I don't know why.
Another question I want to know, I use the
/solu
antype,1
bucopt,lanb,4
mxpand,4,,,1
solve
finish
to calculate the modal and I use Gui/General postproc/ read results / by pick,
there are four frequencies,They are what I write  [2871,6168,8689,11468]. I want to know if they are not modal's frequencies?
Here is my order stream: curved box.txt

Hi,
For what I see, this mathematical problem seems to be a bit hard to solve, and I see the Mapdl Lanczos algorithm making a lot of iterations to converge.
My feeling is that scipy func gives wrong results, because of the numerical difficulties.
I've checked using an alternative Modal Algorithm for Symmetric Matrices ( MODOPT,SUBSPACE,4,1), an I get very close solutions to Lanczos, but SUBSPACE converge faster because it's more robust in this numerical situation :
FREQUENCY RANGE REQUESTED=    4 MODES ABOVE   1.00000     HERTZ
1     8.528801109020
2     12.49936641695
3     14.83630288045
4     17.04379504882

Hi,
For what I see, this mathematical problem seems to be a bit hard to solve, and I see the Mapdl Lanczos algorithm making a lot of iterations to converge.
My feeling is that scipy func gives wrong results, because of the numerical difficulties.
I've checked using an alternative Modal Algorithm for Symmetric Matrices ( MODOPT,SUBSPACE,4,1), an I get very close solutions to Lanczos, but SUBSPACE converge faster because it's more robust in this numerical situation :
FREQUENCY RANGE REQUESTED= 4 MODES ABOVE 1.00000 HERTZ
1     8.528801109020
2     12.49936641695
3     14.83630288045
4     17.04379504882

Thanks for your help, it's so good to have a math genius like you to help my question. I'll select Subspace and Lanczos to solve
generalized eigenvalue questions in engineering.

Thanks @FredAns!

Thanks @FredAns!

I have known why the answer is [2871,6168,8689,11468] in the begining. Because
sqrt(2871)/(2pi)=8.5280
sqrt(6168)/(2pi)=12.499
sqrt(8689)/(2pi)=14.836
sqrt(11468)/(2pi)=17.044
This is similiar to what you calculated in Lanczos
1     8.528010965666
2     12.48564886186
3     14.80078189537
4     17.03364386520  .
Because what in Lanczos calculated is circular frequency(ω), and the Ansys's gui calculated is linear frequency (f).  So what you calculated and ansys calculated are both right.
That's all,
Thanks for your help.

Good point :)

Closing this as it seems to be resolved.  In future releases we should be able to send/receive matrices directly to and from Python.


file upload button is Not type='button'

Describe the bug
I download the toastui-editor-all.min.js file by entering 'https://uicdn.toast.com/editor/latest/toastui-editor-all.min.js' on the browser url.
but i got some problem when i use toast editor with <form> tag.
when i click file upload button (파일 선택), the form submits.
soon i knew that the file upload button's type attribute was missing.
To Reproduce
Steps to reproduce the behavior:

Click '파일 선택'
the form submits

Expected behavior
dont submit form.
Screenshots

Desktop (please complete the following information):

OS: [Window10]
Browser: [Chrome]
Version [91.0.4472.114]

@devToroko
It will be fixed in next release.


Why does image appear less sharp compared with that viewed in QGIS?

Hi
Thanks for these very handy tools to work with large image files.  I'm kind of new to this and am wondering if I've misunderstood how images are rendered in different places.
If I open the original tif file (which is not a valid cog) in QGIS and zoom onto the area of interest.  It looks like this (screenshot):

If I use rio-cogeo to convert it into a COG with:
rio cogeo create '{str(path_in)}' out_jpeg.tif -b 1,2,3 --add-mask --cog-profile jpeg

and then grab the corresponding tile with
tile, mask = rt_main.tile('out_jpeg.tif', *tiles.iloc[idx].xyz, tilesize=1024)
tile = tile.transpose([1, 2, 0])
_, ax = plt.subplots(figsize=(8, 8))
ax.imshow(tile, interpolation='none')
ax.axis('off');

, I see an image like this (screenshot), which seems slightly blurry, or to have softer lines, compared with what's seen in QGIS:

Has the original image become less sharp going through rio-cogeo and rio-tiler, or it's just QGIS doing some processing to make the image appear very sharp?    If the former, ere there some optional arguments that can be specified in either rio-cogeo or rio-tiler to preserve the original image resolution?
Thanks

Hi @qAp, thanks for the issue.
is your input GTIFF ({str(path_in)}) using JPEG compression too?, if not, this will be the cause. JPEG compression is NOT lossless, meaning you'll get a really light file but you'll loose some information (add blur).
When displaying the file at low resolution QGIS will do nearest resampling which is also the default for creating the overview in rio-cogeo. I think there is an option i rio-cogeo --overview-resampling that let you change this to bilinear. This might also make the overview a bit less blurry

Can't actually tell the type compression of the tif file {str(path_in)} using either file or identify.  Does this mean there is something unusual about this tif file?
I have tried various different --overview-resampling values and --cog-profile values, like lzw, but the quality of the subsequent tile time seems to be unchanged from before.
Using gdal like this
ds = gdal.Open(os.path.join(path,filename))
    band = ds.GetRasterBand(1)
    xsize = band.XSize
    ysize = band.YSize

for i in range(0, xsize, tile_size_x):
        for j in range(0, ysize, tile_size_y):
            com_string = "gdal_translate -of JPEG -srcwin " + str(i)+ ", " + str(j) + ", " + str(tile_size_x) + ", " + str(tile_size_y) + " " + f'\"{os.path.join(path,filename)}\"' + " " + f'\"{os.path.join(out_path, output_filename + str(i) + "_" + str(j) + ".jpeg" )}\"'
            os.system(com_string)

yields tile images like this:

whereas similar building structures using rio-cogeo and rio-tiler look like

which seems more blurry (but maybe I'm splitting hairs here.)

@qAp did you figure this out ?


Remove expression sandbox

What kind of change does this PR introduce? (Bug fix, feature, docs update, ...)
refactor
What is the current behavior? (You can also link to an open issue here)
The angular expression parsers attempts to sandbox the expressions to prevent unrestricted access to the global context.
What is the new behavior (if this is a feature change)?
The sandbox is removed, as it was not a real security feature.
Does this PR introduce a breaking change?
No
Please check if the PR fulfills these requirements

[x] The commit message follows our guidelines: https://github.com/angular/angular.js/blob/master/CONTRIBUTING.md#commit-message-format
[x] Tests for the changes have been added (for bug fixes / features)
[x] Docs have been added / updated (for bug fixes / features)

Other information:
TODO:

fill out the commit message
more refactoring to remove unnecessary vars
look into further performance improvements

Once it has a proper commit message and a couple of unused vars at the top of the file are removed (I only found some cached methods, that's all) I think it's ready to land. Performance improvements can be addressed in subsequent PRs/commits, I like that this one mostly cleanly removes stuff.
Other than that, LGTM.

The changes LGTM. Wrt commit message:

[ ] According to this well-respected resource, the type should be refactor :grin:
[ ] Update with URL to blog post. (Just putting it here as a reminder.)

Oh! In a project I work on, we have expressions that can be edited by users. They're used in dynamically composed templates. For us, it is a real security feature. :confused:

This is precisely why we are removing it. It is not a security feature.

@gkalpak - I always get that wrong :-(


Computes UV coordinates of the canvas_item vertices of StyleBoxFlat

Fixes https://github.com/godotengine/godot/issues/31885

Thanks!


check missing entities

sometimes a few entities go missing, please check.

check the nosql writer

no more threads, use an executor service
separate beans from executing code
name classes more appropriate e.g. Consumer, Writer, etc


If few openebs install job fails then also other openebs function and chaos job shoud run

Fix Pr: https://github.com/mayadata-io/oep-e2e-aws/pull/22
      #### https://github.com/mayadata-io/oep-e2e-aws/pull/21


code - window_bar component

#4
todo

[x] code - window_bar component

@input 📦 home_page_v0.0.10 from https://github.com/dat-ecosystem/organization/issues/85
@input 📦 buttons_v0.0.3 from #1
[x] code the window_bar component
[x] adding notify to close parent component
@output 📦 window_bar_v0.0.1 from comment

[x] update window_bar_v0.0.1

@input 📦 window_bar_v0.0.1
[x] condition to add action buttons
[x] condition to toggle button icon
[x] updated theme
@output 📦 window_bar_v0.0.2 from comment

[x] update window_bar_v0.0.2

@input 📦 window_bar_v0.0.2
[x] code protocol and listen
@output 📦 window_bar_v0.0.3 from comment

[x] update window_bar_v0.0.3

@input 📦 window_bar_v0.0.3
[x] update img to svgs
[x] update toggle functions
@output 📦 window_bar_v0.0.4 from comment

[x] update window_bar_v0.0.4

@input 📦 window_bar_v0.0.4
[x] update adding icon buttons and protocols
@output 📦 window_bar_v0.0.5 from comment

[x] update window_bar_v0.0.5

@input 📦 window_bar_v0.0.5
[x] update pattern
@output 📦 window_bar_v0.0.6 from comment

[x] finalize window_bar_v?.?.?

@output :question: window_bar_v?.?.? from comment

next #6
next #7
next #10
next #12
next #13
next #14

tasks - 2023.05.26

coded window_bar_v0.0.1

code the window_bar component - 2h11min
adding notify to close parent component - 33min
discussion on 2023.05.24 - 23min
create worklog comment, task, issues and pull request - 12min
@output 📦 button_v0.0.3

worklog

feedback

coded window bar component
coded notify to connect app_cover with window_bar to close them
added icons
coded toggle functions and responsive designs

proposal

tasks - 2023.06.03

coded window_bar_v0.0.1

code the window_bar component - 2h11min
adding notify to close parent component - 33min
discussion on 2023.05.24 - 23min
create worklog comment, task, issues and pull request - 12min
@output 📦 window_bar_v0.0.1

worklog

feedback

coded window bar component
coded notify to connect app_cover with window_bar to close them
added icons
coded toggle functions and responsive designs

proposal

feedback 2023.06.08
Same thing again, there should never be a my_theme required in any component.
This is true for all components even though i didn't always write it.
Reasining i gave in another issue reply.
Basically, because the theme should only contain some hard coded invariants for certain components where we know this will never change, e.g. obvious example would be a "logo component" of a company that will always in every theme use the same logo shape until the company decides to update their logo in general.
Theme is otherwise always passed as opts (=props) and/or as a protocol message to update it while the user uses an app
Also, even though we don't always use bi-directional communication, we might want to anyway use the pattern that enables bi-directional communication, so
function window_bar (props, notify) {
// ...

    close_window_btn.addEventListener('click', function() {
        notify( { active_state : 'active' } )
    });
    //...
}

should become the following also using our standard message format:
const message = { head, refs, type, data, meta }
function window_bar (props, protocol) {

   const notify = protocol(listen)

   // ...

    close_window_btn.onclick = event => notify({ type: 'state', data: 'active' })

    //...

    return el
    function listen (message) { }
}

Attaching event listener in this style is a bit more concise.
The message above is lacking { head, refs, ..., meta } but just to start simple.
At least it features the vital properties { type, data } to signal a state change
Of couse, we have an empty listen funciton function listen (message) { }
Because we are currently not listening to any messages, but this might change once we listen to dynamic theme changes later on.
Lastly:
    const { action_buttons } = props

    if (action_buttons) { // to support proper standardjs

        // adding additional actions wrapper
        const actions_wrapper = shadow.querySelector('.actions_wrapper')
        actions_wrapper.append(...action_buttons.map(txt => sm_icon_button_alt({ src: txt }))

        // adding toggle button for action wrapper
        const actions_toggle_btn = sm_icon_button_alt({
          src: icon_arrow_down_light,
          src_active: icon_arrow_up_light
        })
        actions_toggle_btn.classList.add('actions_toggle_btn')

        actions_toggle_btn.onclick = event => {
            shadow.querySelector('.window_bar_actions').classList.toggle('active')
        }

        window_bar_actions.append(actions_toggle_btn)

    }

But even the above is not enough.
The boundary between component instances is "HOLY" and can never be violated.
First, it won't work anyway, because we are using a "close shadow",
but second, the proper way to do it is send a message.
We are dealing here with sm_icon_button_alt instances:

And you are toggeling a class
and you are attaching click event listener

The proper way would be to let the sm_icon_button_alt do those things internally,
but you communicate using the protocol, thus:

you send a message to the button of type { type: 'toggle_state' }

or if you want to be explicit just send: { type: 'state', data: 'active' }

you listen for message of type 'click', so like this:

    const window_bar_actions = shadow.querySelector('.window_bar_actions')
    const { action_buttons } = props

    if (action_buttons) { // to support proper standardjs

        // adding additional actions wrapper
        const actions_wrapper = shadow.querySelector('.actions_wrapper')
        actions_wrapper.append(...action_buttons.map(txt => sm_icon_button_alt({ src: txt }))

        // adding toggle button for action wrapper
        const actions_toggle_btn = sm_icon_button_alt({
          src: icon_arrow_down_light,
          src_active: icon_arrow_up_light
        }, send => message => { // protocol to talk to this component instance
          if (message.type === 'click') window_bar_actions.classList.toggle('active')
        })
        actions_toggle_btn.classList.add('actions_toggle_btn')

        window_bar_actions.append(actions_toggle_btn)

    }

tasks - 2023.06.23

update window_bar_v0.0.2

code protocol and listen - 2h
create worklog comment, task, issues and pull request - 8min
@output 📦 window_bar_v0.0.2

worklog
worklog-158

feedback

proposal

tasks - 2023.07.06

update window_bar_v0.0.3 - 18min

update img to svgs
update toggle functions
create - worklog comment, and task- 11min
@output 📦 window_bar_v0.0.4

worklog
worklog-161
worklog-162

feedback

proposal

tasks - 2023.07.29

update window_bar_v0.0.4 - 40min

update adding icon buttons and protocols
create - worklog comment, and task- 12min
@output 📦window_bar_v0.0.5

worklog
worklog-167

feedback

proposal

tasks - 2023.11.11

update window_bar_v0.0.5

update pattern - 7min
@output 📦 window_bar_v0.0.6

worklog
worklog-181

feedback

proposal

tasks - 2023.12.06

update window_bar_v0.0.6

update protocol - 5min
create and update tasks - 5min
@output 📦window_bar_v0.0.7

worklog
worklog-185

feedback

proposal

tasks - 2023.12.08

update window_bar_v0.0.7

update navigation protocol - 15min
create and update tasks - 8min
@output 📦window_bar_v0.0.8

worklog
worklog-186

feedback

proposal


Upgrade terraform-provider-datadog to v3.34.0

Release details: https://github.com/DataDog/terraform-provider-datadog/releases/tag/v3.34.0

Testing; please ignore. Will close when ready.


Line's border width is cut in half when aligning with the x axis.

Expected Behavior
Line between two data points that are 0 should keep it's border width so there's a consistent line between all data points.
Current Behavior
Line width is cut in half whenever aligning with the x-axis.

Possible Solution
Allow to add an option to draw all line within the rect and not on the actual border itself (which will cut-off the other half that is outside).

This is fixed by #6642


⚠ Bofrost.be has degraded performance

In b454a21, Bofrost.be (https://www.bofrost.be) experienced degraded performance:

HTTP code: 200
Response time: 1154 ms

Resolved: Bofrost.be performance has improved in bfb16e1.


Support all ESDT

@PaulHendi I am sure we can add support for all the ESDT 👍

I included a script for classical ESDT airdrop. Tested with Golden token, it works :)


OreDictionary Preferences

Issue Description:
In previous versions of EnderIO, we had an oredictionary preferences xml file (like this one). I cannot find a similar file in the current version of EnderIO. For some reason, EIO has defaulted to the ugliest dusts we have in the pack, and I'd love to fix this.
What happens:
Sagmill gives me libvulpes dusts.
What you expected to happen:
I'd just like to have the ability to change what dust that EIO machines give.
Steps to reproduce:
N/A

Affected Versions (Do not use "latest"):

EnderIO: 5.0.22
EnderCore: 0.5.22
Minecraft: 1.12.2
Forge: <IP_ADDRESS>55

Your most recent log file where the issue was present:
N/A

Thank you for your suggestion, but we manage suggestions on http://ideas.theideawall.com/EnderIO
This ticket will be closed because we want the list of open tickets to reflect known issues.

Just for reference enderio defaults to the first entry for that oredict.


Improve how text streaming is displayed

Problem
Currently the text streaming looks quite choppy compared to other LM platforms. The text is not displayed evenly or at a consistent pace, which makes the UI look janky.
Compare how the text is displayed in different apps:
Claude
The text is displayed line by line, with the Claude icon at the bottom simulating a cursor.
https://github.com/user-attachments/assets/d73e2b64-1e71-4da7-af50-af0c355d3a42
ChatGPT (desktop)
The text seems to be mostly displayed word by word, with a "rounded cursor" at the current end of text.
https://github.com/user-attachments/assets/09b11f82-b6fb-4c01-8ade-07ce3cefb6f7
Moxin
Different-sized blocks of text are displayed at different speeds. This might not look as bad on this video, but it feels a lot worse on slower hardware.
https://github.com/user-attachments/assets/e2d20b47-abd3-49df-bbeb-01c315c5feaf
Proposed fix
We could take inspiration from other LM tools and implement some pacing that makes the text be displayed more evenly. We could do this either line by line like Claude, word by word (ChatGPT), or some other way that introduces consistency.
Adding some type of cursor indication at the end of the text seems like a good idea to make clear that text is being streamed. This could also help with the issue described in https://github.com/moxin-org/moxin/issues/199. This could be something simple like temporarily adding a special character like the ChatGPT “●” (U+25CF) at the end of the text until the streaming has finished. Or it could be a separate icon beneath the text like in Claude (it could also be animated).
Notes
Issue https://github.com/moxin-org/moxin/issues/199 describes how to look for when the model has finished streaming. Also there mechanisms in place already being used to hide the "Stop Streaming" button once streaming has finished.

I am currently facing an issue with how to select the AI's ongoing reply for caching the message content. At present, Moxin renders all the messages in the store at once for the chat_line rendering. In this case, determining how to select the AI's current reply in the conversation with the user becomes a challenge. So far, I haven't found a corresponding solution."
This is the set_text function
This is the render function

@Guocork besides our offline conversations, I think a good starting point could be to add a character (e.g. ●) simulating the cursor at the end of the message (while streaming). If you're struggling with controlling how the text is updated on the screen, this could be a starting point that will make you identify when and and where to update the message content (ChatLine - draw_walk()), and also it'll require you to pass a flag from ChatPanel to ChatLine indicating wether that's the end of the stream or if there's more text incoming.
With that implemented we can check if pacing the text display is necessary or not.

I fix that in pr #262

For the time being we'll close this with the improvement in #262


Cannot change the value using react-forms with value set on the field

hey @omeralpi ,
First of all many thanks, this library is awesome.
I wonder if you faced a similar issue using react-forms.
I'm adding the following code on my form:

const form = useForm({
    resolver: zodResolver(FormSchema),
    defaultValues: {
      phone: userData?.phone || '',
    },
  });

...
         <FormField
            control={form.control}
            name="phone"
            render={({ field }) => ( // field already has a value
              <FormItem className="dialog-form-field">

                  <FormLabel>Phone Number</FormLabel>

                  <FormControl>
                    <PhoneInput placeholder="Phone" {...field} className="w-4/5 inline-flex" />
                  </FormControl>

              </FormItem>
            )}

It works when value is empty but when a value is already set the flag dropdown does not open:

Any idea where the issue can be?

Hey,
Can you send a video about the problem?

Hi @omeralpi ,
Thanks for your quick response. Of course:
https://github.com/omeralpi/shadcn-phone-input/assets/2423937/3618ac8c-dba0-4d88-92ea-a545daf48f59
Basically when i provide an existing value for the form, the flag dropdown does not open. Can be also a problem in my app though but as i share in the snippet earlier, I don't see anything strange.
Thanks in advance,
Pablo

Hi @omeralpi ,
Thanks for your quick response. Of course:
Screen.Recording.2024-06-04.at.3.47.25.PM.mov
Basically when i provide an existing value for the form, the flag dropdown does not open. Can be also a problem in my app though but as i share in the snippet earlier, I don't see anything strange.
Thanks in advance,
Pablo

Currently, there doesn't seem to be any problem, but can you share your package.json? 🤔

Hey,
I've tried the library outside my app and works perfectly fine, so it's definitively something there o the app.
Many thanks for taking the time to check.
Will close the issue.


sync README.md files with the current state of the code

Closes #356
Closes #447

let's wait for @uniemimu's approval too ;)


Contact list

We need to display the friends list of the user so he can interact with them later on.
For now, we are in a very early stage therefore we will focus on making it work rather than making it pretty.

Seems like the idflex library already retrieves a friends list. Does it not work properly? I have been unable to actually add a contact on my POD so i couldn't test if any information was being retrieved at all from the pod

Yes, indeed. Right now we can already retrieve each friends' WebID with ldflex. There's still a few things left.

It would be nice to get a formatted version of the friend's name in order to display it to the user in the contact list
We will need to be able to access a friend's inbox once we start doing the chat itself
There's still all the presentation side of the friend list yet to be done. We need to think (and implement) how to display this information to the user so he can start chatting with one.

I have an idea for the first point. The rest is still pending, if anyone wants to explore it, that would be great.

We can probably retrieve the friend and user's inboxes no problem as long as we have their WebId's. However i am unsure how can we get the friend's WebId since we've got no session object corresponding to them?
As for the friend list, i think we could make the friend list clickable and send the user to a new screen, where the message input field and chat would be.

Closing until further modification is needed


[Deque Re-Analysis] Submission "critical" accessibility issues

Describe the bug
In Sept 2021, Deque did a reanalysis of the issues described in #1171 which were fixed in #1252.
They found the following 9 accessibility issues were either "Not Fixed" or only "Partially Fixed":
Partially Fixed Issues

(Issue ID: 469979) "Keyboard Navigation" - Arrows above/below date field (to change year/month/day) are inaccessible to keyboard navigation (you cannot tab to them).  (Mostly works now)

Deque's re-test notes: "The arrow icon buttons to increment and decrement values are receiving keyboard focus. However, they lack focus indicator and sighted keyboard users will not be able to identify the elements in focus."

(Issue ID 470065) Section headers open/close icons (up/down arrows) have incorrect names/labels.

Deque's re-test notes: "The expand/collapse arrow icon buttons are provided with aria-label="Close section". More clear name like "Expand Describe Section", "Collapse Describe Section" is recommended."

Not Fixed Issues
While Deque flagged these as "Not Fixed", some behave slightly better, but the overall accessibility problem is not fixed.

(Issue ID: 469700) "Button does not have a name or role" - "browse" link in the file drag & drop component (<input type="file">) has no "role".

Deque's re-test notes: "The Browse element is not being exposed as a button to NVDA. Please provide role="button" to the <label> element instead of on the <input> element."

(Issue ID 469984) Group of form controls is missing "group" label for the "Date of Issue" field.

Deque's re-test notes: "The three fields of day, month and year are not grouped as "Date of Issue *". Use <fieldset> element to group the elements and provide group name ("Date of Issue *") using <legend> element."

(Issue ID 469993) Two comboboxes are missing roles/attributes. Specifically "dc.type" and "dc.language.iso" fields.

Deque's re-test notes: "The comboboxes Type and Language do not have sufficient ARIA attributes. They are current being announced as edit boxes."

(Issue 470001) No focus indicator exists on File buttons (Download, Edit, Delete...and Save/Cancel after clicking Edit).

Deque's re-test notes: "Focus indicator is missing for the icon button. outline: 0; is provided to these elements which is removing the focus indicator."

(Issue ID 470010) Form field with error not identified. Occurs on the "Grant Access" fields in the Bitstream Embargo section. When an error occurs it is not linked to the field.

Deque's re-test notes: "Either include the form field name in the error message like "Grant access from date is required". Or associate the error message element with its form field using aria-describedby."

(Issue ID: 470011) Bitstream embargo fields have a keyboard trap (Upload a file -> Edit -> "Access Condition" to "embargo" or "lease" -> Attempt to tab past the "Grant access..." field). It's not possible to tab past them until you fill them out (Keyboard focus is trapped within the field).

Deque's re-test notes: "Keyboard trap is still present at Grant access from and until fields in errored state."

(Issue ID 470062) Section status icons are missing alt text. When a submission section has a green checkmark, it has all required fields filled. Otherwise it shows an orange exclamation mark. Both those status icons are missing alt text.

Deque's re-test notes: "The informative icons are provided with aria-hidden="true" incorrectly."

The above issues should be fixed

@atarix83 : I know you have a lot on your plate right now. But this ticket is the result of a re-test by Deque of the accessibility issues you tried to fix in #1252.  While a lot of the fixes in that old PR were successful, there were 9 listed here that are not quite fixed.  Many of these could be quick to fix if you or someone on your team finds time before the 7.1 release.


Add support for fitting seasonality daily

This is done in the following way:

It uses fractional days since epoch (instead of integer days)

Adds an extra feature component for daily seasonality (question: are 3 components enough for the fourier series?)

In testing in on my use case (time series with one sample every 30 minutes) it seemed to work okay.

I think this is great and is definitely something that we want for v0.2, but I'm going to hold off on merging this now since we want to keep parity between the R and Python versions and so need to implement this in R as well. There it will be a little more complicated since the datatype we currently use is only for days, and not datetime, so it will require a few more changes.

I think this is great and is definitely something that we want for v0.2, but I'm going to hold off on merging this now since we want to keep parity between the R and Python versions and so need to implement this in R as well. There it will be a little more complicated since the datatype we currently use is only for days, and not datetime, so it will require a few more changes.

Ok, thanks. My R knowledge is very minimal, so I am not sure I can take on implementing that.

No problem, we'll just leave this open in the meantime.

Thanks @hellais for this commit.
In my use case I'm trying to fit ~20 days worth of data with a 1-minute resolution. I've included this patch, and declared:
m = Prophet(daily_seasonality=True)

However, it seems to be considerably under-fitting the daily seasonality. Here is the output:

Am I doing anything wrong?
Thanks again.

@mino98 I haven't worked with 1-minute granularity data, but only with data quantised on 30-minutes.
Could you share the plot of the predictions dfs (that should show upper and lower bounds on the predictions) as obtained by running:
# df == your dataframe with columns 'ds' and 'y'
pm = Prophet(daily_seasonality=True)
pm.fit(df)
predictions = pm.predict(df[['ds', 'y']])
predictions.set_index('ds')[['y', 'yhat', 'yhat_lower', 'yhat_upper']]\
   .iplot() # Or equivalent plotting function if not using plotly

Do you mean something like this (I used matplotlib)?

@mino98
Maybe what you can try is increasing the order of the fourier series used for the daily seasonality forecasting. For my use-case order 3 was working fine, but maybe you can try using something like 10 (as in the yearly seasonality calculation).
The line you need to change is this one: https://github.com/facebookincubator/prophet/pull/135/files#diff-68340cfab7044eb1fa0c823dd808c539R397

Thanks @hellais, that's much better!

@hellais Can you tell me please how can I update this feature in python ?
Thanks.:)

@ofir887

Can you tell me please how can I update this feature in python?

If you are using pip to install python dependencies you can install this specific branch by running:
pip install https://github.com/hellais/prophet/archive/feature/daily-seasonality.zip

@hellais Thanks!
Do I need to do some modifications in order to predict every 30 minutes? (like freq & periods)?

Unfortunately this installation method fails on my computer. Is there a way to  merge this change locally, since it appears it is going to be at least another year before anyone is going to put in the effort to fix the R version?

Unfortunately this installation method fails on my computer. Is there a way to merge this change locally, since it appears it is going to be at least another year before anyone is going to put in the effort to fix the R version and thus make this pull request mergeable?

I guess that is a call for @bletham to make, I personally don't see it a big issue to have the R library misaligned (or missing features) with the python library and in the long run it's probably going to be quite a hassle to ensure both are in perfect sync.

Do I need to do some modifications in order to predict every 30 minutes? (like freq & periods)?

I have used this branch as is to make predictions on time that is quantized to 30 minutes. You can find a full working example here: https://ooni.torproject.org/notebooks/internet-blackouts-google-traffic.html

Thanks for the new feature!

In this code the daily and hourly seasonality is added:
forecaster.py
`# Copyright (c) 2017-present, Facebook, Inc.
All rights reserved.

This source code is licensed under the BSD-style license found in the
LICENSE file in the root directory of this source tree. An additional grant
of patent rights can be found in the PATENTS file in the same directory.
from future import absolute_import
from future import division
from future import print_function
from future import unicode_literals
from collections import defaultdict
from datetime import timedelta
import logging
logger = logging.getLogger(name)
from matplotlib import pyplot as plt
from matplotlib.dates import MonthLocator, num2date
from matplotlib.ticker import FuncFormatter
import numpy as np
import pandas as pd
fb-block 1 start
from fbprophet.models import prophet_stan_models
fb-block 1 end
try:
import pystan
except ImportError:
logger.error('You cannot run prophet without pystan installed')
raise
fb-block 2
class Prophet(object):
"""Prophet forecaster.
Parameters
----------
growth: String 'linear' or 'logistic' to specify a linear or logistic
    trend.
changepoints: List of dates at which to include potential changepoints. If
    not specified, potential changepoints are selected automatically.
n_changepoints: Number of potential changepoints to include. Not used
    if input `changepoints` is supplied. If `changepoints` is not supplied,
    then n_changepoints potential changepoints are selected uniformly from
    the first 80 percent of the history.
yearly_seasonality: Fit yearly seasonality. Can be 'auto', True, or False.
weekly_seasonality: Fit weekly seasonality. Can be 'auto', True, or False.
holidays: pd.DataFrame with columns holiday (string) and ds (date type)
    and optionally columns lower_window and upper_window which specify a
    range of days around the date to be included as holidays.
    lower_window=-2 will include 2 days prior to the date as holidays.
seasonality_prior_scale: Parameter modulating the strength of the
    seasonality model. Larger values allow the model to fit larger seasonal
    fluctuations, smaller values dampen the seasonality.
holidays_prior_scale: Parameter modulating the strength of the holiday
    components model.
changepoint_prior_scale: Parameter modulating the flexibility of the
    automatic changepoint selection. Large values will allow many
    changepoints, small values will allow few changepoints.
mcmc_samples: Integer, if greater than 0, will do full Bayesian inference
    with the specified number of MCMC samples. If 0, will do MAP
    estimation.
interval_width: Float, width of the uncertainty intervals provided
    for the forecast. If mcmc_samples=0, this will be only the uncertainty
    in the trend using the MAP estimate of the extrapolated generative
    model. If mcmc.samples>0, this will be integrated over all model
    parameters, which will include uncertainty in seasonality.
uncertainty_samples: Number of simulated draws used to estimate
    uncertainty intervals.
"""

def __init__(
        self,
        growth='linear',
        changepoints=None,
        n_changepoints=25,
        yearly_seasonality='auto',
        weekly_seasonality='auto',
        daily_seasonality='auto',
        hourly_seasonality='auto',
        holidays=None,
        seasonality_prior_scale=10.0,
        holidays_prior_scale=10.0,
        changepoint_prior_scale=0.05,
        mcmc_samples=0,
        interval_width=0.80,
        uncertainty_samples=1000,

):
    self.growth = growth

    self.changepoints = pd.to_datetime(changepoints)
    if self.changepoints is not None:
        self.n_changepoints = len(self.changepoints)
    else:
        self.n_changepoints = n_changepoints

    self.yearly_seasonality = yearly_seasonality
    self.weekly_seasonality = weekly_seasonality
    self.daily_seasonality = daily_seasonality
    self.hourly_seasonality = hourly_seasonality

    if holidays is not None:
        if not (
            isinstance(holidays, pd.DataFrame)
            and 'ds' in holidays
            and 'holiday' in holidays
        ):
            raise ValueError("holidays must be a DataFrame with 'ds' and "
                             "'holiday' columns.")
        holidays['ds'] = pd.to_datetime(holidays['ds'])
    self.holidays = holidays

    self.seasonality_prior_scale = float(seasonality_prior_scale)
    self.changepoint_prior_scale = float(changepoint_prior_scale)
    self.holidays_prior_scale = float(holidays_prior_scale)

    self.mcmc_samples = mcmc_samples
    self.interval_width = interval_width
    self.uncertainty_samples = uncertainty_samples

    # Set during fitting
    self.start = None
    self.y_scale = None
    self.t_scale = None
    self.changepoints_t = None
    self.stan_fit = None
    self.params = {}
    self.history = None
    self.history_dates = None
    self.validate_inputs()

def hola(x):
    return x

def validate_inputs(self):
    """Validates the inputs to Prophet."""
    if self.growth not in ('linear', 'logistic'):
        raise ValueError(
            "Parameter 'growth' should be 'linear' or 'logistic'.")
    if self.holidays is not None:
        has_lower = 'lower_window' in self.holidays
        has_upper = 'upper_window' in self.holidays
        if has_lower + has_upper == 1:
            raise ValueError('Holidays must have both lower_window and ' +
                             'upper_window, or neither')
        if has_lower:
            if max(self.holidays['lower_window']) > 0:
                raise ValueError('Holiday lower_window should be <= 0')
            if min(self.holidays['upper_window']) < 0:
                raise ValueError('Holiday upper_window should be >= 0')
        for h in self.holidays['holiday'].unique():
            if '_delim_' in h:
                raise ValueError('Holiday name cannot contain "_delim_"')
            if h in ['zeros', 'yearly', 'weekly', 'yhat', 'seasonal',
                     'trend']:
                raise ValueError('Holiday name {} reserved.'.format(h))

def setup_dataframe(self, df, initialize_scales=False):
    """Prepare dataframe for fitting or predicting.

    Adds a time index and scales y. Creates auxillary columns 't', 't_ix',
    'y_scaled', and 'cap_scaled'. These columns are used during both
    fitting and predicting.

    Parameters
    ----------
    df: pd.DataFrame with columns ds, y, and cap if logistic growth.
    initialize_scales: Boolean set scaling factors in self from df.

    Returns
    -------
    pd.DataFrame prepared for fitting or predicting.
    """
    if 'y' in df:
        df['y'] = pd.to_numeric(df['y'])
    df['ds'] = pd.to_datetime(df['ds'])

    df = df.sort_values('ds')
    df.reset_index(inplace=True, drop=True)

    if initialize_scales:
        self.y_scale = df['y'].max()
        self.start = df['ds'].min()
        self.t_scale = df['ds'].max() - self.start

    df['t'] = (df['ds'] - self.start) / self.t_scale
    if 'y' in df:
        df['y_scaled'] = df['y'] / self.y_scale

    if self.growth == 'logistic':
        assert 'cap' in df
        df['cap_scaled'] = df['cap'] / self.y_scale

    return df

def set_changepoints(self):
    """Set changepoints

    Sets m$changepoints to the dates of changepoints. Either:
    1) The changepoints were passed in explicitly.
        A) They are empty.
        B) They are not empty, and need validation.
    2) We are generating a grid of them.
    3) The user prefers no changepoints be used.
    """
    if self.changepoints is not None:
        if len(self.changepoints) == 0:
            pass
        else:
            too_low = min(self.changepoints) < self.history['ds'].min()
            too_high = max(self.changepoints) > self.history['ds'].max()
            if too_low or too_high:
                raise ValueError('Changepoints must fall within training data.')
    elif self.n_changepoints > 0:
        # Place potential changepoints evenly throuh first 80% of history
        max_ix = np.floor(self.history.shape[0] * 0.8)
        cp_indexes = (
            np.linspace(0, max_ix, self.n_changepoints + 1)
            .round()
            .astype(np.int)
        )
        self.changepoints = self.history.ix[cp_indexes]['ds'].tail(-1)
    else:
        # set empty changepoints
        self.changepoints = []
    if len(self.changepoints) > 0:
        self.changepoints_t = np.sort(np.array(
            (self.changepoints - self.start) / self.t_scale))
    else:
        self.changepoints_t = np.array([0])  # dummy changepoint

def get_changepoint_matrix(self):
    """Gets changepoint matrix for history dataframe."""
    A = np.zeros((self.history.shape[0], len(self.changepoints_t)))
    for i, t_i in enumerate(self.changepoints_t):
        A[self.history['t'].values >= t_i, i] = 1
    return A

@staticmethod
def fourier_series(dates, period, series_order):
    """Provides Fourier series components with the specified frequency
    and order.

    Parameters
    ----------
    dates: pd.Series containing timestamps.
    period: Number of days of the period.
    series_order: Number of components.

    Returns
    -------
    Matrix with seasonality features.
    """
    # convert to days since epoch
    t = np.array(
        (dates - pd.datetime(1970, 1, 1))
        .dt.days
        .astype(np.float)
    )
    return np.column_stack([
        fun((2.0 * (i + 1) * np.pi * t / period))
        for i in range(series_order)
        for fun in (np.sin, np.cos)
    ])

@classmethod
def make_seasonality_features(cls, dates, period, series_order, prefix):
    """Data frame with seasonality features.

    Parameters
    ----------
    cls: Prophet class.
    dates: pd.Series containing timestamps.
    period: Number of days of the period.
    series_order: Number of components.
    prefix: Column name prefix.

    Returns
    -------
    pd.DataFrame with seasonality features.
    """
    features = cls.fourier_series(dates, period, series_order)
    columns = [
        '{}_delim_{}'.format(prefix, i + 1)
        for i in range(features.shape[1])
    ]
    return pd.DataFrame(features, columns=columns)

def make_holiday_features(self, dates):
    """Construct a dataframe of holiday features.

    Parameters
    ----------
    dates: pd.Series containing timestamps used for computing seasonality.

    Returns
    -------
    pd.DataFrame with a column for each holiday.
    """
    # A smaller prior scale will shrink holiday estimates more
    scale_ratio = self.holidays_prior_scale / self.seasonality_prior_scale
    # Holds columns of our future matrix.
    expanded_holidays = defaultdict(lambda: np.zeros(dates.shape[0]))
    # Makes an index so we can perform `get_loc` below.
    row_index = pd.DatetimeIndex(dates)

    for _ix, row in self.holidays.iterrows():
        dt = row.ds.date()
        try:
            lw = int(row.get('lower_window', 0))
            uw = int(row.get('upper_window', 0))
        except ValueError:
            lw = 0
            uw = 0
        for offset in range(lw, uw + 1):
            occurrence = dt + timedelta(days=offset)
            try:
                loc = row_index.get_loc(occurrence)
            except KeyError:
                loc = None

            key = '{}_delim_{}{}'.format(
                row.holiday,
                '+' if offset >= 0 else '-',
                abs(offset)
            )
            if loc is not None:
                expanded_holidays[key][loc] = scale_ratio
            else:
                # Access key to generate value
                expanded_holidays[key]

    # This relies pretty importantly on pandas keeping the columns in order.
    return pd.DataFrame(expanded_holidays)

def make_all_seasonality_features(self, df):
    """Dataframe with seasonality features.

    Parameters
    ----------
    df: pd.DataFrame with dates for computing seasonality features.

    Returns
    -------
    pd.DataFrame with seasonality.
    """
    seasonal_features = [
        # Add a column of zeros in case no seasonality is used.
        pd.DataFrame({'zeros': np.zeros(df.shape[0])})
    ]

    # Seasonality features
    if self.yearly_seasonality:
        seasonal_features.append(self.make_seasonality_features(
            df['ds'],
            365.25,
            10,
            'yearly',
        ))

    if self.weekly_seasonality:
        seasonal_features.append(self.make_seasonality_features(
            df['ds'],
            7,
            3,
            'weekly',
        ))
    if self.daily_seasonality:
        seasonal_features.append(self.make_seasonality_features(
            df['ds'],
            1,
            3,
            'daily',
        ))

    if self.hourly_seasonality:
        seasonal_features.append(self.make_seasonality_features(
            df['ds'],
            1 / 24,
            2,
            'hourly',
        ))

    if self.holidays is not None:
        seasonal_features.append(self.make_holiday_features(df['ds']))
    return pd.concat(seasonal_features, axis=1)

def set_auto_seasonalities(self):
    """Set seasonalities that were left on auto.

    Turns on yearly seasonality if there is >=2 years of history.
    Turns on weekly seasonality if there is >=2 weeks of history, and the
    spacing between dates in the history is <7 days.

    Turns on hourly seasonality if there is >=2 hours of history, and the
    spacing between dates in the history is <1 hour.
    """
    first = self.history['ds'].min()
    last = self.history['ds'].max()
    if self.yearly_seasonality == 'auto':
        if last - first < pd.Timedelta(days=730):
            self.yearly_seasonality = False
            logger.info('Disabling yearly seasonality. Run prophet with '
                        'yearly_seasonality=True to override this.')
        else:
            self.yearly_seasonality = True
    if self.weekly_seasonality == 'auto':
        dt = self.history['ds'].diff()
        min_dt = dt.iloc[dt.nonzero()[0]].min()
        if ((last - first < pd.Timedelta(weeks=2)) or
                (min_dt >= pd.Timedelta(weeks=1))):
            self.weekly_seasonality = False
            logger.info('Disabling weekly seasonality. Run prophet with '
                        'weekly_seasonality=True to override this.')
        else:
            self.weekly_seasonality = True

    if self.daily_seasonality == 'auto':
        dt = self.history['ds'].diff()
        min_dt = dt.iloc[dt.nonzero()[0]].min()
        if ((last - first < pd.Timedelta(days=2)) or
                (min_dt >= pd.Timedelta(days=1))):
            self.daily_seasonality = False
            print('Disabling daily seasonality. Run prophet with '
                  'daily_seasonality=True to override this.')
        else:
            self.daily_seasonality = True

    if self.hourly_seasonality == 'auto':
        dt = self.history['ds'].diff()
        min_dt = dt.iloc[dt.nonzero()[0]].min()
        if ((last - first < pd.Timedelta(hours=2)) or
                (min_dt >= pd.Timedelta(hours=1))):
            self.hourly_seasonality = False
            print('Disabling hourly seasonality. Run prophet with '
                  'hourly_seasonality=True to override this.')
        else:
            self.hourly_seasonality = True

@staticmethod
def linear_growth_init(df):
    """Initialize linear growth.

    Provides a strong initialization for linear growth by calculating the
    growth and offset parameters that pass the function through the first
    and last points in the time series.

    Parameters
    ----------
    df: pd.DataFrame with columns ds (date), y_scaled (scaled time series),
        and t (scaled time).

    Returns
    -------
    A tuple (k, m) with the rate (k) and offset (m) of the linear growth
    function.
    """
    i0, i1 = df['ds'].idxmin(), df['ds'].idxmax()
    T = df['t'].ix[i1] - df['t'].ix[i0]
    k = (df['y_scaled'].ix[i1] - df['y_scaled'].ix[i0]) / T
    m = df['y_scaled'].ix[i0] - k * df['t'].ix[i0]
    return (k, m)

@staticmethod
def logistic_growth_init(df):
    """Initialize logistic growth.

    Provides a strong initialization for logistic growth by calculating the
    growth and offset parameters that pass the function through the first
    and last points in the time series.

    Parameters
    ----------
    df: pd.DataFrame with columns ds (date), cap_scaled (scaled capacity),
        y_scaled (scaled time series), and t (scaled time).

    Returns
    -------
    A tuple (k, m) with the rate (k) and offset (m) of the logistic growth
    function.
    """
    i0, i1 = df['ds'].idxmin(), df['ds'].idxmax()
    T = df['t'].ix[i1] - df['t'].ix[i0]

    # Force valid values, in case y > cap.
    r0 = max(1.01, df['cap_scaled'].ix[i0] / df['y_scaled'].ix[i0])
    r1 = max(1.01, df['cap_scaled'].ix[i1] / df['y_scaled'].ix[i1])

    if abs(r0 - r1) <= 0.01:
        r0 = 1.05 * r0

    L0 = np.log(r0 - 1)
    L1 = np.log(r1 - 1)

    # Initialize the offset
    m = L0 * T / (L0 - L1)
    # And the rate
    k = L0 / m
    return (k, m)

# fb-block 7
def fit(self, df, **kwargs):
    """Fit the Prophet model.

    This sets self.params to contain the fitted model parameters. It is a
    dictionary parameter names as keys and the following items:
        k (Mx1 array): M posterior samples of the initial slope.
        m (Mx1 array): The initial intercept.
        delta (MxN array): The slope change at each of N changepoints.
        beta (MxK matrix): Coefficients for K seasonality features.
        sigma_obs (Mx1 array): Noise level.
    Note that M=1 if MAP estimation.

    Parameters
    ----------
    df: pd.DataFrame containing the history. Must have columns ds (date
        type) and y, the time series. If self.growth is 'logistic', then
        df must also have a column cap that specifies the capacity at
        each ds.
    kwargs: Additional arguments passed to the optimizing or sampling
        functions in Stan.

    Returns
    -------
    The fitted Prophet object.
    """
    if self.history is not None:
        raise Exception('Prophet object can only be fit once. '
                        'Instantiate a new object.')
    history = df[df['y'].notnull()].copy()
    self.history_dates = pd.to_datetime(df['ds']).sort_values()

    history = self.setup_dataframe(history, initialize_scales=True)
    self.history = history
    self.set_auto_seasonalities()
    seasonal_features = self.make_all_seasonality_features(history)

    self.set_changepoints()
    A = self.get_changepoint_matrix()

    dat = {
        'T': history.shape[0],
        'K': seasonal_features.shape[1],
        'S': len(self.changepoints_t),
        'y': history['y_scaled'],
        't': history['t'],
        'A': A,
        't_change': self.changepoints_t,
        'X': seasonal_features,
        'sigma': self.seasonality_prior_scale,
        'tau': self.changepoint_prior_scale,
    }

    if self.growth == 'linear':
        kinit = self.linear_growth_init(history)
    else:
        dat['cap'] = history['cap_scaled']
        kinit = self.logistic_growth_init(history)

    model = prophet_stan_models[self.growth]

    def stan_init():
        return {
            'k': kinit[0],
            'm': kinit[1],
            'delta': np.zeros(len(self.changepoints_t)),
            'beta': np.zeros(seasonal_features.shape[1]),
            'sigma_obs': 1,
        }

    if self.mcmc_samples > 0:
        stan_fit = model.sampling(
            dat,
            init=stan_init,
            iter=self.mcmc_samples,
            **kwargs
        )
        for par in stan_fit.model_pars:
            self.params[par] = stan_fit[par]

    else:
        params = model.optimizing(dat, init=stan_init, iter=1e4, **kwargs)
        for par in params:
            self.params[par] = params[par].reshape((1, -1))

    # If no changepoints were requested, replace delta with 0s
    if len(self.changepoints) == 0:
        # Fold delta into the base rate k
        self.params['k'] = self.params['k'] + self.params['delta']
        self.params['delta'] = np.zeros(self.params['delta'].shape)

    return self

# fb-block 8
def predict(self, df=None):
    """Predict using the prophet model.

    Parameters
    ----------
    df: pd.DataFrame with dates for predictions (column ds), and capacity
        (column cap) if logistic growth. If not provided, predictions are
        made on the history.

    Returns
    -------
    A pd.DataFrame with the forecast components.
    """
    if df is None:
        df = self.history.copy()
    else:
        df = self.setup_dataframe(df)

    df['trend'] = self.predict_trend(df)
    seasonal_components = self.predict_seasonal_components(df)
    intervals = self.predict_uncertainty(df)

    df2 = pd.concat((df, intervals, seasonal_components), axis=1)
    df2['yhat'] = df2['trend'] + df2['seasonal']
    return df2

@staticmethod
def piecewise_linear(t, deltas, k, m, changepoint_ts):
    """Evaluate the piecewise linear function.

    Parameters
    ----------
    t: np.array of times on which the function is evaluated.
    deltas: np.array of rate changes at each changepoint.
    k: Float initial rate.
    m: Float initial offset.
    changepoint_ts: np.array of changepoint times.

    Returns
    -------
    Vector y(t).
    """
    # Intercept changes
    gammas = -changepoint_ts * deltas
    # Get cumulative slope and intercept at each t
    k_t = k * np.ones_like(t)
    m_t = m * np.ones_like(t)
    for s, t_s in enumerate(changepoint_ts):
        indx = t >= t_s
        k_t[indx] += deltas[s]
        m_t[indx] += gammas[s]
    return k_t * t + m_t

@staticmethod
def piecewise_logistic(t, cap, deltas, k, m, changepoint_ts):
    """Evaluate the piecewise logistic function.

    Parameters
    ----------
    t: np.array of times on which the function is evaluated.
    cap: np.array of capacities at each t.
    deltas: np.array of rate changes at each changepoint.
    k: Float initial rate.
    m: Float initial offset.
    changepoint_ts: np.array of changepoint times.

    Returns
    -------
    Vector y(t).
    """
    # Compute offset changes
    k_cum = np.concatenate((np.atleast_1d(k), np.cumsum(deltas) + k))
    gammas = np.zeros(len(changepoint_ts))
    for i, t_s in enumerate(changepoint_ts):
        gammas[i] = (
            (t_s - m - np.sum(gammas))
            * (1 - k_cum[i] / k_cum[i + 1])
        )
    # Get cumulative rate and offset at each t
    k_t = k * np.ones_like(t)
    m_t = m * np.ones_like(t)
    for s, t_s in enumerate(changepoint_ts):
        indx = t >= t_s
        k_t[indx] += deltas[s]
        m_t[indx] += gammas[s]
    return cap / (1 + np.exp(-k_t * (t - m_t)))

def predict_trend(self, df):
    """Predict trend using the prophet model.

    Parameters
    ----------
    df: Prediction dataframe.

    Returns
    -------
    Vector with trend on prediction dates.
    """
    k = np.nanmean(self.params['k'])
    m = np.nanmean(self.params['m'])
    deltas = np.nanmean(self.params['delta'], axis=0)

    t = np.array(df['t'])
    if self.growth == 'linear':
        trend = self.piecewise_linear(t, deltas, k, m, self.changepoints_t)
    else:
        cap = df['cap_scaled']
        trend = self.piecewise_logistic(
            t, cap, deltas, k, m, self.changepoints_t)

    return trend * self.y_scale

def predict_seasonal_components(self, df):
    """Predict seasonality broken down into components.

    Parameters
    ----------
    df: Prediction dataframe.

    Returns
    -------
    Dataframe with seasonal components.
    """
    seasonal_features = self.make_all_seasonality_features(df)
    lower_p = 100 * (1.0 - self.interval_width) / 2
    upper_p = 100 * (1.0 + self.interval_width) / 2

    components = pd.DataFrame({
        'col': np.arange(seasonal_features.shape[1]),
        'component': [x.split('_delim_')[0] for x in seasonal_features.columns],
    })
    # Remove the placeholder
    components = components[components['component'] != 'zeros']

    if components.shape[0] > 0:
        X = seasonal_features.as_matrix()
        data = {}
        for component, features in components.groupby('component'):
            cols = features.col.tolist()
            comp_beta = self.params['beta'][:, cols]
            comp_features = X[:, cols]
            comp = (
                np.matmul(comp_features, comp_beta.transpose())
                * self.y_scale
            )
            data[component] = np.nanmean(comp, axis=1)
            data[component + '_lower'] = np.nanpercentile(comp, lower_p,
                                                          axis=1)
            data[component + '_upper'] = np.nanpercentile(comp, upper_p,
                                                          axis=1)

        component_predictions = pd.DataFrame(data)
        component_predictions['seasonal'] = (
            component_predictions[components['component'].unique()].sum(1))
    else:
        component_predictions = pd.DataFrame(
            {'seasonal': np.zeros(df.shape[0])})
    return component_predictions

def predict_uncertainty(self, df):
    """Predict seasonality broken down into components.

    Parameters
    ----------
    df: Prediction dataframe.

    Returns
    -------
    Dataframe with uncertainty intervals.
    """
    n_iterations = self.params['k'].shape[0]
    samp_per_iter = max(1, int(np.ceil(
        self.uncertainty_samples / float(n_iterations)
    )))

    # Generate seasonality features once so we can re-use them.
    seasonal_features = self.make_all_seasonality_features(df)

    sim_values = {'yhat': [], 'trend': [], 'seasonal': []}
    for i in range(n_iterations):
        for _j in range(samp_per_iter):
            sim = self.sample_model(df, seasonal_features, i)
            for key in sim_values:
                sim_values[key].append(sim[key])

    lower_p = 100 * (1.0 - self.interval_width) / 2
    upper_p = 100 * (1.0 + self.interval_width) / 2

    series = {}
    for key, value in sim_values.items():
        mat = np.column_stack(value)
        series['{}_lower'.format(key)] = np.nanpercentile(mat, lower_p,
                                                          axis=1)
        series['{}_upper'.format(key)] = np.nanpercentile(mat, upper_p,
                                                          axis=1)

    return pd.DataFrame(series)

def sample_model(self, df, seasonal_features, iteration):
    """Simulate observations from the extrapolated generative model.

    Parameters
    ----------
    df: Prediction dataframe.
    seasonal_features: pd.DataFrame of seasonal features.
    iteration: Int sampling iteration to use parameters from.

    Returns
    -------
    Dataframe with trend, seasonality, and yhat, each like df['t'].
    """
    trend = self.sample_predictive_trend(df, iteration)

    beta = self.params['beta'][iteration]
    seasonal = np.matmul(seasonal_features.as_matrix(), beta) * self.y_scale

    sigma = self.params['sigma_obs'][iteration]
    noise = np.random.normal(0, sigma, df.shape[0]) * self.y_scale

    return pd.DataFrame({
        'yhat': trend + seasonal + noise,
        'trend': trend,
        'seasonal': seasonal,
    })

def sample_predictive_trend(self, df, iteration):
    """Simulate the trend using the extrapolated generative model.

    Parameters
    ----------
    df: Prediction dataframe.
    seasonal_features: pd.DataFrame of seasonal features.
    iteration: Int sampling iteration to use parameters from.

    Returns
    -------
    np.array of simulated trend over df['t'].
    """
    k = self.params['k'][iteration]
    m = self.params['m'][iteration]
    deltas = self.params['delta'][iteration]

    t = np.array(df['t'])
    T = t.max()

    if T > 1:
        # Get the time discretization of the history
        dt = np.diff(self.history['t'])
        dt = np.min(dt[dt > 0])
        # Number of time periods in the future
        N = np.ceil((T - 1) / float(dt))
        S = len(self.changepoints_t)

        prob_change = min(1, (S * (T - 1)) / N)
        n_changes = np.random.binomial(N, prob_change)

        # Sample ts
        changepoint_ts_new = sorted(np.random.uniform(1, T, n_changes))
    else:
        # Case where we're not extrapolating.
        changepoint_ts_new = []
        n_changes = 0

    # Get the empirical scale of the deltas, plus epsilon to avoid NaNs.
    lambda_ = np.mean(np.abs(deltas)) + 1e-8

    # Sample deltas
    deltas_new = np.random.laplace(0, lambda_, n_changes)

    # Prepend the times and deltas from the history
    changepoint_ts = np.concatenate((self.changepoints_t,
                                     changepoint_ts_new))
    deltas = np.concatenate((deltas, deltas_new))

    if self.growth == 'linear':
        trend = self.piecewise_linear(t, deltas, k, m, changepoint_ts)
    else:
        cap = df['cap_scaled']
        trend = self.piecewise_logistic(t, cap, deltas, k, m,
                                        changepoint_ts)

    return trend * self.y_scale

def make_future_dataframe(self, periods, freq='D', include_history=True):
    """Simulate the trend using the extrapolated generative model.

    Parameters
    ----------
    periods: Int number of periods to forecast forward.
    freq: Any valid frequency for pd.date_range, such as 'D' or 'M'.
    include_history: Boolean to include the historical dates in the data
        frame for predictions.

    Returns
    -------
    pd.Dataframe that extends forward from the end of self.history for the
    requested number of periods.
    """
    last_date = self.history_dates.max()
    dates = pd.date_range(
        start=last_date,
        periods=periods + 1,  # An extra in case we include start
        freq=freq)
    dates = dates[dates > last_date]  # Drop start if equals last_date
    dates = dates[:periods]  # Return correct number of periods

    if include_history:
        dates = np.concatenate((np.array(self.history_dates), dates))

    return pd.DataFrame({'ds': dates})

def plot(self, fcst, ax=None, uncertainty=True, plot_cap=True, xlabel='ds',
         ylabel='y'):
    """Plot the Prophet forecast.

    Parameters
    ----------
    fcst: pd.DataFrame output of self.predict.
    ax: Optional matplotlib axes on which to plot.
    uncertainty: Optional boolean to plot uncertainty intervals.
    plot_cap: Optional boolean indicating if the capacity should be shown
        in the figure, if available.
    xlabel: Optional label name on X-axis
    ylabel: Optional label name on Y-axis

    Returns
    -------
    A matplotlib figure.
    """
    if ax is None:
        fig = plt.figure(facecolor='w', figsize=(10, 6))
        ax = fig.add_subplot(111)
    else:
        fig = ax.get_figure()
    ax.plot(self.history['ds'].values, self.history['y'], 'k.')
    ax.plot(fcst['ds'].values, fcst['yhat'], ls='-', c='#0072B2')
    if 'cap' in fcst and plot_cap:
        ax.plot(fcst['ds'].values, fcst['cap'], ls='--', c='k')
    if uncertainty:
        ax.fill_between(fcst['ds'].values, fcst['yhat_lower'],
                        fcst['yhat_upper'], color='#0072B2',
                        alpha=0.2)
    ax.grid(True, which='major', c='gray', ls='-', lw=1, alpha=0.2)
    ax.set_xlabel(xlabel)
    ax.set_ylabel(ylabel)
    fig.tight_layout()
    return fig

def hola(x):
    return x

def plot_components(self, fcst, uncertainty=True, plot_cap=True,
                    hourly_start=0, weekly_start=0, yearly_start=0):
    """Plot the Prophet forecast components.

    Will plot whichever are available of: trend, holidays, weekly
    seasonality, and yearly seasonality.

    Parameters
    ----------
    fcst: pd.DataFrame output of self.predict.
    uncertainty: Optional boolean to plot uncertainty intervals.
    plot_cap: Optional boolean indicating if the capacity should be shown
        in the figure, if available.
    weekly_start: Optional int specifying the start day of the weekly
        seasonality plot. 0 (default) starts the week on Sunday. 1 shifts
        by 1 day to Monday, and so on.
    yearly_start: Optional int specifying the start day of the yearly
        seasonality plot. 0 (default) starts the year on Jan 1. 1 shifts
        by 1 day to Jan 2, and so on.

    Returns
    -------
    A matplotlib figure.
    """
    # Identify components to be plotted
    components = [('trend', True),
                  ('holidays', self.holidays is not None),
                  ('weekly', 'weekly' in fcst),
                  ('yearly', 'yearly' in fcst)]
    components = [plot for plot, cond in components if cond]
    npanel = len(components)

    fig, axes = plt.subplots(npanel, 1, facecolor='w',
                             figsize=(9, 3 * npanel))

    for ax, plot in zip(axes, components):
        if plot == 'trend':
            self.plot_trend(
                fcst, ax=ax, uncertainty=uncertainty, plot_cap=plot_cap)
        elif plot == 'holidays':
            self.plot_holidays(fcst, ax=ax, uncertainty=uncertainty)
        elif plot == 'weekly':
            self.plot_weekly(
                ax=ax, uncertainty=uncertainty, weekly_start=weekly_start)
        elif plot == 'yearly':
            self.plot_yearly(
                ax=ax, uncertainty=uncertainty, yearly_start=yearly_start)
        elif plot == 'hourly':
            self.plot_hourly(
                ax=ax, uncertainty=uncertainty, hourly_start=hourly_start)

    fig.tight_layout()
    return fig

def plot_trend(self, fcst, ax=None, uncertainty=True, plot_cap=True):
    """Plot the trend component of the forecast.

    Parameters
    ----------
    fcst: pd.DataFrame output of self.predict.
    ax: Optional matplotlib Axes to plot on.
    uncertainty: Optional boolean to plot uncertainty intervals.
    plot_cap: Optional boolean indicating if the capacity should be shown
        in the figure, if available.

    Returns
    -------
    a list of matplotlib artists
    """

    artists = []
    if not ax:
        fig = plt.figure(facecolor='w', figsize=(10, 6))
        ax = fig.add_subplot(111)
    artists += ax.plot(fcst['ds'].values, fcst['trend'], ls='-',
                       c='#0072B2')
    if 'cap' in fcst and plot_cap:
        artists += ax.plot(fcst['ds'].values, fcst['cap'], ls='--', c='k')
    if uncertainty:
        artists += [ax.fill_between(
            fcst['ds'].values, fcst['trend_lower'], fcst['trend_upper'],
            color='#0072B2', alpha=0.2)]
    ax.grid(True, which='major', c='gray', ls='-', lw=1, alpha=0.2)
    ax.set_xlabel('ds')
    ax.set_ylabel('trend')
    return artists

def plot_holidays(self, fcst, ax=None, uncertainty=True):
    """Plot the holidays component of the forecast.

    Parameters
    ----------
    fcst: pd.DataFrame output of self.predict.
    ax: Optional matplotlib Axes to plot on. One will be created if this
        is not provided.
    uncertainty: Optional boolean to plot uncertainty intervals.

    Returns
    -------
    a list of matplotlib artists
    """
    artists = []
    if not ax:
        fig = plt.figure(facecolor='w', figsize=(10, 6))
        ax = fig.add_subplot(111)
    holiday_comps = self.holidays['holiday'].unique()
    y_holiday = fcst[holiday_comps].sum(1)
    y_holiday_l = fcst[[h + '_lower' for h in holiday_comps]].sum(1)
    y_holiday_u = fcst[[h + '_upper' for h in holiday_comps]].sum(1)
    # NOTE the above CI calculation is incorrect if holidays overlap
    # in time. Since it is just for the visualization we will not
    # worry about it now.
    artists += ax.plot(fcst['ds'].values, y_holiday, ls='-',
                       c='#0072B2')
    if uncertainty:
        artists += [ax.fill_between(fcst['ds'].values,
                                    y_holiday_l, y_holiday_u,
                                    color='#0072B2', alpha=0.2)]
    ax.grid(True, which='major', c='gray', ls='-', lw=1, alpha=0.2)
    ax.set_xlabel('ds')
    ax.set_ylabel('holidays')
    return artists

def plot_hourly(self, ax=None, uncertainty=True, hourly_start=0):
    """Plot the weekly component of the forecast.

            Parameters
            ----------
            ax: Optional matplotlib Axes to plot on. One will be created if this
                is not provided.
            uncertainty: Optional boolean to plot uncertainty intervals.
            hourly_start: Optional int specifying the start hour of the hourly
                seasonality plot. 0 (default) starts the day at Midnight. 1 shifts
                by 1 hour to 01:00 (AM), and so on.

            Returns
            -------
            a list of matplotlib artists
            """
    artists = []
    if not ax:
        fig = plt.figure(facecolor='w', figsize=(10, 6))
        ax = fig.add_subplot(111)
    # Compute weekly seasonality for a Sun-Sat sequence of dates.
    hours = (pd.date_range(start='2017-01-01', periods=7) +
            pd.Timedelta(hours=hourly_start))
    df_w = pd.DataFrame({'ds': hours, 'cap': 1.})
    df_w = self.setup_dataframe(df_w)
    seas = self.predict_seasonal_components(df_w)
    hours = hours.hours
    artists += ax.plot(range(len(hours)), seas['weekly'], ls='-',
                       c='#0072B2')
    if uncertainty:
        artists += [ax.fill_between(range(len(hours)),
                                    seas['weekly_lower'], seas['weekly_upper'],
                                    color='#0072B2', alpha=0.2)]
    ax.grid(True, which='major', c='gray', ls='-', lw=1, alpha=0.2)
    ax.set_xticks(range(len(hours)))
    ax.set_xticklabels(hours)
    ax.set_xlabel('Hour')
    ax.set_ylabel('hourly')
    return artists

def plot_weekly(self, ax=None, uncertainty=True, weekly_start=0):
    """Plot the weekly component of the forecast.

    Parameters
    ----------
    ax: Optional matplotlib Axes to plot on. One will be created if this
        is not provided.
    uncertainty: Optional boolean to plot uncertainty intervals.
    weekly_start: Optional int specifying the start day of the weekly
        seasonality plot. 0 (default) starts the week on Sunday. 1 shifts
        by 1 day to Monday, and so on.

    Returns
    -------
    a list of matplotlib artists
    """
    artists = []
    if not ax:
        fig = plt.figure(facecolor='w', figsize=(10, 6))
        ax = fig.add_subplot(111)
    # Compute weekly seasonality for a Sun-Sat sequence of dates.
    days = (pd.date_range(start='2017-01-01', periods=7) +
            pd.Timedelta(days=weekly_start))
    df_w = pd.DataFrame({'ds': days, 'cap': 1.})
    df_w = self.setup_dataframe(df_w)
    seas = self.predict_seasonal_components(df_w)
    days = days.weekday_name
    artists += ax.plot(range(len(days)), seas['weekly'], ls='-',
                       c='#0072B2')
    if uncertainty:
        artists += [ax.fill_between(range(len(days)),
                                    seas['weekly_lower'], seas['weekly_upper'],
                                    color='#0072B2', alpha=0.2)]
    ax.grid(True, which='major', c='gray', ls='-', lw=1, alpha=0.2)
    ax.set_xticks(range(len(days)))
    ax.set_xticklabels(days)
    ax.set_xlabel('Day of week')
    ax.set_ylabel('weekly')
    return artists

def plot_yearly(self, ax=None, uncertainty=True, yearly_start=0):
    """Plot the yearly component of the forecast.

    Parameters
    ----------
    ax: Optional matplotlib Axes to plot on. One will be created if
        this is not provided.
    uncertainty: Optional boolean to plot uncertainty intervals.
    yearly_start: Optional int specifying the start day of the yearly
        seasonality plot. 0 (default) starts the year on Jan 1. 1 shifts
        by 1 day to Jan 2, and so on.

    Returns
    -------
    a list of matplotlib artists
    """
    artists = []
    if not ax:
        fig = plt.figure(facecolor='w', figsize=(10, 6))
        ax = fig.add_subplot(111)
    # Compute yearly seasonality for a Jan 1 - Dec 31 sequence of dates.
    df_y = pd.DataFrame(
        {'ds': pd.date_range(start='2017-01-01', periods=365) +
         pd.Timedelta(days=yearly_start), 'cap': 1.})
    df_y = self.setup_dataframe(df_y)
    seas = self.predict_seasonal_components(df_y)
    artists += ax.plot(df_y['ds'], seas['yearly'], ls='-',
                       c='#0072B2')
    if uncertainty:
        artists += [ax.fill_between(
            df_y['ds'].values, seas['yearly_lower'],
            seas['yearly_upper'], color='#0072B2', alpha=0.2)]
    ax.grid(True, which='major', c='gray', ls='-', lw=1, alpha=0.2)
    months = MonthLocator(range(1, 13), bymonthday=1, interval=2)
    ax.xaxis.set_major_formatter(FuncFormatter(
        lambda x, pos=None: '{dt:%B} {dt.day}'.format(dt=num2date(x))))
    ax.xaxis.set_major_locator(months)
    ax.set_xlabel('Day of year')
    ax.set_ylabel('yearly')
    return artists`

In this code the daily and hourly seasonality is added:

You may want to look into using the github pull request feature.


Include the artifactory-request-out log

Including the artifactory-request-out log for monitoring calls to remotes in Datadog

@MahithaB - I opened this PR a long time ago as we rely on this config as part of our Artifactory deployment. At the moment we patch in this log, but would be great to get this PR in so we can remove our patch.

@cawilson thank you for the PR. We've added the changes to include artifactory-request-out log in the config.


Update game.js

Added a timeout for bot  moves and animation bot will also play the sequence

updated simon game also want to add the start btn for mobile devices and make it responsive


Benchmark sometimes fail on macOS

Description
Benchmarks are performance sensitive and sometimes fail on macOS.
Reproduction Steps
Expected behavior
Benchmarks always pass.
Environment Information

OS: macOS
Toolchain/Compiler: Clang11
Version: 7bae86f

Additional context
Probably widen the range.

Not happening so far. Closing for now.

Happened again, should be resolved for good.


Throw an Exception if genesis block doesn't match

@Geod24 in general I don't like the way the code works. The setGenesis(new) scope (exit) setGenesis(old) reminds me a lot about enableGagging in DMD. Just the use of globals seems bad..
And in retrospect having the genesis block be defined in a config file is error-prone. If the file is corrupt, or if a user copy-pastes a configuration from some malicious website, they may end up on the wrong chain.
I'll have to think about a better alternative to setGenesisBlock..

So we discussed this IRL with @AndrejMitrovic and talked about providing CLI switches instead. Where are we on this ?

See the comments in #821

Updated, using TestLedger now.


update project READMEs

We should update the READMEs for each of the projects to help people get setup easily.

It would be nice to have a wiki where we could list out possible ecological function/human uses. Stuff like that.


Change default value for check_aux_coords argument of concatenate_cube()

To maintain compatibility with previous versions of Iris, this PR changes the default value of the check_aux_coords argument of CubeList.concatenate_cube() from True to False

@djkirkham the change is good, but you need to target the v1.10.x branch...

New PR: https://github.com/SciTools/iris/pull/2109