Jordan Petridis
098c5755b0
hammond-data: Do batch indexing of new_episodes.
2018-01-18 13:47:40 +02:00
Jordan Petridis
93372a30d0
1.21 Gigawatts. Remove non-future indexing loop.
...
Tried to have a seperate futures loop but it's too confusign having
too write a Trait2, functon2, etc version of everything and keep it together.
Futures are functional sort of, so the synchronous versioun can be removed.
It still needs a ton of work though to be ready, or even get near the perf
of of the sync+rayon version.
2018-01-18 11:38:24 +02:00
Jordan Petridis
5d88998180
hammond_data: Add an EpisodeMinimal Diesel model.
2018-01-17 10:21:16 +02:00
Jordan Petridis
574cfae5c6
rustfmt: enable reorder imports.
2018-01-17 08:57:02 +02:00
Jordan Petridis
e707087e72
hammond_data: Split models::queriables into multiple modules.
2018-01-17 08:45:14 +02:00
Jordan Petridis
3a6f6d8931
hammond_data: Split models::insertables into multiple modules.
2018-01-17 08:27:39 +02:00
Jordan Petridis
3766db2b14
hammond_data: Move parse_podcast into a NewEpisode method.
2018-01-17 07:31:24 +02:00
Jordan Petridis
8174fe0bac
hammond_data: Move parse_episode from parser into a NewEpisode method.
2018-01-17 07:16:59 +02:00
Jordan Petridis
bd9844f012
Dbquerries: Use map_err() instead of ? into Ok() pattern.
2018-01-17 07:13:34 +02:00
Jordan Petridis
f64779f70a
Parser: Add NewEpisodeMinimal struct.
...
Parsing whole episodes can be expensive and we only need
a subset to determine if it should be indexed/updated or ignored.
2018-01-16 18:33:32 +02:00
Jordan Petridis
978e5a61f6
Dbquerries: add EXIST querries for podcast and episode.
2018-01-16 17:26:41 +02:00
Jordan Petridis
fa03c20b00
Feed: add parse_channel_items_future method.
2018-01-16 14:52:20 +02:00
Jordan Petridis
bf4f655ed2
Pipeline: remove submodule hack.
2018-01-16 14:37:51 +02:00
Jordan Petridis
b3460b15a2
hammond_data::Feed: add parse_podcast_future method.
2018-01-16 13:44:33 +02:00
Jordan Petridis
7f78e87551
hammond_data::Feed: Remove unused parts.
2018-01-16 12:35:08 +02:00
Jordan Petridis
3358fcd0b3
hammond_data::Feed: general cleanup of no longer needed stuff.
2018-01-15 11:03:40 +02:00
Jordan Petridis
10345ffda7
Pipeline: add matching on status code.
2018-01-15 10:16:01 +02:00
Jordan Petridis
2f7a22355f
Minor cleanup of the mess.
2018-01-14 08:27:50 +02:00
Jordan Petridis
e63a366fdc
I don't know why the benchmarks keep cycling.
2018-01-14 06:48:42 +02:00
Jordan Petridis
1031315cdd
Getting somewhere?
2018-01-14 06:37:47 +02:00
Jordan Petridis
a5fd79e220
I hate futures error handling.
2018-01-13 14:47:38 +02:00
Jordan Petridis
e162f8fd3f
Non-working yet benchmark.
2018-01-13 08:31:16 +02:00
Jordan Petridis
e4d77a6ba4
Cleanup the id() method mess of the diesel models.
2018-01-13 07:46:56 +02:00
Jordan Petridis
6517956987
This works somehow...
2018-01-13 07:09:59 +02:00
Jordan Petridis
ee9cede921
hammond_data: Remove Source dependancy from Feed struct.
2018-01-13 06:08:51 +02:00
Jordan Petridis
1dd25f91fd
Minor cleanup
2018-01-13 05:47:23 +02:00
Jordan Petridis
771b7b3804
Something Something futures.
2018-01-13 01:28:02 +02:00
Jordan Petridis
8e367b7e86
Dont ask me how this is working.
2018-01-12 08:50:16 +02:00
Jordan Petridis
955845110b
Merge branch 'master' into 33-downloader-re-work
2018-01-09 12:06:38 +02:00
Jordan Petridis
ea70addbc6
Removed some unwrap()s.
2018-01-09 09:21:07 +02:00
Jordan Petridis
345d4b3865
Check more http status codes when parsing feeds.
2018-01-08 02:41:37 +02:00
Jordan Petridis
81b1ec810c
Apply clippy suggestions.
2018-01-07 08:36:02 +02:00
Jordan Petridis
074284d286
Move unsub show logic and get_download_folder func to hammond-data::utils.
2018-01-07 08:26:38 +02:00
Jordan Petridis
bb5c25d87f
Write unit tests for the itunes_duration_extension parser.
2018-01-06 03:09:24 +02:00
Jordan Petridis
e9dd297bf3
Merge branch 'master' into 33-downloader-re-work
2018-01-05 21:43:41 +02:00
Jordan Petridis
eafad35860
Upgrade diesel to 1.0.0 and update rest dependancies.
2018-01-05 19:57:11 +02:00
Jordan Petridis
2d71a99121
Change the auto-updater and checkup startup scheduling.
2018-01-02 20:47:49 +02:00
Jordan Petridis
37e9b6fbf0
'How hard could it be'
2018-01-02 07:23:53 +02:00
Jordan Petridis
785a5f80c6
hammond-data::utils: Small and minor refactor.
2017-12-31 01:42:28 +02:00
Jordan Petridis
e727734443
DB Schema: Remove episode.published_date column.
2017-12-30 22:23:10 +02:00
Jordan Petridis
5730e71e2a
Clear TODO and FIXME that are no longer releavant.
2017-12-30 20:54:23 +02:00
Jordan Petridis
036292284d
hammond-data: Do not clean source url's cause some feeds use queries as identifiers for some reason.
2017-12-29 20:21:36 +02:00
Jordan Petridis
ca06a16bd9
Closes #2 . Kudo to @jwykeham for the fix!.
2017-12-29 02:45:02 +02:00
Jordan Petridis
1d78ab7c42
Cargo upgrade.
2017-12-28 23:19:36 +02:00
Jordan Petridis
8a7d6d9f1b
Add an empty CHANGELOG that follows keepachangelog.com format. Closes #23 .
2017-12-28 22:41:22 +02:00
Jordan Petridis
7d7b09ff0f
hammond-data::parser: Prefer itunes_summary over channel.description if it exists. Finishes #20 .
2017-12-27 17:59:40 +02:00
Jordan Petridis
1f0a2b5c6c
hammond-data::parser: Prefer itunes_summary over item.description if it exists. Finishes #20 .
2017-12-27 17:40:31 +02:00
Jordan Petridis
bfb74c4dba
hammond-data: Index_loop does not wait for GET request to finish now.
2017-12-26 18:18:48 +02:00
Jordan Petridis
933fec55a5
hammond-data::parser: Refactor Episode tests.
2017-12-26 13:40:52 +02:00
Jordan Petridis
fde4bedce6
hammond-data::parser: Refactor Podcast tests.
2017-12-25 23:29:42 +02:00
Jordan Petridis
2d6f02c407
EpisodeWidget: Do not display size if it's 0 bytes.
2017-12-23 15:44:28 +02:00
Jordan Petridis
31b19dd88e
Just things apple force you to do.
2017-12-22 18:28:18 +02:00
Jordan Petridis
4512790f2d
hammond-data: Add duration column to the episode table.
2017-12-22 17:30:07 +02:00
Jordan Petridis
5541b18a6a
hammond-data::parser: Add itunesext_duration parser.
2017-12-22 16:55:45 +02:00
Jordan Petridis
b820ee4db7
hammond-data::parser : Do not clean image urls cause feeds put redirects for uri for some reason.
2017-12-22 14:51:57 +02:00
Jordan Petridis
e416bca963
Implemented a pixbuf cache mechanism.
...
Since gdk_pixbuf::Pixbuf is refference counted and every episode,
use the cover of the Podcast Feed/Show, We can only create a Pixbuf
cover per show and pass around the Rc pointer.
GObjects do not implement Send trait, so SendCell is a way around that.
Also lazy_static requires Sync trait, so that's what the mutexes are.
2017-12-21 17:36:07 +02:00
Jordan Petridis
db59bed69d
hammond-data::utils: Modified EpisodeCleaner Diesel model to also be used with played_cleaner.
2017-12-20 19:18:20 +02:00
Jordan Petridis
3c24b9f9d9
hammond-data::utils: Added new Diesel model for the download checker.
2017-12-20 19:00:14 +02:00
Jordan Petridis
2e06205eda
Define a new Diesel Model for PodcastCover querries.
...
Define new Diesel Model and impl From<Podcast> trait,
Change the signature of downloader::cache_image function,
Change and merge hammond-gtk::utils::get_pixbuf_from_path functions.
2017-12-20 18:19:31 +02:00
Jordan Petridis
914cad72f5
EpisodesView: Custom Diesel model not really necessary.
2017-12-19 20:45:40 +02:00
Jordan Petridis
895591f628
EpisodesView: Use show cover image.
2017-12-19 13:19:38 +02:00
Jordan Petridis
ad9a932143
Fix diesel query.
2017-12-19 12:56:16 +02:00
Jordan Petridis
f602e8c36d
EpisodesView: Create a DieselModel for EpisodeViewWidget.
2017-12-19 11:04:44 +02:00
Jordan Petridis
61bd7893c7
EpisodeWidget: Remove Podcast depndancy from new() method.
2017-12-18 19:10:50 +02:00
Jordan Petridis
f7af05a650
EpisodeWidget: Dim/Grey out widget if episode is played.
2017-12-16 20:30:25 +02:00
Jordan Petridis
0ac78fcff1
Added date label into EpisodeWidget.
2017-12-14 18:03:37 +02:00
Jordan Petridis
afdb79b712
Set the progress bar into activity mode.
2017-12-14 13:38:18 +02:00
Jordan Petridis
8fe6b526a5
Add a new Diesel Model for the EpisodeWidget.
2017-12-14 12:01:35 +02:00
Jordan Petridis
ad7f5013f3
Extend ShowsMachine functionality.
2017-12-11 11:15:54 +02:00
Jordan Petridis
851be1fcfb
Schema hotfix.
2017-12-10 06:47:10 +02:00
Jordan Petridis
d8281c70a5
Better naming of a dbquery function.
2017-12-10 06:11:03 +02:00
Jordan Petridis
bc57c33491
Change episode table schema.
2017-12-09 17:38:46 +02:00
Jordan Petridis
999a2a1fc1
Strip whitespace during parsing.
2017-12-09 10:22:09 +02:00
Jordan Petridis
b528c48e3b
Run rustfmt.
2017-12-09 05:57:16 +02:00
Jordan Petridis
48e61c6377
Added small a utility that eats extra whitespace.
2017-12-07 18:31:45 +02:00
Jordan Petridis
c07d240532
Sanitize html during feed parsing.
2017-12-07 06:38:31 +02:00
Jordan Petridis
05e056481f
Take advantage of <Into> trait from the generated builders.
2017-12-05 09:28:13 +02:00
Jordan Petridis
8e6cc4d2f2
Remove custom builders in favor of derive_builder crate.
2017-12-05 09:10:02 +02:00
Jordan Petridis
7727bc5ec3
Remove unnecessary Arc.
2017-12-02 01:46:37 +02:00
Jordan Petridis
74a1aae168
Upgraded diesel to 0.99 and switched from the git master into crated.io.
...
Also did a cargo upgrade of all the dependancies.
Notable upgrades are r2d2, lazystatic 1.0, loggerv 0.6 and rss 1.2
2017-12-02 01:39:28 +02:00
Jordan Petridis
9312366f62
Enable more lints.
2017-12-02 00:06:42 +02:00
Jordan Petridis
2051f0f58e
Enable some extra clippy warnings.
2017-11-28 09:50:30 +02:00
Jordan Petridis
c07215d357
Set hammond-data to build with Deny missing_docs.
2017-11-28 09:44:26 +02:00
Jordan Petridis
e4a7a7becf
Add module Doc comments.
2017-11-28 09:38:05 +02:00
Jordan Petridis
7dd8755bcb
Document hammond-data/src/lib.rs xdg_dirs module.
2017-11-28 09:24:12 +02:00
Jordan Petridis
76f9f36609
Document Source Model.
2017-11-28 09:01:38 +02:00
Jordan Petridis
4d669ef297
Document Podcast Model.
2017-11-28 08:40:27 +02:00
Jordan Petridis
3707b70b41
Document Episode Model.
2017-11-28 08:25:04 +02:00
Jordan Petridis
d81f9c6ad7
Fixed stable unit tests.
2017-11-28 04:42:15 +02:00
Jordan Petridis
34e808182c
Documented a bit hammond-data/src/feed.rs.
2017-11-27 06:06:14 +02:00
Jordan Petridis
c4030e51df
Added some doc comments in hammond-data/src/utils.rs.
2017-11-27 05:25:16 +02:00
Jordan Petridis
cb65782829
Extended the NewModel builder and added a bunch of unit tests.
2017-11-27 04:32:21 +02:00
Jordan Petridis
ee4f08c5cf
Implemented a BuildPattern for NewModels. Swiched stuff to pub(crate).
...
Added a Constructor for NewModels in order to make their fields private.
Also added getter methods that are mainly used in Unit tests.
2017-11-26 23:30:56 +02:00
Jordan Petridis
9e03aaae27
Upgrade gtk-rs crates.
2017-11-26 21:25:46 +02:00
Jordan Petridis
866904a687
Use generic IntoIterator instead of &[foo] slices.
2017-11-26 07:03:21 +02:00
Jordan Petridis
9d0df9de46
Tweak cargo privacy settings.
2017-11-26 06:21:14 +02:00
Jordan Petridis
7eff93ecf1
Remove rand from the downloader.
2017-11-26 03:47:43 +02:00
Jordan Petridis
6336f7cac1
Moved update funtions into a Trait and impl for NewModels.
2017-11-26 00:56:38 +02:00
Jordan Petridis
39dff5e27a
Moved insert functions into an Insert trait and implemented it for each NewModel.
2017-11-26 00:40:03 +02:00
Jordan Petridis
ce3a76aee1
Update episode rows instead of replacing them.
2017-11-26 00:13:51 +02:00
Jordan Petridis
2f801043c1
In place update a podcast row instead of replace.
2017-11-25 18:22:56 +02:00
Jordan Petridis
f83f894e44
Replaced pool.get().unwrap() statements with get()?.
2017-11-25 03:29:06 +02:00
Jordan Petridis
f0a0fdfa83
Added a truncate_db helper function.
2017-11-25 03:01:21 +02:00
Jordan Petridis
62da8bbb52
Use source_id as the unique identifieble field for the podcast table.
2017-11-25 01:56:10 +02:00
Jordan Petridis
12ffe5c231
Use a db transaction when deleting a feed.
2017-11-24 20:40:32 +02:00
Jordan Petridis
d9ef4fed53
Truncate the temp db before running test_complete_index. Should close #17 .
2017-11-24 20:31:10 +02:00
Jordan Petridis
4a195eb1b7
Moved set_played_now into a method of the Episode model.
2017-11-24 19:48:20 +02:00
Jordan Petridis
f7f0a16858
Minor cleanup.
2017-11-24 19:28:34 +02:00
Jordan Petridis
91f4a30c01
Moved episode indexing into a transaction.
2017-11-23 21:30:17 +02:00
Jordan Petridis
5570fdd118
Switched to using r2d2 instead of a databaseConnection object.
2017-11-23 21:09:50 +02:00
Jordan Petridis
aa7c493e81
hammond-data/src/feed.rs api changes.
2017-11-23 18:48:41 +02:00
Jordan Petridis
a58671ea9a
Moved database related things into their own module.
2017-11-22 21:54:09 +02:00
Jordan Petridis
02b27d33b6
Move XDG global variables into their own module.
2017-11-22 21:07:21 +02:00
Jordan Petridis
3bcb23b39a
Redefine NewModels to hold Option<String> instead of Option<&str>.
2017-11-22 09:53:52 +02:00
Jordan Petridis
31328355b1
Methods renaming, Api changes.
2017-11-22 06:25:34 +02:00
Jordan Petridis
9bc6df2cba
Made Disel NewModels private.
2017-11-22 05:50:25 +02:00
Jordan Petridis
7be0b5bbf8
Added a url cleanr. Closes #4 .
2017-11-20 22:06:29 +02:00
Jordan Petridis
5890fe2bb4
Moved all diesel queries into hammond-data/src/dbquerries.rs module.
2017-11-20 20:05:00 +02:00
Jordan Petridis
0ce4107be4
Revert back to using an Arc<Mutex<Connection>> instead of the pool.
2017-11-20 18:09:53 +02:00
Jordan Petridis
ce29602431
I hate everything.
2017-11-20 16:57:27 +02:00
Jordan Petridis
e07683c046
partially-fixed the tests and benchmarks.
2017-11-20 01:08:34 +02:00
Jordan Petridis
5ad52fa4b2
Refactored to remove from Database refferences.
2017-11-19 23:32:37 +02:00
Jordan Petridis
beaeeffba8
Refactored dbquerries.
...
Refactored so dbquerries.rs module *should* be the only piece of code
that interacts with a Database connection. Insert stuff will be moved too.
2017-11-19 23:08:36 +02:00
Jordan Petridis
47b32ac000
Removed the Database type alias.
2017-11-19 22:39:51 +02:00
Jordan Petridis
55442529a8
Initial switch to using r2d2.
2017-11-19 22:06:10 +02:00
Jordan Petridis
a6dbf65575
Move stuff from hammond_data/src/lib.rs into utils.
2017-11-18 20:55:19 +02:00
Jordan Petridis
563f249a48
Reverted index_channel items to the synchronous version.
...
Using a transaction yields ~1/3 better performance as of now.
2017-11-18 19:57:50 +02:00
Jordan Petridis
5c84b77434
Converted iter_mut into into_iter wherever possible.
2017-11-18 19:23:25 +02:00
Jordan Petridis
494761beaf
Removed id function from diesel models.
...
It's automaticly declared upon derive identifiable,
And there's no need to overwrite it.
2017-11-18 15:23:40 +02:00
Jordan Petridis
1bf5187e48
Renamed hammond-data/src/feedparser.rs module into parser.rs.
2017-11-18 15:05:55 +02:00
Jordan Petridis
6894f33895
Renamed hammond-data/src/index_feeds.rs module into feed.rs.
2017-11-18 14:32:58 +02:00
Jordan Petridis
450643371c
Renamed index_feeds and fetch_feeds functions.
2017-11-18 14:24:52 +02:00
Jordan Petridis
1ec4370c7b
Moved Database type-alias into hammond-data/src/lib.rs.
2017-11-17 21:37:55 +02:00
Jordan Petridis
e66a337468
Moved indexing episode into a NewEpisode method.
...
There is a performance reggresion caused by the introduction of
Arc<Mutex<Connection>> instead of Connection that should be refactored.
Also removed the db transcaction as it was incomplete.
2017-11-17 20:01:59 +02:00
Jordan Petridis
282a29e7dd
Move some indexing functions into methods of insertable models.
2017-11-17 18:58:44 +02:00
Jordan Petridis
8eca46871d
Moved insert_return functions into methods of insertable models.
2017-11-17 18:10:29 +02:00
Jordan Petridis
2152aa9034
Convert Feed struct-tuple into a plain struct.
2017-11-17 16:24:48 +02:00
Jordan Petridis
3d108d4a7a
Make Feed fields private.
2017-11-17 16:12:11 +02:00
Jordan Petridis
54a0f17588
Initial draft of hammond-data/src/index_feed.rs API redesign.
2017-11-17 13:26:39 +02:00
Jordan Petridis
3c6176b1c0
Renamed hammond-data/src/dbcheckup.rs into utils.rs
2017-11-16 23:32:40 +02:00
Jordan Petridis
cfb876e006
Split hammond-data/models into sub modules.
2017-11-15 17:42:22 +02:00
Jordan Petridis
118846f255
Added extra columns to podcast and episode tables.
2017-11-15 16:58:21 +02:00
Jordan Petridis
a286014a62
Refactored indexing functions of the Diesel models.
2017-11-14 17:56:28 +02:00
Jordan Petridis
6996759db5
Removed the file open syscall from the benchmark.
2017-11-14 15:02:32 +02:00
Jordan Petridis
58e11f17b7
Use bigger rss feeds for benchmarking.
2017-11-14 14:49:43 +02:00
Jordan Petridis
edd694e7d6
Added initial feed indexing benchmark.
2017-11-14 14:09:13 +02:00
Jordan Petridis
03637903c6
Update dependancies.
2017-11-14 13:06:09 +02:00
Jordan Petridis
fe571df1d5
Changed the XDG folder from Hammond to hammond, Added make tar.
2017-11-10 14:09:05 +02:00
Jordan Petridis
1fbacad8b9
Updated rayon.
2017-11-10 11:42:11 +02:00
Jordan Petridis
07c1395c29
Cleanup of downloader module.
2017-11-09 17:47:38 +02:00
