toxblh/podcasts
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
1,193 Commits 6 Branches 16 Tags 9.2 MiB
0.4.0
Commit Graph

375 Commits

Author SHA1 Message Date
Jordan Petridis
e84e7df3f1
Remove unwrap call. 2018-02-05 15:59:34 +02:00
Jordan Petridis
8ba9f928d6
Switch to using DataError instead of failure::Error. 2018-02-05 02:43:56 +02:00
Jordan Petridis
435ce05ac7
Add rest of the From impls for the Error Type. 2018-02-05 01:53:01 +02:00
Jordan Petridis
ede4c21e30
Merge the Errors into a single Enum. 2018-02-05 01:39:50 +02:00
Jordan Petridis
0892fe26ba
Manually implement From Trait for the DatabaseError. 2018-02-05 01:20:01 +02:00
Jordan Petridis
1085d468b9
cargo fmt 2018-02-04 18:45:19 +02:00
Jordan Petridis
de43cae015
Switch rest stuff of data/downloader to Failure Crate. 2018-02-04 17:36:27 +02:00
Jordan Petridis
f9f015a211
Hammond-data: switch database module to use Failure. 2018-02-04 16:33:10 +02:00
Jordan Petridis
d131c279b9
Add batch indexing fallback. 
If an RSS feed triggered a unique constrains violation, none of it's
items would be indexed resulting in a Podcast without episodes.

This adds a fallback that indexes each item individually if the batch index fails.
 2018-02-03 16:02:42 +02:00
Jordan Petridis
92d01ec51a
Feed: enchance get_stuff unit test. 2018-02-03 15:55:26 +02:00
Jordan Petridis
e8ae0b0251
Thats a hack, a terrible one, but without it the user 
would have to manually click refresh if the user that was added
happen to redirect somewhere else.

The problem is in the hammond-data/src/models/source.rs
fn request_constructor. It has to do with futures and lifetimes
and I am not sure how to tackle it yet.
 2018-02-03 00:06:02 +02:00
Jordan Petridis
80fd4e9fc5
Source: Set Etag and last modified to None upon redirect. 2018-02-01 19:35:35 +02:00
Jordan Petridis
7b62ef203d
Source: Pass owenership around instead of &mut, cause futures. 2018-02-01 15:55:19 +02:00
Jordan Petridis
2f062afb07
Clippy suggestions 2018-01-29 19:09:53 +02:00
Jordan Petridis
815639fe5f
Add an order by tittle asc clasuse to the Podcast query. Address #34. 2018-01-29 12:30:13 +02:00
Jordan Petridis
b1c3b4f9cc
Pipeline: Fetch sources without etag support fist as they are more likely to need parsing. 2018-01-29 12:20:21 +02:00
Jordan Petridis
332a439b7a
hammond-data: Factor out save method of diesel models into a Trait. 2018-01-28 22:09:06 +02:00
Jordan Petridis
fb5264c479
NewSource: Rename into_source -> to_source and take reffence &self instead. 2018-01-27 18:34:03 +02:00
Jordan Petridis
a3705d424b
NewSource: Remove Insert implemantation and replace it with a insert_or_ignore method. 2018-01-27 18:09:40 +02:00
Jordan Petridis
77a52bdc8c
hammond-data: Add a source_exists query. 2018-01-27 14:23:19 +02:00
Jordan Petridis
18e55e23ee
Headerbar: Do not allow insertion of invalid urls. 2018-01-27 14:11:02 +02:00
Jordan Petridis
d14973cf0d
Use num_cpus instead of hardcoding the number of threads for dns look_up. 
In the next hyper_tls relase it should be possible to use the existing
cpu_pool executor.
 2018-01-27 11:08:57 +02:00
Jordan Petridis
120d494280
Update hyper and futures. 2018-01-27 09:45:25 +02:00
Jordan Petridis
39aa86157b
Switch rest of the tests to point to the internet archive urls. 2018-01-26 17:25:35 +02:00
Jordan Petridis
df85a4ea9e
Pipeline: avoid panicking by checking the lenght and returning an error. 2018-01-26 14:17:43 +02:00
Jordan Petridis
2912bad110
Pipeline: Split the pipeline from the cpu-pool and the tokio-core. 2018-01-26 13:34:04 +02:00
Jordan Petridis
b370af35d4
Feed: Add a bunch of tests. 2018-01-24 18:42:33 +02:00
Jordan Petridis
d6817aaa1f
Parser: pass an rss::ItunesExtension to instead of an rss::Item. 2018-01-24 15:42:08 +02:00
Jordan Petridis
5da002fe6d
Use map_err(From::from) instead of ? into Ok() wrapping pattern. 2018-01-24 15:27:36 +02:00
Jordan Petridis
0e16f0acb0
NewEpisode: rename into_episode -> to_episode, change its signature and add unit test. 2018-01-24 14:31:33 +02:00
Jordan Petridis
d5a7fa9de8
NewEpisode: Add index method unit test. 2018-01-24 12:11:49 +02:00
Jordan Petridis
9274aba849
NewEpisode: Add update method unit test. 2018-01-24 11:22:03 +02:00
Jordan Petridis
457e43978f
Pipeline: Fix a panic if the list of futures was 0. 2018-01-23 19:07:11 +02:00
Jordan Petridis
f9096e5fac
Pipeline: Print the error right away instead of waiting till all futures complete. 2018-01-23 18:18:56 +02:00
Jordan Petridis
e74a2df27f
Source: On Http 301 response, update Source.uri . 2018-01-23 16:43:02 +02:00
Jordan Petridis
1c4138c166
Benches: Remove dead stuff. 2018-01-23 13:24:08 +02:00
Jordan Petridis
050fe9c52b
Pipeline: offload more stuff to the threadpool. 2018-01-23 11:43:37 +02:00
Jordan Petridis
8095919bbe
This sort of works? 2018-01-23 10:12:32 +02:00
Jordan Petridis
f7f6087d70
NewPodcast: On update unit test assert .id and .source_id fields stay the same. 2018-01-22 21:09:06 +02:00
Jordan Petridis
ea75d50053
NewEpisode: Add insert unit test. 2018-01-22 17:25:23 +02:00
Jordan Petridis
44d7a4e222
NewEpisodeMinimal: Add unit test for into_new_episode method. 2018-01-22 15:49:58 +02:00
Jordan Petridis
2d6f34fa4d
NewEpisode: Clean up tests a bit. 2018-01-22 15:34:57 +02:00
Jordan Petridis
d231cc165f
Fix batch indexing. 2018-01-22 14:51:31 +02:00
Jordan Petridis
7bd361fbbb
Upgrade dependancies. 2018-01-22 11:47:17 +02:00
Jordan Petridis
66e816fc5f
Upgrade Diesel to 1.1 2018-01-22 11:22:01 +02:00
Jordan Petridis
20f36a67a0
Upgrade log and loggerv. 2018-01-21 15:22:13 +02:00
Jordan Petridis
eb75e68b72
Fix benchmarks. 2018-01-21 14:52:21 +02:00
Jordan Petridis
dc9a28c0c7
Remove xml files. 2018-01-21 14:49:54 +02:00
Jordan Petridis
843feabf25
NewEpisode: Move the parsing tests, delete old xml files. 2018-01-21 13:35:35 +02:00
Jordan Petridis
937bd61d5e
NewPodcast: Add unit test for the to_podcast() method. 2018-01-21 12:17:47 +02:00
Jordan Petridis
4ee5888b7f
NewPodcast: Rename into_podcast method -> to_podcast since it now does not take owenership. 2018-01-21 11:04:15 +02:00
Jordan Petridis
73eeb1b545
NewPodcast: Add NewPodcast.index unit test. 2018-01-21 10:04:53 +02:00
Jordan Petridis
16bbfb5478
NewPodcast: Add (lackign) NewPodcast.update unit test. 2018-01-21 09:35:48 +02:00
Jordan Petridis
410657c98b
NewPodcast: Remove the boilerplate on the unit tests. 2018-01-21 08:57:50 +02:00
Jordan Petridis
1625f773c2
hammond-data: Fix PartialEq implementations, add NewPodcast.insert unit test. 2018-01-21 08:39:50 +02:00
Jordan Petridis
da8c3a7827
Implement PartialEq accross Insertable and Queryable diesel models. 2018-01-20 19:09:25 +02:00
Jordan Petridis
6c9e0984a5
Migrate podcast tests to use the internet archive snapshots. 2018-01-20 18:02:07 +02:00
Jordan Petridis
df17a10014
Source: refactor into_feed test. 2018-01-20 16:43:32 +02:00
Jordan Petridis
ef52a026bc
Feed: Add Builder pattern. 2018-01-20 08:46:05 +02:00
Jordan Petridis
7d9781052f
hammond-data: Rework the modules privacy. 2018-01-20 07:28:30 +02:00
Jordan Petridis
855b1517a7
Feed: Removee unimplemented! hack to guard against my future stupidity. 2018-01-19 19:20:21 +02:00
Jordan Petridis
1606ceaadb
Source: split response_to_channel into more futures. 2018-01-19 16:29:50 +02:00
Jordan Petridis
46b6c0d27e
hammond-data: Remove leftover synchronous code. 2018-01-19 15:41:37 +02:00
Jordan Petridis
b92ba7be76
Switch from libtest to criterion.rs for benchmarks. 2018-01-19 14:32:43 +02:00
Jordan Petridis
44dbb06dcc
Add benchmarks for the async path. 2018-01-19 13:15:21 +02:00
Jordan Petridis
e6b0cfccb5
Apply clippy suggestions. 2018-01-19 10:32:25 +02:00
Jordan Petridis
5c5faafc72
Source: Move around code and change some methods signature. 
request_constructor is now a Source method.
update_etags now takes &mut self instead of mut self.
update_etags now uses source setters methods instead of raw fields.
changed the naming of into_feed to to_feed according to rust convention.
 2018-01-19 07:51:55 +02:00
Jordan Petridis
9dc555cad7
Feed: Add an index_async implementation. 2018-01-18 18:57:44 +02:00
Jordan Petridis
1394366f91
hammond-data: Delete some dead code. 2018-01-18 17:04:57 +02:00
Jordan Petridis
6abf2535b0
hammond_data: Make index a trait that requires Insert + Update. 2018-01-18 16:33:07 +02:00
Jordan Petridis
1c96288178
hammond_data: Add more benchmark cases. 2018-01-18 15:50:48 +02:00
Jordan Petridis
098c5755b0
hammond-data: Do batch indexing of new_episodes. 2018-01-18 13:47:40 +02:00
Jordan Petridis
93372a30d0
1.21 Gigawatts. Remove non-future indexing loop. 
Tried to have a seperate futures loop but it's too confusign having
too write a Trait2, functon2, etc version of everything and keep it together.

Futures are functional sort of, so the synchronous versioun can be removed.
It still needs a ton of work though to be ready, or even get near the perf
of of the sync+rayon version.
 2018-01-18 11:38:24 +02:00
Jordan Petridis
5d88998180
hammond_data: Add an EpisodeMinimal Diesel model. 2018-01-17 10:21:16 +02:00
Jordan Petridis
574cfae5c6
rustfmt: enable reorder imports. 2018-01-17 08:57:02 +02:00
Jordan Petridis
e707087e72
hammond_data: Split models::queriables into multiple modules. 2018-01-17 08:45:14 +02:00
Jordan Petridis
3a6f6d8931
hammond_data: Split models::insertables into multiple modules. 2018-01-17 08:27:39 +02:00
Jordan Petridis
3766db2b14
hammond_data: Move parse_podcast into a NewEpisode method. 2018-01-17 07:31:24 +02:00
Jordan Petridis
8174fe0bac
hammond_data: Move parse_episode from parser into a NewEpisode method. 2018-01-17 07:16:59 +02:00
Jordan Petridis
bd9844f012
Dbquerries: Use map_err() instead of ? into Ok() pattern. 2018-01-17 07:13:34 +02:00
Jordan Petridis
f64779f70a
Parser: Add NewEpisodeMinimal struct. 
Parsing whole episodes can be expensive and we only need
a subset to determine if it should be indexed/updated or ignored.
 2018-01-16 18:33:32 +02:00
Jordan Petridis
978e5a61f6
Dbquerries: add EXIST querries for podcast and episode. 2018-01-16 17:26:41 +02:00
Jordan Petridis
fa03c20b00
Feed: add parse_channel_items_future method. 2018-01-16 14:52:20 +02:00
Jordan Petridis
bf4f655ed2
Pipeline: remove submodule hack. 2018-01-16 14:37:51 +02:00
Jordan Petridis
b3460b15a2
hammond_data::Feed: add parse_podcast_future method. 2018-01-16 13:44:33 +02:00
Jordan Petridis
7f78e87551
hammond_data::Feed: Remove unused parts. 2018-01-16 12:35:08 +02:00
Jordan Petridis
3358fcd0b3
hammond_data::Feed: general cleanup of no longer needed stuff. 2018-01-15 11:03:40 +02:00
Jordan Petridis
10345ffda7
Pipeline: add matching on status code. 2018-01-15 10:16:01 +02:00
Jordan Petridis
2f7a22355f
Minor cleanup of the mess. 2018-01-14 08:27:50 +02:00
Jordan Petridis
e63a366fdc
I don't know why the benchmarks keep cycling. 2018-01-14 06:48:42 +02:00
Jordan Petridis
1031315cdd
Getting somewhere? 2018-01-14 06:37:47 +02:00
Jordan Petridis
a5fd79e220
I hate futures error handling. 2018-01-13 14:47:38 +02:00
Jordan Petridis
e162f8fd3f
Non-working yet benchmark. 2018-01-13 08:31:16 +02:00
Jordan Petridis
e4d77a6ba4
Cleanup the id() method mess of the diesel models. 2018-01-13 07:46:56 +02:00
Jordan Petridis
6517956987
This works somehow... 2018-01-13 07:09:59 +02:00
Jordan Petridis
ee9cede921
hammond_data: Remove Source dependancy from Feed struct. 2018-01-13 06:08:51 +02:00
Jordan Petridis
1dd25f91fd
Minor cleanup 2018-01-13 05:47:23 +02:00
Jordan Petridis
771b7b3804
Something Something futures. 2018-01-13 01:28:02 +02:00
Jordan Petridis
8e367b7e86
Dont ask me how this is working. 2018-01-12 08:50:16 +02:00
Jordan Petridis
955845110b
Merge branch 'master' into 33-downloader-re-work 2018-01-09 12:06:38 +02:00
Jordan Petridis
ea70addbc6
Removed some unwrap()s. 2018-01-09 09:21:07 +02:00
Jordan Petridis
345d4b3865
Check more http status codes when parsing feeds. 2018-01-08 02:41:37 +02:00
Jordan Petridis
81b1ec810c
Apply clippy suggestions. 2018-01-07 08:36:02 +02:00
Jordan Petridis
074284d286
Move unsub show logic and get_download_folder func to hammond-data::utils. 2018-01-07 08:26:38 +02:00
Jordan Petridis
bb5c25d87f
Write unit tests for the itunes_duration_extension parser. 2018-01-06 03:09:24 +02:00
Jordan Petridis
e9dd297bf3
Merge branch 'master' into 33-downloader-re-work 2018-01-05 21:43:41 +02:00
Jordan Petridis
eafad35860
Upgrade diesel to 1.0.0 and update rest dependancies. 2018-01-05 19:57:11 +02:00
Jordan Petridis
2d71a99121
Change the auto-updater and checkup startup scheduling. 2018-01-02 20:47:49 +02:00
Jordan Petridis
37e9b6fbf0
'How hard could it be' 2018-01-02 07:23:53 +02:00
Jordan Petridis
785a5f80c6
hammond-data::utils: Small and minor refactor. 2017-12-31 01:42:28 +02:00
Jordan Petridis
e727734443
DB Schema: Remove episode.published_date column. 2017-12-30 22:23:10 +02:00
Jordan Petridis
5730e71e2a
Clear TODO and FIXME that are no longer releavant. 2017-12-30 20:54:23 +02:00
Jordan Petridis
036292284d
hammond-data: Do not clean source url's cause some feeds use queries as identifiers for some reason. 2017-12-29 20:21:36 +02:00
Jordan Petridis
ca06a16bd9
Closes #2. Kudo to @jwykeham for the fix!. 2017-12-29 02:45:02 +02:00
Jordan Petridis
1d78ab7c42
Cargo upgrade. 2017-12-28 23:19:36 +02:00
Jordan Petridis
8a7d6d9f1b
Add an empty CHANGELOG that follows keepachangelog.com format. Closes #23. 2017-12-28 22:41:22 +02:00
Jordan Petridis
7d7b09ff0f
hammond-data::parser: Prefer itunes_summary over channel.description if it exists. Finishes #20. 2017-12-27 17:59:40 +02:00
Jordan Petridis
1f0a2b5c6c
hammond-data::parser: Prefer itunes_summary over item.description if it exists. Finishes #20. 2017-12-27 17:40:31 +02:00
Jordan Petridis
bfb74c4dba
hammond-data: Index_loop does not wait for GET request to finish now. 2017-12-26 18:18:48 +02:00
Jordan Petridis
933fec55a5
hammond-data::parser: Refactor Episode tests. 2017-12-26 13:40:52 +02:00
Jordan Petridis
fde4bedce6
hammond-data::parser: Refactor Podcast tests. 2017-12-25 23:29:42 +02:00
Jordan Petridis
2d6f02c407
EpisodeWidget: Do not display size if it's 0 bytes. 2017-12-23 15:44:28 +02:00
Jordan Petridis
31b19dd88e
Just things apple force you to do. 2017-12-22 18:28:18 +02:00
Jordan Petridis
4512790f2d
hammond-data: Add duration column to the episode table. 2017-12-22 17:30:07 +02:00
Jordan Petridis
5541b18a6a
hammond-data::parser: Add itunesext_duration parser. 2017-12-22 16:55:45 +02:00
Jordan Petridis
b820ee4db7
hammond-data::parser : Do not clean image urls cause feeds put redirects for uri for some reason. 2017-12-22 14:51:57 +02:00
Jordan Petridis
e416bca963
Implemented a pixbuf cache mechanism. 
Since gdk_pixbuf::Pixbuf is refference counted and every episode,
use the cover of the Podcast Feed/Show, We can only create a Pixbuf
cover per show and pass around the Rc pointer.

GObjects do not implement Send trait, so SendCell is a way around that.
Also lazy_static requires Sync trait, so that's what the mutexes are.
 2017-12-21 17:36:07 +02:00
Jordan Petridis
db59bed69d
hammond-data::utils: Modified EpisodeCleaner Diesel model to also be used with played_cleaner. 2017-12-20 19:18:20 +02:00
Jordan Petridis
3c24b9f9d9
hammond-data::utils: Added new Diesel model for the download checker. 2017-12-20 19:00:14 +02:00
Jordan Petridis
2e06205eda
Define a new Diesel Model for PodcastCover querries. 
Define new Diesel Model and impl From<Podcast> trait,
Change the signature of downloader::cache_image function,
Change and merge hammond-gtk::utils::get_pixbuf_from_path functions.
 2017-12-20 18:19:31 +02:00
Jordan Petridis
914cad72f5
EpisodesView: Custom Diesel model not really necessary. 2017-12-19 20:45:40 +02:00
Jordan Petridis
895591f628
EpisodesView: Use show cover image. 2017-12-19 13:19:38 +02:00
Jordan Petridis
ad9a932143
Fix diesel query. 2017-12-19 12:56:16 +02:00
Jordan Petridis
f602e8c36d
EpisodesView: Create a DieselModel for EpisodeViewWidget. 2017-12-19 11:04:44 +02:00
Jordan Petridis
61bd7893c7
EpisodeWidget: Remove Podcast depndancy from new() method. 2017-12-18 19:10:50 +02:00
Jordan Petridis
f7af05a650
EpisodeWidget: Dim/Grey out widget if episode is played. 2017-12-16 20:30:25 +02:00
Jordan Petridis
0ac78fcff1
Added date label into EpisodeWidget. 2017-12-14 18:03:37 +02:00
Jordan Petridis
afdb79b712
Set the progress bar into activity mode. 2017-12-14 13:38:18 +02:00
Jordan Petridis
8fe6b526a5
Add a new Diesel Model for the EpisodeWidget. 2017-12-14 12:01:35 +02:00
Jordan Petridis
ad7f5013f3
Extend ShowsMachine functionality. 2017-12-11 11:15:54 +02:00
Jordan Petridis
851be1fcfb
Schema hotfix. 2017-12-10 06:47:10 +02:00
Jordan Petridis
d8281c70a5
Better naming of a dbquery function. 2017-12-10 06:11:03 +02:00
Jordan Petridis
bc57c33491
Change episode table schema. 2017-12-09 17:38:46 +02:00
Jordan Petridis
999a2a1fc1
Strip whitespace during parsing. 2017-12-09 10:22:09 +02:00
Jordan Petridis
b528c48e3b
Run rustfmt. 2017-12-09 05:57:16 +02:00
Jordan Petridis
48e61c6377
Added small a utility that eats extra whitespace. 2017-12-07 18:31:45 +02:00
Jordan Petridis
c07d240532
Sanitize html during feed parsing. 2017-12-07 06:38:31 +02:00
Jordan Petridis
05e056481f
Take advantage of <Into> trait from the generated builders. 2017-12-05 09:28:13 +02:00
Jordan Petridis
8e6cc4d2f2
Remove custom builders in favor of derive_builder crate. 2017-12-05 09:10:02 +02:00
Jordan Petridis
7727bc5ec3
Remove unnecessary Arc. 2017-12-02 01:46:37 +02:00